How to make backwards compatible changes to a Solana program
Given some of the resources out there, it may surprise you to learn that most Solana programs (“smart contracts”) are not immutable! In fact, most Solana programs get updated on a regular basis.
If you’re a Solana developer, and have deployed a program on mainnet-beta, it’s likely that you’ll eventually need to change it (e.g. to fix a bug, or add a new feature). In this post, we’ll discuss how to make program changes in a safe way. Specifically, we’ll cover which program changes are backwards compatible, and which ones aren’t.
Backwards compatibility
Let’s start at the beginning—what is backwards compatibility?
Backwards compatibility refers to whether or not a software change is compatible with previous versions of software that have already been distributed.
In modern, full-stack software development, the most common scenario where this can happen is when a backend change rolls out with breaking API changes — e.g., an API expects a different set of inputs from clients. All old (or “stale”) clients will send the wrong set of inputs to the API and cause it to fail.
Solana programs run into a similar problem, where program changes are analogous to the API change described above. For example, if a new program version is deployed and expects a different set of accounts to be passed in, all stale clients passing the old set of accounts will fail.
Changes that are backwards compatible
- ✅ Adding more fields to the end of an account struct (as long as there is enough space)
Note that the new field will (usually) be represented by zeros. E.g. booleans will be false, numbers will be 0, Option
s will be None
, etc.
⚠️ However, if your account structure has an Option
as its last field, this may not be the case — read on for more info.
Let’s consider a single FooAccount
account that goes through these steps:
pubkey
gets set toSome(...)
. This gets serialized as a1
, followed by the pubkey.pubkey
gets set toNone
. This gets serialized as a0
. However, the pubkey following the option byte (the byte that denotes whether the option isSome
orNone
) does not get zeroed out (unless you manually do this).- After adding the
test
field, we deserialize the account. But sincepubkey
isNone
, the first byte of the previously existing pubkey will get deserialized astest
. This is undesirable, because you can’t rely on the value oftest
defaulting to false for all old accounts.
2. ✅ Wrapping an account with Box
Note that Box
is typically used when you are deserializing too many accounts with Anchor, which blows the stack. When that happens, you’ll get an error like this:
logs: [
'Program Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS invoke [1]',
'Program Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS consumed 999 of 200000 compute units',
'Program failed to complete: Access violation in stack frame 3 at address 0x200003fd0 of size 8 by instruction #2515',
'Program Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS failed: Program failed to complete'
],
programErrorStack: ProgramErrorStack { stack: [ [PublicKey] ] }
3. ✅ Removing an instruction’s last argument
If you pass an instruction more data than it uses, it will just ignore the extra data when deserializing it (and hence not throw).
4. ✅ Changing a mutable account to be immutable
This may surprise you, because account addresses are ordered by mutability (see this diagram). Here’s how this works.
The transaction the stale client sends will look like this:
- Addresses:
[bar_address, foo_address]
(bar_address
comes first because the account is marked as mutable) - Account address indexes for instruction:
[1, 0]
Then, when the new program is processing this transaction, it sees that the first account address is at index 1, which is foo_address
—just as expected.
5. ✅ Decreasing account sizes
⚠️ This change is not always backwards compatible. But I’m putting it in this section because I think in most scenarios where someone wants to do this, it is backwards compatible.
Specifically, it is only backwards compatible if either of these conditions are true:
- No accounts have been created yet OR
- No program code reads from/writes to inaccessible data. For example, if you decrease an account size from 20 bytes to 10 bytes, your program code should not read from/write to bytes 11–20. Note that if you’re reducing the account size to 10 bytes, you shouldn’t be reading from/writing to the remaining bytes anyways!
If neither of these two conditions are met, then this change is not backwards compatible.
Changes that are NOT backwards compatible
1. 🛑 Adding an account
Programs expect a certain number of accounts, so if you require an additional account to be passed to a program’s instruction, old clients will fail.
💡 There’s a way to do this that is backwards compatible; see the last section for more details.
2. 🛑 Reordering accounts
Programs expect accounts to be passed in a specific order, so re-ordering accounts is not backwards compatible.
This transaction diagram may help understand how this works. Basically, the transaction a stale client sends will look like this:
- Addresses:
[foo_address, bar_address]
- Account address indexes for instruction:
[0, 1]
Then, when the new program is processing this transaction, it will see that the first account address is at index 0, which is foo_address
. However, the new program expects the first account to be a BarAccount
, and so it will throw an error.
3. 🛑 Changing an immutable account to be mutable
This does not work, because the client must specify which accounts are mutable when sending a transaction.
4. 🛑 Adding an instruction argument
The program will fail when trying to deserialize the data, because it expects more data than the client gives it. Note that even if you add a new argument that is an Option
, it still breaks backwards compatibility.
💡 If you do need to add a new instruction argument, you can add a new instruction that takes an additional argument, e.g. test_ix_v2
; see the last section for more details.
5. 🛑 Increasing account sizes
💡 This change does not break backwards compatibility if no accounts have already been initialized.
💡If some accounts have already been initialized, it is still possible to make this change in a backwards compatible way if you re-alloc all the old accounts first.
Let’s consider an example. Suppose we have a bunch of FooAccount
s that are 100 bytes. We want to store more data in these accounts, and so we increase the size—now all newly created FooAccount
s are 200 bytes.
If we start reading from/writing to bytes 101–200, the program will break when doing so for old FooAccount
s that are only 100 bytes.
6. 🛑 Removing an instruction’s middle argument
Depending on what instruction data you pass, this may or may not break things. However, in general, it is not backwards compatible because it causes the instruction data to be deserialized in a different way.
How to perform commonly needed backwards compatible updates
Adding another account to an instruction
⚠️ This approach assumes you have control over all the program’s clients. If you do not, then it does not work.
1. Modify the client to pass in an extra account
const tx = await program.methods
.testIx()
.accounts({
account1: getRandomPubkey(),
// @ts-ignore
account2: getRandomPubkey(),
})
.rpc();
This will work even if the program looks like this:
#[derive(Accounts)]
pub struct TestIx<'info> {
account1: UncheckedAccount<'info>,
}
2. Add the extra account to the program
You must add it at the end, since program instructions expect accounts to be passed in a certain order. For example, if account2
is added before account1
, and the program is deployed but a client is not updated, then the client will pass up the accounts in reverse order, and the program will interpret account2
as account1
(and vice-versa). This transaction diagram may help understand how this works.
#[derive(Accounts)]
pub struct TestIx<'info> {
// DO NOT ADD IT HERE!
// account2: UncheckedAccount<'info>,
account1: UncheckedAccount<'info>,
account2: UncheckedAccount<'info>,
}
Adding an additional argument to an instruction
Adding an arg to an existing instruction is not backwards compatible. Thus, you must create a new version of the instruction, e.g. ix_v2
, and then switch to calling it in client code. You can remove the old version afterwards if no clients are using it.
Final Thoughts
If you enjoyed this article, please support us by giving us a 👏, leaving a comment with your thoughts, or by simply sharing this article with your network. You can also reach us on Twitter (@Formfunction) or by joining our Discord where one of our team members would be happy to meet and chat with you.
Stay tuned for more posts like these from the Formfunction engineering team. And if you’re also passionate about building a better world for creators, come join us!