How to make backwards compatible changes to a Solana program

Given some of the resources out there, it may surprise you to learn that most Solana programs (“smart contracts”) are not immutable! In fact, most Solana programs get updated on a regular basis.

If you’re a Solana developer, and have deployed a program on mainnet-beta, it’s likely that you’ll eventually need to change it (e.g. to fix a bug, or add a new feature). In this post, we’ll discuss how to make program changes in a safe way. Specifically, we’ll cover which program changes are backwards compatible, and which ones aren’t.

Backwards compatibility

Let’s start at the beginning—what is backwards compatibility?

Backwards compatibility refers to whether or not a software change is compatible with previous versions of software that have already been distributed.

In modern, full-stack software development, the most common scenario where this can happen is when a backend change rolls out with breaking API changes — e.g., an API expects a different set of inputs from clients. All old (or “stale”) clients will send the wrong set of inputs to the API and cause it to fail.

Solana programs run into a similar problem, where program changes are analogous to the API change described above. For example, if a new program version is deployed and expects a different set of accounts to be passed in, all stale clients passing the old set of accounts will fail.

Changes that are backwards compatible

1. ✅ Adding more fields to the end of an account struct (as long as there is enough space)

Changing the old code to the new code is backwards compatible.

Note that the new field will (usually) be represented by zeros. E.g. booleans will be false, numbers will be 0, Options will be None, etc.

⚠️ However, if your account structure has an Option as its last field, this may not be the case — read on for more info.

If you’re adding a new field after an option, be wary of old data that has not been zeroed out.

Let’s consider a single FooAccount account that goes through these steps:

• pubkey gets set to Some(...). This gets serialized as a 1, followed by the pubkey.
• pubkey gets set to None. This gets serialized as a 0. However, the pubkey following the option byte (the byte that denotes whether the option is Some or None) does not get zeroed out (unless you manually do this).
• After adding the test field, we deserialize the account. But since pubkey is None, the first byte of the previously existing pubkey will get deserialized as test. This is undesirable, because you can’t rely on the value of test defaulting to false for all old accounts.

2. ✅ Wrapping an account with Box

Note that Box is typically used when you are deserializing too many accounts with Anchor, which blows the stack. When that happens, you’ll get an error like this:

logs: [
  'Program Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS invoke [1]',
  'Program Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS consumed 999 of 200000 compute units',
  'Program failed to complete: Access violation in stack frame 3 at address 0x200003fd0 of size 8 by instruction #2515',
  'Program Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS failed: Program failed to complete'
],
programErrorStack: ProgramErrorStack { stack: [ [PublicKey] ] }

Changing the old code to the new code is backwards compatible.

3. ✅ Removing an instruction’s last argument

If you pass an instruction more data than it uses, it will just ignore the extra data when deserializing it (and hence not throw).

Changing the old code to the new code is backwards compatible.

4. ✅ Changing a mutable account to be immutable

Changing the old code to the new code is backwards compatible.

This may surprise you, because account addresses are ordered by mutability (see this diagram). Here’s how this works.

The transaction the stale client sends will look like this:

• Addresses: [bar_address, foo_address] (bar_address comes first because the account is marked as mutable)
• Account address indexes for instruction: [1, 0]

Then, when the new program is processing this transaction, it sees that the first account address is at index 1, which is foo_address—just as expected.

5. ✅ Decreasing account sizes

⚠️ This change is not always backwards compatible. But I’m putting it in this section because I think in most scenarios where someone wants to do this, it is backwards compatible.

Specifically, it is only backwards compatible if either of these conditions are true:

• No accounts have been created yet OR
• No program code reads from/writes to inaccessible data. For example, if you decrease an account size from 20 bytes to 10 bytes, your program code should not read from/write to bytes 11–20. Note that if you’re reducing the account size to 10 bytes, you shouldn’t be reading from/writing to the remaining bytes anyways!

If neither of these two conditions are met, then this change is not backwards compatible.

Changes that are NOT backwards compatible

1. 🛑 Adding an account

Programs expect a certain number of accounts, so if you require an additional account to be passed to a program’s instruction, old clients will fail.

Changing the old code to the new code is NOT backwards compatible.

💡 There’s a way to do this that is backwards compatible; see the last section for more details.

2. 🛑 Reordering accounts

Programs expect accounts to be passed in a specific order, so re-ordering accounts is not backwards compatible.

Changing the old code to the new code is NOT backwards compatible.

This transaction diagram may help understand how this works. Basically, the transaction a stale client sends will look like this:

• Addresses: [foo_address, bar_address]
• Account address indexes for instruction: [0, 1]

Then, when the new program is processing this transaction, it will see that the first account address is at index 0, which is foo_address. However, the new program expects the first account to be a BarAccount, and so it will throw an error.

3. 🛑 Changing an immutable account to be mutable

This does not work, because the client must specify which accounts are mutable when sending a transaction.

Changing the old code to the new code is NOT backwards compatible.

4. 🛑 Adding an instruction argument

The program will fail when trying to deserialize the data, because it expects more data than the client gives it. Note that even if you add a new argument that is an Option, it still breaks backwards compatibility.

Changing the old code to the new code is NOT backwards compatible.

💡 If you do need to add a new instruction argument, you can add a new instruction that takes an additional argument, e.g. test_ix_v2; see the last section for more details.

5. 🛑 Increasing account sizes

💡 This change does not break backwards compatibility if no accounts have already been initialized.

💡If some accounts have already been initialized, it is still possible to make this change in a backwards compatible way if you re-alloc all the old accounts first.

Let’s consider an example. Suppose we have a bunch of FooAccounts that are 100 bytes. We want to store more data in these accounts, and so we increase the size—now all newly created FooAccounts are 200 bytes.

If we start reading from/writing to bytes 101–200, the program will break when doing so for old FooAccounts that are only 100 bytes.

Changing the old code to the new code is NOT backwards compatible.

6. 🛑 Removing an instruction’s middle argument

Depending on what instruction data you pass, this may or may not break things. However, in general, it is not backwards compatible because it causes the instruction data to be deserialized in a different way.

Changing the old code to the new code is NOT backwards compatible.

How to perform commonly needed backwards compatible updates

Adding another account to an instruction

⚠️ This approach assumes you have control over all the program’s clients. If you do not, then it does not work.

1. Modify the client to pass in an extra account

const tx = await program.methods
  .testIx()
  .accounts({
    account1: getRandomPubkey(),
    // @ts-ignore
    account2: getRandomPubkey(),
  })
  .rpc();

This will work even if the program looks like this:

#[derive(Accounts)]
pub struct TestIx<'info> {
    account1: UncheckedAccount<'info>,
}

2. Add the extra account to the program

You must add it at the end, since program instructions expect accounts to be passed in a certain order. For example, if account2 is added before account1, and the program is deployed but a client is not updated, then the client will pass up the accounts in reverse order, and the program will interpret account2 as account1 (and vice-versa). This transaction diagram may help understand how this works.

#[derive(Accounts)]
pub struct TestIx<'info> {
    // DO NOT ADD IT HERE!
    // account2: UncheckedAccount<'info>,
    account1: UncheckedAccount<'info>,
    account2: UncheckedAccount<'info>,
}

Adding an additional argument to an instruction

Adding an arg to an existing instruction is not backwards compatible. Thus, you must create a new version of the instruction, e.g. ix_v2, and then switch to calling it in client code. You can remove the old version afterwards if no clients are using it.

Final Thoughts

If you enjoyed this article, please support us by giving us a 👏, leaving a comment with your thoughts, or by simply sharing this article with your network. You can also reach us on Twitter (@Formfunction) or by joining our Discord where one of our team members would be happy to meet and chat with you.

Stay tuned for more posts like these from the Formfunction engineering team. And if you’re also passionate about building a better world for creators, come join us!