Dependencies

• Working knowledge of entities and actors - Schema Versions and Schema Version Migrations are part of the backend's entities ecosystem. Take a look at Introduction to Entities and Actors if you're not familiar with them yet.

Key Concepts

• Every entity has an associated integer called Schema Version that tracks what version of data model it currently has. This number should be incremented when adding new members to an entity class. When an entity is persisted to the database, its Schema Version is stored alongside the payload.
• A Schema Version Migration (or just Migration) is an operation that increments the Schema Version and upgrades the entity to match the following model data layout.

Overview

Over a game's lifetime, it changes, receives updates, and eventually, old logic gets deprecated. Schema Version Migrations allow you to update older persisted entities to match the current data models. It's also possible to eliminate the old code if necessary, but we don't recommend it.

Here are the Schema Version operations covered on this page:

1. Defining the Initial Schema Version
2. Implementing Migrations
3. Retiring Legacy Members
4. Manually Migrating Entities

Defining the Schema Version

You can mark a persisted state object to support Migrations by implementing the ISchemaMigratable interface and then using the [SupportedSchemaVersions(oldestSupportedSchemaVersion, currentSchemaVersion)] attribute to specify the range of Schema Versions that you wish to support for the type. The oldestSupportedSchemaVersion is the oldest Schema Version we're willing to support migrating from.

Note that the base class often implements the ISchemaMigratable interface and does not need to be explicitly implemented anymore. For example, with PlayerModel, the interface is already implemented by PlayerModelBase.

As an example, let's define a simplified initial player model that holds information on the number of fruits that the player has:

PlayerModel.cs

// Right now, the backend only supports Schema Version 
// 1, with it being both the CurrentSchemaVersion 
// and the OldestSupportedSchemaVersion.
[MetaSerializableDerived(1)]
[SupportedSchemaVersions(1, 1)]
public class PlayerModel : PlayerModelBase<...>
{
    [MetaMember(1)] int NumApples;
    [MetaMember(2)] int NumOranges;

    ...
}

When a new entity is created, its Schema Version is initialized to the maximum Schema Version. The maximum Schema Version is also the current Schema Version, while the Minimum Schema Version is the oldest version we want to support Migrations from.

Implementing the Migration

The Migration is implemented with a set of Migration functions, each representing a Migration operation from a given Schema Version and updates the model to the next.

Continuing our example, we could introduce a new set of members to record the highest number of fruits a player has ever held. These new members would need to be initialized through a Migration.

The simplest way of implementing the Migrations is to declare the individual Migration functions as Model methods tagged with the [MigrateFromVersion(fromVersion)] attribute, grouped together into a C# region:

PlayerModel.cs

[MetaSerializableDerived(1)]
[SupportedSchemaVersions(1, 2)] // CurrentSchemaVersion increased from v1 to v2
public class PlayerModel : PlayerModelBase<...>
{
    [MetaMember(1)] int NumApples;
    [MetaMember(2)] int NumOranges;
    [MetaMember(3)] int MaxNumApples;
    [MetaMember(4)] int MaxNumOranges;

    ...

    #region Schema migrations

    // Migration from version 1 to 2:
    // Added a new field to record highest number of fruit
    // that a player has ever held.
    [MigrateFromVersion(1)]
    void Migrate1To2()
    {
        MaxNumApples = model.NumApples;
        MaxNumOranges = model.NumOranges;
    }

    #endregion
}

Now, let's imagine that we want to remove apples from the game and that we'll compensate players at an exchange rate of 3 oranges for every apple:

PlayerModel.cs

[MetaSerializableDerived(1)]
[SupportedSchemaVersions(1, 3)] // Bumped CurrentSchemaVersion to v3
public class PlayerModel : PlayerModelBase<...>
{
    [MetaMember(1)] int LegacyNumApples;    // No longer used in-game
    [MetaMember(2)] int NumOranges;
    [MetaMember(3)] int LegacyMaxNumApples; // No longer used in-game
    [MetaMember(4)] int MaxNumOranges;

    ...

    #region Schema migrations

    // Migration from version v1 to v2:
    [MigrateFromVersion(1)]
    void Migrate1To2()
    {
        MaxNumApples = model.NumApples;
        MaxNumOranges = model.NumOranges;
    }

    // Migration from version v2 to v3:
    [MigrateFromVersion(2)]
    void Migrate2To3()
    {
        // Compensate players with three oranges for every apple that they held.
        NumOranges += model.LegacyNumApples * 3;

        // It's good practice to clear these legacy values as we will no longer use them.
        LegacyNumApples = 0;
        LegacyMaxNumApples = 0;

        // Ensure NumOranges isn't above MaxNumOranges.
        MaxNumOranges = Math.Max(MaxNumOranges, NumOranges);
    }

    #endregion
}

The SDK calls the Migration functions automatically when a persisted entity is being "woken up", i.e., restored from the database, if the entity was persisted with a Schema Version lower than current. The entity sequentially invokes each Migration step required to bring it to the current version, and only then is the entity ready to start running.

The Migrations are performed whenever a persisted entity wakes up. This can happen for many reasons, but the most typical ones are players logging, an entity (player or other) being viewed from the dashboard, or an entity spawning automatically when the server starts. Migrations are also invoked when an entity is imported with a Schema Version older than the current one.

Note that after you deploy the Migration code, it does not run automatically for all entities. An entity stays at the old Schema Version until it is woken up and then persisted again. This means that you may need to keep the old code around indefinitely to support the old Migration code. See the section Manually Migrating Entities for a technique to address this

Using RegisterMigrationFunction

As your model's number of Migration functions grows, maintaining separate methods for each Migration might become impractical. Therefore, it is also possible to declare Migration functions per version by declaring a single RegisterMigrationsFunction static method that returns a lambda function per Migration version:

PlayerModel.cs

[SupportedSchemaVersions(1, 3)]
public class PlayerModel : PlayerModelBase<...>
{
    [RegisterMigrationsFunction]
    static Action<PlayerModel> RegisterMigrations(int fromVersion)
    {
        switch (fromVersion)
        {
            // Migrate from v1 to v2: Initialize MaxNumApples and MaxNumOranges
            case 1: return model => {
                model.MaxNumApples = model.NumApples;
                model.MaxNumOranges = model.NumOranges;
            }
            // Migrate from v2 to v3: Remove apples from the game
            case 2: return model => {
                model.NumOranges += model.LegacyNumApples * 3;
                model.LegacyNumApples = 0;
                model.LegacyMaxNumApples = 0;
                model.MaxNumOranges = Math.Max(MaxNumOranges, NumOranges);
            }
            default: return null;
        }
   }

   ...

}

These two examples yield identical results, the only difference being the convenience of declaring the functions. The latter RegisterMigrationsFunction approach is especially convenient if your Version Migrations reuse code. For example, you want to clear some version-dependent state on the model every time a Migration executes.

It is also possible to declare both individual MigrateFromVersion functions and a RegisterMigrationsFunction, but you can only declare a single Schema Version Migration function via one of the methods. If a MigrationFromVersion function is provided, then the corresponding call to RegisterMigrations must return null. On conflicting declarations, the Metaplay SDK will raise an error during initialization.

Retiring Legacy Members

DANGER

Be careful! If you wish to drop support for legacy Schema Versions, you must ensure that every entity has migrated to the current Version. If not, the Metaplay SDK will fail to deserialize entities with deprecated Schema Versions, and their data will be wiped. We recommend you avoid removing legacy members, but if you must, make sure no entities remain on the corresponding Schema Version.

In general, after adding code for a Migration, you should keep it around indefinitely. As long as there is a possibility that the game backend will run against a database that holds entities of old Schema Versions, the code to support Migrating from those Schema Versions cannot be removed without making those entities unable to run.

Remember the OldestSupportedSchemaVersion parameter to [SupportedSchemaVersions]? It marks the lowest Schema Version that your code still supports. The SDK will discard any entity with a Schema Version lower than this (and initialize it with a fresh state), instead performing a Migration. Therefore, you'll likely only want to change this if you can ensure that all entities are at least at that Schema Version. For player entities in particular, this likely means that you don't want to change OldestSupportedSchemaVersion once your game is live unless you take extraordinary measures to ensure all existing players have been Migrated. The Schema Migrator Job does precisely that and will be discussed in the next section, Manually Migrating Entities.

Manually Migrating Entities

For many types of entities, there's no natural time after which they'd all have Migrated. For example, there's no time period after which we'd know for sure that all existing players have logged in at least once after a new Schema Version was added.

A practical way to address this is to actively wake up all Entities of a given type. Metaplay has a schema migrator maintenance job for this purpose; see the Scan Jobs System vs. Maintenance Jobs section in Database Scan Jobs. The schema migrator job scans through an entity table, wakes up all the entities on an old Schema Version, and produces a report indicating whether all entities were successfully Migrated to the current Schema Version. Note that the job is intended to be run during the normal operation of a game server and can take a significant amount of time to complete, depending on the number of entities.

DANGER

Caution: When removing code dealing with old Schema Versions, consider all the environments in which the code may run. For example, even if all the player entities in your production environment have migrated, you might still need to consider your staging and development environments.

After you are sure all entities of a given type have migrated to the maximum Schema Version, the code dealing with the old Schema Versions can be safely removed.