Key Concepts

• An Entity is an ephemeral Actor that exists on a Metaplay Server. Each Entity is spawned by the Game Server and exists in isolation, communicating with other Entities through Messages. An Entity could represent a Player, a Guild, the Admin API, etc.
• A Message is simply an immutable piece of context-specific data that inherits from MetaMessage and defines a unique TypeCode.

Overview

There are a number of different Entity-to-Entity communication patterns available in the Metaplay SDK. Each pattern has a specific use case and comes with its own set of pros and cons. Below you will see the five most commonly used patterns: Message Cast, EntityAsk, EntitySynchronize, EntitySubscribe, and native Akka.Net messaging. These patterns can be used as-is or as the basis for more advanced messaging patterns.

Messages vs Actions

While Messages and Actions seem superficially similar, each of them are intended for quite different purposes:

• Messages are simple containers of data and contain no logic. Actions always target a Model and are intended to be executed against it. For example, PlayerAction targets PlayerModel.
• Messages are the lower-level communication primitive and can be exchanged between Entities on the Server(s) or between client and Server. Actions must be wrapped in a Message in order to be transmitted.
• Actions always live in a context where a valid set of Game Configs are present and they can refer to such data. Messages have no such guarantees and cannot refer to Game Config data.
• Actions automatically work on both client and server due to the shared code architecture. Messages require custom handling code on the receiving end.

As a rule of thumb, when implementing general game logic, Actions are usually used. When implementing backend features, Messages are usually the right choice.

Message Cast

The most fundamental operation is sending and receiving a single message between two Entities. There is no guarantee of delivery in this pattern and no way to automatically detect if the Message was received. This pattern is often referred to as "fire and forget" messaging.

Example: Send message to another Entity.

// Define a new message type with type code 1001.
// Use ServerInternal direction as this message should never leave
// the Game Backend. It is an internal Entity-to-Entity message.
[MetaMessage(typeCode: 1001, MessageDirection.ServerInternal)]
class MyMessage : MetaMessage
{
    int MyCustomData;

    private MyMessage() { }
    public MyMessage(int myCustomData)
    {
        MyCustomData = myCustomData;
    }
}

// Send message to entityId.
CastMessage(entityId, new MyMessage(123))

class MyEntityActor : PersistedEntityActor<...>
{
    // Synchronous handler with fromEntityId example:
    [MessageHandler]
    void HandleMyMessage(EntityId fromEntityId, MyMessage message)
    {
        // Received MyMessage from fromEntityId.
        // Do something with it.
        // We could also reply back that we have handled the message.
        // CastMessage(fromEntityId, new MyMessageDone())
    }

    // Async handler without fromEntityId example:
    [MessageHandler]
    async Task HandleMyMessage(MyMessage message)
    {
        await DoSomeWork(...);
        // We could also reply back that we have handled the message.
        // CastMessage(fromEntityId, new MyMessageDone())
    }
}

The calling Entity casts the Message to the callee Entity with CastMessage(). The Message is serialized and delivered to the callee, where it's placed into the message queue of that Entity. If needed, the callee is woken up as discussed in Introduction to Entities and Actors. The callee handles the messages in its message queue in order, one by one.

TIP

The calling and callee may exist on different server nodes but that doesn't matter - the Message passing handles this under the hood and the system still works in exactly the same way from both the Entity and the programmer's point of view.

In the case of a MyMessage, the message is handled by a method decorated with the [MessageHandler] attribute. An Entity may also define a HandleUnknownMessage() catch-all handler.

INFO

In exception stack traces, only the handler method's name is shown, so it's recommended to name the handlers such that they contain all the information needed to know which message was received.

Messages between two Entities are always delivered in order. If Entity A sends Messages M1 and M2 to Entity B, Entity B is guaranteed to receive the Messages in the order M1 followed by M2. This ordering guarantee applies only between two Entities - in particular, it is not transitive and the Triangle Inequality does not hold. If Entity A sends a message to Entity B and then to Entity C, and Entity B sends a message to Entity C upon receiving Entity A's message, then Entity C might receive either A or B's message first.

The Message delivery is unreliable and the Messages are delivered with at-most-once semantics. If the target Entity crashes or there is a networking issue between Game Servers, the cast Message may be lost. The caller is expected to detect and handle this situation. In any case however, the sent Messages will not be received more than once.

Synchronous Request-Response With EntityAsk

A common pattern with messaging is a Request-Response pair. Metaplay SDK implements the pattern with an available EntityAsk.

Example: Send a Request to an Entity and wait for a Response.

// Call and wait for a response.
// This caller Entity's execution is suspended until the target Entity replies
// or the EntityAsk times out.
RequestMessage request = new RequestMessage();
ResponseMessage response = await EntityAskAsync<ResponseMessage>(entityId, request);

// Callee.
[EntityAskHandler]
ResponseMessage HandleRequestMessage(RequestMessage request)
{
    // Compute response & send it back.
    ResponseMessage response = new ResponseMessage();
    return response;
}

From a communications perspective, EntityAsk is very similar to manually sending request-response Messages. The crucial difference comes from the local behavior: unlike with normal messaging where Messages are processed in order to completion, the response to the EntityAsk always bypasses the Entity message queue. This allows an Entity to await for EntityAsks within a message handler. This cannot be achieved with normal Messages as awaiting would suspend the Entity Message processing and the response would never be processed.

To prevent suspending entity processing indefinitely, EntityAsk has an implicit timeout of 10 seconds. Hence, EntityAsk is not suitable for long-running Request-Responses.

An Entity may provide a HandleUnknownEntityAsk() handler to handle any otherwise unhandled EntityAsks.

Handling Errors During EntityAsks

If an unhandled exception happens during an EntityAsk in the callee, the callee Entity is terminated and an UnexpectedEntityAskError exception is thrown on the caller side with details of the exception (exception type, message, and stack trace) concatenated into an error string. The error is thrown if a) the Entity Actor crashes while handling the Request, or b) if the Entity Actor fails to spawn when triggered by the Request.

The intention is to provide human-readable error messages on the caller side for logging or displaying on the dashboard.

try
{
  await EntityAskAsync<SomeResponse>(entityId, new SomeRequestThatThrows());
}
catch (UnexpectedEntityAskError ex)
{
  _log.Warning("EntityAsk failed: {ErrorMessage}", ex.HandlerErrorMessage);
}

EntityAskHandler implementations can also communicate controlled error situations by throwing exceptions derived from EntityAskError. Similarly to UnexpectedEntityAskError, the exception is serialized and thrown on the caller side. Contrary to other types of unhandled exceptions, the callee Entity will not be terminated upon throwing an EntityAskError. Therefore, special care must be taken that the callee Entity remains in a consistent and operational state. The Metaplay SDK provides a simple concrete EntityAskError type InvalidEntityAsk that only supports passing a single error string, for more complicated error reporting you should declare your own type of EntityAskError.

[EntityAskHandler]
public SomeResponse HandleSomeRequest(SomeRequest request)
{
    if (!IsValid(request))
        throw new InvalidEntityAsk($"Request was not valid {request}");
  ...
}

EntitySynchronize

EntityAsk only supports a single request-response pair, which can be limiting in some cases. For example, an Entity might first Ask whether something is possible, and only then Ask to commit to it. With EntityAsk, this pattern requires two separate requests and the target Entity state might change between those requests, causing unexpected or undefined behavior. This could be fixed with a retry loop where the caller tries again until race does not happen, but it is complex.

For these use-cases, Metaplay SDK ships EntitySynchronize. EntitySynchronize is a context in which two Entities can exchange Messages while bypassing the normal message queue. As with EntityAsk, this can be used within a Message handler. Essentially, this is a superset of EntityAsk that allows for an arbitrary number of requests and responses.

Example: Coordinate with another Entity to perform specific operations in a well-defined order. The caller Entity first sends a request and callee chooses if it wants to continue with the operation. If so, the caller then also gets to decides whether it wants to continue. If both Entities agree, the caller does a placeholder "commit" operation first, and only then does the callee do the same.

// Caller.
// Establish synchronized execution context.
// The await completes when target Entity starts executing the handler
FirstRequest request = new FirstRequest()
using (EntitySynchronize sync = await EntitySynchronizeAsync(entityId, request))
{
    // Wait for target to respond.
    FirstResponse firstResponse = await sync.Receive<FirstResponse>()

    // If target is angry, end. If happy, continue.
    if (firstResponse.bad)
        return;

    // Are we ready to commit?
    if (!(we want to commit))
    {
        sync.Send(SecondRequest(commit: false));
        return;
    }

    // Commit first, then let callee know.
    commit()
    sync.Send(SecondRequest(commit: true));
}

// Callee.
[EntitySynchronizeHandler]
async Task HandleFirstRequest(EntitySynchronize sync, FirstRequest request)
{
    // Handshake message is delivered as an argument.
    // Check that the request is good.
    if (request is bad)
    {
        sync.Send(new FirstResponse(bad: true))
        return;
    }
    sync.Send(new FirstResponse(bad: false))

    // Wait for the caller to commit (or not).
    SecondRequest secondReq = await sync.Receive<SecondRequest>()
    if (!secondReq.commit)
        return;

    // Caller has committed. We commit as well.
    commit()
}

To prevent suspending entity processing indefinitely, EntitySynchronize has an implicit timeout of 10 seconds.

EntitySubscribe

In addition to one-off communication, Entities can form longer-term communication relations by subscribing to a topic of another Entity. The pattern works like a radio: when the Entity Publishes a Message on a particular topic, all subscribing Entities receive that message. This pattern is often referred to as the Pub-Sub pattern.

Example:

// Payload message for the Request to open a subscription channel
class HandshakeRequest : MetaMessage {}
// Payload message for the succesful Response to open a subscription channel
class HandshakeResponse : MetaMessage { int Tick; }
// Payload message to be sent on the channel
class TickCount : MetaMessage { int numTicks }


// Entity that subscribes to other entity
class TickListenerEntity : EntityActor
{
    int _tick;

    async Task BeginListeningTickAsync()
    {
        // Subscribe to Entity on Topic of Spectator, with the payload of
        // HandshakeRequest. On completion, we get a handshake Response.
        (EntitySubscription subscription, HandshakeResponse response) =
            await SubscribeToAsync<HandshakeResponse>(announcerEntityId, EntityTopic.Spectator, new HandshakeRequest());

        _tick = response.Tick;
    }

    // Message from the subscription.
    // For subscriber messages, we would use "void HandlePubSubMessage(EntitySubscriber subscriber, TickCount tickCount)".
    // If we don't care where the data came from, we could use
    // "[MessageHandler] void HandleTickCount([EntityId fromEntityId,] TickCount tickCount)".
    [PubSubMessageHandler]
    void HandleTickCount(EntitySubscription subscription, TickCount tickCount)
    {
        // Got message on from the other Entity. Their clock is now tickCount.NumTicks forward
        _tick += tickCount.NumTicks;
    }

    async Task EndListeningTickAsync()
    {
        // When we are done with this
        await UnsubscribeFromAsync(subscription)
    }
}

// The entity the listener subscribes to
class TickAnnouncerEntity : EntityActor
{
    int _tick;

    // Keep the entity alive while we have active subscribers.
    // Note: our subscriptions (if we have subscribed to other Entities) do not matter
    protected override AutoShutdownPolicy ShutdownPolicy => AutoShutdownPolicy.ShutdownAfterSubscribersGone(...)

    // Implement OnNewSubscriber to accept new subscriptions
    protected override async Task<MetaMessage> OnNewSubscriber(EntitySubscriber subscriber, MetaMessage message)
    {
        // Message is HandshakeRequest
        // subscriber.Topic is EntityTopic.Spectator

        // Return the current tick count on success response.
        return new HandshakeResponse(_tick)
    }

    // Every now and then, advance tick and announce change to listeners.
    void AdvanceTick()
    {
        int numTicksToAdvance = 1;
        _tick += numTicksToAdvance;
        PublishMessage(EntityTopic.Spectator, new TickCount(numTicksToAdvance))
    }
}

In case there is no declared handler for a given Message type, the HandleUnknownMessage() method of the Entity is invoked instead.

The OnNewSubscriber() method cannot be overloaded.

While Entity subscriptions are directional (publishing only delivers data to subscribers, not the other way around), the Message channel is bidirectional. After forming a subscription channel, either Entity may send Messages to the other with SendMessage().

A Subscription channel provides stronger reliability guarantees than normal messaging. Each Message contains a counter and the receiver Entity checks the counter matches the next expected value. Dropping a Message leads to a counter mismatch, and tearing down of the Subscription channel. Hence, receiving a Message guarantees that all preceding Messages of that channel were also received. However, as this requires the receiver to receive a Message before it can tell that previous Message(s) have been lost, the delay for detecting the lost Messages is unbounded.

When an Entity shuts down, all subscriptions to and from that Entity are automatically torn down. This applies even if the Entity shutdown is not clean. Subscriptions are also automatically torn down if the Game Server that the peer Entity resides on is lost.

If an Entity defines its AutoShutdownPolicy as ShutdownAfterSubscribersGone, the Entity will automatically extend its lifetime until all subscribing Entities have unsubscribed (or torn down). This allows for Entities to conveniently wake up, stay running and stop on-demand. For more information on Entity lifecycle, see Introduction to Entities and Actors.

Subscription Rejection

In the case the Entity being subscribed to does not want to accept another Entity's request to subscribe, it can reject the subscriber. As with EntityAsks, the OnNewSubscriber handler method can throw an exception deriving from EntityAskRefusal, which rejects the subscription and causes the exception to be thrown on the call site of SubscribeToAsync.

Extending the previous example:

// Subscription rejection error
[MetaSerializableDerived(12345)]
class TickAnnouncerTooBusyError : EntityAskRefusal
{
    [MetaMembe(1)] public MetaTime TryAgainAt;
}

class TickAnnouncerEntity : EntityActor
{
    ...

    protected override async Task<MetaMessage> OnNewSubscriber(EntitySubscriber subscriber, MetaMessage message)
    {
        // Reject if we are too busy.
        if (tooBusy)
            throw new TickAnnouncerTooBusyError(tryAgainAt: MetaTime.Now + MetaDuration.FromSeconds(5));
    }
}

class TickListenerEntity : EntityActor
{
    ...

    async Task BeginListeningTickAsync()
    {
        try
        {
            (EntitySubscription subscription, HandshakeResponse response) =
                await SubscribeToAsync<HandshakeResponse>(announcerEntityId, EntityTopic.Spectator, new HandshakeRequest());

            // Success path
            _tick = response.Tick;
        }
        catch (TickAnnouncerTooBusyError tooBusy)
        {
            // Subscribing failed.
            // We should try again at tooBusy.TryAgainAt
        }
    }
}

Native Akka.NET Messaging

All Entities in the server are Akka.NET actors and can use the native IActorRef to Tell commands to each other. This has both advantages and disadvantages over normal Entity messaging.

Example:

// Send to the current Entity. Self is a reference to the current Akka.NET actor.
// This is NOT handled immediately but pushed into the Entity's message queue.
Self.Tell(new MyCommandObject(), sender: null)

// Callee.
[CommandHandler]
void HandleMyCommandObject(MyCommandObject command)
{
    // Received command from somewhere. This is the same Entity instance.
}

The Entity may provide a HandleUnknownCommand() handler to handle any Commands for which there was no handler declared.

Unlike with Entity messaging where all communication must be represented as MetaMessage and hence must be MetaSerializable, Akka.NET can send arbitrary CLR objects. This can be beneficial if an Entity must send non-serializable objects, such as TaskCompletionSource<>. Note that this only works when the caller and callee are on the same server node. When passing such an object over a node-to-node-hop, the data is lost.

Entity Messages are addressed using EntityIds, which uniquely identify a single Entity over its entire life cycle. Akka.NET messages instead are addressed with IActorRefs, which denote a single Entity instance. If an Entity is shut down, all native Messages addressed to it are lost since the instance they are addressed to no longer exists. Any pending Entity Messages instead cause the Entity to wake up again, and the Messages are then delivered to the new Entity instance. Note that this possibility of shutdown applies also when sending Messages to the Self actor. If an actor is in the process of shutting down, it will not stop for pending Messages.

Due to this discrepancy between addressing an Entity and Entity Instance, getting the IActorRef for any other Entity is generally impossible. Even if it was possible, the actor reference could be or could become, stale at any moment.

Hence in practice, native messaging is only useful in very limited scenarios. The most common uses are:

• Target is Self and losing the message is desired if entity shuts down:

  For example, an Entity might enqueue a request to persist in the entity state. In case of a shutdown, the entity will be persisted automatically, and hence the request is no longer relevant.

  Alternatively, an Entity might enqueue an operation that needs to be replied to its subscriber. In case the Entity shuts down, it means that the subscriber has been lost (since alive subscription would keep the Entity awake), and the request can be safely ignored.

• Interoperability with Akka.NET tools:

  Akka.NET contains many useful tools, such as Timers that target Akka.NET actors. Interoperating with such tools may require using IActorRefs and/or HandleCommands

• Target has no EntityId:

  For example, if an Entity needs to communicate with its manager EntityShard object. EntityShard manages lifecycle and Message routing for Entities and it itself has no EntityId.