Metaplay Docs

Appearance

Integration Testing

This page describes how to run all the different test suites in a single run.

Dependencies ​

  • An overview of automated testing in the Metaplay SDK - You should have an idea of how automated testing works across the Metaplay SDK as described in Introduction to Automated Testing.

Overview ​

The integration tests allow running all available test suites in a single operation. This includes both the built-in SDK core tests and game-specific test suites recognized by the SDK. The intention is to catch problems in the interactions between the different parts of the SDK and the game-specific backend,since this aspect is often the least tested by the developers and human QA.

The primary purpose of the integration tests is to run the regularly as part of your CI process to catch regressions before publishing. The tests can also be invoked locally to catch or debug issues more easily on your own machine.

Included Test Suites ​

The following test suites are run as part of the integration tests:

Running the Tests Locally ​

We provide a Python script that uses Docker to run all the tests in the Metaplay SDK. You can run it locally with the following steps:

  1. Make sure Docker is running on your machine.
  2. Install the dependencies by running pip install -r MetaplaySDK/Scripts/requirements.txt.
  3. Run the tests with python MetaplaySDK/Scripts/integration-tests.py.

If your project is not located in the root of your repository, use --project-dir=<path> to help integration-tests.py find the project.

Please note that your first time running this script will take a while as it downloads the necessary Docker images and builds the projects. Repeated runs will be significantly faster.

Running the Tests in CI ​

You should also set up CI runs to ensure the integration tests run at regular intervals. The steps for this are the same as when running locally, you just need to wrap them in a job (or the equivalent task or action) for the CI you're using. Below are the steps in shell script form:

bash
pip install -r MetaplaySDK/Scripts/requirements.txt
python MetaplaySDK/Scripts/integration-tests.py

Take a look at your CI system's documentation for steps on how to create a job to run them. Various files, such as screenshots from failing test cases, are output into a directory named results/. You should ideally capture that directory as part of your CI run to diagnose failures faster.

You should run the tests at least once per day and for all versions that are planned to be published. The test runs usually take approximately 15 minutes, but this may increase with more test coverage.

Sufficient Resources

The integration tests require at least 6GB of memory to run. Some CI systems, for example Bitbucket Pipelines, do not provide this by default, and this manifests with the CI job just hanging indefinitely. Make sure you provide enough memory for the tests.

How Does it Work? ​

The integration tests are essentially an orchestrator around Docker image builds and container runs.

  1. Initially, the relevant containers are built using MetaplaySDK/Dockerfile.server. This includes the game server image itself but also some additional Playwright images for running the tests.
  2. Next, the various tests are run. The game server first starts running in the background, and then the various test suites (simple load testing with BotClient and the functional Playwright tests) run against the game server.

The tests are done using Docker containers to simulate the final target environment closely, as well as to avoid any tooling dependencies on the host machine where the tests are run.

Special Directory Names ​

There are some special directory names that the integration tests recognize in the game-specific backend directory. These tests are invoked during the testing stages if the directories exist. You must adhere to these names if you want the integration tests to invoke your tests!

  • .NET unit tests in Backend/SharedCode.Tests and Backend/Server.Tests. These are invoked without any game server running.
  • Dashboard tests in Backend/Dashboard/tests/unit and Backend/Dashboard/tests/e2e. These are invoked against the running game server and dashboard.
  • Playwright.NET system tests in Backend/System.Tests. These are invoked against the running game server and dashboard.

Customize directories

The special directory names are currently hard-coded in MetaplaySDK/Dockerfile.server. We will eventually replace this with a more customizable system that allows declaring the directories. In the meantime, you can customize the Dockerfile to adjust the paths.