Backend environment for match hosting for lance.blue
arena DEVELOPING.md
9.9 kB

Developing #

Docker is the only requirement. There is no host build.

Docker #

./scripts/build.sh       build the image
./scripts/test.sh        lint, syntax, unit and compile checks
./scripts/run.sh         play a match; http://localhost:8080/ lists the
                         per-seat clients
./scripts/push.sh        build and publish; REGISTRY overrides the default
./scripts/deploy.sh      push, then point infra at what was pushed

Driving a match from a script #

./scripts/run.sh is for a person: it holds the terminal until Ctrl-C. ./scripts/arena.sh is the same match, same image and same manifest, for something that has to start it, wait for it, measure something and stop it.

./scripts/arena.sh up          # returns once the endpoint answers
./scripts/arena.sh status      # one JSON object
./scripts/arena.sh logs -f
./scripts/arena.sh down        # safe when nothing is running

up returns when http://localhost:8080/ answers, not when docker accepted the run, and when it never answers it fails with the container's last log lines rather than hanging. Exit codes say why: 2 a match is already running, 3 it never served, 4 nothing is running.

Both scripts cap every container at the production shape: 2 CPUs, 8GB, no swap, 512 pids. Three JVMs with no cap take the whole machine, and a run on sixteen cores measures a shape nothing is deployed on. ARENA_CPUS, ARENA_MEMORY and ARENA_PIDS move the caps within bounds — 1 to 8 CPUs, 2 to 12GB — and nothing removes them. A value outside the bounds is clamped, and the clamp is printed along with the caps in effect.

One match per session, not one per machine. Several sessions work on this box at once - agents, and you - so a container is owned by the checkout that started it (ARENA_SESSION overrides), gets its own name, its own port and its own state directory. down stops that session's match and names anything else it found; down --all takes every arena container, which is somebody else's work, so ask first. ARENA_MAX_MATCHES (default 2) is what protects the laptop now that other sessions no longer block a start.

down stops before it removes, because the container's whole exit path hangs off SIGTERM: it flushes profiles, then runs finalize. ARENA_STOP_TIMEOUT bounds the wait at 60s, after which docker kills it anyway.

A match that plays itself #

./scripts/arena.sh up --spectate

Nobody plays. Princess takes every faction, the game starts without waiting for a lobby Done and runs to victory on its own, and the browser joins as ArenaSpectator - flagged observer server-side, so it never owns a turn and never stalls the game. HUMAN=- ./scripts/run.sh is the same thing interactively.

This is what profiling wants: a board that repaints continuously with nothing driving it. An ordinary match sits in the lobby until someone presses Done, so a profile of one is mostly an idle EDT.

There is no new manifest field. A manifest with no human players has no other sensible meaning, so container/init/20-identity.sh reads it as spectator mode; headquarters never sends one.

Profiling a local match #

ARENA_PROFILE=1 ./scripts/arena.sh up
# play, or drive it; the longer it runs the more there is to look at
./scripts/arena.sh down

Both JVMs get async-profiler - the client on wall clock, the match host on ctimer - and down keeps what finalize could not upload in .arena-artifacts/<container>/: the collapsed stacks and the stats table reduced from Suramadu's per-session metrics. ARENA_ARTIFACTS_DIR moves that.

Collapsed stacks are one line per stack, frames separated by ;, sample count last. To see what the EDT was doing rather than what every parked thread was, filter to it first:

awk '/EventDispatchThread.run/ && !/getNextEvent/' profile-client.collapsed

A short run may end before the client JVM has a session to flush, and then only profile-host.collapsed is there.

Which build is in play #

A machine that has built this repo for a week holds a dozen tags, across two Suramadu versions and one renaming. ./scripts/image.sh answers which is which without reading tag strings by eye.

./scripts/image.sh list                   what is on disk
./scripts/image.sh resolve                the tag versions.env names
./scripts/image.sh resolve newest         the most recent build
./scripts/image.sh resolve c836f26        the build made from that commit
./scripts/image.sh verify newest          resolve it, and say what it is
./scripts/image.sh diff 4e5403f c836f26   what changed between two builds

resolve prints one reference on stdout and nothing else, so a script can ask it rather than composing IMAGE_NAME and IMAGE_TAG for itself. It fails if the image is not on the machine, and names ./scripts/build.sh when the miss is the ordinary one — versions.env names a tag for this commit and no build has produced it yet.

list prints one row per image, not per tag: build.sh tags every build latest as well, so those two names are one set of layers and two rows would imply disk usage that is not there. The size is docker image inspect .Size. The SIZE column in docker images is much larger and is not a second opinion — this daemon uses the containerd snapshotter, so that column adds the unpacked snapshot to the content. IN REPO is whether the commit in the tag is one you can still git show; no means nothing can rebuild that image.

Two builds can carry the same Created timestamp to the nanosecond, and docker image ls orders those ties differently on each run. newest breaks them with git — the newest of a tied set is the one with the others behind it — and refuses rather than guessing when they are not on one line of history.

diff is read-only: sizes, labels, the commit range, and where the layer histories part. diff --deep adds one throwaway container per image to read versions.env and the container scripts out of them, which is the only way to compare what a build actually contains.

Nothing here deletes an image. list reports what untagged images are holding and leaves the command to you.

./scripts/arena.sh up -i <selector> takes the same selectors, so a match can be started against a named build rather than whatever versions.env points at.

Deploying #

./scripts/deploy.sh runs the push and then writes the tag it produced into the sibling infra checkout's envs/lance.blue/releases.auto.tfvars.json. It leaves one uncommitted diff there and stops: applying is the deploy, and this script applies nothing. It prints the two tofu commands to run.

It refuses before building if the tree is dirty — a tag nobody can rebuild is worse than a failed deploy. Any branch can be deployed, because the branch name is part of the tag and what is serving stays legible. On main it also refuses unless HEAD is exactly origin/main, neither ahead nor behind: that is the one branch a build off it is expected to match. The state of the infra checkout is not its business: headquarters writes its own keys in the same file, so an uncommitted bump there is normal. INFRA_DIR overrides where the infra checkout is; AWS_PROFILE is required.

headquarters has the same script for its own two artifacts.

test.sh builds the test stage, so the tests see the same JDK and MegaMek jar the build does. It tags nothing — the stage passes or the build fails.

Editing container scripts #

container/ is copied into the image, so an edit to an init, watch or exit script needs ./scripts/build.sh before it takes effect. A stale image fails exactly the way the old code did, which is an easy way to lose ten minutes.

To iterate without rebuilding, mount the working copy instead:

LIVE_SCRIPTS=1 ./scripts/run.sh

Off by default, so a plain ./scripts/run.sh tests what was actually built. The banner says which mode you are in.

Working on one stage #

Stages are base, jre, megamek, suramadu, arena, test, profiler, runtime. Build just one, then poke around inside it:

docker build --target megamek -t arena-mm .
docker run --rm -it --entrypoint /bin/bash arena-mm

Useful when a patch stops applying or a compile fails: the stage stops with everything still in place.

Clearing caches #

Two separate caches, and the distinction matters because one is expensive to refill.

what command
cache mounts: downloads (~1.1GB) and apt docker builder prune -f --filter type=exec.cachemount
all build cache: layers, mounts, frontends docker builder prune -af
everything Docker holds, incl. images and volumes docker system prune -af --volumes

See what is there first with docker system df -v.

Notes:

  • --no-cache rebuilds every layer but does not empty the download cache. Dropping the downloads needs the exec.cachemount filter above.
  • exec.cachemount is the BuildKit cache-record type for --mount=type=cache. A filter of type=cache matches nothing and prunes nothing while looking like it worked.
  • docker system prune -af --volumes deletes any image not attached to a running container — including lance-blue/arena — and any unused volume from any project. Use it when you want the disk back, not to fix a build.

A corrupt download #

Downloads are fetched to .part, verified, and only then renamed, so an interrupted transfer never becomes a cache entry. A cached file that fails its check is deleted and refetched. Pruning is not the fix — just build again.

Every archive is pinned by SHA-256 in versions.env and verified before the cache entry is created — the JDK against the vendor's published digest, MegaMek and Suramadu against digests recorded from a verified fetch.