Skip to content

Reading Along: ZoneWorld

Guide Home | Previous: Reading Along: GameQuest | Next: 15. E2E Testing — Verifying the Whole System with a Client

View in another language — C++ · C#/.NET · Java · Kotlin · Node/TypeScript

What you get from this chapter

You can open the ZoneWorld sample in an editor and follow, in the code, the path of a player who enters the world and crosses a zone boundary onto another node, and the path by which the ops console's announcements and maintenance reach every node. The code in this chapter runs as it stands in framework/languages/java/samples/java/ZoneWorld.

Picking a Sample introduced what this sample demonstrates. This chapter is what you read after that introduction — the roles and where their code lives, the message flow of the main scenarios, and, for each flow, the framework feature it uses and the chapter that explains it, in the order the source is laid out. This chapter has no spec document that owns a contract. The requirements, message contract and verification criteria are owned by the ZoneWorld scenario, and this chapter does not restate them.

1. What This Sample Demonstrates

The world is divided into zones, and each zone is a User Spot with an id. Which ZoneNode owns which zone is decided by Location Store placement, not by configuration. A player is an Actor with an id, and crossing a zone boundary means joining the neighbouring zone Spot — when the owner differs, the framework moves the Actor, and the client's connection stays as it is. The ops console sends announcements and maintenance without a list of nodes.

↗ View larger

The flow this chapter follows is entry and binding → moving and the boundary join → relocation and join completion → the zone tick's pushes and border snapshots → the ops fanout and node observation. The subject of this sample is that "doing something to several nodes" needs a different surface in each situation, and the code for each surface appears in that order.

2. Roles and Where the Code Lives

Role Processes Owns Code
Gateway 1 The game STREAM, player Actor binding, relay and pushes Server/Gateway
ZoneNode 2 The Entry Spot, zone Spots, player Actors, bot timers, local reports Server/ZoneNode
Ops 1 The ops STREAM, runtime event collection, fanout publish, the maintenance store Server/Ops
Client browser The game screen and the ops console Client

Movement rules and zone coordinates live in Server/ZoneNode/Domain and reference no framework type. Messages are set by the JSON contract in shared.

3. Server Configuration — Mesh, Capacity and Fanout

ZoneNode registers the Entry Spot, the player Actor factory and the zone Spot factory. The zone Spot factory declares a per-node capacity — every ZoneNode requests every zone, and the capacity is what makes each node own only some of them.

Server/src/main/java/systems/zlink/samples/zoneworld/server/Program.java

mesh.objects()
        .server()
        .addEntrySpot(ZoneEntrySpot.class)
        .addSpotFactory(
                ZoneWorldNames.ZONE_SPOT_TYPE,
                ZoneSpot.class,
                factory ->
                        factory.stableTypeLimit(topology.zoneCapacityValue())
                                .disableRelocation())
        .addActorFactory(
                ZoneWorldNames.PLAYER_ACTOR_TYPE,
                PlayerActor.class,
                PlayerActorFactory.class,
                factory ->
                        factory.preserveStateWith(
                                PlayerActorRelocationAdapter.class));

The player Actor factory names an adapter that preserves state, and the zone Spot factory disables relocation. Separately from the mesh, the node registers a subscriber on the classic fanout channel.

Server/src/main/java/systems/zlink/samples/zoneworld/server/Program.java

options.addFanoutChannel(ZoneWorldNames.BROADCAST_CHANNEL)
        .enableSubscriber()
        .addHandlerGroup(ZoneWorldNames.BROADCAST_HANDLER_GROUP);

The Ops side is the publisher on the same fanout channel.

Server/src/main/java/systems/zlink/samples/zoneworld/server/Program.java

options.addFanoutChannel(ZoneWorldNames.BROADCAST_CHANNEL)
        .setRoutingIdPrefix("zoneworld-ops-broadcast")
        .enablePublisher();

Placement by capacity is covered by Activation and Lifetime, and the fanout channel by Channel Messaging.

4. Entry — Binding and the First Zone Join

When the browser connects to Gateway and sends a join, the session creates or finds the Actor by PlayerId, binds it to the current session, and relays the join packet to that Actor.

↗ View larger

Server/src/main/java/systems/zlink/samples/zoneworld/server/gateway/GameSession.java

return actors.getOrCreate(request.playerId(), ZoneWorldNames.PLAYER_ACTOR_TYPE)
        .inMesh(ZoneWorldNames.MESH)
        .request(ZLinkMessage.empty())
        .submit()
        .thenCompose(
                result -> {
                    if (result instanceof ZLinkActorCreateResult.Rejected rejected) {
                        return CompletableFuture.failedFuture(
                                new IllegalStateException(
                                        rejected.reply().isEmpty()
                                                ? "Player actor creation rejected"
                                                : rejected.reply()
                                                        .decode(
                                                                Messages.EnterWorldRes
                                                                        .class)
                                                        .error()));
                    }
                    ActorRef actorRef =
                            result instanceof ZLinkActorCreateResult.Created created
                                    ? created.actor()
                                    : ((ZLinkActorCreateResult.Existing) result).actor();
                    return context.actors()
                            .bindOrGet(actorRef)
                            .thenCompose(actor -> actor.relay(dispatch, payload));
                });

The Actor starts in the Entry Spot. The Entry Spot's handler derives the zone from the coordinates and reserves a join into that zone Spot.

Server/src/main/java/systems/zlink/samples/zoneworld/server/zone/handlers/EntryZoneJoinHandler.java

actor.prepareEntry(ZoneWorldSpec.SPAWN_X, ZoneWorldSpec.SPAWN_Y, false, 0, 0);
String zone = ZoneWorldSpec.zoneOf(ZoneWorldSpec.SPAWN_X, ZoneWorldSpec.SPAWN_Y);
actor.context()
        .joinSpot(
                zone,
                new Messages.EnterZoneReq(
                        actor.actorId(),
                        ZoneWorldSpec.SPAWN_X,
                        ZoneWorldSpec.SPAWN_Y,
                        false,
                        true,
                        "",
                        false))
        .timeout(Duration.ofSeconds(10))
        .defer();

The join runs after the handler returns, and its result arrives at the Actor's completion callback. The entry reply is sent from that callback — receiving it means admission into the target zone has completed. Binding is covered by Session and Actor, and the reservation rules by Actor Membership.

5. Movement — Inside a Zone and Across a Boundary

A move request is relayed to the bound Actor and arrives at the zone Spot's actor send handler. The domain decides, and the result splits into a rejection, a move inside the zone, or a zone change.

Server/src/main/java/systems/zlink/samples/zoneworld/server/zone/spots/ZoneSpot.java

ZoneWorldSpec.MoveDecision decision =
        ZoneWorldSpec.validateMove(actor.x(), actor.y(), targetX, targetY);
if (!decision.accepted()) {
    if (!actor.isBot()) {
        return actor.send(
                new Messages.MoveRejectedNotify(decision.reason(), actor.x(), actor.y()));
    }
    return CompletableFuture.completedFuture(null);
}
String targetZone = ZoneWorldSpec.zoneOf(targetX, targetY);
if (!decision.zoneChanged()) {
    actor.updatePosition(targetX, targetY);
    census.record(context.spotId(), residents.size());
    context.outbound()
            .sendToSpot(
                    context.spotId(),
                    new Messages.UpdatePositionMsg(
                            actor.actorId(), targetX, targetY, actor.isBot()))
            .submit();
    if (actor.isBot()) return CompletableFuture.completedFuture(null);
    return actor.send(new Messages.ZoneStateNotify(context.spotId(), tick, statePlayers()));
}

Inside the zone, the Actor updates its coordinate and sends the copy update to the zone Spot. When the zone changes, it reserves a join into the target zone Spot.

Server/src/main/java/systems/zlink/samples/zoneworld/server/zone/spots/ZoneSpot.java

actor.prepareMove(targetX, targetY, targetZone);
actor.context()
        .joinSpot(
                targetZone,
                new Messages.EnterZoneReq(
                        actor.actorId(),
                        targetX,
                        targetY,
                        actor.isBot(),
                        false,
                        actor.zoneId(),
                        false))
        .timeout(Duration.ofSeconds(10))
        .defer();

The application does not distinguish whether the target's owner is the same node or another. On the same node only the membership changes; on another node the framework moves the Actor inside the join. Messages that reach the previous owner during the move are forwarded to the target.

6. Relocation and Join Completion

When the owner changes, the Actor's application state is serialized by the adapter named on the factory. It carries only the coordinate, the zone, the bot direction and the pending join; queues and timers are moved by the framework.

↗ View larger

Server/src/main/java/systems/zlink/samples/zoneworld/server/zone/actors/PlayerActorRelocationAdapter.java

@Override
public CompletionStage<byte[]> capture(
        PlayerActor actor, ZLinkRelocationCancellation cancellation) {
    try {
        return CompletableFuture.completedFuture(
                JSON.writeValueAsBytes(
                        new State(
                                actor.x(),
                                actor.y(),
                                actor.zoneId(),
                                actor.isBot(),
                                actor.dirX(),
                                actor.dirY(),
                                actor.pendingX(),
                                actor.pendingY(),
                                actor.pendingZone(),
                                actor.pendingJoin(),
                                actor.pendingPurpose(),
                                actor.completedJoins().stream()
                                        .map(
                                                value ->
                                                        new OperationId(
                                                                value.high(), value.low()))
                                        .toList())));
    } catch (Exception error) {
        return CompletableFuture.failedFuture(error);
    }
}

The target zone Spot's admission is the final judge. If that node is under maintenance it rejects the join; otherwise it records the join as pending and accepts.

Server/src/main/java/systems/zlink/samples/zoneworld/server/zone/spots/ZoneSpot.java

if (maintenance.rejectsArrival(topology.nodeId(), zone, join.fromZoneId())) {
    return CompletableFuture.completedFuture(
            ZLinkSpotActorJoinResult.reject(
                    new Messages.EnterZoneRes(zone, "ZoneMaintenance")));
}
if (join.crashBoundaryProbe()) {
    System.out.println("crash-boundary join pending zone=" + zone + " actor=" + actorId);
    return new CompletableFuture<>();
}
pendingJoins.put(actorId, join);
return CompletableFuture.completedFuture(
        ZLinkSpotActorJoinResult.accept(new Messages.EnterZoneRes(zone, null)));

The join's result arrives at the Actor's completion callback. The callback filters duplicates by the operation id, then sends the entry reply on acceptance or a failure notify on rejection to the bound session. A rejected move leaves the coordinate where it was.

Server/src/main/java/systems/zlink/samples/zoneworld/server/zone/actors/PlayerActor.java

@Override
public CompletionStage<Void> onJoinCompleted(ZLinkActorJoinCompletion completion) {
    ZLinkActorJoinOperationId operationId =
            completion instanceof ZLinkActorJoinCompletion.Accepted accepted
                    ? accepted.operationId()
                    : completion instanceof ZLinkActorJoinCompletion.Rejected rejected
                            ? rejected.operationId()
                            : ((ZLinkActorJoinCompletion.Failed) completion).operationId();
    if (completedJoins.contains(operationId)) return CompletableFuture.completedFuture(null);
    rememberJoin(operationId);

    pendingJoin = false;
    if (completion instanceof ZLinkActorJoinCompletion.Accepted accepted) {
        Messages.EnterZoneRes reply = accepted.reply().decode(Messages.EnterZoneRes.class);
        String joinedZone =
                reply.zoneId().isBlank()
                        ? pendingZone == null || pendingZone.isBlank()
                                ? ZoneWorldSpec.zoneOf(pendingX, pendingY)
                                : pendingZone
                        : reply.zoneId();
        applyAtZone(pendingX, pendingY, joinedZone, isBot);
        pendingZone = null;
        if (pendingPurpose == JoinPurpose.INITIAL_HUMAN) {
            pendingPurpose = JoinPurpose.NONE;
            return send(new Messages.JoinWorldNotify(actorId, joinedZone, x, y, null));
        }
        if (pendingPurpose == JoinPurpose.CRASH_PROBE) {
            pendingPurpose = JoinPurpose.NONE;
            return send(new Messages.CrashRelocationProbeRes(null));
        }
        pendingPurpose = JoinPurpose.NONE;
        return CompletableFuture.completedFuture(null);
    }

    String reason =
            completion instanceof ZLinkActorJoinCompletion.Rejected rejected
                    ? rejected.reply().decode(Messages.EnterZoneRes.class).error()
                    : mapFailure(((ZLinkActorJoinCompletion.Failed) completion).kind().name());
    pendingZone = null;
    if (pendingPurpose == JoinPurpose.INITIAL_HUMAN) {
        pendingPurpose = JoinPurpose.NONE;
        return send(
                new Messages.JoinWorldNotify(
                        actorId,
                        ZoneWorldSpec.zoneOf(pendingX, pendingY),
                        pendingX,
                        pendingY,
                        reason));
    }
    if (pendingPurpose == JoinPurpose.CRASH_PROBE) {
        pendingPurpose = JoinPurpose.NONE;
        return send(new Messages.CrashRelocationProbeRes(reason));
    }
    pendingPurpose = JoinPurpose.NONE;
    if (!isBot) return send(new Messages.MoveRejectedNotify(reason, x, y));
    reverseDirection();
    return CompletableFuture.completedFuture(null);
}

What survives a move and the adapter's part are covered by Relocation, and the rule that the connection is kept during the move by How Session Binding Works.

7. The Zone Tick — Pushes and Border Snapshots

The zone Spot ticks on a timer and builds a state notify from its own zone and the neighbouring snapshots. The notify is sent to each player Actor, and the Actor's handler pushes it to the bound session. A bot is an Actor of the same type without a bound session.

▶ Try it in the interactive tour — RouteMesh pub/sub

Server/src/main/java/systems/zlink/samples/zoneworld/server/zone/actors/PlayerActor.java

public CompletionStage<Void> send(Object message) {
    if (isBot) return CompletableFuture.completedFuture(null);
    return context.boundSession().send(message).submit();
}

The state near the border is published on a topic per neighbouring zone. Both the sending and the receiving zone are in the topic name, so unrelated zones never receive it.

Server/src/main/java/systems/zlink/samples/zoneworld/server/zone/spots/ZoneSpot.java

CompletionStage<Void> published = CompletableFuture.completedFuture(null);
for (String target : ZoneWorldSpec.adjacentZones(context.spotId())) {
    List<Messages.PlayerView> border =
            residents.values().stream()
                    .filter(
                            actor ->
                                    ZoneWorldSpec.inBorderBand(
                                            actor.x(), actor.y(), context.spotId(), target))
                    .map(
                            actor ->
                                    new Messages.PlayerView(
                                            actor.actorId(),
                                            actor.x(),
                                            actor.y(),
                                            context.spotId(),
                                            actor.isBot()))
                    .sorted(
                            Comparator.comparing(
                                    Messages.PlayerView::playerId,
                                    ZoneWorldSpec.UTF8_ORDER))
                    .toList();
    Messages.ZoneBorderEvent event =
            new Messages.ZoneBorderEvent(context.spotId(), target, tick, border);
    published =
            published.thenCompose(
                    ignored ->
                            context.outbound()
                                    .publish(
                                            ZoneWorldNames.ZONE_CHANNEL,
                                            ZoneWorldNames.borderTopic(
                                                    context.spotId(), target),
                                            event)
                                    .submit());
}
return published;

Each zone Spot subscribes only to the topics coming from its own neighbours.

Server/src/main/java/systems/zlink/samples/zoneworld/server/zone/handlers/BorderSubscriptionHandlers.java

public static Class<?> forRoute(String fromZoneId, String toZoneId) {
    return switch (ZoneWorldNames.borderTopic(fromZoneId, toZoneId)) {
        case ZoneWorldNames.NW_NE -> NorthWestToNorthEast.class;
        case ZoneWorldNames.NW_SW -> NorthWestToSouthWest.class;
        case ZoneWorldNames.NE_NW -> NorthEastToNorthWest.class;
        case ZoneWorldNames.NE_SE -> NorthEastToSouthEast.class;
        case ZoneWorldNames.SW_NW -> SouthWestToNorthWest.class;
        case ZoneWorldNames.SW_SE -> SouthWestToSouthEast.class;
        case ZoneWorldNames.SE_NE -> SouthEastToNorthEast.class;
        case ZoneWorldNames.SE_SW -> SouthEastToSouthWest.class;
        default ->
                throw new IllegalArgumentException(
                        "unknown border route: " + fromZoneId + " -> " + toZoneId);
    };
}

Timers are covered by Timers and Workers, and Logical Multicast that picks targets by topic by How Channels Work.

8. Operations — Fanout and Node Observation

Ops publishes announcements and the maintenance desired state over fanout. The publisher holds no list of nodes — adding a node changes nothing on this side.

↗ View larger

Server/src/main/java/systems/zlink/samples/zoneworld/server/ops/OpsSession.java

return fanout.publish(
                ZoneWorldNames.BROADCAST_CHANNEL,
                ZoneWorldNames.MAINTENANCE_TOPIC,
                new Messages.NodeMaintenanceChangedEvent(
                        request.nodeId(), request.enabled()))
        .submit()
        .thenCompose(
                ignored ->
                        reply(
                                new Messages.SetMaintenanceRes(
                                        request.nodeId(),
                                        request.enabled(),
                                        node.zones(),
                                        null)));

The subscriber on each ZoneNode applies only its own NodeId's share to the local policy. Even when this cache is stale, the final judge is the target admission of the previous section.

Server/src/main/java/systems/zlink/samples/zoneworld/server/zone/handlers/NodeMaintenanceSubscriber.java

@Override
public CompletionStage<Void> handle(
        Messages.NodeMaintenanceChangedEvent message, ZLinkPublishMessageContext context) {
    state.apply(message.nodeId(), message.enabled());
    store.set(message.nodeId(), message.enabled());
    if (topology.nodeId().equals(message.nodeId())) {
        System.out.println(
                "maintenance state node=" + message.nodeId() + " enabled=" + message.enabled());
        ZoneStatusReporter reporter = statusReporter.getIfAvailable();
        if (reporter != null) {
            return reporter.reportNow();
        }
    }
    return CompletableFuture.completedFuture(null);
}

Whether a node is registered and connected is learned by observing runtime status, not by a request. Ops observes the mesh's status and turns changes in the set of Ready peers into node connection state.

Server/src/main/java/systems/zlink/samples/zoneworld/server/ops/NodeLivenessObserver.java

observation = runtime.observe(ZoneWorldNames.MESH, 32);
observation.subscribe(
        new Flow.Subscriber<ZLinkObservedStatus<ZLinkMeshNodeSnapshot>>() {
            @Override
            public void onSubscribe(Flow.Subscription subscription) {
                subscription.request(Long.MAX_VALUE);
            }

            @Override
            public void onNext(ZLinkObservedStatus<ZLinkMeshNodeSnapshot> observed) {
                registry.applyLiveRoutingIds(readyRoutingIds(observed.status()));
            }

            @Override
            public void onError(Throwable error) {
                System.out.println(
                        "node observation error mesh="
                                + ZoneWorldNames.MESH
                                + " detail="
                                + error.getMessage());
            }

            @Override
            public void onComplete() {}
        });

The forms of fanout are covered by How Channels Work, and runtime status and drain by Operations and Lifecycle.

9. Running and Verifying

One runner starts the Redis container, the server processes and the browser scenario together.

framework/languages/java/samples/java/ZoneWorld/run_sample.sh

The browser scenario checks the boundary move, the connection that survives relocation, the arrival of announcements, and the admission rejection under maintenance. The checks and the scenario ids are set by the ZoneWorld scenario.