DistrictServer exits when WorldServer TCP control link drops; world keeps stale district registrations that block re-registration

[eaxiumnet]

Summary

Two coupled lifecycle gaps make every WorldServer restart (or a district's own restart) bring the whole stack down and leave the world unable to re-accept district registrations until the WorldServer itself is restarted. Observed three times in one day of testing (2026-08-17) during routine deploys.

1. DistrictServer hard-exits when the WorldServer TCP control connection dies

DistrictServer/DistrictServer.cpp main() (end of file):

while (true)
{
    std::unique_ptr<char[]> prefixBuffer(network.Receive(1));
    if (!prefixBuffer)
        break;
    ...
}
Logger(lERROR, "main()", "WorldServer control connection ended; exiting in 5 seconds...");
Sleep(5000);
return 1;

When the WorldServer process stops (restart, crash, deploy), its TCP socket dies (WSAECONNRESET 10054), Receive() fails, and the DistrictServer exits unconditionally — no reconnect, no re-register, no retry loop anywhere in the process. There is no world-reconnect logic in the DistrictServer at all.

Observed log line from the Social instance on 2026-08-17 ~00:45 during a WorldServer restart:

Network::Receive() failed 10054 → WorldServer control connection ended; exiting in 5 seconds

Both district instances exited the same way. Consequences:

• Any WorldServer restart takes every district player offline (active sessions die mid-game, as happened at 00:31 when a deploy dropped a live Financial session and triggered a client GPF).
• Operational ordering becomes load-bearing: districts must be started after the world and manually re-run after every world restart.

Fix direction: replace return 1 with a reconnect loop — on control-link failure, log, sleep (configurable backoff), Connect() again, re-send the registration record — so a world restart only needs the districts to be (re)started once, or is even self-healing if the loop lives in the DS.

2. WorldServer keeps stale district registrations; cleanup code is broken

WorldServer/District/Listener.cs handleDistrict() teardown on disconnect:

lock (DistrictsTcp)
{
    foreach (KeyValuePair<TcpClient, uint> dtcp in DistrictsTcp)   // <-- iteration order = insertion order
    {
        lock (Districts)
        {
            foreach (KeyValuePair<uint, District> dis in Districts)
            {
                if (dis.Key == dtcp.Value)
                {
                    Log.Error("Listener", dis.Value.ToString() + " disconnected!");   // logs, then falls out of the inner loop
                    break;
                }
            }
            Districts.Remove(dtcp.Value);   // removes the code of the FIRST DistrictsTcp entry, not the disconnecting client
            break;                          // <-- outer loop stops after the first entry
        }
    }
    DistrictsTcp.Remove(tcpClient);         // only this line actually targets the dying client
}

The teardown only works if the disconnecting district happens to be the first one ever registered in this world process. In every other case:

• The wrong district's entry is removed from Districts (the first-inserted one), while
• the actual disconnected district's entry survives forever — the world keeps a stale registration for a dead TCP socket.

On top of that, RegisterDistrict.Register() (second overload) rejects a re-registering instance with 0x30 0x32 (Log.Error("RegisterDistrict", "Fail try of district registration that already exists!")) when its IP differs from the stale entry, and DistrictsTcp accumulates dead TcpClient objects on every re-register:

Program.districtsListener.DistrictsTcp.Add(district.tcp, code);   // old dead clients are never pruned

Observed 2026-08-17 ~01:00: after restarting only the three district instances (WorldServer kept running), the Social instance's re-registration was not accepted — the world held a stale Social entry from the dead process. Restarting the WorldServer cleared it and all three registered again within seconds (Safe-Social-EN-1 was registered! (127.0.0.1:6969), Financial 6970, Waterfront 6971).

Fix directions:

• Fix the teardown: look up the code paired with the actual disconnecting tcpClient in DistrictsTcp, remove exactly that entry from both dictionaries, and drop the stray Districts.Remove(dtcp.Value) / outer break.
• On collision in RegisterDistrict.Register(): replace the stale entry unconditionally (the same-IP path already tries this; the foreach + break + mid-loop mutation makes it unreliable).
• Optionally prune DistrictsTcp of dead clients and log a warning when a re-registration replaces a still-listed entry.

Repro

1. Start Lobby → World → three districts; confirm all three register.
2. Restart only the district instances (or only the WorldServer).
3. Observe: every district exits within ~5 s of a world restart ("exiting in 5 seconds") and/or re-registration is rejected/silently misattributed while the world runs with the stale entries.
4. Restarting the WorldServer clears the stale state.

Impact

• Every world restart takes the whole district stack down with it (hit three times in one day of testing).
• Stale registrations can reject fresh instances or leave the world advertising a dead district, surfacing client-side as "connecting to district" loops with no packet activity.

Related

• LobbyServer/WorldServer crash with NullReferenceException when launched headless (stdin closed) #2 (headless NRE) — the same restart cadence that exposed both problems
• Support multiple concurrently registered district servers #7 (multi-district registration PR) — makes stale-entry handling more visible with 3 concurrent instances