Admission gate has no migration path: pre-#1438 records can never reclaim their name

[khaliqgant]

Summary

The v11.4.2 admission gate (#1438, 5c2ad8ee3) is correct and should not be weakened. It has one gap: it has no migration path for records created before it shipped, so a broker that registered under a name before 2026-08-06 can never reclaim that name after an unclean shutdown. The name is stranded permanently.

// crates/broker/src/relaycast/auth.rs:990 — admit_agent_registration
let reclaims_same_work_unit = matches!(
    (identity_key, existing_identity),
    (Some(ours), Some(theirs)) if hash_identity_key(ours) == theirs
);

Pre-#1438 records carry no stamped identity_key, so existing_identity is None and the arm is unsatisfiable for every possible caller key. No RELAY_AGENT_IDENTITY_KEY value recovers such a name. This is not a matter of finding the right key: stable_node_identity_key() derives it deterministically from the broker's state-directory path, and the stored side simply is not there.

This stranded the name chief during the 2026-08-07 node outage. It is unrecoverable until AgentWorkforce/relaycast#309 (name release without history loss) ships.

Read this before designing the fix: the gate covers one of three registration paths

The incident brief this came from compared two agent records — one refused, one that reclaimed its id twice with no identity_key — and treated the difference as gate behaviour. It is not. They went through different code and were never subject to the same rule. Anyone fixing this needs the full picture, because a fix built on the two-record comparison would be built on a false model.

Path 1 — broker self-registration. GATED.
Rust AuthClient::startup_session_* → register_agent_with_workspace_key (crates/broker/src/relaycast/auth.rs:725) → admit_agent_registration (:959) → POST /v1/agents → 409 → the check above. This is the only caller of admit_agent_registration in the tree. This is the path with the migration gap.

Path 2 — broker-spawned worker. Gated differently, server-side.
Node-control agent.register → relaycast registerAgentViaNode (packages/engine/src/engine/node.ts:1118). An onConflictDoUpdate on (workspace_id, name) that overwrites tokenHash, guarded by:

setWhere: or(
  ne(agents.status, 'active'),
  and(
    eq(agents.locationType, 'via_node'),
    or(
      eq(agents.locationNodeId, nodeId),
      sql`${agents.locationNodeId} = 'node_direct_' || ${agents.id}`,
    ),
  ),
),

Reclaim here is proved by node identity, not by a caller-supplied key. It never consults identity_key.

Path 3 — HTTP fallback when path 2 is unavailable. UNGATED.
ws.rs::register_agent_token (crates/broker/src/relaycast/ws.rs:129) → SDK registerOrRotate (relaycast packages/sdk-typescript/src/relay.ts:485) → on 409, agents.get(name) + agents.rotateToken(name), returning the incumbent's id and a fresh token. The name alone suffices. Taken as the documented fallback at crates/broker/src/runtime/relaycast_events.rs:493-510.

Live evidence that the two populations are path-separated, read from the workspace on 2026-08-07:

agent	type	metadata	path
chief	human	{}	1 — refused, stranded
chief-broker	human	identityKey present	1 — created post-gate
restart-probe-0807	human	identityKey present	1 — created post-gate
chief-khaliq	agent	fleet.nodeId, nodeId, registeredAt: 09:47:06.932Z	2 — reclaimed
marketing-lead	agent	fleet.nodeId, nodeId, registeredAt: 09:47:09.421Z	2 — reclaimed

The registeredAt stamps on the path-2 records are the reclaims themselves. Those agents never reached the gate.

The trap: status is a staleness flag, not a liveness signal — and the API view disagrees with the column

Any adopt-on-not-live design needs a signal for "the incumbent is provably gone". status is not it, for two separate reasons.

1. Nothing ever clears it. sweepStaleAgents (packages/engine/src/engine/agent.ts:298) is written to flip 'active' → 'offline' after STALE_THRESHOLD_MS (5 minutes, :11) — but it has no caller. The only other references in the tree are its own .d.ts declarations under dist/. The loop that drove it was removed in the Fly.io → Cloudflare Workers migration and has not been restored; AgentWorkforce/relaycast#306 tracks re-wiring it and is still open.

Confirmed against the live workspace rather than inferred from the missing caller: of the 329 records whose stored status is 'active', 305 have a lastSeen older than five minutes, and the oldest has been 'active' for 23.9 days. If a five-minute sweep were running, that count would be 24.

So 'active' is not a stale-for-under-five-minutes signal. It means "was alive at some point and never explicitly said otherwise" — it is never revoked on a timer. The 536 'offline' rows come from explicit disconnect and teardown paths (a2a.ts:433, action.ts:1170, inboundWebhook.ts:35, and the node lifecycle in node.ts), not from a sweep, so any code branching on 'offline' is branching on whether something once took the trouble to say so.

2. The serialized value does not match the stored value. Measured against a live workspace of 864 agents on 2026-08-07:

probe	result
GET /v1/agents?status=active (SQL filter on the stored column)	329 rows
GET /v1/agents?status=offline	536 rows
GET /v1/agents?status=online / ?status=unknown	0 rows
status field in the response body for those same 329 rows	"unknown", all 329

The filter matches 'active' in the column (listAgents does eq(agents.status, status) before any serialization, packages/engine/src/engine/agent.ts:109-113), so the stored value is 'active' — but every one of those records serializes as "unknown". Two agents confirmed present in the ?status=active set were actively exchanging messages at the time of the read.

Nothing in the engine ever writes 'unknown' ('active' at :59 and :294, 'offline' at :303), so the divergence is introduced on the way out, in the deployed build. It could not be located in this checkout, which may not match what is deployed.

Consequences for anyone building on this field:

1. Read the column, not the response body. A liveness rule written against the serialized value would see zero live agents in a fully healthy workspace.

2. Validate any candidate signal against the living population, not just a known-dead record. A signal that correctly identifies chief as dead is worthless if it also identifies every running agent as dead. Testing against chief alone would have shown offline and looked correct.

3. Path 2's guard is real and, as deployed, permanent. Its setWhere first disjunct is ne(agents.status, 'active'). The stored column genuinely holds 'active' for any agent that ever came up, and — because nothing sweeps — it keeps holding it indefinitely. So for those records the disjunct is permanently false and the node-identity condition always does the work. Path 2 is more strongly gated than its source suggests, not less.

   The cost is the mirror image, and it belongs with Freeing an agent name requires deleting its history — and the delete is refused for any agent that has ever spoken relaycast#309: an agent that dies uncleanly stays 'active' forever, so its name can only ever be reclaimed by the same node. If that node is gone or its id changes, the name is stranded permanently — the same failure that stranded chief, reached by a different route, on the path the entire resident roster registers through. Releasing the name is the only recovery on either path.

   Note also that restoring the sweep (relaycast#306) would change this security property, not just fix presence reporting: every record it flips to 'offline' becomes claimable by any node, by name alone. That coupling should be deliberate rather than discovered later.

Options, and the one worth arguing for

Option 1 — do not adopt; release instead. Land relaycast#309 and let an operator explicitly release a stranded name. The next honest registration then takes the create branch and stamps identity_key naturally. No inference, no change to the boundary #1438 just established, auditable, and the recovery is a documented command.

Option 2 — adopt-and-stamp on provable non-liveness. When existing_identity is None and the incumbent is provably not live, adopt the record and stamp the hash, converting legacy records on first honest restart. No operator action.

The honest case against Option 2: a legacy path-1 record carries no ownership proof of any kind. chief has empty metadata, and post-gate path-1 records (chief-broker) carry no node binding either, so there is no locationNodeId to fall back on the way path 2 does. Adoption therefore cannot verify ownership — it can only infer it from absence. That is the weakest possible foundation for a change to the exact boundary that exists to stop AR-448 duplicate-agent takeover, and it buys automation of a recovery that Option 1 already makes safe and explicit.

Sizing, since it bears on whether the manual path is tolerable: 226 records in this workspace are type: human with no identityKey — the shape a legacy path-1 record has. How many are broker self-registrations rather than genuine human registrations is not determined by that view and should be established before assuming the migration is small. In practice, though, only one name per node needs reclaiming after any given incident.

Recommendation: Option 1, with Option 2 held until #309 is in and the population is actually characterized. If Option 2 is taken anyway, it needs the most review of anything in this cluster — getting it wrong reopens AR-448.

If Option 2 proceeds, the minimum bar:

• liveness from lastSeen age plus absence of a live node connection, never from status
• the signal demonstrated against currently-running agents, with the false-positive rate stated
• adoption stamped and logged as a distinct, greppable event, so a wrong adoption is visible after the fact rather than silent
• a test that a live incumbent is never adopted away, exercised against the live-population signal rather than a fixture

Related

• Freeing an agent name requires deleting its history — and the delete is refused for any agent that has ever spoken relaycast#309 — release a name without destroying history. Prerequisite for recovering chief, and the whole of Option 1.
• node up spawns AGENT_RELAY_BIN unvalidated, bypassing the broker resolver — 10.x hosts get 'unknown command init' #1451 — node up binary resolution; the fault that triggered this incident.
• Path 3 above is an ungated, name-only takeover of any agent record, reachable from the MCP register_agent tool whenever strict worker identity is off. Same class fix(broker): spawn admission fails closed on agent-name collision #1438 exists to stop. Not filed pending a scope decision — flagged here so it is not lost.
• PR Stop silently discarding agent metadata on registration and spawn #1436 was raised as a risk to this gate on the theory that caller-supplied metadata could write identity_key onto another agent's record. It cannot: registerOrRotate drops metadata entirely on the conflict branch, so supplied metadata only lands when creating a new agent, and Stop silently discarding agent metadata on registration and spawn #1436's own verify_metadata would report metadata_verified: false in the collision case. Separately, PATCH /v1/agents/:name (relaycast packages/engine/src/routes/agent.ts:291) already shallow-merges arbitrary metadata onto any agent under workspace-key auth alone, so that capability exists today without Stop silently discarding agent metadata on registration and spawn #1436. Worth knowing when reasoning about what this gate does and does not defend.

Verification

Rust line references read from origin/main (the gate is not in every local checkout — HEAD at 08c2f290d predates it). Relaycast references from main at 08ddec7. Agent records and status distribution read live from the workspace on 2026-08-07, not from a persisted state file.

Context

Filed from an incident brief covering the 2026-08-07 node outage. Do not merge without Khaliq — he owns the merge gate.

[khaliqgant]

Correction to the status measurement in this issue, applied to the body above.

An earlier revision of this issue stated that status = 'active' was zero across all 864 agents. That was measured from the agent list response body and is wrong about the stored column. Probing the SQL filter directly (?status=active, which matches on the column before serialization) returns 329 rows.

The stored column does hold 'active' for live agents. The response body renders those same records as "unknown".

This matters because the wrong reading supported a much larger conclusion — that path 2's ne(status,'active') disjunct is always true and its node-identity guard therefore does nothing. That conclusion does not hold. The guard is real for any agent inside the 5-minute staleness window. What survives is narrower and is now in the body: once sweepStaleAgents flips a record to 'offline', the node-identity requirement drops away.

Recording the error rather than quietly editing it, because the near-miss is the useful part: the field that was trusted was checked against a dead record, which looked correct, and only a measurement against provably-live agents exposed the divergence — and then a second measurement against the column exposed that the first measurement was itself reading a lying view.

[khaliqgant]

Second correction, applied to the body: sweepStaleAgents has no caller, so the "five-minute window" in the previous revision was wrong.

The prior revision said path 2's node-identity guard lapses after five minutes, once the sweep flips a record to 'offline'. The sweep does not run. sweepStaleAgents has exactly one reference in the tree — its own definition — plus .d.ts declarations under dist/. Its driving loop was removed in the Fly.io → Cloudflare Workers migration; AgentWorkforce/relaycast#306 tracks restoring it and is still open.

Measured against the live workspace rather than inferred from the absent caller: of 329 records stored as 'active', 305 have lastSeen older than five minutes and the oldest has been 'active' for 23.9 days.

The correction runs in the safe direction — path 2 is gated more strongly than stated, effectively permanently, not for five minutes. But it inverts the consequence: because 'active' is never revoked, an agent that dies uncleanly can only ever have its name reclaimed by the same node, so a lost or renamed node strands the name permanently. That is the same failure that stranded chief, on the path the whole resident roster uses, and it makes AgentWorkforce/relaycast#309 the only recovery on either path.

It also means relaycast#306 is not purely a presence fix: restoring the sweep would make every swept record claimable by any node by name alone. Worth deciding deliberately.

Recording rather than silently editing — this is the second time in this issue that a conclusion about status was wrong, both times because the field was read without checking whether anything maintains it.

[khaliqgant]

Provenance note for anyone checking these references.

The gate is not in every working tree. All Rust references in this issue (crates/broker/src/relaycast/auth.rs) were read from origin/main. The v11.4.2 admission gate landed in 5c2ad8ee3, and a checkout that has not pulled since will not contain admit_agent_registration at all — the local tree this was investigated from sits at 08c2f290d, which predates it. If you open the file and the function is missing, pull before concluding the reference is wrong.

The deployed artifact is not necessarily any checkout. relaycast references here (registerAgentViaNode, sweepStaleAgents, the engine's status writers) were read from relaycast main at 08ddec7. The deployed engine demonstrably carries code no local tree has — see AgentWorkforce/relaycast#312, where stored 'active' serializes as "unknown" via a mapping that exists nowhere in the source. Treat file:line here as provisional against the running build.

What does not depend on either caveat: every measurement in this issue was taken against the live workspace, not from source and not from a persisted state file — the 329/536 status partition, the 305 records stale beyond five minutes, the 23.9-day-old 'active' record, the 226 legacy-shaped records, and the per-agent metadata shapes. Those stand regardless of which build is deployed.