Skip to content

engine: node.register resets delivery readiness and silently drops deliver frames until agent.register/inventory.sync #270

Description

@willwashburn

Found while root-causing a cross-node "reply never injected" report (fixed on the relay side by AgentWorkforce/relay#1273 — that repro turned out to be drive-mode hold, but this engine hazard was confirmed during the trace and can produce the same symptom on its own).

Problem

The 6.0.3 broker cursor negotiation (b510538, #254) introduced an agent_scoped delivery-readiness gate with two silent failure modes:

  • Every node.register resets the provider connection's ready-set to empty (src/engine/node.ts ~1499–1505; adapter src/adapters/node/realtime.ts ~362–375), and node-level replay is deliberately skipped for cursor-negotiating brokers (node.ts ~1527–1535). Identities only become ready again via a per-agent agent.register or inventory.sync.
  • While an identity is not ready, live dispatch silently returnsroutes/deliveryRouting.ts ~419–427 leaves the row queued with no log, no last_dispatch_error, no retry — and sendToProvider re-drops deliver frames at adapters/node/realtime.ts ~244–249 (only action.invoke frames are queued in memory; deliver frames are not).

If a broker re-sends node.register (reconnect, Wi-Fi blip, engine restart losing in-memory readiness) and does not follow with inventory.sync/agent.register for every hosted agent, every deliver frame to that node is dropped silently. Sending still works (token-authenticated REST), so the failure is asymmetric and invisible until the sender gets delivery.failed / ttl_expired ~1h later. nodeDeliver.ts ~132–140 gates ephemeral events on the same predicate, so all frames to the node go dark.

Suggested fixes

  • Log (at warn) when a deliver frame is skipped for readiness, and stamp the delivery row (last_dispatch_error) so the drop is observable.
  • Consider replaying/marking ready from the broker's inventory.sync-equivalent state automatically on re-register, or fail the node.register negotiation loudly when readiness can never be satisfied.
  • Conformance tests cover the reset (__tests__/conformance/delivery.test.ts ~550–704, 921–999) but there's no test asserting the recovery path when a broker re-registers on the same connection without re-announcing agents.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions