[Epic] Idempotent receiver: move the unit of work, inbox and outbox processing to the delivery boundary

[brenpike]

Problem

UnitOfWorkBehavior<TMessage>, InboxBehavior<TMessage> and OutboxProcessingBehavior<TMessage> are all ICommandBehavior<TMessage>. The transaction and the inbox's deduplication therefore live inside each Command's pipeline, not around the delivery that the broker handed us. That placement is the root cause of a family of defects:

• In-memory Inbox silently skips a Command dispatched with context.InMemory() from inside a received handler #534: a Command a received handler dispatches in-process was treated as the delivery itself and silently skipped (in-memory Inbox), re-claimed (relational Inbox), or allowed to complete the delivery's marker early (standalone Cosmos Inbox).
• The In-memory Inbox silently skips a Command dispatched with context.InMemory() from inside a received handler #534 fix (ADR-0041) can only approximate the delivery: it gates the first Command that reaches the Inbox in each receive attempt. As a result:
  • when a broker-delivered Event's handler dispatches several Commands, only the first is deduplicated; later ones are at-least-once on retry or redelivery (ADR-0041 R1);
  • a direct call to the public DispatchReceivedMessageAsync bypasses the per-attempt reset (ADR-0041 R2);
  • broker-delivered Events are not deduplicated at all.
• Cosmos document tier silently discards a nested participant command's writes #538: the Cosmos document tier's batch gate has the same root class and silently discards a nested participant Command's writes.

Target design

The standard idempotent receiver / transactional inbox pattern (as NServiceBus's outbox and MassTransit's transactional inbox apply it): the reliability envelope wraps the handling of one delivery.

For each receive attempt of a delivery:

1. Open the unit of work for the delivery.
2. Check and claim the delivery's message id in the Inbox; if it is already handled, stop.
3. Run every handler: the delivered Command or Event and every Command dispatched in-process during it.
4. Record the message id as handled and commit it atomically with all the handler work and any outbox rows.

A failed attempt rolls back its claim with its work, so a retry starts clean. Nested and fan-out dispatch need no special rule, and Events are deduplicated too.

BrokeredMessageReceiver already creates one TransactionContext per delivery, which is the natural seam.

Scope and open decisions

• Packages: Chatter.MessageBrokers, Chatter.MessageBrokers.Reliability.EntityFramework, Chatter.MessageBrokers.Reliability.Cosmos (including Cosmos document tier silently discards a nested participant command's writes #538's document-tier design).
• ADRs to amend or supersede: ADR-0006 (behavior order / two-tier reliability), ADR-0025 (unit of work vs retrying execution strategy), ADR-0041 (interim Delivery Entry).
• Registration surface: WithBehavior(typeof(InboxBehavior<>)), EF WithInboxBehavior<TContext>(), the Cosmos inbox registration — how they map to a delivery-level envelope, and what that means for versioning and consumer code.
• Commands dispatched outside any delivery (no broker context): what, if anything, provides a unit of work for them.
• Non-transactional resources inside a handler remain at-least-once; state that explicitly.

This needs a planning session before implementation.

Related

#534, #538, ADR-0006, ADR-0025, ADR-0041.

[brenpike]

Design input from the #534 review: every inbox tier keys on the inbound MessageId, which the producer controls (reused, colliding or absent ids). The current posture is recorded in ADR-0009 (D2) and ADR-0015. When this epic redesigns the inbox at the delivery boundary, decide explicitly which identity it deduplicates on.