Skip to content

docs: add the rationale doc for AG2-015 - #70

Open
bradAGI wants to merge 1 commit into
trustabl:mainfrom
bradAGI:docs/ag2-015-idempotency-rationale
Open

docs: add the rationale doc for AG2-015#70
bradAGI wants to merge 1 commit into
trustabl:mainfrom
bradAGI:docs/ag2-015-idempotency-rationale

Conversation

@bradAGI

@bradAGI bradAGI commented Aug 24, 2026

Copy link
Copy Markdown

Pairs with trustabl/trustabl-rules#101 (AG2-015) and trustabl/trustabl#138 (fixture + tests).

Why AG2-015 exists

Six packs ship an idempotency rule — CSDK-006/016, OAI-009/019, ADK-006, MCP-007, CREW-006, PYD-007. AutoGen ships none.

The argument the doc makes

A retry is not an unusual event in an agent — it is the normal response to an ambiguous result. That is at-least-once delivery with a language model as the retry policy, and the retry policy cannot inspect the remote's state.

AutoGen adds two retry paths a single-agent loop does not have, and both come from its conversation model rather than the tool:

  1. Persistence. A tool response is a message re-sent in full on every later turn. An inconclusive call does not scroll out of reach — it stays in front of the model, with its arguments, for the rest of the run. The window for a duplicate is not the next turn; it is every turn.
  2. A retry issued by a different agent. In a GroupChat the manager selects who speaks next, and any selected agent can re-issue the tool. It reads the same transcript but holds no private memory of having committed the side effect, so the duplicate is issued by a participant structurally unable to know it is a duplicate.

Confidence 0.55 is defended, not glossed

It is the lowest in this pack, and the doc says why in both directions: the rule reasons from a name and a parameter list. update_cache prefix-matches and is harmless; book_flight and transfer_funds carry no listed prefix and are never examined at all. The prefix list is a heuristic, not a taxonomy of side effects.

The exclusions lead with the non-fix

The most important limitation is the one that looks like a fix:

Whether the key is stable. A key derived from a fresh uuid4() at call time is regenerated on the retry, so it deduplicates nothing while looking exactly like a fix.

Also named: a declared key the body never forwards, a downstream service that ignores keys entirely, and deduplication done in a wrapper the signature check cannot see.

One recommendation specific to this framework

In a GroupChat, assume the retry may come from a different agent than the original caller. The key has to travel in the transcript, not in one agent's private state.

And: put "reuse the same idempotency_key when retrying" in the tool description, because the model chooses the argument — if it is not told, it invents a new one on the retry.

Gate

Two pre-existing errors remain on main (OAI-112, PYD-106), fixed by #53. AG2-015 is covered and consistent.

Pairs with AG2-015 in trustabl-rules. Follows the template guide and carries
the machine-checked front-matter, so check_rulebook.py covers the rule.

The threat model turns on AutoGen's conversation model supplying two retry
paths a single-agent loop lacks. A tool response is a message re-sent in full
on every later turn, so an inconclusive call stays re-invitable for the whole
run rather than the next turn. And in a group chat the manager may select a
different speaker to re-issue it — an agent that reads the same transcript but
holds no private memory of having committed the side effect, and so is
structurally unable to know it is issuing a duplicate.

Confidence is 0.55, the lowest in the pack, and the doc defends that rather
than glossing it: the rule reasons from a name and a parameter list, so
update_cache is a false positive and book_flight is never examined at all.

The exclusions lead with the failure that matters most in practice — a key
derived from a fresh uuid4() per call is regenerated on the retry, so it
deduplicates nothing while looking exactly like a fix.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant