feat(hooks): flag-gated same-batch read coalescer

[Brian Krabach (bkrabach)]

Summary

Adds hooks-tool-dedupe, a new additive, default-OFF hook module that
coalesces byte-identical repeated read_file / load_skill results within a
single parallel tool batch of a single session.

Measured basis (5 gpt-5.6 runs; full spec: yoc-dedup-spec.md, Section 2):

Class	Reads	Share	Share of dup tokens
Same LLM turn, same parallel batch	1 196	37%	87%
Cross-session (rejected as out of scope)	129	4%	12%

• Content byte-identical in 100% of the 1,196 measured same-batch repeats
  (0 divergences) -- this is why the mechanism can be lossless: it coalesces
  an exact-content match, never guesses from a staleness heuristic.
• Deduplicating just this class removes 16.4% of all input tokens across
  the measured runs, with zero information loss, because the first
  occurrence always carries the content and the marker for every duplicate
  is delivered in the same tool-result turn, adjacent to it.

Mechanism

A tool:post hook can already replace the result content the LLM sees, via
HookResult(action="modify", data={..., "result": <new str>}). Verified
against loop-streaming's parallel (_execute_tool_only) and sequential
tool-execution paths, both of which detect a hook-modified result by object
identity (returned_result is not result_data). hooks-tool-truncation
(amplifier-bundle-attractor) is the existing, exercised precedent for this
exact shape -- this hook is a sibling of it, registered at a lower priority
(5 vs 10) so truncation sees an already-short marker and no-ops.

Hard constraints honored

• Object identity, never in-place mutation. The hook always returns a
  new dict with a new str under "result".
• Sequential calls are never deduped. The sequential tool-call path never
  carries a parallel_group_id (None there), so "no batch id -> no
  dedupe" falls out of the payload shape, not a separate check.
• Per-session, in-memory state only. One dict on the hook instance
  (parallel_group_id -> key -> holder entry), bounded by an
  insertion-ordered LRU (max_batches, default 8). No tree id, no
  cross-session storage, no cleanup path needed -- it dies with the session.
• Exact-content invalidation, not a heuristic. A same-key sha256
  mismatch within a batch is delivered untouched and counted as
  dedupe:divergent (the canary for mid-batch file mutation; should be ~0).
• Skill-load dedupe is zero incremental code. load_skill is in the
  default tools allowlist, so it dedupes via the exact same code path --
  not a separate mechanism, as the spec's Section 0 (row 4) argues.
• Default-OFF. enabled: false in behaviors/agents.yaml; mounting this
  hook has zero runtime effect until a caller opts in. Not added to
  tasks.yaml (same ships-dark pattern as hooks-delegation-batching).

Telemetry

dedupe:coalesced (one per marker, carrying tool_name,
parallel_group_id, holder/duplicate call ids, target, bytes_saved,
sha256) and dedupe:divergent, both declared via
register_contributor("observability.events", ...) so the existing session
logging hooks auto-capture them -- no new instrumentation needed.

Note on the falsification metric: the spec's "re-read after marker"
guardrail (Section 9) -- the single most informative signal in its DTU
measurement plan -- is computed externally, post-hoc, by joining
dedupe:coalesced events against ordinary read_file tool:post events in
the following turn. It is intentionally not tracked as live in-hook
state: doing so would require remembering marked keys across a turn
boundary, which is exactly the cross-turn state this module's scope excludes
(spec Section 6.2, rejected outright). No new instrumentation is needed for
this measurement either -- dedupe:coalesced's existing target field is
sufficient for the external analysis to join against.

Testing

• 25 new pure-function/hook-level tests in
  modules/hooks-tool-dedupe/tests/test_batch_dedupe.py, covering the
  spec's full test plan (Section 4.8): sequential-never-deduped, first-read
  passthrough, second-read marked, new-object/str-result identity checks,
  divergent-content passthrough, offset/limit key normalisation and
  isolation, min_bytes/failed-result/unknown-tool/malformed-input/
  missing-content passthroughs, load_skill dedupe, concurrent-batch
  single-content-holder (pins the no-await-between-lookup-and-store
  invariant), max_batches LRU eviction, disabled-is-total-noop, telemetry
  payload shape, and mount() wiring (priority=5, observability.events
  contributor, default-disabled).
• Module suite: 25 passed.
• Repo suite: 1634 passed, 1 skipped.
• python_check (ruff-lint, ruff-format, pyright, stub-check): clean.

Scope

Purely additive: one new module directory
(modules/hooks-tool-dedupe/) plus one behavior-YAML entry
(behaviors/agents.yaml). No edits to read.py, tool-skills,
tool-delegate, loop-streaming, amplifier-core, or any provider module.
bundle.md needs no change -- the hook arrives via the agents behavior it
already includes, matching how tool-delegate is wired.

Rollback is either flipping enabled: false (already the default) or
deleting the behaviors/agents.yaml entry.

This PR is Stage 1 only (per the spec's staged rollout, Section 10):
merge with enabled: false, zero runtime effect. Flipping the default and
running the DTU A/B (Section 9) is a follow-up, not part of this PR. Stage 2
(ACI windowed reads) is a separate repo/PR entirely.

Not merging this PR myself -- flagging for review.

🤖 Generated with Amplifier

[Brian Krabach (bkrabach)]

Admin-merge disclosure — self-authored PR, explicit user/operator direction

This PR is self-authored by the maintainer. Per the documented admin-merge pattern for this situation (self-approval is not possible; merge proceeds only under explicit operator direction — consistent with ~14 PRs merged this way this week), this will be merged via gh pr merge --admin once CI is green.

Basis for merge:

• New module hooks-tool-dedupe: a same-batch read coalescer for duplicate tool reads within a single batch.
• Measured basis: 1,196 measured same-batch duplicate reads across 5 eval runs; those duplicates account for 87% of duplicate-read tokens; content was byte-identical in 100% of measured cases; duplicate reads amplified input tokens by ~16.4%.
• Ships dark: registered with enabled: false in the behaviors entry — zero behavior change until an operator opts in.
• Test evidence: 25 module tests + 1634 repo tests green.
• CI status: all required checks green at time of merge.

Merging squash, branch retained (--delete-branch=false).

[Brian Krabach (bkrabach)]

CORRECTION: this module's premise has been refuted by payload-level validation. The "1,196 same-batch duplicate reads / 16.4% amplified tokens" figures were a measurement artifact — duplicate snapshot copies of session files counted the same events 2-4x (identical tool_call_id + nanosecond timestamps prove it); deduped, same-batch duplicate excess is 0 across 314 sessions / 10,320 reads. Additionally the openai provider chains server-side (deltas only on the wire), so the assumed re-send amplification does not exist; measured real value ≈ 19k tokens (~$0.003) across 5 runs. Module removed in #328; the loader-classification regression test from #326 is retained (it guards a real core loader fragility). The verified successor lever is windowed reads targeting first-read volume (39.1% of billed input) — tracked separately.