Skip to content

feat(tool-delegate): record routing-matrix provenance on spawn telemetry - #349

Merged
Brian Krabach (bkrabach) merged 1 commit into
mainfrom
lane/wok-spawn-telemetry-matrix
Sep 3, 2026
Merged

feat(tool-delegate): record routing-matrix provenance on spawn telemetry#349
Brian Krabach (bkrabach) merged 1 commit into
mainfrom
lane/wok-spawn-telemetry-matrix

Conversation

@bkrabach

Copy link
Copy Markdown
Collaborator

What

delegate:agent_spawned records which routing matrix file produced a
delegation's provider_preferences, not just the preferences themselves.

"routing_matrix": {
  "matrix_name": "anthropic",
  "matrix_path": "/home/u/.amplifier/routing/anthropic.yaml",
  "matrix_source": "user",
  "shadowed_paths": ["/opt/bundles/routing-matrix/routing/anthropic.yaml"]
}

Why

A user file in ~/.amplifier/routing/ silently outranks the bundle's own
same-named matrix. Until now a surprising resolution in the event stream —
e.g. a fast-role delegate served on gpt-5.6-sol@xhigh — was
indistinguishable from a shadowed matrix, a shipped-matrix change, or no
routing at all. Two prior investigations read the shipped file, reasoned about
a matrix that was not in effect, and reached confidently wrong mechanisms.

Consumed, not re-derived

matrix_path / matrix_source / shadowed_paths are published by
hooks-routing
on the model_role_resolver capability object this tool
already holds. That module's own docstring names "a spawn-time telemetry
payload"
as the intended consumer. Nothing here re-implements matrix
precedence — a second implementation of that precedence is exactly the drift
this reads published state to avoid.

Verified against the real MatrixModelRoleResolver object (not a mock) in
shadowed / clean / no-origin configurations. Foundation does not and must not
depend on a routing bundle — the capability is duck-typed by design — so the
committed tests use a stand-in mirroring the published attribute contract
exactly.

Where

Provenance read _matrix_provenance(), modules/tool-delegate/.../__init__.py
Captured execute(), at the one site that consults the resolver
Recorded _spawn_new_session()delegate:agent_spawned
Also recorded delegate:model_role_unresolved

Captured at the resolver-lookup site rather than re-fetched at the emit site,
so the recorded identity is the strategy that produced these preferences and
cannot drift from it if the capability is swapped mid-session.

Note: the originating work item asserted that delegate:agent_spawned "DOES
NOT EXIST as a real emit". That is falsified — it is emitted at
__init__.py:1648, registered in observability.events at __init__.py:297,
present identically in the installed cached bundle, and dating to b73813b
(#44). No new event name was needed.

Backward compatibility

Additive, and omitted when unknown.

  • Consumers that ignore the field are unaffected. No existing key's name,
    type, or value changes.
  • Analyzers reading OLD captures still work. Read with
    payload.get("routing_matrix"). Absent means UNKNOWN, never "no
    shadowing."
    Every capture on disk today lacks the key; an analyzer that
    treats absence as a negative assertion would silently clear exactly the
    shadowed sessions this field exists to catch. Pinned by a test that reads an
    old-shape and a new-shape payload side by side.
  • Default behaviour byte-identical. The key is absent for: no model_role;
    an explicit provider_preferences pin; an agent-level default; no routing
    bundle installed; a resolver that reports no source (alternate strategy, or
    an older routing bundle). Pinned against a literal set of the pre-existing
    keys, so a rename or drop fails loudly.
  • Never claims a matrix it didn't consult. An explicit pin records no
    matrix — the resolver was never asked.
  • Not added to the tool result. The provider_routing dict returned to the
    model is deliberately unchanged: filesystem paths in model-facing context are
    token cost and leakage, and the event stream is the forensic record.

Tests

17 new (test_delegate_spawn_matrix_provenance.py). Verified non-vacuous:
removing the payload injection fails 4 of them, including the headline
shadowing case.

  • a spawn records the matrix path that actually produced its preferences
  • a shadowed load records the SHADOWING file, not the shipped one
  • unresolved role records which matrix failed
  • four byte-identical-default cases + explicit-pin + old-capture-reader
  • reader hardening: a bare str in shadowed_paths is not iterated
    character-wise; non-str entries dropped, not coerced

Full suite: 1857 passed, 1 skipped. One pre-existing unrelated failure
(test_grpc_adapter_main.py::test_non_isinstance_object_with_mount_passes,
Python 3.12 @runtime_checkable) — confirmed failing on a clean tree.

`delegate:agent_spawned` recorded WHICH provider_preferences a delegation
resolved to, but not WHICH matrix file produced them. A user file in
~/.amplifier/routing/ silently outranks the bundle's own same-named matrix,
so a surprising resolution in the event stream was indistinguishable from a
shadowed matrix, a shipped-matrix change, or no routing at all.

Adds an optional `routing_matrix` key (matrix_name / matrix_path /
matrix_source / shadowed_paths) READ FROM the model_role_resolver
capability's published attributes -- hooks-routing publishes them, and its
own docstring names "a spawn-time telemetry payload" as the intended
consumer. Nothing here re-derives matrix precedence; a second implementation
of that precedence is the drift this avoids.

Captured at the one site that actually consults the resolver (execute()),
threaded to the emit site as a keyword-only arg, so the recorded identity is
the strategy that produced THESE preferences and cannot drift from it.

Additive and omitted when unknown:
- Consumers ignoring the field are unaffected: no existing key's name, type
  or value changes.
- Absent means UNKNOWN, never "no shadowing". Every capture on disk today
  lacks the key; an analyzer treating absence as a negative assertion would
  silently clear exactly the shadowed sessions this exists to catch.
- No model_role, an explicit provider_preferences pin, an agent-level
  default, no routing bundle, or a resolver that reports no source all leave
  the payload byte-identical to before.

Same key added to `delegate:model_role_unresolved`, where "which matrix
failed to serve this role" is the first question asked.

17 new tests; verified non-vacuous (4 fail with the injection removed).
@bkrabach
Brian Krabach (bkrabach) force-pushed the lane/wok-spawn-telemetry-matrix branch from bfb6a65 to 41fbef8 Compare September 3, 2026 00:46
@bkrabach
Brian Krabach (bkrabach) marked this pull request as ready for review September 3, 2026 00:46
@bkrabach

Copy link
Copy Markdown
Collaborator Author

Merge-queue verification — lane wok

Fresh scratch clone (scratch/merge9/amplifier-foundation). PR branch was opened before #348 (016dadb, the session-naming leak fix) merged, so it was one commit behind main.

Conflict check: no textual conflict. git diff origin/main pr-head --stat (pre-update) already showed the full 7-file diff was #348's own session-naming changes appearing only because the branch predated them — not an overlap. Confirmed independence by inspecting both diffs: #348 touches only modules/hooks-session-naming/*; #349 touches only modules/tool-delegate/*. git rebase origin/main on the lane branch applied cleanly with zero conflicts, proving the union — pushed that rebase to lane/wok-spawn-telemetry-matrix (force-with-lease) so the PR now diffs cleanly against current main (confirmed: gh pr diff --name-only now shows exactly 3 files, all under modules/tool-delegate/). Re-ran the full suite on the updated branch (see below) rather than trusting the git merge alone.

Gate Method Result
(a) spawn records the matrix path that produced its preferences test_spawn_records_the_matrix_that_produced_its_preferences PASS
(b) a SHADOWED load records the shadowing file, not the shipped one test_shadowed_load_records_the_shadowing_file_not_the_shipped_one — asserts matrix_path == USER_MATRIX, matrix_source == "user", shadowed_paths == [BUNDLE_MATRIX], and explicitly matrix_path != BUNDLE_MATRIX. Real, non-vacuous test (not a mock rubber-stamp). PASS
(c) consumes ell's published fields rather than re-deriving precedence Read _matrix_provenance(): reads resolver.name / .matrix_path / .matrix_source / .shadowed_paths off the duck-typed model_role_resolver capability — no precedence logic reimplemented anywhere in the diff. PASS
(d) backward compat: old captures parse, consumers ignoring field unaffected test_old_capture_reader_pattern_still_works, test_pre_existing_keys_are_unchanged_when_matrix_is_recorded PASS
(e) default otherwise byte-identical test_no_model_role_payload_is_byte_identical, test_no_routing_bundle_installed_payload_is_byte_identical, test_resolver_without_provenance_payload_is_byte_identical, test_explicit_pin_records_no_matrix — all assert set(payload) == _PRE_EXISTING_SPAWN_KEYS (a literal pinned key set) PASS
(f) full suite green uv sync --extra grpc-adapter then: CI-scoped uv run pytest tests/ -q1691 passed, 1 skipped; full testpaths run (tests/ + modules/tool-delegate/tests/, which is where the new 17 tests live) → 1858 passed, 1 skipped, 0 failures. (PR body claimed one pre-existing unrelated failure on Python 3.12; on this host's Python 3.13.11 it passes — re-ran it directly, confirmed green, not a regression either way.) ruff check modules/tool-delegate/ clean. PASS
Diff scope matches title 3 files, all under modules/tool-delegate/: README.md, __init__.py, new test file. Nothing in hooks-session-naming or elsewhere touched. PASS
Performance/measurement claims None made by this PR. N/A

All gates pass, including the required non-vacuity check on gate (b) (the headline shadowing test).

Merging with --admin (required-review ruleset on this repo; I am the PR author).

@bkrabach
Brian Krabach (bkrabach) merged commit 2a3cb19 into main Sep 3, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant