fix(routing): mark a matrix that shadows a same-named matrix in routing list/show - #293
Conversation
…ing list`/`show` `amplifier routing list` showed a user matrix and a bundle matrix of the same name as peers. Only one is ever loaded: hooks-routing's mount() searches `[*custom_routing_dirs, bundle routing/]` and takes the first hit, so a file in ~/.amplifier/routing/ silently makes the shipped bundle matrix dead -- and nothing in the CLI said so. That is why a matrix change shipped in the bundle can be completely inert on a host. The precedence rule is NOT re-derived here. It is consumed from hooks-routing's own `resolve_matrix_source()` (routing-matrix PR #52), loaded by file path out of the same cached bundle directory the CLI already globs -- app-cli does not depend on hooks-routing as a distribution, and `routing list` never mounts a bundle, so neither an import nor the session-time `model_role_resolver` capability is reachable from this process. When the cached bundle predates PR #52 (no `resolve_matrix_source`), the CLI draws no marker at all rather than guessing the search order: a wrong shadowing claim is worse than none. Unshadowed output is byte-identical to before, in both text and JSON. - amplifier_app_cli/lib/routing_provenance.py: locate + load the bundle's matrix_loader, classify custom vs bundle routing dirs, resolve one MatrixSource per matrix name. - commands/routing.py: row marker (`⚠ shadows bundle`), a footer naming the file in use and each file it suppresses, the same note on `routing show`, and MatrixSource.to_dict() in `--format json`. - tests/test_routing_shadowing.py: 15 tests -- shadowed marks the winner, unshadowed output byte-identical, no user routing dir, and the old-bundle degradation path.
7aa6bd9 to
450f309
Compare
Merge-queue verification — lane adqFresh scratch clone ( Conflict check: no textual conflict.
Known incompleteness (not a blocker, noted per instructions): the underlying All gates pass. Merging with |
…-write-wins (#294) * fix(routing): mark a matrix that shadows a same-named matrix in `routing list`/`show` `amplifier routing list` showed a user matrix and a bundle matrix of the same name as peers. Only one is ever loaded: hooks-routing's mount() searches `[*custom_routing_dirs, bundle routing/]` and takes the first hit, so a file in ~/.amplifier/routing/ silently makes the shipped bundle matrix dead -- and nothing in the CLI said so. That is why a matrix change shipped in the bundle can be completely inert on a host. The precedence rule is NOT re-derived here. It is consumed from hooks-routing's own `resolve_matrix_source()` (routing-matrix PR #52), loaded by file path out of the same cached bundle directory the CLI already globs -- app-cli does not depend on hooks-routing as a distribution, and `routing list` never mounts a bundle, so neither an import nor the session-time `model_role_resolver` capability is reachable from this process. When the cached bundle predates PR #52 (no `resolve_matrix_source`), the CLI draws no marker at all rather than guessing the search order: a wrong shadowing claim is worse than none. Unshadowed output is byte-identical to before, in both text and JSON. - amplifier_app_cli/lib/routing_provenance.py: locate + load the bundle's matrix_loader, classify custom vs bundle routing dirs, resolve one MatrixSource per matrix name. - commands/routing.py: row marker (`⚠ shadows bundle`), a footer naming the file in use and each file it suppresses, the same note on `routing show`, and MatrixSource.to_dict() in `--format json`. - tests/test_routing_shadowing.py: 15 tests -- shadowed marks the winner, unshadowed output byte-identical, no user routing dir, and the old-bundle degradation path. * docs(adq): lane DONE-NOTE — findings, seam rationale, honest test limitation * fix(routing): select the listed matrix by the loader's rule, not last-write-wins `amplifier routing list` built its rows with matrices[data["name"]] = (data, path) # over sorted(discovered files) which diverges from hooks-routing in two independent ways at once: it keyed on the `name:` field INSIDE each YAML (the loader resolves by file STEM), and it let the LAST file in sort order win (the loader takes the FIRST hit in `[*custom_routing_dirs, bundle routing/]`). The two rules agreed only by alphabetical accident -- `~/.amplifier/cache/...` sorts before `~/.amplifier/routing/...` because "c" < "r". When they disagree the command asserts something false: it names a file as in use that the loader would never read. A user file `my-fast.yaml` declaring `name: balanced` overwrote the row for the real `balanced` matrix outright. Rows are now keyed by file stem, and the file behind each row is `MatrixSource.path` -- the value hooks-routing's own `resolve_matrix_source()` assigns to `matrix_path` and loads. Those fields are reachable from the CLI through the seam PR #293 already built, so precedence is not re-derived a third time. Only the winning file is parsed, so a shadowed file can no longer supply a row's description, `updated:` date or compatibility count. Also: `routing use` now writes the filename the loader resolves (it could write an unloadable internal name before), a `name:`/stem disagreement is surfaced rather than silently keyed on the internal name, and every JSON row carries `matrix_file`. When the cached bundle predates routing-matrix PR #52 there is no `resolve_matrix_source` to ask. The shadowing MARKER is still withheld (#293's rule: a wrong marker is worse than none), but a row must point at some file, so selection falls back to the first candidate in `[*custom_dirs, *bundle_dirs]` -- the same list hooks-routing builds as `search_dirs`, in one labelled function. Tests: `tests/test_routing_winner_selection.py` constructs both disagreement classes explicitly, each with a non-vacuity gate that re-runs the old algorithm inline and asserts it picks the other file. Full suite green (1605 passed); existing routing list/show/use tests unmodified. --------- Co-authored-by: Amplifier Lane adq <lane-adq@localhost> Co-authored-by: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
The defect
amplifier routing list— the command a human runs to ask "what matrices doI have?" — shows a user matrix and a bundle matrix of the same name as
peers, with no indication that one is suppressing the other.
Only one of them is ever loaded. hooks-routing's
mount()searches[*custom_routing_dirs, bundle routing/]and takes the first hit, so asame-named file in
~/.amplifier/routing/silently makes the shipped bundlematrix dead. This is why a matrix change shipped in the bundle can be
completely inert on a host (proven on n=99 wire captures by
model_performance-8vq; it is what closed lanedf1's PR).model_performance-ell(routing-matrix #52, mergedd17d03c) made thisobservable at load time — an INFO/WARNING naming the winner and every
suppressed file, plus
matrix_path/matrix_source/shadowed_pathsonthe
model_role_resolvercapability. This PR is the missing half: making itobservable in the command a human actually runs.
What the CLI shows today (file:line)
listcommandamplifier_app_cli/commands/routing.py:297-354_discover_matrix_files()— same file,:98-141_load_all_matrices()— same file,:153-160routing listrenders one row per matrix — name, active arrow, description,covered/total rolescompatibility,updated— from aname -> datadictbuilt by
_load_all_matrices(). Nothing in that pipeline carries the file arow came from, so nothing can say that two rows collapsed into one, or which
file won.
Reachability: the fields ell publishes are not importable here
Stated precisely, because it drove the design:
resolve_matrix_sourcecannot be imported.hooks-routingis abundle module. It is not a distribution
amplifier-app-clidepends onand it is not on
sys.path.matrix_path/matrix_source/shadowed_pathsare published on themodel_role_resolvercapability at session start.amplifier routing listnever mounts a bundle, so no coordinator and no capability exist inthat process.
The smallest seam that still avoids re-implementing precedence: the
routing-matrix bundle is on disk, in the very cache directory
_discover_matrix_files()already globs. So the CLI loadsmodules/hooks-routing/amplifier_module_hooks_routing/matrix_loader.pybyfile path and calls the real
resolve_matrix_source. One rule, one home —this PR adds no precedence logic of its own.
Degradation is deliberate: a cached bundle older than #52 has no
resolve_matrix_source, and in that case no marker is drawn at all. Awrong shadowing claim is worse than none.
(A smaller long-term seam exists and is worth a maintainer's opinion:
hooks-routing shipping
resolve_matrix_sourcesomewhere app-cli can importoutright. That is a cross-repo packaging decision, not this lane's call.)
Before / after
Real run against a fixture home carrying the actual upstream
matrix_loader.pyatd17d03c(not a stub), with~/.amplifier/routing/openai.yamlshadowing the bundle's
openai.yaml.Before
Two
openai.yamlfiles exist. One is dead. Nothing says so.After
A marker on the row (visible in every view, including
--compact), then therelationship itself: the winning path, and each path it suppresses, one per
line. Two extra lines per collision — not a wall of text.
--format jsonadditionally carriesMatrixSource.to_dict()verbatim(
matrix_name/matrix_path/matrix_source/matrix_shadowed/shadowed_paths) on shadowed entries — the same field names ell publishes onthe capability, so a script reads one vocabulary either way.
Unshadowed output is unchanged
Not "looks the same" — asserted byte-identical. Two tests render the same
unshadowed tree twice, once through the new path and once with provenance
forced unavailable (the pre-change path), and compare the strings exactly.
Text and JSON both. Nothing at all is printed when nothing is shadowed.
Tests
tests/test_routing_shadowing.py, 15 tests:— including a non-vacuity assertion: the same tree through the
pre-change path shows no marker;
routing listworks with no user routing dir at all;matrix_loader.py, or present without the function);(
fn.__code__.co_filename), not from this repo;MatrixSourceresolution.Full suite: 1588 passed, 1 skipped, 1 xfailed.
ruff checkclean on alltouched files (8 pre-existing
session.pyfindings unchanged).Known follow-up, not fixed here
_load_all_matrices()(routing.py:153-160) keys rows by thename:fieldinside the YAML with last-write-wins over
sorted(files). That is notthe loader's rule — it agrees with it today only by alphabetical accident
(
cache<routing) and can diverge. This PR reads provenance by the winningrow's file stem, which is the loader's actual key, so the marker is
correct either way; but the row-selection defect itself is untouched and
deserves its own item.