Skip to content

map: one versioned capability contract instead of five copies - #137

Merged
patchstackdave merged 3 commits into
mainfrom
davejong/map-capability-contract
Aug 17, 2026
Merged

map: one versioned capability contract instead of five copies#137
patchstackdave merged 3 commits into
mainfrom
davejong/map-capability-contract

Conversation

@patchstackdave

@patchstackdave patchstackdave commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Stacked on #136 — base is davejong/map-import-inventory, so this diff shows only its own change. Rebase after #136 squash-merges.

Why

The closed vocabularies that describe what the map can see — sink kinds, argument roles, candidate families, confidence tiers — were declared independently in five places: the TypeScript unions here, plus separate copies in the layers that consume the map (rule authoring, and the platform that binds a coordinate into a rule).

Adding a capability meant editing all of them, and missing one fails silently: a sink kind this side emits but an authoring layer rejects makes the capability unauthorable; one the binding layer doesn't know makes every flow of that kind unusable. Nothing throws — the capability just never matches, which reads exactly like "not reachable".

That is the same shape as the wrong-pin bugs in the map hardening track: three instances, one lossy representation. The fix is the representation, not a third patch.

What

  • src/map/capabilities.ts is the single definition, versioned, with the reasoning that a new member is a contract change rather than a list edit — and that a new sink family owes adversarial corpus cases before it may generate rules, since one capability admits a whole package family at once.
  • The TypeScript unions are derived from it ((typeof SINK_KINDS)[number]) rather than declared beside it, which would have been a second copy with the same drift risk.
  • capabilities.json is generated and committed, so a vocabulary change shows up as a reviewable diff in the contract instead of appearing silently in dist. npm run capabilities regenerates; --check verifies.
  • Deliberately not added to package.json files. The consumers vendor it from source, and the published surface is separately reviewed — the existing pack-safety test caught my first attempt at widening it, which was the correct outcome.

Verification

977 tests, typecheck clean. The drift guard was verified by actually causing drift: adding a member to SINK_KINDS without regenerating fails with "capabilities.json is stale — run npm run capabilities".

The tests defend the contract rather than the values: the manifest must be byte-identical to the emitter's output, must agree with the TS member for member, the auto-promotable tier must remain the single strongest proven tier, and everything the extractor actually emits must fall inside the vocabulary — with a non-vacuity check first, since a fixture producing no sinks would satisfy that trivially.

Consuming layers

The vocabulary is consumed outside this repo. Those layers now vendor capabilities.json and assert conformance against it, so a stale copy is a test failure rather than a silent mismatch. One consumer is still a hand-maintained copy and is tracked separately — until it reads the manifest, it remains the one place drift can occur.

@coderbuds

coderbuds Bot commented Aug 17, 2026

Copy link
Copy Markdown

Cohesively centralizes and version-controls the capability contract with robust codegen and tests.

🎯 Quality: 94% Elite · 📦 Size: Large — consider splitting if possible

📈 This month: Your 57th PR — above team average · Averaging Excellent

See how your team is trending →

@patchstackdave
patchstackdave force-pushed the davejong/map-capability-contract branch from cc7f8cc to f755047 Compare August 17, 2026 09:11
@patchstackdave

Copy link
Copy Markdown
Contributor Author

/review

Base automatically changed from davejong/map-import-inventory to main August 17, 2026 10:25
The closed vocabularies that describe what the map can see — sink kinds, argument roles, candidate
families, confidence tiers — were declared independently in five places: the TypeScript unions here,
and separate copies in the rule-authoring and rule-binding layers that consume the map. Adding a
capability meant editing all of them, and the failure mode of missing one is silent: a sink kind
this side emits and an authoring layer rejects makes the capability unauthorable, one the binding
layer does not know makes every flow of that kind unusable. Nothing errors. The capability simply
never matches, which reads exactly like "not reachable".

That is the same shape as the wrong-pin bugs in the map hardening track: three instances, one
lossy representation. The fix is the representation.

- `src/map/capabilities.ts` is now the single definition, versioned, with the reasoning for why a
  new member is a contract change and not a list edit.
- The TypeScript unions are DERIVED from it (`(typeof SINK_KINDS)[number]`) rather than declared
  beside it, which would have been a second copy with the same drift risk.
- `capabilities.json` is generated and committed, so a vocabulary change is a reviewable diff in
  the contract. It is not added to package.json `files`: the consumers vendor from source, and the
  published surface is separately reviewed.
- Tests defend the contract, not just the values: the committed manifest must be byte-identical to
  what the emitter produces (verified by adding a sink kind and watching it fail), the manifest must
  agree with the TS member for member, the auto-promotable tier must remain the single strongest
  proven tier, and everything the extractor actually emits must fall inside the vocabulary — with a
  non-vacuity check first, since a fixture producing no sinks would satisfy that trivially.
…e must bump the version

Two ways the contract could still drift silently.

A capability could be declared with nothing behind it. Adding a sink kind, regenerating the manifest
and teaching every consumer to accept it succeeded even if no recognizer ever emitted that kind — a
vocabulary entry with no behaviour, unlocking a rule family that can never fire. The existing
"everything emitted is inside the vocabulary" test cannot see it: that only checks the kinds that DO
emit are legal, and a fixture exercising the other four still passes a "we emit some sinks"
assertion.

Each kind now names its own control — the minimal shape that must produce a sink of exactly that
kind — and the tests run every control, assert the kind appears, and for a rule-generatable kind
assert a candidate with a real family compiles. Because the matrix is typed Record<SinkKind, …>,
adding a kind without a control is a TYPE error: the compiler asks before CI does. A kind that can
compile a rule can also compile a WRONG one, so those additionally require an adversarial lookalike
in the corpus, asserted rather than assumed.

And the version could stay put. A member added or removed without moving CAPABILITY_VERSION left
every vendoring consumer unable to tell it was behind — the same silent drift the manifest exists to
remove, one level up. check-capability-version.mjs compares the manifest against the base branch and
classifies: a member or field added needs a minor bump, a member removed or a scalar redefined is
BREAKING and needs a major one, because a consumer pinned to the old list keeps emitting a value
that can no longer match. Wired as its own CI job with full history.

Both verified by simulation: declaring an unrecognized capability fails three tests (no sink, no
candidate, no adversarial coverage), and the four version paths — additive without a bump, additive
with one, a removal under a minor bump, and a redefined scalar — each behave as specified.
…s is exact

Two ways the coverage checks could pass without proving anything.

Adversarial coverage was a text search over the corpus file: any occurrence of an API name counted,
so a comment, a positive fixture, or an unrelated case satisfied it. That is coverage that can be
true by coincidence — the same substring reasoning this suite exists to reject.

The cases now live in tests/map/corpus-cases.ts with stable ids, and each rule-generatable
capability names the adversarial case that covers it. The test resolves that id to a real case,
asserts it is in the adversarial category, and asserts it declares no expected candidates — then
BUILDS it and checks the extractor agrees, including the corpus non-vacuity rule, since a case that
detects nothing at all would satisfy a zero-candidate assertion for the wrong reason.

That exposed a genuine gap while wiring it up: eval had no adversarial case. It is recognized only
as a bare global call or `new Function`, gated on the name not being locally bound, and nothing
asserted the gate. adv/local-eval-lookalikes covers it — a locally-declared `Function` and a member
`.eval()` on an untraceable receiver, neither of which may compile code injection.

And the controls only required "some rule-generatable flow with some family". A recognizer that
classified a flow as the wrong mitigation class would have passed, which is a rule that inspects the
wrong thing and blocks the wrong traffic. Each rule-generatable control now declares its exact
argumentRole and candidateFamily and the test asserts equality. The type is a discriminated union,
so a capability declared rule-generatable without a role, a family and a case id does not compile.

Verified by simulation: a wrong family fails, an id that resolves to nothing fails, and an id
pointing at a case that legitimately expects candidates fails.
@patchstackdave
patchstackdave force-pushed the davejong/map-capability-contract branch from 016cac7 to 0727b66 Compare August 17, 2026 10:25
@patchstackdave
patchstackdave merged commit a6b12f0 into main Aug 17, 2026
6 checks passed
@patchstackdave
patchstackdave deleted the davejong/map-capability-contract branch August 17, 2026 10:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants