Skip to content

A dormant deterrent must not look dead, and the ceiling must not depend on a silence - #89

Merged
OsherElhadad merged 2 commits into
eranra:mainfrom
OsherElhadad:pr/evidence-integrity
Sep 4, 2026
Merged

A dormant deterrent must not look dead, and the ceiling must not depend on a silence#89
OsherElhadad merged 2 commits into
eranra:mainfrom
OsherElhadad:pr/evidence-integrity

Conversation

@OsherElhadad

Copy link
Copy Markdown
Collaborator

Clause evidence integrity — closes #81 and #85

Two issues, one theme: clause evidence that is wrong in a way nothing complains about. Neither is
a feature. Base origin/main (fd87f2a), branch pr/evidence-integrity, two commits — one per
issue, in one branch because #85 changes ablate.ts's options and #81 asserts ablate.ts's ceiling
against select.ts, so a reviewer reading either wants the other in front of them; and because one
branch means one plugin/lib regeneration instead of two that conflict.

make check: 106 files / 2,722 tests → 107 files / 2,749 tests, green. make plugin run last,
plugin/lib committed, ci/check-plugin-lib.sh reports "plugin/lib is the current build of src/".
No new dependency, no .py, no new code path that can produce an approval.


#81 — the rendered-ceiling exemption depends on another module and nothing asserted it

renderedCount counts only patterns.length === 0, so a matchable clause is exempt from the per-tier
prompt ceiling. That exemption is what makes pipeline output structurally unable to consume the prompt
budget — every mined candidate carries an anchored matcher by construction. It is not a property of
the schema, and ablate.ts says so itself two lines above the function.

Test only, no runtime check, and the justification is that no runtime input can reach the bad state.
The filters are code. There is no SelectOptions field, no PluginSettings key, and no settings.json
knob that reintroduces an unfiltered bundle; a hand-edited compiled artifact cannot do it either,
because both render paths filter on patterns at render time regardless of what the artifact holds.
Reaching this state takes a source edit, which is exactly what CI sees. A runtime observation would be
a per-decision branch guarding against a code change, producing a counter nobody reads — a silence
that reads as success, which is the bug shape this wave is hunting rather than a defence against it.

What the test asserts (src/test/policy/ablate.test.ts, 4 tests). Over a corpus of 35 matchable
mined clauses — deliberately past CEILING_PER_TIER:

  1. renderedCount is 0 for both learned tiers and selectClauses reports all 35 as
    evaluated-missed, with nothing selected and zero budget spent.
  2. A clause that does match is rendered for that call and the count still does not move — the
    exemption is per call, not per corpus.
  3. The same clauses as prose consume the whole ceiling, so the zero is the pattern test and not the
    fixture failing to be a learned clause at all.
  4. coreClauses / renderCore exempt them too.

Mutation evidence

Mutation Result
Remove selectClauses's evaluated-missed branch (patterned clauses fall through to fill) tests 1 and 2 fail on the selector assertions. renderedCount kept returning 0 throughout — which is the wrong-population failure #81 describes, demonstrated rather than argued
Relax coreClauses to drop its patterns.length === 0 filter test 4 fails
Full suite under the selector mutation 4 failures: 2 mine, 2 in select.test.ts

Honest scoping of what was and was not already covered. The selector's drop behaviour was
already tested in select.test.ts, and coreClauses's filter in compile.test.ts. What was missing
is the link: a reviewer who removes the filter sees select.test.ts fail, reasonably concludes the
test encodes the old behaviour, updates it, and never learns the prompt ceiling depended on it. These
tests fail with the ceiling named in the failure. That is the whole contribution and it is not more
than that.

Found, not fixed

pipeline.ts reports the ceiling outcome as a hardcoded rendered: 0 for both learned tiers
(with a comment saying why). It is honest given the code, but it is a literal, so it cannot move the
day the exemption stops holding — which is precisely the day the run line ought to move. Making it
real needs the compiled artifact inside propose(), which only has Clause[]. Left alone
deliberately: what the literal claims is now asserted by test 1, so the claim is checked even though
the number is not computed.


#85 — a dormant deterrent looked dead after the trail rotates

lifetimeFires came from the trail; the trail rotates at 4 MiB keeping one generation. A red that
fired steadily for months and is quiet this week was indistinguishable from one that never fired, so
classify() returned dead-weight? or insufficient-exposure where the right answer is deterrent
— and a retirement got proposed for a clause whose whole value is that it stopped something being
tried.

pipeline/citations.json (src/policy/citations.ts, new): one number per clause, folded by the
same offset-driven Stage A pass, read by ablate as lifetimeFires. It reuses mine.ts's
readNewBytes and tailShaAt rather than writing a second offset machine (both are now exported;
SourceState too).

Monotonic in effect — structurally, not by discipline

raise is the only path that writes a count and it assigns max(prior, candidate). No
subtraction, no reset, no other assignment in the module. "The count went down" is therefore not a bug
that can be introduced by getting a branch wrong; it is a value the writer cannot express.

Two consequences that are asserted, not asserted-about: a rotation that shrinks the trail cannot
shrink the count, and new fires after that rotation still add on top of the held number.

Idempotent

The fold commits an offset plus a tail hash per generation, so a re-run reads no lines and changes
nothing. When an offset fails its tail-hash check — rotation, truncation — both generations are re-read
whole and the recount is merged with max instead of added, so re-reading already-counted bytes cannot
double them either.

The ceiling, documented in place: fires living only in bytes rotated away before any fold saw
them
are lost, so a post-rotation count can lag the truth. It can never exceed it and never fall,
which is the direction that matters — classify() asks lifetimeFires >= 1, so an undercount can
only make a deterrent look under-exposed, never make a live clause look dead. Upgrade path noted: fold
on rotation rather than at session end.

Why its own file, and why not a record field

Not a DecisionRecord field: the trail is append-only, so a stored derived value goes stale with no
way to correct it and no way to detect that it has.

Not shapes.json, even though that file already carries offsets over the same bytes: readShapes
discards a file whose version does not match and rebuilds it from the trail. That is right for
derived counts and fatal for a lifetime one — a rebuild from a rotated trail resets the count downward,
the exact fabricated-dead-clause failure. The two files have opposite disposability, so they are two
files. readCitations is correspondingly the mirror image of readShapes: on a version mismatch it
keeps the counts and drops the offsets, because the counts are the part that cannot be rebuilt.
Cost: a second read of the same ≤8 MiB at session end.

The class-transition table — every entry produced by calling classify()

classify(level, changed, lifetimeFires, misses, matches) reads lifetimeFires on exactly one path:
changed === 0 && matches === 0 && isSafetyLevel(level). Everywhere else it is dead input. So there
are two transitions and four no-ops, and all six are asserted at lifetimeFires 0 and 1:

Case lifetimeFires: 0 lifetimeFires: 1 Moves?
red, changed 0, near-miss present — the misclassification #85 is about dead-weight? deterrent yes
orange, changed 0, no near-miss insufficient-exposure deterrent yes
red that still decides something (changed 3) in-service in-service no — changed short-circuits
red pre-empted by another rung (matches 4) shadowed shadowed no — matches short-circuits
green, changed 0 retire retire no — not a safety level
yellow, changed 5 in-service in-service no

Both transitions move towards deterrent, i.e. towards do not retire this. Verified rather than
assumed: EvidenceClass has six members with retire first (ablate.ts:70); all six are asserted
reachable by construction; retire is unreachable for red/orange and the three safety classes are
unreachable for green/yellow, because isSafetyLevel is red-or-orange. No class string literal appears
as an expectation anywhere in the new tests.

Blast radius: none on what gets proposed. retirement_candidate is
changed === 0 && !isSafetyLevel(level) and does not read the count at all, asserted across
lifetimeFires of 0, 1 and 10,000 for both a red and a green. Fail-closed holds: this changes a
label a human reads, never a verdict and never a proposal.

Callers, all of them

classify has exactly one caller, ablate; ablate is reached through ablateAll from
src/cli/learn.ts and src/policy/cli.ts. Both now pass citations. AblationOptions.citations is
the corpus-wide form of the existing lifetimeFires seam, because ablateAll ablates every clause
against one options object and a single number cannot serve a corpus. It combines with the record scan
by max, never replacing it: the durable count lags fires that arrived since the last fold, the
scan lags a rotation, and each is a floor under the truth. session-sitter learn folds via
accumulate('cli') before ablating, so it reads the freshest count available; policy ablate reports
without folding, so an unfolded counter leaves the record scan in charge — the pre-existing behaviour.

Mutation evidence — each half reverted independently

Mutation Tests failed
raise becomes a plain assignment 3 — direct max, rotation-holds, add-on-top
Offsets never committed (always full re-read) 5 — both idempotence tests, both-generations, and both rotation tests
The re-read branch adds instead of merging with max 2 — both rotation tests
classify requires two fires for deterrent (>= 1>= 2) 2 — the transition table and the reachability assertion
ablate ignores opts.citations 2 — the end-to-end reclassification and the max-combination
?? instead of max when combining (takes a stale zero over a live scan) 1 — the max-combination
accumulate no longer folds citations 1 — the pipeline test, which reads the file rather than greps for the call

Also found, not fixed

Osher Elhadad added 2 commits September 4, 2026 20:12
…ra#81)

`renderedCount` exempts every matchable clause from the per-tier ceiling, and
that exemption is what makes a mining pipeline structurally unable to consume
the prompt budget. It is not a property of the schema: it holds only because
the render paths test a patterned clause against the call and drop it. That is
a cross-module dependency, and `ablate.ts` says so two lines above the
function.

Asserting `renderedCount === 0` alone would pass for the wrong reason the day
a render path stops dropping — the count would simply be over the wrong
population. So these assertions name the behaviour instead:

  - over a corpus of matchable mined clauses past the ceiling, `renderedCount`
    is 0 *and* `selectClauses` reports every one of them as `evaluated-missed`
    with nothing selected and no budget spent;
  - a clause that does match is rendered for that call and the count still
    does not move, so the exemption is per call rather than per corpus;
  - the same clauses as prose consume the whole ceiling, so the zero is the
    pattern test and not the fixture;
  - `coreClauses` / `renderCore` exempt them too. eranra#81 names only the selector,
    but there are two channels a patternless clause reaches the prompt
    through, and relaxing the core one would push every mined red into the
    cached prefix — the standing instruction budget, plus a revision-hash
    change on every running session. Strictly worse, and equally silent.

Mutation-proved both ways: removing the selector's `evaluated-missed` branch
fails the first two while `renderedCount` keeps returning 0, which is exactly
the wrong-population failure the issue describes; relaxing `coreClauses`
fails the fourth.

A test rather than a runtime invariant, because no runtime input can reach
this state: the filters are code, there is no `SelectOptions` field or
setting that reintroduces an unfiltered bundle, and a hand-edited artifact
cannot make a patterned clause render either. A per-decision branch guarding
against a source edit CI already sees would be a counter nobody reads.
…ranra#85)

`lifetimeFires` was computed from the trail, and the trail rotates at 4 MiB
keeping one generation. So a red that fired steadily for months and is quiet
this week was indistinguishable from one that never fired: `classify()` called
it `dead-weight?` or `insufficient-exposure` instead of `deterrent`, and the
listing a human reads proposed retiring a clause whose whole value is that it
stopped something being tried.

`pipeline/citations.json` is one number per clause, folded by the same
offset-driven Stage A pass that already advances the trail offset, and read by
`ablate` as `lifetimeFires`.

**Monotonic, structurally.** `raise` is the only writer and it assigns
`max(prior, candidate)`. There is no subtraction, no reset and no other
assignment in the module, so "the count went down" is not a bug to avoid but a
value the writer cannot express. A rotation that shrinks the trail therefore
cannot shrink the count, and new fires still add on top of the held number.

**Idempotent.** The fold commits an offset and a tail hash per generation, so
a re-run reads nothing. When an offset fails its tail-hash check the module
re-reads both generations whole and merges the recount with `max` rather than
adding it, so re-reading counted bytes cannot double them either. Its ceiling
is documented in place: fires living only in bytes rotated away before any
fold saw them are lost, so the count can lag the truth but never exceed it and
never fall — and an undercount can only make a deterrent look under-exposed,
never make a live clause look dead.

Its own file rather than `shapes.json`, even though that file already carries
offsets over the same bytes: `readShapes` discards a file whose version does
not match and rebuilds it from the trail, which is right for derived counts
and fatal for a lifetime one. The two have opposite disposability. Not a
`DecisionRecord` field either — the trail is append-only, so a stored derived
value goes stale with no way to correct or detect it.

`AblationOptions.citations` is the corpus-wide form of the `lifetimeFires`
seam, because `ablateAll` ablates every clause against one options object.
Combined with the record scan by `max`, never replacing it: the durable count
lags fires that arrived since the last fold, the scan lags a rotation, and
each is a floor under the truth.

The transition table, every entry produced by calling `classify()`:
`lifetimeFires` is read on exactly one path (`changed === 0 && matches === 0
&& isSafetyLevel(level)`), so of six cases two move — `dead-weight?` →
`deterrent` and `insufficient-exposure` → `deterrent` — and four are no-ops
(`in-service`, `shadowed`, `retire` twice). `retirement_candidate` is
`changed === 0 && !isSafetyLevel(level)` and does not read the count at all,
so no value of it can propose a retirement that was not already proposed.
All six `EvidenceClass` members are asserted reachable by construction.

Mutation-proved, each half independently: `raise` as a plain assignment fails
3 tests; never committing an offset fails 5; the re-read branch adding instead
of merging fails 2; `classify` requiring two fires fails 2; `ablate` ignoring
the durable count fails 2; `??` instead of `max` fails 1; `accumulate` not
folding fails 1.
Copilot AI lite review requested due to automatic review settings September 4, 2026 18:34

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@OsherElhadad
OsherElhadad merged commit 8b50df2 into eranra:main Sep 4, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The rendered-ceiling exemption depends on the selector, and nothing asserts it

2 participants