Consolidate two clauses that say the same thing, when it can be proved - #90
Merged
Merged
Conversation
`proposals.merges` was typed `never[]`: the offline pipeline could add a clause and retire one, and never consolidate two. Retirement answers "is this clause carrying its weight?" and ablation answers it; merging asks "are these two the same rule written twice?", which ablation cannot see, because both halves of a duplicated pair are individually in service. Only one of the three detections in the design survives contact with what is provable without a regex prover: same-tier, same-level containment. `A`'s matched set is a proven superset of `B`'s, so the pair decides exactly what `A` alone decides. It is a static fact about the pattern text — no window, no replay, no traffic — so it holds on a corpus nobody has exercised and it does not weaken when the trail rotates. Same level is not decoration on that rule, it is what makes containment sufficient. A broad clause and its narrow exception are supposed to overlap, and that is exactly a level difference; an exception deciding the same way as the rule it excepts is a no-op, so a same-level narrower clause carries nothing the broader one does not. Intersection without containment emits nothing at all, not even a report. The evidence for it — "both ablate to the same changed set" — is a bounded claim wearing an unbounded one's clothes, and the case is not detectable anyway: between two anchored command matchers one literal is a token prefix of the other or the languages are disjoint, with no third outcome. The consolidation is a retirement of the subsumed clause rather than a new clause carrying `supersedes: [A, B]`. That shape is right when the merged matcher is neither clause's, which is the case that got refused; for containment it would be a byte-for-byte duplicate of a clause already in the corpus under a fresh id, and +1 −2 is the same net as −1. So this lane writes no file and no `supersedes`, which is not a weaker guarantee than "our supersedes is acyclic" — it is the same guarantee with nothing left to get wrong. Non-widening is set equality, not an inequality, and it is asserted with the runtime's own evaluator: `ablate(drop, corpus, records).changed === 0`. Writing that test found the trap it exists for — `RECORDED` injections replay a model-allowed call's own allow, so removing a green over an all-allow window reports zero whatever the clause does. The first draft passed against a window in which nothing could change; the liveness assertion is now beside the zero. Guards, each with a test that fails alone when the guard is reverted: same level and same tier; only a `learned` clause is ever dropped; red and orange are listed and never proposed; mutual containment is broken by id so equal clauses yield one finding rather than a two-cycle; a clause with no patterns is refused on both sides, since "every pattern is covered" is vacuously true of prose. `--no-retire` suppresses the static pass too: it is cheap, and it is still a retirement. 2,722 -> 2,736 tests, 106 files.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #82.
proposals.mergeswas typednever[], so the offline pipeline could add a clause and retire one andnever consolidate two. It is now
MergeFinding[]and populated — but by a narrower rule than thedesign asked for, and on a different channel, and both of those are the substance of the PR.
The design question: what evidence justifies proposing a merge?
11-mine-v2.md§8.3 offers three detections. One survives, and it is the one that needs no windowat all.
Accepted — containment, same tier and same level. The kept clause's matched set is a proven
superset of the dropped clause's, so the pair decides exactly what the kept clause alone decides and
the dropped one is dead text. This is a static fact about the pattern strings: no window, no replay,
no traffic. It therefore holds on a corpus that has never been exercised, and it does not weaken when
the decision trail rotates at 4 MiB keeping one generation — the bound that makes every
replay-derived claim in this pipeline a claim about a window rather than about a rule.
Containment is proved two ways and no others:
in the command literal of an anchored matcher. A substring matches anywhere in the haystack, so it
is enough that it occurs inside the other pattern's text.
(
commandMatcher).subsumesMatcherrecognises that form by parsing back its own literal parts— it is a closed, versioned shape we generate — and compares the literals. A longer command prefix
matches strictly fewer commands, but only when the extra text starts on a token boundary:
npdoes not subsume
npm test, because nothing afternpinnpmis whitespace, so the shortermatcher's own lookahead refuses the very command the longer one accepts. That off-by-one is the
whole reason this is a function with a test rather than a
startsWith.Any other regex returns null. There is no regex prover here and there should not be: a containment
prover would be subtly wrong for two years, and wrong in the direction that merges an exception away.
Same level is not decoration on the containment rule — it is what makes containment sufficient.
The objection in the ticket is exactly right: a broad clause and a narrow exception are supposed to
overlap. But that is a level difference. An exception that decides the same way as the rule it
excepts is a no-op, so a same-level narrower clause carries no information the broader one does not
already carry. Different levels with intersecting patterns is never a merge.
Rejected — patterns that intersect without containment, which §8.3 would propose when "both
ablate to the same
changedset". Two reasons, either sufficient:this window" is not "these are the same rule", and the window is one rotating generation.
a token prefix of the other — which is containment, case 1 — or no command can start with both and
the languages are disjoint. There is no third outcome. Between two arbitrary substrings,
intersection-without-containment needs the prover we just refused to write.
So it emits nothing at all, not even a report. A list of candidate pairs assembled by a heuristic
is worse than no list: it teaches a reviewer to skim, and the first time it is wrong it will be wrong
about an exception.
Rejected — different levels. Left as-is. It is a conflict report, not a merge, and #83/#81 are
closer to that surface than this ticket is.
Why the output is a retirement and not a new clause carrying
supersedes§8.3 shapes a merge as one new
proposedclause withsupersedes: [A, B], forced byassertWritable— the pipeline cannot edit or delete an existing clause file. That shape is right for a merge whose
matcher is neither clause's. That is the intersection case, and it is refused.
For containment the merged matcher would be the kept clause's, byte for byte. The new file would be
a duplicate of a clause already in the corpus under a fresh id, and its arithmetic (+1 −2 = −1) is
exactly the arithmetic of retiring the subsumed clause (−1). It buys nothing and costs the two failure
modes #60 had to add refusals for: a
supersedesnaming an id that does not exist, and a supersessioncycle — which a containment merge would have walked straight into, because a content-derived id over
the kept clause's own matcher can collide with the kept clause's own id and produce a self-supersession.
So the finding goes on the channel
planRetirementsalready uses: a run-line entry, no file written, ahuman acts. This lane writes no
supersedesat all. That is not a weaker guarantee than "oursupersedesis acyclic"; it is the same guarantee with nothing left to get wrong.Non-widening, and it is set equality
L(keep) ⊇ L(drop)givesL(keep) = L(keep) ∪ L(drop), so the corpus after the retirement decidesexactly what it decided before: a green allows no command it did not allow, and a red denies every
command it denied. Asserted behaviourally with the runtime's own evaluator rather than argued —
ablate(drop, corpus, records).changed === 0.Three narrowings keep that proof true rather than merely plausible:
learnedclause is ever dropped. A machine proposal does not initiate retiring a human'spractice, in either direction — the same rule
compareLadderencodes.Clause.originabsent meanshuman, and that default is the safe one.present in the context this one is. Equality of decided sets only follows when both are loaded
together.
sound for a red too; the house rule stands regardless — the pipeline does not get to initiate
disarming a safety clause.
The trap that test found
The first draft of the non-widening test passed against a window in which nothing could ever change.
RECORDEDinjections replay a model-allowed call's own recorded allow, so removing a green over anall-allow window reports
changed: 0whatever the clause does. Another instance of the wave's bugshape — a silence that reads as success. The window is now built from fail-closed denies, which is both
the traffic a green is actually learned from and the only traffic a green's ablation can move, and the
zero sits next to a liveness assertion: removing the kept clause does move decisions.
What could not form a cycle, and the test for it
Findings are directed edges
keep → drop. Two properties prevent a ring:drop. Otherwise a reviewer is asked to retire the sameclause once per clause that covers it.
thing each subsume the other, and emitting both directions is a two-cycle: accept both and the rule
disappears entirely, which is the annihilation
supersessionCyclesexists to refuse. This was areal bug in the first implementation, not a hypothetical. Across the kinds of proof mutual
containment cannot arise — an anchored matcher never subsumes a free substring — so the tie-break
only ever fires on genuinely equal languages.
The test asserts acyclicity structurally and hands the findings to the real
compilePolicyas thesupersedesedges §8.3 would have written, through the realparseLearnedClause. So even the shapethis PR declined to emit compiles. Removing the tie-break fails it.
Mutation evidence
Every guard reverted independently,
propose.test.ts+pipeline.test.ts(108 tests) each time:a broad clause and its narrower exception at a different level are never merged,never merges across tiers— 2 failed, 106 passeda red containment is listed, never proposed— 1 failed, 107 passedtwo clauses saying the same thing yield one finding, not a two-cycle,the findings can never form a supersession cycle— 2 failed, 106 passednever drops a human clause, whichever way containment runs— 1 failed, 69 passednever proposes retiring a prose clause on the strength of it having no matchers— 1 failed, 69 passedSurfaces
RunLine.proposals.merges—never[]can now be typed properly, and is:MergeFinding[], withkeep,drop,tier,level,proof,proposedand a verbatimnote. Still[]rather thanabsent when there is nothing, so a reader never has to distinguish "no merges" from "this version
did not report merges".
candidates.mergedcounts only the proposed ones, and it now enters both the headline arithmetic(
clauses: +1 −0 merge 1 = net 0) andexitReasonFor. A run whose only output is a merge findinghas something a reviewer can act on; reporting it as
no-shape-cleared-floorwould be the samesilence-reads-as-success bug pointing the other way.
session-sitter learnprints one line per finding,=proposed,?listed.--no-retiresuppresses the static pass too. It is cheap and window-independent, and it is still aretirement — the flag's own help text says "so no retirement is proposed", and one flag meaning two
things is how a governance surface starts lying.
Relationship to what ablation already finds
Ablation does reach the same clauses, as
shadowed— the dropped clause matches real calls andremoving it changes nothing, so
shadowed_bynames the other clause. Two differences make the staticfinding worth having beside it:
shadowedis deliberately ambiguous — "delete as redundant or narrow to cover what the otherrung does not" — because ablation cannot tell those apart. Same-level containment can: there is
nothing to narrow onto, because a same-level narrower clause adds nothing.
Constraints
Zero runtime dependencies, TypeScript only, no model on the path, nothing written outside
assertWritable's permitted set (this lane writes nothing at all). Every fixture invented.make checkgreen: 2,722 → 2,736 tests, 106 files.make pluginrun last;ci/check-plugin-lib.shreportsplugin/lib is the current build of src/.Known ceilings, marked in the source
token.
expiresis not consulted, because nothing consumes it yet (the schema gap The pipeline can add and retire a clause, but never merge two #82's sibling ticketsare about). Once decay ships, a pair whose
expiresdiffer must be refused: the kept clause couldexpire first and take the dropped clause's coverage with it. Noted at the call site.