Skip to content

Put the authored rule shapes in CI, where they cannot rot - #144

Merged
patchstackdave merged 1 commit into
mainfrom
feature/protect-rule-shape-corpus
Aug 19, 2026
Merged

Put the authored rule shapes in CI, where they cannot rot#144
patchstackdave merged 1 commit into
mainfrom
feature/protect-rule-shape-corpus

Conversation

@patchstackdave

Copy link
Copy Markdown
Contributor

Independent of #142 and #143 — this touches only tests/protect/, so it can merge in any order.

Why

The authored rule shapes were verified once, by hand, in a local working directory no CI runs. They are
the only evidence that the shipped rules are not inert — and that is the failure mode the verification
exists for: a rule the engine silently rejects, or one whose match never fires, is indistinguishable from
protection. It is in the bundle, it shows in the dashboard, it blocks nothing. Four of these were inert at
some point for exactly that class of reason: an inline-flag regex, a phase left at the default, a
parameter source that does not exist.

rule-corpus.test.ts already proves the engine can express one canonical rule per vulnerability class.
This is a different claim about a different artifact: these are the shapes as authored — the AND-ed
gadget spellings, the mutations, the extra carriers — and the question is whether the engine executes them
as written.

What lands

Five shapes, each run through the real guard, each payload paired with a benign request — a rule that
blocks the attack and also blocks ordinary traffic has mitigated nothing either. Plus structural checks
for the three ways one of these went inert before: undelimited regexes, unknown parameter sources, and a
rule that screens egress while defaulting to the request phase.

Both mutation directions are checked, so neither half can rot into a tautology:

mutation fails
drop the base64 condition from the marker shape the base64-transport blocking test
broaden the gadget rule to match one half of the spelling the false-positive test for ordinary prose

The boundary

Shapes are keyed by mechanism, and the fixture is payload shapes only: no advisory identifiers, no
affected version ranges, no mapping from a shape to what it covers. A test asserts that, rather than
leaving it to review — this repository is public, and the coverage mapping is the product, so it stays
where the advisories are triaged.

Per-advisory duplication collapses with the same cut: several advisories share one shape byte for byte,
and what actually differs between them is the version scope, which is not a payload shape.

A drift this surfaced

One shape was missing the third condition its own prose described — the carrier that reaches a cookie-borne
payload — while the rule that had actually been verified included it. So the recorded shape was weaker than
both its documentation and the artifact the assertions covered, and regenerating instances from it would
have quietly dropped that coverage. The verified form is what landed here.

Verification

Full suite 1083 passed, typecheck clean.

The authored rule shapes were verified once, by hand, in a local working directory no CI runs. They are
the only evidence that the shipped rules are not inert — and that failure mode is the whole reason the
verification exists: a rule the engine silently rejects, or one whose match never fires, is
indistinguishable from protection. It is in the bundle, it shows in the dashboard, it blocks nothing.
Four of these were inert at some point for exactly that kind of reason: an inline-flag regex, a phase
left at the default, a parameter source that does not exist.

The five shapes now run through the real guard on every build, each payload paired with a benign request,
because a rule that blocks the attack and also blocks ordinary traffic has mitigated nothing either.
`rule-corpus.test.ts` already proves the engine can express one canonical rule per class; this asserts
that these shapes, as authored — AND-ed gadget spellings, mutations, extra carriers — execute as written.

Shapes are keyed by mechanism, and the fixture carries payload shapes only: no advisory identifiers, no
affected version ranges, no mapping from a shape to what it covers. That boundary is asserted rather than
left to review, because this repository is public and the coverage mapping is the product. Per-advisory
duplication collapses with it: several advisories share one shape byte for byte, and what actually
differs between them is the version scope.

Writing this found a drift in the source corpus worth recording: one shape was missing the third
condition its own prose described — the carrier that reaches a cookie-borne payload — while the rule that
had been verified included it. The shape was weaker than both its documentation and the artifact the
assertions covered, and regenerating from it would have quietly dropped that coverage. Both mutation
directions are now checked: weakening a rule fails a blocking assertion, broadening one fails a
false-positive assertion.
@coderbuds

coderbuds Bot commented Aug 18, 2026

Copy link
Copy Markdown

Comprehensive tests ensure all authored rule shapes execute as intended.

🎯 Quality: 86% Excellent · 📦 Size: Large — consider splitting if possible

🛡️ Standards: no pre-flight fit check ran for this change — wire assess-change-fit into your coding agents to catch size before opening.

📈 This month: Your 80th PR — above team average · Averaging Good

See how your team is trending →

@patchstackdave

Copy link
Copy Markdown
Contributor Author

/review

@patchstackdave
patchstackdave merged commit 0b6f0ab into main Aug 19, 2026
6 checks passed
@patchstackdave
patchstackdave deleted the feature/protect-rule-shape-corpus branch August 19, 2026 07:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants