fix(policy): fail closed when set_check extraction breaks - #379
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a2f8628e4c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…ch count The perl compile oracle only proved a regex compiles; a capture-free regex that also matches nothing in the extraction loop never runs the `defined($1) or die` guard, so it vacuously passed as a clean zero-match result. Add an independent capture-group count check (via a synthetic always-matching alternation against @+) plus a canary covering exactly that combination.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 84ea548aec
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Summary
tools/policy_check.shtest_set_checkcould not tell a broken regex extraction apart from a genuine zero-match result: both collapsed to an empty capture{}. A zero-countsubsetfixture (notablytests/policy/safety-destructive-fix-gate-no-second-copy.json) could therefore pass vacuously if its extractor ever broke. This PR makes the check fail closed.Closes #373.
What changed
REenvironment variable and is compiled withqr//. It is never spliced into program source, so an unescaped/can no longer break the program.qr//instead ofgrep -P, a different engine.jqaggregation's exit status are each checked, and each has its own finding. A match with no group-1 capture now fails.{}can only come from a successful extraction that found nothing.grep -oP/sed -Efallback is deleted. It could not parse the PCRE(?:...)syntax the fixtures use, and it returned zero captures silently. If perl is missing, the check now fails loudly and names perl. perl is an essential package on theubuntu-latestCI runner, and the script already depends ongrep -Pelsewhere with no fallback./passes and capturesa/b.tests/policy/README.mdline 10 citedtools/policy_check.sh:1473, which was already stale. It now points at theSAFETY_FIXTURESdiscovery block by name. This is backlog item 6 of the prompt-audit follow-ups, folded in because this PR already edits the README.Files by plan step
tests/policy/fixtures/set-check-zero-match-canary.md(slash marker line, prose describing the five canary branches)tools/policy_check.sh(test_set_checkrewrite, canary assertions 3-5)tests/policy/README.md(new section 8 "Runtime dependencies", line 10 pointer)Validation
bash tools/validate.sh --changedescalated to the full suite plus--self-testbecausetools/**changed. Exit 0, all suites pass.policy_check.sh --strict: Checks passed 72/73, 0 new findings (unchanged baseline).|| extract_out="") makes the canary fail, so the canary catches the regression it guards.Versioning
No bump. Only
tools/andtests/changed; noplugin/file is touched.Unresolved
None. The script-wide discovery gaps in other
findsites remain tracked in #377.