test(ci): pin the no-visual-change waiver so it cannot rot - #553
Open
chaitanyagiri wants to merge 1 commit into
Open
chaitanyagiri wants to merge 1 commit into
chaitanyagiri wants to merge 1 commit into
Conversation
… template THE ESCAPE ALREADY EXISTS. `.github/workflows/pr-evidence.yml` has had the `no-visual-change` waiver since the gate shipped, at lines 55 to 59, checked before any parsing and returning early. The bot's own failure comment already tells the author about it, and the label already exists in the repository. I used it on #547 today and the check passed in three seconds. So this change does NOT build the escape. It does the two things that were actually missing. ONE. IT WAS COMPLETELY UNTESTED. A branch that lets a required status check pass, with nothing asserting when it may and may not fire, is the dangerous shape: it can rot in two directions and neither is loud. If it stops working, a CI tweak or a typo has no way through a required check. If it starts always firing, the gate is decorative on every pull request and nobody notices. Eight tests, in the style already in this file: the label name, the ordering and the guard are read OUT of the workflow rather than restated, so editing the workflow carelessly fails here instead of on a contributor's pull request. the label still matches the template and the bot comment the waiver is checked BEFORE the evidence is computed the waiver returns early rather than falling through it FIRES with the label, at any position in the list it is case insensitive, because labels are typed by hand it does NOT fire without the label <- the always-pass guard it needs the WHOLE label, so `needs-no-visual-change` does not walk through All four mutations go red, checked before committing: rename the label in the workflow only 3 tests fail substring match instead of exact membership 6 tests fail remove the early return 1 test fails move the waiver after the evidence parsing 1 test fails TWO. THE TEMPLATE ONLY MENTIONED IT INSIDE AN HTML COMMENT, so it never rendered. There is now a visible note under `## Evidence` naming the label and saying it is the supported way past the check. Two further tests prove that adding visible text changed nothing: an unfilled template still fails, and the note itself is not mistaken for evidence. `.github/workflows/pr-evidence.yml` IS NOT MODIFIED BY THIS COMMIT. WHY 42 PULL REQUESTS ARE STUCK, since that was the reason for the work: not a missing escape. Of 93 open pull requests exactly one carries the label. The escape works and nobody is applying it. Labelling them is a maintainer action and is not in this change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PogxmT9ULRQSYbfs1oigLx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The escape already exists. This pins it.
.github/workflows/pr-evidence.ymlhas had theno-visual-changewaiver since the gate shipped, at lines 55 to 59: checked before any parsing, returning early, exact label membership, case insensitive. The bot's own failure comment already tells the author about it. The label already exists in this repository.I used it on #547 today and
Before / after evidencepassed in three seconds.So this PR does not build the escape.
pr-evidence.ymlis not modified by this commit at all. It does the two things that were actually missing.1. It was completely untested
A branch that lets a required status check pass, with nothing asserting when it may and may not fire, is the dangerous shape. It can rot in two directions and neither is loud:
Eight tests, in the style already in this file: the label name, the ordering and the guard are read out of the workflow rather than restated, so editing the workflow carelessly fails here instead of on a contributor's PR.
needs-no-visual-changewalking throughMutation results, run before committing
Baseline is 15 pass, 0 fail.
2. The template only mentioned it inside an HTML comment
So it never rendered. There is now a visible note under
## Evidencenaming the label and saying it is the supported way past the check.Two further tests prove that adding visible text changed nothing: an unfilled template still fails the gate, and the note itself is not mistaken for evidence.
Why 42 PRs are stuck
Since that was the reason for the work: not a missing escape. Of 93 open pull requests, exactly one carries the label (#498). The escape works and nobody is applying it.
Labelling them is a maintainer action and is not in this change.
Before and after
No visual surface. This is a test file and three lines of template text, so the
no-visual-changelabel applies to this PR by its own rule.Evidence that the tests do their job is the mutation table above: four ways of breaking the waiver, all caught.
🤖 Generated with Claude Code
https://claude.ai/code/session_01PogxmT9ULRQSYbfs1oigLx