Skip to content

fix(rules): restore the RE001 structural detectors deleted by #773 - #827

Merged
hyperpolymath merged 1 commit into
mainfrom
fix/826-re001-helpers
Sep 22, 2026
Merged

hyperpolymath merged 1 commit into
mainfrom
fix/826-re001-helpers

Conversation

@hyperpolymath

Copy link
Copy Markdown
Owner

Closes #826.

What happened

PR #773 says in its own body: "Expand the RE001 rule documentation… Validation was not run; this is a documentation-only change."

Its measured diff is 10 insertions, 104 deletions — only 4 of the inserted lines are prose. It deleted eight private helpers and replaced the RE001 job analysis with two regexes over raw job text.

The tell is that two of the deleted helpers are named for the exact fixes whose tests now fail: harden_runner_uses?/1 (from ab1f855) and block_scalar?/1 (from e047d46).

A regex that replaces a parser breaks in both directions at once

direction input regex behaviour consequence
false positive - uses: "step-security/harden-runner@main" requires an unquoted value, so it does not match RE001 fires against a job that is hardened
false negative the same string inside run: | or env: > matches shell/data as if it were a step a genuine finding is suppressed

The runs-on half had the same shape: ~r/^\s+runs-on:/m matches a runs-on nested inside an env: block scalar, and it also cannot tell a job's own field from a sibling's — which is precisely what direct_job_field?/2 exists to decide, since a reusable job delegates its runtime to the source workflow.

The fix

Restores direct_job_field?/2, harden_runner_step?/1, job_field_indent/1, lines_in_steps/2, step_indent/2, harden_runner_uses?/1 and block_scalar?/1, and re-points the two detectors at them.

The @doc is kept and genuinely expanded — which is what #773 claimed to do but did not. It reworded the prose slightly while deleting the logic, and in doing so dropped the "without a direct runs-on field" precision that direct_job_field?/2 implements. The new section documents why the check is structural rather than line-oriented, naming both failure directions so the next person is less likely to "simplify" it back into a regex.

Mutants — a passing suite proves nothing until one dies

Every red below was verified to be an assertion failure, not a compile error:

# Mutant Result
1 harden detection reverted to the #773 regex BOTH tests red — quoted-uses false positive and block-scalar false negative
2 runs-on detection reverted to the #773 regex :120 red, reporting a finding at line 8 for a runs-on nested inside an env: block scalar
3 block_scalar?/1 neutered to always-false :120 red

Clean control: 1593 tests, 0 failures. The suite is green for the first time since 2026-09-12.

Secondary defect — why five red CI runs were undiagnosable

tests/e2e.sh printed tail -120 of the run on failure. ExUnit prints failure diagnostics inline, as each test fails — it does not collect them at the end. At ~1600 tests under --trace the log is dominated by two trace lines per test, so no fixed tail window can contain a failure block.

Measured directly here: the failing run's log is 4819 lines, and tail -120 contains zero failure blocks. That is why the reds could not be read from CI at all and had to be reproduced locally — and why the failing tests were initially misidentified from passing trace lines that happened to land inside the window.

The harness now selects failure blocks by content, not by position — from the first N) test … to the end, capped at 500 lines — and falls back to a tail only when there is no failure block, which means the run never reached the tests (a compile error or a startup crash), where the tail is the informative part. HYPATIA_KEEP_MIX_LOG=1 retains the full log.

Verification

  • mix test → 1593 tests, 0 failures, 242 excluded
  • mix format --check-formatted → clean
  • bash -n tests/e2e.sh → syntax OK
  • three mutants killed, each on an assertion; clean control green
  • the diff removes only the two regex detectors, the now-unused active_content, and the vaguer prose

🤖 Generated with Claude Code

https://claude.ai/code/session_0113HQM9LVGkNCzU1WwkJZSV

)

PR #773 described itself as "documentation-only" and stated that validation
was not run. Its measured diff is 10 insertions and 104 deletions: it removed
eight private helpers and replaced the RE001 job analysis with two regexes
over raw job text.

That broke the rule in BOTH directions at once:

  * false positive — `uses: "step-security/harden-runner@..."` is valid YAML
    and does install the hardener, but the regex requires an unquoted value,
    so RE001 fired against jobs that were already hardened;
  * false negative — the same string inside a `run: |` or `env: >` block
    scalar is shell or data, not a step, but the regex matched it and
    suppressed a genuine finding.

Two of the deleted helpers are named for the exact fixes whose tests now
fail: `harden_runner_uses?/1` (ab1f855) and `block_scalar?/1` (e047d46).

Restores `direct_job_field?/2`, `harden_runner_step?/1`, `job_field_indent/1`,
`lines_in_steps/2`, `step_indent/2`, `harden_runner_uses?/1` and
`block_scalar?/1`, and re-points the two detectors at them.

The @doc is kept and genuinely expanded — which is what #773 claimed to do but
did not; it reworded the prose slightly while deleting the logic, and dropped
the "without a direct `runs-on` field" precision that `direct_job_field?/2`
implements. The new section documents why the check is structural rather than
line-oriented, naming both failure directions.

Mutants (a passing suite proves nothing until one dies) — every red below is
an assertion failure, not a compile error:

  1. harden detection reverted to the #773 regex  -> BOTH tests red
     (quoted-uses false positive AND block-scalar false negative)
  2. runs-on detection reverted to the #773 regex -> :120 red, reporting a
     finding at line 8 for a `runs-on` nested inside an `env:` block scalar
  3. `block_scalar?/1` neutered to always-false   -> :120 red
  Clean control: 1593 tests, 0 failures.

Secondary defect, same issue: tests/e2e.sh printed `tail -120` on failure.
ExUnit prints failure diagnostics INLINE as each test fails, not collected at
the end, so at ~1600 tests under --trace no fixed tail window can contain
one. Measured here: the failing run's log is 4819 lines and `tail -120`
contains zero failure blocks — which is why five consecutive red CI runs were
undiagnosable and this had to be reproduced locally. The harness now selects
failure blocks by content (from the first `N) test` to the end, capped), and
falls back to a tail only when no failure block exists, which means the run
never reached the tests.

Closes #826

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0113HQM9LVGkNCzU1WwkJZSV
@coderabbitai

coderabbitai Bot commented Sep 22, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

Next included review available in 22 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 5f44265e-776c-4010-9d25-6307d83844fd

📥 Commits

Reviewing files that changed from the base of the PR and between 54f14da and b8cca58.

📒 Files selected for processing (2)
  • lib/rules/research_extensions.ex
  • tests/e2e.sh

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@hyperpolymath
hyperpolymath merged commit f079bc0 into main Sep 22, 2026
78 of 95 checks passed
@hyperpolymath
hyperpolymath deleted the fix/826-re001-helpers branch September 22, 2026 11:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RE001 regressed by #773: a "documentation-only" PR deleted the quoted-uses and block-scalar handling — E2E Elixir job red on main

1 participant