Skip to content

fix(scorecard): fire the reusable's pull-request job on PRs - #108

Merged
hyperpolymath merged 1 commit into
mainfrom
fix/scorecard-pr-trigger-20260915
Sep 15, 2026
Merged

hyperpolymath merged 1 commit into
mainfrom
fix/scorecard-pr-trigger-20260915

Conversation

@hyperpolymath

@hyperpolymath hyperpolymath commented Sep 15, 2026 •

Copy link
Copy Markdown
Owner

What was broken

scorecard.yml called standards/.github/workflows/scorecard-reusable.yml on
push only. The reusable's pull-request: job — which measures the repo-level
Scorecard checks and merges them into the PR-tree SARIF — is gated on
github.event_name == 'pull_request', so it never fired. The code_scanning
rule on main requires three tools (CodeQL, Hypatia, Scorecard), and two of
the three Scorecard categories can only come from that job. Result: every PR
waited forever on a check that could not be produced.

Two limbs, both in the caller:

  1. pull_request: added to on:, so the reusable's PR job actually runs.
  2. permissions: raised to the superset the callee needs
    (security-events: write, id-token: write). A called workflow can only
    narrow the caller's token, and read-all cannot cover a write scope — so
    before this, the run was a startup_failure.

Why the pin is da2c748a and not main

main is unusable by every caller in the estate, and so are the three
commits before it. GitHub validates a called reusable against the callee
repository's own .github/workflows/actions.lock
, which a caller cannot see,
fix or override. Dependabot's 2cea69eb bumped github/codeql-action inside
scorecard-reusable.yml without regenerating actions.lock, so every caller
now dies with:

Invalid dependency lockfile … references actions not present in the lockfile

Capability is therefore not monotonic in time — newer is not safer:

pin date pull-request: job lock-consistent usable
81dbf2dd 07-21 no yes no (no feature)
d200ddca 09-08 yes yes yes
8f2ee508 09-08 yes yes yes
da2c748a 09-10 yes yes YES ← newest usable
2cea69eb 09-12 yes no no
c78f9148 09-12 yes no no
c27611ff 09-14 yes no no
317101e0 09-14 (tip) yes no no

actionlint returns rc=0 on both caller and callee at every pin — a linter
structurally cannot catch this class, because the defect is a cross-repo
relation, not a property of either file.

Measured result

Run 34924268842: success in 43s. Run Scorecard PR executed for 38s and
uploaded SARIF carrying exactly three categories —

category results
supply-chain/local 6
supply-chain/branch-protection 1
supply-chain/online-scm 2

The code_scanning rule's own merge-protection check (Scorecard, app
github-advanced-security) reports success — "No new alerts in code changed
by this pull request"
, alongside Hypatia and CodeQL.

⚠ The SARIF lands on refs/pull/108/head, not /merge. The job checks out
head.sha and passes no ref: to upload-sarif, so the ref is inferred from
the checked-out tree; Hypatia and CodeQL land on /merge. A probe querying only
one ref reads a working cure as dead. The merge-protection check consumes the
head-ref analysis correctly.

Known gap, deliberately accepted

da2c748a predates the Select SARIF to upload and Fail if reconciliation did not succeed steps added at c27611ff. Here reconciliation was a no-op —
actions-lock-audit.json is [] and the reconciled SARIF is byte-identical to
its input (73778 B) — which on this pin passes silently rather than failing
loudly. Harmless in this run because the no-op preserved all three runs, but a
reconciler that dropped runs would ship fewer categories past the green
test -eq 3 assertion (that assertion runs before reconciliation and
asserts on a different file than the one uploaded). Those guards exist only
on lockfile-poisoned commits, so they are not available at any usable pin. The
durable cure belongs in standards: a CI gate asserting uses ⊆ actions.lock,
or adding the lockfile to Dependabot's update scope.

🤖 Generated with Claude Code

https://claude.ai/code/session_01HfgwLCdKNd5iZVo6VTiSim

@coderabbitai

coderabbitai Bot commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Advanced

Run ID: a34e0909-0450-4270-b96f-a591df461e4d

📥 Commits

Reviewing files that changed from the base of the PR and between ed9e276 and 1b07914.

📒 Files selected for processing (1)
  • .github/workflows/scorecard.yml

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.

📜 Recent review details
⏰ Context from checks skipped due to timeout. (4)
  • GitHub Check: CodeQL
  • GitHub Check: hypatia / Hypatia Neurosymbolic Analysis
  • GitHub Check: Idris2 proofs + tests
  • GitHub Check: analyze (actions, none)

📝 Summary

Summary by CodeRabbit

  • Chores
    • Pull requests targeting the main and master branches now trigger the project’s security checks.
    • Updated the underlying security analysis workflow to use a newer validated version.
    • Security results can now be incorporated into pull-request reporting when applicable.

Walkthrough

The Scorecard workflow now runs for pull requests targeting main or master. It also uses an updated pinned commit for the reusable Scorecard workflow.

Changes

Scorecard workflow execution

Layer / File(s) Summary
Workflow trigger and reusable workflow pin
.github/workflows/scorecard.yml
The workflow adds a pull_request trigger for main and master. It updates the pinned reusable workflow commit.

Priority: ➖ Normal

Estimated code review effort: 1 (Trivial) | ~5 minutes

Change: Bug fix

Merge Risk: ⚪ Minimal · up to 1b079

The workflow can upload Scorecard results for pull-request runs as intended. No actionable merge-blocking risk remains.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly and concisely describes the main change: enabling the reusable Scorecard pull-request job to run for pull requests.
Description check ✅ Passed The description is directly related to the Scorecard workflow changes and explains the trigger, permissions, reusable workflow pin, observed result, and known gap.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

A rabbit checks the workflow trail
Pull requests now join the tale
Main and master open the gate
A newer pin keeps work up to date
Scorecard hops with steady feet

Comment @coderabbitai help to get the list of available commands.

coderabbitai[bot]
coderabbitai Bot previously approved these changes Sep 15, 2026
ochrance requires the `code_scanning` ruleset rule with Scorecard among its
tools, but no pull_request event ever produced Scorecard SARIF, so the rule
waited forever and every PR sat BLOCKED with all checks green.

Two independent reasons the cure never fired, both fixed here:

1. `on:` had no `pull_request:`. The reusable in hyperpolymath/standards
   already contains a `pull-request:` job gated on
   `if: github.event_name == 'pull_request'`, but the caller never raised
   that event, so the job could not run.

2. The pin was @81dbf2dd (2026-07-21), which predates the `pull-request:`
   job entirely. That job entered the reusable at d200ddca on 2026-09-08.

Pinned to da2c748a rather than to main. Measured: `standards` main tip is
NOT usable by any caller. Dependabot's 2cea69eb (2026-09-12) bumped
github/codeql-action cdf488f5 -> b96794f0 inside scorecard-reusable.yml
without regenerating .github/workflows/actions.lock, so every caller
pinning 2cea69eb or later dies before any job starts with
"references actions not present in the lockfile". da2c748a (2026-09-10) is
the newest commit that both carries the `pull-request:` job and whose
lockfile is consistent with its own workflow.

Workflow-level permissions are widened to exactly the superset the callee's
jobs request (security-events: write, id-token: write); a called workflow
can only narrow the caller's token, and read-all cannot cover a write scope.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HfgwLCdKNd5iZVo6VTiSim
@hyperpolymath
hyperpolymath force-pushed the fix/scorecard-pr-trigger-20260915 branch from 1b07914 to 009692b Compare September 15, 2026 03:15
@hyperpolymath
hyperpolymath merged commit a9148b9 into main Sep 15, 2026
40 checks passed
@hyperpolymath
hyperpolymath deleted the fix/scorecard-pr-trigger-20260915 branch September 15, 2026 03:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant