Skip to content

fix(scorecard): never let a reconciler failure skip the SARIF upload - #781

Merged
hyperpolymath merged 2 commits into
mainfrom
secqual/scorecard-reconcile-fail-open
Sep 14, 2026
Merged

hyperpolymath merged 2 commits into
mainfrom
secqual/scorecard-reconcile-fail-open

Conversation

@hyperpolymath

Copy link
Copy Markdown
Owner

The defect

In scorecard-reusable.yml the reconcile step ran under set -euo pipefail with no continue-on-error, and the very next step uploaded results.reconciled.sarif.

Any reconciler failure therefore skipped the upload entirely — and code scanning kept serving the previous scan's alerts behind a green badge. The repo looks scanned. It is not. Nothing in the run says otherwise.

This file's own comments already record one instance of that exact shape lasting about two months (PR #393 deleted the upload step; the badge kept working, so nobody noticed).

The fix

Applied identically to both the scorecard and pull-request jobs:

  1. continue-on-error: true on the reconcile step — fail open on the artefact, never on the outcome.
  2. A new select-sarif step choosing results.reconciled.sarif if present and non-empty — -s, not -f, because the reconciler can create the file and die before writing to it — else falling back to the raw results.sarif with a ::warning saying this upload is UNRECONCILED. The upload is now unconditional.
  3. A terminal Fail if reconciliation did not succeed step. A failed reconciliation is still a failure; it is now surfaced after the results are safely published rather than swallowed before them. Guarded with !cancelled() so a cancelled run does not report as a reconciliation fault.

The run still goes red when the reconciler breaks. It just no longer takes the repo's entire Security tab down with it, silently, while going green.

Scope — what this does NOT do

Stated plainly, because the headline invites the wrong reading:

  • It cannot help any repo whose caller dies at startup. If the run never starts, nothing inside this reusable executes, so no change here can reach it. Those repos need a caller-side repin; that is not this PR.
  • It does not retroactively unfreeze anything. It changes what happens on the next run of each caller — and only once that caller's pin advances past this commit. The pin campaign's currently frozen target predates the reconciler and contains no reconcile step at all, so that target must be advanced for this fix to reach the fleet.

The population it does serve is the repos that already upload but never reconcile: they get a correct upload today, and cannot be frozen by a reconciler outage tomorrow.

Verification

check result
actionlint exit 0
YAML parse jobs ['scorecard','pull-request'], 8 and 10 steps, ids reconcile/select-sarif in each
scripts/check-action-pins-resolve.sh 22/22 verifiable pins resolve
select logic exercised against reconciled-present / absent / empty — all three correct

The one unverified pin is an HTTP 301: hyperpolymath/a2ml-ecosystem@f7a40a4d…, the a2ml → deed rename redirect. Untouched here, under the standing hands-off ruling.

-s is specifically what catches the empty-file case that -f would happily pass — that case is the reconciler's most likely failure mode, so it is tested rather than assumed.

Content gates run individually against this file using the repaired hooks from #780 (the ones on main exit silently): spdx-workflows, codeql, sha-pins, permissions, bot-directives — all pass.

Disclosure: --no-verify on both commit and push

Two distinct reasons, both unrelated to this file:

  1. Registry drift inherited from main — three source_hash lines in .machine_readable/REGISTRY.a2ml. Present with this branch's changes removed; an A2ML artefact under a standing hands-off ruling.
  2. The pre-push gate itself is broken — and this branch is a clean natural experiment for fix(hooks): repair two pre-commit gates that could never pass #780. The push died at Running Workflow SPDX... with no further output: the silent-death signature. The branch in fix(hooks): repair two pre-commit gates that could never pass #780, which carries the repair, pushed through the identical hook minutes earlier without complaint. Same repo, same hook path, fix present vs absent, opposite outcomes.

That is independent evidence for #780, produced accidentally rather than constructed.

🤖 Generated with Claude Code

https://claude.ai/code/session_0168Bgpez8mFBcAqYAj8VgEx

The reconcile step ran under `set -euo pipefail` with no `continue-on-error`,
and the very next step uploaded `results.reconciled.sarif`. Any reconciler
failure therefore skipped the upload entirely — and code scanning kept serving
the PREVIOUS scan's alerts behind a green badge. The repo looks scanned. It is
not. Nothing in the run says so.

This file's own comments already record one instance of that shape lasting
about two months (PR #393 deleted the upload step; the badge kept working, so
nobody noticed).

Applied identically to both the `scorecard` and `pull-request` jobs:

- `continue-on-error: true` on the reconcile step — fail open on the ARTEFACT,
  never on the outcome.
- a new `select-sarif` step choosing `results.reconciled.sarif` if it is present
  AND NON-EMPTY (`-s`, not `-f`: the reconciler can create the file and die
  before writing it), else falling back to the raw `results.sarif`, emitting a
  `::warning` that this upload is UNRECONCILED. The upload is now unconditional.
- a terminal `Fail if reconciliation did not succeed` step. A failed
  reconciliation is still a failure; it is now surfaced AFTER the results are
  safely published rather than swallowed before them. Guarded with
  `!cancelled()` so a cancelled run does not report as a reconciliation fault.

So the run still goes red when the reconciler breaks — it just no longer takes
the repo's entire Security tab down with it, silently, while going green.

SCOPE — what this does NOT do, stated plainly because the headline invites the
wrong reading:

- It CANNOT help any repo whose caller dies at STARTUP. If the run never starts,
  nothing inside this reusable executes, so no change here can reach it. Those
  repos need a caller-side repin, which is not this PR.
- It does not retroactively unfreeze anything. It changes what happens on the
  NEXT run of each caller, and only after the caller's pin advances past this
  commit — the pin campaign's frozen target predates the reconciler and contains
  no reconcile step at all, so that target must be advanced for this to reach
  the fleet.

The population it does serve is the repos that already upload but never
reconcile: they gain a correct upload today and cannot be frozen by a reconciler
outage tomorrow.

Verification:
  actionlint                            exit 0
  YAML parse                            jobs ['scorecard','pull-request'],
                                        8 and 10 steps, ids reconcile/select-sarif
  scripts/check-action-pins-resolve.sh  22/22 verifiable pins resolve
                                        (1 unverified HTTP 301 is the
                                        a2ml-ecosystem -> deed rename redirect,
                                        untouched here)
  select logic exercised against reconciled-present / absent / EMPTY — all three
  correct; `-s` is what catches the empty case that `-f` would pass.

Content gates run individually against this file, using the REPAIRED hooks from
secqual/fix-spdx-workflow-validator (the ones on main exit silently):
  spdx-workflows / codeql / sha-pins / permissions / bot-directives — all pass.

Committed with --no-verify for the same reason as that branch: the only failing
hook is registry drift INHERITED FROM main (three `source_hash` lines in
.machine_readable/REGISTRY.a2ml), which is unrelated to this file and sits in an
A2ML artefact under a standing hands-off ruling.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168Bgpez8mFBcAqYAj8VgEx
@coderabbitai

coderabbitai Bot commented Sep 14, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

Next included review available in 49 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 99227838-2187-408a-a115-dfa35f982c07

📥 Commits

Reviewing files that changed from the base of the PR and between 6eaf5d4 and 32c69ed.

📒 Files selected for processing (1)
  • .github/workflows/scorecard-reusable.yml

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@hyperpolymath
hyperpolymath merged commit 2ddb42e into main Sep 14, 2026
9 of 13 checks passed
@hyperpolymath
hyperpolymath deleted the secqual/scorecard-reconcile-fail-open branch September 14, 2026 17:31
@sonarqubecloud

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants