Skip to content

cargo-audit.yml template: vulnerability issues carry no findings, no acceptance criteria and never close (rpa-elysium #113/#143 are the same issue twice) #1265

Open

Description

@hyperpolymath

Summary

The templated cargo-audit.yml publishes an issue that cannot be acted on and cannot be closed by the thing that opened it. This is the dependency-vulnerability member of the same family as #1070: reds and findings that reach the tracker without a determination.

The create-issue job (present in the estate template and in every consumer repo) is:

  create-issue:
    needs: audit
    if: failure() && github.event_name != 'pull_request'
    steps:
      - name: Create vulnerability issue
        run: |
          EXISTING=$(gh issue list --label "security,dependencies" --state open --json number -q '.[0].number')
          if [ -z "$EXISTING" ]; then
            gh issue create \
              --title "Security: Dependency vulnerabilities detected" \
              --body "cargo audit found vulnerabilities. Run \`cargo audit\` locally for details." \
              --label "security,dependencies"
          fi

Four problems, all of which the estate's own rules already forbid:

  1. No findings. The body is a pointer to a local command. The advisory IDs, the crate, the affected version and the suggested fix exist in the job that failed, and are thrown away.
  2. No acceptance criteria. "Depends on" triage, but the issue carries none, so it cannot be closed on evidence and cannot be prioritised against anything.
  3. Never closes. A passing audit does nothing; the issue stays open after the fix lands. In hyperpolymath/rpa-elysium that produced Wave 8: Nix-mirror retirement (januskey…laniakea) #113 and docs(licence-policy): A6 hard-exclusions + A7 multi-SPDX FP ignore-list (ledger #2/#3) #143 — the same opaque issue twice, both already remediated when they were read (the second was opened by the audit that found RUSTSEC-2026-0316 in Wasmtime 36.0.15, which was fixed by the 36.0.16 update).
  4. Dedupe by label, not identity. .[0].number picks whichever issue the API returns first, so an unrelated open security issue suppresses the creation of a real one, and the marker of "which audit produced this" is lost.

Suggested shape (implemented and verified in one consumer)

hyperpolymath/rpa-elysium now runs both audits blocking (--deny warnings and --deny unmaintained, unchanged), tees their output, and hands it to a reconciler script:

  • a stable marker in the body identifies the issue (not the label list);
  • a failing run creates one issue containing the audit output, the run URL and acceptance criteria, or comments the new findings onto the existing one — so the history of findings survives;
  • a passing run comments the green run and closes the marked issue, i.e. the gate that opened it closes it, with the run as evidence;
  • an issue created by this older publisher (same title, same labels, no marker) is adopted rather than duplicated;
  • pull requests never touch issues.

PR: hyperpolymath/rpa-elysium#148 (scripts/reconcile-audit-issue.sh plus its 10 fail-closed controls; the pattern is repo-local and lifts into the template unchanged).

Proposal

Change the templated job (and its consumers) to:

  1. capture the audit output instead of discarding it, and put it in the issue;
  2. add acceptance criteria that name what closing means ("exits 0 on a run of the default branch");
  3. close the issue from the same workflow when the audit passes, citing the run;
  4. identify the issue by a marker rather than by label ordering;
  5. keep the audit steps blocking — the proposal changes the issue lifecycle, not the gate.

If the estate prefers not to carry a reconciler per repo, the same logic belongs in a reusable workflow called by cargo-audit.yml; the decision is the owner's, the failure mode is measured.

Found while closing out hyperpolymath/rpa-elysium#143 and hyperpolymath/rpa-elysium#138.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions