CI: trivy-scan gate is red on every PR (pre-existing dep CVEs since ~2026-08-04)

[pdettori]

Summary

The trivy-scan job (workflow .github/workflows/security-scans.yml, "Security Scans") fails on effectively every pull request and has done so since at least 2026-08-04. Because it surfaces as a red required check, it blocks normal PR merges (each merge needs an admin override) and drowns out real, PR-specific signal.

This is not caused by any individual PR — it is a repo-wide baseline failure of the vulnerability gate.

Evidence

• The "Security Scans" workflow triggers only on pull_request → [main], so it never runs on push to main (the main run list looks empty, which hides the problem).
• Across recent PRs the job is 8/8 failures back to 2026-08-04, on unrelated branches (multi-protocol model provider, demo doc fix, dependabot, context-service-provider, workload-routing, …).
• Most recently observed on fix(ci): make pi codegen hermetic to unblock the check job (Closes #160) #161, where every other check is green and only trivy-scan is red.

Root cause

The step runs:

trivy fs .   # severity: CRITICAL,HIGH   exit-code: 1

Trivy scans the repo's own dependency manifests (Go modules + pnpm — "Number of language-specific files num=3"), finds CRITICAL/HIGH vulnerabilities, and exits 1. It is a vulnerability-gate failure, not an infra/tooling error. No .trivyignore or severity relaxation is in place, so any HIGH/CRITICAL advisory in a transitive dep turns the gate red until remediated.

Note: actions/checkout in this workflow does not set submodules, so the pi-fork submodule is not scanned — the findings are entirely in the harness's own gomod/pnpm dependencies.

Impact

• Required check is permanently red → PRs cannot merge without --admin / branch-protection override.
• Real security regressions introduced by a specific PR are indistinguishable from the standing baseline noise.

Proposed remediation (pick one, or stage them)

1. Remediate the deps — bump the Go modules / pnpm packages flagged by trivy fs . until the CRITICAL/HIGH set is empty. (Run trivy fs . --severity CRITICAL,HIGH locally to get the current list.)
2. Track + suppress with expiry — add a .trivyignore listing each accepted CVE ID with a justification and a review/expiry date, so the gate goes green but accepted risk stays visible and time-boxed.
3. Decouple blocking from reporting — keep a non-blocking (exit-code: 0) informational trivy run that uploads SARIF, and add a separate blocking gate that fails only on newly introduced vulns (diff against main), so a PR is judged on what it changes rather than the standing baseline.

Acceptance

• trivy-scan passes on a clean PR against main.
• Any remaining accepted CRITICAL/HIGH findings are explicitly tracked (option 2) rather than silently overridden.

---

Filed while landing #161 (pi hermetic-build fix). Assisted-By: Claude Code.