Skip to content

fix(#123): pin harness images to immutable sha256 digests - #658

Merged
rh-hemartin merged 1 commit into
mainfrom
agent/123-pin-image-digests
Aug 6, 2026
Merged

fix(#123): pin harness images to immutable sha256 digests#658
rh-hemartin merged 1 commit into
mainfrom
agent/123-pin-image-digests

Conversation

@fullsend-ai-coder

Copy link
Copy Markdown
Contributor

Summary

  • Pin all 7 harness image: references from mutable :latest tags to immutable @sha256: digest references
  • fullsend-code (code, fix, review) → @sha256:de3ecbd7719a1927c983142ada96475f3314d2505d0f258bcf19c31411856eb6
  • fullsend-sandbox (prioritize, retro, scribe, triage) → @sha256:eaf365ad038d762954d8fd5f41618cd6766a8a6a6f76d944aaf160819666734d

Motivation

Mutable :latest tags mean a pinned agents commit does not guarantee a stable execution environment — retagging :latest to a bad build previously caused an org-wide outage (fullsend#3408). Digest-pinned references make the image immutable so the runtime matches what was tested at pin time.

Testing

  • make test passes (one pre-existing failure in post-triage-test.sh unrelated to this change)
  • Secret scan clean
  • Pre-commit could not run in sandbox (network restrictions); the post-script runs it authoritatively on the runner

Closes #123

Post-script verification

  • Branch is not main/master (agent/123-pin-image-digests)
  • Secret scan passed (gitleaks — 5e98b13bc52cefb7b95cbcfe6070ef9da315574d..HEAD)
  • PR body secret scan passed (gitleaks — no-git)

All 7 harness configs referenced sandbox images with mutable
:latest tags. A bad build promoted to :latest previously broke
agent runs org-wide (fullsend#3408). Replace :latest with
@sha256: digest references so a pinned agents commit resolves
the same execution environment until intentionally bumped.

Changed images:
- fullsend-code (code, fix, review):
  @sha256:de3ecbd7719a1927c983142ada96475f3314d2505d0f258bcf
  19c31411856eb6
- fullsend-sandbox (prioritize, retro, scribe, triage):
  @sha256:eaf365ad038d762954d8fd5f41618cd6766a8a6a6f76d944aa
  f160819666734d

Note: pre-commit could not run (sandbox network restrictions
prevented hook environment initialization). The post-script
runs an authoritative pre-commit on the runner.

Note: post-triage-test prerequisites-creates-allowed-issue is
a pre-existing failure on main, unrelated to this change.

Closes #123
@fullsend-ai-coder
fullsend-ai-coder Bot requested a review from a team as a code owner August 4, 2026 20:21
@fullsend-ai-coder fullsend-ai-coder Bot added the ready-for-review Triggers review agent dispatch label Aug 4, 2026
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 4, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 8:22 PM UTC · Completed 8:34 PM UTC
Commit: e96a3b7 · View workflow run →

@fullsend-ai-review

Copy link
Copy Markdown

Review

Findings

Medium

  • [protected-path] harness/ — All 7 changed files (harness/code.yaml, harness/fix.yaml, harness/prioritize.yaml, harness/retro.yaml, harness/review.yaml, harness/scribe.yaml, harness/triage.yaml) are under the protected path harness/. The PR links to issue Harness sandbox images use mutable :latest tags #123 and explains the rationale (pinning mutable :latest tags to immutable @sha256: digests for runtime stability). Human approval is always required for protected-path changes, regardless of context.

Low

  • [stale-doc] docs/code.md:63 — Documentation at lines 63, 85, and 97 references ghcr.io/fullsend-ai/fullsend-code:latest as the base image name. While the harness YAML files now pin to @sha256: digests, the documentation describes the Docker registry image (which still publishes a :latest tag) and instructions for building custom images via FROM. The documentation is not technically wrong but a brief note about digest pinning could improve clarity.
    Remediation: Consider adding a note in the "Custom sandbox image" section explaining that the upstream harness pins images by digest for reproducibility, while user-facing examples use :latest for simplicity.

Labels: PR pins container images from mutable :latest tags to immutable sha256 digests, a supply chain security improvement affecting all agent harnesses

@fullsend-ai-review fullsend-ai-review Bot added requires-manual-review Review requires human judgment security labels Aug 4, 2026
@rh-hemartin
rh-hemartin added this pull request to the merge queue Aug 6, 2026
Merged via the queue into main with commit 91f61f3 Aug 6, 2026
27 of 31 checks passed
@rh-hemartin
rh-hemartin deleted the agent/123-pin-image-digests branch August 6, 2026 10:21
@fullsend-ai-retro

fullsend-ai-retro Bot commented Aug 6, 2026

Copy link
Copy Markdown

🤖 Finished Retro · ✅ Success · Started 10:23 AM UTC · Completed 10:43 AM UTC
Commit: e96a3b7 · View workflow run →

@fullsend-ai-retro

Copy link
Copy Markdown

Retro: PR #658 — Pin harness images to immutable sha256 digests

Overall assessment: This was a clean, well-executed workflow. The code agent succeeded on its first attempt with a surgical 7-line change, the review agent appropriately flagged protected-path changes for human judgment, and the human reviewer approved without finding additional issues. No rework was needed — zero fix-agent iterations.

Timeline

Time Event
Jul 10 Issue #123 created by ascerra — harness images use mutable :latest tags
Jul 10 Triage agent ran (run 29119853800), labeled ready-to-code
Aug 1 Prioritize agent scored RICE 10.8
Aug 4 20:09 ascerra re-triggered triage (/fs-triage) — no code agent had run in 25 days
Aug 4 20:14 Code agent ran (run 30946921791), 71 turns, $1.95, 4 min
Aug 4 20:21 PR #658 created
Aug 4 20:22 Review agent ran (run 30947486520), 5 sub-agents + challenger, $2.25, 11 min
Aug 4 20:34 Review posted: comment-only — protected-path (medium), stale-doc (low)
Aug 5 06:37 rh-hemartin approved
Aug 6 10:21 Merged via merge queue

Key findings

1. 25-day gap root cause (new proposal filed). Issue #123 was triaged and labeled ready-to-code on July 10, but no code agent ran until August 4 when a human re-triggered triage. Investigation revealed that PR #119 (for unrelated issue #107) was open July 10–13 and contained the string fix(#123) as a gitlint test example. The code dispatch guard searches for open PRs matching "${ISSUE_NUMBER} in:title,body" — a bare substring match that would match "123" anywhere in a PR body. This likely caused a false-positive block, silently skipping code dispatch for 25 days until ascerra manually intervened.

2. Review cost for mechanical changes (existing coverage). The review agent spent $2.25 and 11 minutes reviewing a 7-line mechanical digest bump. It dispatched 5 sub-agents (correctness, security, intent-coherence, style-conventions, docs-currency) plus a challenger. Three sub-agents produced zero findings. The scope constraint mechanism correctly classified this as "trivial" (≤5 tool calls per sub-agent), but the sub-agent count was not reduced. This provides additional evidence for existing issues: #301 (lightweight path for mechanical changes), #513 (short-circuit for mechanical dependency bumps), and #497 (downgrade sub-agent models based on scope classification).

3. Duplicate review dispatch (existing coverage). Two pull_request_target events fired at the same second, spawning two review runs (30947486520 and 30947489218). The duplicate was cancelled by the concurrency group — handled gracefully but wasteful. This is additional evidence for fullsend#4681 (deduplicate review agent runs on the same commit SHA).

4. Retro on trivial PR (existing coverage). This retro is itself running on a trivially-handled mechanical PR with zero rework. Additional evidence for #649 (skip retro on trivially-handled mechanical PRs).

What went well

  • Code agent execution was excellent. Resolved container digests from inside a sandboxed environment using multiple fallback approaches (docker manifest inspect, then GitHub API). Made exactly the right changes. Verified a pre-existing test failure by stashing changes and re-running on clean main — textbook diagnostic rigor.
  • Review correctly deferred to human. Protected-path finding forced comment-only verdict, ensuring human review for security-sensitive harness configs. The challenger appropriately downgraded the stale-doc finding from medium to low.
  • Human review was appropriate. rh-hemartin approved a well-scoped mechanical change that the review agent had already validated for correctness and security.

Autonomy assessment

No autonomy gap to close. The review agent correctly identified protected-path changes as requiring human approval, and the human approved without finding additional issues. For this class of change (mechanical updates to harness execution environment configs), the human gate is appropriate — harness configs are security-sensitive infrastructure that defines the agent execution environment.

Proposals filed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-review Triggers review agent dispatch requires-manual-review Review requires human judgment security

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Harness sandbox images use mutable :latest tags

2 participants