Skip to content

fix(#2002): mock HTTP server in TestResolveLinuxBinary_Download - #2007

Closed
fullsend-ai-coder[bot] wants to merge 1 commit into
mainfrom
agent/2002-mock-download-test
Closed

fix(#2002): mock HTTP server in TestResolveLinuxBinary_Download#2007
fullsend-ai-coder[bot] wants to merge 1 commit into
mainfrom
agent/2002-mock-download-test

Conversation

@fullsend-ai-coder

Copy link
Copy Markdown
Contributor

Replace live GitHub CDN calls in TestResolveLinuxBinary_Download with an httptest.Server serving mock checksums.txt and tarball data. This eliminates the external network dependency that caused merge queue ejections during GitHub CDN outages (e.g., the June 8 incident affecting PRs #1238, #1239, #1468, #1531, #1905).

The refactored test follows the same pattern already used by TestDownloadReleaseBinary_ChecksumMatch and TestDownloadReleaseBinary_ChecksumMismatch in the same file: build a tar.gz with mock content, compute its SHA256, serve both via httptest, and override the package-level releaseBaseURL var.

The ELF validation assertion was removed since the mock binary is not a real ELF — validateLinuxBinary is independently tested by TestValidateLinuxBinary_* tests elsewhere in the file.

Note: Go tests could not run in the sandbox (requires Go 1.26.0, sandbox has Go 1.24.13). The change is syntactically valid (gofmt passes) and follows the exact pattern of adjacent passing tests. Manual verification of go test is required.


Closes #2002

Post-script verification

  • Branch is not main/master (agent/2002-mock-download-test)
  • Secret scan passed (gitleaks — be348d5073262d3ada97382950d093c71441dcf1..HEAD)
  • Pre-commit hooks passed (authoritative run on runner)
  • Tests ran inside sandbox

Replace live GitHub CDN calls in TestResolveLinuxBinary_Download with
an httptest.Server serving mock checksums.txt and tarball data. This
eliminates the external network dependency that caused merge queue
ejections during GitHub CDN outages (e.g., the June 8 incident
affecting PRs #1238, #1239, #1468, #1531, #1905).

The refactored test follows the same pattern already used by
TestDownloadReleaseBinary_ChecksumMatch and
TestDownloadReleaseBinary_ChecksumMismatch in the same file:
build a tar.gz with mock content, compute its SHA256, serve both
via httptest, and override the package-level releaseBaseURL var.

The ELF validation assertion was removed since the mock binary is
not a real ELF — validateLinuxBinary is independently tested by
TestValidateLinuxBinary_* tests elsewhere in the file.

Note: Go tests could not run in the sandbox (requires Go 1.26.0,
sandbox has Go 1.24.13). The change is syntactically valid (gofmt
passes) and follows the exact pattern of adjacent passing tests.
Manual verification of go test is required.

Closes #2002
@github-actions

github-actions Bot commented Jun 8, 2026

Copy link
Copy Markdown

Site preview

Preview: https://9289b6a5-site.fullsend-ai.workers.dev

Commit: 0268e059b7d78697bd45e360a9a89b71533de138

@fullsend-ai-review

Copy link
Copy Markdown

Review

Findings

Low

  • [test-coverage-regression] internal/cli/run_test.go:617 — The removal of the validateLinuxBinary assertion reduces integration-level validation that downloadReleaseBinary produces a valid ELF. This is a necessary trade-off for the mock strategy, since the mock binary is not a real ELF, and validateLinuxBinary is independently tested by TestValidateLinuxBinary_* tests (lines 454–475). Consider adding a brief comment explaining why validateLinuxBinary is not called here.

Info

  • [scope-alignment] internal/cli/run_test.go:566 — Change aligns with issue TestResolveLinuxBinary_Download should not make live HTTP calls to GitHub CDN #2002 authorization. The mock server pattern follows established codebase conventions used by TestDownloadReleaseBinary_ChecksumMatch, TestRefreshOIDCToken_*, and other tests in the same file.

  • [test-consistency] internal/cli/run_test.go:590 — The releaseBaseURL save/restore pattern and httptest.NewServer usage are consistent with five other tests in this file that mock HTTP endpoints.

Comment thread internal/cli/run_test.go
data, err := os.ReadFile(binPath)
require.NoError(t, err)
assert.Equal(t, "mock fullsend binary", string(data))
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[low] test-coverage-regression

The removal of the validateLinuxBinary assertion reduces integration-level validation that downloadReleaseBinary produces a valid ELF. This is a necessary trade-off for the mock strategy, since the mock binary is not a real ELF, and validateLinuxBinary is independently tested by TestValidateLinuxBinary_* tests (lines 454-475).

Suggested fix: Consider adding a comment explaining why validateLinuxBinary is not called (e.g., '// Mock binary is not a real ELF; validateLinuxBinary is tested separately in TestValidateLinuxBinary_*').

@fullsend-ai-review fullsend-ai-review Bot added the ready-for-merge All reviewers approved — ready to merge label Jun 8, 2026
@rh-hemartin rh-hemartin self-assigned this Jun 15, 2026
@rh-hemartin

Copy link
Copy Markdown
Member

Closing as superseded. The functions this PR tests (extractFullsendFromTarGz, downloadReleaseBinary, releaseBaseURL) were moved to internal/binary during the refactor in PRs #2015/#2053. The equivalent tests (TestExtractFullsendFromTarGz_PathTraversal, TestExtractFullsendFromTarGz_ValidEntry, TestDownloadReleaseBinary_ChecksumMatch) already exist in internal/binary/download_test.go. The rebase cannot be cleanly applied without duplicating that coverage.

@rh-hemartin
rh-hemartin deleted the agent/2002-mock-download-test branch June 15, 2026 06:45
@fullsend-ai-retro

fullsend-ai-retro Bot commented Jun 15, 2026

Copy link
Copy Markdown

🤖 Finished Retro · ✅ Success · Started 6:49 AM UTC · Completed 7:01 AM UTC
Commit: 0268e05 · View workflow run →

@fullsend-ai-retro

Copy link
Copy Markdown

Retro: PR #2007 — mock HTTP server in TestResolveLinuxBinary_Download

Overall: The fullsend pipeline worked efficiently end-to-end — retro agent filed issue #2002, triage labeled it ready-to-code within 5 minutes, the code agent produced a clean PR in 7 minutes, and the review agent approved with only a low-severity suggestion. The code change itself (replacing live CDN calls with httptest.Server) was well-crafted and followed existing codebase patterns.

What went wrong: The PR sat with ready-for-merge for a week without being merged. During that time, PRs #2015/#2053 refactored the target functions (downloadReleaseBinary, releaseBaseURL) into internal/binary/, making PR #2007 unmergeable. On June 15, a fix agent ran and failed (wasted tokens), a review agent dispatched after the human had already closed the PR, and this retro was triggered on a closed-without-merge PR. Issue #2002 remains open despite the human confirming equivalent tests already exist.

Existing issues covering most findings:

  • #2004 — Proactive staleness alert for ready-for-merge PRs (covers the week-long stale period)
  • #2176 — Skip retro dispatch for closed-without-merge PRs with immediate successor
  • #1439 / #1870 — Skip review/fix dispatch when PR is already closed
  • #491 / #1403 — Sandbox toolchain version discovery (code agent couldn't run go test due to Go version mismatch)

One novel proposal filed below for the orphaned-issue lifecycle gap not covered by existing issues.

Proposals filed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-merge All reviewers approved — ready to merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TestResolveLinuxBinary_Download should not make live HTTP calls to GitHub CDN

1 participant