diff --git a/docs/guides/dev/cli-internals.md b/docs/guides/dev/cli-internals.md index a8580e0ef6..c964086fc8 100644 --- a/docs/guides/dev/cli-internals.md +++ b/docs/guides/dev/cli-internals.md @@ -235,7 +235,19 @@ Install: process 1→7 (forward) Uninstall: process 7→1 (reverse) ``` -Per-repo mode does not use the layer stack — it runs the same phases inline in `runPerRepoInstall()` since there's no need for composable uninstall ordering with a single repo. Binary vendoring (when `--vendor-fullsend-binary` is set) and stale binary cleanup are handled inline rather than through `VendorBinaryLayer`. +Per-repo mode does not use the layer stack — it runs the same phases inline in `runPerRepoInstall()` and `runGitHubSetupPerRepo()` since there's no need for composable uninstall ordering with a single repo. Binary vendoring (when `--vendor-fullsend-binary` is set) and stale binary cleanup are handled inline or via shared helpers; per-org mode uses `VendorBinaryLayer`. + +### Binary acquisition (`internal/binary`) + +Linux binary resolution for `fullsend run` and vendoring lives in `internal/binary`: + +| Function | Policy | +|----------|--------| +| `ResolveForRun` | Release download (released CLI only) → cross-compile → latest release | +| `ResolveForVendor` | Cross-compile → matching release (released CLI only) → fail (no latest) | +| `ResolveExplicit` | Validate linux/{arch} ELF for `--fullsend-binary` | + +Vendoring commit messages use title + body (upload and stale delete). `admin analyze` reports stale vendored binaries at `bin/fullsend` or `.fullsend/bin/fullsend` without install-intent flags. --- diff --git a/docs/guides/getting-started/github-setup.md b/docs/guides/getting-started/github-setup.md index 7163e80aad..158cabc4a6 100644 --- a/docs/guides/getting-started/github-setup.md +++ b/docs/guides/getting-started/github-setup.md @@ -118,9 +118,16 @@ fullsend github setup acme-corp \ | `--app-set` | No | `fullsend-ai` | App set name prefix for GitHub Apps | | `--enroll-all` | No | `false` | Enroll all repositories without prompting (per-org only) | | `--enroll-none` | No | `false` | Skip enrollment without prompting (per-org only) | -| `--vendor-fullsend-binary` | No | `false` | Build and upload the fullsend binary to the config repo for local dev testing (e.g., macOS with a Podman Linux VM) | +| `--vendor-fullsend-binary` | No | `false` | Resolve and upload a linux/amd64 fullsend binary for CI (see [Vendoring the CLI binary](#vendoring-the-cli-binary)) | +| `--fullsend-binary` | No | | Path to a Linux fullsend binary when vendoring (skips auto-resolution) | | `--dry-run` | No | `false` | Preview changes without making them | +### Vendoring the CLI binary + +Same policy as [admin install](installation.md#vendoring-the-cli-binary): `--fullsend-binary` → checkout cross-compile → matching release (released CLI only) → fail. Per-repo setup now wires vendoring and stale-binary cleanup when the flag is off. + +`fullsend admin analyze ` reports when a stale vendored binary is present (no install-intent flags on analyze). + ## Per-repo setup Per-repo mode bootstraps a single repository with a `.fullsend/` directory, shim workflow, and repo-level secrets: diff --git a/docs/guides/getting-started/installation.md b/docs/guides/getting-started/installation.md index d2b7671dc9..35e0aa6015 100644 --- a/docs/guides/getting-started/installation.md +++ b/docs/guides/getting-started/installation.md @@ -256,7 +256,8 @@ The installer automatically provisions [Workload Identity Federation (WIF)](http | `--skip-mint-check` | `false` | Skip mint validation, GCP provisioning, and app setup; requires `--mint-url` | | `--enroll-all` | `false` | Enroll all repositories without prompting (per-org only) | | `--enroll-none` | `false` | Skip repository enrollment without prompting (per-org only) | -| `--vendor-fullsend-binary` | `false` | Cross-compile and vendor the fullsend binary for development iteration | +| `--vendor-fullsend-binary` | `false` | Resolve and upload a linux/amd64 fullsend binary for CI (see [Vendoring the CLI binary](#vendoring-the-cli-binary)) | +| `--fullsend-binary` | | Path to a Linux fullsend binary to upload when `--vendor-fullsend-binary` is set (skips auto-resolution) | The `--skip-mint-check` flag bypasses all mint validation, GCP provisioning, and app setup. It requires `--mint-url` to be set and only validates that the URL uses HTTPS. This is useful when the mint infrastructure is managed externally or you want to skip GCP API calls entirely. @@ -266,6 +267,25 @@ The installer automatically detects when the deployed mint function is up-to-dat A single token mint can serve multiple GitHub organizations. See [Mint service administration — Multi-org setup](../infrastructure/mint-administration.md#multi-org-setup) for the complete multi-org workflow. +### Vendoring the CLI binary + +Use `--vendor-fullsend-binary` to upload a linux/amd64 `fullsend` binary into the config repo (`bin/fullsend`) or per-repo path (`.fullsend/bin/fullsend`). CI workflows prefer this file over downloading from GitHub releases. + +When the flag is set, the binary is resolved in this order: + +1. **`--fullsend-binary `** — upload that file (validated as linux/amd64 ELF) +2. **Checkout build** — cross-compile from the fullsend module root (`go env GOMOD`), stamped `{version}-vendored` +3. **Release fetch** — only if step 2 is unavailable **and** the running CLI is a released version (e.g. `0.4.0`); downloads the matching GitHub release (no `-vendored` suffix) +4. **Fail** — dev CLI outside a checkout fails with a clear error (no “latest release” fallback) + +When the flag is **off**, any existing vendored binary is removed so CI uses released versions. + +**Notes:** + +- Vendoring the CLI alone does not air-gap the full pipeline (OpenShell, gateway, sandbox image, upstream scaffold still download at runtime). +- Release fallback requires network access at install time; CI consumes the uploaded file. +- Works from any directory inside the module checkout (module root discovery via `GOMOD`). + ### Merge enrollment PRs If you chose to enroll repositories during install, the installer dispatches a workflow that creates an enrollment PR in each enrolled repo. These PRs add a shim workflow (`.github/workflows/fullsend.yaml`) that wires events to the agent pipeline. diff --git a/e2e/admin/admin_test.go b/e2e/admin/admin_test.go index 6022346c65..948832d44d 100644 --- a/e2e/admin/admin_test.go +++ b/e2e/admin/admin_test.go @@ -24,6 +24,7 @@ import ( "github.com/fullsend-ai/fullsend/internal/config" "github.com/fullsend-ai/fullsend/internal/forge" gh "github.com/fullsend-ai/fullsend/internal/forge/github" + "github.com/fullsend-ai/fullsend/internal/layers" ) // e2eEnv holds the shared state for an e2e test run. @@ -651,3 +652,33 @@ func runUnenrollmentTest(t *testing.T, env *e2eEnv) { require.True(t, forge.IsNotFound(err), "shim should be removed from %s after unenrollment", testRepo) t.Log("Verified shim is gone") } + +// TestVendorFromSubdirectory verifies that --vendor-fullsend-binary cross-compiles +// when the CLI is run from a subdirectory inside the module (GOMOD discovery). +func TestVendorFromSubdirectory(t *testing.T) { + env := setupE2ETest(t) + ctx := context.Background() + + subdir := filepath.Join(moduleRoot(t), "internal", "cli") + installArgs := []string{ + "admin", "install", env.org, + "--skip-app-setup", + "--skip-mint-check", + "--mint-url", env.cfg.mintURL, + "--app-set", e2eAppSet, + "--enroll-none", + "--vendor-fullsend-binary", + } + runCLIFromDir(t, env.binary, env.token, subdir, installArgs...) + + _, err := env.client.GetFileContent(ctx, env.org, forge.ConfigRepoName, layers.VendoredBinaryPath) + require.NoError(t, err, "vendored binary should exist at %s", layers.VendoredBinaryPath) + + registerRepoCleanup(t, env.client, env.org, forge.ConfigRepoName) + + runCLI(t, env.binary, env.token, + "admin", "uninstall", env.org, + "--yolo", + "--app-set", e2eAppSet, + ) +} diff --git a/e2e/admin/testutil.go b/e2e/admin/testutil.go index d4e3bdbf8e..b19d46330b 100644 --- a/e2e/admin/testutil.go +++ b/e2e/admin/testutil.go @@ -260,19 +260,19 @@ func buildCLIBinary(t *testing.T) string { } // runCLI executes the fullsend CLI with the given args, passing GITHUB_TOKEN. -// The working directory is set to the module root so that --vendor-fullsend-binary -// can find ./cmd/fullsend/ (same as a user running from the repo root). +// By default the working directory is the module root. Use runCLIFromDir to +// run from a subdirectory (GOMOD discovery makes this work for vendoring). func runCLI(t *testing.T, binary, token string, args ...string) string { - t.Helper() - t.Logf("[cli] fullsend %s", strings.Join(args, " ")) + return runCLIFromDir(t, binary, token, moduleRoot(t), args...) +} - modRoot, err := exec.Command("go", "list", "-m", "-f", "{{.Dir}}").Output() - if err != nil { - t.Fatalf("finding module root for runCLI: %v", err) - } +// runCLIFromDir runs the CLI with cwd set to dir. +func runCLIFromDir(t *testing.T, binary, token, dir string, args ...string) string { + t.Helper() + t.Logf("[cli] fullsend %s (cwd=%s)", strings.Join(args, " "), dir) cmd := exec.Command(binary, args...) - cmd.Dir = strings.TrimSpace(string(modRoot)) + cmd.Dir = dir cmd.Env = append(os.Environ(), "GITHUB_TOKEN="+token, "CI=true") out, runErr := cmd.CombinedOutput() output := string(out) @@ -283,6 +283,15 @@ func runCLI(t *testing.T, binary, token string, args ...string) string { return output } +func moduleRoot(t *testing.T) string { + t.Helper() + modRoot, err := exec.Command("go", "list", "-m", "-f", "{{.Dir}}").Output() + if err != nil { + t.Fatalf("finding module root: %v", err) + } + return strings.TrimSpace(string(modRoot)) +} + // retryOnNotFound retries an operation up to maxAttempts times with linear // backoff when it returns a not-found error (GitHub eventual consistency). func retryOnNotFound(ctx context.Context, maxAttempts int, fn func() error) error { diff --git a/internal/binary/acquire.go b/internal/binary/acquire.go new file mode 100644 index 0000000000..0f7e70d9ad --- /dev/null +++ b/internal/binary/acquire.go @@ -0,0 +1,115 @@ +package binary + +import ( + "fmt" + "os" + "path/filepath" +) + +// Source identifies how a Linux fullsend binary was obtained. +type Source int + +const ( + SourceExplicitPath Source = iota + SourceCheckoutBuild + SourceReleaseDownload +) + +// AcquireResult holds the path to an acquired binary and metadata for callers. +type AcquireResult struct { + TmpDir string // caller must RemoveAll when non-empty + Path string + Source Source +} + +// ResolveExplicit validates that path is a Linux ELF for arch. +func ResolveExplicit(path, arch string) error { + return ValidateLinuxBinary(path, arch) +} + +// ResolveForRun obtains a Linux binary using the run policy: +// release download (if released) → cross-compile → latest release. +func ResolveForRun(version, arch string) (AcquireResult, error) { + tmpDir, err := os.MkdirTemp("", "fullsend-linux-*") + if err != nil { + return AcquireResult{}, fmt.Errorf("creating temp dir: %w", err) + } + binaryPath := filepath.Join(tmpDir, "fullsend") + + // 1. Released version → download matching release asset. + if IsReleasedVersion(version) { + fmt.Fprintf(os.Stderr, "Downloading fullsend %s for linux/%s from GitHub Release...\n", version, arch) + if dlErr := DownloadRelease(version, arch, binaryPath); dlErr == nil { + fmt.Fprintf(os.Stderr, "Downloaded fullsend for linux/%s\n", arch) + return AcquireResult{TmpDir: tmpDir, Path: binaryPath, Source: SourceReleaseDownload}, nil + } else { + fmt.Fprintf(os.Stderr, "WARNING: release download failed: %v\n", dlErr) + } + } + + // 2. Try cross-compilation (requires Go toolchain + module checkout). + fmt.Fprintf(os.Stderr, "Cross-compiling fullsend for linux/%s...\n", arch) + if ccErr := CrossCompile(CrossCompileOpts{ + Version: version, + Arch: arch, + DestPath: binaryPath, + VersionStamp: "-crosscompiled", + }); ccErr == nil { + fmt.Fprintf(os.Stderr, "Cross-compiled fullsend for linux/%s\n", arch) + return AcquireResult{TmpDir: tmpDir, Path: binaryPath, Source: SourceCheckoutBuild}, nil + } else { + fmt.Fprintf(os.Stderr, "WARNING: cross-compilation failed: %v\n", ccErr) + } + + // 3. Last resort → download latest release. + fmt.Fprintf(os.Stderr, "Downloading latest fullsend release for linux/%s...\n", arch) + latestErr := DownloadLatestRelease(arch, binaryPath) + if latestErr == nil { + fmt.Fprintf(os.Stderr, "Downloaded latest fullsend for linux/%s\n", arch) + return AcquireResult{TmpDir: tmpDir, Path: binaryPath, Source: SourceReleaseDownload}, nil + } + fmt.Fprintf(os.Stderr, "WARNING: latest release download failed: %v\n", latestErr) + + os.RemoveAll(tmpDir) + return AcquireResult{}, fmt.Errorf("all strategies failed for linux/%s: provide --fullsend-binary or install Go toolchain", arch) +} + +// ResolveForVendor obtains a Linux binary using the vendoring policy: +// cross-compile from checkout → matching release (released CLI only) → fail. +// No latest-release fallback. +func ResolveForVendor(version, arch string) (AcquireResult, error) { + tmpDir, err := os.MkdirTemp("", "fullsend-linux-*") + if err != nil { + return AcquireResult{}, fmt.Errorf("creating temp dir: %w", err) + } + binaryPath := filepath.Join(tmpDir, "fullsend") + + // 1. Cross-compile from checkout. + fmt.Fprintf(os.Stderr, "Cross-compiling fullsend for linux/%s...\n", arch) + if ccErr := CrossCompile(CrossCompileOpts{ + Version: version, + Arch: arch, + DestPath: binaryPath, + VersionStamp: "-vendored", + }); ccErr == nil { + fmt.Fprintf(os.Stderr, "Cross-compiled fullsend for linux/%s\n", arch) + return AcquireResult{TmpDir: tmpDir, Path: binaryPath, Source: SourceCheckoutBuild}, nil + } else { + fmt.Fprintf(os.Stderr, "WARNING: cross-compilation failed: %v\n", ccErr) + } + + // 2. Release fetch only for released CLI versions. + if IsReleasedVersion(version) { + fmt.Fprintf(os.Stderr, "Downloading fullsend %s for linux/%s from GitHub Release...\n", version, arch) + if dlErr := DownloadRelease(version, arch, binaryPath); dlErr == nil { + fmt.Fprintf(os.Stderr, "Downloaded fullsend for linux/%s\n", arch) + return AcquireResult{TmpDir: tmpDir, Path: binaryPath, Source: SourceReleaseDownload}, nil + } else { + os.RemoveAll(tmpDir) + return AcquireResult{}, fmt.Errorf("cross-compilation unavailable and release download failed for v%s: %w", version, dlErr) + } + } + + os.RemoveAll(tmpDir) + return AcquireResult{}, fmt.Errorf("cannot vendor binary: not in fullsend source tree and CLI version %s is a dev build — use --fullsend-binary, run from a checkout, or use a released CLI", version) +} diff --git a/internal/binary/crosscompile.go b/internal/binary/crosscompile.go new file mode 100644 index 0000000000..d71b0407ae --- /dev/null +++ b/internal/binary/crosscompile.go @@ -0,0 +1,64 @@ +package binary + +import ( + "fmt" + "os" + "os/exec" + "path/filepath" + "strings" +) + +// CrossCompileOpts configures a cross-compilation build. +type CrossCompileOpts struct { + Version string // CLI version to embed (before stamp suffix) + Arch string + DestPath string + VersionStamp string // e.g. "-vendored", "-crosscompiled", or "" +} + +// ModuleRoot returns the fullsend module root directory, or an error if not +// inside a Go module checkout. +func ModuleRoot() (string, error) { + goPath, lookErr := exec.LookPath("go") + if lookErr != nil { + return "", fmt.Errorf("Go toolchain not found: %w", lookErr) + } + modRootCmd := exec.Command(goPath, "env", "GOMOD") + modOutput, err := modRootCmd.Output() + if err != nil { + return "", fmt.Errorf("finding module root: %w", err) + } + modPath := strings.TrimSpace(string(modOutput)) + if modPath == "" || modPath == os.DevNull { + return "", fmt.Errorf("not in a Go module") + } + return filepath.Dir(modPath), nil +} + +// CrossCompile builds a Linux fullsend binary and writes it to DestPath. +// Requires the Go toolchain and a fullsend module checkout (go env GOMOD). +func CrossCompile(opts CrossCompileOpts) error { + goPath, lookErr := exec.LookPath("go") + if lookErr != nil { + return fmt.Errorf("Go toolchain not found — install Go or use a released version of fullsend: %w", lookErr) + } + + modRoot, err := ModuleRoot() + if err != nil { + return fmt.Errorf("not in a Go module — run from the fullsend source tree or use a released version: %w", err) + } + + versionLD := opts.Version + opts.VersionStamp + buildCmd := exec.Command(goPath, "build", + "-ldflags", fmt.Sprintf("-X github.com/fullsend-ai/fullsend/internal/cli.version=%s", versionLD), + "-o", opts.DestPath, + "./cmd/fullsend/", + ) + buildCmd.Dir = modRoot + buildCmd.Env = append(os.Environ(), "GOTOOLCHAIN=auto", "GOOS=linux", "GOARCH="+opts.Arch, "CGO_ENABLED=0") + buildCmd.Stderr = os.Stderr + if err := buildCmd.Run(); err != nil { + return fmt.Errorf("cross-compiling for linux/%s: %w", opts.Arch, err) + } + return nil +} diff --git a/internal/binary/download.go b/internal/binary/download.go new file mode 100644 index 0000000000..8714a34555 --- /dev/null +++ b/internal/binary/download.go @@ -0,0 +1,184 @@ +package binary + +import ( + "archive/tar" + "bufio" + "bytes" + "compress/gzip" + "crypto/sha256" + "encoding/hex" + "encoding/json" + "fmt" + "io" + "net/http" + "os" + "path/filepath" + "strings" + "time" +) + +// ReleaseBaseURL is the GitHub releases download base URL. Tests may override. +// Not safe for concurrent test mutation. +var ReleaseBaseURL = "https://github.com/fullsend-ai/fullsend/releases/download" + +// HTTPClient is used for release downloads. Tests may override. +// Not safe for concurrent test mutation. +var HTTPClient = &http.Client{Timeout: 120 * time.Second} + +const defaultMaxDownloadSize = 200 * 1024 * 1024 // 200 MB compressed + +// maxDownloadSize caps release asset downloads. Tests may lower temporarily. +var maxDownloadSize = defaultMaxDownloadSize + +const maxBinarySize = 500 * 1024 * 1024 // 500 MB — reasonable upper bound for a Go binary + +// DownloadRelease downloads the fullsend binary for linux/{arch} from the +// GitHub Release matching the given version, verifies its SHA256 checksum +// against the release checksums.txt, and writes it to destPath. +func DownloadRelease(ver, arch, destPath string) error { + cleanVer := strings.TrimPrefix(ver, "v") + assetName := fmt.Sprintf("fullsend_%s_linux_%s.tar.gz", cleanVer, arch) + + expectedHash, err := downloadChecksumForAsset(ver, assetName) + if err != nil { + return fmt.Errorf("fetching checksum for %s: %w", assetName, err) + } + + url := fmt.Sprintf("%s/v%s/%s", ReleaseBaseURL, cleanVer, assetName) + resp, err := HTTPClient.Get(url) //nolint:gosec // URL is constructed from known constants + if err != nil { + return fmt.Errorf("fetching %s: %w", url, err) + } + defer resp.Body.Close() + + if resp.StatusCode != http.StatusOK { + return fmt.Errorf("GET %s returned %d", url, resp.StatusCode) + } + + maxSize := int64(maxDownloadSize) + var buf bytes.Buffer + if _, err := io.Copy(&buf, io.LimitReader(resp.Body, maxSize+1)); err != nil { + return fmt.Errorf("reading %s: %w", assetName, err) + } + if int64(buf.Len()) > maxSize { + return fmt.Errorf("download of %s exceeds maximum size (%d bytes)", assetName, maxSize) + } + + h := sha256.Sum256(buf.Bytes()) + actualHash := hex.EncodeToString(h[:]) + if actualHash != expectedHash { + return fmt.Errorf("checksum mismatch for %s: got %s, want %s", assetName, actualHash, expectedHash) + } + + return ExtractFullsendFromTarGz(bytes.NewReader(buf.Bytes()), destPath) +} + +func downloadChecksumForAsset(ver, assetName string) (string, error) { + cleanVer := strings.TrimPrefix(ver, "v") + url := fmt.Sprintf("%s/v%s/checksums.txt", ReleaseBaseURL, cleanVer) + + resp, err := HTTPClient.Get(url) //nolint:gosec // URL is constructed from known constants + if err != nil { + return "", fmt.Errorf("fetching checksums: %w", err) + } + defer resp.Body.Close() + + if resp.StatusCode != http.StatusOK { + return "", fmt.Errorf("GET %s returned %d", url, resp.StatusCode) + } + + scanner := bufio.NewScanner(io.LimitReader(resp.Body, 64*1024)) + for scanner.Scan() { + line := scanner.Text() + parts := strings.Fields(line) + if len(parts) == 2 && parts[1] == assetName { + hash := strings.ToLower(parts[0]) + if len(hash) != 64 { + return "", fmt.Errorf("invalid hash length for %s in checksums.txt", assetName) + } + if _, err := hex.DecodeString(hash); err != nil { + return "", fmt.Errorf("invalid hex hash for %s in checksums.txt: %w", assetName, err) + } + return hash, nil + } + } + if err := scanner.Err(); err != nil { + return "", fmt.Errorf("reading checksums: %w", err) + } + return "", fmt.Errorf("asset %s not found in checksums.txt", assetName) +} + +// DownloadLatestRelease resolves the latest release tag from the GitHub API +// and downloads the Linux binary for the given arch. +func DownloadLatestRelease(arch, destPath string) error { + tag, err := resolveLatestReleaseTag() + if err != nil { + return err + } + return DownloadRelease(tag, arch, destPath) +} + +func resolveLatestReleaseTag() (string, error) { + resp, err := HTTPClient.Get("https://api.github.com/repos/fullsend-ai/fullsend/releases/latest") //nolint:gosec + if err != nil { + return "", fmt.Errorf("fetching latest release: %w", err) + } + defer resp.Body.Close() + + if resp.StatusCode != http.StatusOK { + return "", fmt.Errorf("GitHub API returned %d", resp.StatusCode) + } + + var release struct { + TagName string `json:"tag_name"` + } + if err := json.NewDecoder(io.LimitReader(resp.Body, 1024*1024)).Decode(&release); err != nil { + return "", fmt.Errorf("parsing release JSON: %w", err) + } + if release.TagName == "" { + return "", fmt.Errorf("empty tag_name in latest release") + } + return release.TagName, nil +} + +// ExtractFullsendFromTarGz reads a tar.gz stream and extracts the "fullsend" +// binary to destPath. +func ExtractFullsendFromTarGz(r io.Reader, destPath string) error { + gz, err := gzip.NewReader(r) + if err != nil { + return fmt.Errorf("gzip reader: %w", err) + } + defer gz.Close() + + tr := tar.NewReader(gz) + for { + hdr, err := tr.Next() + if err == io.EOF { + return fmt.Errorf("fullsend binary not found in archive") + } + if err != nil { + return fmt.Errorf("reading tar: %w", err) + } + clean := filepath.Clean(hdr.Name) + if strings.Contains(clean, "..") || filepath.IsAbs(clean) { + continue + } + if filepath.Base(clean) == "fullsend" && hdr.Typeflag == tar.TypeReg { + f, err := os.OpenFile(destPath, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0o755) + if err != nil { + return fmt.Errorf("creating %s: %w", destPath, err) + } + n, copyErr := io.Copy(f, io.LimitReader(tr, maxBinarySize+1)) + if copyErr != nil { + f.Close() + return fmt.Errorf("extracting fullsend: %w", copyErr) + } + if n > maxBinarySize { + f.Close() + os.Remove(destPath) + return fmt.Errorf("binary exceeds maximum size (%d bytes)", maxBinarySize) + } + return f.Close() + } + } +} diff --git a/internal/binary/download_test.go b/internal/binary/download_test.go new file mode 100644 index 0000000000..23b20db993 --- /dev/null +++ b/internal/binary/download_test.go @@ -0,0 +1,580 @@ +package binary + +import ( + "archive/tar" + "bytes" + "compress/gzip" + "crypto/sha256" + "encoding/hex" + "fmt" + "io" + "net/http" + "net/http/httptest" + "os" + "path/filepath" + "runtime" + "strings" + "sync/atomic" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +type redirectTransport struct { + srvURL string + base http.RoundTripper +} + +func (t redirectTransport) RoundTrip(req *http.Request) (*http.Response, error) { + clone := req.Clone(req.Context()) + clone.URL.Scheme = "http" + clone.URL.Host = strings.TrimPrefix(strings.TrimPrefix(t.srvURL, "https://"), "http://") + if t.base == nil { + t.base = http.DefaultTransport + } + return t.base.RoundTrip(clone) +} + +func withTestReleaseServer(t *testing.T, srv *httptest.Server) { + t.Helper() + origClient := HTTPClient + origBaseURL := ReleaseBaseURL + HTTPClient = &http.Client{ + Transport: redirectTransport{srvURL: srv.URL}, + Timeout: 120 * time.Second, + } + ReleaseBaseURL = srv.URL + t.Cleanup(func() { + HTTPClient = origClient + ReleaseBaseURL = origBaseURL + }) +} + +func TestExtractFullsendFromTarGz_PathTraversal(t *testing.T) { + var buf bytes.Buffer + gw := gzip.NewWriter(&buf) + tw := tar.NewWriter(gw) + + content := []byte("malicious binary content") + require.NoError(t, tw.WriteHeader(&tar.Header{ + Name: "../../../tmp/fullsend", + Size: int64(len(content)), + Mode: 0o755, + Typeflag: tar.TypeReg, + })) + _, err := tw.Write(content) + require.NoError(t, err) + require.NoError(t, tw.Close()) + require.NoError(t, gw.Close()) + + destPath := filepath.Join(t.TempDir(), "fullsend") + err = ExtractFullsendFromTarGz(&buf, destPath) + assert.Error(t, err) + assert.Contains(t, err.Error(), "not found in archive") +} + +func TestExtractFullsendFromTarGz_ValidEntry(t *testing.T) { + var buf bytes.Buffer + gw := gzip.NewWriter(&buf) + tw := tar.NewWriter(gw) + + content := []byte("valid binary content") + require.NoError(t, tw.WriteHeader(&tar.Header{ + Name: "fullsend_0.4.0_linux_amd64/fullsend", + Size: int64(len(content)), + Mode: 0o755, + Typeflag: tar.TypeReg, + })) + _, err := tw.Write(content) + require.NoError(t, err) + require.NoError(t, tw.Close()) + require.NoError(t, gw.Close()) + + destPath := filepath.Join(t.TempDir(), "fullsend") + err = ExtractFullsendFromTarGz(&buf, destPath) + require.NoError(t, err) + + data, err := os.ReadFile(destPath) + require.NoError(t, err) + assert.Equal(t, "valid binary content", string(data)) +} + +func TestDownloadChecksumForAsset_ParsesLine(t *testing.T) { + body := "1b4f0e9851971998e732078544c96b36c3d01cedf7caa332359d6f1d83567014 fullsend_1.0.0_linux_arm64.tar.gz\n" + + "60303ae22b998861bce3b28f33eec1be758a213c86c93c076dbe9f558c11c752 fullsend_1.0.0_linux_amd64.tar.gz\n" + + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + fmt.Fprint(w, body) + })) + defer srv.Close() + + origBaseURL := ReleaseBaseURL + ReleaseBaseURL = srv.URL + defer func() { ReleaseBaseURL = origBaseURL }() + + hash, err := downloadChecksumForAsset("1.0.0", "fullsend_1.0.0_linux_amd64.tar.gz") + require.NoError(t, err) + assert.Equal(t, "60303ae22b998861bce3b28f33eec1be758a213c86c93c076dbe9f558c11c752", hash) +} + +func TestDownloadChecksumForAsset_AssetNotFound(t *testing.T) { + body := "60303ae22b998861bce3b28f33eec1be758a213c86c93c076dbe9f558c11c752 fullsend_1.0.0_linux_amd64.tar.gz\n" + + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + fmt.Fprint(w, body) + })) + defer srv.Close() + + origBaseURL := ReleaseBaseURL + ReleaseBaseURL = srv.URL + defer func() { ReleaseBaseURL = origBaseURL }() + + _, err := downloadChecksumForAsset("1.0.0", "fullsend_1.0.0_linux_arm64.tar.gz") + require.Error(t, err) + assert.Contains(t, err.Error(), "not found in checksums.txt") +} + +func TestDownloadChecksumForAsset_InvalidHex(t *testing.T) { + body := "ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ fullsend_1.0.0_linux_amd64.tar.gz\n" + + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + fmt.Fprint(w, body) + })) + defer srv.Close() + + origBaseURL := ReleaseBaseURL + ReleaseBaseURL = srv.URL + defer func() { ReleaseBaseURL = origBaseURL }() + + _, err := downloadChecksumForAsset("1.0.0", "fullsend_1.0.0_linux_amd64.tar.gz") + require.Error(t, err) + assert.Contains(t, err.Error(), "invalid hex hash") +} + +func TestDownloadReleaseBinary_ChecksumMismatch(t *testing.T) { + var tarBuf bytes.Buffer + gw := gzip.NewWriter(&tarBuf) + tw := tar.NewWriter(gw) + content := []byte("fake binary") + require.NoError(t, tw.WriteHeader(&tar.Header{ + Name: "fullsend", + Size: int64(len(content)), + Mode: 0o755, + Typeflag: tar.TypeReg, + })) + _, err := tw.Write(content) + require.NoError(t, err) + require.NoError(t, tw.Close()) + require.NoError(t, gw.Close()) + + wrongHash := "0000000000000000000000000000000000000000000000000000000000000000" + checksumBody := fmt.Sprintf("%s fullsend_1.0.0_linux_amd64.tar.gz\n", wrongHash) + + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path == "/v1.0.0/checksums.txt" { + fmt.Fprint(w, checksumBody) + } else if r.URL.Path == "/v1.0.0/fullsend_1.0.0_linux_amd64.tar.gz" { + w.Write(tarBuf.Bytes()) + } else { + http.NotFound(w, r) + } + })) + defer srv.Close() + + origBaseURL := ReleaseBaseURL + ReleaseBaseURL = srv.URL + defer func() { ReleaseBaseURL = origBaseURL }() + + destPath := filepath.Join(t.TempDir(), "fullsend") + err = DownloadRelease("1.0.0", "amd64", destPath) + require.Error(t, err) + assert.Contains(t, err.Error(), "checksum mismatch") +} + +func TestDownloadReleaseBinary_ChecksumMatch(t *testing.T) { + var tarBuf bytes.Buffer + gw := gzip.NewWriter(&tarBuf) + tw := tar.NewWriter(gw) + content := []byte("good binary") + require.NoError(t, tw.WriteHeader(&tar.Header{ + Name: "fullsend", + Size: int64(len(content)), + Mode: 0o755, + Typeflag: tar.TypeReg, + })) + _, err := tw.Write(content) + require.NoError(t, err) + require.NoError(t, tw.Close()) + require.NoError(t, gw.Close()) + + tarBytes := tarBuf.Bytes() + h := sha256.Sum256(tarBytes) + correctHash := hex.EncodeToString(h[:]) + checksumBody := fmt.Sprintf("%s fullsend_2.0.0_linux_amd64.tar.gz\n", correctHash) + + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path == "/v2.0.0/checksums.txt" { + fmt.Fprint(w, checksumBody) + } else if r.URL.Path == "/v2.0.0/fullsend_2.0.0_linux_amd64.tar.gz" { + w.Write(tarBytes) + } else { + http.NotFound(w, r) + } + })) + defer srv.Close() + + origBaseURL := ReleaseBaseURL + ReleaseBaseURL = srv.URL + defer func() { ReleaseBaseURL = origBaseURL }() + + destPath := filepath.Join(t.TempDir(), "fullsend") + err = DownloadRelease("2.0.0", "amd64", destPath) + require.NoError(t, err) + + data, err := os.ReadFile(destPath) + require.NoError(t, err) + assert.Equal(t, "good binary", string(data)) +} + +func TestDownloadRelease_Live(t *testing.T) { + if testing.Short() { + t.Skip("skipping download test in short mode") + } + + destPath := filepath.Join(t.TempDir(), "fullsend") + err := DownloadRelease("0.4.0", "amd64", destPath) + require.NoError(t, err) + + info, err := os.Stat(destPath) + require.NoError(t, err) + assert.True(t, info.Size() > 0) +} + +func TestCrossCompile_ProducesBinary(t *testing.T) { + if runtime.GOOS == "linux" { + t.Skip("cross-compilation test only meaningful on non-Linux hosts") + } + if testing.Short() { + t.Skip("skipping cross-compilation in short mode") + } + + tmpDir := t.TempDir() + binPath := filepath.Join(tmpDir, "fullsend") + err := CrossCompile(CrossCompileOpts{ + Version: "dev", + Arch: runtime.GOARCH, + DestPath: binPath, + VersionStamp: "-crosscompiled", + }) + require.NoError(t, err) + + info, err := os.Stat(binPath) + require.NoError(t, err) + assert.True(t, info.Size() > 0) +} + +func TestValidateLinuxBinary_RejectsNonELF(t *testing.T) { + tmp := filepath.Join(t.TempDir(), "not-elf") + require.NoError(t, os.WriteFile(tmp, []byte("#!/bin/sh\necho hello"), 0o755)) + err := ValidateLinuxBinary(tmp, "amd64") + require.Error(t, err) + assert.Contains(t, err.Error(), "not a valid ELF binary") +} + +func TestValidateLinuxBinary_RejectsMissing(t *testing.T) { + err := ValidateLinuxBinary("/tmp/nonexistent-fullsend-binary-12345", "amd64") + require.Error(t, err) +} + +func TestValidateLinuxBinary_AcceptsHostBinary(t *testing.T) { + if runtime.GOOS != "linux" { + t.Skip("host binary is only ELF on Linux") + } + exe, err := os.Executable() + require.NoError(t, err) + assert.NoError(t, ValidateLinuxBinary(exe, runtime.GOARCH)) +} + +func TestResolveForVendor_DevNoCheckoutFails(t *testing.T) { + // Force no module by running from a temp dir without go.mod. + origDir, err := os.Getwd() + require.NoError(t, err) + tmpDir := t.TempDir() + require.NoError(t, os.Chdir(tmpDir)) + t.Cleanup(func() { _ = os.Chdir(origDir) }) + + _, err = ResolveForVendor("dev", "amd64") + require.Error(t, err) + assert.Contains(t, err.Error(), "dev build") +} + +func TestResolveForVendor_NoLatestFallback(t *testing.T) { + var latestCalls atomic.Int32 + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if strings.Contains(r.URL.Path, "/releases/latest") { + latestCalls.Add(1) + } + http.NotFound(w, r) + })) + defer srv.Close() + + origClient := HTTPClient + origBaseURL := ReleaseBaseURL + HTTPClient = srv.Client() + ReleaseBaseURL = srv.URL + defer func() { + HTTPClient = origClient + ReleaseBaseURL = origBaseURL + }() + + origDir, err := os.Getwd() + require.NoError(t, err) + tmpDir := t.TempDir() + require.NoError(t, os.Chdir(tmpDir)) + t.Cleanup(func() { _ = os.Chdir(origDir) }) + + _, err = ResolveForVendor("0.4.0", "amd64") + require.Error(t, err) + assert.Equal(t, int32(0), latestCalls.Load(), "vendor path must not call latest release API") + assert.NotContains(t, err.Error(), "latest") +} + +func TestResolveForVendor_ReleaseFallback(t *testing.T) { + var tarBuf bytes.Buffer + gw := gzip.NewWriter(&tarBuf) + tw := tar.NewWriter(gw) + content := []byte("release binary") + require.NoError(t, tw.WriteHeader(&tar.Header{ + Name: "fullsend", + Size: int64(len(content)), + Mode: 0o755, + Typeflag: tar.TypeReg, + })) + _, err := tw.Write(content) + require.NoError(t, err) + require.NoError(t, tw.Close()) + require.NoError(t, gw.Close()) + + tarBytes := tarBuf.Bytes() + h := sha256.Sum256(tarBytes) + correctHash := hex.EncodeToString(h[:]) + checksumBody := fmt.Sprintf("%s fullsend_0.4.0_linux_amd64.tar.gz\n", correctHash) + + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path == "/v0.4.0/checksums.txt" { + fmt.Fprint(w, checksumBody) + } else if r.URL.Path == "/v0.4.0/fullsend_0.4.0_linux_amd64.tar.gz" { + w.Write(tarBytes) + } else { + http.NotFound(w, r) + } + })) + defer srv.Close() + + origBaseURL := ReleaseBaseURL + ReleaseBaseURL = srv.URL + defer func() { ReleaseBaseURL = origBaseURL }() + + origDir, err := os.Getwd() + require.NoError(t, err) + tmpDir := t.TempDir() + require.NoError(t, os.Chdir(tmpDir)) + t.Cleanup(func() { _ = os.Chdir(origDir) }) + + result, err := ResolveForVendor("0.4.0", "amd64") + require.NoError(t, err) + t.Cleanup(func() { os.RemoveAll(result.TmpDir) }) + assert.Equal(t, SourceReleaseDownload, result.Source) + + data, err := os.ReadFile(result.Path) + require.NoError(t, err) + assert.Equal(t, "release binary", string(data)) +} + +func TestResolveForRun_PrefersReleaseBeforeCrossCompile(t *testing.T) { + // Build mock release assets. + var tarBuf bytes.Buffer + gw := gzip.NewWriter(&tarBuf) + tw := tar.NewWriter(gw) + content := []byte("release binary") + require.NoError(t, tw.WriteHeader(&tar.Header{ + Name: "fullsend", + Size: int64(len(content)), + Mode: 0o755, + Typeflag: tar.TypeReg, + })) + _, err := tw.Write(content) + require.NoError(t, err) + require.NoError(t, tw.Close()) + require.NoError(t, gw.Close()) + + tarBytes := tarBuf.Bytes() + h := sha256.Sum256(tarBytes) + correctHash := hex.EncodeToString(h[:]) + checksumBody := fmt.Sprintf("%s fullsend_0.4.0_linux_amd64.tar.gz\n", correctHash) + + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path == "/v0.4.0/checksums.txt" { + fmt.Fprint(w, checksumBody) + } else if r.URL.Path == "/v0.4.0/fullsend_0.4.0_linux_amd64.tar.gz" { + w.Write(tarBytes) + } else { + http.NotFound(w, r) + } + })) + defer srv.Close() + + origBaseURL := ReleaseBaseURL + ReleaseBaseURL = srv.URL + defer func() { ReleaseBaseURL = origBaseURL }() + + // Run from non-module dir — cross-compile would fail if attempted after release. + origDir, err := os.Getwd() + require.NoError(t, err) + tmpDir := t.TempDir() + require.NoError(t, os.Chdir(tmpDir)) + t.Cleanup(func() { _ = os.Chdir(origDir) }) + + result, err := ResolveForRun("0.4.0", "amd64") + require.NoError(t, err) + t.Cleanup(func() { os.RemoveAll(result.TmpDir) }) + assert.Equal(t, SourceReleaseDownload, result.Source) +} + +func TestDownloadRelease_ExceedsMaxSize(t *testing.T) { + origLimit := maxDownloadSize + maxDownloadSize = 512 + t.Cleanup(func() { maxDownloadSize = origLimit }) + + content := bytes.Repeat([]byte("x"), 2000) + + var tarBuf bytes.Buffer + gw, err := gzip.NewWriterLevel(&tarBuf, gzip.NoCompression) + require.NoError(t, err) + tw := tar.NewWriter(gw) + require.NoError(t, tw.WriteHeader(&tar.Header{ + Name: "fullsend", + Size: int64(len(content)), + Mode: 0o755, + Typeflag: tar.TypeReg, + })) + _, err = tw.Write(content) + require.NoError(t, err) + require.NoError(t, tw.Close()) + require.NoError(t, gw.Close()) + + tarBytes := tarBuf.Bytes() + h := sha256.Sum256(tarBytes) + checksumBody := fmt.Sprintf("%s fullsend_1.0.0_linux_amd64.tar.gz\n", hex.EncodeToString(h[:])) + + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path == "/v1.0.0/checksums.txt" { + fmt.Fprint(w, checksumBody) + } else if r.URL.Path == "/v1.0.0/fullsend_1.0.0_linux_amd64.tar.gz" { + w.Write(tarBytes) + } else { + http.NotFound(w, r) + } + })) + defer srv.Close() + withTestReleaseServer(t, srv) + + destPath := filepath.Join(t.TempDir(), "fullsend") + err = DownloadRelease("1.0.0", "amd64", destPath) + require.Error(t, err) + assert.Contains(t, err.Error(), "exceeds maximum size") +} + +func TestResolveForRun_CrossCompileFallback(t *testing.T) { + if testing.Short() { + t.Skip("skipping cross-compilation in short mode") + } + + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + http.NotFound(w, r) + })) + defer srv.Close() + withTestReleaseServer(t, srv) + + result, err := ResolveForRun("0.4.0", "amd64") + require.NoError(t, err) + t.Cleanup(func() { os.RemoveAll(result.TmpDir) }) + assert.Equal(t, SourceCheckoutBuild, result.Source) +} + +func TestResolveForRun_LatestReleaseFallback(t *testing.T) { + var tarBuf bytes.Buffer + gw := gzip.NewWriter(&tarBuf) + tw := tar.NewWriter(gw) + content := []byte("latest release binary") + require.NoError(t, tw.WriteHeader(&tar.Header{ + Name: "fullsend", + Size: int64(len(content)), + Mode: 0o755, + Typeflag: tar.TypeReg, + })) + _, err := tw.Write(content) + require.NoError(t, err) + require.NoError(t, tw.Close()) + require.NoError(t, gw.Close()) + + tarBytes := tarBuf.Bytes() + h := sha256.Sum256(tarBytes) + correctHash := hex.EncodeToString(h[:]) + checksumBody := fmt.Sprintf("%s fullsend_9.9.9_linux_amd64.tar.gz\n", correctHash) + + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path == "/repos/fullsend-ai/fullsend/releases/latest" { + fmt.Fprint(w, `{"tag_name":"v9.9.9"}`) + } else if r.URL.Path == "/v9.9.9/checksums.txt" { + fmt.Fprint(w, checksumBody) + } else if r.URL.Path == "/v9.9.9/fullsend_9.9.9_linux_amd64.tar.gz" { + w.Write(tarBytes) + } else { + http.NotFound(w, r) + } + })) + defer srv.Close() + withTestReleaseServer(t, srv) + + origDir, err := os.Getwd() + require.NoError(t, err) + tmpDir := t.TempDir() + require.NoError(t, os.Chdir(tmpDir)) + t.Cleanup(func() { _ = os.Chdir(origDir) }) + + result, err := ResolveForRun("dev", "amd64") + require.NoError(t, err) + t.Cleanup(func() { os.RemoveAll(result.TmpDir) }) + assert.Equal(t, SourceReleaseDownload, result.Source) +} + +func TestResolveForRun_AllStrategiesFail(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + http.NotFound(w, r) + })) + defer srv.Close() + withTestReleaseServer(t, srv) + + origDir, err := os.Getwd() + require.NoError(t, err) + tmpDir := t.TempDir() + require.NoError(t, os.Chdir(tmpDir)) + t.Cleanup(func() { _ = os.Chdir(origDir) }) + + _, err = ResolveForRun("dev", "amd64") + require.Error(t, err) + assert.Contains(t, err.Error(), "all strategies failed") +} + +func TestResolveExplicit_ValidatesELF(t *testing.T) { + tmp := filepath.Join(t.TempDir(), "not-elf") + require.NoError(t, os.WriteFile(tmp, []byte("not binary"), 0o644)) + err := ResolveExplicit(tmp, "amd64") + require.Error(t, err) +} + +// Ensure io is used in download tests. +var _ = io.Discard diff --git a/internal/binary/validate.go b/internal/binary/validate.go new file mode 100644 index 0000000000..64decfb0ae --- /dev/null +++ b/internal/binary/validate.go @@ -0,0 +1,40 @@ +package binary + +import ( + "debug/elf" + "fmt" +) + +// DefaultArch is the architecture used for vendored binaries (linux/amd64 GHA runners). +const DefaultArch = "amd64" + +var validArchs = map[string]bool{"amd64": true, "arm64": true} + +// ValidateLinuxBinary checks that the file at path is a Linux ELF executable +// for the expected architecture. Returns a descriptive error if the file is +// missing, not ELF, not Linux, or the wrong architecture. +func ValidateLinuxBinary(path, arch string) error { + f, err := elf.Open(path) + if err != nil { + return fmt.Errorf("not a valid ELF binary (is this a macOS Mach-O?): %w", err) + } + defer f.Close() + + if f.OSABI != elf.ELFOSABI_NONE && f.OSABI != elf.ELFOSABI_LINUX { + return fmt.Errorf("ELF OS/ABI is %s, expected Linux or NONE", f.OSABI) + } + + archToMachine := map[string]elf.Machine{ + "amd64": elf.EM_X86_64, + "arm64": elf.EM_AARCH64, + } + if expected, ok := archToMachine[arch]; ok && f.Machine != expected { + return fmt.Errorf("ELF machine is %s, expected %s for %s", f.Machine, expected, arch) + } + return nil +} + +// ValidArch reports whether arch is a supported linux target (amd64 or arm64). +func ValidArch(arch string) bool { + return validArchs[arch] +} diff --git a/internal/binary/version.go b/internal/binary/version.go new file mode 100644 index 0000000000..82fcc42936 --- /dev/null +++ b/internal/binary/version.go @@ -0,0 +1,20 @@ +package binary + +import "strings" + +// IsReleasedVersion returns true if version looks like a release tag +// (e.g. "0.4.0", "v0.4.0") rather than a dev build (e.g. "dev", +// "0.4.0-3-gabcdef", "0.4.0-vendored"). +func IsReleasedVersion(v string) bool { + v = strings.TrimPrefix(v, "v") + if v == "" || v == "dev" { + return false + } + // A released version is purely digits and dots (e.g. "0.4.0"). + for _, c := range v { + if c != '.' && (c < '0' || c > '9') { + return false + } + } + return true +} diff --git a/internal/binary/version_test.go b/internal/binary/version_test.go new file mode 100644 index 0000000000..5c1f3213d7 --- /dev/null +++ b/internal/binary/version_test.go @@ -0,0 +1,28 @@ +package binary + +import ( + "testing" + + "github.com/stretchr/testify/assert" +) + +func TestIsReleasedVersion(t *testing.T) { + tests := []struct { + version string + expected bool + }{ + {"0.4.0", true}, + {"v0.4.0", true}, + {"1.0.0", true}, + {"dev", false}, + {"", false}, + {"0.4.0-3-gabcdef", false}, + {"0.4.0-vendored", false}, + {"0.4.0-crosscompiled", false}, + } + for _, tt := range tests { + t.Run(tt.version, func(t *testing.T) { + assert.Equal(t, tt.expected, IsReleasedVersion(tt.version), "version=%q", tt.version) + }) + } +} diff --git a/internal/cli/admin.go b/internal/cli/admin.go index a2b5736804..0e23ad809d 100644 --- a/internal/cli/admin.go +++ b/internal/cli/admin.go @@ -150,6 +150,7 @@ type perRepoInstallConfig struct { SkipMintCheck bool AppSet string VendorBinary bool + FullsendBinary string } // wifProviderPattern validates the full WIF provider resource name format @@ -226,6 +227,7 @@ func newInstallCmd() *cobra.Command { var dryRun bool var skipAppSetup bool var vendorBinary bool + var fullsendBinary string var enrollAllFlag bool var enrollNoneFlag bool var inferenceProject string @@ -270,6 +272,9 @@ Inference authentication: if err := appsetup.ValidateAppSet(appSet); err != nil { return fmt.Errorf("invalid --app-set: %w", err) } + if err := validateVendorBinaryFlags(vendorBinary, fullsendBinary); err != nil { + return err + } arg := args[0] if strings.Contains(arg, "/") { @@ -304,6 +309,7 @@ Inference authentication: SkipMintCheck: skipMintCheck, AppSet: appSet, VendorBinary: vendorBinary, + FullsendBinary: fullsendBinary, }) } @@ -490,7 +496,7 @@ Inference authentication: printer.Blank() if dryRun { - return runDryRun(ctx, client, printer, org, repos, roles, inferenceProvider, inferenceProviderName, skipMintCheck, mintURL, allRepos) + return runDryRun(ctx, client, printer, org, repos, roles, inferenceProvider, inferenceProviderName, skipMintCheck, mintURL, allRepos, vendorBinary, fullsendBinary) } if err := checkInstallScopes(ctx, client, printer); err != nil { @@ -533,14 +539,15 @@ Inference authentication: agentCreds = creds } - return runInstall(ctx, client, printer, org, repos, roles, agentCreds, inferenceProvider, inferenceProviderName, vendorBinary, mintProvider, mintProject, mintRegion, mintSourceDir, mintSkipDeploy, mintURL, skipMintCheck, allRepos) + return runInstall(ctx, client, printer, org, repos, roles, agentCreds, inferenceProvider, inferenceProviderName, vendorBinary, fullsendBinary, mintProvider, mintProject, mintRegion, mintSourceDir, mintSkipDeploy, mintURL, skipMintCheck, allRepos) }, } cmd.Flags().StringVar(&agents, "agents", strings.Join(config.DefaultAgentRoles(), ","), "comma-separated agent roles") cmd.Flags().BoolVar(&dryRun, "dry-run", false, "preview changes without making them") cmd.Flags().BoolVar(&skipAppSetup, "skip-app-setup", false, "skip GitHub App creation/setup") - cmd.Flags().BoolVar(&vendorBinary, "vendor-fullsend-binary", false, "cross-compile and vendor the fullsend binary for development iteration") + cmd.Flags().BoolVar(&vendorBinary, "vendor-fullsend-binary", false, "resolve and upload a linux/amd64 fullsend binary for CI") + cmd.Flags().StringVar(&fullsendBinary, "fullsend-binary", "", "path to a Linux fullsend binary to upload when vendoring (default: auto-resolve)") cmd.Flags().BoolVar(&enrollAllFlag, "enroll-all", false, "enroll all repositories without prompting") cmd.Flags().BoolVar(&enrollNoneFlag, "enroll-none", false, "skip repository enrollment without prompting") cmd.Flags().StringVar(&inferenceProject, "inference-project", "", "GCP project ID for inference (Agent Platform)") @@ -577,6 +584,7 @@ func runPerRepoInstall(ctx context.Context, c perRepoInstallConfig) error { mintSkipDeploy := c.MintSkipDeploy skipMintCheck := c.SkipMintCheck vendorBinary := c.VendorBinary + fullsendBinary := c.FullsendBinary if strings.Contains(repoFullName, "://") || strings.HasPrefix(repoFullName, "www.") { return fmt.Errorf("expected owner/repo format, got a URL — use just the owner/repo portion (e.g. acme/widget)") @@ -829,7 +837,7 @@ func runPerRepoInstall(ctx context.Context, c perRepoInstallConfig) error { } if vendorBinary { printer.Blank() - printer.StepInfo(fmt.Sprintf("Would cross-compile and upload vendored binary to %s", layers.VendoredBinaryPathPerRepo)) + printer.StepInfo(vendorDryRunMessage(fullsendBinary, layers.VendoredBinaryPathPerRepo)) } else { printer.Blank() printer.StepInfo(fmt.Sprintf("Would remove stale vendored binary at %s (if present)", layers.VendoredBinaryPathPerRepo)) @@ -1018,22 +1026,12 @@ func runPerRepoInstall(ctx context.Context, c perRepoInstallConfig) error { printer.StepDone(fmt.Sprintf("Set %d repository secrets", len(repoSecrets))) if vendorBinary { - if err := vendorFullsendBinary(ctx, client, printer, owner, repo); err != nil { + if err := acquireAndVendorFullsendBinary(ctx, client, printer, owner, repo, fullsendBinary); err != nil { return fmt.Errorf("vendoring binary: %w", err) } } else { - // Clean up any vendored binary left from a previous install. - // Mirrors VendorBinaryLayer.Install cleanup logic for per-org mode. - _, err := client.GetFileContent(ctx, owner, repo, layers.VendoredBinaryPathPerRepo) - if err == nil { - printer.StepStart("removing stale vendored binary") - if err := client.DeleteFile(ctx, owner, repo, layers.VendoredBinaryPathPerRepo, "chore: remove vendored binary"); err != nil { - printer.StepFail("failed to remove vendored binary") - return fmt.Errorf("deleting vendored binary: %w", err) - } - printer.StepDone("removed stale vendored binary") - } else if !forge.IsNotFound(err) { - return fmt.Errorf("checking for vendored binary: %w", err) + if err := removeStaleVendoredBinary(ctx, client, printer, owner, repo, layers.VendoredBinaryPathPerRepo); err != nil { + return err } } @@ -1042,54 +1040,6 @@ func runPerRepoInstall(ctx context.Context, c perRepoInstallConfig) error { return nil } -// vendorFullsendBinary cross-compiles the fullsend binary for linux/amd64 -// and uploads it via layers.VendorBinary. Per-org mode uploads to bin/fullsend -// in the .fullsend config repo; per-repo mode uploads to .fullsend/bin/fullsend -// in the target repo. -func vendorFullsendBinary(ctx context.Context, client forge.Client, printer *ui.Printer, owner, repo string) error { - destPath := layers.VendoredBinaryPath - if repo != forge.ConfigRepoName { - destPath = layers.VendoredBinaryPathPerRepo - } - - printer.StepStart("Cross-compiling fullsend for linux/amd64") - - tmpBinary, err := os.CreateTemp("", "fullsend-linux-amd64-*") - if err != nil { - return fmt.Errorf("creating temp file: %w", err) - } - tmpBinary.Close() - defer os.Remove(tmpBinary.Name()) - - buildCmd := exec.Command("go", "build", - "-ldflags", fmt.Sprintf("-X github.com/fullsend-ai/fullsend/internal/cli.version=%s-vendored", version), - "-o", tmpBinary.Name(), - "./cmd/fullsend/", - ) - buildCmd.Env = append(os.Environ(), "GOTOOLCHAIN=auto", "GOOS=linux", "GOARCH=amd64", "CGO_ENABLED=0") - buildCmd.Stderr = os.Stderr - if err := buildCmd.Run(); err != nil { - printer.StepFail("Cross-compilation failed") - return fmt.Errorf("cross-compiling: %w", err) - } - printer.StepDone("Cross-compiled fullsend for linux/amd64") - - printer.StepStart(fmt.Sprintf("Uploading vendored binary to %s", destPath)) - if err := layers.VendorBinary(ctx, client, owner, repo, destPath, tmpBinary.Name()); err != nil { - printer.StepFail("Failed to upload vendored binary") - return err - } - - info, _ := os.Stat(tmpBinary.Name()) - if info != nil { - printer.StepDone(fmt.Sprintf("Uploaded vendored binary (%d MB)", info.Size()/(1024*1024))) - } else { - printer.StepDone("Uploaded vendored binary") - } - - return nil -} - func newUninstallCmd() *cobra.Command { var yolo bool var appSet string @@ -1183,7 +1133,7 @@ func newAnalyzeCmd() *cobra.Command { // runDryRun builds a layer stack with empty credentials and analyzes. // If discoveredRepos is non-nil, it will be used instead of calling ListOrgRepos. -func runDryRun(ctx context.Context, client forge.Client, printer *ui.Printer, org string, enabledRepos, roles []string, inferenceProvider inference.Provider, inferenceProviderName string, skipMintCheck bool, mintURL string, discoveredRepos []forge.Repository) error { +func runDryRun(ctx context.Context, client forge.Client, printer *ui.Printer, org string, enabledRepos, roles []string, inferenceProvider inference.Provider, inferenceProviderName string, skipMintCheck bool, mintURL string, discoveredRepos []forge.Repository, vendorBinary bool, fullsendBinary string) error { printer.Header("Dry run - analyzing what install would do") printer.Blank() @@ -1244,7 +1194,7 @@ func runDryRun(ctx context.Context, client forge.Client, printer *ui.Printer, or } else { dispatcher = gcf.NewProvisioner(gcf.Config{}, nil) } - stack := buildLayerStack(org, client, cfg, printer, user, privateRepo, enabledRepos, agentCreds, enrolledRepoIDs, inferenceProvider, false, nil, dispatcher) + stack := buildLayerStack(org, client, cfg, printer, user, privateRepo, enabledRepos, agentCreds, enrolledRepoIDs, inferenceProvider, vendorBinary, makeVendorFunc(fullsendBinary), dispatcher) if err := runPreflight(ctx, stack, layers.OpInstall, client, printer); err != nil { return err @@ -1505,7 +1455,7 @@ func validateEnabledRepos(enabledRepos, discoveredNames []string) error { // runInstall performs the full installation. // If discoveredRepos is non-nil, it will be used instead of calling ListOrgRepos. -func runInstall(ctx context.Context, client forge.Client, printer *ui.Printer, org string, enabledRepos, roles []string, agentCreds []layers.AgentCredentials, inferenceProvider inference.Provider, inferenceProviderName string, vendorBinary bool, mintProvider, mintProject, mintRegion, mintSourceDir string, mintSkipDeploy bool, mintURL string, skipMintCheck bool, discoveredRepos []forge.Repository) error { +func runInstall(ctx context.Context, client forge.Client, printer *ui.Printer, org string, enabledRepos, roles []string, agentCreds []layers.AgentCredentials, inferenceProvider inference.Provider, inferenceProviderName string, vendorBinary bool, fullsendBinary, mintProvider, mintProject, mintRegion, mintSourceDir string, mintSkipDeploy bool, mintURL string, skipMintCheck bool, discoveredRepos []forge.Repository) error { var allRepos []forge.Repository var err error @@ -1597,7 +1547,7 @@ func runInstall(ctx context.Context, client forge.Client, printer *ui.Printer, o }, gcf.NewLiveGCFClient(mintProject)) } - stack := buildLayerStack(org, client, cfg, printer, user, privateRepo, enabledRepos, agentCreds, enrolledRepoIDs, inferenceProvider, vendorBinary, vendorFullsendBinary, disp) + stack := buildLayerStack(org, client, cfg, printer, user, privateRepo, enabledRepos, agentCreds, enrolledRepoIDs, inferenceProvider, vendorBinary, makeVendorFunc(fullsendBinary), disp) if err := runPreflight(ctx, stack, layers.OpInstall, client, printer); err != nil { return err diff --git a/internal/cli/github.go b/internal/cli/github.go index 75cbecac56..ed695b7213 100644 --- a/internal/cli/github.go +++ b/internal/cli/github.go @@ -60,6 +60,7 @@ type githubSetupConfig struct { enrollAll bool enrollNone bool vendorBinary bool + fullsendBinary string dryRun bool } @@ -89,6 +90,9 @@ values (mint URL, WIF provider, project ID) are provided as flags.`, if err := appsetup.ValidateAppSet(cfg.appSet); err != nil { return fmt.Errorf("invalid --app-set: %w", err) } + if err := validateVendorBinaryFlags(cfg.vendorBinary, cfg.fullsendBinary); err != nil { + return err + } if err := validateMintURLHTTPS(cfg.mintURL); err != nil { return err @@ -132,7 +136,8 @@ values (mint URL, WIF provider, project ID) are provided as flags.`, cmd.Flags().StringVar(&cfg.appSet, "app-set", appsetup.DefaultAppSet, "app set name prefix for GitHub Apps") cmd.Flags().BoolVar(&cfg.enrollAll, "enroll-all", false, "enroll all repositories without prompting") cmd.Flags().BoolVar(&cfg.enrollNone, "enroll-none", false, "skip repository enrollment without prompting") - cmd.Flags().BoolVar(&cfg.vendorBinary, "vendor-fullsend-binary", false, "cross-compile and upload the fullsend binary") + cmd.Flags().BoolVar(&cfg.vendorBinary, "vendor-fullsend-binary", false, "resolve and upload a linux/amd64 fullsend binary for CI") + cmd.Flags().StringVar(&cfg.fullsendBinary, "fullsend-binary", "", "path to a Linux fullsend binary to upload when vendoring (default: auto-resolve)") cmd.Flags().BoolVar(&cfg.dryRun, "dry-run", false, "preview changes without making them") return cmd @@ -266,6 +271,13 @@ func runGitHubSetupPerRepo(ctx context.Context, client forge.Client, printer *ui for _, name := range secretNames { printer.StepInfo(fmt.Sprintf(" %s", name)) } + if cfg.vendorBinary { + printer.Blank() + printer.StepInfo(vendorDryRunMessage(cfg.fullsendBinary, layers.VendoredBinaryPathPerRepo)) + } else { + printer.Blank() + printer.StepInfo(fmt.Sprintf("Would remove stale vendored binary at %s (if present)", layers.VendoredBinaryPathPerRepo)) + } return nil } @@ -305,6 +317,16 @@ func runGitHubSetupPerRepo(ctx context.Context, client forge.Client, printer *ui } printer.StepDone(fmt.Sprintf("Set %d repository secrets", len(repoSecrets))) + if cfg.vendorBinary { + if err := acquireAndVendorFullsendBinary(ctx, client, printer, owner, repo, cfg.fullsendBinary); err != nil { + return fmt.Errorf("vendoring binary: %w", err) + } + } else { + if err := removeStaleVendoredBinary(ctx, client, printer, owner, repo, layers.VendoredBinaryPathPerRepo); err != nil { + return err + } + } + printer.Blank() printer.StepDone(fmt.Sprintf("Per-repo setup complete for %s/%s", owner, repo)) return nil @@ -452,7 +474,7 @@ func runGitHubSetupPerOrg(ctx context.Context, client forge.Client, printer *ui. var vendorFn layers.VendorFunc if cfg.vendorBinary { - vendorFn = vendorFullsendBinary + vendorFn = makeVendorFunc(cfg.fullsendBinary) } stack := buildLayerStack(org, client, orgCfg, printer, user, privateRepo, enabledRepos, agentCreds, enrolledRepoIDs, inferenceProvider, cfg.vendorBinary, vendorFn, dispatcher) diff --git a/internal/cli/run.go b/internal/cli/run.go index 16341341e1..1dcce94897 100644 --- a/internal/cli/run.go +++ b/internal/cli/run.go @@ -1,14 +1,7 @@ package cli import ( - "archive/tar" - "bufio" - "bytes" - "compress/gzip" "context" - "crypto/sha256" - "debug/elf" - "encoding/hex" "encoding/json" "fmt" "io" @@ -24,6 +17,7 @@ import ( "github.com/spf13/cobra" + "github.com/fullsend-ai/fullsend/internal/binary" "github.com/fullsend-ai/fullsend/internal/config" "github.com/fullsend-ai/fullsend/internal/envfile" "github.com/fullsend-ai/fullsend/internal/fetch" @@ -893,7 +887,7 @@ func bootstrapCommon(sandboxName, fullsendBinary string, h *harness.Harness) err if localBinary == "" { if needsCrossCompilation() { targetArch := sandboxArch() - dir, binPath, err := resolveLinuxBinary(targetArch) + result, err := binary.ResolveForRun(version, targetArch) if err != nil { if h.FailModeClosed() { return fmt.Errorf("could not obtain linux/%s binary for security scan (fail_mode: closed): %w\nUse --fullsend-binary to provide a pre-built Linux binary", targetArch, err) @@ -902,8 +896,8 @@ func bootstrapCommon(sandboxName, fullsendBinary string, h *harness.Harness) err fmt.Fprintf(os.Stderr, "WARNING: skipping sandbox-side security scan (fail_mode: open). Use --fullsend-binary to provide a pre-built Linux binary.\n") localBinary = "" } else { - tmpBinaryDir = dir - localBinary = binPath + tmpBinaryDir = result.TmpDir + localBinary = result.Path } } else { var err error @@ -917,8 +911,8 @@ func bootstrapCommon(sandboxName, fullsendBinary string, h *harness.Harness) err defer os.RemoveAll(tmpBinaryDir) } if localBinary != "" { - if err := validateLinuxBinary(localBinary); err != nil { - return fmt.Errorf("fullsend binary %q is not valid for the sandbox: %w", localBinary, err) + if err := binary.ValidateLinuxBinary(localBinary, sandboxArch()); err != nil { + return fmt.Errorf("fullsend binary %q is not valid for the sandbox: %w\nSet FULLSEND_SANDBOX_ARCH to override the target architecture", localBinary, err) } // Use UploadDir (tarball-based) instead of Upload for the binary. // Upload silently fails for large files (~16MB); the tarball @@ -1239,6 +1233,8 @@ func runOIDCRefresh(ctx context.Context, sandboxName, oidcURL, oidcAuth string, } } +var oidcHTTPClient = &http.Client{Timeout: 120 * time.Second} // matches pre-refactor shared httpClient timeout + func refreshOIDCToken(ctx context.Context, sandboxName, oidcURL, oidcAuth string) error { req, err := http.NewRequestWithContext(ctx, "GET", oidcURL, nil) if err != nil { @@ -1246,7 +1242,7 @@ func refreshOIDCToken(ctx context.Context, sandboxName, oidcURL, oidcAuth string } req.Header.Set("Authorization", oidcAuth) - resp, err := httpClient.Do(req) + resp, err := oidcHTTPClient.Do(req) if err != nil { return fmt.Errorf("fetching OIDC token: %w", err) } @@ -1605,31 +1601,6 @@ func needsCrossCompilation() bool { return runtime.GOOS != "linux" } -// validateLinuxBinary checks that the file at path is a Linux ELF executable -// for the expected sandbox architecture. Returns a descriptive error if the -// file is missing, not ELF, not Linux, or the wrong architecture. -func validateLinuxBinary(path string) error { - f, err := elf.Open(path) - if err != nil { - return fmt.Errorf("not a valid ELF binary (is this a macOS Mach-O?): %w", err) - } - defer f.Close() - - if f.OSABI != elf.ELFOSABI_NONE && f.OSABI != elf.ELFOSABI_LINUX { - return fmt.Errorf("ELF OS/ABI is %s, expected Linux or NONE", f.OSABI) - } - - arch := sandboxArch() - archToMachine := map[string]elf.Machine{ - "amd64": elf.EM_X86_64, - "arm64": elf.EM_AARCH64, - } - if expected, ok := archToMachine[arch]; ok && f.Machine != expected { - return fmt.Errorf("ELF machine is %s, expected %s for %s (set FULLSEND_SANDBOX_ARCH to override)", f.Machine, expected, arch) - } - return nil -} - // copyFile copies src to dst, preserving permissions. func copyFile(src, dst string) error { in, err := os.Open(src) @@ -1655,8 +1626,6 @@ func copyFile(src, dst string) error { return os.Chmod(dst, info.Mode()) } -var validArchs = map[string]bool{"amd64": true, "arm64": true} - // sandboxArch returns the target architecture for the sandbox binary. // Defaults to the host arch (correct when sandbox image matches host, e.g. // arm64 Mac → arm64 sandbox image). Override with FULLSEND_SANDBOX_ARCH @@ -1664,7 +1633,7 @@ var validArchs = map[string]bool{"amd64": true, "arm64": true} // on an arm64 host via emulation). Only amd64 and arm64 are supported. func sandboxArch() string { if arch := os.Getenv("FULLSEND_SANDBOX_ARCH"); arch != "" { - if !validArchs[arch] { + if !binary.ValidArch(arch) { fmt.Fprintf(os.Stderr, "WARNING: FULLSEND_SANDBOX_ARCH=%q is not a supported architecture (amd64, arm64), using host arch %s\n", arch, runtime.GOARCH) return runtime.GOARCH } @@ -1673,260 +1642,6 @@ func sandboxArch() string { return runtime.GOARCH } -// resolveLinuxBinary obtains a Linux fullsend binary for the given arch. -// Strategy: download from GitHub Release first (fast, no toolchain needed), -// fall back to cross-compilation if the download fails or version is "dev". -// Returns the temp directory (caller must clean up), the binary path, and any error. -func resolveLinuxBinary(arch string) (tmpDir string, binaryPath string, err error) { - tmpDir, err = os.MkdirTemp("", "fullsend-linux-*") - if err != nil { - return "", "", fmt.Errorf("creating temp dir: %w", err) - } - binaryPath = filepath.Join(tmpDir, "fullsend") - - // 1. Released version → download matching release asset. - if isReleasedVersion(version) { - fmt.Fprintf(os.Stderr, "Downloading fullsend %s for linux/%s from GitHub Release...\n", version, arch) - if dlErr := downloadReleaseBinary(version, arch, binaryPath); dlErr == nil { - fmt.Fprintf(os.Stderr, "Downloaded fullsend for linux/%s\n", arch) - return tmpDir, binaryPath, nil - } else { - fmt.Fprintf(os.Stderr, "WARNING: release download failed: %v\n", dlErr) - } - } - - // 2. Dev build → try cross-compilation (requires Go toolchain + module in CWD). - fmt.Fprintf(os.Stderr, "Cross-compiling fullsend for linux/%s...\n", arch) - if ccErr := crossCompileFullsend(arch, binaryPath); ccErr == nil { - fmt.Fprintf(os.Stderr, "Cross-compiled fullsend for linux/%s\n", arch) - return tmpDir, binaryPath, nil - } else { - fmt.Fprintf(os.Stderr, "WARNING: cross-compilation failed: %v\n", ccErr) - } - - // 3. Last resort → download latest release (version won't match exactly, - // but the scan context command interface is stable across patch versions). - fmt.Fprintf(os.Stderr, "Downloading latest fullsend release for linux/%s...\n", arch) - if dlErr := downloadLatestReleaseBinary(arch, binaryPath); dlErr == nil { - fmt.Fprintf(os.Stderr, "Downloaded latest fullsend for linux/%s\n", arch) - return tmpDir, binaryPath, nil - } else { - fmt.Fprintf(os.Stderr, "WARNING: latest release download failed: %v\n", dlErr) - } - - os.RemoveAll(tmpDir) - return "", "", fmt.Errorf("all strategies failed for linux/%s: provide --fullsend-binary or install Go toolchain", arch) -} - -// isReleasedVersion returns true if version looks like a release tag -// (e.g. "0.4.0", "v0.4.0") rather than a dev build (e.g. "dev", -// "0.4.0-3-gabcdef", "0.4.0-vendored"). -func isReleasedVersion(v string) bool { - v = strings.TrimPrefix(v, "v") - if v == "" || v == "dev" { - return false - } - // A released version is purely digits and dots (e.g. "0.4.0"). - for _, c := range v { - if c != '.' && (c < '0' || c > '9') { - return false - } - } - return true -} - -var releaseBaseURL = "https://github.com/fullsend-ai/fullsend/releases/download" - -var httpClient = &http.Client{Timeout: 120 * time.Second} - -// downloadReleaseBinary downloads the fullsend binary for linux/{arch} from -// the GitHub Release matching the given version, verifies its SHA256 checksum -// against the release checksums.txt, and writes it to destPath. -func downloadReleaseBinary(ver, arch, destPath string) error { - cleanVer := strings.TrimPrefix(ver, "v") - assetName := fmt.Sprintf("fullsend_%s_linux_%s.tar.gz", cleanVer, arch) - - expectedHash, err := downloadChecksumForAsset(ver, assetName) - if err != nil { - return fmt.Errorf("fetching checksum for %s: %w", assetName, err) - } - - url := fmt.Sprintf("%s/v%s/%s", releaseBaseURL, cleanVer, assetName) - resp, err := httpClient.Get(url) //nolint:gosec // URL is constructed from known constants - if err != nil { - return fmt.Errorf("fetching %s: %w", url, err) - } - defer resp.Body.Close() - - if resp.StatusCode != http.StatusOK { - return fmt.Errorf("GET %s returned %d", url, resp.StatusCode) - } - - const maxDownloadSize = 200 * 1024 * 1024 // 200 MB compressed - var buf bytes.Buffer - if _, err := io.Copy(&buf, io.LimitReader(resp.Body, maxDownloadSize)); err != nil { - return fmt.Errorf("reading %s: %w", assetName, err) - } - - h := sha256.Sum256(buf.Bytes()) - actualHash := hex.EncodeToString(h[:]) - if actualHash != expectedHash { - return fmt.Errorf("checksum mismatch for %s: got %s, want %s", assetName, actualHash, expectedHash) - } - - return extractFullsendFromTarGz(bytes.NewReader(buf.Bytes()), destPath) -} - -// downloadChecksumForAsset fetches the checksums.txt from the GitHub Release -// for the given version and returns the SHA256 hash for assetName. -// GoReleaser format: " \n" -func downloadChecksumForAsset(ver, assetName string) (string, error) { - cleanVer := strings.TrimPrefix(ver, "v") - url := fmt.Sprintf("%s/v%s/checksums.txt", releaseBaseURL, cleanVer) - - resp, err := httpClient.Get(url) //nolint:gosec // URL is constructed from known constants - if err != nil { - return "", fmt.Errorf("fetching checksums: %w", err) - } - defer resp.Body.Close() - - if resp.StatusCode != http.StatusOK { - return "", fmt.Errorf("GET %s returned %d", url, resp.StatusCode) - } - - scanner := bufio.NewScanner(io.LimitReader(resp.Body, 64*1024)) - for scanner.Scan() { - line := scanner.Text() - parts := strings.Fields(line) - if len(parts) == 2 && parts[1] == assetName { - hash := strings.ToLower(parts[0]) - if len(hash) != 64 { - return "", fmt.Errorf("invalid hash length for %s in checksums.txt", assetName) - } - if _, err := hex.DecodeString(hash); err != nil { - return "", fmt.Errorf("invalid hex hash for %s in checksums.txt: %w", assetName, err) - } - return hash, nil - } - } - if err := scanner.Err(); err != nil { - return "", fmt.Errorf("reading checksums: %w", err) - } - return "", fmt.Errorf("asset %s not found in checksums.txt", assetName) -} - -// downloadLatestReleaseBinary resolves the latest release tag from the GitHub -// API and downloads the Linux binary for the given arch. -func downloadLatestReleaseBinary(arch, destPath string) error { - tag, err := resolveLatestReleaseTag() - if err != nil { - return err - } - return downloadReleaseBinary(tag, arch, destPath) -} - -func resolveLatestReleaseTag() (string, error) { - resp, err := httpClient.Get("https://api.github.com/repos/fullsend-ai/fullsend/releases/latest") //nolint:gosec - if err != nil { - return "", fmt.Errorf("fetching latest release: %w", err) - } - defer resp.Body.Close() - - if resp.StatusCode != http.StatusOK { - return "", fmt.Errorf("GitHub API returned %d", resp.StatusCode) - } - - var release struct { - TagName string `json:"tag_name"` - } - if err := json.NewDecoder(io.LimitReader(resp.Body, 1024*1024)).Decode(&release); err != nil { - return "", fmt.Errorf("parsing release JSON: %w", err) - } - if release.TagName == "" { - return "", fmt.Errorf("empty tag_name in latest release") - } - return release.TagName, nil -} - -const maxBinarySize = 500 * 1024 * 1024 // 500 MB — reasonable upper bound for a Go binary - -// extractFullsendFromTarGz reads a tar.gz stream and extracts the "fullsend" -// binary to destPath. -func extractFullsendFromTarGz(r io.Reader, destPath string) error { - gz, err := gzip.NewReader(r) - if err != nil { - return fmt.Errorf("gzip reader: %w", err) - } - defer gz.Close() - - tr := tar.NewReader(gz) - for { - hdr, err := tr.Next() - if err == io.EOF { - return fmt.Errorf("fullsend binary not found in archive") - } - if err != nil { - return fmt.Errorf("reading tar: %w", err) - } - clean := filepath.Clean(hdr.Name) - if strings.Contains(clean, "..") || filepath.IsAbs(clean) { - continue - } - if filepath.Base(clean) == "fullsend" && hdr.Typeflag == tar.TypeReg { - f, err := os.OpenFile(destPath, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0o755) - if err != nil { - return fmt.Errorf("creating %s: %w", destPath, err) - } - n, copyErr := io.Copy(f, io.LimitReader(tr, maxBinarySize+1)) - if copyErr != nil { - f.Close() - return fmt.Errorf("extracting fullsend: %w", copyErr) - } - if n > maxBinarySize { - f.Close() - os.Remove(destPath) - return fmt.Errorf("binary exceeds maximum size (%d bytes)", maxBinarySize) - } - return f.Close() - } - } -} - -// crossCompileFullsend builds a Linux fullsend binary for the given arch -// and writes it to destPath. Requires the Go toolchain. -func crossCompileFullsend(arch, destPath string) error { - goPath, lookErr := exec.LookPath("go") - if lookErr != nil { - return fmt.Errorf("Go toolchain not found — install Go or use a released version of fullsend: %w", lookErr) - } - - // Find the module root so `go build ./cmd/fullsend/` resolves correctly - // regardless of the caller's working directory. - modRootCmd := exec.Command(goPath, "env", "GOMOD") - modOutput, err := modRootCmd.Output() - if err != nil { - return fmt.Errorf("finding module root: %w", err) - } - modPath := strings.TrimSpace(string(modOutput)) - if modPath == "" || modPath == os.DevNull { - return fmt.Errorf("not in a Go module — run from the fullsend source tree or use a released version") - } - modRoot := filepath.Dir(modPath) - - buildCmd := exec.Command(goPath, "build", - "-ldflags", fmt.Sprintf("-X github.com/fullsend-ai/fullsend/internal/cli.version=%s-crosscompiled", version), - "-o", destPath, - "./cmd/fullsend/", - ) - buildCmd.Dir = modRoot - buildCmd.Env = append(os.Environ(), "GOTOOLCHAIN=auto", "GOOS=linux", "GOARCH="+arch, "CGO_ENABLED=0") - buildCmd.Stderr = os.Stderr - if err := buildCmd.Run(); err != nil { - return fmt.Errorf("cross-compiling for linux/%s: %w", arch, err) - } - return nil -} - func titleCase(s string) string { words := strings.Fields(s) for i, w := range words { diff --git a/internal/cli/run_test.go b/internal/cli/run_test.go index 9c7f163ef7..8a91ad00f2 100644 --- a/internal/cli/run_test.go +++ b/internal/cli/run_test.go @@ -1,12 +1,8 @@ package cli import ( - "archive/tar" "bytes" - "compress/gzip" "context" - "crypto/sha256" - "encoding/hex" "fmt" "io" "net/http" @@ -23,6 +19,7 @@ import ( "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" + "github.com/fullsend-ai/fullsend/internal/binary" "github.com/fullsend-ai/fullsend/internal/ui" ) @@ -501,16 +498,15 @@ func TestSandboxArch_InvalidFallsBack(t *testing.T) { } func TestValidateLinuxBinary_RejectsNonELF(t *testing.T) { - // A plain text file should be rejected. tmp := filepath.Join(t.TempDir(), "not-elf") require.NoError(t, os.WriteFile(tmp, []byte("#!/bin/sh\necho hello"), 0o755)) - err := validateLinuxBinary(tmp) + err := binary.ValidateLinuxBinary(tmp, "amd64") require.Error(t, err) assert.Contains(t, err.Error(), "not a valid ELF binary") } func TestValidateLinuxBinary_RejectsMissing(t *testing.T) { - err := validateLinuxBinary("/tmp/nonexistent-fullsend-binary-12345") + err := binary.ValidateLinuxBinary("/tmp/nonexistent-fullsend-binary-12345", "amd64") require.Error(t, err) } @@ -520,115 +516,7 @@ func TestValidateLinuxBinary_AcceptsHostBinary(t *testing.T) { } exe, err := os.Executable() require.NoError(t, err) - assert.NoError(t, validateLinuxBinary(exe)) -} - -func TestIsReleasedVersion(t *testing.T) { - tests := []struct { - version string - expected bool - }{ - {"0.4.0", true}, - {"v0.4.0", true}, - {"1.0.0", true}, - {"dev", false}, - {"", false}, - {"0.4.0-3-gabcdef", false}, - {"0.4.0-vendored", false}, - {"0.4.0-crosscompiled", false}, - } - for _, tt := range tests { - t.Run(tt.version, func(t *testing.T) { - assert.Equal(t, tt.expected, isReleasedVersion(tt.version), "version=%q", tt.version) - }) - } -} - -func TestExtractFullsendFromTarGz_PathTraversal(t *testing.T) { - // Create a tar.gz with a path-traversal entry named "../../../tmp/fullsend". - var buf bytes.Buffer - gw := gzip.NewWriter(&buf) - tw := tar.NewWriter(gw) - - content := []byte("malicious binary content") - require.NoError(t, tw.WriteHeader(&tar.Header{ - Name: "../../../tmp/fullsend", - Size: int64(len(content)), - Mode: 0o755, - Typeflag: tar.TypeReg, - })) - _, err := tw.Write(content) - require.NoError(t, err) - require.NoError(t, tw.Close()) - require.NoError(t, gw.Close()) - - destPath := filepath.Join(t.TempDir(), "fullsend") - err = extractFullsendFromTarGz(&buf, destPath) - assert.Error(t, err, "should reject traversal entry and report binary not found") - assert.Contains(t, err.Error(), "not found in archive") -} - -func TestExtractFullsendFromTarGz_ValidEntry(t *testing.T) { - var buf bytes.Buffer - gw := gzip.NewWriter(&buf) - tw := tar.NewWriter(gw) - - content := []byte("valid binary content") - require.NoError(t, tw.WriteHeader(&tar.Header{ - Name: "fullsend_0.4.0_linux_amd64/fullsend", - Size: int64(len(content)), - Mode: 0o755, - Typeflag: tar.TypeReg, - })) - _, err := tw.Write(content) - require.NoError(t, err) - require.NoError(t, tw.Close()) - require.NoError(t, gw.Close()) - - destPath := filepath.Join(t.TempDir(), "fullsend") - err = extractFullsendFromTarGz(&buf, destPath) - require.NoError(t, err) - - data, err := os.ReadFile(destPath) - require.NoError(t, err) - assert.Equal(t, "valid binary content", string(data)) -} - -func TestCrossCompileFullsend_ProducesBinary(t *testing.T) { - if runtime.GOOS == "linux" { - t.Skip("cross-compilation test only meaningful on non-Linux hosts") - } - if testing.Short() { - t.Skip("skipping cross-compilation in short mode") - } - - tmpDir := t.TempDir() - binPath := filepath.Join(tmpDir, "fullsend") - err := crossCompileFullsend(runtime.GOARCH, binPath) - require.NoError(t, err) - - info, err := os.Stat(binPath) - require.NoError(t, err) - assert.True(t, info.Size() > 0, "binary should be non-empty") -} - -func TestResolveLinuxBinary_Download(t *testing.T) { - if testing.Short() { - t.Skip("skipping download test in short mode") - } - - tmpDir := t.TempDir() - binPath := filepath.Join(tmpDir, "fullsend") - err := downloadReleaseBinary("0.4.0", "amd64", binPath) - require.NoError(t, err) - - info, err := os.Stat(binPath) - require.NoError(t, err) - assert.True(t, info.Size() > 0, "downloaded binary should be non-empty") - - // Verify the downloaded artifact is a valid Linux ELF for the requested arch. - t.Setenv("FULLSEND_SANDBOX_ARCH", "amd64") - assert.NoError(t, validateLinuxBinary(binPath), "downloaded binary should be a valid Linux/amd64 ELF") + assert.NoError(t, binary.ValidateLinuxBinary(exe, runtime.GOARCH)) } func TestAgentWorkingDirExcludes_ContainsKnownPatterns(t *testing.T) { @@ -840,148 +728,6 @@ func TestRunHeartbeat_NoNoticeWhenNotCI(t *testing.T) { assert.Empty(t, buf.String(), "should not emit any ::notice:: when not in CI") } -func TestDownloadChecksumForAsset_ParsesLine(t *testing.T) { - body := "1b4f0e9851971998e732078544c96b36c3d01cedf7caa332359d6f1d83567014 fullsend_1.0.0_linux_arm64.tar.gz\n" + - "60303ae22b998861bce3b28f33eec1be758a213c86c93c076dbe9f558c11c752 fullsend_1.0.0_linux_amd64.tar.gz\n" - - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - fmt.Fprint(w, body) - })) - defer srv.Close() - - origBaseURL := releaseBaseURL - releaseBaseURL = srv.URL - defer func() { releaseBaseURL = origBaseURL }() - - hash, err := downloadChecksumForAsset("1.0.0", "fullsend_1.0.0_linux_amd64.tar.gz") - require.NoError(t, err) - assert.Equal(t, "60303ae22b998861bce3b28f33eec1be758a213c86c93c076dbe9f558c11c752", hash) -} - -func TestDownloadChecksumForAsset_AssetNotFound(t *testing.T) { - body := "1b4f0e9851971998e732078544c96b36c3d01cedf7caa332359d6f1d83567014 fullsend_1.0.0_linux_amd64.tar.gz\n" - - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - fmt.Fprint(w, body) - })) - defer srv.Close() - - origBaseURL := releaseBaseURL - releaseBaseURL = srv.URL - defer func() { releaseBaseURL = origBaseURL }() - - _, err := downloadChecksumForAsset("1.0.0", "fullsend_1.0.0_linux_arm64.tar.gz") - require.Error(t, err) - assert.Contains(t, err.Error(), "not found in checksums.txt") -} - -func TestDownloadChecksumForAsset_InvalidHex(t *testing.T) { - body := "ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ fullsend_1.0.0_linux_amd64.tar.gz\n" - - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - fmt.Fprint(w, body) - })) - defer srv.Close() - - origBaseURL := releaseBaseURL - releaseBaseURL = srv.URL - defer func() { releaseBaseURL = origBaseURL }() - - _, err := downloadChecksumForAsset("1.0.0", "fullsend_1.0.0_linux_amd64.tar.gz") - require.Error(t, err) - assert.Contains(t, err.Error(), "invalid hex hash") -} - -func TestDownloadReleaseBinary_ChecksumMismatch(t *testing.T) { - // Build a valid tar.gz containing a "fullsend" binary. - var tarBuf bytes.Buffer - gw := gzip.NewWriter(&tarBuf) - tw := tar.NewWriter(gw) - content := []byte("fake binary") - require.NoError(t, tw.WriteHeader(&tar.Header{ - Name: "fullsend", - Size: int64(len(content)), - Mode: 0o755, - Typeflag: tar.TypeReg, - })) - _, err := tw.Write(content) - require.NoError(t, err) - require.NoError(t, tw.Close()) - require.NoError(t, gw.Close()) - - tarBytes := tarBuf.Bytes() - - // Serve a checksums.txt with a WRONG hash for the asset. - wrongHash := "0000000000000000000000000000000000000000000000000000000000000000" - checksumBody := fmt.Sprintf("%s fullsend_1.0.0_linux_amd64.tar.gz\n", wrongHash) - - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if r.URL.Path == "/v1.0.0/checksums.txt" { - fmt.Fprint(w, checksumBody) - } else if r.URL.Path == "/v1.0.0/fullsend_1.0.0_linux_amd64.tar.gz" { - w.Write(tarBytes) - } else { - http.NotFound(w, r) - } - })) - defer srv.Close() - - origBaseURL := releaseBaseURL - releaseBaseURL = srv.URL - defer func() { releaseBaseURL = origBaseURL }() - - destPath := filepath.Join(t.TempDir(), "fullsend") - err = downloadReleaseBinary("1.0.0", "amd64", destPath) - require.Error(t, err) - assert.Contains(t, err.Error(), "checksum mismatch") -} - -func TestDownloadReleaseBinary_ChecksumMatch(t *testing.T) { - var tarBuf bytes.Buffer - gw := gzip.NewWriter(&tarBuf) - tw := tar.NewWriter(gw) - content := []byte("good binary") - require.NoError(t, tw.WriteHeader(&tar.Header{ - Name: "fullsend", - Size: int64(len(content)), - Mode: 0o755, - Typeflag: tar.TypeReg, - })) - _, err := tw.Write(content) - require.NoError(t, err) - require.NoError(t, tw.Close()) - require.NoError(t, gw.Close()) - - tarBytes := tarBuf.Bytes() - h := sha256.Sum256(tarBytes) - correctHash := hex.EncodeToString(h[:]) - - checksumBody := fmt.Sprintf("%s fullsend_2.0.0_linux_amd64.tar.gz\n", correctHash) - - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if r.URL.Path == "/v2.0.0/checksums.txt" { - fmt.Fprint(w, checksumBody) - } else if r.URL.Path == "/v2.0.0/fullsend_2.0.0_linux_amd64.tar.gz" { - w.Write(tarBytes) - } else { - http.NotFound(w, r) - } - })) - defer srv.Close() - - origBaseURL := releaseBaseURL - releaseBaseURL = srv.URL - defer func() { releaseBaseURL = origBaseURL }() - - destPath := filepath.Join(t.TempDir(), "fullsend") - err = downloadReleaseBinary("2.0.0", "amd64", destPath) - require.NoError(t, err) - - data, err := os.ReadFile(destPath) - require.NoError(t, err) - assert.Equal(t, "good binary", string(data)) -} - func TestValidationFailMessage_UsesOutputWhenPresent(t *testing.T) { msg := validationFailMessage([]byte("check failed: lint errors"), fmt.Errorf("exit status 1")) assert.Equal(t, "check failed: lint errors", msg) diff --git a/internal/cli/vendor.go b/internal/cli/vendor.go new file mode 100644 index 0000000000..bf455a4f78 --- /dev/null +++ b/internal/cli/vendor.go @@ -0,0 +1,118 @@ +package cli + +import ( + "context" + "fmt" + "os" + + "github.com/fullsend-ai/fullsend/internal/binary" + "github.com/fullsend-ai/fullsend/internal/forge" + "github.com/fullsend-ai/fullsend/internal/layers" + "github.com/fullsend-ai/fullsend/internal/ui" +) + +const vendorArch = binary.DefaultArch + +func validateVendorBinaryFlags(vendorBinary bool, fullsendBinary string) error { + if fullsendBinary != "" && !vendorBinary { + return fmt.Errorf("--fullsend-binary requires --vendor-fullsend-binary") + } + return nil +} + +// makeVendorFunc returns a VendorFunc closure that uploads a fullsend binary +// using the vendoring acquisition policy. +func makeVendorFunc(fullsendBinary string) layers.VendorFunc { + return func(ctx context.Context, client forge.Client, printer *ui.Printer, owner, repo string) error { + return acquireAndVendorFullsendBinary(ctx, client, printer, owner, repo, fullsendBinary) + } +} + +// acquireAndVendorFullsendBinary resolves a Linux binary and uploads it to the +// target repo using the vendoring policy. +func acquireAndVendorFullsendBinary(ctx context.Context, client forge.Client, printer *ui.Printer, owner, repo, fullsendBinary string) error { + destPath := layers.VendoredBinaryPath + if repo != forge.ConfigRepoName { + destPath = layers.VendoredBinaryPathPerRepo + } + + var ( + binPath string + source binary.Source + tmpDir string + ) + + if fullsendBinary != "" { + printer.StepStart(fmt.Sprintf("Using provided binary: %s", fullsendBinary)) + if err := binary.ResolveExplicit(fullsendBinary, vendorArch); err != nil { + printer.StepFail("Invalid --fullsend-binary") + return fmt.Errorf("validating --fullsend-binary: %w", err) + } + binPath = fullsendBinary + source = binary.SourceExplicitPath + printer.StepDone("Validated linux/amd64 ELF binary") + } else { + result, err := binary.ResolveForVendor(version, vendorArch) + if err != nil { + printer.StepFail("Failed to obtain binary for vendoring") + return err + } + tmpDir = result.TmpDir + binPath = result.Path + source = result.Source + } + + if tmpDir != "" { + defer os.RemoveAll(tmpDir) + } + + info, err := os.Stat(binPath) + if err != nil { + return fmt.Errorf("stat binary: %w", err) + } + + commitMsg := layers.VendorCommitMessage(source, version, destPath, info.Size()) + + printer.StepStart(fmt.Sprintf("Uploading vendored binary to %s", destPath)) + if err := layers.VendorBinary(ctx, client, owner, repo, destPath, binPath, commitMsg); err != nil { + printer.StepFail("Failed to upload vendored binary") + return err + } + + printer.StepDone(fmt.Sprintf("Uploaded vendored binary (%d MB)", info.Size()/(1024*1024))) + return nil +} + +// removeStaleVendoredBinary deletes a stale vendored binary when vendoring is disabled. +func removeStaleVendoredBinary(ctx context.Context, client forge.Client, printer *ui.Printer, owner, repo, destPath string) error { + _, err := client.GetFileContent(ctx, owner, repo, destPath) + if err != nil { + if forge.IsNotFound(err) { + return nil + } + return fmt.Errorf("checking for vendored binary: %w", err) + } + + printer.StepStart("removing stale vendored binary") + deleteMsg := layers.RemoveStaleBinaryCommitMessage(destPath) + if err := client.DeleteFile(ctx, owner, repo, destPath, deleteMsg); err != nil { + printer.StepFail("failed to remove vendored binary") + return fmt.Errorf("deleting vendored binary: %w", err) + } + printer.StepDone("removed stale vendored binary") + return nil +} + +// vendorDryRunMessage returns a dry-run line describing what vendoring would do. +func vendorDryRunMessage(fullsendBinary, destPath string) string { + if fullsendBinary != "" { + return fmt.Sprintf("Would upload provided binary from %s to %s", fullsendBinary, destPath) + } + if _, err := binary.ModuleRoot(); err == nil { + return fmt.Sprintf("Would cross-compile and upload vendored binary to %s", destPath) + } + if binary.IsReleasedVersion(version) { + return fmt.Sprintf("Would download release %s and upload vendored binary to %s", version, destPath) + } + return fmt.Sprintf("Would fail: dev CLI outside checkout cannot vendor to %s", destPath) +} diff --git a/internal/cli/vendor_test.go b/internal/cli/vendor_test.go new file mode 100644 index 0000000000..f8a4c60eae --- /dev/null +++ b/internal/cli/vendor_test.go @@ -0,0 +1,84 @@ +package cli + +import ( + "context" + "os" + "runtime" + "strings" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/fullsend-ai/fullsend/internal/forge" + "github.com/fullsend-ai/fullsend/internal/layers" + "github.com/fullsend-ai/fullsend/internal/ui" +) + +func TestValidateVendorBinaryFlags(t *testing.T) { + require.NoError(t, validateVendorBinaryFlags(false, "")) + require.NoError(t, validateVendorBinaryFlags(true, "")) + require.NoError(t, validateVendorBinaryFlags(true, "/tmp/fullsend")) + + err := validateVendorBinaryFlags(false, "/tmp/fullsend") + require.Error(t, err) + assert.Contains(t, err.Error(), "--fullsend-binary requires --vendor-fullsend-binary") +} + +func TestInstallCmd_HasFullsendBinaryFlag(t *testing.T) { + cmd := newInstallCmd() + flag := cmd.Flags().Lookup("fullsend-binary") + require.NotNil(t, flag, "expected --fullsend-binary flag") + assert.Equal(t, "", flag.DefValue) +} + +func TestGitHubSetupCmd_HasFullsendBinaryFlag(t *testing.T) { + cmd := newGitHubSetupCmd() + flag := cmd.Flags().Lookup("fullsend-binary") + require.NotNil(t, flag, "expected --fullsend-binary flag") +} + +func TestVendorDryRunMessage(t *testing.T) { + msg := vendorDryRunMessage("/tmp/fullsend", layers.VendoredBinaryPathPerRepo) + assert.Contains(t, msg, "/tmp/fullsend") + assert.Contains(t, msg, layers.VendoredBinaryPathPerRepo) +} + +func TestAcquireAndVendorFullsendBinary_ExplicitPath(t *testing.T) { + if runtime.GOOS != "linux" { + t.Skip("needs Linux ELF binary") + } + exe, err := os.Executable() + require.NoError(t, err) + + client := &forge.FakeClient{} + var buf strings.Builder + printer := ui.New(&buf) + + err = acquireAndVendorFullsendBinary(context.Background(), client, printer, "org", "my-repo", exe) + require.NoError(t, err) + + key := "org/my-repo/" + layers.VendoredBinaryPathPerRepo + require.Contains(t, client.FileContents, key) + require.NotEmpty(t, client.CreatedFiles) + assert.Contains(t, client.CreatedFiles[0].Message, "\n\n") + assert.Contains(t, client.CreatedFiles[0].Message, "Source: --fullsend-binary") +} + +func TestAcquireAndVendorFullsendBinary_CheckoutBuild(t *testing.T) { + if testing.Short() { + t.Skip("skipping cross-compile in short mode") + } + + client := &forge.FakeClient{} + var buf strings.Builder + printer := ui.New(&buf) + + err := acquireAndVendorFullsendBinary(context.Background(), client, printer, "org", forge.ConfigRepoName, "") + require.NoError(t, err) + + key := "org/" + forge.ConfigRepoName + "/" + layers.VendoredBinaryPath + require.Contains(t, client.FileContents, key) + require.NotEmpty(t, client.CreatedFiles) + assert.Contains(t, client.CreatedFiles[0].Message, "cross-compiled from checkout") +} diff --git a/internal/layers/vendor.go b/internal/layers/vendor.go index 0bbddb2c16..6ddd0639e5 100644 --- a/internal/layers/vendor.go +++ b/internal/layers/vendor.go @@ -4,7 +4,9 @@ import ( "context" "fmt" "os" + "strings" + "github.com/fullsend-ai/fullsend/internal/binary" "github.com/fullsend-ai/fullsend/internal/forge" ) @@ -18,7 +20,7 @@ const ( // VendorBinary uploads a pre-built fullsend binary to the given destPath. // CI workflows detect this file and use it instead of downloading from // GitHub releases, enabling development iteration without cutting a release. -func VendorBinary(ctx context.Context, client forge.Client, owner, repo, destPath, binaryPath string) error { +func VendorBinary(ctx context.Context, client forge.Client, owner, repo, destPath, binaryPath, commitMsg string) error { const maxBinarySize = 100 * 1024 * 1024 // 100 MB (GitHub Contents API limit) info, err := os.Stat(binaryPath) if err != nil { @@ -34,9 +36,62 @@ func VendorBinary(ctx context.Context, client forge.Client, owner, repo, destPat if err != nil { return fmt.Errorf("reading binary %s: %w", binaryPath, err) } - if err := client.CreateOrUpdateFile(ctx, owner, repo, - destPath, "chore: vendor fullsend binary for development", data); err != nil { + if err := client.CreateOrUpdateFile(ctx, owner, repo, destPath, commitMsg, data); err != nil { return fmt.Errorf("uploading vendored binary: %w", err) } return nil } + +// VendorCommitMessage returns a GitHub commit message (title + body) for upload. +func VendorCommitMessage(source binary.Source, version, destPath string, sizeBytes int64) string { + const arch = "linux/amd64" + var title string + var bodyLines []string + + switch source { + case binary.SourceExplicitPath: + title = "chore: vendor fullsend binary for development" + bodyLines = []string{ + "Source: --fullsend-binary", + fmt.Sprintf("Path: %s", destPath), + fmt.Sprintf("Size: %d bytes", sizeBytes), + fmt.Sprintf("Arch: %s", arch), + } + case binary.SourceCheckoutBuild: + title = "chore: vendor fullsend binary for development" + bodyLines = []string{ + "Source: cross-compiled from checkout", + fmt.Sprintf("CLI version: %s", version), + fmt.Sprintf("Binary stamp: %s-vendored", version), + fmt.Sprintf("Path: %s", destPath), + fmt.Sprintf("Size: %d bytes", sizeBytes), + fmt.Sprintf("Arch: %s", arch), + } + case binary.SourceReleaseDownload: + cleanVer := strings.TrimPrefix(version, "v") + title = fmt.Sprintf("chore: vendor fullsend v%s binary from release", cleanVer) + bodyLines = []string{ + fmt.Sprintf("Source: GitHub Release v%s", cleanVer), + fmt.Sprintf("Path: %s", destPath), + fmt.Sprintf("Size: %d bytes", sizeBytes), + fmt.Sprintf("Arch: %s", arch), + "Note: binary retains release version stamp (no -vendored suffix)", + } + default: + title = "chore: vendor fullsend binary for development" + bodyLines = []string{fmt.Sprintf("Path: %s", destPath)} + } + + return title + "\n\n" + strings.Join(bodyLines, "\n") +} + +// RemoveStaleBinaryCommitMessage returns title + body for stale binary deletion. +func RemoveStaleBinaryCommitMessage(destPath string) string { + title := "chore: remove vendored fullsend binary" + body := strings.Join([]string{ + "Reason: --vendor-fullsend-binary not set; removing stale binary so CI uses released versions", + fmt.Sprintf("Path: %s", destPath), + "Note: re-run install with --vendor-fullsend-binary to upload again", + }, "\n") + return title + "\n\n" + body +} diff --git a/internal/layers/vendor_test.go b/internal/layers/vendor_test.go new file mode 100644 index 0000000000..4c19c5936b --- /dev/null +++ b/internal/layers/vendor_test.go @@ -0,0 +1,69 @@ +package layers + +import ( + "strings" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/fullsend-ai/fullsend/internal/binary" +) + +func TestVendorCommitMessage_HasTitleAndBody(t *testing.T) { + tests := []struct { + name string + source binary.Source + ver string + path string + size int64 + want []string + }{ + { + name: "explicit path", + source: binary.SourceExplicitPath, + ver: "dev", + path: ".fullsend/bin/fullsend", + size: 1024, + want: []string{"Source: --fullsend-binary", "Path: .fullsend/bin/fullsend", "Size: 1024 bytes"}, + }, + { + name: "checkout build", + source: binary.SourceCheckoutBuild, + ver: "dev", + path: "bin/fullsend", + size: 2048, + want: []string{"Source: cross-compiled from checkout", "Binary stamp: dev-vendored", "Path: bin/fullsend"}, + }, + { + name: "release download", + source: binary.SourceReleaseDownload, + ver: "0.4.0", + path: "bin/fullsend", + size: 4096, + want: []string{"Source: GitHub Release v0.4.0", "no -vendored suffix"}, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + msg := VendorCommitMessage(tt.source, tt.ver, tt.path, tt.size) + require.Contains(t, msg, "\n\n", "commit message must have title and body separated by blank line") + for _, line := range tt.want { + assert.Contains(t, msg, line) + } + }) + } +} + +func TestRemoveStaleBinaryCommitMessage_HasTitleAndBody(t *testing.T) { + msg := RemoveStaleBinaryCommitMessage(".fullsend/bin/fullsend") + require.Contains(t, msg, "\n\n") + assert.Contains(t, msg, "chore: remove vendored fullsend binary") + assert.Contains(t, msg, "Path: .fullsend/bin/fullsend") + assert.Contains(t, msg, "--vendor-fullsend-binary not set") +} + +func TestVendorCommitMessage_ReleaseTitle(t *testing.T) { + msg := VendorCommitMessage(binary.SourceReleaseDownload, "v0.4.0", "bin/fullsend", 100) + assert.True(t, strings.HasPrefix(msg, "chore: vendor fullsend v0.4.0 binary from release")) +} diff --git a/internal/layers/vendorbinary.go b/internal/layers/vendorbinary.go index 15d326540b..901920a0fc 100644 --- a/internal/layers/vendorbinary.go +++ b/internal/layers/vendorbinary.go @@ -83,7 +83,8 @@ func (l *VendorBinaryLayer) Install(ctx context.Context) error { } l.ui.StepStart("removing stale vendored binary") - if err := l.client.DeleteFile(ctx, l.org, l.repo, path, "chore: remove vendored binary"); err != nil { + deleteMsg := RemoveStaleBinaryCommitMessage(path) + if err := l.client.DeleteFile(ctx, l.org, l.repo, path, deleteMsg); err != nil { l.ui.StepFail("failed to remove vendored binary") return fmt.Errorf("deleting vendored binary: %w", err) } @@ -117,10 +118,10 @@ func (l *VendorBinaryLayer) Analyze(ctx context.Context) (*LayerReport, error) { if l.enabled { report.Status = StatusInstalled - report.Details = append(report.Details, "vendored binary present") + report.Details = append(report.Details, fmt.Sprintf("vendored binary present at %s", l.binaryPath())) } else { report.Status = StatusDegraded - report.Details = append(report.Details, "stale vendored binary present") + report.Details = append(report.Details, fmt.Sprintf("stale vendored binary present at %s", l.binaryPath())) report.WouldFix = append(report.WouldFix, "delete vendored binary") } return report, nil diff --git a/internal/layers/vendorbinary_test.go b/internal/layers/vendorbinary_test.go index d0c1304cb7..72ee7d1e05 100644 --- a/internal/layers/vendorbinary_test.go +++ b/internal/layers/vendorbinary_test.go @@ -4,6 +4,7 @@ import ( "bytes" "context" "errors" + "strings" "testing" "github.com/stretchr/testify/assert" @@ -89,6 +90,8 @@ func TestVendorBinaryLayer_DisabledDeletesBinary(t *testing.T) { assert.Equal(t, "test-org", client.DeletedFiles[0].Owner) assert.Equal(t, ".fullsend", client.DeletedFiles[0].Repo) assert.Equal(t, "bin/fullsend", client.DeletedFiles[0].Path) + assert.Contains(t, client.DeletedFiles[0].Message, "\n\n") + assert.Contains(t, client.DeletedFiles[0].Message, "Path: bin/fullsend") // File should no longer be in FileContents _, ok := client.FileContents["test-org/.fullsend/bin/fullsend"] @@ -143,7 +146,7 @@ func TestVendorBinaryLayer_Analyze_EnabledPresent(t *testing.T) { require.NoError(t, err) assert.Equal(t, "vendor-binary", report.Name) assert.Equal(t, StatusInstalled, report.Status) - assert.Contains(t, report.Details, "vendored binary present") + assert.True(t, strings.Contains(strings.Join(report.Details, " "), "vendored binary present at")) } func TestVendorBinaryLayer_Analyze_EnabledAbsent(t *testing.T) { @@ -169,7 +172,7 @@ func TestVendorBinaryLayer_Analyze_DisabledPresent(t *testing.T) { report, err := layer.Analyze(context.Background()) require.NoError(t, err) assert.Equal(t, StatusDegraded, report.Status) - assert.Contains(t, report.Details, "stale vendored binary present") + assert.True(t, strings.Contains(strings.Join(report.Details, " "), "stale vendored binary present at")) assert.Contains(t, report.WouldFix, "delete vendored binary") } @@ -245,7 +248,7 @@ func TestVendorBinaryLayer_PerRepo_Analyze_EnabledPresent(t *testing.T) { report, err := layer.Analyze(context.Background()) require.NoError(t, err) assert.Equal(t, StatusInstalled, report.Status) - assert.Contains(t, report.Details, "vendored binary present") + assert.True(t, strings.Contains(strings.Join(report.Details, " "), "vendored binary present at")) } func TestVendorBinaryLayer_PerRepo_Analyze_DisabledPresent(t *testing.T) { @@ -261,7 +264,7 @@ func TestVendorBinaryLayer_PerRepo_Analyze_DisabledPresent(t *testing.T) { report, err := layer.Analyze(context.Background()) require.NoError(t, err) assert.Equal(t, StatusDegraded, report.Status) - assert.Contains(t, report.Details, "stale vendored binary present") + assert.True(t, strings.Contains(strings.Join(report.Details, " "), "stale vendored binary present at")) } func TestVendorBinaryLayer_PerRepo_EnabledCallsVendorFn(t *testing.T) {