Bridge linkers: run + benchmark reld on Windows and macOS now by delegating to lld (rust-lld) per platform

[zackees]

Part of #2. Complements — does not replace — #7 (native PE/COFF) and #8 (native Mach-O).

Proposal

reld v0 is three linkers in one binary. Linux links natively via the inherited wild core (working today). Windows and macOS link via a bridge: reld dispatches to the fastest available open-source linker for that format, so reld is invocable through -Clinker/-fuse-ld= on all three OSes now, and the published benchmark graph gets three platforms of data instead of one.

The native backends (#7, #8) are measured in months. Until they land, reld on Windows/macOS exits with bail!, the benchmark covers one OS, and all cross-platform CI plumbing remains unproven. The bridge fixes all three cheaply, and when a native backend lands it replaces the bridge behind the same dispatch point — with the benchmark immediately showing the native-vs-bridge delta on the day of the switch.

Investigation: fastest open-source linker per platform

Platform	Choice	Why
Windows PE/COFF	lld-link (LLD COFF driver)	Default bridge. Ships inside every Rust toolchain as rust-lld — zero build/install cost, Apache-2.0 WITH LLVM-exception. The fastest broadly-available OSS COFF linker.
Windows (alternative)	radlink (EpicGamesExt/raddebugger, MIT)	Optional bridge + benchmark competitor. ~50% faster link on multi-GB-debug-info projects, fully MSVC-compatible CLI, real PDB output. x64-Windows-only, younger codebase — alternative, not default.
macOS Mach-O	ld64.lld (LLD Mach-O driver)	The only viable option. mold's Mach-O backend was removed (the commercial sold fork was discontinued when mold 2.0 went MIT); zld is archived and itself recommends ld64.lld; Apple's ld-prime is faster but closed-source. Also inside rust-lld.
Linux ELF	wild core (inherited)	Native already. No bridge.

The pivotal fact: rust-lld is one binary containing all three drivers selected by flavor (ld.lld / lld-link / ld64.lld), and it is already installed with every rustc. The bridge is therefore: locate rust-lld (or system lld), exec the right flavor with argv passed through, propagate exit status and diagnostics. No LLVM build, no C++ FFI, no vendored source.

There is no production-grade Rust COFF or Mach-O linker to vendor instead — wild's Mach-O backend is a feature-gated skeleton with 40 todo!()s (#8), and nothing else exists. Subprocess delegation to lld is the entire realistic option space for "runs everywhere this quarter."

What already exists in-tree (verified)

• Dispatch is done (D2): argv[0] multi-call + -flavor parsing live in crates/reld-core/src/args.rs (from_flavor ~line 239, from_executable_name ~line 251); a reld-link binary already builds via crates/reld/src/bin/reld-link.rs.
• The plug-in points are single lines: crates/reld-core/src/lib.rs:267 — Args::Coff(_) => bail!("COFF linking is not implemented until Phase 3") — and the Mach-O arm beside it at line 266. The bridge replaces these two bail!s.
• The benchmark n/a is hardcoded: crates/reld-testkit/src/bin/reld-bench.rs:127 prints n/a unconditionally; default competitor set ["bfd","lld","mold","wild"] (line ~70) is Linux-only; the workflow .github/workflows/benchmark-stats.yml runs on a single ubuntu-24.04 runner; history.jsonl lines (ci/benchmark_stats.py:329) carry no OS/target key, so multi-platform history needs a schema bump.
• Cross-platform groundwork already present: probe_linker degrades missing linkers to n/a; .exe naming and -fPIC are cfg!(windows)-guarded; chart-font lookup already knows Windows/macOS paths.

Relationship to locked decisions — read this before objecting

docs/plan/01-DECISIONS.md D6 says an external tool dependency in the hot path "defeats the entire premise of a fast dev-loop linker." The bridge deliberately and temporarily violates that rationale, and says so out loud. Resolution:

1. The bridge starts as scaffolding — it lights up CI, toolchain integration, and the benchmark on three platforms before the native backends exist — but the dispatch architecture is permanent: reld is a router over multiple engines. The native wild-core engine becomes the default for a platform the day its backend lands (P3: PE/COFF backend, MinGW (windows-gnu) ABI #7/P4: Mach-O backend, aarch64-apple-darwin #8); bridge engines remain behind it for what the native engine will never do (LTO, release-fidelity links — see B8) and as the differential-test oracle.
2. Every benchmark datum produced while bridging is labeled mode: "bridge:lld" in the JSON and in the graph caption. A bridge number presented as a reld number would violate the project's honesty policy (README claim 4, DESIGN §6). We are publishing "reld orchestrating lld," and the graph will say exactly that.
3. D1 (in-tree backends only) is untouched — the bridge adds no backend; it adds a delegation arm ahead of the backend dispatch.

Design decisions (proposed answers)

• B1 — subprocess, not liblld in-process. In-process saves ~10–30 ms of Windows process spawn but costs an LLVM build dependency and C++ FFI. Not worth it for scaffolding. The spawn overhead is disclosed in the graph caption instead.
• B2 — discovery order: RELD_BRIDGE_LINKER env override → rust-lld in the active toolchain (rustc --print sysroot → lib/rustlib/<host>/bin/) → lld-link/ld64.lld on PATH → hard error naming exactly what to install (until BR-5, when the embedded payload becomes the final fallback and this error can no longer occur). Never silently fall back to MSVC link.exe or Apple ld — closed-source tools would poison benchmark comparability and mask discovery bugs.
• B3 — flag surface: pass-through. lld already speaks each platform's native dialect (MSVC link.exe args for COFF, ld64 args for Mach-O), which is the same dialect reld's dispatch already selects per D2. reld intercepts only its own diagnostics/flags; unknown-flag policy is lld's.
• B4 — bridge output and CI gates: differential/corpus harnesses run against the bridge on Windows/macOS from day one. That validates the harness on those OSes and produces the reference oracle the native backends will be diffed against. Gate scripts must never conflate "bridge green" with "P3: PE/COFF backend, MinGW (windows-gnu) ABI #7/P4: Mach-O backend, aarch64-apple-darwin #8 done."
• B5 — exit-status and diagnostic fidelity: propagate the child's exit code verbatim; stream stderr through unmodified with a single reld: delegating to <linker> (bridge mode) note-level line so a user always knows which engine linked.

Distribution: self-installing engine payloads

The bridge must not depend on the user having a Rust toolchain or lld on PATH. Instead, reld ships as a payload binary: each per-platform reld build embeds a compressed archive (zstd) of that platform's bridge engine(s), self-extracted on first run into a versioned cache. reld is then a single-file install that links out of the box on every OS.

• B6 — payload, with discovery as override. Embedded payload is the guarantee; RELD_BRIDGE_LINKER / sysroot / PATH discovery (B2) becomes an override order searched before falling back to the payload, so power users can pin their own engine. Embedding beats download-on-first-run: works offline, no supply-chain surface at runtime, checksummable at build time. Payloads are sourced from official upstream release artifacts (llvm.org LLVM releases or the rust-lld shipped in rustc dist tarballs), pinned by version + sha256 in-repo. Per-platform builds embed only their own platform's engines (~20–30 MB compressed for lld) — not all three platforms.
• B7 — versioned cache layout. ~/.reld/engines/<platform>/<engine>/<engine-version>/ (e.g. ~/.reld/engines/x86_64-pc-windows-msvc/lld/19.1.7/). Each reld release carries a manifest pinning exactly which engine versions it uses, keyed by content hash — so a reld update that improves only one engine extracts only that engine, and older reld versions keep working against their pinned versions. Engines coexist by default (they are tens of MB; correctness beats tidiness); reld cache prune removes engines no installed manifest references. Extraction is atomic — extract to a temp dir, fsync, rename into place — because parallel build systems spawn many linker processes at once and first run will race with itself; losers of the rename just use the winner's copy.
• B8 — flag-based engine routing. The router inspects the link request and picks the engine, not just the platform:
  • -flto / /GL / -Clto → route to lld (which has real LTO support) even on Linux where the native engine exists. This upgrades DESIGN's "LTO flags are rejected with a clear diagnostic" to "LTO links are delegated" — strictly better for the user, still zero LTO code in reld.
  • Release-shaped links (e.g. explicit --engine=lld, or future heuristics) → full-fidelity engine; dev-loop links → fastest engine for that platform.
  • Every delegation prints the one-line reld: engine=<name> (<reason>) note (B5), and the benchmark records which engine produced each number (B3's mode field generalizes to engine).

Phases

BR-1 — bridge module + replace the two bail!s (small)
reld-core gains a bridge module (discovery per B2, exec, status propagation). lib.rs:266-267 route Coff/MachO args to it. Linux path untouched. Gate: existing Linux CI stays green; on a dev Windows box reld-link /?-style smoke passes through to lld-link.

BR-2 — Windows CI proof (medium)
windows-latest job: cargo build of a nontrivial crate with -Clinker=reld (msvc target) produces a running .exe. Bridge-discovery unit tests (env override, sysroot lookup, PATH, error text). Optional: radlink smoke behind RELD_BRIDGE_LINKER.

BR-3 — macOS CI proof (medium)
Same on macos-latest (arm64): cargo build -Clinker=reld → binary runs. Note ld64.lld performs ad-hoc signing itself, so D6 is deferred, not solved. Gate: binary executes on the runner with default Gatekeeper.

BR-4 — benchmark goes three-platform (medium)
Workflow becomes a 3-OS matrix. Per-OS competitor sets: Linux bfd/lld/mold/wild, Windows link.exe/lld-link (+radlink optional), macOS ld64/ld64.lld. reld-bench.rs:127 replaced with a real reld invocation (bridge mode on Win/mac, native on Linux). Schema bump: SCHEMA_VERSION=2, results/history lines gain target and mode (native | bridge:lld | bridge:radlink); renderer produces per-OS charts or a faceted chart; README caption documents bridge labeling.

BR-5 — payload packaging + versioned cache (medium)
Embed per-platform zstd engine archives at build time (build script keyed off target triple); first-run atomic extraction into ~/.reld/engines/... per B7; manifest + reld cache prune; RELD_BRIDGE_LINKER/discovery kept as override per B6. Gate: on a clean runner with no rustup and no lld on PATH, a single downloaded reld binary links a program on all three OSes.

BR-6 — flag-based engine routing (LTO etc.) (small)
Router recognizes LTO flags and --engine= and routes per B8; benchmark mode field generalizes to engine. Gate: -Clto build succeeds via delegation on Linux while non-LTO links still use the native engine.

BR-7 — competitor definition + native handoff (small, standing)
The bridge columns define the number the native backends must beat. When #7/#8 land: flip default to native, keep bridge as --bridge fallback and as the differential-test reference linker. Benchmark keeps both series across the transition so the graph shows the delta.

Non-goals

• Vendoring LLVM/lld (or radlink) source into the repo — engines are embedded as prebuilt, checksummed binary payloads (B6), never built from source in our tree
• Beating anything on speed while bridging — bridge ≈ lld + spawn overhead, by construction
• PDB (D5) and code-signing internals (D6) — inherited from lld for free while bridging; still owed by the native backends
• MinGW/win-gnu bridge dialect work beyond what pass-through gives (win-gnu native path is P3: PE/COFF backend, MinGW (windows-gnu) ABI #7's scope)

[zackees]

BR-1 landed — Windows COFF bridge merged in #20 (squash 8eb143b), closing #18. On a Windows/MSVC host reld/reld-link now delegates COFF links to lld-link (rust-lld) via raw-argv pass-through, with -flavor de-duplication and verbatim exit-code propagation. Validated by an e2e link of a two-crate, 16-file, 4-dependency project including rusqlite (bundled → SQLite C amalgamation): binary links via reld→lld-link, runs, exits 0. Next: BR-2 (Windows CI proof job) / BR-3 (macOS ld64.lld bridge).

[zackees]

BR-2 landed — merged in #22 (squash 20c4678), closing #21. The Windows COFF bridge is now a permanent CI gate: the windows-msvc leg builds ci/e2e/sqlite-bridge (two-crate, 16-file, 4-dep incl. rusqlite bundled → SQLite C amalgamation) with reld-link.exe as the linker and runs it, asserting exit 0 + OK/linked SQLite markers. The fixture is its own workspace (root-excluded), Cargo.lock pinned. Verified green in CI on the windows-msvc runner, not just locally. Next: BR-3 (macOS ld64.lld bridge — note it can't be e2e'd on the current Windows dev box) or BR-4 (3-platform benchmark + schema v2).

[zackees]

BR-4a landed — merged in #24 (squash e14246e), closing #23. reld-bench now measures reld via its ld.reld driver shim like any other linker, replacing the hardcoded n/a: a real reld number lands on the Linux benchmark (native ELF engine), with honest n/a + reason where the clang harness can't drive reld (Windows/macOS use the rustc -Clinker path). Table-scraping contract with benchmark_stats.py unchanged. An independent review reproduced a test race (two discovery tests racing on the real exe dir); fixed by making the sibling-dir lookup injectable so tests are hermetic (5/5 under parallelism). Deferred to BR-4b: SCHEMA_VERSION=2 (mode/engine/target fields), the 3-OS workflow matrix, and a Windows/macOS rustc-based reld bench path.

[zackees]

BR-4b (schema) landed — merged in #26 (squash 0801bb8), closing #25. Benchmark JSON is now SCHEMA_VERSION=2: history.jsonl lines carry target (so per-OS runs don't collide once the graph goes 3-platform), and results entries carry a per-series mode (native reld on Linux, bridge on Windows/macOS, reference for competitors). Schema/metadata only — parser, markdown-table contract, and chart renderers untouched; nothing reads back old history lines, so published data stays valid. Independent review: APPROVE (no findings). Remaining BR-4 slices: the 3-OS workflow matrix + per-OS competitor sets, and a Windows/macOS rustc-based reld bench path (clang can't drive reld off-Linux).

[zackees]

BR-4c landed — merged in #28 (squash 42974ac), closing #27. The windows-msvc and macos-arm64 CI legs now run reld-bench (clang -fuse-ld=lld), assert a well-formed table, and upload it as a per-OS artifact — verified: the run produced benchmark-windows-msvc + benchmark-macos-arm64 artifacts. This proves the bench harness + per-OS competitor probing work off-Linux on every PR (not just the nightly job), de-risking the future 3-OS publish matrix. reld charts honest n/a on these legs (clang can't drive reld off-Linux). Independent review empirically confirmed no false-green on either leg. Remaining: the 3-OS publish/aggregation matrix + multi-target rendering, the Windows/macOS rustc-based reld measurement (real reld number off-Linux), and BR-3 (native macOS ld64.lld bridge).

[zackees]

BR-3 landed — reld now links on all three platforms. Merged in #31 (squash 71ee065), closing #29. The bridge is now format-aware (BridgeTarget{Coff,MachO}): Windows COFF → lld-link (-flavor link), macOS Mach-O → ld64.lld (-flavor darwin), same discovery + raw-argv pass-through + exit-code propagation. The macos-arm64 CI e2e passed on the first try — reld built and ran the rusqlite(bundled, C-linking) fixture on macOS via -Clinker-flavor=ld64.lld. COFF path verified unchanged (Windows rusqlite e2e green).

Milestone: the same 16-file rusqlite e2e now passes on Linux (native wild), Windows (COFF→lld-link bridge), and macOS (Mach-O→ld64.lld bridge). "Runs everywhere" is real.

Bridge/benchmark phases landed: BR-1 (Windows COFF), BR-2 (Windows CI proof), BR-3 (macOS Mach-O), BR-4a (bench participation), BR-4b (schema v2), BR-4c (cross-platform bench smoke). Remaining: the 3-OS benchmark publish matrix; the Windows/macOS rustc-based reld measurement; the polylinker router + LTO-via-routing (#30); and native COFF/Mach-O codegen (#7/#8) behind the same dispatch points.

[zackees]

Benchmark/publication slice update: #69 is merged at 58066f9, and run 32668402943 published real reld measurements for every canonical scenario on all three targets. Schema v4 records Linux native/reld, Windows bridge/lld-link, and macOS bridge/ld64.lld; strict coverage and exact-SHA freshness both passed. Keeping this issue open for its broader bridge/router/payload scope.

[zackees]

Wave 0 audit (#153): keeping this open. Five of the seven slices are delivered; two are not, which matches this issue's own last comment ("keeping this issue open for its broader bridge/router/payload scope").

Delivered:

• BR-1 bridge module replacing the two bail!s — crates/reld-core/src/bridge.rs.
• BR-2 / BR-3 Windows and macOS CI proof — phase1-native covers windows-msvc, windows-gnu and macos-arm64; the sqlite/rusqlite e2e links and runs on both.
• BR-4 three-platform benchmark — schema v4 records Linux native/reld, Windows bridge/lld-link, macOS bridge/ld64.lld, with strict coverage and exact-SHA freshness.
• BR-6 flag-based engine routing — the capability table in bridge.rs, now being replaced by the declarative flag table (Polylinker no-gaps gate: classify every flag, route by target not host, prove coverage before reld becomes soldr's default #123 Phase 1, PRs feat(flag-table): bootstrap the declarative flag table (reld#123 Phase 1 / D1) #139/test(flag-table): add NativeControl flags + well-formedness tests (reld#123 Phase 1) #140).

Outstanding:

• BR-5 — payload packaging + versioned cache. discover_linker still ends at a hard error: RELD_BRIDGE_LINKER → rust-lld via rustc --print sysroot → gcc-ld/<driver> → PATH → error naming what to install. The embedded-payload fallback described in B2 does not exist, so a host without a Rust toolchain still cannot bridge. Note that reld#148 now ships per-platform release archives, which changes the options here: the payload could be a release asset rather than something embedded in the binary.
• BR-7 — competitor definition + native handoff (standing; closes out when the native backends land, P3: PE/COFF backend, MinGW (windows-gnu) ABI #7/P4: Mach-O backend, aarch64-apple-darwin #8, tracked in meta: the long-horizon program — native backends, incremental, daemon #154).

Suggest narrowing this issue to BR-5 + BR-7, since BR-1/2/3/4/6 are done and the title no longer describes what is left.