meta: the long-horizon program — native backends, incremental, daemon

[zackees]

Scope

One place for the work that comes after the near-term burn-down. Tracks #7, #8, #9, #10, #11, #19 plus the Windows research/implementation chain #91, #95, #96.

Relationship to the other metas:

• meta: burn-down — unblock soldr's default linker (#123) and clear the debt around it #153 is the near-term burn-down: unblocking soldr's default linker (Polylinker no-gaps gate: classify every flag, route by target not host, prove coverage before reld becomes soldr's default #123) and the CI/test debt around it. Nothing here should start ahead of it, with the two exceptions noted under Sequencing.
• meta(perf): externalize mimalloc-pprof and optimize the standalone LLVM COFF driver #97 stays the authority for the allocator + zackees/llvm-ld program. This meta links to it and does not restate it.
• Meta: road to a working cross-platform link (Windows / Linux / macOS) #2 is the original roadmap and is now badly stale — it still describes a pre-linker world ("crates/reld is 28 lines", "everything else exits 2", the benchmark reld column reading n/a). reld links real programs on three platforms today. Recommend closing Meta: road to a working cross-platform link (Windows / Linux / macOS) #2 in favor of meta: burn-down — unblock soldr's default linker (#123) and clear the debt around it #153 (near term) and this issue (long horizon), or rewriting its status section. Its D1–D6 decisions are worth preserving wherever it lands.

Decisions already made — do not re-litigate

• Fork wild's ELF backend rather than reimplement (Meta: road to a working cross-platform link (Windows / Linux / macOS) #2 D1). Done; reld is that fork.
• Windows goes through an LLD-COFF fork, not a COFF retrofit into wild (Research: Windows PE/COFF fast-linking strategy - optimize LLD-COFF fork vs retrofit COFF into Wild #91, decided in its final comment). The sequencing lives in meta(perf): externalize mimalloc-pprof and optimize the standalone LLVM COFF driver #97: pin the allocator, extract zackees/llvm-ld as a C/C++-only minimal COFF library with a narrow C ABI, keep all Rust in reld, then prove byte-identical EXE/PDB against stock LLD before measuring gains.
• mun-lang/lld-rs is prior art, not the base (LLVM-14 era, links every driver).

Track A — Windows native COFF

The active chain. Most of the current motion is in the zackees/llvm-ld repo, which already has merged bootstrap work and its own open issues (llvm-ld#6 Tier 2 LTO corpus, llvm-ld#15 remaining link-speed work).

• Research: Windows PE/COFF fast-linking strategy - optimize LLD-COFF fork vs retrofit COFF into Wild #91 — research complete and the decision is recorded in its final comment. Audit and close, so the decision stops looking open.
• Phase 2: bootstrap zackees/llvm-ld as a minimal COFF C++ library with mimalloc-pprof #95 — Phase 2: bootstrap zackees/llvm-ld as a minimal COFF C++ library with mimalloc-pprof. Partly delivered in that repo; reconcile this issue with what has merged there and either close it or list the remainder.
• Phase 3: add reld-owned Rust bindings and route Windows through vendored llvm-ld #96 — Phase 3: reld-owned Rust bindings over the C ABI, vendored _vendor/llvm-ld import, and route Windows through it while keeping the pinned lld-link oracle and escape hatch.
• P5: MSVC ABI + resolve the PDB decision (D5) #9 — P5: MSVC ABI and the PDB decision (Meta: road to a working cross-platform link (Windows / Linux / macOS) #2 D5). A dev-loop linker whose output cannot be debugged is not a dev-loop linker, so D5 needs an answer before this starts, not during.
• P3: PE/COFF backend, MinGW (windows-gnu) ABI #7 — P3: PE/COFF backend, MinGW ABI. Open question: its original premise (retrofit COFF into wild) is superseded by the Research: Windows PE/COFF fast-linking strategy - optimize LLD-COFF fork vs retrofit COFF into Wild #91 decision. Re-scope it to the MinGW/windows-gnu ABI work that the llvm-ld route still needs, or close it as superseded. It should not sit open implying a second, competing Windows implementation.

Track B — macOS native Mach-O

• P4: Mach-O backend, aarch64-apple-darwin #8 — P4: Mach-O backend for aarch64-apple-darwin. Bridged to ld64.lld today, so this is a performance and independence goal, not a functionality gap. Note Meta: road to a working cross-platform link (Windows / Linux / macOS) #2 D6: ad-hoc code signing is mandatory on Apple Silicon, since unsigned arm64 Mach-O binaries do not execute.

Track C — Incremental linking

This is the product thesis, and it is deliberately last.

• DECISION: two contradictions in DESIGN.md that block incremental #10 — resolve the two DESIGN.md contradictions that block incremental. This is a decision, not an implementation, and it is cheap; doing it early stops the contradictions propagating into Track A/B design choices.
• I0-I7: Incremental linking (do not start before P4 is green) #11 — I0–I7. Its own text says: do not start before P4 (P4: Mach-O backend, aarch64-apple-darwin #8) is green.

Track D — Linker daemon

• Linker daemon: thin reld-link client + broker-versioned daemon holding the object graph (running-process broker v2, zccache-embedded instrumentation) #19 — thin reld-link client plus a broker-versioned daemon holding the object graph, with zccache-embedded instrumentation. Mostly orthogonal to Tracks A–C, but it wants a stable object graph underneath it, so it follows Linux being proven (P2: Prove Linux — self-host, corpus pass rate, populate the benchmark column #6) rather than preceding it. Open question: is this still wanted now that soldr drives reld as a subprocess, or does the daemon belong to soldr's side of the boundary?

Sequencing

1. meta: burn-down — unblock soldr's default linker (#123) and clear the debt around it #153 first. The near-term gate (Polylinker no-gaps gate: classify every flag, route by target not host, prove coverage before reld becomes soldr's default #123) is what unblocks a real user (soldr), and it keeps the native engine honest, which every later track depends on.
2. Two exceptions may run in parallel: DECISION: two contradictions in DESIGN.md that block incremental #10, because it is a decision and its absence corrupts later design; and the llvm-ld repo work, which is a separate repository with its own CI and is already moving.
3. Then Track A (Windows is the largest user-visible gap and already has a decided strategy), then Track B, then Track C, with Track D slotted once Linux is proven.

Open questions for the owner

Answered: Q2 — #7 closed as superseded. Q3 — the daemon stays in the product, gated on #6. Q4 — Track B stays a goal, gated after Track A. D5 (PDB) — via llvm-ld. Still open: Q1 (#2) and #10.

1. Close Meta: road to a working cross-platform link (Windows / Linux / macOS) #2, or rewrite its status section?
2. Is P3: PE/COFF backend, MinGW (windows-gnu) ABI #7 superseded by the Research: Windows PE/COFF fast-linking strategy - optimize LLD-COFF fork vs retrofit COFF into Wild #91 LLD-COFF decision, or re-scoped to MinGW ABI?
3. Is the daemon (Linker daemon: thin reld-link client + broker-versioned daemon holding the object graph (running-process broker v2, zccache-embedded instrumentation) #19) still in the product, given soldr already drives reld as a subprocess?
4. Does Track B (P4: Mach-O backend, aarch64-apple-darwin #8) stay a goal at all, or is ld64.lld bridging the permanent macOS answer? The same question will eventually apply to Track A once llvm-ld is fast.

Checklist

• Research: Windows PE/COFF fast-linking strategy - optimize LLD-COFF fork vs retrofit COFF into Wild #91 audit and close — CLOSED; decision recorded, route built in llvm-ld
• Phase 2: bootstrap zackees/llvm-ld as a minimal COFF C++ library with mimalloc-pprof #95 reconcile with merged llvm-ld work — CLOSED; every requirement delivered in llvm-ld (evidence on the issue)
• Phase 3: add reld-owned Rust bindings and route Windows through vendored llvm-ld #96 Rust bindings and Windows routing — gated on meta: burn-down — unblock soldr's default linker (#123) and clear the debt around it #153 (owner decision); needs dependency approval when it starts
• P5: MSVC ABI + resolve the PDB decision (D5) #9 MSVC ABI — D5 decided: PDB via llvm-ld (LLD's writer, byte-identical); follows Phase 3: add reld-owned Rust bindings and route Windows through vendored llvm-ld #96
• P3: PE/COFF backend, MinGW (windows-gnu) ABI #7 — CLOSED as superseded by the Research: Windows PE/COFF fast-linking strategy - optimize LLD-COFF fork vs retrofit COFF into Wild #91 LLD-COFF route (MinGW via llvm-ld / Phase 3: add reld-owned Rust bindings and route Windows through vendored llvm-ld #96)
• P4: Mach-O backend, aarch64-apple-darwin #8 Mach-O backend — kept, gated after Track A (owner decision)
• DECISION: two contradictions in DESIGN.md that block incremental #10 DESIGN contradictions resolved
• I0-I7: Incremental linking (do not start before P4 is green) #11 incremental (gated on P4: Mach-O backend, aarch64-apple-darwin #8)
• Linker daemon: thin reld-link client + broker-versioned daemon holding the object graph (running-process broker v2, zccache-embedded instrumentation) #19 daemon — kept, gated on P2: Prove Linux — self-host, corpus pass rate, populate the benchmark column #6 (owner decision)