You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Research: Windows (PE/COFF) fast-linking strategy — fork/optimize LLD-COFF vs retrofit COFF into Wild
Purpose
Decide, with evidence, which of two strategies reld pursues to dramatically improve Windows link performance. This issue collects the research; the decision will be recorded here once the research phases (appended as comments) are complete. macOS is explicitly out of scope for this cycle. Windows only.
The two candidate strategies
Option 1 — Fork LLD (lld-link / rust-lld) and optimize its COFF driver
Optimize for non-LTO, thin-LTO, and full-LTO regimes.
Pro: lld-link already links Windows correctly, covering the full flag/feature surface. Every optimization is testable with a brutal, cheap oracle: does the optimized path produce a byte-identical deterministic binary vs the unoptimized path? Low bring-up risk.
Con: presumably a lower performance ceiling — we inherit lld's architecture (memory model, serialization points, PDB pipeline).
Option 2 — Retrofit PE/COFF into the Wild linker
Wild is dramatically faster on ELF, which makes it the tempting choice.
Pro: highest performance ceiling; modern parallel architecture; written in Rust.
Con: Wild is ELF-only today. Windows linking has a long tail of machinery with no ELF analog (PDB/CodeView, import libraries, dllimport thunks, SEH/.pdata, /GUARD:CF, resources, manifests, delay-load). Bring-up could burn days-to-weeks of effort and heavy token spend before we even learn whether a blocker (most likely PDB) is fatal. Byte-identity testing vs lld is impossible by construction — a different linker produces different bytes — so the test oracle is weaker (behavioral equivalence).
Fixed constraints (decided up front)
Both toolchain flavors are in scope: MSVC (link.exe-style, *-pc-windows-msvc) and GNU/MinGW (*-pc-windows-gnu). (soldr preferred MSVC early on; we've since evolved — both now.)
This is a general-purpose C/C++/Rust linker. Linkers are language-agnostic: rustc emits ordinary COFF objects and .rlibs (plain ar archives whose .rmeta member a linker must skip — standard archive handling). No Rust-specific format support exists or is needed; "Rust support" = correct COFF/archive handling + accepting the flags rustc passes. (Research must verify this claim, not assume it.)
All link modes matter: non-LTO (dev inner loop), thin-LTO, full-LTO. Once the design is right, an agent grinds each mode until it is as fast as possible.
Full link.exe flag/feature compatibility is mandatory — the whole surface (/WHOLEARCHIVE, /ALTERNATENAME, /EXPORT, /DEF, /DELAYLOAD, /GUARD:CF, /DEBUG + PDB, def files, manifests, resources, …). Sole MVP exception: /INCREMENTAL may be deferred. Everything else is required.
CRT selection /MT vs /MD is super important and must be handled correctly (defaultlib resolution, /NODEFAULTLIB, mixed-CRT diagnostics).
Zig's self-hosted COFF linker is potentially super important prior art — their bring-up experience (esp. PDB) is direct evidence for the Wild-retrofit cost estimate.
Research questions
A. Scoping / baseline
A1. Where does Windows link time actually go today (lld-link vs link.exe; symbols vs no symbols; LTO vs not)? Profile evidence, not vibes. If PDB emission dominates, that reshapes everything.
A2. Verify constraint Meta: road to a working cross-platform link (Windows / Linux / macOS) #2: confirm rustc's linker contract on windows-msvc and windows-gnu (flags passed, rlib handling, self-contained mode, -fuse-ld/linker-flavor mechanics). Is there truly zero Rust-specific linker work?
A3. In thin/full LTO, is link time dominated by LLVM codegen inside the LTO plugin rather than link mechanics? If yes, does linker choice even matter for LTO builds, meaning the decision should be driven by the non-LTO dev loop?
B. LLD-fork path
B1. How fast is lld-link today vs GNU-ld/mold/wild on comparable inputs — how much headroom exists?
B2. Where does lld-link spend time (symbol resolution, /ICF, PDB emission, LTO)? What has upstream already parallelized (e.g. parallel PDB type merging, ghash)? What serialization points remain?
B3. Is lld-link deterministic/reproducible by default (/Brepro, timestamps, PDB GUIDs)? The byte-identity oracle depends on it.
B4. Fork maintenance: COFF-driver churn rate upstream; can optimizations be upstreamed instead of forked; how hard is shipping a patched lld to Rust/C++ users?
B5. What did Rust's "lld by default on Windows" efforts learn? Known blockers?
C. Wild-retrofit path
C1. What fraction of Wild is format-generic (symbol resolution, layout, parallelism, string merging) vs hard-wired ELF (relocations, section model, dynamic linking, TLS)? Is there any abstraction boundary today, or is ELF baked into core types? This is the load-bearing feasibility question.
C2. Has Wild upstream stated any position on COFF/PE (or Mach-O) support? Would they accept a port, or is this a hard fork of a fast-moving project?
C3. Enumerate the COFF machinery with no ELF analog that must be written from scratch: import/export tables + import libraries, dllimport thunks, SEH/.pdata unwind, safeseh, /GUARD:CF metadata, resources (.res), manifest embedding, delay-load, TLS callbacks, base relocations, and — the monster — PDB/CodeView writing. Estimated cost of each; which can be borrowed from existing Rust crates or LLVM?
C4. Where does Wild's speed actually come from — and does the advantage survive once COFF's costs (esp. PDB) are added? Risk: weeks of porting to land at lld-speed anyway.
C5. What is Wild's correctness/test story, and what would our COFF test oracle be given byte-identity vs lld is impossible?
C6. Licensing of Wild vs LLVM (Apache-2.0 WITH LLVM-exception) — any shipping constraint?
D. Prior art & third options
D1. Zig's self-hosted COFF linker: current status, what works, what stalled, PDB story, lessons on effort. Direct evidence for Option 2's cost.
D2. mold/sold history: mold's author split macOS support into a separate product (sold) and later abandoned it. What does that say about cross-format code sharing in a single-format fast linker?
D3. Any other fast-COFF efforts (e.g. Microsoft's link.exe improvements, radlink, others)?
D4. Is there a hybrid/staged option: optimize lld now (ships this cycle) while a strictly time-boxed Wild-COFF spike answers C1–C4? GNU-flavor-first as a stepping stone (avoids some MSVC machinery)?
E. Decision criteria & kill criteria (to be finalized before any build starts)
E1. Estimated bring-up cost to a minimum viable milestone per path: links a real Rust hello-world + one mid-size real C++/Rust project on windows-msvc and windows-gnu; binary runs; tests pass.
E2. Written kill criteria per path (e.g. Wild: "if PDB or import-library handling has no feasible design within N effort, stop"). Purpose: never discover infeasibility at the bottom of a rabbit hole.
E3. Is byte-for-byte reproducibility a product requirement for reld (content-addressed caching) or only a testing convenience? If product: heavy thumb on the scale for lld.
Known corner-case watchlist (adversarial pass will expand this)
PDB writing (the classic underestimated monster) · /INCREMENTAL expectations (deferred for MVP but users assume it) · full link.exe flag pass-through surface · dllimport-from-static-lib fixups · CRT selection /MT vs /MD and defaultlib resolution · import library flavors (short vs long format, MinGW .dll.a) · ARM64/ARM64EC · authenticode signing space · antivirus/mmap I/O behavior on Windows · string/COMDAT merging semantics differences · /Brepro + PDB GUID determinism.
Process
Opus research agents gather evidence per section (LLD-COFF, Wild internals, Rust linker contract, Zig/mold prior art, link.exe compatibility surface). Each phase's findings are appended to this issue as a comment with links.
Fable reads all findings and runs an adversarial/follow-up pass (corner cases, feasibility gaps); follow-up research appended likewise.
Decision recorded as a final comment on this issue: chosen option, rationale, staged plan, kill criteria.
Research: Windows (PE/COFF) fast-linking strategy — fork/optimize LLD-COFF vs retrofit COFF into Wild
Purpose
Decide, with evidence, which of two strategies reld pursues to dramatically improve Windows link performance. This issue collects the research; the decision will be recorded here once the research phases (appended as comments) are complete. macOS is explicitly out of scope for this cycle. Windows only.
The two candidate strategies
Option 1 — Fork LLD (
lld-link/ rust-lld) and optimize its COFF driverOptimize for non-LTO, thin-LTO, and full-LTO regimes.
Option 2 — Retrofit PE/COFF into the Wild linker
Wild is dramatically faster on ELF, which makes it the tempting choice.
Fixed constraints (decided up front)
link.exe-style,*-pc-windows-msvc) and GNU/MinGW (*-pc-windows-gnu). (soldr preferred MSVC early on; we've since evolved — both now.).rlibs (plainararchives whose.rmetamember a linker must skip — standard archive handling). No Rust-specific format support exists or is needed; "Rust support" = correct COFF/archive handling + accepting the flags rustc passes. (Research must verify this claim, not assume it.)link.exeflag/feature compatibility is mandatory — the whole surface (/WHOLEARCHIVE,/ALTERNATENAME,/EXPORT,/DEF,/DELAYLOAD,/GUARD:CF,/DEBUG+ PDB, def files, manifests, resources, …). Sole MVP exception:/INCREMENTALmay be deferred. Everything else is required./MTvs/MDis super important and must be handled correctly (defaultlib resolution,/NODEFAULTLIB, mixed-CRT diagnostics).Research questions
A. Scoping / baseline
windows-msvcandwindows-gnu(flags passed, rlib handling, self-contained mode,-fuse-ld/linker-flavor mechanics). Is there truly zero Rust-specific linker work?B. LLD-fork path
/Brepro, timestamps, PDB GUIDs)? The byte-identity oracle depends on it.C. Wild-retrofit path
D. Prior art & third options
E. Decision criteria & kill criteria (to be finalized before any build starts)
Known corner-case watchlist (adversarial pass will expand this)
PDB writing (the classic underestimated monster) ·
/INCREMENTALexpectations (deferred for MVP but users assume it) · full link.exe flag pass-through surface · dllimport-from-static-lib fixups · CRT selection/MTvs/MDand defaultlib resolution · import library flavors (short vs long format, MinGW.dll.a) · ARM64/ARM64EC · authenticode signing space · antivirus/mmap I/O behavior on Windows · string/COMDAT merging semantics differences ·/Brepro+ PDB GUID determinism.Process
🤖 Generated with Claude Code