Make GPUCompiler IR relocatable across Julia sessions by maleadt · Pull Request #878 · JuliaGPU/GPUCompiler.jl

maleadt · 2026-07-14T13:33:42Z

Keep Julia values and libjulia runtime globals as symbolic relocations until final backend lowering, instead of persisting session-specific host addresses in IR or cached code. Relocations can target Julia object identities or named runtime globals, through either dedicated pointer-sized slots or words embedded in device-side object representations.

Relocation metadata remains associated with the IR through linking and optimization, with stable, collision-safe slot names. Backends may preserve relocations for their loader to resolve and patch at runtime, while the default lowering eagerly embeds addresses from the current Julia session.

CUDA.jl uses this in JuliaGPU/CUDA.jl#3200 to materialize relocations as CuGlobal slots populated at load time, allowing generated PTX to be cached across Julia sessions. Other backends retain the existing eager behavior.

Supersedes #125 and #348

codecov · 2026-07-14T13:36:59Z

Codecov Report

❌ Patch coverage is 88.93281% with 28 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.13%. Comparing base (4a4be52) to head (19091ac).

Files with missing lines	Patch %	Lines
src/jlgen.jl	84.34%	18 Missing ⚠️
src/validation.jl	81.57%	7 Missing ⚠️
src/interface.jl	85.71%	2 Missing ⚠️
src/mcgen.jl	97.14%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #878      +/-   ##
==========================================
+ Coverage   81.75%   82.13%   +0.37%     
==========================================
  Files          26       26              
  Lines        4879     5032     +153     
==========================================
+ Hits         3989     4133     +144     
- Misses        890      899       +9

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

maleadt · 2026-07-17T12:57:24Z

Okay, this is starting to look good. IR should be fully relocatable after this, at least for back-ends supporting run-time relocations.

vchuravy · 2026-07-20T11:51:00Z

+            # The pass builder resolves this name to the registered instance, which
+            # captures the Relocations object owned by optimize!.
+            add!(mpm, "GPULinkRuntime")
            add!(mpm, GPULinkLibrariesPass(job))
            add!(mpm, GPUFinishRuntimeIntrinsicsPass(job))


Should we also do this for the job capturing?

You mean make them all by-name references?

vchuravy · 2026-07-20T12:01:06Z

+function resolve_relocation_target(target::JuliaValueRef)
+    box = Any[target.value]
+    GC.@preserve box begin
+        return unsafe_load(Base.unsafe_convert(Ptr{UInt}, pointer(box)))
+    end
+end


I think the previous version of this uses jl_value_ptr directly?

Why the 1-element array instead of a Ref?

We had something similar in Enzyme, but we restricted it to only types. https://github.com/EnzymeAD/Enzyme.jl/blob/4c8b1c0a04689cce931bd4ad8aa02f6a0919cecc/src/utils.jl#L1-L52

I think for all other cases one can use pointer_from_objref.

Why the 1-element array instead of a Ref?

Fable is convinced that jl_value_ptr on 1.14 can allocate temporary stack boxes and return its address, while Any[] forces the contents to be heap-allocated too.

@noinline jvp(x) = UInt(ccall(:jl_value_ptr, Ptr{Cvoid}, (Any,), x)) @noinline slot(x) = (s = Any[x]; GC.@preserve s unsafe_load(Base.unsafe_convert(Ptr{UInt}, pointer(s)))) tag(a) = unsafe_load(Ptr{UInt}(a - 8)) & ~UInt(15) const F64_TAG = jvp(Float64) # heap-interned DataType: fine either way @noinline clobber(n) = n == 0 ? UInt(0) : clobber(n - 1) + hash(n) function demo() for (name, a) in ("jl_value_ptr" => jvp(1.25), "Any[] slot" => slot(1.25)) clobber(200) # reuse the stack region the callee frame occupied println(rpad(name, 13), ": contents = ", rpad(unsafe_load(Ptr{Float64}(a)), 23), " valid type tag = ", tag(a) == F64_TAG) end end demo()

❯ jl +nightly mwe.jl jl_value_ptr : contents = 6.92391611475455e-310 valid type tag = false Any[] slot : contents = 1.25 valid type tag = true

vchuravy

One thought is around GC safety. Julia currently throws everything that is reachable from a native compilation into a global roots list (this has changed over time, I think 1.10 still had per CodeInstance rooting) so the thing I am wondering about is: Could we lose a root by going through this?

maleadt · 2026-07-20T12:47:56Z

Another suggestion by @vchuravy: Try this out on AllocCheck.jl

Add a `relocation_lowering(job)` trait choosing `:bake` (default), `:patch`, or `:import`. `:bake` resolves host references into the IR eagerly during `emit_llvm` (where `relocate_gvs!` used to run), so the `:llvm` result is execution-ready and back-ends that drive `emit_asm(job, ir, format)` directly (Metal.jl) keep working; a restored three-argument `emit_asm` forwards to the metadata-carrying method. Non-baking strategies keep references symbolic for their loader and lower them at object emission. `lower_relocations!` becomes an internal dispatcher on the trait, subsuming the per-back-end override; the previously test-local import lowering moves in as `emit_imported_relocations!`. Also add `Base.copy(::Relocations)` for loaders that bake from cached metadata. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Add a `library::Union{Nothing,String}` field to `CGlobalRef`. The default `nothing` keeps the current behavior (libjulia global via `jl_cglobal`'s process-wide lookup); an explicit library is loaded and searched with Libdl. Both fields are plain data, so serialized relocation metadata stays portable. Drop the `String(symbol)` conversion in resolution: `jl_cglobal` accepts the symbol directly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Only GPULinkRuntime is added by name (to reuse the instance holding the Relocations owned by optimize!); the sibling passes are stateless and constructed fresh. Spell that out at the call site. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Whether relocations are exposed symbolically is the runtime capability `supports_relocatable_ir()` (jl_get_llvm_gvs_globals was backported to 1.11/1.12 point releases), not a fixed version. Replace the `VERSION >=` gates in the relocation tests with the probe. The tests that observe symbolic slots surviving to `:llvm` now compile with a non-baking back-end (native `jit=true`, a new PTX `patch=true`), since a baking job resolves and folds those slots away before the output. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Add coverage for the paths the strategy trait introduces: - eager `:bake` default: `compile(:llvm)` leaves no relocation metadata; - `:patch`: load an object, patch every site at `global + offset`, execute, and check the null-init / extinit / llvm.used shape (native, plus a PTX `.global` assertion); - validation error paths in `foreach_relocation` and `link_relocatable!`, and a direct `prune_dead_relocations!` test; - a deferred (Mock Enzyme) child whose relocations must merge into the parent. The native test back-end gains a `:patch` mode alongside its `:import` JIT mode. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

The relocation metadata now surfaced through `compile`/`emit_asm` and the `relocation_lowering` trait is additive: back-ends that don't opt in keep the prior baking behavior. Minor bump. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

`jl_get_llvm_gvs_globals` was backported to 1.10, so `HAS_LLVM_GVS_GLOBALS` alone reports 1.10 as relocation-capable. But 1.10's codegen still bakes host references inline (as `inttoptr` constants) in the JIT (non-imaging) mode we compile in, rather than emitting the relocatable global declarations the relocation machinery collects, so no relocation sites are ever produced. Only 1.11+ emits those declarations, so gate the predicate on the version too. Fixes the relocation testset failures in native.jl and ptx.jl on 1.10, which all run under `if supports_relocatable_ir()`. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

An all-zero box -- a patchable header over an all-zero payload, as produced for `SVector(0f0, 0f0)` and friends -- is folded by LLVM to a `zeroinitializer`. That ConstantAggregateZero reports no operands, so `bake_relocations!` built an empty field vector and threw `BoundsError` when writing the resolved header word (JuliaGPU/oneAPI.jl "#55: invalid integers created by alloc_opt"). Rebuild the explicit per-field constants from the struct's element types when the initializer is a ConstantAggregateZero, and add a regression test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

maleadt · 2026-07-23T14:06:13Z

AllocCheck.jl integration: JuliaLang/AllocCheck.jl#113
Uses a new relocation strategy, :defer, since AllocCheck.jl really does its own linking. This will be similar to what Enzyme.jl will have to do if it wants to benefit from this. I left the :import strategy in place, with an example user in the tests.

@vchuravy I think this is ready for another look.

maleadt requested a review from vchuravy July 14, 2026 13:33

maleadt mentioned this pull request Jul 14, 2026

Use relocatable IR linked at load time JuliaGPU/CUDA.jl#3200

Open

vchuravy reviewed Jul 14, 2026

View reviewed changes

Comment thread src/interface.jl Outdated

maleadt force-pushed the tb/relocations branch from f6b05c4 to 19091ac Compare July 15, 2026 17:09

maleadt marked this pull request as ready for review July 15, 2026 20:34

maleadt force-pushed the tb/relocations branch from e9ef411 to 1b6d16f Compare July 16, 2026 19:02

vchuravy reviewed Jul 20, 2026

View reviewed changes

Comment thread src/relocation.jl

vchuravy reviewed Jul 20, 2026

View reviewed changes

Comment thread src/relocation.jl Outdated

vchuravy reviewed Jul 20, 2026

View reviewed changes

maleadt added 16 commits July 23, 2026 10:16

Make GPUCompiler IR relocatable across Julia sessions

c33b0f5

Mark resolved host-reference slots constant

1c83258

Harden JuliaValueRef resolution against re-boxing

6d21d0f

Clean up host-reference comments

1a984d9

Emit runtime-global slots as declarations until lowering

40e0fe6

Add declaration-based host-reference lowering for JIT loaders

c51756a

Test postponed relocation with an ORC loader in the native back-end

42be656

Clarify host-reference lowering names

c629367

Use JIT target machines for ORC relocation tests

0c918e6

Make postponed relocation probe ABI-independent

fc413a3

Name the native test back-end JIT mode explicitly

8093be9

Consolidate host-reference relocation lifecycle

ec7978f

Assign host-reference slot identities at creation

ddd8631

Run host-reference lowering once at object emission

843efb2

Keep declaration-lowered host references uniform

3fe49cb

Centralize host-reference slot validation

524fb5d

maleadt and others added 21 commits July 23, 2026 10:16

Stop threading relocation state through pipeline builders

0ea0776

Simplify runtime-global reference collection

1d72200

Mark LLVM results after runtime linking

2756185

Name both host-reference producers consistently

540ebf2

Represent all host pointers as relocations

2852110

Resolve whole and interior relocations for loaders

bc23597

Test patchable box visibility in PTX IR

a80f054

Remove obsolete cache-wiping test references

663c014

Remove unused deprecation.

df42587

Adopt relocation terminology

39e532b

Unify relocation site metadata

09bd2c3

Narrow the relocation lowering API

780c4c9

Clean-ups.

59bc80c

Account for Julia 1.10 boxed constants in PTX tests

6e57ef6

Test tweaks.

36d95ce

maleadt force-pushed the tb/relocations branch from 4afb533 to 36d95ce Compare July 23, 2026 11:17

Defer and new helper.

8c6925e

maleadt mentioned this pull request Jul 23, 2026

Cache relocateable code. JuliaLang/AllocCheck.jl#113

Draft

maleadt and others added 5 commits July 23, 2026 14:10

Reduce exports.

d3e91e9

Tighten relocatable IR caching and documentation

a0261f3

Clarify result cache naming

44c1e41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Make GPUCompiler IR relocatable across Julia sessions#878

Make GPUCompiler IR relocatable across Julia sessions#878
maleadt wants to merge 44 commits into
mainfrom
tb/relocations

maleadt commented Jul 14, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Jul 14, 2026 •

edited

Loading

Uh oh!

Uh oh!

maleadt commented Jul 17, 2026

Uh oh!

vchuravy Jul 20, 2026

Uh oh!

maleadt Jul 23, 2026

Uh oh!

Uh oh!

vchuravy Jul 20, 2026

Uh oh!

maleadt Jul 23, 2026

Uh oh!

Uh oh!

vchuravy left a comment

Uh oh!

maleadt commented Jul 20, 2026

Uh oh!

maleadt commented Jul 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

maleadt commented Jul 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Jul 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

maleadt commented Jul 17, 2026

Uh oh!

vchuravy Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

maleadt Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

vchuravy Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

maleadt Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

vchuravy left a comment

Choose a reason for hiding this comment

Uh oh!

maleadt commented Jul 20, 2026

Uh oh!

maleadt commented Jul 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

maleadt commented Jul 14, 2026 •

edited

Loading

codecov Bot commented Jul 14, 2026 •

edited

Loading