aot: make precompiled modules run without libxls#962
Closed
dank-openai wants to merge 3 commits into
Closed
Conversation
# Problem Solved Runtime consumers that only execute precompiled XLS modules should not need to ship the compiler and JIT runtime after those modules have already been compiled. The existing generated wrappers still instantiate the heavyweight execution stack, so copied runtime artifacts retain an avoidable dependency on `xlsynth`, `xlsynth-sys`, `libxls`, and LLVM even when the AOT object code is already linked into the final binary. # Mental model `xlsynth` remains the build-time compiler. At runtime, a precompiled module is represented by its linked object code, generated wrapper glue, and a small runtime-only support layer that knows how to marshal buffers and surface assertion events. Compiler and JIT-capable tools can continue to use the full runtime path when they need it. # Non-goals - This does not add standalone support for runtime `trace` or `cover` callbacks. - This does not promise cross-version compatibility for generated artifacts. - This does not remove the heavyweight runtime path from tools that genuinely need compiler or JIT features. # Tradeoffs - Runtime assertions remain part of the first standalone artifact contract because current shipped AOT inputs already depend on them. - Reachable `trace` and `cover` usage is rejected during generation instead of being accepted and failing later through unsupported callbacks. - Generated wrapper source is canonicalized by the generator so artifact bytes do not depend on whichever host `rustfmt` binary happened to run after emission. # Architecture - Add `xlsynth-aot-runtime` for artifact-agnostic runner support that can be linked directly into runtime consumers. - Make generated AOT wrappers standalone by default and keep their artifact-specific glue separate from reusable runtime support. - Carry explicit ABI metadata in each artifact, preserve assertion delivery, and scope feature legality checks to the selected top function closure rather than unrelated dead siblings. # Observability Generated wrappers expose runtime-visible artifact metadata, including ABI version and supported feature flags, and keep assertion messages available through the standalone runner APIs. Unsupported callback-bearing artifacts now fail during generation with explicit errors instead of reaching a latent runtime abort path. # Tests Validation covered standalone artifact generation, runtime execution, generated-source stability, and downstream integration behavior: - `cargo test -p xlsynth-aot-standalone-test-crate --tests --no-fail-fast` - `cargo test -p xlsynth-aot-test-crate --tests --no-fail-fast` - `cargo test --manifest-path xlsynth/tests/aot-test-crate/Cargo.toml generated_wrappers_match_golden_references` Focused downstream integration validation passed on both macOS and Ubuntu 24.04. Copied-artifact validation on both platforms also confirmed that representative runtime carriers no longer had a runtime `libxls` dependency. pr:standalone-runtime-aot
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5df7abff5f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
dank-openai
force-pushed
the
dank-spr/standalone-runtime-aot
branch
2 times, most recently
from
acc6c7e to
a28d4b2
Compare
Preserve the standalone allocator callback contract for generated heap allocas by routing over-aligned requests through posix_memalign while keeping the malloc fast path for ordinary alignments.
Cover heap-scratch allocator behavior directly, exercise a generated counted_for artifact above the LLVM stack-allocation cutoff, and compare standalone execution with the legacy runner for direct-call and assertion-bearing artifacts.
dank-openai
force-pushed
the
dank-spr/standalone-runtime-aot
branch
from
9e14e19 to
42af3a5
Compare
dank-openai
changed the title
Contributor
Author
|
Closing as superseded by #964. This draft was the initial standalone runtime-AOT prototype. The follow-up design moved runtime ABI ownership into XLS itself and replaced the Rust-owned shadow runtime with thin integration against the XLS-owned runtime archive. That successor path landed through #964 and the later rollout PRs. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Generate standalone wrappers for precompiled AOT modules and move reusable execution support into
xlsynth-aot-runtime. That lets runtime consumers execute linked object code without carrying thecompiler/JIT stack or an accompanying
libxlsDSO.Problem Solved
Runtime consumers that only execute precompiled XLS modules should not need to ship the compiler and JIT runtime after those modules have already been compiled. The existing generated wrappers still instantiate the heavyweight execution stack, so copied runtime artifacts retain an avoidable dependency on
xlsynth,xlsynth-sys,libxls, and LLVM even when the AOT object code is already linked into the final binary.Mental model
xlsynthremains the build-time compiler. At runtime, a precompiled module is represented by its linked object code, generated wrapper glue, and a small runtime-only support layer that knows how to marshal buffers and surface assertion events. Compiler and JIT-capable tools can continue to use the full runtime path when they need it.Non-goals
traceorcovercallbacks.Tradeoffs
traceandcoverusage is rejected during generation instead of being accepted and failing later through unsupported callbacks.rustfmtbinary happened to run after emission.Architecture
xlsynth-aot-runtimefor artifact-agnostic runner support that can be linked directly into runtime consumers.Observability
Generated wrappers expose runtime-visible artifact metadata, including ABI version and supported feature flags, and keep assertion messages available through the standalone runner APIs. Unsupported callback-bearing artifacts now fail during generation with explicit errors instead of reaching a latent runtime abort path.
Tests
Validation covered standalone artifact generation, runtime execution, generated-source stability, and downstream integration behavior:
cargo test -p xlsynth-aot-standalone-test-crate --tests --no-fail-fastcargo test -p xlsynth-aot-test-crate --tests --no-fail-fastcargo test --manifest-path xlsynth/tests/aot-test-crate/Cargo.toml generated_wrappers_match_golden_referencesFocused downstream integration validation passed on both macOS and Ubuntu 24.04. Copied-artifact validation on both platforms also confirmed that representative runtime carriers no longer had a runtime
libxlsdependency.Stack: