Skip to content

Commit 71afde3

Browse files
created/adapted documentation refered to fast llvm extraction and usage
1 parent cf9b610 commit 71afde3

3 files changed

Lines changed: 154 additions & 1 deletion

File tree

‎README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@ validates the setup, and how QNX-specific authentication and licensing fit in.
3131
- [Overview](docs/overview.md)
3232
- [Repository layout](docs/repository_layout.md)
3333
- [Extension API](docs/extension_api.md)
34+
- [Fast LLVM extraction (opt-in)](docs/fast_llvm_repo.md)
3435
- [Generation flow](docs/generation_flow.md)
3536
- [Toolchain features](docs/features.md)
3637
- [Migration guide (explicit-feature model)](docs/migration_guide.md)

‎docs/extension_api.md‎

Lines changed: 55 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -182,4 +182,58 @@ behavior explicitly, see the
182182
`gcc.toolchain(...)`. See [Toolchain features](features.md#sanitizers-linux-opt-in)
183183
for the injected feature names.
184184
- QNX toolchains use additional licensing and include-path parameters that do
185-
not apply to Linux toolchains.
185+
not apply to Linux toolchains.
186+
187+
## LLVM Extension (Optional, Faster Alternative)
188+
189+
The `llvm` module extension in
190+
`@score_bazel_cpp_toolchains//extensions:llvm.bzl` is an optional, opt-in
191+
alternative for consumers who already use (or plan to use)
192+
[`toolchains_llvm`](https://github.com/bazel-contrib/toolchains_llvm) and want
193+
faster LLVM archive extraction. It is not a replacement for `toolchains_llvm`
194+
in general — it only swaps the repository that provides the LLVM
195+
distribution. See [Fast LLVM repository](fast_llvm_repo.md) for the
196+
rationale and measured performance numbers (~220s → ~70s).
197+
198+
### Consumer Entry Point
199+
200+
```starlark
201+
bazel_dep(name = "score_bazel_cpp_toolchains", version = "0.5.4")
202+
203+
llvm = use_extension("@score_bazel_cpp_toolchains//extensions:llvm.bzl", "llvm")
204+
llvm.toolchain(
205+
name = "llvm_toolchain",
206+
version = "19.1.1",
207+
)
208+
209+
use_repo(llvm, "llvm_toolchain", "llvm_toolchain_pkg")
210+
```
211+
212+
Consumers do not need their own `bazel_dep(toolchains_llvm)`:
213+
`score_bazel_cpp_toolchains` already depends on `toolchains_llvm`, and the
214+
extension loads it internally.
215+
216+
### `llvm.toolchain(...)` Attributes
217+
218+
- `name` (mandatory): name of the generated LLVM toolchain repository. The
219+
companion package repository (produced by `fast_llvm_repo`) is generated
220+
alongside it as `<name>_pkg`.
221+
- `version` (mandatory): LLVM version to extract. Must be one of the versions
222+
pinned in [`fast_llvm_repo.bzl`](../rules/fast_llvm_repo.bzl).
223+
224+
### Activation In A Workspace
225+
226+
```text
227+
--extra_toolchains=@llvm_toolchain//:<toolchain-target>
228+
```
229+
230+
### Behavior Notes
231+
232+
- The extension is intended for the root module only, the same restriction as
233+
`gcc`.
234+
- It is an alternative, not a replacement, for `toolchains_llvm`.
235+
- Only the versions and architectures pinned in `fast_llvm_repo.bzl` are
236+
supported; requesting any other version fails at package-fetch time.
237+
- Each `llvm.toolchain(...)` tag creates its own package repo. There is no
238+
sharing or deduplication across multiple toolchain tags requesting the same
239+
version — this is intentionally kept simple.

‎docs/fast_llvm_repo.md‎

Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,98 @@
1+
<!--
2+
# *******************************************************************************
3+
# Copyright (c) 2026 Contributors to the Eclipse Foundation
4+
#
5+
# See the NOTICE file(s) distributed with this work for additional
6+
# information regarding copyright ownership.
7+
#
8+
# This program and the accompanying materials are made available under the
9+
# terms of the Apache License Version 2.0 which is available at
10+
# https://www.apache.org/licenses/LICENSE-2.0
11+
#
12+
# SPDX-License-Identifier: Apache-2.0
13+
# *******************************************************************************
14+
-->
15+
16+
# Fast LLVM repository
17+
18+
## Purpose
19+
20+
`fast_llvm_repo` creates the prebuilt LLVM distribution repository consumed by
21+
[`toolchains_llvm`](https://github.com/bazel-contrib/toolchains_llvm). It is a
22+
performance-oriented alternative to the repository that `toolchains_llvm`
23+
normally generates internally.
24+
25+
The rule is useful when LLVM setup is a significant part of the developer or
26+
CI feedback time. In the measurements that motivated it, LLVM setup decreased
27+
from roughly 220 seconds to 70 seconds. The exact result depends on the host;
28+
the rule is not required for correctness.
29+
30+
`fast_llvm_repo` is an internal repository rule consumed by the `llvm` module
31+
extension; it is not a public entry point and is not meant to be loaded
32+
directly from a consumer's `MODULE.bazel`.
33+
34+
## Usage with Bzlmod
35+
36+
The recommended way to consume this rule is through the `llvm` module
37+
extension in `extensions/llvm.bzl`, which wires `fast_llvm_repo` together with
38+
`toolchains_llvm`'s `llvm_toolchain` for you from a single version value:
39+
40+
```starlark
41+
module(name = "score_llvm_probe")
42+
bazel_dep(name = "score_bazel_cpp_toolchains", version = "0.5.4")
43+
44+
llvm = use_extension("@score_bazel_cpp_toolchains//extensions:llvm.bzl", "llvm")
45+
llvm.toolchain(name = "probe_llvm", version = "19.1.1")
46+
47+
use_repo(llvm, "probe_llvm", "probe_llvm_pkg")
48+
```
49+
50+
Consumers do not need their own `bazel_dep(toolchains_llvm)` and only need to
51+
track a single `version` value. See
52+
[Extension API](extension_api.md#llvm-extension-optional-faster-alternative)
53+
for the full attribute reference.
54+
55+
## What the rule does
56+
57+
For the requested LLVM version, the rule:
58+
59+
1. rejects non-Linux hosts and selects the archive for the host architecture,
60+
2. downloads it with a pinned SHA-256 checksum,
61+
3. extracts it with parallel `xz -T0` output streamed to `tar`, using Bash's
62+
`pipefail` to check both process exit codes,
63+
4. renders the BUILD file expected by `toolchains_llvm`.
64+
65+
The parallel XZ pipeline is the main optimization. It changes the extraction
66+
implementation while leaving the LLVM archive, checksum verification, and
67+
toolchain target layout intact.
68+
69+
## Why not `http_archive`?
70+
71+
`http_archive` can functionally download and unpack a `tar.xz`. However,
72+
`toolchains_llvm` already uses Bazel's standard repository download-and-extract
73+
mechanism for the LLVM repository it creates internally. That
74+
`download_and_extract()` path is the slow part this rule is intended to
75+
replace.
76+
77+
`http_archive` does not provide a switch to use the parallel `xz -T0` pipeline,
78+
and `patch_cmds` run only after extraction. Using `http_archive` would therefore
79+
not provide the performance improvement.
80+
81+
If the additional extraction time is acceptable, the original
82+
`toolchains_llvm` setup remains the simpler option. This rule exists for the
83+
case where the measured setup time justifies the additional repository-rule
84+
maintenance.
85+
86+
## Scope and trade-offs
87+
88+
This is intentionally a small, specialized rule rather than a replacement for
89+
`http_archive` or `toolchains_llvm`:
90+
91+
- only the versions and Linux architectures listed in
92+
[`fast_llvm_repo.bzl`](../rules/fast_llvm_repo.bzl) are supported;
93+
- the fast path requires `bash`, `tar`, and `xz` on the host;
94+
- the version/checksum table must be kept aligned with `toolchains_llvm`;
95+
- the rule uses `toolchains_llvm`'s internal BUILD template, so upgrades of
96+
that dependency need to be checked; and
97+
- `fast_llvm_repo` itself is not meant to be used directly outside of
98+
`extensions/llvm.bzl`.

0 commit comments

Comments
 (0)