Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .github/workflows/actiond-production.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,6 @@ jobs:
run: |
work="$RUNNER_TEMP/actiond-production"
git -C "$work/actiond" apply "$GITHUB_WORKSPACE/experiments/actiond/actiond-advice.patch"
git -C "$work/actiond" apply "$GITHUB_WORKSPACE/experiments/actiond/actiond-input-rootfs.patch"
cd "$work/actiond"
bazelisk --output_base="$work/worker-output" build --bes_backend= --remote_executor= --remote_cache= --spawn_strategy=local --jobs=2 //cmd/linux-actiond:linux-actiond_linux_x86_64 > "$work/worker-build.log" 2>&1
worker=$(bazelisk --output_base="$work/worker-output" cquery --bes_backend= //cmd/linux-actiond:linux-actiond_linux_x86_64 --output=starlark '--starlark:expr=providers(target)["DefaultInfo"].files_to_run.executable.path')
Expand Down
5 changes: 3 additions & 2 deletions docs/actiond-migration.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ migrations have landed.
checks stay in host E2E targets; inherited environment and origin exceptions
do not carry over.

The memory-advice patch is upstream PR #48; the declared-rootfs patch remains
isolated locally for upstreaming. Native macOS VM execution and cross-architecture
The memory-advice patch is covered by upstream actiond PR #33. VRT now relocates
declared executable copies and no longer requires the rootfs patch in actiond
PR #49. Native macOS VM execution and cross-architecture
pixel equivalence remain unvalidated. These baselines require a Linux amd64 worker.
22 changes: 15 additions & 7 deletions docs/actiond.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,12 @@ from their own OCI image. Host E2E and component tests use host browsers.

## Worker and Bazel configuration

The worker currently needs the memory-advice and declared-rootfs patches in
[`experiments/actiond`](../experiments/actiond). The memory-advice change is
[upstream PR #48](https://github.com/hermeticbuild/actiond/pull/48); the rootfs
patch is maintained locally for upstreaming. The production workflow builds and
runs that exact patched worker. Linux VM workers require KVM and vhost-vsock.
The worker currently needs the memory-advice kernel patch in
[`experiments/actiond`](../experiments/actiond), covered by
[upstream PR #33](https://github.com/hermeticbuild/actiond/pull/33).
The production workflow builds actiond with only that patch. No `input-rootfs`,
`libc`, or `requires-bash` execution properties are needed.
Linux VM workers require KVM and vhost-vsock.

With a patched worker listening on `127.0.0.1:8980`, put this in the consumer's
Bazel configuration:
Expand All @@ -34,8 +35,9 @@ Use a remote worker address when appropriate. ARM64 clients need an amd64 worker
for these baselines. The validated CI worker uses 6 GiB RAM for at most two
concurrent actions; size workers for fixture and staging memory as well as
Chromium. The macOS native VM backend is not yet validated by these
checks. Local fallback must remain disabled; `/workspace` runtime executables
are meaningful inside the worker's declared rootfs.
checks. Keep VRT's explicit remote strategies and local fallback disabled.
The bootstrap also rejects ordinary host roots with system shell/loader paths
before creating temporary runtime launchers.

```sh
bazel test --config=vrt //path:visual_test
Expand All @@ -56,6 +58,12 @@ commands needed by fixtures. OCI extraction verifies declared blobs and never
contacts a registry. Acquisition and image construction happen before execution;
Docker, registry credentials, Testcontainers, and Ryuk are absent from the action.

The declared ELF loader starts the bootstrap. It prepares a private Node/Bash
launcher in the action's temporary directory, then relocates executable copies
in the staged inputs. Libraries and fonts use explicit paths. A VRT-only Node
preload directs `spawn(..., {shell: true})` to declared Bash, preserving native
Playwright `webServer` behavior without `/bin/sh`. Host tests do not load it.

The action has loopback-only networking. Start fixture services inside it using
`server` or native Playwright `webServer`, and declare their files in `data`.
External assets and APIs need local fixtures. Live deployed checks belong in
Expand Down
28 changes: 22 additions & 6 deletions docs/browser-runtime.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@

The actiond migration accepts caller-owned Linux runtime files through
`browser_runtime`. The runtime must contain Chromium, Node, their ELF loader and
shared libraries, and the fonts/fontconfig used for screenshots. Native
Playwright `webServer` commands also need `/bin/sh`. Bazel `js_binary` fixture
launchers additionally need `/usr/bin/env`, Bash, and their shell utilities
(including `dirname`, `uname`, and `readlink`) in the declared runtime.
shared libraries, and the fonts/fontconfig used for screenshots. Include Bash
and the shell utilities used by fixture launchers (including `dirname`, `uname`,
and `readlink` for Bazel `js_binary`). These files stay inside the runtime tree;
they are not installed at system paths.

A caller can produce a flattened filesystem tar and unpack it during the Bazel
build:
Expand All @@ -25,6 +25,8 @@ browser_runtime(
root = ":runtime_files",
executable = "chromium/chrome-headless-shell",
node = "bin/node",
loader = "lib/ld-linux-x86-64.so.2",
bash = "bin/bash",
library_dirs = ["lib"],
fontconfig = "etc/fonts",
)
Expand All @@ -46,8 +48,22 @@ Archive extraction uses a Bazel-provided Python interpreter and makes no network
requests. Absolute image symlinks are resolved within the image root, then links
are materialized into regular files and directories for the output tree. Missing
link targets are errors, so runtime packaging cannot silently borrow host files.
Font configuration should use image paths or paths relative to the configuration
file, rather than a build-machine path.
Font configuration must use paths relative to its configuration file, rather
than absolute image or build-machine paths.

`loader` and `bash` default to the paths shown above. Execution starts the
declared loader directly. The VRT runner creates `/tmp/rules-web-vrt` inside its
isolated action, copies the loader/Node/Bash there, and rewrites staged ELF64
interpreter paths to that loader. This retains executable identity for Chromium
subprocesses. Interpreter segments too short for the replacement are rejected.
Original declared inputs are never modified.

Executable scripts with ordinary `/bin/sh`, `/bin/bash`, `/usr/bin/env bash`,
or Node shebangs are redirected to the declared launchers. The VRT subprocess
adapter supplies Bash for Playwright's `shell: true` launches. Other hardcoded
system paths, interpreters, and complex `env -S` shebangs need caller-owned
wrappers or packaging changes; arbitrary OCI images are not automatically
relocatable.

For a caller-owned OCI image layout directory, use `browser_runtime_oci` instead:

Expand Down
13 changes: 10 additions & 3 deletions experiments/actiond/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ The local host has no `/dev/kvm`; KVM validation ran on GitHub's Ubuntu runner.

## Production validation

The production workflow builds actiond at `8a42c3d` with both patches below,
The production workflow builds actiond at `8a42c3d` with the memory-advice patch,
starts a Linux amd64 VM, and runs `prepare-public.mjs` / `run-public-actiond.sh`.
The fixture constructs a caller-owned OCI layout through Bazel and extracts its
runtime with `browser_runtime_oci`. Public `.update` and test targets execute
Expand All @@ -96,14 +96,21 @@ is separate from the VM fixtures and does not establish a full consumer CI migra
## Local actiond patches

- `actiond-advice.patch`: enables memory-advice syscalls in both kernel configs;
submitted as [upstream PR #48](https://github.com/hermeticbuild/actiond/pull/48).
- `actiond-input-rootfs.patch`: exposes selected directories from a declared
covered by [upstream PR #33](https://github.com/hermeticbuild/actiond/pull/33).
- `actiond-input-rootfs.patch`: historical alternative, submitted as
[upstream PR #49](https://github.com/hermeticbuild/actiond/pull/49), and no longer
applied by production CI. It exposes selected directories from a declared
runtime input tree at normal Linux paths, preserving executor-owned devices,
`/proc`, temporary storage, and network isolation. `input-rootfs-env` resolves
its path from a declared command variable, which supports Bazel output paths.
Maintained separately in local actiond commits `66e2dca` and `f713bca` for
upstreaming. The full actiond build and both unit-test targets pass.

The current VRT runner instead relocates staged executable interpreter paths,
supplies explicit library/font paths, and directs Node shell launches to declared
Bash. The production isolation fixture asserts that the former system runtime
paths are absent. Image extraction remains a caller-side Bazel action.

The native macOS VM backend has not been exercised here. ARM64 clients must
select an amd64 worker for these baseline inputs.

Expand Down
3 changes: 3 additions & 0 deletions experiments/actiond/isolation.visual.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@ test('the whole VRT action is offline and can still serve its fixture', async ({
expect(process.platform).toBe('linux')
expect(process.arch).toBe('x64')
expect(fs.existsSync('/var/run/docker.sock')).toBe(false)
// No input-rootfs mapping or packaged libc/Bash runtime was requested.
for (const file of ['/bin/bash', '/bin/sh', '/usr/bin/env', '/lib64/ld-linux-x86-64.so.2'])
expect(fs.existsSync(file), file).toBe(false)
const fixture = JSON.parse(process.env.ACTIOND_FIXTURE!)
expect(JSON.parse(fs.readFileSync(fixture.package, 'utf8')).name).toBeTruthy()
const error = await new Promise<NodeJS.ErrnoException>(resolve => {
Expand Down
7 changes: 7 additions & 0 deletions experiments/actiond/prepare-public.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,13 @@ browser_runtime(
library_dirs = ["lib"],
fontconfig = "etc/fonts",
)
visual_test(
name = "actiond_local_rejection_test",
browser = ":actiond_browser",
config = ":native_config",
tests = ":native_visual_specs",
baseline_dir = "__actiond_local_rejection__",
)
visual_test(
name = "actiond_native_test",
browser = ":actiond_browser",
Expand Down
15 changes: 15 additions & 0 deletions experiments/actiond/run-public-actiond.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,21 @@ flags=(
--noremote_cache_compression --remote_download_outputs=all
)
bazel_cmd=("${ACTIOND_BAZEL:-bazelisk}" --output_base="$work/public-bazel-output")
# This target is never executed remotely, so it cannot reuse a successful
# capture from the action cache and accidentally skip the rejection check.
mkdir -p "$work/results"
if "${bazel_cmd[@]}" build //:actiond_local_rejection_test_capture \
--remote_executor= --remote_cache= --disk_cache= --spawn_strategy=sandboxed,local \
> "$work/results/local-rejection.log" 2>&1; then
echo 'VRT unexpectedly executed on the host' >&2
exit 1
fi
python3 - "$work/results/local-rejection.log" <<'PY'
from pathlib import Path
import sys
log = Path(sys.argv[1]).read_text()
assert 'VRT requires an isolated action without system runtimes' in log, log
PY
"${bazel_cmd[@]}" run //:actiond_native_test.update "${flags[@]}"
"${bazel_cmd[@]}" run //:actiond_gallery_test.update "${flags[@]}"
test -s __actiond_native__/saved.png
Expand Down
14 changes: 9 additions & 5 deletions internal/remote.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -43,20 +43,25 @@ def _remote_impl(ctx):
"args": [ctx.expand_location(value, targets = locations) for value in ctx.attr.args],
"output": output.path,
"capture": ctx.attr.capture,
"runtime": dict(browser.descriptor, path = root.path),
}))
descriptor = browser.descriptor
ctx.actions.run(
executable = "/workspace/" + root.path + "/" + descriptor["node"],
arguments = [ctx.file._bootstrap.path, job.path],
executable = root.path + "/" + descriptor["loader"],
arguments = [
"--library-path",
":".join([root.path + "/" + p for p in descriptor["libraryDirs"]]),
root.path + "/" + descriptor["node"],
ctx.file._bootstrap.path,
job.path,
],
inputs = depset(files + [root, job, ctx.file._bootstrap]),
outputs = [output],
env = {
"VRT_RUNTIME_ROOT": root.path,
"HOME": "/tmp",
"TMPDIR": "/tmp",
"LANG": "C.UTF-8",
"TZ": "UTC",
"LD_LIBRARY_PATH": ":".join(["/workspace/" + root.path + "/" + p for p in descriptor["libraryDirs"]]),
},
execution_requirements = {"no-local": "1"},
mnemonic = "VrtCapture" if ctx.attr.capture else "VrtCompare",
Expand Down Expand Up @@ -96,7 +101,6 @@ def remote_browser_test(name, browser, env, args, tags, timeout, data, target_pl
capture = capture,
data = data,
target_platform = target_platform,
exec_properties = {"input-rootfs-env": "VRT_RUNTIME_ROOT"},
exec_compatible_with = [Label("@platforms//os:linux"), Label("@platforms//cpu:x86_64")],
tags = ["manual"],
)
Expand Down
4 changes: 4 additions & 0 deletions playwright/defs.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ def _browser_runtime_impl(ctx):
"root": runfile(ctx.file.root),
"executable": _relative_path(ctx.attr.executable, "executable"),
"node": _relative_path(ctx.attr.node, "node"),
"loader": _relative_path(ctx.attr.loader, "loader"),
"bash": _relative_path(ctx.attr.bash, "bash"),
"libraryDirs": [_relative_path(p, "library_dirs") for p in ctx.attr.library_dirs],
"fontconfig": _relative_path(ctx.attr.fontconfig, "fontconfig"),
"arch": ctx.attr.arch,
Expand All @@ -34,6 +36,8 @@ browser_runtime = rule(
"root": attr.label(mandatory = True, allow_single_file = True),
"executable": attr.string(mandatory = True),
"node": attr.string(mandatory = True),
"loader": attr.string(default = "lib/ld-linux-x86-64.so.2"),
"bash": attr.string(default = "bin/bash"),
"library_dirs": attr.string_list(mandatory = True),
"fontconfig": attr.string(mandatory = True),
"arch": attr.string(default = "x64", values = ["x64", "arm64"]),
Expand Down
10 changes: 10 additions & 0 deletions runtime/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,9 @@ js_library(
"host-browser.js",
"matching.js",
"network.js",
"relocation.js",
"remote-job.js",
"vrt-processes.cjs",
"package.json",
"server.js",
"static-server.js",
Expand Down Expand Up @@ -87,6 +90,13 @@ js_test(
entry_point = "remote-result.test.js",
)

js_test(
name = "relocation_test",
size = "small",
data = ["package.json", ":typecheck", "vrt-processes.cjs"],
entry_point = "relocation.test.js",
)

npm_package(
name = "package",
srcs = [
Expand Down
68 changes: 68 additions & 0 deletions runtime/relocation.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
import assert from 'node:assert/strict'
import fs from 'node:fs'
import os from 'node:os'
import path from 'node:path'
import {spawnSync} from 'node:child_process'
import {fileURLToPath} from 'node:url'
import {test} from 'node:test'
import {relocateExecutable, relocateInputs, runtimeDirectory} from './relocation.js'

function elf(interpreter = '/lib64/ld-linux-x86-64.so.2') {
const data = Buffer.alloc(256, 0x51)
data.set([0x7f, 0x45, 0x4c, 0x46, 2, 1])
data.writeBigUInt64LE(64n, 32)
data.writeUInt16LE(56, 54)
data.writeUInt16LE(1, 56)
data.writeUInt32LE(3, 64)
data.writeBigUInt64LE(128n, 72)
data.writeBigUInt64LE(BigInt(interpreter.length + 1), 96)
data.write(interpreter + '\0', 128)
return data
}

test('ELF relocation replaces only the interpreter bytes and rejects invalid segments', () => {
const input = elf()
const output = relocateExecutable(input)!
assert.deepEqual(output.subarray(0, 128), input.subarray(0, 128))
assert.deepEqual(output.subarray(156), input.subarray(156))
assert.equal(output.subarray(128, 128 + runtimeDirectory.length + 7).toString(), `${runtimeDirectory}/ld.so\0`)
assert.equal(input.subarray(128, 156).toString(), '/lib64/ld-linux-x86-64.so.2\0')
assert.throws(() => relocateExecutable(elf('/ld.so')), /too short/)
const bad = elf(); bad.writeBigUInt64LE(999n, 72)
assert.throws(() => relocateExecutable(bad), /Invalid ELF interpreter/)
assert.throws(() => relocateExecutable(input.subarray(0, 100)), /program-header/)
})

test('staging relocates executable ELF and common shebangs without changing data or following links', () => {
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'vrt-relocation-'))
try {
const binary = path.join(root, 'node'); fs.writeFileSync(binary, elf(), {mode: 0o555})
const script = path.join(root, 'server'); fs.writeFileSync(script, '#!/usr/bin/env bash\nprintf hello\n', {mode: 0o755})
const data = path.join(root, 'data'); fs.writeFileSync(data, elf(), {mode: 0o644})
fs.symlinkSync('data', path.join(root, 'alias'))
relocateInputs(root)
assert.deepEqual(fs.readFileSync(binary), relocateExecutable(elf()))
assert.equal(fs.statSync(binary).mode & 0o777, 0o555)
assert.equal(fs.readFileSync(script, 'utf8'), `#!${runtimeDirectory}/bash\nprintf hello\n`)
assert.deepEqual(fs.readFileSync(data), elf())
assert.ok(fs.lstatSync(path.join(root, 'alias')).isSymbolicLink())
assert.equal(relocateExecutable(Buffer.from('#!/usr/bin/env python3\nprint(1)\n')), undefined)
} finally { fs.rmSync(root, {recursive: true, force: true}) }
})

test('VRT subprocess adapter supplies a declared shell while preserving explicit shell selection', {skip: process.platform === 'win32'}, () => {
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'vrt-shell-'))
try {
const shell = path.join(root, 'bash')
fs.writeFileSync(shell, '#!/bin/sh\nprintf "declared:"\nexec /bin/sh "$@"\n', {mode: 0o755})
const hook = fileURLToPath(new URL('./vrt-processes.cjs', import.meta.url))
const probe = `const {spawn}=require('node:child_process'); const c=spawn('printf hello',{shell:SHELL});c.stdout.pipe(process.stdout);c.on('exit',code=>process.exit(code));`
for (const [selection, expected] of [['true', 'declared:hello'], ['"/bin/sh"', 'hello']]) {
const result = spawnSync(process.execPath, ['--require', hook, '-e', probe.replace('SHELL', selection)], {
env: {...process.env, VRT_BASH: shell}, encoding: 'utf8',
})
assert.equal(result.status, 0, result.stderr)
assert.equal(result.stdout, expected)
}
} finally { fs.rmSync(root, {recursive: true, force: true}) }
})
Loading
Loading