Skip to content

refactor(orch): render the distro selection in provision.sh's own template - #3463

Closed
tomassrnka wants to merge 2 commits into
mainfrom
distro-selector-template
Closed

tomassrnka wants to merge 2 commits into
mainfrom
distro-selector-template

Conversation

@tomassrnka

@tomassrnka tomassrnka commented Jul 30, 2026

Copy link
Copy Markdown
Member

Follow-up promised in the #3411 review (@dobrac's three comments: Go templating for the selector, profiles-as-data with selection living in provision.sh, init blocks in separate files). #3459 has merged; this sits directly on main.

What changed: the distro-selection block (case arms, rejected-id guard, ID_LIKE loop, all error/warning text) moved verbatim from the ShellSelector() string-builder into provision.sh as inline text/template actions; the distro package now contributes data only (distro.NewTemplateData()), with shell-quoting kept unit-testable in Go and no FuncMap (repo precedent). Init-setup blocks moved to distro/init-{systemd,openrc,nixos}.sh, go:embed'd.

Cache key: selection structure is covered by the raw embedded provision.sh that Hash() already hashes; distro.Fingerprint() (%#v of the view) covers the spliced data, so new fields fingerprint automatically. Version 1 → 2 — the dev/fallback provision version rotates once on deploy; production rollout stays gated by the BuildProvisionVersion LD flag.

Verification: rendered output diffed against the old ShellSelector() — byte-identical for the whole selection function, only deliberate comment/blank-line additions in the tail. Full e2e on an E2B Local dev slot (cold cache, orchestrator rebuilt from this branch): oraclelinux:9 rejected with byte-identical ERROR lines; kali builds with the byte-identical ID_LIKE WARNING and boots; alpine (OpenRC path) builds and boots with envd healthy; ubuntu and debian build and boot. sh -n, bash -n, shellcheck -s sh clean on the three new files and the rendered script.

Tests: selection-text assertions moved to base/provision_test.go against the fully rendered script (render seam and tests are build-tag-free, so they run on darwin); every prior assertion preserved, incl. the errexit guard (no if e2b_select_profile) and guard-before-fallback ordering; new per-field %q-vs-sh quoting pins, byte-exact customer-message pins, and a leftover-{{ render check. New TestProvisionScriptSplicesEveryProfileField asserts every ProfileView field — including the init-setup body now sourced from a file — reaches its rendered case arm: a dropped template action would leave the Go-side tests green while every distro hits an undefined variable under set -eu. It subsumes the removed TestProvisionScriptInitSetup.

Rebase onto current main (#3440, #3478): semantic, not textual — both touch the moved code. #3440's chrony wiring (the systemd $E2B_TIMESYNC_UNIT.service.d drop-in, OpenRC's e2b-chrony-source install, the unconditional command_args="-F 0") now lives in init-openrc.sh/init-systemd.sh, verified byte-for-byte against main's literals; its tests are kept, with the chrony-deferral test now reading the rendered script. #3478's InitBinary: "/sbin/e2b-nixos-init" survives the distro.go rewrite and renders as E2B_INIT_BIN="/sbin/e2b-nixos-init". Tree swept for orphans of the moved code (E2B_CHRONY_PHC, command_args="-F 0", e2b-chrony-source, E2B_TIMESYNC_UNIT, E2B_INIT_BIN, /sbin/e2b-nixos-init, ShellSelector, DistroSelector): every consumer has a definition, none stranded. Re-verified after the rebase: rendered script still byte-identical to main's ShellSelector() output apart from the comments.

🤖 Generated with Claude Code

@cla-bot cla-bot Bot added the cla-signed label Jul 30, 2026
@cursor

cursor Bot commented Jul 30, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Changes sit on the template base provisioning path and rotate cache keys; runtime selection logic is intended to stay byte-identical, but mistakes in templating or quoting could break builds or guest provisioning across all distros.

Overview
Distro profile selection no longer comes from a Go-built shell string; it is rendered inline in provision.sh via text/template, while the distro registry only supplies structured view data (quoted scalars, shell fragments, ID patterns, and embedded init-setup bodies from separate init-*.sh files).

Base-layer cache versioning splits responsibility: the embedded provision.sh template covers selection structure, and distro.Fingerprint() hashes the spliced data with Version bumped to 2, so dev/fallback caches rotate once on deploy. Tests now render the full script (including on darwin) and assert per-profile field splicing, customer-visible errors, and shell-quoting safety so a dropped template action cannot pass Go-only checks.

Reviewed by Cursor Bugbot for commit 77d963e. Bugbot is set up for automated code reviews on this repo. Configure here.

@codecov

codecov Bot commented Jul 30, 2026

Copy link
Copy Markdown

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
3741 1 3740 9
View the top 1 failed test(s) by shortest run time
github.com/e2b-dev/infra/packages/api/internal/sandbox/storage/redis::TestSubscriptionManager_PubSubEndToEnd
Stack Traces | 6.52s run time
=== RUN   TestSubscriptionManager_PubSubEndToEnd
=== PAUSE TestSubscriptionManager_PubSubEndToEnd
=== CONT  TestSubscriptionManager_PubSubEndToEnd
    subscription_manager_test.go:245: 
        	Error Trace:	.../storage/redis/subscription_manager_test.go:245
        	Error:      	did not receive PubSub notification
        	Test:       	TestSubscriptionManager_PubSubEndToEnd
--- FAIL: TestSubscriptionManager_PubSubEndToEnd (6.52s)

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@tomassrnka
tomassrnka force-pushed the distro-selector-template branch from 9437ec6 to 4b6d895 Compare July 30, 2026 11:08
Base automatically changed from warn-unsupported to main July 30, 2026 11:31
@tomassrnka
tomassrnka force-pushed the distro-selector-template branch from 4b6d895 to 3f449f6 Compare July 30, 2026 12:10
The three initSetup bodies leave their Go string literals for
init-{systemd,openrc,nixos}.sh, go:embed'd with the trailing newline
trimmed so they splice exactly like the literals did. Selector output is
unchanged apart from the entry comments moving in as # lines.

Rebased over #3440, which added chrony-source wiring to both init blocks
after this was written: the systemd drop-in, OpenRC's e2b-chrony-source
install and the unconditional command_args="-F 0" move into the new
files verbatim, checked byte-for-byte against main's literals.

First half of the #3411 follow-up (dobrac: init.go:21).
…plate

The selection structure (case arms, rejected-id guard, ID_LIKE loop,
error/warning text) moves verbatim from the ShellSelector string-builder
into provision.sh as inline template actions; the distro package now
contributes data only — distro.NewTemplateData(), with all shell-quoting
kept in Go. Fingerprint hashes the view data (%#v, so new fields are
covered automatically); the structure is already hashed via the raw
embedded template. Version 1 -> 2.

The render seam and its tests live in ungated files so they keep running
on darwin. Selection-text assertions move to base/provision_test.go
against the rendered script, with the guard-ordering anchor rewritten to
the fallback loop line (E2B_ID_LIKE is assigned earlier in the full
script). New guards: %q-vs-sh quoting assumptions pinned per profile
field, a leftover-"{{" render check, and a test that every ProfileView
field — including the init-setup body now sourced from a file — reaches
the rendered case arm. A dropped template action would otherwise leave
the Go-side tests green while the guest hits an undefined variable under
set -u.

#3440's chrony-deferral test now asserts on the rendered script rather
than the raw template: the init blocks moved out, so only the rendered
form covers both halves.

Rendered output verified byte-identical to main's ShellSelector output
modulo the added comments; customer-visible messages pinned byte-exact.

Completes the #3411 follow-up (dobrac: distro.go:160, provision.sh:29).
@blacksmith-sh

blacksmith-sh Bot commented Aug 3, 2026

Copy link
Copy Markdown

Found 1 test failure on Blacksmith runners:

Failure

Test View Logs
github.com/e2b-dev/infra/packages/api/internal/sandbox/storage/redis/
TestSubscriptionManager_PubSubEndToEnd
View Logs

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

@tvi tvi closed this Aug 4, 2026
@ValentaTomas
ValentaTomas deleted the distro-selector-template branch September 11, 2026 03:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants