Skip to content

Commit c824366

Browse files
kalyazinclaude
andcommitted
docs: document the resume-time envd live-upgrade flow
ARCHITECTURE.md described resume without the envd live-upgrade step and listed no /upgrade endpoint or handover responsibility. Document envd's POST /upgrade + same-PID handover in the Envd section, and the resume-time upgrade (best-effort, failing the resume only on an unrecoverable post-exec failure) in the Pause/resume flow. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Nikita Kalyazin <nikita.kalyazin@e2b.dev>
1 parent 5546769 commit c824366

1 file changed

Lines changed: 16 additions & 1 deletion

File tree

‎docs/ARCHITECTURE.md‎

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -174,9 +174,18 @@ The agent inside every VM (started by systemd very early in boot), port 49983, c
174174
stdout/stderr, stdin, signals, PTYs — this is what SDKs use to "run code".
175175
- **Filesystem service** (`spec/filesystem/filesystem.proto`): stat/list/make/move/remove/watch.
176176
- **REST**: `/health`, `/metrics`, `/files` upload/download, `/init` (orchestrator pushes env
177-
vars, access token, metadata after boot/resume), freeze/thaw hooks used during pause.
177+
vars, access token, metadata after boot/resume), `/upgrade` (live self-upgrade, below),
178+
freeze/thaw hooks used during pause.
178179
- **Auth**: `X-Access-Token` header checked against a token delivered via Firecracker MMDS;
179180
signed URLs for file endpoints.
181+
- **Live upgrade** (`internal/services/process/upgrade.go`): an authenticated `POST /upgrade` lets
182+
the orchestrator swap envd inside a *running* sandbox at resume. It streams the new binary in the
183+
request body and envd `syscall.Exec`s into it **with the same PID**, carrying the workload's
184+
stdio/PTY fds, process table, recently-retained exit codes and filesystem watchers forward via a
185+
tmpfs handover blob. The workload cgroups stay frozen until the post-upgrade `/init` restores the
186+
access token (so no re-adopted process runs unauthenticated), and the handover outcome
187+
(procs/watchers re-adopted, plus any failures) rides back on that `/init`'s `X-Envd-Handover`
188+
header for fleet visibility.
180189
- Scans guest ports and forwards them so any port a user process opens becomes reachable through
181190
sandbox URLs. **`pkg/version.go` must be bumped on every behavioral change** — the API and the
182191
orchestrator gate features on the envd version recorded in each template build.
@@ -289,6 +298,12 @@ sequenceDiagram
289298
- **Resume**: same path as creation, but placement prefers the **origin node** — if the snapshot
290299
is still in its local cache, resume avoids any object-storage reads. `Checkpoint` is a
291300
pause+resume in place used to persist state while keeping the sandbox running.
301+
- **Envd live-upgrade on resume**: the orchestrator can upgrade the sandbox's envd to a newer
302+
node-local build during resume (gated by the `envd-upgrade-target` flag in
303+
`packages/shared/pkg/featureflags`), via envd's `POST /upgrade` (see the envd section). It is
304+
best-effort — a delivery failure before the `exec` leaves the old envd serving — except an
305+
unrecoverable post-`exec` failure (the new envd never re-initializes), which fails the resume
306+
rather than return a permanently unusable sandbox.
292307
- Auto-pause/auto-resume make sandboxes effectively serverless: idle sandboxes pause, traffic
293308
resumes them (see traffic flow above).
294309

0 commit comments

Comments
 (0)