Skip to content

feat: add Linux Kernel Builder benchmark mode#79

Open
HeyGarrison wants to merge 9 commits intomasterfrom
feat/compute-perf-mode
Open

feat: add Linux Kernel Builder benchmark mode#79
HeyGarrison wants to merge 9 commits intomasterfrom
feat/compute-perf-mode

Conversation

@HeyGarrison
Copy link
Copy Markdown
Contributor

@HeyGarrison HeyGarrison commented Apr 15, 2026

Summary

  • Add a new linux-kernel-builder benchmark mode for LKB (Linux Kernel Builder) to measure sustained sandbox performance on a real workload (defconfig + bzImage).
  • Introduce a dedicated provider cohort, LKB scoring, terminal table output, and JSON result writer under src/linux-kernel-builder/.
  • Wire LKB into src/run.ts, add npm scripts for full-run and per-provider runs, and include workload metadata (linux-kernel-builder, LKB) in results.
  • Add a dedicated GitHub Actions workflow at .github/workflows/linux-kernel-builder-benchmarks.yml that runs on push/PR for relevant path changes.
  • Fix command execution to run the benchmark as a multiline shell script (resolving the prior bash then; syntax failure).

Validation

  • Ran npx tsc --noEmit
  • Ran npm run -s bench -- --mode linux-kernel-builder --provider namespace --iterations 1 (exec path verified; provider skipped without local credentials)

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 15, 2026

Storage Benchmark Results

10MB Files

# Provider Score Download Throughput Upload Status
1 Tigris 95.9 0.10s 810.7 Mbps 0.58s 10/10
2 AWS S3 93.6 0.12s 697.8 Mbps 0.36s 10/10
3 Cloudflare R2 82.9 0.39s 215.7 Mbps 0.92s 10/10

View full run · SVGs available as build artifacts

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 15, 2026

Sandbox Benchmark Results

Sequential

# Provider Score Median TTI P95 P99 Status
1 daytona 98.2 0.11s 0.29s 0.29s 10/10
2 e2b 96.2 0.35s 0.44s 0.44s 10/10
3 vercel 95.9 0.38s 0.46s 0.46s 10/10
4 blaxel 95.4 0.44s 0.50s 0.50s 10/10
5 hopx 87.7 1.10s 1.41s 1.41s 10/10
6 modal 84.1 1.40s 1.87s 1.87s 10/10
7 runloop 84.0 1.38s 1.93s 1.93s 10/10
8 cloudflare 81.7 1.53s 2.27s 2.27s 10/10
9 namespace 74.2 1.82s 3.74s 3.74s 10/10
10 codesandbox 68.9 2.62s 3.84s 3.84s 10/10

Staggered

# Provider Score Median TTI P95 P99 Status
1 daytona 98.0 0.13s 0.30s 0.30s 10/10
2 blaxel 95.5 0.43s 0.48s 0.48s 10/10
3 vercel 95.2 0.37s 0.65s 0.65s 10/10
4 e2b 94.3 0.33s 0.92s 0.92s 10/10
5 hopx 88.7 1.09s 1.19s 1.19s 10/10
6 namespace 81.3 1.73s 2.06s 2.06s 10/10
7 modal 81.3 1.56s 2.33s 2.33s 10/10
8 runloop 80.6 1.88s 2.04s 2.04s 10/10
9 cloudflare 73.9 1.75s 3.91s 3.91s 10/10
10 codesandbox 62.3 3.63s 3.98s 3.98s 10/10

Burst

# Provider Score Median TTI P95 P99 Status
1 daytona 98.0 0.13s 0.31s 0.31s 10/10
2 blaxel 94.9 0.47s 0.56s 0.56s 10/10
3 e2b 94.6 0.46s 0.65s 0.65s 10/10
4 vercel 87.8 0.35s 2.52s 2.52s 10/10
5 hopx 82.1 1.71s 1.92s 1.92s 10/10
6 modal 81.5 1.71s 2.06s 2.06s 10/10
7 namespace 79.0 1.92s 2.36s 2.36s 10/10
8 runloop 68.3 2.97s 3.47s 3.47s 10/10
9 cloudflare 63.4 2.22s 5.81s 5.81s 10/10
10 codesandbox 60.1 3.45s 4.79s 4.79s 10/10

View full run · SVGs available as build artifacts

@HeyGarrison HeyGarrison changed the title feat: add compute-perf mode for kernel build benchmarks feat: add Linux Kernel Builder benchmark mode Apr 16, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 16, 2026

Linux Kernel Builder Benchmark Results

# Provider Score Build Median Build P95 Build P99 Status Sample Failure
1 blaxel 0.0 -- -- -- 0/1 Command failed with exit code 2: Missing downloader: curl or wget
2 daytona 0.0 -- -- -- 0/1 Command failed with exit code 2: linux-6.12.24.tar.xz: OK tar: Unexpected EOF in archive tar: Unexpected EOF in archive
3 e2b 0.0 -- -- -- 0/1 Command failed with exit code 3: linux-6.12.24.tar.xz: OK Missing dependency: bison
4 hopx 0.0 -- -- -- 0/1 Command failed with exit code 124: Command timed out after 30 seconds linux-6.12.24.tar.xz: OK
5 namespace 0.0 -- -- -- 0/1 Command failed with exit code 1: time="2026-04-16T02:09:39Z" level=fatal msg="exec failed with exit code 3" linux-6.12.
6 runloop 0.0 -- -- -- 0/1 Command failed with exit code 2: tar (child): xz: Cannot exec: No such file or directory tar (child): Error is not recov

View full run · JSON artifacts available in the run artifacts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant