Skip to content

Add API benchmark suite#535

Open
gwskier11-design wants to merge 2 commits into
SecureBananaLabs:mainfrom
gwskier11-design:codex/api-benchmark-suite
Open

Add API benchmark suite#535
gwskier11-design wants to merge 2 commits into
SecureBananaLabs:mainfrom
gwskier11-design:codex/api-benchmark-suite

Conversation

@gwskier11-design
Copy link
Copy Markdown

@gwskier11-design gwskier11-design commented May 21, 2026

Summary

  • add a dependency-free benchmark runner under benchmarks/ covering /health and every mounted /api/* route
  • include realistic JSON/form payloads, generated benchmark admin token support, p50/p95/p99 latency, p99 TTFB, RPS, peak RPS, and error-rate reporting
  • add reviewable thresholds, .env.benchmark.example, endpoint coverage verification, and committed JSON/Markdown benchmark reports
  • add demos/api-benchmark-demo.mp4 as the requested short demo artifact
  • fix the API test script glob so existing Node tests run correctly

Validation

  • node --check benchmarks/run-benchmarks.mjs
  • node --check benchmarks/endpoints.mjs
  • node --check benchmarks/verify-endpoint-coverage.mjs
  • npm run benchmark:coverage
  • npm test
  • npm run benchmark:smoke — 21 endpoints, 21 requests, 0% errors, max p99 16.6 ms
  • npm run benchmark — 21 endpoints, 168 requests, 0% errors, max p99 13.01 ms, max p99 TTFB 12.74 ms
  • ffprobe demos/api-benchmark-demo.mp4 — 14.0s, 1280x720, H.264
  • git diff --check

Reports are committed at benchmarks/results/full-latest.* and benchmarks/results/smoke-latest.*.

Benchmark Environment

Hardware

  • CPU model & core count: Apple M4, 10 cores
  • RAM: 32 GiB total
  • Storage type: Apple internal SSD / APFS
  • Network interface: loopback (127.0.0.1) for local benchmarks
  • Machine type: local Mac workstation
  • OS & version: macOS 26.5 (25F71), arm64

Runtime

  • Node.js version: v22.22.0
  • npm version: 11.11.0
  • Resource limits applied: none intentionally applied
  • Other significant processes running: normal local desktop/Codex background processes

If submitted by or with an AI agent

  • Agent/tool name: OpenAI Codex desktop app
  • Underlying model/version: GPT-5-family Codex coding agent
  • Inference provider: OpenAI
  • Orchestration framework: none beyond Codex desktop shell/GitHub tooling
  • Execution mode: user-initiated, agent-executed
  • Shell/tool access during execution: yes
  • Internet access during execution: yes
  • Benchmark commands run by: agent directly
  • Known constraints/sandboxing: local workstation execution; benchmark target uses local loopback by default; no production/staging target or secrets were used.

/claim #30

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant