Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions .github/workflows/benchmark-smoke.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
name: API Benchmark Smoke

on:
pull_request:
paths:
- "apps/api/**"
- "benchmarks/**"
- "package*.json"
- ".github/workflows/benchmark-smoke.yml"

jobs:
api-benchmark-smoke:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 22
cache: npm
- run: npm ci
- run: npm run benchmark:smoke
- uses: actions/upload-artifact@v4
if: always()
with:
name: api-benchmark-results
path: benchmarks/results/
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,6 @@ node_modules
dist
.env
.env.*
!benchmarks/.env.benchmark.example
coverage
*.log
2 changes: 1 addition & 1 deletion apps/api/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
"scripts": {
"dev": "node src/server.js",
"start": "node src/server.js",
"test": "node --test src/tests"
"test": "node --test \"src/tests/**/*.test.js\""
},
"dependencies": {
"cors": "^2.8.5",
Expand Down
10 changes: 10 additions & 0 deletions benchmarks/.env.benchmark.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Optional. When unset, the benchmark runner starts the local Express app.
BENCHMARK_BASE_URL=http://127.0.0.1:4000

# Optional for protected routes. When unset for the local app, a benchmark JWT
# is generated with the development JWT secret.
BENCHMARK_AUTH_TOKEN=

# Tunables for local runs.
BENCHMARK_REQUESTS=5
BENCHMARK_CONCURRENCY=2
47 changes: 47 additions & 0 deletions benchmarks/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# API Benchmarks

This directory contains a reproducible benchmark suite for the platform API.

## Commands

```console
npm run benchmark
npm run benchmark:smoke
```

By default, the runner starts the local Express app on a random port and benchmarks every `/api/` endpoint. To run against an already running local or staging server, set `BENCHMARK_BASE_URL`.

## Reviewer Demo

Use `benchmark:smoke` for a short end-to-end review pass:

```console
npm ci
npm test
npm run benchmark:smoke
sed -n '1,80p' benchmarks/results/api-benchmark-latest.md
```

The smoke run starts the local API automatically, exercises all configured routes once, checks the thresholds in `thresholds.json`, and writes the JSON and Markdown reports under `benchmarks/results/`. See `demo-walkthrough.md` for a recording-ready walkthrough.

## Environment

Copy `.env.benchmark.example` into your local environment or CI secret store and set:

- `BENCHMARK_BASE_URL`: target server URL. If omitted, the local app is started automatically.
- `BENCHMARK_AUTH_TOKEN`: bearer token for protected routes. If omitted for a local run, a benchmark JWT is generated.
- `BENCHMARK_REQUESTS`: requests per endpoint. Defaults to `5`, or `1` in smoke mode.
- `BENCHMARK_CONCURRENCY`: concurrent requests per endpoint. Defaults to `2`, or `1` in smoke mode.

## Output

Each run writes:

- `benchmarks/results/api-benchmark-latest.json`
- `benchmarks/results/api-benchmark-latest.md`

The JSON result is machine-readable for regression tracking. The Markdown summary is designed to be pasted into PR descriptions.

## Thresholds

`thresholds.json` stores reviewable smoke-test gates. The runner fails when an endpoint exceeds its p99 latency threshold or error-rate threshold.
36 changes: 36 additions & 0 deletions benchmarks/demo-walkthrough.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# API Benchmark Demo Walkthrough

This is a short, recording-ready walkthrough for reviewers to verify the benchmark suite added for bounty issue #30.

## Goal

Show that the benchmark runner can start the local API, exercise the platform endpoints, enforce latency/error thresholds, and publish reviewable benchmark output.

## Demo Steps

```console
npm ci
npm test
npm run benchmark:smoke
sed -n '1,80p' benchmarks/results/api-benchmark-latest.md
```

## Expected Result

The smoke run should complete successfully and write:

- `benchmarks/results/api-benchmark-latest.json`
- `benchmarks/results/api-benchmark-latest.md`

The committed smoke output covers 20 API routes with 1 request per endpoint, 1 concurrent request per endpoint, and a passing threshold result.

## What To Check

- `benchmarks/routes.mjs` lists the route inventory and request payloads.
- `benchmarks/thresholds.json` defines reviewable p99 latency and error-rate gates.
- `benchmarks/run-api-benchmarks.mjs` starts a local app when `BENCHMARK_BASE_URL` is not set, or benchmarks an external target when it is set.
- `.github/workflows/benchmark-smoke.yml` runs the smoke benchmark on relevant pull requests and uploads `benchmarks/results/` as an artifact.

## CI Artifact

The `API Benchmark Smoke` workflow uploads the generated benchmark report as `api-benchmark-results`, so maintainers can inspect the exact JSON and Markdown generated by the pull request run.
1 change: 1 addition & 0 deletions benchmarks/results/.gitkeep
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@

Loading
Loading