Skip to content

perf(phoenix-bandit): speed up crud and benchmark controllers#924

Open
nakrovati wants to merge 1 commit into
MDA2AV:mainfrom
nakrovati:perf/speed-up-phoenix-bandit
Open

perf(phoenix-bandit): speed up crud and benchmark controllers#924
nakrovati wants to merge 1 commit into
MDA2AV:mainfrom
nakrovati:perf/speed-up-phoenix-bandit

Conversation

@nakrovati

@nakrovati nakrovati commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Description

Optimize the POST /upload endpoint, and for POST /crud/items and PUT /crud/items/{id}, return only the data described in the guide.


PR Commands — comment on this PR to trigger (requires collaborator approval):

Command Description
/benchmark -f <framework> Run all benchmark tests
/benchmark -f <framework> -t <test> Run a specific test
/benchmark -f <framework> --save Run and save results (updates leaderboard on merge)

Always specify -f <framework>. Results are automatically compared against the current leaderboard.


Run benchmarks locally

You can validate and benchmark your framework locally with the lite script — no CPU pinning, fixed connection counts, all load generators run in Docker.

./scripts/validate.sh <framework>
./scripts/benchmark-lite.sh <framework> baseline
./scripts/benchmark-lite.sh --load-threads 4 <framework>

Requirements: Docker Engine on Linux. Load generators (gcannon, h2load, h2load-h3, wrk, ghz) are built as self-contained Docker images on first run.

@nakrovati nakrovati marked this pull request as ready for review June 24, 2026 01:41
@MDA2AV

MDA2AV commented Jun 26, 2026

Copy link
Copy Markdown
Owner

/benchmark -f phoenix-bandit

@github-actions

Copy link
Copy Markdown
Contributor

👋 /benchmark request received. A collaborator will review and approve the run.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: phoenix-bandit | Test: all tests

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 503,092 6542.0% 352MiB -0.3% +6.3%
baseline 4096 493,547 6428.1% 496MiB +2.3% +2.1%
pipelined 512 654,498 6520.0% 361MiB +1.1% +7.1%
pipelined 4096 632,610 6406.8% 512MiB +1.1% +3.9%
limited-conn 512 401,317 6283.1% 414MiB +0.9% -2.4%
limited-conn 4096 401,161 6237.7% 450MiB +0.5% +7.7%
json 4096 193,968 6297.9% 996MiB -3.2% -0.4%
json-comp 512 111,147 6319.2% 771MiB +1.2% +0.3%
json-comp 4096 93,113 6272.8% 1.8GiB -1.4% +5.9%
json-comp 16384 90,111 6050.5% 1.8GiB -4.7% -5.3%
upload 32 2,132 2103.5% 354MiB +202.8% -59.7%
upload 256 2,816 5584.5% 846MiB +311.1% -73.3%
api-4 256 10,301 399.0% 348MiB -1.3% -1.7%
api-16 1024 40,244 1573.0% 758MiB +0.3% -3.6%
static 1024 27,118 1192.6% 404MiB ~0% +0.5%
static 4096 26,484 1211.8% 568MiB +3.2% -1.9%
static 6800 26,067 1220.2% 610MiB +2.1% +1.5%
async-db 1024 9,612 1197.1% 522MiB -0.7% -7.3%
crud 4096 338,679 5594.3% 727MiB -0.5% -4.6%
baseline-h2 256 216,284 6429.4% 819MiB +0.4% +0.7%
baseline-h2 1024 203,876 6442.5% 1.7GiB ~0% ~0%
static-h2 256 24,927 1460.3% 978MiB +2.3% +22.7%
static-h2 1024 24,364 1538.1% 1.3GiB +1.1% ~0%
echo-ws 512 1,428,953 6822.6% 380MiB +36.1% +8.9%
echo-ws 4096 1,372,345 6732.6% 498MiB +1.1% +0.8%
echo-ws 16384 1,270,422 6325.1% 675MiB -0.3% -1.5%
echo-ws-pipeline 512 1,999,510 6773.2% 333MiB +0.9% +1.2%
echo-ws-pipeline 4096 1,842,282 6578.4% 575MiB +0.3% -0.5%
echo-ws-pipeline 16384 1,821,854 6702.1% 683MiB +4.1% -11.5%
Full log
  Bandwidth:  13.34MB/s
  WS upgrades: 512
  WS frames:   9997551
  Latency samples: 10005492 / 9997551 responses (100.1%)
[info] CPU 6773.2% | Mem 333MiB

=== Best: 1999510 req/s (CPU: 6773.2%, Mem: 333MiB) ===
[info] saved results/echo-ws-pipeline/512/phoenix-bandit.json
httparena-bench-phoenix-bandit
httparena-bench-phoenix-bandit

==============================================
=== phoenix-bandit / echo-ws-pipeline / 4096c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   29.64ms   35.40ms   37.50ms   51.40ms   78.10ms

  9273652 frames sent in 5.00s, 9211412 frames received
  Throughput: 1.84M req/s
  Bandwidth:  12.48MB/s
  WS upgrades: 4096
  WS frames:   9211412
  Latency samples: 9211411 / 9211412 responses (100.0%)
[info] CPU 6578.4% | Mem 575MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   30.04ms   35.30ms   37.50ms   43.70ms   61.30ms

  9256075 frames sent in 5.00s, 9201115 frames received
  Throughput: 1.84M req/s
  Bandwidth:  12.45MB/s
  WS upgrades: 4095
  WS frames:   9201115
  Latency samples: 9201093 / 9201115 responses (100.0%)
[info] CPU 6726.7% | Mem 574MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   30.27ms   35.40ms   37.10ms   44.10ms   58.30ms

  9168823 frames sent in 5.00s, 9109863 frames received
  Throughput: 1.82M req/s
  Bandwidth:  12.34MB/s
  WS upgrades: 4095
  WS frames:   9109863
  Latency samples: 9109852 / 9109863 responses (100.0%)
[info] CPU 6441.9% | Mem 582MiB

=== Best: 1842282 req/s (CPU: 6578.4%, Mem: 575MiB) ===
[info] saved results/echo-ws-pipeline/4096/phoenix-bandit.json
httparena-bench-phoenix-bandit
httparena-bench-phoenix-bandit

==============================================
=== phoenix-bandit / echo-ws-pipeline / 16384c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   37.37ms   42.20ms   61.20ms   67.50ms   84.30ms

  9195910 frames sent in 5.00s, 9090822 frames received
  Throughput: 1.82M req/s
  Bandwidth:  12.45MB/s
  WS upgrades: 6598
  WS frames:   9090822
  Latency samples: 9090822 / 9090822 responses (100.0%)
[info] CPU 6355.0% | Mem 692MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   43.57ms   50.30ms   64.20ms   68.60ms   78.40ms

  9215216 frames sent in 5.00s, 9109273 frames received
  Throughput: 1.82M req/s
  Bandwidth:  12.48MB/s
  WS upgrades: 6976
  WS frames:   9109273
  Latency samples: 9109256 / 9109273 responses (100.0%)
[info] CPU 6702.1% | Mem 683MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   48.94ms   54.40ms   69.10ms   75.50ms   85.60ms

  8960698 frames sent in 5.00s, 8843050 frames received
  Throughput: 1.77M req/s
  Bandwidth:  12.16MB/s
  WS upgrades: 7353
  WS frames:   8843050
  Latency samples: 8843050 / 8843050 responses (100.0%)
[info] CPU 6248.5% | Mem 759MiB

=== Best: 1821854 req/s (CPU: 6702.1%, Mem: 683MiB) ===
[info] saved results/echo-ws-pipeline/16384/phoenix-bandit.json
httparena-bench-phoenix-bandit
httparena-bench-phoenix-bandit
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-16-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-4-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/async-db-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-h2-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-h2-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/crud-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-6800.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-h2-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-h2-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-32.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
httparena-redis
[info] restoring loopback MTU to 65536

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants