Skip to content

poundingwater/the-event-loop-death-test

Repository files navigation

The Event-Loop Death Test

A production-grade benchmark suite for Go, Node.js (Express), and Python (FastAPI)
Built for the System Design Breakdown series — aimed at founders and CTOs who want real numbers, not synthetic toy tests.


🏗️ Architecture

                ┌─────────────────────────────────────────────┐
                │            docker-compose.yml               │
                │                                             │
                │  ┌───────────┐    ┌────────────────────┐   │
                │  │  go-api   │◄──►│  postgres-go       │   │
                │  │  :8081    │    │  (isolated network) │   │
                │  └───────────┘    └────────────────────┘   │
                │                                             │
                │  ┌───────────┐    ┌────────────────────┐   │
                │  │ node-api  │◄──►│  postgres-node      │   │
                │  │  :8082    │    │  (isolated network) │   │
                │  └───────────┘    └────────────────────┘   │
                │                                             │
                │  ┌───────────┐    ┌────────────────────┐   │
                │  │python-api │◄──►│  postgres-python    │   │
                │  │  :8083    │    │  (isolated network) │   │
                │  └───────────┘    └────────────────────┘   │
                └─────────────────────────────────────────────┘

Every container is constrained to 1 vCPU / 512 MB RAM. Every database starts with a 100,000-row seed. Connection pools are capped at 50 connections across all three runtimes.


📂 Repository Structure

.
├── docker-compose.yml         # Orchestration (isolated Docker profiles)
├── run-benchmarks.sh          # One-command benchmark runner
├── db/
│   └── init.sql               # Schema + 100k seed rows
├── k6/
│   ├── event_loop_test.js     # CPU stress + DB read (up to 200 VUs)
│   └── cloud_bill_test.js     # Webhook throughput (up to 500 VUs)
├── go-backend/
│   ├── main.go                # net/http + pgx/v5 pool
│   ├── go.mod
│   ├── go.sum
│   └── Dockerfile
├── node-backend/
│   ├── server.js              # Express + pg-pool (intentionally blocks event loop)
│   ├── package.json
│   └── Dockerfile
└── python-backend/
    ├── main.py                # FastAPI + SQLAlchemy async + asyncpg
    ├── requirements.txt
    └── Dockerfile

⚡ Quick Start

Prerequisites

Tool Version
Docker + Docker Compose v2 latest
k6 ≥ 0.49

Install k6: https://k6.io/docs/get-started/installation/

Run all benchmarks

./run-benchmarks.sh          # runs Go → Node → Python sequentially

Run a single language

./run-benchmarks.sh go
./run-benchmarks.sh node
./run-benchmarks.sh python

Results are saved to ./results/ as JSON + log files.


🧪 Benchmark Tests

Test 1: The Event-Loop Death Test (k6/event_loop_test.js)

Phase VUs Duration
Warm-up 0 → 50 30s
Sustained 50 90s
Stress ramp 50 → 200 60s
Peak stress 200 120s
Cool-down 200 → 0 30s

Endpoints hit:

  • GET /api/data?user_id=<random> — fetches a user row from Postgres (light I/O)
  • POST /api/compute — runs 200k iterations of SHA-256 chaining (heavy CPU)

Key insight: Under concurrent load, the Node.js and Python event loops are fully blocked by the synchronous compute handler. All other requests queue up behind it. Go runs each request in its own goroutine so compute and I/O requests never block each other.

Test 2: The $10,000 Cloud-Bill Test (k6/cloud_bill_test.js)

Phase VUs Duration
Viral spike 0 → 100 30s
Sustained spike 100 60s
Overload ramp 100 → 500 60s
Peak (cloud bill) 500 120s
Partial recovery 500 → 50 30s
Degraded normal 50 60s
Cool-down 50 → 0 30s

Endpoint hit:

  • POST /api/webhook — validates Authorization: Bearer <token> against DB, writes an event_logs row, returns 201

Key insight: This test stresses DB connection pool exhaustion. At 500 VUs with a 50-connection pool, every framework must queue or shed load. The difference in error rates and p99 latency reveals pool management quality.


📊 Metrics Captured

Metric Description
http_req_duration End-to-end latency (avg, p90, p95, p99, max)
http_req_failed Error rate
http_reqs Total RPS
compute_endpoint_latency CPU endpoint latency only
data_endpoint_latency DB-read endpoint latency only
webhook_p99_latency Webhook endpoint p99 latency

🔬 API Reference

All three backends expose identical endpoints:

Method Path Description
GET /health Liveness probe + pool stats
GET /api/data?user_id=<n> Fetch user record from Postgres
POST /api/compute CPU-bound SHA-256 hash chain
POST /api/webhook Auth + append-only event log write

POST /api/compute

{
  "iterations": 200000,
  "data": [{"index": 0, "value": 123.45, "label": "item_0"}]
}

POST /api/webhook

Authorization: Bearer tok_static_benchmark_k6_00000001
Content-Type: application/json
{
  "event": "user.signup",
  "timestamp": "2025-01-01T00:00:00Z",
  "data": {"user_id": 42}
}

🛠️ Connection Pool Config

Setting Go (pgx/v5) Node (pg-pool) Python (asyncpg/SQLAlchemy)
Max connections 50 50 50
Min / idle connections 25
Max overflow 0 (strict)
Max conn lifetime 30 min 30 min
Idle timeout 5 min 10 s 5 min (pool_recycle)

📝 Notes for Content Creators

  • The Node compute handler is intentionally synchronous (no worker_threads) so the event-loop blocking is observable. This is the pedagogically correct representation of the common mistake.
  • The Python compute handler similarly runs synchronous hashing in an async handler, demonstrating GIL + event-loop interaction.
  • The Go handler runs each request in a separate goroutine; the SHA-256 computation does not block any other requests.
  • Each language run fully destroys its Docker volumes before the next language starts. This eliminates "warm Postgres cache" bias.

About

A production-grade benchmark suite for Go, Node.js (Express), and Python (FastAPI).

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Contributors