feat(benchmarks): report PostgreSQL's own time next to the wall clock

[jonaas-dev]

A review of what the benchmarks measure, rather than whether they run. Three findings, all verified against 100k rows.

1. The wall clock was mostly the client, not the database

	server (EXPLAIN)	total (wall)	client
SELECT * LIMIT 10000	0.31 ms	17.22 ms	98%
SELECT id, name, email LIMIT 10000	0.46 ms	2.87 ms	84%

98% of the headline result was psycopg2 building tuples. And server-side SELECT * is faster than the three-column projection — returning a stored tuple needs no projection work, while picking three columns means building a new one. The number pointed the opposite way to the database it claimed to be about.

measure() now also captures Execution Time from EXPLAIN (ANALYZE, TIMING OFF). Every table and chart reports both; solid lines are the total, dashed are PostgreSQL alone.

SELECT * is still worth avoiding — but for the true reason, which is that it strains your application and your network, not the planner.

2. The index benchmark undersold indexes by ~4x

	no index	indexed	speedup
server	5.25 ms	0.68 ms	7.7x
total	8.16 ms	3.83 ms	2.1x

Shipping ~1,600 wide rows costs the same however they were found, so SELECT * swamps the lookup. A benchmark titled "why B-tree indexes are not optional" was showing 2.1x for a reason that has nothing to do with indexes. Both numbers are now visible, which is the real lesson: an index speeds up finding rows, not sending them.

3. JOIN vs IN vs EXISTS compared three spellings of one plan

The seed did SELECT id FROM users WHERE random() < 0.3 — every user got at most one order. With 1:1 data JOIN never fans out, so all three returned identical row sets (24,208) and IN/EXISTS produced byte-identical plans (Hash Semi Join, cost 875.51..4680.36). The distinction the benchmark exists to show was the one the data prevented.

Orders are now 1:N (up to 5 per user, avg 2.58). At amount > 50, JOIN emits 225,245 rows against 95,077 — it is the fastest of the three on the server (40.8 vs 47.1 / 55.8 ms) and the slowest overall (160.6 vs 83.4 / 78.3 ms), because it ships 2.4x more rows.

README

Corrected throughout, including a claim I introduced myself: that the TEXT bio column "dominates the row width". It averages 17.9 bytes, 10.2% of the row. The real ratio is 143 vs 30 bytes, which is what the ~5x gap actually reflects.

Verification

• ruff check . clean, 60 tests (was 54), 92.8% coverage.
• New integration gates: every benchmark must report server timings, and the orders table must be 1:N with a demonstrable fan-out. Confirmed the 1:N gate fails on the old data model (max 1 order/user, JOIN 2441 = IN 2441) before trusting it.
• Full compose stack rebuilt: all four benchmarks return 200 with charts and server columns.

🤖 Generated with Claude Code

https://claude.ai/code/session_014abw4B6YUf54giaEyPQpbo