feat(benchmarks): end every benchmark with a conclusion, not just a chart - #48
Merged
Conversation
…hart The tool showed a chart, a table and an EXPLAIN plan, then left the reader to work out what any of it meant. For a project whose purpose is teaching basic SQL behaviour, the lesson was the missing part. Every benchmark now returns a Takeaway: a verdict, the evidence behind it, and the advice that follows. Every number in it is computed from the run that just happened, so it cannot drift away from the chart above it — and the dataclass refuses to be built without evidence. The organising idea is CostCentre: who actually pays. - select_star -> CLIENT. PostgreSQL is barely involved; the time goes to the driver decoding 16 columns instead of 3, and in a real service to re-serializing them to JSON. The bill lands in the API process and on the frontend waiting for it, which is the whole reason not to return everything. - index_usage -> TRANSFER. The index is worth ~8x to the database but ~2x to the caller. An index speeds up finding rows, never sending them. - pagination -> DATABASE. Unlike the above, this really is PostgreSQL doing the extra work, so only a query change fixes it. - join_vs_subquery -> TRANSFER. The three patterns answer different questions; JOIN ships 2.4x more rows for the same users. Read together they make one point: "the query is slow" is not a diagnosis. The same symptom has a different cure depending on whether the time goes to the planner, the wire or the driver. Conclusions are rendered above the chart in both the live and the stored result views, and persisted with the run so history keeps them. Also drops a misleading growth ratio in the pagination takeaway that divided by 0.01 ms — the resolution floor of EXPLAIN output, so the figure was noise. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014abw4B6YUf54giaEyPQpbo
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The tool showed a chart, a table and an EXPLAIN plan, then left the reader to work out what any of it meant. For a project whose purpose is teaching basic SQL behaviour, the lesson was the missing part.
Every benchmark now returns a
Takeaway: a verdict, the evidence behind it, and the advice that follows. Every number is computed from the run that just happened, so it cannot drift away from the chart above it — and the dataclass refuses to be constructed without evidence.The organising idea: who actually pays
CostCentreis what makes the four benchmarks a curriculum rather than four charts, because it decides the fix.select_starindex_usagepaginationjoin_vs_subqueryselect_staris the case that motivated this. PostgreSQL is barely involved — the time goes to the driver decoding 16 columns instead of 3, and in a real service to re-serializing them to JSON. The bill lands in your API process and on the frontend waiting for it, which is the actual reason not to return everything. Saying "SELECT * is slow" without saying where hides the fix.paginationis the deliberate counterexample: there the database really is doing the extra work, and no client tuning helps.Read together: "the query is slow" is not a diagnosis. The same symptom has a different cure depending on whether the time goes to the planner, the wire, or the driver. The README now opens with that arc.
Also
Verification
ruff check .clean, 71 tests (was 60), 93.4% coverage.select_starmust blame the client andpaginationmust blame the database — if a future change makesSELECT *look like a database problem, CI fails.🤖 Generated with Claude Code
https://claude.ai/code/session_014abw4B6YUf54giaEyPQpbo