perf(stats): fold the Statistics page's readings out of the statistics rollups - #282
Merged
Merged
Conversation
…s rollups stats_scan_since reads query_stats_quarter and query_stats_metrics_hour, plus the table rows each window starts inside, in one four-arm statement. The API-only readers follow: window_metrics_since folds the metrics rollup, and timeline_multi_since and hourly_heatmap_since fold the quarter rollup whenever the bucket and offset are whole quarters. The API rounds tz_offset to the nearest quarter hour so it always is. No reader names idx_query_logs_ts_metrics any more. Page misses on a 1.48 M-row database (main -> this): Statistics 30d visit 11 641 -> 3 644 Statistics 7d visit 10 060 -> 1 015 API timeline 30d 9 476 -> 45 API heatmap 6 030 -> 38 Dashboard and query log readings unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #282 +/- ##
==========================================
+ Coverage 91.32% 91.36% +0.03%
==========================================
Files 31 31
Lines 11625 11691 +66
==========================================
+ Hits 10617 10681 +64
- Misses 1008 1010 +2 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fifth of the rollup series (#278 benches, #279 log counter, #280 rollups, #281 dashboard).
What changes
stats_scan_since— the Statistics page's one read for its breakdowns, latency and chart series — is now a single four-arm statement:query_stats_quarterfrom the earlier window's first whole quarter,query_stats_metrics_hourfrom the range's first whole hour, and thequery_logsrows each window starts inside (viaidx_query_logs_timestamp). A leading column tells the arms apart, so each window keeps its edge to the millisecond.window_metrics_since(breakdowns, highlights' latency),timeline_multi_sinceandhourly_heatmap_since(quarter rollup when bucket and offset are whole quarters; a table count otherwise)./api/stats/v2/timelineand/api/stats/v2/heatmaproundtz_offsetto the nearest 15 minutes. Every zone in use already is; the rounding keeps every API call on the rollup path.idx_query_logs_ts_metricsany more, which unblocks dropping it (next PR). Its plan and migration tests stay until then.Page misses
stats_page_miss_bench,BENCH_NOW=1789212354, main (d801b9c) → this branch.dashboard_page_miss_benchandlogs_page_miss_benchare identical to main on both databases (tick 227 / 190, logs sum 368 963 / 100 029).Tests
statistics_readings_equal_a_recount_of_the_table— scan, window metrics, timeline (5 buckets) and heatmap against a direct count ofquery_logs, for window starts before the data, on an hour, on a quarter, inside each and after; heatmap windows before/equal/after the range; offsets 0, +8h, −5h, +5:30, +5:45, −9:30 and an unaligned 7 min. Breaking each table arm in turn (scan, window, timeline, heatmap) makes it fail.the_statistics_readings_fold_rollups_rather_than_the_table— fails on main (stats scan 107 pages vs 168 for a full scan).stats_v2_tz_offset_is_rounded_to_a_quarter_hour— fails with the rounding removed.the_page_and_its_charts_are_one_metrics_scanandthe_heatmap_reads_the_narrowest_index_that_covers_it, which asserted index choices no reader makes now.🤖 Generated with Claude Code