perf(stats): answer top domains and top clients from one index scan

[henry40408]

Summary

The Statistics page and every dashboard tick ask for top domains and top clients together, but each list read its own index: (domain, timestamp) and (client_ip, doh_token, timestamp). Both indexes put the grouped columns first, so the time window could not narrow either scan. A 24-hour question skip-scanned most of each index anyway.

• Migration 15 drops idx_query_logs_client_ts (only top clients used it) and adds idx_query_logs_ts_domain_client ON (timestamp, domain, client_ip, doh_token).
• traffic_lists_since groups by (domain, client_ip, doh_token) once and builds top domains, the distinct-domain count and top clients in Rust. compute_range_stats (Statistics) and the new compute_top_domains_and_clients (dashboard SSR and the stats snapshot) call it once. top_clients_since now reads from it.
• idx_query_logs_domain_ts stays. The query log's domain search seeks it by prefix. Without it, searching a prefix nobody has queried reads the whole table:

  GLOB 'note*'  with domain_ts: 2117  without: 231
  GLOB 'zzz*'  with domain_ts: 3  without: 12170
  LIKE '%telemetry%'  with domain_ts: 212  without: 212
  

  Replacing both indexes measured cheaper on every other axis and was rejected for this reason.
• domain_stats_since (domain suggestions, API) keeps its statement. Both spellings now break ties by name, so they return the same rows.

Page misses

All figures come from the same 370k-row database (7 days of data, default retention).

Statistics page, stats_page_miss_bench, 30d (whole table). The previous run on main read 9 758:

  range_stats (all readings, charts, both lists)      7923       30.9
  db_health                                         14        0.1
    top_domains alone (domain index)              3973       15.5
    both lists alone                              5770       22.5
  PAGE TOTAL (whole visit)                        7937       31.0

Dashboard tick, each statement on a fresh connection. The folded lists match the two statements they replace (diffed):

domains identical
clients identical
v14 count_1m 3
v14 summary 2152
v14 domains 1831
v14 clients 558
v14 upstreams 211
v15 count_1m 3
v15 summary 2152
v15 lists 747
v15 upstreams 211
TICK v14 4755 v15 3113

	before	after
Statistics visit (whole table)	9 758	7 937 (−19%)
Dashboard tick	4 755	3 113 (−35%)
Index space	client index 14.1 MiB	new index 22.5 MiB (+8.4 MiB)
Pages written per 500-row batch	53	52

Tests

• stats_db_test::traffic_lists_answer_what_the_separate_lists_did: 7 domains × 5 IPs × with/without a DoH token, with tied counts and a row just before the window. Domains match domain_stats_since row for row, and clients match counts computed directly from the entries. An empty-window case is included.
• stats_page_miss_test::a_short_window_reads_a_short_stretch_of_the_traffic_lists: a tenth of the window must read under a quarter of the pages, which holds the index to being timestamp-first. the_page_reads_less_than_its_readings_do_separately now also counts the client list.
• db::tests: the fresh schema has the new index and no client index; a v9 database migrates to the same state; the lists' plan is COVERING INDEX idx_query_logs_ts_domain_client.
• cargo nextest run: 680 passed. fmt and clippy clean. E2E was not run locally; CI runs it.

🤖 Generated with Claude Code

[codecov]

Codecov Report

❌ Patch coverage is 96.39640% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.14%. Comparing base (c82c657) to head (77d3471).

Files with missing lines	Patch %	Lines
src/admin/pages.rs	88.23%	2 Missing ⚠️
src/db.rs	97.50%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #277      +/-   ##
==========================================
- Coverage   91.27%   91.14%   -0.13%     
==========================================
  Files          31       31              
  Lines       11342    11401      +59     
==========================================
+ Hits        10352    10392      +40     
- Misses        990     1009      +19     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.