Skip to content

Keep the site's own monitor out of the statistics, and draw each tile its own line - #173

Open
pstayets wants to merge 1 commit into
TeoSlayer:mainfrom
pstayets:fix/stats-monitor-traffic
Open

pstayets wants to merge 1 commit into
TeoSlayer:mainfrom
pstayets:fix/stats-monitor-traffic

Conversation

@pstayets

Copy link
Copy Markdown
Contributor

What changed

Follow-up to #160, from a review of the numbers the dashboard showed on its first day: 91 installs against 3 copied commands, 56 "new" machines with nobody returning, and the installs and sessions tiles spiking together.

What was going on

  • The site's own monitor was being counted as people. Check the published downloads on a schedule #162's Downloads workflow runs every 30 minutes and does a real install on three fresh GitHub runners, plus downloads three sample binaries. Per run the dashboard recorded 6 installs, 5 installer fetches (3 of them "read in a browser"), 3 "ok" outcomes and 3 to 4 new installer machines. Runner addresses are never reused, so every one of them was "new" and none came back: the 100% churn was the schedule. Six per run times fifteen runs is 91.
  • Two classifier defaults made it worse. Anything that was not curl, wget, the CLI or a self-described bot counted as a desktop browser, so Node's fetch and PowerShell's web cmdlets were people. A Windows install has therefore never counted as an installer run, only as someone reading the script.
  • The installs tile drew the sessions line. The trend had no installs series, so both tiles passed sessions to their sparkline and spiked together by construction.
  • "New" could not mean anything yet. People have been counted since Sep 15 and machines were re-keyed with Fix the statistics funnel's measurement, lay the dashboard out as a story, and count what it could not see #160, so everyone was new and nobody could be returning until a whole range has passed.

Fixes

  • Both install scripts take SHELL_ONLINE_INSTALL_CHECK=1: every request then carries the user agent shell.online-install-check and nothing is reported. The workflow sets it, and puts the same agent on its own curl and PowerShell fetches of the scripts. The Worker counts that agent, shell.online-downloads-check, and monitors in general (uptime, pingdom, statuscake, checkly, site24x7, synthetic, "monitor") as crawlers: in the ledger, out of every figure about people.
  • HTTP libraries (node, undici, go-http-client, python, java, okhttp, axios, libwww, httpie) and PowerShell classify as tools.
  • The trend carries installs and started as their own series, with crawlers left out of every line; the installs and sessions tiles draw them.
  • "Since" is tracked per surface. While a whole range has not yet passed since a surface's people were first counted, the tiles, the funnel and the strip say "counted since ; new or returning cannot be told until " instead of a forced split. Machine rows keyed the old way (on or before 2026-09-15) are dropped once, marked as done, so they do not depress the first retention cohort as machines that never came back.
  • Pages a browser fetched ahead of time (Sec-Purpose: prefetch, Purpose: prefetch) are not views.
  • The 24h range says people are counted by UTC day, so over two days.

The sessions spike itself is not the workflow: nothing in CI creates sessions. After this deploy, the "CLI clients" panel and the machines cohort will say whether it is one version from many never-returning addresses (an automation) or people.

Verification

  • Regression tests: monitors as crawlers and libraries as tools, prefetch skipped (tests/analytics.test.ts); the trend's new series and crawler exclusion, per-surface since, and the one-time machine cleanup done exactly once, against SQLite (tests/stats-database.test.ts, worker/stats-store.test.ts); a check run sends the agent on the download and reports nothing (scripts/test-install.sh)
  • npm run check and npm run build:web: typecheck clean, 164 tests and the installer, Docker entrypoint, deploy guard, landing SEO and mobile checks pass; the web build succeeds
  • Rendered the dashboard headlessly from a snapshot (not committed): no errors, and the people notes read as intended
  • The workflow parses, and its install step carries the check agent
  • Not run: install.ps1 (no PowerShell here). Its change is a splatted -UserAgent on the two downloads and an early return in Report

Deploy notes

  • The workflow change is live on merge and stops the monitor being counted from its next run.
  • The Worker needs a deploy for the classifier, the trend series and the dashboard; the machine cleanup runs itself on the object's first start after that.
  • Nothing changes in the accounts app.

Changelog

Fixed entries under Unreleased.

🤖 Generated with Claude Code

… its own line

The scheduled download check installed on three fresh GitHub runners every
half hour, and the dashboard counted it: six installs, five installer
fetches, three "ok" outcomes and three or four new machines per run, which
is a day of ninety installs and fifty-six machines that never came back.
Both install scripts now take SHELL_ONLINE_INSTALL_CHECK=1, which puts a
check user agent on every request and reports nothing; the workflow sets
it, and the Worker counts that agent, and monitors in general, as crawlers.
HTTP libraries and PowerShell are tools rather than desktop browsers, so a
Node script is not a person and a Windows install is a run.

The installs tile drew the sessions line, since the trend had no installs
series; it now has installs and started sessions, crawlers left out of
every line. "New" is measured per surface from the day its people were
first counted, and until a whole range has passed since then the split is
replaced by the day it becomes meaningful. Machine rows keyed the old way
are dropped once. Prefetched pages are not views, and the 24h range says
people are counted by UTC day.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants