Publish install counts as a stats.json sidecar - #98
Open
andrewkchan wants to merge 1 commit into
Open
Conversation
BB has collected a `plugin_installed` telemetry event since the store shipped, but nothing ever read it back, so the marketplace had no way to show how popular a listing is. Add a daily job that queries those counts and publishes them beside the manifest. `scripts/build-stats.mjs` queries PostHog's HogQL API for `count(DISTINCT distinct_id)` of `plugin_installed` grouped by `properties.plugin_id`, and writes `dist/stats.json`. Distinct installations, not raw events: a reinstall loop must not inflate a listing. Rows whose id fails the manifest's own id pattern, or whose count is not a non-negative safe integer, are dropped with a warning; keys are emitted sorted so an unchanged run republishes identical bytes and keeps its ETag. A run that ends with no usable counts exits non-zero without writing, so an outage or a rotated key leaves the last published sidecar serving instead of zeroing every counter in the store. The counts are a separate document rather than a field in marketplace.json on purpose: that schema is strict, so an unknown field there rejects the whole catalog on an older desktop, and the numbers move daily while the manifest sits unchanged behind a 304. `.github/workflows/stats.yml` runs it daily and uploads to the same R2 bucket as the manifest publish. It needs the POSTHOG_API_KEY secret and the POSTHOG_PROJECT_ID variable in the production environment, alongside the Cloudflare credentials publish.yml already uses. Verified against a stub PostHog endpoint: the request carries the bearer key and HogQL body, unusable rows are dropped, and the emitted document parses with BB's own `parseMarketplaceStatsJson`. Confirmed an empty result set fails without writing, and that `npm run build` still composes the manifest unchanged. > AGENT GENERATED
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was wrong
BB servers have sent a
plugin_installedtelemetry event since the store shipped, but nothing ever read it back. The registry had no way to tell anyone how many people use a listing, and the store card had no popularity signal at all.What changed
scripts/build-stats.mjs(new,npm run build:stats) queries PostHog's HogQL API forcount(DISTINCT distinct_id)ofplugin_installedgrouped byproperties.plugin_idand writesdist/stats.json.--printwrites to stdout for local inspection..github/workflows/stats.yml(new) runs it daily plus on demand and uploads tobb-marketplace/stats.jsonin the same bucket as the manifest publish.Counts are a separate document rather than a field in
marketplace.jsonon purpose: that schema is strict, so an unknown field there rejects the whole catalog on an older desktop, and the numbers move daily while the manifest sits unchanged behind a 304.Deploy requirement: the
productionenvironment needs a newPOSTHOG_API_KEYsecret (a personal API key with project read access) and aPOSTHOG_PROJECT_IDvariable. SetPOSTHOG_HOSTif the project is not on US cloud. Until they exist the scheduled job fails and uploads nothing; the manifest publish is unaffected.The consuming side is get-bb/bb: the server fetches this file on every catalog refresh and renders the number in the store, on mobile, and in
bb plugin search.How you verified
generatedAt.parseMarketplaceStatsJsonin a scratch test in the bb checkout — it parses and reports the right count.dist/stats.json.npm run buildstill composes the manifest unchanged (63 entries).Not run: the workflow itself, which needs the credentials above.
Fixes #