squadrats-club

Club site for captured squares (squadrats/squadratinhos): profiles, places (municipality/district/country), history and personal analysis, served via GitHub Pages. Was squadrats-map until it outgrew the name.

The cycle is automatic: fetch-map-data.yml runs 6×/day, fetches the data directly from Squadrats' vector tiles (no manual KML export) and publishes if there is a real difference. It also publishes data/squadrats.json, plain totals (no geographic breakdown) for the club athletes, consumed by folha-do-clube the same way as its prs.json.

Data on its own branch (2026-08-14): the files the cron regenerates (the full list is in env.FILES of fetch-map-data.yml) live on the data branch, not main, committing on main made Pages (classic mode) rebuild on every run, and a code push of ours at the same time as the cron running collided with the bot's commit. The consumers (analise.html, club.html, folha-do-clube) read those files from raw.githubusercontent.com/JustAnotherDud/squadrats-club/data/data/..., not the Pages URL, faster (no rebuild) and without that collision risk. The border geojsons and adjacency.json are static, they stay only on main, normal relative path.

Until 2026-08-18 those dynamic files were also still committed on main, frozen at the migration day, 1.2 MB stating 14 Aug values while the site already served today's, ready to mislead anyone debugging from them. They moved to .gitignore: they exist only on the data branch. To work with them locally:

git checkout origin/data -- data/

Structure

index.html: the landing/hub: cards for the club map, profiles and history. GitHub Pages serves it at the root.
nav.js: the shared top nav bar (hub · Mapa · Perfis · Lugares · Histórico), self-injecting, marks the current page. Loaded by every page except analise.html. Full-screen map pages set <body data-nav="overlay"> so the bar is position:fixed and they offset their own layout by 38px. Also registers the service worker (sw.js) for the PWA.
PWA: manifest.json (root, start_url = the hub, display: standalone, colours from the site's dark palette: #0f1115 / #12141a), sw.js (minimal: caches the static shell, serves it from cache when the network fails; the data still comes from raw.githubusercontent), and the icons (icon.svg, icon-192/512.png, icon-maskable-512.png, apple-touch-icon.png, favicon.ico) generated by pipeline/gen_icons.py (needs pymupdf + pillow) from one shape: the Twemoji maple-leaf silhouette (assets/svg/1f341.svg, Twemoji, CC-BY 4.0), recoloured orange #ef7722, on Squadrats purple #663399 (the brand colour; their own logo is fill="#639"). The <link> tags are in every page (via the two stub templates for the generated ones), analise.html included for a consistent favicon; installing from anywhere still opens the hub (start_url). analise.html stays out of nav.js and the hub as before, so it is not reachable by navigation.
analise.html: the map owner's personal detailed map (was mapa.html, and index.html before that). Deliberately not in nav.js or the hub, it is personal analysis, more detailed and a different kind of thing from the club pages. Still lives at its own URL and is kept current by the pipeline. Public repo: this is "not in the navigation", not "private".
club.html: the club page: squadratinhos of the athletes with a Squadrats account. One colour per person, and two modes for the shared ones: How many (grey scale by number of owners, legible from afar, it is the default) and Who (stripes in the owners' colours, to zoom in and investigate). Neither view answers both questions well, and inventing a colour per combination does not scale: with 5 athletes that would be 26. Below 10px per square the stripes get thinner than legible and the Who mode falls back to the grey scale. No per-municipality classification or trophies. That is a different subject, a different data file (data/club.json). Squares drawn on canvas (~9000; in SVG the map choked when dragging).
atletas/: one static profile page per club athlete, at /atletas/<slug>.html (e.g. /atletas/ze.html), plus /atletas/index.html listing them. Pure stats, no map: the 8 counts, the athlete's daily-gains slice (click a +N in the Squadratinhos column to see which concelhos/ distritos it fell in, z17 only, so that column alone drills down), the squadratinhos overlap (exclusive/shared) and their position per region, with region names linking to lugares/<c|d>-<slug>.html. perfil.js + perfil.css are hand-written; the <slug>.html stubs are generated by pipeline/gen_profile_stubs.py (in the workflow, committed to main only when the athlete set changes, same pattern as the heartbeat). Each page fetches data/atletas/<slug>.json from the data branch via raw, like club.html does with club.json.
historico.html: club event feed: ranking changes per region (concelho + distrito, PT only), overtakes, new leaders, first square in a region, squadratinho milestones. Reverse-chronological, filters by athlete/region. Reads data/events.json + data/club_regioes.json from the data branch via raw. History starts 2026-07-26: from 15 Aug it comes from the per-region breakdown generated each run; before that it is reconstructed from club.json (recon_snapshots.py). Region names link to lugares/<c|d>-<slug>.html.
lugares/: one static page per concelho/distrito with activity (~100 today), at lugares/c-<slug>.html (concelho) / lugares/d-<slug>.html (distrito), the c-/d- prefix disambiguates the 18 names that are both. Shows the current ranking (+ %), the region's z14/z17 totals, a step-line chart of each athlete's captured squares over time (who led when), this region's events (filtered from events.json), and the neighbouring regions (data/adjacency.json). regiao.js + regiao.css are hand-written; the <key>.html stubs are generated by pipeline/gen_lugar_stubs.py (in the workflow, committed to main when the active-region list changes). Each page fetches data/regioes/<key>.json from the data branch via raw. lugares/index.html is the browsable index of all active regions, grouped by district (the district row is the group header, concelhos below), ordered by club union (squadratinhos any member covers, shared ones counted once; from club_regioes.json["uniao"], written by classify_club.py), with the region coverage percentage shown muted beside it. Data from data/regioes_index.json (one aggregate file, written by append_regioes.py). Linked from the navbar ("Lugares"), the hub, each region page's footer, and the history sidebar. lugares/pais-pt.html and lugares/pais-es.html are hand-written country pages sharing pais.js, built in the browser from club_regioes.json (uniao.by_pais, uniao.by_region for foreign provinces, per-athlete country/by_region) and stats.json (country_*, by_region_* totals); no per-country data file, no pipeline step of their own. Linked from the region index. Other countries (AD/DE/MA) are near-empty and intentionally skipped.
data/: files consumed by analise.html (simplified geometry, square classification) + squadrats.json (club totals) + trophies.json (yard/übersquadrat shapes, for the map's optional layers) + atletas/<slug>.json (per-athlete profile bundle, one file each + index.json; built by pipeline/build_profiles.py from the files above, no extra network) + events.json (club event feed, append-only; seeded once by pipeline/backfill_events.py, real club_regioes.json from 15 Aug, reconstructed from club.json for 26 Jul→14 Aug via recon_snapshots.py, kept current by pipeline/append_events.py as a run_all.py step that compares the previous club_regioes.json day by day, appends new events, idempotent across the 6 runs/day) + lugares/<key>.json (one per active region; append_regioes.py rewrites every active region's ranking/totals/union/neighbours from the current club_regioes.json each run, idempotent, no history) + gains_regioes.json (per-day squadratinho gains by concelho/distrito per athlete, the drill-down when you click a +N in a profile's daily-gains table; pipeline/backfill_gains_regioes.py seeds it from club.json history with the Classifier + an (x,y)→region cache, append_gains_regioes.py keeps it current by diffing two consecutive club_regioes.json days, no new classification. z17 only, so only the Squadratinhos column drills down)
pipeline/: fetches the map owner's squares and produces the files in data/
- tiles_fetch.py: main source: direct fetch from Squadrats' vector tiles (tiles1.squadrats.com/{uid}/trophies/{ts}/{z}/{x}/{y}.pbf, undocumented endpoint, see "Usage rules" below). Cascading discovery (z4→z7→z10, each level only explores children of tiles with coverage at the previous level), robust to an athlete having captures anywhere in the world, with no fixed bbox to guess. Mandatory self-validation: the reconstructed count of squadrats/squadratinhos must match exactly the size the server itself reports, if it does not, build_analise.py/fetch_club_totais.py blow up instead of publishing wrong data. The cascade only runs when needed: each athlete's z10 coverage is stored in data/scan_cache.json (derived coordinates, never raw tiles; coarser than what club.json already publishes) and the next run goes straight to those z10 tiles, it was ~2/3 of each run's requests. Safe because coverage never shrinks and the self-validation above invalidates the cache on its own: if the athlete captured in a new area, the reconstruction does not match size, full discovery runs (without repeating tiles already fetched) and the cache is rebuilt. Fetched at z10, not z12 (2026-08-15, see tiles_fetch.FETCH_ZOOM): tested live that the server does not simplify geometry at coarser zooms, z10 returns the exact total, with 16× fewer tiles than z12 for the same area.
- run_all.py: workflow entry point: runs the three consumers in the same process, so each athlete is scanned only once. In separate steps, the first athlete was scanned three times per run and two others twice, ~1200 duplicated requests. The cache is in tiles_fetch.scan_athlete. The three scripts still run standalone, for debugging.
- suggestions.py: progress suggestions -> data/suggestions.json. verify_rules() is the guard: it confirms we reproduce the yard/übersquadrat the server reports before publishing anything. corredores() looks for the cheapest path to connect a closed cluster to the yard, that is where the big gains are (isolated captures never go beyond +4/+5). The published gain always comes from recomputing the clusters, never from arithmetic on sizes.
- build_analise.py: builds the owner's detailed personal map. py build_analise.py --uid <firebase_uid> <out_dir>
- fetch_club_totais.py: plain totals (all 8 layers) for the club athletes → data/squadrats.json. Also updates data/daily_gains.json (gains per day, see daily_gains.py, baseline is the ultimo_total stored in the file itself, never git: in CI the checkout is shallow and history does not exist). Initial history (24 Jul onward) reconstructed once with backfill_daily_gains.py.
- fetch_club_squares.py: squadratinhos of the same athletes, with a bitmask of who has each square → data/club.json (fetch_club_totais.py only brings totals; this one brings the coordinates). The order of the ATLETAS list fixes the bit assignment: do not reorder without regenerating the file.
- kml_parse.py: XYZ grid maths: lon/lat ↔ tile conversion and reconstruction of individual squares (x, y, zoom). Historical name, it was born parsing hand-exported KML, that path removed 2026-08-18; only the geometry remained, shared by almost the whole pipeline (compute_grid_totals, classify_club, fetch_club_squares, fetch_club_totais, build_analise)
- classify.py: classifies each square into municipality/district (point-in-polygon with STRtree)
- compute_grid_totals.py: one-off, run manually, never by the pipeline: computes how many tiles (zoom14/17) exist in total in each municipality/district/country, using the same criterion as classify.py (tile centre). Output committed to refdata/grid_totals.json, recompute whenever the borders OR the classification rule (classify.py) change, so the totals never diverge from the criterion used on the captured ones.
- compute_adjacency.py: one-off: adjacency (which regions share a border, geom.buffer(eps).intersects()) + greedy coloring over analise.html's categorical palette so neighbours never share a colour in "Colours: region" mode. Output committed to data/adjacency.json. Buckets: concelhos/distritos (PT), provincias_es/laender_de/regioes_ma/ paroquias_ad (foreign level 2), municipios_es/municipios_de/ municipios_ma (foreign level 3, computed over the clipped foreign_muni/<CC>.geojson; re-run this whenever clip.py changes that set), and paises (country-country borders, from outlines/europe.geojson, for the neighbours section of country pages). Recompute if borders change or the clip set changes.
- refdata/: municipality/district borders at classification precision (coordinates rounded to 6 dp / ~0.1 m, which is lossless for the max-area classify: 0 flips in 310 k boundary tiles of every foreign municipality the club has touched). The data/ copies are the same geometry, Douglas-Peucker simplified for the browser to draw only (analise.html never classifies with them). Both are regenerated by refdata/prep.py (see its docstring); prep.py --check fails if a file on disk drifts from what prep would produce. Source is GADM 4.1, fetched and reshaped once by hand; prep.py does not re-fetch. Also here: grid_totals.json
- refdata/foreign/: precision geometry of foreign regions, one file per country (ES.geojson). Adding a new country (e.g. France) = add FR.geojson in the same format (properties.country + properties.region per feature), add it to prep.py's table, run prep.py; zero code changes in classify.py/build_analise.py.
- refdata/foreign_muni/: fine (municipality) geometry. Two files per country:
  - <CC>.full.geojson.gz: the source. Full GADM 4.1 municipality set, reshaped to {country, region, parent}, 6 dp, minified, gzipped. Lives on main (frozen GADM data, ~6.5 MB total for DE+ES+MA). Never loaded at runtime, only clip.py reads it. Germany is 10978 Gemeinden here.
  - <CC>.geojson: the artefact classify.py actually loads (~50 KB). Generated by clip.py, never hand-edited. Only the municipalities within 10 km of a captured square (real km, equirectangular-local projection). Re-running clip.py rebuilds it deterministically from the source + data/club.json, so it is reproducible by construction and the clip is no longer a one-way narrowing. clip.py also prunes grid_totals.json to the kept set. The geojson may hold a few municipalities with no grid_totals entry (in the buffer but no activity yet); a page for one of those shows captured squares without a total until compute_grid_totals.py is re-run. See "Foreign municipalities: clip to visited area" below.
- refdata/outlines/europe.geojson: country outlines (Natural Earth Admin 0, public domain, ~2 km simplified, {country, nome} per feature). Not for drawing, only so classify.py can put a country on a square captured somewhere with no region data yet, instead of dropping it. py prep.py --fetch-outlines regenerates it from source.
- spikes/: test/validation scripts used during development, not part of the production pipeline
.github/workflows/
- fetch-map-data.yml: the only one, cron 6×/day (01/05/09/13/17/20h UTC since 2026-09-11, was …/22h; last slot moved earlier because GitHub was landing it after midnight UTC ~half the days and gains near the boundary got dated to the wrong day, see the note in the workflow). ~4h of slack from the last slot to midnight now. It also carries the heartbeat step that keeps the schedule alive. If GitHub's scheduler keeps dropping/delaying runs after this: the next step (not done) is to trigger workflow_dispatch from a punctual external cron (Cloudflare Worker cron → the dispatch REST endpoint with a single-repo fine-grained PAT, or Task Scheduler on an always-on local box). Keeps all the Actions plumbing, fixes both the missed runs and the day boundary. Deliberately not adopted yet: it is new infra to maintain for a gain the earlier slot already covers in practice.

What each layer's `size` means

size does not mean the same thing in every layer:

Layer	`size` is…	Verified (map owner)
`squadrats` / `squadratinhos`	number of squares visited	392 squares / 5050 squares
`yard` / `yardinho`	number of squares in the largest closed cluster	90 squares in 1 cluster / 480 in 1
`ubersquadrat` / `ubersquadratinho`	the N of the largest full NxN square	6 → 6x6

"Closed cluster" = contiguous set of visited squares where each one also has its 4 neighbours (N/E/S/W) visited.

backyards / backyardinhos (number of closed clusters, the main one included) were dropped on 2026-09-04: they counted clusters rather than squares and their geometry contained the yard's, so the number was more confusing than useful. The server still sends the layer; the pipeline no longer reads it. The reverse-engineering is in git history up to that commit.

On the map: the yard/über shapes are in data/trophies.json and can be toggled in the chips at the top (off by default), following the active grid, in Squadrats they show yard/über, in Squadratinhos the "-inho" versions.

Vector-tile endpoint usage rules

tiles1.squadrats.com is not a public or documented API, it is data the athletes made public and whose links they shared voluntarily (the map URL at squadrats.com/map/{uid}/17 already exposes the UID). To avoid abuse:

Run at most 6×/day (fetch-map-data.yml, cron 7 1,5,9,13,17,20 * * *). It was weekly, then daily, went to 2×/day on 2026-08-08 (only justifiable after tiles_fetch.py's coverage cache cut the per-run cost by 68%) and to 6×/day on 2026-08-14, following the 1-request/athlete probe (see below), an intermediate step on purpose, not straight to 1×/hour. Worst case today (all 5 changed since the last run, FETCH_ZOOM=10, see below): ~138 requests (it was ~2032 at z12, before 2026-08-15), at 6×/day that is ~828/day if it always happened, but it does not: with the probe, whoever did not change costs only 1 request, and running more often does not make anyone change more often. In practice most runs are well below this. Do not raise the frequency without recomputing this budget by the WORST case, not the average. Investigated 2026-09-11 and decided against raising it: GitHub already drops ~1/3 of the 6 daily triggers and delays the rest 2-3h, so a higher nominal frequency does not buy proportional real frequency, and it is more load on a third party's server with no new information on quiet intervals. The last slot moved 22h→20h that day for the day-boundary reason (above); that is placement, not frequency.
Identifiable User-Agent (squadrats-club-sync/1.0 (+github.com/...), see tiles_fetch.py)
Low concurrency (DISCOVERY_CONCURRENCY=4/FETCH_CONCURRENCY=6 in tiles_fetch.py, up from 4 on 2026-08-08, tested without 500s/new slowness; do not raise further without repeating that check)
Never publish the raw tiles, only the derived aggregates (tile_info_*.json, stats.json, squadrats.json)
The {TS} in the URL has to be fresh (int(time.time()*1000)) on every request, the server ignores the value but the URL is the cache key; a fixed timestamp can return frozen data with no error at all (silent failure). Updated 2026-08-14: tested live (with fixed ts and with real ts) and the response always comes with Cache-Control: no-store, not max-age=31536000 as this note said before, there is no CDN or response cache to exploit (not worth trying to save requests that way). Keep ts fresh anyway: the "frozen data" warning may not be only about HTTP cache, and there is no need to touch it, the real request saving is in tiles_fetch.py's probe (see below).
1-request probe before scanning for real (2026-08-14, tiles_fetch._probe_sem_alteracoes): reads the global squadratinhos total embedded in any already-known tile and compares it with the last published one (data/squadrats.json). Squadratinhos is the finest grid, any gain in the other 7 layers always implies a new squadratinho, never the other way round, so this single count is enough to confirm "nothing changed" and skip the rest of that athlete's scan. Cuts an unchanged run of ~2032 requests to ~5 (1 per athlete); only pays the full cost for whoever actually captured something. Strict-equality comparison (not "≥"): an activity deleted/cropped after being published can make the total go DOWN, and that also has to trigger the full scan.
Fine fetch at z10, not z12 (2026-08-15, tiles_fetch.FETCH_ZOOM): tested live, the server returns the exact square geometry regardless of the requested zoom, only clipped to a larger bbox, without simplifying. Confirmed with full reconstruction == declared size at z10/z9/z8/z7 (the 5 club athletes, and also on a 3984-cell z10 account to test scale). It fails from z4 (loses squares) and blows up at z2/z0, never go near that. z10 leaves 4 zooms of margin, and cuts the cost of whoever actually changed by another 16× (672 -> 42 requests for the map owner). There would still be margin for z9/z8/z7, deliberately not used, the marginal gain does not justify risking closer to the unknown limit.

Note on duplicate municipality names

Two pairs of municipalities have the same name in Portugal: Calheta (Azores/Madeira) and Lagoa (Azores/Algarve). The border files (concelhos_pt.geojson, refdata and display) already come with this disambiguated, Calheta (Açores), Calheta (Madeira), Lagoa (Açores), Lagoa (Faro), generated from GADM's NAME_1 (district/region). Without this, the two municipalities collided on the same key and one of them lost all captures/totals in the aggregation.

Foreign regions (Spain and future ones)

Squares outside Portugal are classified by Spanish province when geometry is available (refdata/foreign/ES.geojson, 52 provinces/GADM ESP level 2, names corrected, Asturias, Cantabria, Madrid, León, etc., without GADM's missing spaces). Without geometry for the country in question (e.g. a square in France, today), classify.py falls back to the country outline (refdata/outlines/europe.geojson, Natural Earth) and returns country=<CC>, region=null: the square shows up on analise.html and club.html with a flag (bandeiraSvg gives a generic pennant for countries without a specific one) and "sem dados por região", counted in stats.foreign[zkey].by_country_nodata, instead of vanishing into unclassified. Only outside Europe (or mid-ocean) does it stay fully null.

Foreign regions now have totals (stats.json → by_region_es / by_region_de / by_region_ma / by_region_ad, and by_municipio_*), a club union per region (classify_club.classify_uniao → club_regioes.json uniao.by_region, plus exclusivos.by_region), and adjacency (adjacency.json → provincias_es, laender_de, regioes_ma, paroquias_ad). The pais-es country page uses all of it: a "Por região" table with the club union, % and leader per province, each row expandable to the per-athlete ranking and the province's neighbours.

Standalone province pages (lugares/es-p-<slug>.html, one per active ES province) are deliberately not generated. The data, the union, the adjacency and the sub-region table component all exist now, but each page would carry a ranking + union + neighbours and nothing else (no events, since eventos.py only tracks PT buckets; no dispute history), for ~10 provinces nobody contests. The country-page expansion covers the same ground without 10 thin pages or a URL per province. If the club starts riding Spain enough that provinces get contested, what a real page would take: generalise regioes.py from (nivel, nome) to (cc, nivel, nome) with a key scheme (es-p-…, es-m-…), teach gen_lugar_stubs.py + regiao.js the foreign level, add the foreign regions to the append_regioes.py loop, and extend eventos.py to foreign buckets (or accept no events feed). Foreign municipality pages stay out regardless: no municipal adjacency, and the pages would be trivially thin.

Foreign municipalities: clip to visited area

The full cycle:

Source: refdata/foreign_muni/<CC>.full.geojson.gz. The whole GADM 4.1 municipality set (Gemeinde / municipio / cercle) reshaped to {country, region, parent}, 6 dp, minified, gzipped. On main, ~6.5 MB for DE+ES+MA. Never loaded by classify.py or anything at runtime, only by clip.py.
Artefact: refdata/foreign_muni/<CC>.geojson. What classify.py loads (~50 KB). Generated by clip.py, never hand-edited. Holds only the municipalities within 10 km of a captured square (real km: clip.py projects country + points to a local equirectangular frame, x·cos(lat0), before the dwithin; a plain degree threshold gave ~6 km east-west at 53°N). The seed is the data/club.json square points, not the classified municipalities (that was circular: a square with no municipality match never seeded the clip and stayed excluded).
Reproducible: re-running clip.py rebuilds <CC>.geojson from <CC>.full.geojson.gz + the current club.json. Same inputs → same output. The clip is no longer a one-way narrowing: a municipality that drops out comes back automatically when a square lands near it again.
grid_totals.json: clip.py prunes it to the kept set. It never adds entries (only compute_grid_totals.py computes totals). So <CC>.geojson can hold municipalities with no grid_totals row: in the buffer but no activity. A page for one of those shows captured squares without a % until compute_grid_totals.py is re-run. Harmless: those municipalities have no activity by definition.

When a member rides somewhere new:

New country (no foreign_muni/<CC>.full.geojson.gz yet): download the country from GADM 4.1 (gadm.org → level 4 GeoJSON, e.g. gadm41_CHE_4.json), reshape each feature to properties = {country: "CH", region: <municipality name>, parent: <region name>} (disambiguate duplicate names the way Calheta (Açores) / Lagoa (Faro) are), truncate coordinates to 6 dp, minify, gzip to pipeline/refdata/foreign_muni/CH.full.geojson.gz, then run the clip (step 3). Also add refdata/foreign/CH.geojson (level 2 regions, same shape) if you want a region level, and both to prep.py's table.
Same country, wider area than before: nothing to download, the source .full.geojson.gz is already there. Just run the clip.
Run the clip: git checkout origin/data -- data/club.json (the seed points live there), then py pipeline/refdata/clip.py CH (or no argument for all). It rewrites foreign_muni/<CC>.geojson and prunes grid_totals.json. Commit both. Idempotent.
If the kept set changed, re-run py pipeline/compute_adjacency.py so municipios_<cc> in adjacency.json matches, and commit that too.

Degradation is visible, not silent. If a member captures squares in a country that has a foreign_muni file but outside the 10 km clip, those squares classify to region level only (no municipality). build_analise.py and classify_club.py count them and print AVISO: <CC>: N square(s) ... fora do recorte de 10 km at the end of every run, and the count is in stats.foreign.<zkey>.muni_clip_misses. That is the signal to re-run the clip. (Coastal squares just off a municipality's land edge get that municipality by proximity (_foreign_municipio, same-country only), so they are not counted as misses.)

Running the pipeline manually

py -m pip install -r requirements.txt
py pipeline/build_analise.py --uid <FIREBASE_UID> data   # a club athlete, via vector tiles
py pipeline/fetch_club_totais.py data                 # totals for the club athletes

Produces data/tile_info_squadrats.json, data/tile_info_squadratinhos.json, data/stats.json and data/squadrats.json.

ATHLETES_JSON (env) holds {name: firebase_uid, ...} for the club athletes, kept out of the code because it is third-party data (see pipeline/atletas.py). In CI it comes from a repo secret; locally, export it by hand. The JSON order fixes the bitmask bit order, do not reorder without regenerating data/club.json.

Architecture

Vector tiles (`fetch-map-data.yml`, cron 6×/day)

[fetch-map-data.yml, cron 6x/day or workflow_dispatch]
                    |
                    v
[tiles_fetch.py] --fetch--> tiles1.squadrats.com/{uid}/trophies/{ts}/{z}/{x}/{y}.pbf
   cascading discovery z4->z7->z10, then fetch those z10 tiles directly
   self-validation: reconstructed count == server `size`, else blow up
                    |
      +-------------+-------------+
      |                           |
      v                           v
[build_analise.py --uid]  [fetch_club_totais.py]
  map owner -> classify.py   club athletes -> plain totals (8 layers)
  -> tile_info_*.json,       -> data/squadrats.json
     data/stats.json
      |                           |
      +-------------+-------------+
                    |
                    v
          conditional commit (only on a real diff)
                    |
                    v
          [GitHub Pages: automatic redeploy]

KML/Drive path, removed (2026-08-18)

There was a second path, kept as a fallback: manual KML export to a Google Drive folder → changes.watch() → two Supabase Edge Functions → repository_dispatch → process-kml.yml. It was removed entirely (workflows, Edge Functions, migrations, download_kml.py, t1_*/t5_* spikes, google-* deps from requirements.txt), it had not run since 27 Jul 2026 and cost two crons, a table and four secrets to sit idle.

If the vector-tile endpoint is blocked without warning (undocumented, no public API, see "Usage rules" above), the path has to be rebuilt; it is in history up to the commit before this removal. The grid geometry (kml_parse.py) stayed, it is shared by the live pipeline.

Secrets to revoke (no longer used by anything): GOOGLE_SA_KEY, GH_PAT, CHANNEL_TOKEN, SETUP_TOKEN, in Supabase, in the GitHub Actions secrets, and the service account key squadrats-drive-sa@garmin-calendar-sync-488923 in Google Cloud. Locally, .secrets/ (git-ignored) only has files from this path.

Known limitations / tech debt

The athlete roster now derives from data at runtime. The pages read the names (in bit order) from club.json (.atletas, has nome + slug) or club_regioes.json (.atletas keys): index.html, club.html, historico.html. The only hand-kept fallback left is COR_FALLBACK in shared.js: one object, name → colour, used only when the page is offline (no club.json, no membros_cores.json). Colours otherwise have their single runtime source in membros_cores.json. atletas/index.html is still a build artefact (gen_profile_stubs.py), so its 5 <li> are literal by design. If someone leaves the club, their historical events/gains still render (grey from COR_FALLBACK's miss, name from the data, profile link slugified).
The pipeline's ATLETAS_ORDEM lives in eventos.py; regioes.py imports it from there. eventos.py deliberately has no atletas import so it stays usable/testable without ATHLETES_JSON.

Debugging: "the map stopped updating"

fetch-map-data.yml runs but fails. The most likely cause is the self-validation catching an incomplete scan, check the run logs, the message says which layer/UID did not match the server's size. If it is an athlete with captures somewhere very remote that the cascade levels (DISCOVERY_LEVELS = (4, 7, 10) in tiles_fetch.py) did not reach, adjust the levels or run manually with a more specific bbox= first to confirm where the missing coverage is.
UID returns 500. SquadratsHttpError, the UID changed or became invalid. Confirm at squadrats.com/map/{uid}/17 that the athlete's map still opens.
The vector-tile endpoint changed or is blocked. It is not documented, it can change without warning (see "Usage rules" above). There is no automatic fallback anymore, the KML/Drive path was removed on 2026-08-18 (see the architecture section). Recovering means rebuilding it from history, or rewriting tiles_fetch.py for whatever new source there is.
GitHub disabled the schedule due to inactivity (60 days without commits). Run workflow_dispatch on fetch-map-data.yml once, it reactivates the schedule. That workflow's heartbeat step exists so this never gets to happen.

Navigation Menu

Repository files navigation

squadrats-club

Structure

What each layer's `size` means

Vector-tile endpoint usage rules

Note on duplicate municipality names

Foreign regions (Spain and future ones)

Foreign municipalities: clip to visited area

Running the pipeline manually

Architecture

Vector tiles (`fetch-map-data.yml`, cron 6×/day)

KML/Drive path, removed (2026-08-18)

Known limitations / tech debt

Debugging: "the map stopped updating"

See also

About

Releases

Packages

Contributors

Languages

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
atletas		atletas
data		data
fonts		fonts
lugares		lugares
pipeline		pipeline
regioes		regioes
.gitattributes		.gitattributes
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
analise.html		analise.html
apple-touch-icon.png		apple-touch-icon.png
club.html		club.html
favicon-16.png		favicon-16.png
favicon-32.png		favicon-32.png
favicon.ico		favicon.ico
ganhos.html		ganhos.html
heartbeat.txt		heartbeat.txt
historico.html		historico.html
icon-192.png		icon-192.png
icon-512.png		icon-512.png
icon-maskable-512.png		icon-maskable-512.png
icon.svg		icon.svg
index.html		index.html
manifest.json		manifest.json
nav.js		nav.js
pytest.ini		pytest.ini
requirements.txt		requirements.txt
shared.js		shared.js
site.css		site.css
sw.js		sw.js

Latest commit

History

Folders and files

Repository files navigation

squadrats-club

Structure

What each layer's size means

Vector-tile endpoint usage rules

Note on duplicate municipality names

Foreign regions (Spain and future ones)

Foreign municipalities: clip to visited area

Running the pipeline manually

Architecture

Vector tiles (fetch-map-data.yml, cron 6×/day)

KML/Drive path, removed (2026-08-18)

Known limitations / tech debt

Debugging: "the map stopped updating"

See also

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages

What each layer's `size` means

Vector tiles (`fetch-map-data.yml`, cron 6×/day)