fix(pipeline): freshness repairs — caps, recent-first fetch, hot-bill pass, honest quiet-week verdict, metrics keyspace#88
Merged
Conversation
MAX_NEW_DECODES (scripts/sync-bills.mjs + the workflow_dispatch input/env default in sync-bills.yml) was still 40, pinning the ascending-pass sync cursor weeks behind against live nightly demand of 373-418 bills. Raise to 120 (workflow_dispatch input stays overridable for a one-time higher-budget catch-up run). COVERAGE_TOP_N (scripts/sync-coverage.mjs) was unbounded (Infinity), so the nightly coverage sweep queried every eligible bill in urgency order and structurally starved anything near the bottom of the ranking of TheNewsAPI's daily quota; cap it at 150, still overridable via env. This is the already-deferred 2026-07-04 proposal, re-sized against the audit's live numbers (pipeline-audit.md §5 item 1). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
… the race The nightly sync queried Congress.gov ascending by updateDate, so the most recently-touched bills (floor votes, markups — exactly what generates press) were structurally the LAST ones reached each run, behind both the MAX_UPDATES window cap and the decode budget. On a deep-backlog night this meant a bill that just had real activity could go unnoticed indefinitely even on a clean, successful run — the root cause behind "worth a call" reading stale/empty independent of the decode-cap fix (pipeline-audit.md §5 item 2). Add a recent-first pass (sort=updateDate+desc, ~100 bills, no cursor floor) that runs before the existing ascending backlog scan: already-known bills refresh for free, brand-new bills decode within a reserved sub-budget (RECENT_DECODE_RESERVE=20, carved out of MAX_NEW_DECODES, not additional). Bills the recent pass resolves are deduped (not re-fetched/re-decoded) when the ascending scan reaches them. Cursor semantics preserved deliberately: the freeze-on-incomplete-work high-water mark (state.lastSync) is advanced ONLY by the ascending pass, so a bill handled out-of-order by the recent-first pass can never push the cursor past unprocessed middle-of-backlog territory. The failed > updated.length/2 abort also stays scoped to the ascending pass exactly as before; the recent pass's failures are logged separately. Extracted the Congress.gov fetch/status/urgency/refresh helpers into scripts/congress-fetch.mjs so scripts/hot-bills.mjs (next commit) can reuse them instead of duplicating. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Add scripts/hot-bills.mjs + .github/workflows/hot-bills.yml: a refresh-only pass at 17:00 and 22:00 UTC that re-fetches the ~100 most-recently-updated Congress.gov bills and updates status/last_action_date/urgency for whichever are already in the corpus. Decodes nothing — a brand-new bill still waits for the nightly sync's decode-before-publish gate — so this is zero Anthropic usage, near-$0 marginal cost (Congress.gov + free public-repo Actions minutes only). Reuses scripts/congress-fetch.mjs (extracted in the prior commit) rather than duplicating the fetch/refresh logic. Commits as the same oravan-sync author pattern as sync-bills.yml/refresh-legislators.yml and shares their `data-sync` concurrency group so it can never race the nightly sync. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…aleness emptyStateVerdict() (lib/freshness-state.ts) collapsed to quiet_week whenever checkedAt/lastRun was recent, regardless of whether the sync cursor (completeThrough/lastSync) or the corpus's own newest last_action_date had gone stale — so a pipeline that ran and committed every night, but made no real forward progress for weeks, read as "fresh" and the site (and the MCP whats_moving tool) confidently claimed "quiet week" over a month-stale corpus (pipeline-audit.md §5 item 4 — the exact live bug: a 29-day-old cursor and newest action both passing as fresh under the old checkedAt-only check). emptyStateVerdict now takes all three freshness signals and fails toward data_stale if EITHER checkedAt is stale/dead (tight FRESHNESS_CLAIM_WINDOW_ DAYS/FRESHNESS_DEAD_WINDOW_DAYS) OR completeThrough/newestAction exceed the wider FRESHNESS_DEAD_WINDOW_DAYS — deliberately wider for the latter two since the sync cursor is expected to lag checkedAt by real days under normal backlog-draining operation (lib/freshness.ts's own doc comment); gating them on the tight window would cry "stale" most nights even when the recent-first pass (previous commit) kept the actually-relevant content current. UrgencyEmptyState and its three call sites (homepage, /reps, BillsBrowser) now pass the full FreshnessSignals object instead of just checkedAt. whatsMoving (lib/core/mcp.ts) passes getFreshness() directly — the shape already matched. Also promotes scripts/verify-sync.mjs's 7-day cursor-age ::warning to a hard failure at 10 days (generous enough that the post-merge backlog-drain window doesn't insta-fail, still well inside the site's own 21-day dead ceiling for the same value) — the warning fired every clean night for weeks and was never acted on, which is exactly the silent-failure shape this verifier exists to catch. Updates the pinned tests this invariant touches (tests/freshness.unit.spec.ts, tests/freshness.spec.ts, tests/mcp-tools.spec.ts) deliberately, with comments explaining why the invariant legitimately changed; unrelated assertions are untouched. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…eal keyspace lib/upstash.ts's keyPrefix() resolves to `process.env.VERCEL_ENV ?? 'dev'` — 'production'/'preview' are only ever set by Vercel itself at request time, so any GitHub Actions job touching an Upstash-backed key builder without explicitly setting VERCEL_ENV silently operates on the 'dev:' keyspace instead of the one the live site actually uses (same pattern already documented for laptop-shell CLI runs in scripts/tenant-admin.mjs). Two live instances, both fixed the same way: - .github/workflows/daily-metrics.yml (flagged, not fixed, in PR #87): readUsageWindow/readMcpClientDay were reading an always-empty 'dev:' keyspace, so the daily digest could only ever report zero MCP/script usage — never caught because a silent zero looks identical to genuinely no traffic. - .github/workflows/sync-bills.yml's pregen step (same root cause, previously unflagged): every combo cached since PREGEN_ENABLED was armed (2026-07-12) has been written under 'dev:script:...' — a keyspace app/api/script/route.ts's live 'production:' requests can never read — so every pregen run since arming has been pure Anthropic Batch API spend with zero possible cache hits. Checked scripts/verify-salt.mjs for the same problem: it does NOT have it — that script iterates the 'production'/'preview' keyspaces explicitly rather than deriving a single prefix via keyPrefix(), so no fix needed there. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the fix package from the 2026-07-16 pipeline audit (delivered to Colby; root causes: decode budget 40 vs ~400/night real demand, oldest-first fetch bias, quiet-week verdict keyed on job-ran instead of data-advanced, plus the now-resolved Anthropic credit outage).
The five commits
MAX_NEW_DECODES40→120 (still dispatch-overridable for the one-time catch-up run),COVERAGE_TOP_N∞→150 (stops the nightly news-quota exhaustion; applies the deferred 2026-07-04 proposal, re-sized to live numbers).cursor/frozenare written only by the ascending pass. Shared fetch helpers extracted toscripts/congress-fetch.mjs.hot-bills.ymlat 17:00/22:00 UTC, refresh-only (status/last_action_date for known bills), zero Anthropic usage, shareddata-syncconcurrency group. New bills still wait for the nightly decode gate (documented).emptyStateVerdictnow tripsdata_staleon cursor (completeThrough) ornewestActionstaleness past the 21-day dead window, not justlastRun;verify-sync's 7-day cursor warning promoted to a hard failure at 10 days. With today's committed data the suite proves the site now says "Data check needed" instead of the false "Quiet week" — pinned tests updated deliberately with invariant-change comments.keyPrefix()resolves todevin Actions (noVERCEL_ENV): the daily digest was reading an always-empty keyspace, and pregen has been writing its script cache todev:script:*since arming on 07-12 — spend with zero possible production cache hits. Both steps now setVERCEL_ENV: production. (verify-salt.mjschecked — not affected.)Costs at new defaults (per audit, /bin/zsh.07–0.15/bill)
Backlog-drain nights: hard-capped .40–18.00. Steady state: .80–12 on active session days, ~/bin/zsh in recess (refreshes are free). Hot-bills: $0. Coverage cap slightly reduces Haiku spend.
Verification
lint ✓ typecheck ✓ full Playwright suite 1290 passed / 14 skipped / 0 failed ✓ check-naming ✓ check-key-namespaces ✓ check-server-json ✓ check-messages-parity ✓ (no new strings; existing dataStale EN/ES keys reused). No live API calls in tests. Built by a Sonnet 5 subagent; orchestrator independently verified branch on origin, cursor-semantics implementation, and the hot-bills workflow's zero-Anthropic surface.
After merge (pending Colby's spend approvals in chat)
One-time catch-up dispatch (~400 decodes, ~–60) to drain the 29-day backlog, then the nightly settles at the new caps.
🤖 Generated with Claude Code
https://claude.ai/code/session_01XexCqeBQ2csGYiBMC62p2G