feat(sync): priority decode gate + hourly RSS newsdesk — spend drops to ~$0–2/day#90
Merged
Conversation
…s a decode Owner directive: reduce spend right now by focusing the AI decode pass on bills with real legislative motion, not the "majority… junk with high odds of never going anywhere." A brand-new bill now only spends a decode if mapStatus's output clears scripts/decode-gate.mjs's passesGate() (markup or later); mere "referred to committee" — sampled at 1,573/1,706 (92.2%) of today's 'committee'-status bills — does not. Extracted the shared decode-before-publish path (fetchBillText/decode/ syncOneBill) out of sync-bills.mjs into scripts/bill-decode.mjs, taking bills/es/bySlug/anthropic as explicit parameters instead of module-scope closures, so the newsdesk (next commit) can decode a press-triggered bill via the exact same path instead of a second copy of the prompts. Gate-skipped bills are NOT stored and count as fully handled: the ascending backlog pass's cursor advances past them exactly as if decoded, which drains the multi-week decode backlog nearly for free. If a gated bill later gets real motion, Congress.gov's own updateDate resurfaces it on a later run and the gate re-evaluates. FORCE_DECODE_SLUGS (env + workflow_dispatch input) bypasses the gate for specific slugs. MAX_NEW_DECODES reverts to a pure safety ceiling (120 -> 60) now that the gate does the real limiting, and per-run logging now reports new-bills-seen/gated/decoded so the digest-era numbers stay honest. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
New scripts/newsdesk.mjs + .github/workflows/newsdesk.yml (hourly, cron '7 * * * *'), the news-headline trigger half of the owner's spend-reduction ask, ALL-IN cost under $5/day. Free RSS only — NO paid news API (TheNewsAPI stays sync-coverage.mjs's own quota) — a politically-balanced 6-feed basket (The Hill/Roll Call/NPR/Fox/CBS + a Google News congressional-keyword query carrying per-article outlet attribution), each verified live 2026-07-16 to return real, parseable RSS/Atom. Matching, cheapest first (scripts/newsdesk-match.mjs, a pure/I-O-free module so it's unit-testable with zero mocking): t1 explicit bill-number citation regex (hr/s/hjres/sjres only — "H. Res. 12" and "US 567" correctly excluded) -> t2 free normalized-token overlap against corpus titles+press_names -> t3 ONE batched claude-haiku-4-5-20251001 call for headlines t2 leaves ambiguous, skipped entirely when that batch is empty. Nonpartisan guardrail (non-negotiable): a bill fires only on (a) an explicit citation from any outlet, or (b) a t2/t3 match corroborated by >=2 distinct outlets, accumulated across runs — single-outlet soft-match triggering would be a prioritization channel data/media-bias.json's display-only lean normalization doesn't cover. On fire, the bill refreshes (free) or, if not yet in the corpus, decodes via the exact same decode-before-publish path as sync-bills.mjs (force-bypassing the priority gate — the press trigger's own corroboration is the worthiness signal here), bounded by both NEWSDESK_DECODE_CAP=3/run and a NEWSDESK_DAILY_DECODE_CAP=10/day so the documented <$2/day ceiling is code-enforced, not just expected. A seen-headlines cache (actions/cache, GitHub's own 7-day-unused eviction) dedupes across hourly runs and degrades gracefully on a cache miss. Never writes data/coverage.json (stays sync-coverage.mjs's) or sync-state.json's nightly cursor. Commits data/ only when scripts/newsdesk.mjs actually changed something, so a no-op hour never triggers a deploy. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
133 live corpus bills carry the hyphenated spelling and would have been wrongly gated out as mere 'committee'. Key check moves from import time to first cg() call so the module's pure exports stay unit-testable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XexCqeBQ2csGYiBMC62p2G
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the owner's 2026-07-16 spend directive: decode only legislation with real motion or real press, everything else tracked free.
Deadline note (the one merge that shouldn't wait)
Tonight's 07:30 UTC (3:30am ET) nightly cron runs UNGATED at cap 120 (~$8–18) unless this merges first. Merged before then, the backlog instead drains gated: ~$3–12 one-time, then the costs below.
Part 1 — priority decode gate
Data-driven line (corpus measured 2026-07-16): 79.5% of the 2,147 bills sit at 'committee', and 92.2% of those are literal referral-with-zero-action. The gate decodes only markup / floor_vote / passed_chamber / conference / signed / vetoed (~20.5% of today's corpus). Gate-skipped bills cost nothing, are never published, and re-enter automatically the moment Congress.gov shows new motion — the cursor advances past them, which drains the 29-day backlog nearly free.
FORCE_DECODE_SLUGS(env + dispatch input) bypasses the gate;MAX_NEW_DECODESdrops 120→60 as a pure safety ceiling.Part 2 — newsdesk (hourly, free RSS, no new secrets)
Hourly Actions job over a politically balanced, live-verified feed basket (The Hill, Roll Call, NPR, Fox, CBS + a Google News congressional query; AP/Politico/WaPo feeds are dead — verified 2026-07-16). Matching, cheapest first: explicit-citation regex → free token overlap vs titles/press_names → one batched Haiku call only for leftovers (skipped when empty). Nonpartisan guardrail: t2/t3 matches fire only on ≥2 distinct outlets (citation matches are unambiguous and fire on one). On fire: free status refresh, or decode-before-publish capped 3/run and 10/day (code-enforced). No-change runs don't commit (no deploy churn); seen-set lives in actions/cache, not the repo.
Costs
Also fixed here
mapStatusmissed hyphenated "Mark-up" action text — 133 live bills with genuine committee activity read as mere 'committee' and would have been wrongly gated out. Two-token fix + pinning tests (orchestrator commit3bac1ee). Congress-fetch's key check moved from import time to first call so its pure exports stay unit-testable.Verification
76 new unit tests (gate table, citation regex hard cases, ≥2-outlet rule, no-commit guard) + full suite 1368 passed / 14 skipped / 0 failed; all repo gates green; re-run after the orchestrator's fix (78/78 on affected specs). Live mock proof: gated bill → no Anthropic call; markup bill → decodes; forced slug → bypasses. Built by a Sonnet 5 subagent; orchestrator independently verified the branch, caught the mapStatus gap from the agent's own report, and fixed it before this PR.
🤖 Generated with Claude Code
https://claude.ai/code/session_01XexCqeBQ2csGYiBMC62p2G