Add Voice Arena database schema - #119
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
📝 WalkthroughWalkthroughA new Alembic migration ( ChangesArena Schema Migration
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Schema-qualify arena objects instead of relying on search_path, make the snapshot count columns uniformly NUMERIC, widen the snapshot lookup index to the full board grouping, and add votes.updated_at to support labeler re-label windowing.
…hecks Prevent a model battling itself, enforce one snapshot row per board (domain NOT NULL with an 'all' sentinel so global rows dedup too), drop the redundant lookup index, and add nonnegative checks on snapshot counts.
Add a BEFORE UPDATE trigger so a re-label always stamps updated_at, even if the writer omits the column — windowed refits can't silently use a stale time.
2 participants
First piece of Voice Arena — just the database tables, nothing wired up yet.
Adds an
arenaschema (next tobenchmarks_v2, same DB) with three tables:battles— a matchup: prompt, the two models, their audio. Model names are text keys matchingbenchmarks_v2.results, so they can be joined later. A model can't battle itself.votes— one judgment per row (A_WIN/B_WIN/TIE), labeler vs external.UNIQUE (battle_id, voter_type, voter_id)is the double-vote backstop;updated_atlets a labeler re-label (upsert) without breaking windowed refits, while external votes stay insert-once.leaderboard_snapshots— computed Elo / Bradley-Terry ratings over time.UNIQUE (computed_at, metric_name, methodology_version, domain, provider, model)enforces one row per model per board;domainisNOT NULL DEFAULT 'all'so the global board dedups too.Additive only: no grants, no
benchmarks_v2changes, nothing reads/writes these yet. ThearenaDB user + grants live in Terraform, landing separately.Greptile Summary
Introduces the
arenaschema alongsidebenchmarks_v2, adding three tables (battles,votes,leaderboard_snapshots) for the Voice Arena feature. The migration is additive-only and well-documented; prior review rounds addressed the missing UNIQUE constraint on snapshots, the absentupdated_attrigger, and type-uniformity for win/loss/tie columns.arena.battlesstores raw matchups with a CHECK that prevents a model from facing itself;arena.votesrecords human judgments with a BEFORE UPDATE trigger that auto-maintainsupdated_at;arena.leaderboard_snapshotscaches computed Elo/BT ratings with a 6-column UNIQUE constraint.leaderboard_snapshotsboard-grouping relies oncomputed_at TIMESTAMPTZ DEFAULT now()as a shared key across rows, butnow()returns the transaction start time — multi-transaction snapshot writes will silently assign different timestamps to rows in the same conceptual run, fragmenting the board and defeating the UNIQUE deduplication guarantee.Confidence Score: 3/5
The migration is additive and won't touch existing tables, but the board-grouping design in leaderboard_snapshots has a structural flaw that should be resolved before any snapshot writer is built against it.
The snapshot table's deduplication guarantee — enforced by a UNIQUE constraint that includes computed_at — breaks as soon as a writer inserts rows across multiple transactions. Each transaction gets a different timestamp from DEFAULT now(), making rows that belong to the same computation run appear as separate boards. Fixing the schema now (before any writer is implemented) is straightforward; waiting until writers exist makes it a much more disruptive change.
runner/src/coval_bench/db/migrations/versions/20260615_0007_arena_init_schema.py — specifically the leaderboard_snapshots table definition and its UNIQUE constraint.
Important Files Changed
Entity Relationship Diagram
%%{init: {'theme': 'neutral'}}%% erDiagram BATTLES { UUID id PK TEXT provider_a TEXT model_a TEXT provider_b TEXT model_b TEXT domain TEXT prompt_text TEXT audio_a_url TEXT audio_b_url TIMESTAMPTZ created_at } VOTES { UUID id PK UUID battle_id FK TEXT outcome TEXT voter_type TEXT voter_id TIMESTAMPTZ created_at TIMESTAMPTZ updated_at } LEADERBOARD_SNAPSHOTS { UUID id PK TIMESTAMPTZ computed_at TEXT metric_name TEXT methodology_version TEXT domain TEXT provider TEXT model NUMERIC rating_elo NUMERIC rating_bt NUMERIC ci_low NUMERIC ci_high NUMERIC ci_half_width INTEGER votes_total NUMERIC wins NUMERIC losses NUMERIC ties TEXT status } BATTLES ||--o{ VOTES : "has" BATTLES }o--o{ LEADERBOARD_SNAPSHOTS : "informs (via refit)"%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%% erDiagram BATTLES { UUID id PK TEXT provider_a TEXT model_a TEXT provider_b TEXT model_b TEXT domain TEXT prompt_text TEXT audio_a_url TEXT audio_b_url TIMESTAMPTZ created_at } VOTES { UUID id PK UUID battle_id FK TEXT outcome TEXT voter_type TEXT voter_id TIMESTAMPTZ created_at TIMESTAMPTZ updated_at } LEADERBOARD_SNAPSHOTS { UUID id PK TIMESTAMPTZ computed_at TEXT metric_name TEXT methodology_version TEXT domain TEXT provider TEXT model NUMERIC rating_elo NUMERIC rating_bt NUMERIC ci_low NUMERIC ci_high NUMERIC ci_half_width INTEGER votes_total NUMERIC wins NUMERIC losses NUMERIC ties TEXT status } BATTLES ||--o{ VOTES : "has" BATTLES }o--o{ LEADERBOARD_SNAPSHOTS : "informs (via refit)"Reviews (4): Last reviewed commit: "Auto-bump arena.votes.updated_at via tri..." | Re-trigger Greptile