Playable brain: deterministic model, real spiking network, per-sentence math, calibration (which caught an inverted trust score) - #92
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces 'Mind-Hack Autopsy' (experiment 013), a new client-side interactive lab allowing users to simulate brain reactions to various content types locally. It updates the arcade layout, landing page, sharing dialogs, and E2E tests to support thirteen experiments, and adds a new evidence export feature for the attractor playground. The review feedback suggests several robustness and compatibility improvements, including defensive checks for clipboard API availability, safer destructuring of parameters in the evidence builder, and appending the temporary download link to the document body to ensure cross-browser compatibility.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
3d48582 to
138dc92
Compare
…ce export
Content Reaction Lab (arcade experiment 013):
- New src/features/gaugegap/ContentReactionLab.jsx: paste an ad / tweet /
landing page / email, run a fully client-side scan (analysisEngine +
layerRouter, no /api round-trip), watch the live 3D brain react, and read
four headline scores — Attention, Trust, Emotional Charge, Manipulation
Risk — with delta badges against the previous run.
- One-click rewrites (build trust / add real urgency / reduce manipulation)
scored in-browser with before→after deltas; new additive "urgency" goal in
improve/rewrite.js. Share score card reuses ShareDialog.
- Registered as lab 013 in ExperimentArcade with XP/daily-mission integration;
the landing's Mind-Hack Autopsy card now opens the lab in place instead of
bouncing to /app ("Open full analysis" remains the escape hatch).
- New src/lib/headlineScores.js maps getBusinessMetrics to the four headline
scores; score card, share text and share preview now show the same four
rows so the playground and shared cards never disagree (this also changes
/app's share card rows from Hook/Viral Pull to the headline set).
Truth layer:
- New src/features/gaugegap/evidence.js: "Export evidence" button on the
Butterfly Effect (Lorenz) lab downloads a content-hashed JSON proofpack
(schema gaugegap.browser_attractor_proofpack.v1) with live parameters,
solver settings, scores, share URL and an explicit claim boundary.
- Deep Foundry link from the landing's research section to the published
gaugegap-foundry Foundry Experience playground.
Tests: new tinyVitest suites for headlineScores and evidence; urgency-goal
coverage in rewrite.test.js; scoreCard test updated to the new rows. Stale
e2e landing specs (pre-arcade selectors) rewritten against the arcade:
content lab local-scan flow asserts no /api/analyze request; 3D-mount and
Reconstruct-nav specs target the current DOM. Full suite: 79 unit tests,
13 e2e passed locally.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
…load link Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
A shared link (?lab=content&state=<text>) now prefills the lab and runs the scan automatically, matching the challenge-link loop of the other arcade labs (same `state` param, so lab-switch URL cleanup applies unchanged). "Share challenge" / "Copy link" actions reuse the DeepLabChrome helpers. Shared text is capped and validated before auto-running. Tests: unit coverage for the URL round-trip, foreign-lab guarding, size cap and stale-run-param cleanup; new e2e spec proves a shared link auto-runs to scored tiles. The auto-run effect keys on the parsed share payload only, so StrictMode's dev double-invoke re-arms the cleared scan timer instead of deadlocking, and keystrokes never re-apply the shared text. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
The 3D brain ran on Math.random(), so the same content produced a different brain every time — scores could not be reproduced, shared or verified. Its ~20 model constants were hardcoded literals inside the step function, and it had no test coverage at all. - New src/features/brain3d/brainModel.js: the 7-region rate dynamics lifted out of the React hook into a pure, seeded, parameterized model. Adds interventions (lesion a region, cut a pathway, inject current) and exposes the excitatory/inhibitory split and per-pathway STDP delta that the old loop computed and discarded. DEFAULT_PARAMS reproduces the original behaviour exactly, so the live visual is unchanged. - runBrainTrial() runs the model headlessly: same seed and inputs always produce an identical trace, which is what makes brain readouts scoreable and verifiable. - New brainMetrics.js: firing rates in Hz, PFC/AMY control ratio, hijack index, gain around the THL->CTX->AMY->BG->|THL control loop, gate integrity, per-region E/I balance, mean pairwise spike correlation, settling time, plasticity, weight drift and net STDP flux — each with a descriptor stating its direction, plus an explicit claim boundary. - New src/lib/rng.js: shared FNV-1a + mulberry32 seeding, matching the implementation solitonLayer.js already used privately. - useBrainSimulation.js becomes a thin wrapper over the model and regains pause/step/reset/burst (dropped during the r3f port) plus stimulate, lesion, cut and speed control. - 23 new tests covering determinism, bounds, interventions and metrics. One pins a real quirk found while testing: activity is capped at 1.0, so spike probability maxes at 0.836 and the spikeMax 0.85 ceiling is unreachable. - tinyVitest gains the toBeLessThan matcher it was missing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
"Manipulation Risk 72" said nothing about why, and the single `confidence` number was asserted rather than estimated. This adds attribution and a real uncertainty estimate by re-running the scorer on modified inputs — genuinely new information, not a restatement of the same features. - New src/lib/ablation.js: leave-one-sentence-out scoring gives each sentence its exact contribution to each headline score. The same replicates are then read as a jackknife to produce a standard error and range per score, which is a standard estimator rather than an invented confidence. Seeded sentence shuffles measure whether a verdict depends on what was said or merely on the running order. - topDrivers() ranks the sentences behind one score with each one's share. - The lab now renders a "Where this score comes from" panel: ranked sentence bars, a tab per headline score, and ± on each tile. On mixed content the attribution correctly assigns the pressure sentence 100% of Manipulation Risk while splitting Trust across both sentences. - Runs inline with the scan: measured at 21-27 ms including the 18-sentence ceiling, so it needs no button gate. - 12 unit tests plus an e2e spec covering attribution and tab switching. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
The ten gallery classics already carried hand-labelled expectations that nothing ever checked. This turns them into a real evaluation, and the first run found a significant defect. - New calibrationCorpus.js: 18 labelled archetypes (the 10 classics plus 8 new originals) scored on four ordinal dimensions. Kept separate from CLASSIC_PRESETS so the evaluation set can grow without changing the UI gallery, and so re-ordering the gallery cannot alter a published figure. - New calibration.js: labels are ordinal, so agreement is reported as Spearman rank correlation plus the count of labelled pairs the engine orders backwards — a falsifiable claim, rather than asserted precision. Measured baseline: 66% of 442 labelled comparisons ranked correctly, mean Spearman 0.214. Urgency is decent (0.64); manipulation risk (0.39) and viral pull (0.33) are weak. FINDING — trust is ANTI-correlated with its labels (Spearman -0.505, 65% of pairs inverted). The engine ranks outrage bait as more trustworthy than a sincere apology. Cause: `trust` rewards trust *vocabulary*, not evidence. TRUST_TERMS includes the bare connective "because", so "share this because once it's gone" earns +13, while "on Tuesday our update broke checkout for six hours, here is what we refunded" earns nothing — none of its concrete, checkable words are in the bank. Thresholds in the test pin MEASURED behaviour, not aspiration, so scoring changes cannot quietly regress. The trust defect is asserted explicitly rather than hidden, and the assertion flips once the signal is fixed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
Trust was anti-correlated with its labels (Spearman -0.505): the engine
ranked outrage bait as more trustworthy than a sincere apology, because
`trust` counted trust *vocabulary* rather than evidence.
Each change was measured against the calibration harness rather than
guessed, and one hypothesis was rejected by the data:
- Removing the connective "because" from TRUST_TERMS made trust WORSE
(-0.505 -> -0.611). It turns out "closes on 30 November because the cohort
starts in December" is a genuine evidence marker. Reverted.
- The real gap was that concrete detail earned nothing. Added a specificity
signal (numerals, dates, durations) and a stated-limitation signal
("not yet", "uncertain", "approximately"), since admitting limits is a
credibility cue. Trust: -0.505 -> +0.560.
- Weighted specifics above evidence-vocabulary, on the principle that
checkable detail is stronger than claiming to be transparent: -> +0.628.
- Dropped trust from viralScore: careful writing is not more viral, and
including it was dragging viral pull down. Viral: 0.333 -> 0.366.
Measured across 442 labelled comparisons:
before after
trust -0.505 +0.628
risk 0.387 0.629
urgency 0.641 0.641
viral pull 0.333 0.366
accuracy 65.8% 80.1%
mean rho 0.214 0.566
Every dimension now ranks in the right direction. Test thresholds move to
the new measured values, and a remaining limitation is pinned rather than
hidden: "verify within 24 hours" still reads as concrete detail, so phishing
scores higher on trust than it should. Discounting numerals adjacent to
urgency terms is the next step.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
Despite the name, BrainSNN was a rate model: 7 scalar units, no membrane
potential, no threshold, no refractory period. This adds the real thing.
- src/lib/snn/lifNetwork.js: sparse random network of leaky integrate-and-fire
neurons in the Brunel (2000) formulation — Dale's law (80/20 excitatory/
inhibitory), fixed-amplitude delta synapses, axonal delays, Poisson external
drive, exact exponential leak. Typed arrays plus CSR adjacency; ~150 ms for
800 neurons over 250 ms of biological time, and fully seeded.
- src/lib/snn/snnMetrics.js: the measurements that characterise a network
state — firing rate, CV of ISI, Fano factor, population synchrony, and a DFT
of the population rate. The last one matters: gamma-band power becomes a
MEASURED property rather than a closed-form assertion.
- brunelValidation.test.js checks published consequences of the model, so the
simulation is falsifiable rather than merely plausible:
* silent below the threshold drive nu_thr, firing above it
* rate monotonically increasing in external drive
* rate monotonically decreasing in inhibition strength g
* irregularity monotonically increasing in g — the central Brunel result
* near clock-regular firing (CV < 0.15) when excitation dominates
* rate never exceeding the refractory ceiling
Found and fixed a delay bug while validating: the ring buffer wrote into the
same slot it read that step, so a spike reached later-indexed neurons with
zero delay instead of 1.5 ms. With correct delays the balanced network settles
at 42 Hz instead of 69 Hz, and the sweeps become cleanly monotonic.
Known limitation, asserted rather than overclaimed: the asynchronous-irregular
regime needs N ~ 10^4-10^5 at epsilon ~ 0.1 for presynaptic independence. At
browser-affordable sizes the network reaches high irregularity but keeps more
population synchrony than true AI, so the tests pin the monotonic trends
rather than claiming all four published regimes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
All twelve existing labs print a mission string that nothing verifies. This one has a real win condition, a real fail state and a finite budget, all evaluated against the deterministic model from brainModel.js. - brainGame.js: mission rules, a pressure profile that ramps threat drive up while suppressing judgment, five interventions (lesion the amygdala, cut the salience or action route, stimulate PFC or cerebellum), and a three-axis score in the arcade's house style where control, stability and budget efficiency pull against each other. - BrainGameLab.jsx: arcade experiment 014. A 2D circuit view of the seven regions and ten pathways, live activity and spikes, cut pathways drawn dashed, lesioned regions crossed out. Sandbox / Mission / Challenge modes, pause, single-step, reset, and a shareable seed. Thresholds were measured, not guessed. Sweeping the hijack limit against the model showed doing nothing breaches for ~179 consecutive ticks, silencing threat or boosting judgment holds the line, and cutting the salience route helps but is not sufficient on its own — so the limit is set at 60 to keep that gradient. A plausible-looking intervention that still loses is what makes the mission worth replaying. Two bugs found while testing: - The loss condition read the breach streak on the final tick, so a run that failed mid-way but recovered was scored as a win. It now latches on the worst streak seen. - Control ratio diverged when a lesioned amygdala drove the denominator to zero (21.73 on screen). Capped at 10, past which the ratio says nothing extra. 17 unit tests plus an e2e that confirms the mission is genuinely losable and that intervening keeps a run alive past the point it previously failed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
Two pieces of the evidence layer. Krippendorff's alpha (src/lib/agreement.js) — a labelled corpus without an agreement statistic says nothing about whether the labels are reproducible by another person. Full implementation with the coincidence-matrix algorithm, supporting any number of annotators, missing ratings, and nominal / ordinal / interval distance. Ordinal matters here: "low vs extreme" should count as a bigger disagreement than "low vs moderate". Reported with the conventional reliability reading (>=0.8 reliable, >=0.667 tentative). Verifiable run proofs (src/features/gaugegap/runProof.js) — because the model is deterministic, a Defend the Brain run is fully described by its seed, mode and the tick at which each intervention was applied. So a score is checkable by RECOMPUTATION: replay the log and see whether the claimed result falls out. Inflating a score means finding a log that actually replays to it, not editing a number. The lab now records intervention ticks and exports a hashed proof. Privacy: a proof carries a seed and a list of tick/id pairs. No text, no personal data — so the lab's "nothing leaves your browser" promise survives even if proofs are later submitted somewhere. Note on scope: durable cross-user leaderboards need storage this deployment does not have. Supabase is wired for magic-link auth only — no tables, no persistence, no rate limiting, and the server is stateless. The verification logic that makes a leaderboard trustworthy is what ships here; the storage behind it is a separate, infrastructural piece of work. Fixed two bugs found by the tests: evaluateRun derived elapsed ticks from the rolling frame window (so a completed replay read as still running), and the proof hash was taken over an object without the hash key while verification spread it back as undefined, which canonical JSON renders as null. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
Completes the evidence path from labels to a defensible number. - src/lib/evalMetrics.js: AUC (ranking), Brier (probabilistic accuracy) and Expected Calibration Error with a reliability table, plus precision/recall/F1 at the best operating point. ECE is the one that matters for honesty: users read these scores as percentages, and a scorer can rank items perfectly while its numbers mean nothing as probabilities. One test pins exactly that case. - scripts/eval-corpus.mjs: evaluate the engine against any externally labelled JSONL corpus. Public datasets are NOT vendored — they carry their own licences — so datasets/ is gitignored and the script points at a local copy. - docs/ANNOTATION_RUBRIC.md: dimensions, ordinal levels, worked anchors, and the rule that a corpus release without a Krippendorff alpha does not get to claim reliability. Maps the published persuasion-technique taxonomies (SemEval-2023 Task 3, SemEval-2020 Task 11 PTC, Webis clickbait) onto our dimensions, and records that only one of the four tactics the firewall detects today maps cleanly to a published class — so those corpora should drive building a real detector, not just validate the current one. Smoke run on a six-item set: AUC 0.778, ECE 0.290 — ranks reasonably, but the scores do not yet behave like percentages. Worth knowing and worth stating. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
…ust gap
Three follow-ups that finish the work rather than extend it.
1. The LIF network was a library nobody could see — tests and no UI. Added
arcade experiment 015: a spike raster, population-rate trace and live
measurements, driven by sliders for inhibition strength and external drive
so you can walk across Brunel's phase diagram yourself. It runs in a Web
Worker (a 2000-neuron run takes ~800 ms, which would drop frames across the
whole arcade on the main thread). Verified in-browser: excitation-dominated
fires at 215 Hz nearly clock-regular (CV 0.05), strong inhibition drops to
31 Hz and turns irregular (CV 0.41), sub-threshold drive is silent.
2. Fixed the trust defect this suite previously pinned as known-broken. The
specificity signal counted numerals, so "verify within 24 hours" read as
concrete evidence and bought phishing credibility. Specifics inside urgency
phrasing are now discounted:
before after
trust rho 0.628 0.700
pair accuracy 80.1% 81.2%
phishing trust 68 60
flash-sale trust 54 46
Crucially the honest deadline ("closes 30 November because the cohort
starts in December") keeps its 96 — the discount targets coercion, not
dates. Two tests now pin both halves of that distinction.
3. XP could only be earned by opening a lab (25 first time, 2 after). Winning
a mission paid nothing. Added recordAchievement with a bounded, one-off
40 XP plus per-lab personal bests, threaded through the arcade to Defend
the Brain. Verified live: a Defense 97 win using 1 of 6 interventions
unlocked both achievements and took XP from 27 to 107.
The progress rules moved into arcadeProgressCore.js because the bare-Node test
runner cannot import .jsx — which is why this economy had no tests at all. It
now has twelve, covering double-pay, streak resets, and migration of progress
saved before the new fields existed.
214 unit tests, 21 e2e, lint and build green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
…tion
The four headline numbers are 0-100 INDICES, but users read "72" as "72%".
The eval harness confirmed the gap: expected calibration error ~0.29 against
any probabilistic reading. Rather than invent a probability the engine cannot
support, this gives each score a statement that is true.
- src/lib/scoreReference.js: percentile of a score against the 18 labelled
archetypes, with the midpoint convention for ties. Each tile now reads
"around the 86th percentile of 18 known archetypes" instead of leaving the
bare index to be misread.
This immediately earns its keep: a phishing email scores a raw 32 on
manipulation risk — which looks safe — while sitting at the 86th percentile
of known archetypes. The index is compressed; the percentile tells the
truth.
- A calibration card in the lab states what the engine has been measured to
do, where the scores are actually read rather than buried in a README:
rank agreement per dimension, how many labelled pairs are ordered wrongly,
and an explicit note that these are indices, not probabilities.
Current measured figures, shown in the card: 81% of 442 labelled comparisons
ranked correctly, mean Spearman 0.585 (trust 0.70, urgency 0.641,
manipulation risk 0.631, viral pull 0.366).
Also fixed an ordinal-suffix bug the browser check surfaced ("81th"), with
the teens case covered.
224 unit tests, 21 e2e, lint and build green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
- The game can now verify a run proof, not just export one. Drop a proof file in and it replays the intervention log and checks the claimed score falls out. Confirmed in-browser: a genuine proof reports "replay matches"; the same proof with its score edited to 100 is rejected with two problems (the score mismatch and the broken content hash). This mirrors the Python verifier already shipped for gaugegap proofpacks — export in one place, check independently in another. - README now states what the engine has actually been measured to do, rather than leaving the claim implicit: the per-dimension rank agreement table, the 81%-of-442-comparisons figure, the trust defect calibration caught and how it was fixed, and the two honesty notes that tests enforce — scores are indices rather than probabilities, and labels are ordinal. Documents the deterministic brain model, the LIF network validated against Brunel, and the claim boundary that ships with every neural readout. 224 unit tests, 21 e2e, lint and build green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
9719c7b to
98b5918
Compare
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 98b5918725
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const stability = Math.round(Math.max(0, 1 - worst / Math.max(1, rules.breachGrace)) * 100); | ||
| const defense = Math.round(Math.max(0, Math.min(100, controlScore * 0.45 + stability * 0.35 + efficiency * 0.2))); |
There was a problem hiding this comment.
Use a defined grace period for challenge scoring
When the player switches to Challenge mode, rules is CHALLENGE, which has no breachGrace property. This makes the stability expression evaluate to NaN, and the derived defense score also becomes NaN; the challenge UI therefore shows invalid scores, achievements cannot record a meaningful best score, and exported challenge proofs fail verification because NaN !== NaN. Use a challenge-specific denominator or define breachGrace for CHALLENGE.
Useful? React with 👍 / 👎.
| const sentences = segments.map((sentence, index) => { | ||
| const without = segments.filter((_, other) => other !== index).join(' '); | ||
| const ablated = scoreText(without); |
There was a problem hiding this comment.
Preserve sentences beyond the attribution display limit
For inputs longer than 18 sentences, splitIntoSegments already truncates the passage, so each leave-one-out scan is built only from the first 18 sentences while the baseline was scored from the complete input. Consequently, signals in sentence 19 and later disappear from every replicate and are falsely attributed to each displayed sentence—for example, an urgent 19th sentence makes otherwise neutral opening sentences appear to add manipulation risk. Build each ablation from the full passage, even if only 18 contributions are displayed.
Useful? React with 👍 / 👎.
| timerRef.current = window.setTimeout(() => { | ||
| const nextResult = scanContentLocally(content); | ||
| setResult((current) => { |
There was a problem hiding this comment.
Cancel a pending scan when the input changes
If the user edits the textarea during the deliberate 550 ms delay after clicking Run, this callback still scans the captured old content while the textarea displays the new text. The resulting scores and sensitivity panel are then presented as though they describe the visible input, and rewrite actions combine that stale result with the newly edited text. Cancel or invalidate the pending timer on input changes, or retain and display the exact scanned snapshot.
Useful? React with 👍 / 👎.
Summary
Turns the arcade's brain from decoration into an instrument, and its numbers from asserted into measured. The new rigor caught one significant scoring defect and three simulation bugs.
224 unit tests · 21 Playwright e2e · lint and build green.
The headline finding
Calibrating against a labelled corpus revealed trust was ranked backwards (Spearman −0.505): the engine scored outrage bait as more trustworthy than a sincere apology, because it counted trust vocabulary rather than evidence.
One of my own hypotheses was rejected by the data: removing "because" from the trust bank made things worse (−0.611), since it's sometimes a genuine evidence marker. Reverted; added specificity and stated-limitation signals instead, then discounted specifics inside urgency phrasing so a fake deadline can't buy credibility — while the honest deadline ("closes 30 November because the cohort starts in December") keeps its score.
Guarded in CI so scoring changes can't silently regress, and surfaced in-product via a calibration card.
What shipped
Deterministic brain — it ran on
Math.random(), so the same text produced a different brain every time and nothing was reproducible. Now pure, seeded and parameterized, with interventions (lesion / cut / stimulate), headless trials, and ten real measurements. It previously had no tests at all.A real spiking network — despite the name, BrainSNN was a rate model. Added a LIF network in the Brunel (2000) formulation (Dale's law, delta synapses, delays, Poisson drive), validated against published regime behaviour, with a lab that puts a spike raster and population rate on screen. It runs in a Web Worker. Verified in-browser: excitation-dominated fires at 215 Hz nearly clock-regular (CV 0.05), strong inhibition drops to 31 Hz and turns irregular (CV 0.41), sub-threshold drive is silent. Gamma-band power is now measured, not asserted.
Per-sentence math — leave-one-sentence-out attribution, the same replicates read as a jackknife for genuine error bars replacing the asserted
confidence. Runs inline (~27 ms).Defend the Brain — the arcade's first machine-checked mission; all fifteen other labs print a mission nothing verifies. Difficulty thresholds were measured: doing nothing breaches for ~179 ticks, silencing threat or boosting judgment holds, and cutting the salience route helps but isn't enough.
Verifiable run proofs — a score is checked by recomputation: the replay must produce it. Confirmed live — a genuine proof reports "replay matches"; the same proof with its score edited to 100 is rejected. Mirrors the Python verifier shipped for gaugegap proofpacks.
Honest score interpretation — the numbers are 0–100 indices, but users read "72" as "72%". Each tile now shows its percentile against the corpus. This earns its keep immediately: a phishing email scores a raw 32 on manipulation risk (looks safe) while sitting at the 86th percentile of known archetypes.
Evidence layer — Krippendorff's alpha, AUC/Brier/ECE metrics, an external-corpus eval script, and an annotation rubric. XP now pays for real accomplishments (verified: a Defense 97 win took XP 27 → 107) instead of only for opening a lab.
Bugs the rigor caught
Honesty notes
Not done, and why
Durable cross-user leaderboard storage: Supabase is wired for magic-link auth only — no tables, no persistence, no rate limiting. The verification logic that makes a leaderboard trustworthy ships here; the storage is separate infrastructure. Public corpora aren't vendored (licensing) — loader, rubric and metrics ship, data is fetched locally.
🤖 Generated with Claude Code
https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD