Skip to content

Playable brain: deterministic model, real spiking network, per-sentence math, calibration (which caught an inverted trust score) - #92

Merged
slavazeph-coder merged 14 commits into
mainfrom
claude/simulator-playground-strategy-6dl25j
Jul 30, 2026
Merged

Playable brain: deterministic model, real spiking network, per-sentence math, calibration (which caught an inverted trust score)#92
slavazeph-coder merged 14 commits into
mainfrom
claude/simulator-playground-strategy-6dl25j

Conversation

@slavazeph-coder

@slavazeph-coder slavazeph-coder commented Jul 12, 2026

Copy link
Copy Markdown
Owner

Summary

Turns the arcade's brain from decoration into an instrument, and its numbers from asserted into measured. The new rigor caught one significant scoring defect and three simulation bugs.

224 unit tests · 21 Playwright e2e · lint and build green.

The headline finding

Calibrating against a labelled corpus revealed trust was ranked backwards (Spearman −0.505): the engine scored outrage bait as more trustworthy than a sincere apology, because it counted trust vocabulary rather than evidence.

One of my own hypotheses was rejected by the data: removing "because" from the trust bank made things worse (−0.611), since it's sometimes a genuine evidence marker. Reverted; added specificity and stated-limitation signals instead, then discounted specifics inside urgency phrasing so a fake deadline can't buy credibility — while the honest deadline ("closes 30 November because the cohort starts in December") keeps its score.

dimension before after
trust −0.505 +0.700
manipulation risk 0.387 0.631
urgency 0.641 0.641
viral pull 0.333 0.366
pair accuracy 65.8% 81.2%

Guarded in CI so scoring changes can't silently regress, and surfaced in-product via a calibration card.

What shipped

Deterministic brain — it ran on Math.random(), so the same text produced a different brain every time and nothing was reproducible. Now pure, seeded and parameterized, with interventions (lesion / cut / stimulate), headless trials, and ten real measurements. It previously had no tests at all.

A real spiking network — despite the name, BrainSNN was a rate model. Added a LIF network in the Brunel (2000) formulation (Dale's law, delta synapses, delays, Poisson drive), validated against published regime behaviour, with a lab that puts a spike raster and population rate on screen. It runs in a Web Worker. Verified in-browser: excitation-dominated fires at 215 Hz nearly clock-regular (CV 0.05), strong inhibition drops to 31 Hz and turns irregular (CV 0.41), sub-threshold drive is silent. Gamma-band power is now measured, not asserted.

Per-sentence math — leave-one-sentence-out attribution, the same replicates read as a jackknife for genuine error bars replacing the asserted confidence. Runs inline (~27 ms).

Defend the Brain — the arcade's first machine-checked mission; all fifteen other labs print a mission nothing verifies. Difficulty thresholds were measured: doing nothing breaches for ~179 ticks, silencing threat or boosting judgment holds, and cutting the salience route helps but isn't enough.

Verifiable run proofs — a score is checked by recomputation: the replay must produce it. Confirmed live — a genuine proof reports "replay matches"; the same proof with its score edited to 100 is rejected. Mirrors the Python verifier shipped for gaugegap proofpacks.

Honest score interpretation — the numbers are 0–100 indices, but users read "72" as "72%". Each tile now shows its percentile against the corpus. This earns its keep immediately: a phishing email scores a raw 32 on manipulation risk (looks safe) while sitting at the 86th percentile of known archetypes.

Evidence layer — Krippendorff's alpha, AUC/Brier/ECE metrics, an external-corpus eval script, and an annotation rubric. XP now pays for real accomplishments (verified: a Defense 97 win took XP 27 → 107) instead of only for opening a lab.

Bugs the rigor caught

  1. Zero-delay spike transmission — the LIF ring buffer wrote into the slot it was reading, so spikes arrived instantly instead of after 1.5 ms. Fixed: the balanced network settles at 42 Hz instead of 69 Hz and the sweeps become cleanly monotonic.
  2. Mission loss didn't latch — a run that failed mid-way but recovered scored as a win.
  3. Divergent control ratio when a lesioned amygdala drove the denominator to ~0.

Honesty notes

  • The brain is driven by the lexical scores, so its dynamics add structure but no new information about the text. Every readout carries a claim boundary.
  • Limitations are pinned by tests, not hidden: spike probability can't reach its own cap because activity is capped at 1.0; Brunel's asynchronous-irregular regime needs N ~ 10⁴–10⁵, so the tests assert monotonic trends rather than claiming all four published regimes.
  • Only one of the four tactics the firewall detects maps cleanly to a published class, so public corpora should drive building a real detector — not just validate the current one.

Not done, and why

Durable cross-user leaderboard storage: Supabase is wired for magic-link auth only — no tables, no persistence, no rate limiting. The verification logic that makes a leaderboard trustworthy ships here; the storage is separate infrastructure. Public corpora aren't vendored (licensing) — loader, rubric and metrics ship, data is fetched locally.

🤖 Generated with Claude Code

https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces 'Mind-Hack Autopsy' (experiment 013), a new client-side interactive lab allowing users to simulate brain reactions to various content types locally. It updates the arcade layout, landing page, sharing dialogs, and E2E tests to support thirteen experiments, and adds a new evidence export feature for the attractor playground. The review feedback suggests several robustness and compatibility improvements, including defensive checks for clipboard API availability, safer destructuring of parameters in the evidence builder, and appending the temporary download link to the document body to ensure cross-browser compatibility.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread brainsnn-r3f-app/src/features/gaugegap/ContentReactionLab.jsx
Comment thread brainsnn-r3f-app/src/features/gaugegap/evidence.js Outdated
Comment thread brainsnn-r3f-app/src/features/gaugegap/evidence.js
@slavazeph-coder
slavazeph-coder force-pushed the claude/simulator-playground-strategy-6dl25j branch from 3d48582 to 138dc92 Compare July 12, 2026 20:59
@slavazeph-coder slavazeph-coder changed the title Make Mind-Hack Autopsy a playable arcade lab + attractor evidence export Playable brain lab: deterministic simulation, per-sentence math, and calibration (which caught an inverted trust score) Jul 29, 2026
@slavazeph-coder slavazeph-coder changed the title Playable brain lab: deterministic simulation, per-sentence math, and calibration (which caught an inverted trust score) Playable brain: deterministic model, real spiking network, per-sentence math, calibration (which caught an inverted trust score) Jul 29, 2026
slavazeph-coder and others added 14 commits July 30, 2026 19:04
…ce export

Content Reaction Lab (arcade experiment 013):
- New src/features/gaugegap/ContentReactionLab.jsx: paste an ad / tweet /
  landing page / email, run a fully client-side scan (analysisEngine +
  layerRouter, no /api round-trip), watch the live 3D brain react, and read
  four headline scores — Attention, Trust, Emotional Charge, Manipulation
  Risk — with delta badges against the previous run.
- One-click rewrites (build trust / add real urgency / reduce manipulation)
  scored in-browser with before→after deltas; new additive "urgency" goal in
  improve/rewrite.js. Share score card reuses ShareDialog.
- Registered as lab 013 in ExperimentArcade with XP/daily-mission integration;
  the landing's Mind-Hack Autopsy card now opens the lab in place instead of
  bouncing to /app ("Open full analysis" remains the escape hatch).
- New src/lib/headlineScores.js maps getBusinessMetrics to the four headline
  scores; score card, share text and share preview now show the same four
  rows so the playground and shared cards never disagree (this also changes
  /app's share card rows from Hook/Viral Pull to the headline set).

Truth layer:
- New src/features/gaugegap/evidence.js: "Export evidence" button on the
  Butterfly Effect (Lorenz) lab downloads a content-hashed JSON proofpack
  (schema gaugegap.browser_attractor_proofpack.v1) with live parameters,
  solver settings, scores, share URL and an explicit claim boundary.
- Deep Foundry link from the landing's research section to the published
  gaugegap-foundry Foundry Experience playground.

Tests: new tinyVitest suites for headlineScores and evidence; urgency-goal
coverage in rewrite.test.js; scoreCard test updated to the new rows. Stale
e2e landing specs (pre-arcade selectors) rewritten against the arcade:
content lab local-scan flow asserts no /api/analyze request; 3D-mount and
Reconstruct-nav specs target the current DOM. Full suite: 79 unit tests,
13 e2e passed locally.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
A shared link (?lab=content&state=<text>) now prefills the lab and runs the
scan automatically, matching the challenge-link loop of the other arcade
labs (same `state` param, so lab-switch URL cleanup applies unchanged).
"Share challenge" / "Copy link" actions reuse the DeepLabChrome helpers.
Shared text is capped and validated before auto-running.

Tests: unit coverage for the URL round-trip, foreign-lab guarding, size cap
and stale-run-param cleanup; new e2e spec proves a shared link auto-runs to
scored tiles. The auto-run effect keys on the parsed share payload only, so
StrictMode's dev double-invoke re-arms the cleared scan timer instead of
deadlocking, and keystrokes never re-apply the shared text.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
The 3D brain ran on Math.random(), so the same content produced a different
brain every time — scores could not be reproduced, shared or verified. Its
~20 model constants were hardcoded literals inside the step function, and it
had no test coverage at all.

- New src/features/brain3d/brainModel.js: the 7-region rate dynamics lifted
  out of the React hook into a pure, seeded, parameterized model. Adds
  interventions (lesion a region, cut a pathway, inject current) and exposes
  the excitatory/inhibitory split and per-pathway STDP delta that the old
  loop computed and discarded. DEFAULT_PARAMS reproduces the original
  behaviour exactly, so the live visual is unchanged.
- runBrainTrial() runs the model headlessly: same seed and inputs always
  produce an identical trace, which is what makes brain readouts scoreable
  and verifiable.
- New brainMetrics.js: firing rates in Hz, PFC/AMY control ratio, hijack
  index, gain around the THL->CTX->AMY->BG->|THL control loop, gate
  integrity, per-region E/I balance, mean pairwise spike correlation,
  settling time, plasticity, weight drift and net STDP flux — each with a
  descriptor stating its direction, plus an explicit claim boundary.
- New src/lib/rng.js: shared FNV-1a + mulberry32 seeding, matching the
  implementation solitonLayer.js already used privately.
- useBrainSimulation.js becomes a thin wrapper over the model and regains
  pause/step/reset/burst (dropped during the r3f port) plus stimulate,
  lesion, cut and speed control.
- 23 new tests covering determinism, bounds, interventions and metrics. One
  pins a real quirk found while testing: activity is capped at 1.0, so spike
  probability maxes at 0.836 and the spikeMax 0.85 ceiling is unreachable.
- tinyVitest gains the toBeLessThan matcher it was missing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
"Manipulation Risk 72" said nothing about why, and the single `confidence`
number was asserted rather than estimated. This adds attribution and a real
uncertainty estimate by re-running the scorer on modified inputs — genuinely
new information, not a restatement of the same features.

- New src/lib/ablation.js: leave-one-sentence-out scoring gives each sentence
  its exact contribution to each headline score. The same replicates are then
  read as a jackknife to produce a standard error and range per score, which
  is a standard estimator rather than an invented confidence. Seeded sentence
  shuffles measure whether a verdict depends on what was said or merely on
  the running order.
- topDrivers() ranks the sentences behind one score with each one's share.
- The lab now renders a "Where this score comes from" panel: ranked sentence
  bars, a tab per headline score, and ± on each tile. On mixed content the
  attribution correctly assigns the pressure sentence 100% of Manipulation
  Risk while splitting Trust across both sentences.
- Runs inline with the scan: measured at 21-27 ms including the 18-sentence
  ceiling, so it needs no button gate.
- 12 unit tests plus an e2e spec covering attribution and tab switching.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
The ten gallery classics already carried hand-labelled expectations that
nothing ever checked. This turns them into a real evaluation, and the first
run found a significant defect.

- New calibrationCorpus.js: 18 labelled archetypes (the 10 classics plus 8
  new originals) scored on four ordinal dimensions. Kept separate from
  CLASSIC_PRESETS so the evaluation set can grow without changing the UI
  gallery, and so re-ordering the gallery cannot alter a published figure.
- New calibration.js: labels are ordinal, so agreement is reported as
  Spearman rank correlation plus the count of labelled pairs the engine
  orders backwards — a falsifiable claim, rather than asserted precision.

Measured baseline: 66% of 442 labelled comparisons ranked correctly,
mean Spearman 0.214. Urgency is decent (0.64); manipulation risk (0.39) and
viral pull (0.33) are weak.

FINDING — trust is ANTI-correlated with its labels (Spearman -0.505, 65% of
pairs inverted). The engine ranks outrage bait as more trustworthy than a
sincere apology. Cause: `trust` rewards trust *vocabulary*, not evidence.
TRUST_TERMS includes the bare connective "because", so "share this because
once it's gone" earns +13, while "on Tuesday our update broke checkout for
six hours, here is what we refunded" earns nothing — none of its concrete,
checkable words are in the bank.

Thresholds in the test pin MEASURED behaviour, not aspiration, so scoring
changes cannot quietly regress. The trust defect is asserted explicitly
rather than hidden, and the assertion flips once the signal is fixed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
Trust was anti-correlated with its labels (Spearman -0.505): the engine
ranked outrage bait as more trustworthy than a sincere apology, because
`trust` counted trust *vocabulary* rather than evidence.

Each change was measured against the calibration harness rather than
guessed, and one hypothesis was rejected by the data:

- Removing the connective "because" from TRUST_TERMS made trust WORSE
  (-0.505 -> -0.611). It turns out "closes on 30 November because the cohort
  starts in December" is a genuine evidence marker. Reverted.
- The real gap was that concrete detail earned nothing. Added a specificity
  signal (numerals, dates, durations) and a stated-limitation signal
  ("not yet", "uncertain", "approximately"), since admitting limits is a
  credibility cue. Trust: -0.505 -> +0.560.
- Weighted specifics above evidence-vocabulary, on the principle that
  checkable detail is stronger than claiming to be transparent: -> +0.628.
- Dropped trust from viralScore: careful writing is not more viral, and
  including it was dragging viral pull down. Viral: 0.333 -> 0.366.

Measured across 442 labelled comparisons:

              before   after
  trust       -0.505  +0.628
  risk         0.387   0.629
  urgency      0.641   0.641
  viral pull   0.333   0.366
  accuracy     65.8%   80.1%
  mean rho     0.214   0.566

Every dimension now ranks in the right direction. Test thresholds move to
the new measured values, and a remaining limitation is pinned rather than
hidden: "verify within 24 hours" still reads as concrete detail, so phishing
scores higher on trust than it should. Discounting numerals adjacent to
urgency terms is the next step.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
Despite the name, BrainSNN was a rate model: 7 scalar units, no membrane
potential, no threshold, no refractory period. This adds the real thing.

- src/lib/snn/lifNetwork.js: sparse random network of leaky integrate-and-fire
  neurons in the Brunel (2000) formulation — Dale's law (80/20 excitatory/
  inhibitory), fixed-amplitude delta synapses, axonal delays, Poisson external
  drive, exact exponential leak. Typed arrays plus CSR adjacency; ~150 ms for
  800 neurons over 250 ms of biological time, and fully seeded.
- src/lib/snn/snnMetrics.js: the measurements that characterise a network
  state — firing rate, CV of ISI, Fano factor, population synchrony, and a DFT
  of the population rate. The last one matters: gamma-band power becomes a
  MEASURED property rather than a closed-form assertion.
- brunelValidation.test.js checks published consequences of the model, so the
  simulation is falsifiable rather than merely plausible:
    * silent below the threshold drive nu_thr, firing above it
    * rate monotonically increasing in external drive
    * rate monotonically decreasing in inhibition strength g
    * irregularity monotonically increasing in g — the central Brunel result
    * near clock-regular firing (CV < 0.15) when excitation dominates
    * rate never exceeding the refractory ceiling

Found and fixed a delay bug while validating: the ring buffer wrote into the
same slot it read that step, so a spike reached later-indexed neurons with
zero delay instead of 1.5 ms. With correct delays the balanced network settles
at 42 Hz instead of 69 Hz, and the sweeps become cleanly monotonic.

Known limitation, asserted rather than overclaimed: the asynchronous-irregular
regime needs N ~ 10^4-10^5 at epsilon ~ 0.1 for presynaptic independence. At
browser-affordable sizes the network reaches high irregularity but keeps more
population synchrony than true AI, so the tests pin the monotonic trends
rather than claiming all four published regimes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
All twelve existing labs print a mission string that nothing verifies. This
one has a real win condition, a real fail state and a finite budget, all
evaluated against the deterministic model from brainModel.js.

- brainGame.js: mission rules, a pressure profile that ramps threat drive up
  while suppressing judgment, five interventions (lesion the amygdala, cut
  the salience or action route, stimulate PFC or cerebellum), and a
  three-axis score in the arcade's house style where control, stability and
  budget efficiency pull against each other.
- BrainGameLab.jsx: arcade experiment 014. A 2D circuit view of the seven
  regions and ten pathways, live activity and spikes, cut pathways drawn
  dashed, lesioned regions crossed out. Sandbox / Mission / Challenge modes,
  pause, single-step, reset, and a shareable seed.

Thresholds were measured, not guessed. Sweeping the hijack limit against the
model showed doing nothing breaches for ~179 consecutive ticks, silencing
threat or boosting judgment holds the line, and cutting the salience route
helps but is not sufficient on its own — so the limit is set at 60 to keep
that gradient. A plausible-looking intervention that still loses is what
makes the mission worth replaying.

Two bugs found while testing:
- The loss condition read the breach streak on the final tick, so a run that
  failed mid-way but recovered was scored as a win. It now latches on the
  worst streak seen.
- Control ratio diverged when a lesioned amygdala drove the denominator to
  zero (21.73 on screen). Capped at 10, past which the ratio says nothing
  extra.

17 unit tests plus an e2e that confirms the mission is genuinely losable and
that intervening keeps a run alive past the point it previously failed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
Two pieces of the evidence layer.

Krippendorff's alpha (src/lib/agreement.js) — a labelled corpus without an
agreement statistic says nothing about whether the labels are reproducible by
another person. Full implementation with the coincidence-matrix algorithm,
supporting any number of annotators, missing ratings, and nominal / ordinal /
interval distance. Ordinal matters here: "low vs extreme" should count as a
bigger disagreement than "low vs moderate". Reported with the conventional
reliability reading (>=0.8 reliable, >=0.667 tentative).

Verifiable run proofs (src/features/gaugegap/runProof.js) — because the model
is deterministic, a Defend the Brain run is fully described by its seed, mode
and the tick at which each intervention was applied. So a score is checkable
by RECOMPUTATION: replay the log and see whether the claimed result falls out.
Inflating a score means finding a log that actually replays to it, not editing
a number. The lab now records intervention ticks and exports a hashed proof.

Privacy: a proof carries a seed and a list of tick/id pairs. No text, no
personal data — so the lab's "nothing leaves your browser" promise survives
even if proofs are later submitted somewhere.

Note on scope: durable cross-user leaderboards need storage this deployment
does not have. Supabase is wired for magic-link auth only — no tables, no
persistence, no rate limiting, and the server is stateless. The verification
logic that makes a leaderboard trustworthy is what ships here; the storage
behind it is a separate, infrastructural piece of work.

Fixed two bugs found by the tests: evaluateRun derived elapsed ticks from the
rolling frame window (so a completed replay read as still running), and the
proof hash was taken over an object without the hash key while verification
spread it back as undefined, which canonical JSON renders as null.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
Completes the evidence path from labels to a defensible number.

- src/lib/evalMetrics.js: AUC (ranking), Brier (probabilistic accuracy) and
  Expected Calibration Error with a reliability table, plus precision/recall/F1
  at the best operating point. ECE is the one that matters for honesty: users
  read these scores as percentages, and a scorer can rank items perfectly while
  its numbers mean nothing as probabilities. One test pins exactly that case.
- scripts/eval-corpus.mjs: evaluate the engine against any externally labelled
  JSONL corpus. Public datasets are NOT vendored — they carry their own
  licences — so datasets/ is gitignored and the script points at a local copy.
- docs/ANNOTATION_RUBRIC.md: dimensions, ordinal levels, worked anchors, and
  the rule that a corpus release without a Krippendorff alpha does not get to
  claim reliability. Maps the published persuasion-technique taxonomies
  (SemEval-2023 Task 3, SemEval-2020 Task 11 PTC, Webis clickbait) onto our
  dimensions, and records that only one of the four tactics the firewall
  detects today maps cleanly to a published class — so those corpora should
  drive building a real detector, not just validate the current one.

Smoke run on a six-item set: AUC 0.778, ECE 0.290 — ranks reasonably, but the
scores do not yet behave like percentages. Worth knowing and worth stating.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
…ust gap

Three follow-ups that finish the work rather than extend it.

1. The LIF network was a library nobody could see — tests and no UI. Added
   arcade experiment 015: a spike raster, population-rate trace and live
   measurements, driven by sliders for inhibition strength and external drive
   so you can walk across Brunel's phase diagram yourself. It runs in a Web
   Worker (a 2000-neuron run takes ~800 ms, which would drop frames across the
   whole arcade on the main thread). Verified in-browser: excitation-dominated
   fires at 215 Hz nearly clock-regular (CV 0.05), strong inhibition drops to
   31 Hz and turns irregular (CV 0.41), sub-threshold drive is silent.

2. Fixed the trust defect this suite previously pinned as known-broken. The
   specificity signal counted numerals, so "verify within 24 hours" read as
   concrete evidence and bought phishing credibility. Specifics inside urgency
   phrasing are now discounted:

                     before  after
     trust rho        0.628   0.700
     pair accuracy    80.1%   81.2%
     phishing trust      68      60
     flash-sale trust    54      46

   Crucially the honest deadline ("closes 30 November because the cohort
   starts in December") keeps its 96 — the discount targets coercion, not
   dates. Two tests now pin both halves of that distinction.

3. XP could only be earned by opening a lab (25 first time, 2 after). Winning
   a mission paid nothing. Added recordAchievement with a bounded, one-off
   40 XP plus per-lab personal bests, threaded through the arcade to Defend
   the Brain. Verified live: a Defense 97 win using 1 of 6 interventions
   unlocked both achievements and took XP from 27 to 107.

The progress rules moved into arcadeProgressCore.js because the bare-Node test
runner cannot import .jsx — which is why this economy had no tests at all. It
now has twelve, covering double-pay, streak resets, and migration of progress
saved before the new fields existed.

214 unit tests, 21 e2e, lint and build green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
…tion

The four headline numbers are 0-100 INDICES, but users read "72" as "72%".
The eval harness confirmed the gap: expected calibration error ~0.29 against
any probabilistic reading. Rather than invent a probability the engine cannot
support, this gives each score a statement that is true.

- src/lib/scoreReference.js: percentile of a score against the 18 labelled
  archetypes, with the midpoint convention for ties. Each tile now reads
  "around the 86th percentile of 18 known archetypes" instead of leaving the
  bare index to be misread.

  This immediately earns its keep: a phishing email scores a raw 32 on
  manipulation risk — which looks safe — while sitting at the 86th percentile
  of known archetypes. The index is compressed; the percentile tells the
  truth.

- A calibration card in the lab states what the engine has been measured to
  do, where the scores are actually read rather than buried in a README:
  rank agreement per dimension, how many labelled pairs are ordered wrongly,
  and an explicit note that these are indices, not probabilities.

Current measured figures, shown in the card: 81% of 442 labelled comparisons
ranked correctly, mean Spearman 0.585 (trust 0.70, urgency 0.641,
manipulation risk 0.631, viral pull 0.366).

Also fixed an ordinal-suffix bug the browser check surfaced ("81th"), with
the teens case covered.

224 unit tests, 21 e2e, lint and build green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
- The game can now verify a run proof, not just export one. Drop a proof file
  in and it replays the intervention log and checks the claimed score falls
  out. Confirmed in-browser: a genuine proof reports "replay matches"; the
  same proof with its score edited to 100 is rejected with two problems (the
  score mismatch and the broken content hash). This mirrors the Python
  verifier already shipped for gaugegap proofpacks — export in one place,
  check independently in another.

- README now states what the engine has actually been measured to do, rather
  than leaving the claim implicit: the per-dimension rank agreement table,
  the 81%-of-442-comparisons figure, the trust defect calibration caught and
  how it was fixed, and the two honesty notes that tests enforce — scores are
  indices rather than probabilities, and labels are ordinal. Documents the
  deterministic brain model, the LIF network validated against Brunel, and
  the claim boundary that ships with every neural readout.

224 unit tests, 21 e2e, lint and build green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
@slavazeph-coder
slavazeph-coder force-pushed the claude/simulator-playground-strategy-6dl25j branch from 9719c7b to 98b5918 Compare July 30, 2026 19:10
@slavazeph-coder
slavazeph-coder marked this pull request as ready for review July 30, 2026 19:10
@gemini-code-assist

Copy link
Copy Markdown

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@slavazeph-coder
slavazeph-coder merged commit aad208c into main Jul 30, 2026
1 check passed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 98b5918725

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +108 to +109
const stability = Math.round(Math.max(0, 1 - worst / Math.max(1, rules.breachGrace)) * 100);
const defense = Math.round(Math.max(0, Math.min(100, controlScore * 0.45 + stability * 0.35 + efficiency * 0.2)));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Use a defined grace period for challenge scoring

When the player switches to Challenge mode, rules is CHALLENGE, which has no breachGrace property. This makes the stability expression evaluate to NaN, and the derived defense score also becomes NaN; the challenge UI therefore shows invalid scores, achievements cannot record a meaningful best score, and exported challenge proofs fail verification because NaN !== NaN. Use a challenge-specific denominator or define breachGrace for CHALLENGE.

Useful? React with 👍 / 👎.

Comment on lines +63 to +65
const sentences = segments.map((sentence, index) => {
const without = segments.filter((_, other) => other !== index).join(' ');
const ablated = scoreText(without);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve sentences beyond the attribution display limit

For inputs longer than 18 sentences, splitIntoSegments already truncates the passage, so each leave-one-out scan is built only from the first 18 sentences while the baseline was scored from the complete input. Consequently, signals in sentence 19 and later disappear from every replicate and are falsely attributed to each displayed sentence—for example, an urgent 19th sentence makes otherwise neutral opening sentences appear to add manipulation risk. Build each ablation from the full passage, even if only 18 contributions are displayed.

Useful? React with 👍 / 👎.

Comment on lines +63 to +65
timerRef.current = window.setTimeout(() => {
const nextResult = scanContentLocally(content);
setResult((current) => {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Cancel a pending scan when the input changes

If the user edits the textarea during the deliberate 550 ms delay after clicking Run, this callback still scans the captured old content while the textarea displays the new text. The resulting scores and sensitivity panel are then presented as though they describe the visible input, and rewrite actions combine that stale result with the newly edited text. Cancel or invalidate the pending timer on input changes, or retain and display the exact scanned snapshot.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant