feat(playtest): settle planet-1's aim question — the aim is not the variable - #24
Merged
Conversation
…ariable Pilot 4 left one experiment owed: no seat ever issued a valid jump from the pit lip, so nothing was known about whether a correctly-aimed pair clears it. Run it scripted (aim-sweep.sh, 68 trials, no LLM) rather than as another pilot, because a free-tier seat dies inside 5.5 minutes and this needs ~70 attempts. Result: no bare jumpAtX cleared the pit at any of 12 values (0 of 48) — including 19 attempts that landed on the bridge and then ran off its far edge, because nothing released the run. jumpAtX with an untilX over the bridge landed 6 for 6 across a 72px span of take-off points. So planet-1 needs no level change; it needs feedback. Two harness defects the sweep exposed, both now Active backlog items: - /move reports `jumped` when the driver pressed jump, not when the astronaut left the ground. Eight trials at jumpAtX >= 646 reported `jumped` having never jumped — the astronaut was already airborne past the x=656 fall edge. This is the ambiguity pilot 4 could only describe from the seat's side. - A landing-then-walk-off is indistinguishable from a miss; both report a plain pit death. aim-sweep.sh also adds two reusable techniques: /planet as a per-trial reset (scene.start, so no state leaks between trials — a bare jump grazes the bridge's top face and ARMS it, which silently expired one mid-flight during development), and a live trajectory trace, possible only because /state is registered off the laptop driver's serial command chain. Docs + shell only, no TS/JS touched, so `npm run check` is not applicable (`biome check --write` would also reformat other sessions' files). Verified with `bash -n`.
F1 Block C's table dropped a trial and miscounted the no-jump row: it is 12 trials, and parks >=636 are 8 (6 never jumped, 1 jumped and fell, 1 jumped and landed), not 7 and "5 of 7". Row 1's park label was wrong too — both cut=none trials parked at 624, both cut=780 at 628. Root cause was hand-tallying, so aim-sweep.sh now emits the per-block/per-cell summary its own header has always promised; the tallies reconcile against the 68-trial log. F2 Referencing backlog items by ordinal breaks on every insert — this PR's new item 4 shifted 429-retryability to 7 and orphaned the pilot report's pointer. Reference by title instead. F3 Follow-up 3 was not just unrouted, it was wrong: it blamed the undiscoverable untilX:780 on the phone seat's blindness, but the phone seat never issues untilX — the laptop does, and the laptop cannot see the bridge because the harness is headless, which DESIGN.md already lists as an accepted v1 limit. Withdrawn and replaced with a plain statement that nothing here routes to constellation, plus the one thing that does: 68 trials, 0 reached the far ledge, so Planet.ts's "uncrossable without a platform" docblock is now evidenced rather than asserted. The BACKLOG Shipped entry no longer claims an unrelated item carries it. F4 trial() discarded bank_platform's failure return and never read the freeze's `solved`, so a trial run at an unbridged pit emitted a TRIAL line indistinguishable from a valid one. Both preconditions are now checked and a failure is labelled SKIPPED in the log instead of silently counted. F5 "2b." is not a CommonMark ordered-list marker — the DESIGN.md entry rendered glued into step 2. Made it step 3 and renumbered. F6 Block A holds ms:4500 fixed, so its 0-of-48 cannot separate an aim failure from a stop failure. Said so, in the headline bullet and the caveats. F7 Measured take-off is biased one-sidedly low (-16..0px), not the "+/-16px" the caveats claimed — the V=566 trials report take-offs below the jumpAtX the driver waits for, which no real take-off can produce. F8 Added the two missing measurement caveats: the two-consecutive-sample landing rule makes 19 a floor, and the ~20Hz trace perturbs the runs it measures. No conclusion changes: it still rests on Block B, which is untouched.
…erclaimed F9 The paragraph that replaced the withdrawn follow-up 3 asserted the sweep evidences constellation's "the pit is still uncrossable without a platform" docblock. It does not, twice over. Every trial ran WITH a bridge banked (plat=1, 68 of 68) — and as of the previous commit's precondition guard a no-platform trial is structurally impossible — so the counterfactual has zero samples; the freeze half of that docblock is equally untested. And the sentence said "19 crossings depended on the bridge" eight words after "nothing ever reached the far ledge": the log has farledge=None in all 68 rows and zero crossings. The 19 were bridge LANDINGS that then ran off and died. Replaced with the claim the data actually supports: 19 Block A trials took a ground-launched jump whose trajectory never contacted the bridge (take-offs 608-644, bridge=None) and none reached the far ledge, so on those the platform is not a confound. Stated as the behavioural counterpart to planet1.test.ts's existing geometric assertion, not a replacement for the untested run. This one had already left the repo — the constellation PR earendil-works#40 comment carried the same error and has been corrected in place, with the correction stated at the top rather than silently edited. F10 The headline bullet claimed the 0-of-48 "is a statement about the missing stop and not about the aim" while the caveat it pointed at said the 0-of-48 cannot separate those two. The report's own table refutes the strong form: of the 48, 8 never jumped and 2 clipped the ceiling — failures no stop can fix. Scoped the bullet to the 19 a stop would actually have saved. F11 The summary's comment claimed it counts anything a reader might otherwise count by eye, but it never parsed take-off or apex, leaving Block A's 2/19/19/8 take-off table hand-tallied off 48 rows — the same failure mode F1 caught, one table over. The summary now derives all four outcome classes with their take-off spans; run against the 68-trial log it reproduces the report's Block A table exactly (bonked 556-560 apexY 394/395, bridge 19 @568-612 touchdown 784-820, missed 19 @608-644, no-jump 8). Nice-to-haves F10/F11 were fixed rather than deferred, extending Kyle's "fix F1-F8" instruction from the previous round to this one.
…tops F12 The headline said the 19 that flew clean past the bridge failed "for reasons a stop cannot fix". Block B refutes that with a matched pair: take-off 644, apex (768,362), identical — bare it dies, with untilX:780 it lands at 788. The root cause is that the report used "a stop" for two maneuvers of different power. Now stated as the split it is: a mid-flight cut over the bridge saves all 38 trials that got airborne past the ceiling; merely releasing the run after touchdown saves only the 19 that landed; 10 are beyond either (8 never jumped, 2 clipped the ceiling). Third round on this sentence, so the headline no longer attributes at all — it states the measured outcomes and the split follows in its own paragraph. F13 Block B's touchdown range was given as 780-796. 780 is the untilX cut VALUE, and the only 780 in the data is a block C landing; block B's measured touchdowns are 788, 788, 788, 788, 792, 796. Corrected to 788-796 and attributed to the same 60ms poll overshoot the report already explains for jumpAtX. F14 Two unguarded None paths in the summary added last commit. span() checked that at least one take-off was set and then ran min() over the raw mix, so one traceless row would abort the entire summary with a TypeError — including the per-cell table block C's numbers come from. And a row with no trace joined none of the three outcome classes, so the counts silently stopped summing to n: F1's failure mode reproduced inside the fix for F1. span() now filters internally and an UNCLASSIFIED line is printed when the classes do not sum. Verified on a synthetic degraded log (traceless + takeoff=None rows): no traceback, unclassified announced. Real log output is unchanged. Fixed under Kyle's re-verify waiver — the 3-dispatch reviewer cap was spent, so these three fixes carry no zero-context verification. Each claim was checked against the 68-trial log before acceptance.
Owner
Author
Adversarial review — CLEAR TO MERGE (rounds: 3)
Waived by Kyle: F12, F13, F14 — "Fix all three, no re-verify". The three-dispatch Coverage: complete at all three rounds — every changed file read in full, plus Three findings had teeth worth naming. F9 caught an unsupported claim that had Zero-context reviewer + author triage on all findings; no judge dispatch (nothing |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pilot 4 (#23) closed with one experiment owed: no seat ever issued a valid jump
from the pit lip, so it could not say whether a correctly-aimed pair clears it.
This runs that experiment as a scripted 68-trial probe rather than another
two-seat pilot — both pilot-4 runs died to a free-tier token-rate 429 inside 5.5
minutes, and this question needs ~70 attempts.
The answer reframes the question
The aim is not the variable.
jumpAtXtrials across 12 values: 0 clears, all fell in the pit.ran off its far edge, because
ms:4500with nountilXkeeps holding rightafter touchdown.
jumpAtX+untilX:780: 6 for 6, every time a jump actually happened,across take-offs from 572 to 644.
So planet-1 does not need a level change — the working maneuver tolerates a 72px
error in the one number a seat picks. It needs feedback.
Two harness defects this exposed
Both are now Active backlog items, and both are the same shape as
diedAtin#21 — the harness knows the answer and discards it before the seat sees it.
/movereportsjumpedwhen the driver pressed jump, not when theastronaut left the ground. Eight trials at
jumpAtX≥ 646 reportedjumpedwith a trace apex of y=476 — never airborne. The driver polls at60ms (14px at run speed), so by the time a poll sees x ≥ 646 the astronaut
has usually already crossed the fall edge at x=656, where the jump input is
inert. This is exactly the ambiguity pilot 4 described from the seat's side:
run B's
jumpAtX: 680returnedjumpedanddiedAt {772, 592}, identicalto its no-jump attempt. They were indistinguishable because they were the
same event.
jumps here reported as plain pit deaths.
Also refuted
The round-3 reviewer simulation on #23 (F17, unexecuted) predicted that without
untilX, ≤606 lands, 608–634 dies, 636–648 clears. Measured: ≤612 touches thebridge but still dies (0/19 survived), 636–648 does not clear, and 646+ produces
no jump at all. It assumed the aim was the variable — the same assumption the
seats made.
What's in the diff
aim-sweep.sh— the probe. Two reusable techniques:/planetas a per-trialreset (a
scene.start, so nothing leaks between trials), and a livetrajectory trace, possible only because
/stateis registered off the laptopdriver's serial command chain and answers during a running
/move. It tees tologs/<date>-aim-sweep.lograther than printing and discarding, which is thegap pilot 4's appendix flags for
verify-rails.sh. Same rule as that script:it encodes the solution, so player prompts must never reference it.
AIM-SWEEP-2026-08-13.md— the report.PILOT-2026-08-13.md— a pointer at the top so finding 2 is read with theanswer in front of it.
DESIGN.md— scripted measurement as verification step 2b.BACKLOG.md— pilot 4 and the sweep into Shipped; the jump-outcome telemetry,the rails-log tee, and the
arm-firedremaining-life item into Active.Testing
Docs + shell only, no TS/JS touched, so
npm run checkis not applicable(
biome check --writewould also reformat other sessions' files).bash -nclean. The 68-trial log is at
.pi/playtest/logs/20260813-185618-aim-sweep.log(git-ignored, like every run artifact); every number in the report is a field
from it.
One open question for you
The report concludes no constellation-side level change is indicated, so
nothing needs filing there on that front. The one game-side item that stands is
the UI feedback ask pilot 4 already recorded as finding 6 (the phone seat cannot
see where the bridge lands, so
untilX:780is not derivable) — the sweep raisesits priority from "helps diagnosis" to "it is the input to the only parameter
that works". I have not opened anything in the constellation repo. Say the word
and I will.