Skip to content

fix: a reference controller scan clock counted the period, not the time - #1

Closed
malgaroshy-maker wants to merge 1 commit into
masterfrom
fix/reference-controller-scan-clock
Closed

malgaroshy-maker wants to merge 1 commit into
masterfrom
fix/reference-controller-scan-clock

Conversation

@malgaroshy-maker

Copy link
Copy Markdown
Owner

What this changes

Every reference controller in tools/grade.py now measures its scan with real
elapsed time instead of assuming its nominal period. One test assertion is
removed, and the reasoning is left in its place.

Why

Nine graded tests failed on Linux CI and passed on Windows. They read as a
flaky grader. They were a controller whose clock was wrong.

run_scan called body(period) — handing every reference controller the
nominal scan period as its dt — and then slept that long. On an idle
machine an iteration really does take about period, so the controller and the
plant agree. On a busy one each iteration takes period plus the body plus
whatever the event loop was doing, and the plant advances by that whole amount,
because it runs on its own wall-clock accumulator. The controller counted only
period, so it under-counted elapsed time by an amount that depends purely on
machine load.

That is AGENTS.md gotcha 3 arriving somewhere new — "stepping once per
sleep(tick_ms) runs the sim slow... both engines accumulate real elapsed
time"
. The engines learned it; the reference controllers did not.

It surfaced in the batch-dosing stopwatch, which computes a cut-off in seconds:
23.8 L against a 22 L pot on Linux, 22.0 L on Windows, from the same code.

Two things I tried first that did not work, since this project's log keeps
those:

  1. I assumed the graded run's own clock was at fault and changed it from
    wall-clock to the plant's clock (already on master). That change is right
    on its own merits — a grader whose mark moves because the marking machine
    was busy is broken regardless — but it fixed nothing here, and I could not
    reproduce the failure with it either way. I was fixing the run's clock
    while the controller's clock was wrong.
  2. Fixing dt took nine failures to one. The survivor was not a bug at all.

That last failure was a claim the controller never made. The test asserted the
stopwatch's first batch lands inside tolerance. A stopwatch with no taper
cuts off at a scan boundary, so it overshoots by up to one scan's worth of
delivery — 2 L/s at the rated flow — and how coarse the scans get is a fact
about the machine, not about the program. The overshoot is also the thing the
scene teaches, so pinning it pins the lesson to a machine. That assertion is
gone; the lesson beside it got stronger, gaining a lower bound where it had
only an upper one.

What you ran

Reproduced on Linux (WSL2, Ubuntu 26.04), which is where CI runs and where the
failure lives — it has never failed on Windows.

Before, single test:

FAILED tests/test_grade.py::test_a_batch_timed_in_seconds_delivers_half_when_the_pump_is_re_rated
1 failed in 0.70s      (2 failures in 3 runs)

The grader's own report named the cause:

[XX] dose1.on_the_number   batch 1 delivered 23.8 L against a 22 L pot,
                           with the pump rated 120 L/min (within 1.5 L)

After the dt fix, same test, same machine:

1 passed in 78.65s (0:01:18)

Full Linux grader suite, after the dt fix and before the assertion change —
nine failures down to one:

1 failed, 50 passed in 1779.49s (0:29:39)

Windows, bus and parity suites, unaffected by this change:

134 passed, 1 warning in 8.57s

The full Linux grader suite is re-running to confirm 51/51 with both changes in.
I will post that output on this PR when it lands rather than claim it now.

Checks

  • python -m pytest -q passes
  • python tools/test_plan.py — section B is the suite above; the full plan
    takes ~45 min with the grader in it and is running
  • dotnet build in engine/ is clean at zero warnings — untouched by
    this change, verified on master at a8e5b41

If you added a test:

  • I broke the thing it checks, watched the test fail, and fixed it again —
    this change runs the other direction: I watched a failing test, found the
    product bug behind it, and removed one assertion that was measuring the
    machine rather than the code. Both are documented above.

Docs

  • No document states anything this makes wrong. The removed assertion's
    reasoning is now a comment at the site, so its absence is legible.

Related

Follows the HP-33 write-gate fix (09f1b44) and the timeout-handler fix
(a8e5b41), both of which were needed before CI could report this failure at
all — the handler meant to print TIMEOUT was raising TypeError instead.

Not fixed here, and worth its own change: CI's Python step is about thirty
minutes because these graded tests are nearly all of it. They should run as
their own job so a fast signal stays fast.

…time

Nine graded tests failed on Linux CI and passed on Windows. They read as a
flaky grader. They were a controller whose clock was wrong.

`run_scan` called `body(period)` -- handing every reference controller the
NOMINAL scan period as its dt -- and then slept that long. On an idle machine
an iteration really does take about `period`, so the controller and the plant
agree. On a busy one each iteration takes period plus the body plus whatever
the event loop was doing, and the plant advances by that whole amount because
it runs on its own wall-clock accumulator. The controller counted only
`period`, so it under-counted elapsed time by an amount that depends purely
on machine load.

That is AGENTS.md gotcha 3 arriving somewhere new: "stepping once per
sleep(tick_ms) runs the sim slow... both engines accumulate real elapsed
time". The engines learned it. The reference controllers did not.

It showed up first in the batch-dosing stopwatch, which computes a cut-off in
seconds: 23.8 L against a 22 L pot on Linux, 22.0 L here, from the same code.
`dt` is now real elapsed time. Nine failures became one.

The last one was a claim the controller never made. The test asserted the
stopwatch's FIRST batch lands inside tolerance. A stopwatch with no taper
cuts off at a scan boundary, so it overshoots by up to one scan's delivery --
2 L/s at the rated flow -- and how coarse the scans get is a fact about the
machine, not the program. Worse, the overshoot is the thing the scene
teaches, so pinning it pins the lesson to a machine. That assertion is gone
and the reasoning is in its place, while the lesson it was standing near got
stronger: batch two delivering about half now has both bounds where it had
one.

Verified: the batch-dosing case passes on Linux where it failed; full Linux
grader suite re-running to confirm 51/51 (it was 50/51 before this).

Not fixed here, and worth its own change: CI's Python step is about thirty
minutes because these graded tests are nearly all of it. They should run as
their own job.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@malgaroshy-maker

Copy link
Copy Markdown
Owner Author

Opened by mistake — closing. The fix itself is unaffected and will land separately.

@malgaroshy-maker
malgaroshy-maker deleted the fix/reference-controller-scan-clock branch September 22, 2026 06:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant