Acceptance run against the box, drip-feed, and FOCAS telemetry - #2
Merged
Conversation
The 113 tests need no hardware - the wire format is built by pure functions and the transfer logic runs against a scripted fake device and a fake serial port. That is exactly why it is worth automating: the golden byte strings in tests/test_protocol.py are the only copy of the reverse-engineered protocol outside the physical box, and nothing else would catch a refactor that quietly changed them. A second job imports every module on Linux with no Fanuc DLL and no serial port present. That guards the fix that stopped focas.py and serial_adapter.py calling sys.exit() at import time - the kind of regression that only shows up on the Raspberry Pi, in production. Evidence: matrix over Python 3.9 / 3.11 / 3.13; checked locally that the code uses no 3.10+ syntax and that all seven modules import with neither DLL nor serial port available.
Everything in this toolkit is tested against a fake device. That proves
the logic is self-consistent; it cannot prove the firmware agrees, and
three open questions have been sitting unanswered since the integrity
work went in:
Q1 does the box store bytes exactly as sent, or does its FatFs
normalise line endings? VERIFY_UPLOAD depends on the answer, and if
it normalises, verification fires on perfectly good programs.
Q2 does a listing really end with the 0xFFFF terminator? list_dir()
raises on silence, which is correct only if the terminator is real.
Q3 does block accounting hold under real UDP traffic, including the
512-byte boundary where an empty final block is sent?
hardware_check.py settles all three in one run and writes a report.
Safety, since this gets run standing next to a machine: it never issues
RunFile or StopDnc, so nothing is started, stopped or sent to the CNC -
the box is treated purely as storage. It refuses to overwrite an existing
file, uses a canary name, cleans up after itself, and --read-only
performs no writes at all while still answering Q2.
Evidence: 8 tests drive the script against a stateful simulator of the
box, including simulated firmware that normalises line endings and
firmware that goes quiet instead of terminating a listing. Each asserts
the DIAGNOSIS, not the exit code - a run that fails without saying which
question it settled would be useless at the machine. Those tests also
caught main() returning None on the success path, which made the exit
code 0 even when checks had failed.
"Copy to memory" needs the whole program to fit in the control's memory. A mould or a 3D finishing path does not fit in 64 KB, and running those is the actual reason shops buy the commercial DNC boxes. Drip-feed streams the program while the control executes it, so size stops being the limit. The port lock is held for the entire job - minutes, not milliseconds. That is deliberate: it is one wire and the machine is cutting from it, so the listener must not interleave reads and eat the flow-control bytes. Automatic receiving is paused until the job finishes. write_timeout is raised to 300s for the duration and restored afterwards. A control can legitimately hold XOFF for minutes during a slow finishing pass, so the 15s default used elsewhere would abort a healthy job - but leaving 300s in place afterwards would make a dead cable look like a hang on the next ordinary send. *** THIS HAS NEVER RUN AGAINST A MACHINE. *** The machine is cutting while it streams: if the feed stops it stops mid-cut, and if the control's buffer overflows it executes garbage. Flow control is not a nicety here, it is the safety mechanism. The full warning, including how to try it for the first time, is at the top of the drip-feed section in serial_adapter.py. Evidence: 9 tests covering the paths that decide what happens when a job goes wrong - that an abort actually stops the wire, that it reports how far it got, that the raised write timeout is restored even on abort, and that a drip-feed is refused while a program is arriving. Not validated: everything about the machine's side of it.
The README has claimed since the first commit that FOCAS gets telemetry for free, because the same library that pushes a program also reports run state, alarms, feedrate and spindle speed. This makes good on it: run state, selected mode, motion, program numbers, the block being executed, active alarms decoded from their bit mask, feed and spindle. Strictly read-only. Nothing here writes a parameter, starts a cycle or clears an alarm. Every reading is taken independently and a failure costs only its own row. Controls differ in which calls they support, and a panel that shows nothing because this particular control lacks cnc_rdexecprog would be worse than useless on a shop floor. *** THIS HAS NEVER RUN AGAINST A CONTROL. *** The ctypes structure layouts are the part no test can check: ODBST field order differs across control series, and the wrong layout does not crash - it returns plausible nonsense. If one field reads as garbage while its neighbours look sane, the layout is the first suspect. Marked # CHECK? in the source. Evidence: 20 tests on the parts that are checkable - alarm bit decoding against the specification order, unknown status codes being surfaced rather than silently passing as normal, error codes becoming exceptions, and a snapshot surviving every single reading failing.
README gains the three new capabilities, a CI badge, and - for the two that have never touched hardware - the warning next to the feature rather than buried in a source file. The drip-feed section says plainly that the machine is cutting while it streams and how to try it the first time. run.bat grows entries for the telemetry watch and both acceptance modes, read-only first, so the safe one is the easier one to pick. Evidence: docs and menus only, no behaviour change - 113 tests still pass.
CI caught this on its first run: the test forced a logging failure with "Z:\nope\cannot\write.csv", which is impossible to write on Windows but a perfectly valid RELATIVE FILENAME on Linux. The write succeeded there, a row was logged, and the assertion that the log stayed empty failed. The test was wrong, not transfer_log. The property it means to check - that a logging failure never aborts a transfer that already succeeded - now gets an unwritable path that holds on any OS: a regular file put where a directory would have to be, so opening a path beneath it raises NotADirectoryError everywhere. Evidence: 113 tests pass locally on Windows; the point of the change is that they should now also pass on the Linux runner, which is what the previous run disproved.
CI found both on its first real run; neither is a defect in the toolkit. socket.timeout is not the builtin TimeoutError before Python 3.10. The box simulator raised the builtin, which sails straight past the `except socket.timeout` in _exchange, so under 3.9 the acceptance script reported a bare "TimeoutError:" instead of its own DncTimeout and the guidance about local UDP port 69. Production code was right; the fake was wrong, and it was wrong in the direction that hides a real diagnostic. The busy-line guard was tested by starting a listener thread, feeding it half a program, and asserting on _receiving from the main thread. That asserts on a state another thread is free to leave at any moment. It passed locally every time and failed on a loaded runner, which is the signature of a race in the test rather than in the code - and I could not reproduce it here, so rather than stretch a timeout until it went quiet, the race is gone: the listener publishing the flag and the guard honouring the flag are two properties and now have two tests, neither of which depends on catching another thread mid-stride. Evidence: 114 tests pass locally, and the two formerly racy files were run five times in a row with no variation. Whether the 3.9 fix holds is for the runner to say - that is the whole point of having one.
This was referenced Aug 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Four things: one that makes the rest verifiable, one that automates the
suite, and two features that have never touched hardware and say so.
Acceptance run —
hardware_check.pyThe integrity work merged in #1 left three questions that only the firmware can
answer, and they have been open ever since:
line endings?
VERIFY_UPLOADdepends on it — if it normalises, verificationfires on perfectly good programs.
0xFFFFterminator?list_dir()raises on silence, which is right only if the terminator is real.
512-byte boundary?
One command settles all three and writes a report.
Safety, since it gets run standing next to a machine: it never issues RunFile or
StopDnc, so nothing is started, stopped or sent to the CNC. It refuses to
overwrite an existing file, uses a canary name, cleans up after itself, and
--read-onlywrites nothing at all while still answering Q2.CI
113 tests, no hardware. A second job imports every module on Linux with no Fanuc
DLL and no serial port — guarding the fix that stopped
focas.pyandserial_adapter.pycallingsys.exit()at import time, a regression that wouldonly surface on the Pi, in production.
Drip-feed
"Copy to memory" needs the program to fit in the control's memory. A mould or a
3D finishing path does not fit in 64 KB, and running those is the actual reason
shops buy the commercial boxes. This streams the program while the control
executes it.
The port lock is held for the whole job, so the listener cannot interleave reads
and eat the flow-control bytes.
write_timeoutis raised to 300 s for theduration — a control can hold XOFF for minutes on a slow pass — and restored
afterwards, so a dead cable does not look like a hang on the next send.
FOCAS telemetry
Read-only: run state, mode, motion, program numbers, the executing block, alarms
decoded from their bit mask, feed and spindle. Every reading is independent, so
a control lacking
cnc_rdexecprogstill reports its run state instead ofshowing nothing.
Verification
113 tests, all off-hardware. The acceptance script is itself driven against a
stateful simulator of the box, including simulated firmware that normalises line
endings and firmware that goes quiet instead of terminating a listing — each
test asserting the diagnosis, not the exit code. Those tests caught
main()returning
Noneon the success path, which made the exit code 0 even whenchecks had failed.
Drip-feed is covered on the paths that matter when a job goes wrong: abort stops
the wire, reports how far it got, and restores the raised write timeout even on
abort.
Not validated
Drip-feed and telemetry have never touched hardware, by construction — the
acceptance script is the thing meant to change that for the DNC box, and there
is no equivalent for a cutting machine short of a scrap part and a hand on feed
hold.