Skip to content

ENG-34: Run scripts under a PTY so terminal-aware tools work - #286

Open
gorandodig wants to merge 67 commits into
mainfrom
node-subprocess-handling
Open

ENG-34: Run scripts under a PTY so terminal-aware tools work#286
gorandodig wants to merge 67 commits into
mainfrom
node-subprocess-handling

Conversation

@gorandodig

@gorandodig gorandodig commented Aug 15, 2026

Copy link
Copy Markdown
Member

Replaces pipe-based script execution with terminal-backed subprocess backends (ADR-001, Option A): scripts run with fds 0/1/2 on one terminal in their own contained process tree, output is VT-rendered to clean transcripts, live terminal queries (DSR/CPR/DA) get answered, and teardown covers the whole tree with bounded escalation. Fixes the Node.js startup failure on macOS.

  • POSIX: PTY-backed sessions (openpty + login_tty launcher, process-group teardown).

  • Windows: ConPTY + Job Object backend (render_machine/_conpty.py), validated by a dedicated windows-2022 lifecycle job plus the windows-latest suite.

  • CODEPLAIN_NO_PTY=1 selects the legacy pipe backend on every platform.

  • CI runs tests on ubuntu/macos/windows, type-checks per platform, and a macOS e2e job runs Node through the installed execute_script().

  • Implementation plan: https://app.notion.com/p/3bd5a3b25692816eb299de519f4a7feb

  • Linear: https://linear.app/codeplain/issue/ENG-34/issue-running-node

The harness reconstructs the topology from ADR-001 - a child spawned into a
new session whose fd 0 still points at the terminal of the session it left -
using only synthetic processes, so it documents the defect independently of
any fix and never validates one. The Node probe is a manual, non-gating
diagnostic whose outcome varies by host.
Locks in today's exit-code passthrough, merged stderr, timeout result,
cancellation, large-output drain, and output sanitizing, so the PTY backend
rewrite can be proven equivalent.
A rendered script inherited fd 0 pointing at the renderer's terminal, and a child
in a new session bypasses SIGTTIN for a terminal it does not own, so reads
succeeded and consumed the user's keystrokes. Pointing fd 0 at /dev/null ends
that and gives an immediate EOF instead; interim until the PTY backend lands.
The macOS bug this branch addresses had no macOS runner, and the unit suite had
no Windows runner at all. Per-platform mypy plus per-module strict error codes
protect the platform-split modules that follow. The removed test step sourced
.env.dev.example, which is not in the repo; coverage still uploads from Ubuntu.
The macOS bug this branch addresses had no macOS runner, and the unit suite had
no Windows runner at all. Per-platform mypy plus per-module strict error codes
protect the platform-split modules that follow. The removed test step sourced
.env.dev.example, which is not in the repo; coverage still uploads from Ubuntu.

The tests job runs one command per step so a failing command fails the job on
Windows, where PowerShell otherwise continues past a failed native command.
Tilde-expansion tests set USERPROFILE alongside HOME, since expanduser reads
USERPROFILE on Windows.

� Conflicts:
�	.github/workflows/lint-and-test.yml
The launcher attaches a PTY slave to fds 0, 1 and 2, verifies the terminal
invariants before exec, and reports progress as typed length-framed records.
An acknowledgment barrier holds the target until the parent has recorded its
process group, and -I -S keeps anything else from running before that point.
One pseudoterminal backs the target's three standard descriptors. The spawn
sequence completes a strict framed handshake and an acknowledgment barrier that
records the process group before the target can run. A single reader thread owns
the master descriptor and services a bounded, byte-accounted input queue.
Adds the fault-injection and lifecycle suite: reader shutdown, ack-barrier
cancellation on both sides, descriptor ownership, escalation and reap ordering,
VEOF delivery, input-queue receipts, and the bounded final drain. Two fixes fell
out: the reader's select unpacking and joining a reader that never started.
Renders the terminal stream into scrollback plus final screen instead of
stripping escape bytes, so repaints and progress rewrites collapse the way
a terminal collapses them, and caps the transcript head and tail.
Fixtures are real recordings of npm, pytest, a spinner, a full-screen
repainter, and a detached run.
The parser now runs in the reader through its byte-feed hook, so a target that
emits a device-status, cursor-position or attributes query gets an answer
instead of hanging to the timeout. Each reply is one non-blocking admission
into the ordered input queue, and its obligation is tracked to completion.
A failure part-way through _open_channels() leaked the descriptors already opened and
escaped as a raw OSError instead of the environment-error channel.
The final drain appended straight to the raw buffer, so output still in flight at close()
never reached read_output().
close_and_fail_all() dropped the in-flight item without closing its transaction, leaving
termios unrestored when an EAGAIN'd VEOF met a close.
submit() copied before validating the size and admitted unlimited zero-length items; the
queue is now bounded in items as well as bytes.
Two tests were not testing what they claimed: the daemon-thread target outlived nothing,
and the pre-ack cancellation raced a timer instead of the delayed-ack hook.
Escape sequences are framed and capped before pyte sees them, so an unterminated OSC string or CSI parameter no longer grows the reader's memory, an oversized sequence is dropped rather than parsed, and a parse failure costs one sequence instead of the rest of an OS-sized read.
One cell can no longer accumulate combining marks without bound.
The normalizer gained an idempotent finalize(), called at reader shutdown beside the decoded-stream flush, so a trailing partial UTF-8 sequence renders as U+FFFD instead of vanishing.
Terminal-reply failures are counted in full but retained as a bounded, deduplicated sample, and failure_detail() names that sample plus how many failures it omits.
Fixture tests assert a committed golden rendering per recording and compute ratios byte to byte; new tests cover the caps, boundary-invariant recovery, a query flood, and two-thread races at the callback and quiesce boundary.
Wraps today's Popen path — merged streams, the drain thread, the Linux pipe
widening — behind the TerminalProcess interface, keeping stdin at DEVNULL so
neither the escape hatch nor the Windows interim can steal keystrokes again.
The backend has no input channel, so its query responder starts quiesced and
output is fed through the normalizer.
Scripts now run on the PTY backend on POSIX and on the legacy pipe backend on
Windows, an interim until the ConPTY backend lands. One arbiter ranks the
conditions that can race — infrastructure failure, cancellation, the deadline,
an undelivered terminal reply, the target's exit — so launch and reader failures
reach the environment-error channel instead of the patcher, and the timeout
message names the absent input driver. The temp file now holds the rendered
transcript, with the raw bytes kept beside it.
Setting CODEPLAIN_NO_PTY=1 in Codeplain's own environment runs scripts on the
legacy pipe backend, read once per spawn at the single construction site and
warned about on every use. It is never selected automatically — a failed
openpty() stays an environment error — and the variable is stripped from the
child's environment so a rendered script cannot branch on it.
Add a validation suite that drives the terminal contract through the real
path: session and foreground-group invariants, Node and shell compatibility,
lifecycle bounds, the documented process-tree limits, PTY exhaustion,
terminal isolation on both backends, the child environment, and a detached run.
The new job builds and installs this checkout's wheel, then runs Node through
the installed execute_script(), so it needs neither Docker nor the API key.
Both existing jobs now set up Python 3.11, matching the project's pin.
Each poll now records every observable fact — target exit, expired deadline, set
cancellation, reader failure — before the rank table picks one, so racing conditions
are arbitrated instead of published in discovery order.
Any backend exception, not only TerminalProcessError, is classified as an environment
failure with its detail, and the launch failure is recorded before teardown so a
cleanup diagnostic follows it rather than replacing it.
Both backends publish a reader that outlives its join bound, the pipe reader publishes
read failures that happen while it is active, and the PTY drain suppresses only PTY EOF
and a closing backend's EBADF.
The raw transcript is written as a sibling of the published output file instead of an
orphaned temp file, and a cancelled run leaves no artifact at all.
Add the missing case for terminating a process group that was stopped mid-run, and
make the detached case launch its parent through the real nohup binary.
Survivors are now registered by pidfile the moment a process starts and swept with
bounded waits and escalation, so cleanup no longer depends on where a case failed.
Pin the backend read size to one byte in the fragmented-stream case, and judge the
renderer-death case on the signal that killed it and on a stable-quiet heartbeat window.
The repr highlighter interleaved ANSI codes inside logged text on
color-capable terminals, breaking the exactly-as-logged contract that
markup=False already promised. Disable highlighting on the same path.
Redirecting stdin to DEVNULL does not stop a Windows child from opening
CONIN$ on the console it inherited, so children are now created with
CREATE_NO_WINDOW and get a console of their own. The legacy backend also
honours a stop event that is already set before it launches anything.
The wrapper joined the render thread for 0.7 seconds, less than the SIGTERM
grace a script teardown runs to its end, so the CLI could exit mid-escalation
and leave a descendant alive. The wait is now the sum of the teardown budgets
the backend can spend, and a thread still running past it is reported.
The characterization harness documents the topology execute_script() produced
before this branch, not the one it produces now. available_backends() has no
callers and only leaked internal backend names into the public surface.
The characterization harness is a session leader whose controlling terminal
is the PTY it opens; Linux delivers SIGHUP when the master closes, killing it
before the report is written. Ignore the hangup — it is teardown noise.

test_terminal_process imported termios at module level, breaking collection
on Windows; move it behind the existing platform guard.
Parametrize lists reference pty_exec at import time, so the skipif mark
cannot save collection; skip the module before anything evaluates.
The arbiter fixture name must end in .ps1 on Windows or execute_script
rejects it before the injected backend is reached. The git-plumbing suites
skip on native Windows: files land on disk with CRLF and GitPython keeps
repository handles open, breaking diff assertions and temp-dir teardown.

The drain-bound test now waits for the escapee's flood before closing, so
the escapee is provably still writing while close() drains.
Every Repo in git_utils leaked its persistent git cat-file children; they
are only reaped by close(). On Windows a live child keeps the repository
directory undeletable, which is what broke temp-dir teardown — and on every
platform the children accumulate for the lifetime of a render. All repos
now close before returning; a closed Repo re-acquires resources lazily, so
the returned handles stay usable.

Tests write files with explicit LF (text mode writes CRLF on Windows, which
the diff assertions would see) and the fixtures collect test-held repos
before the temp directory is removed. The win32 skips are gone.
Marshaling for CreateProcessW (list2cmdline quoting, the double-NUL environment
block, and rejection of embedded NULs, '=' in a name and an empty name), the
bounded input queue with its reserved partition and control lane, and the writer
protocol that owns every write to a synchronous input pipe.

Split from the backend module, which binds kernel32 at import time, so these
rules run in every platform's test suite instead of only on a Windows runner.
One pseudoconsole behind the script's standard handles and one Job Object holding
its process tree, both attached through the same proc-thread attribute list so the
child is created inside the job or not at all. Ownership is incremental from before
the first allocation, and the ordered teardown runs above the rollback stack so a
session that outlives its bound can be handed to a daemon finalizer.

Windows selects it whenever CODEPLAIN_NO_PTY is not set; the escape hatch keeps
selecting the legacy pipe backend on both platforms. Builds below 17763 report an
environment error rather than downgrading to pipes.
The finalizer paths are exercised directly: deadline exhaustion asserting the job
handle closed before ClosePseudoConsole was attempted and the tree died with it, and
a finalizer that cannot start asserting the foreground kept ownership and released
the natives itself.

The query target now reads the reply it asked for, the saturation case samples the
writer's progress and asserts the cancel path ran when it is genuinely parked, the
script helper takes a stop event so cancellation is exercised through execute_script,
and failing WaitForSingleObject, GetExitCodeProcess, QueryInformationJobObject and
TerminateJobObject each assert the exit-69 mapping.
CreateProcessW copies the renderer's own standard handles into the child when they
are not console handles, so on a redirected renderer — every CI run — the target wrote
into the renderer's stdout and read end-of-file from its stdin while still attached to
the pseudoconsole and its job. It is now handed STARTF_USESTDHANDLES with none, and
only when the renderer is redirected, so the interactive path is untouched.

ClosePseudoConsole() also does not always end a parked ReadFile: a session that never
had a client left the reader waiting on a pipe whose write end was gone, and the
stalled thread then outlived the run. The reader opens a handle to itself at startup
and the bounded join cancels its read the way the writer's stop already does.
GetProcessHandleCount is not a leak detector for this backend: CPython allocates a
kernel semaphore per lock object, so the count moved for reasons the tests were not
about and every rollback case failed on it. A ledger of the handles the backend itself
opens and closes replaces it.

A pump left running now fails the test that leaked it rather than the next one, the
pseudoconsole failure case asks for a size Windows Server 2022 actually rejects, the
in-child probe declares the calls it makes instead of truncating handles to int, and a
target that cannot reach the transcript writes its report to a file so the next run
says what it saw.
A job handle created and then assigned by the caller, and a pseudoconsole armed one
statement after the call that made it, both leaked when anything came between: the
Windows run showed the job and the pseudoconsole outstanding after a failure injected
straight after those calls. Each now lands in its owner inside the call itself, as the
pipes, the attribute list and the process handles already did.

The last-resort release stops the input writer instead of assuming it is stuck. A
teardown that gave up at the job-empty wait never reached the stop, so the writer sat
parked on its queue — where only the stop sentinel can release it — and outlived the
run holding the input handle.
The probe exited 1 on Windows with nothing to say why, because the assertion that
caught it printed neither the transcript nor the report. It now takes no argument,
writes its findings beside itself, records a traceback if it raises at all, and every
assertion in the case carries both the transcript and that report.

The abandoned-session case expects the input handles released now that the writer is
stopped there, and the pseudoconsole failure case checks the slot is left unarmed.
The probe reached the runner with an indent no longer shared by every line, so dedent
removed nothing and the target died on line 2 before it could report anything. The
source is provably correct in the blob, so rather than explain the transformation the
program now sits at column zero and carries no escape sequence at all: neither dedent
nor an escape resolved too early has anything left to act on.

write_program compiles what it is about to write, so a malformed program fails the
test at the write with its own text and line, and the probe case reports the first
lines actually on disk beside the transcript and the literal.
Output accumulation, the reply-failure reporting, the cancellation check and
the owner constants were byte-identical in each backend; they now live on
TerminalProcess and in terminal_process/terminal_queries, and the backends keep
only their read loops. The terminal child environment and the reply-resolution
mapping become one shared function each.
Deleted: the ResizePseudoConsole declaration, the ConPTY backend's unread
input-driver field, InputQueue.accepting(), InputWriter.acknowledged_generation,
the responder's never-passed active flag, and the set mirroring the capped
failure list. IsProcessInJob stays: tests call it through _conpty.kernel32. The
repeated job and process-handle closes in _release_handles stay too, now with a
comment saying they are belt and braces.
Each backend now publishes the teardown budget its own constants add up to and
the CLI waits out the longest one reachable on this platform, so a ConPTY
teardown is no longer reported as a render that did not stop. The absent-input
note comes from the backend that ran rather than from sys.platform, which was
wrong under the escape hatch on Windows. The POSIX grace probes the process
group each tick and ends early once it is spent, and the precondition check asks
each repository once for the whole list of previous frids.
spawn() no longer carries pre_ack_delay through the handshake for the tests'
benefit. An empty _pre_ack_hook() sits where the delay ran, and the cases that
need the window open override it on a subclass, as they already do for the
select and master-descriptor seams.
Both tests need the grace loop to reach their injected tick; on a fast
runner the group can read as spent on the first probe and the loop breaks
before the tick fires. The tests' subject is the tick's effect, not
liveness, so the probe is pinned open.
close() no longer calls BufferedReader.close() under a reader parked in
read1(): the read end is redirected to devnull, which never blocks and
cannot recycle the descriptor under the parked read.
terminate_tree() watches the whole group through the grace with the
leader left unreaped, so descendants get their grace, a group that
empties is never SIGKILLed, and the escalation cannot reach a recycled
process group. The reap runs in its own finally so an interrupted grace
still collects the leader.
CAN and SUB abort an escape sequence, an ESC that does not begin ST
exits a control string, and a second ESC restarts the escape state.
An oversized control string is abandoned with its payload reclaimed as
plain output, and finalize() reclaims an unterminated string's payload.
Previously one truncated OSC silently swallowed all subsequent output,
including the failure text the patching loop depends on.
A broken conformance-tests repository raised a raw GitPython error
before the actionable missing-functionality error for the build repo
could be raised. The tests-repo query now propagates only when the
build repo has nothing missing.
A script exiting within the last poll interval of its budget was
published as a 124 timeout: the same poll recorded both the exit and
the expired deadline and the rank table let the timeout win.
A reply discarded by teardown itself after a clean exit converted
exit 0 into an environment error; the escalation now requires a
failing exit, since a passing one proves the reply did not matter.
The full teardown budget is waited out only while execute_script()
holds a terminal backend. Without one the render thread is typically
parked in an API call no cancellation reaches, so the CLI now gives up
after two seconds instead of holding the exiting user for the budget.
The shared note claimed a script waiting for input runs to the
timeout, which is wrong for the backends that hand end-of-file at
spawn and steered the patcher toward nonexistent stdin reads.
ConPTY, which cannot deliver an end-of-file, states its own note.
Both sides appended a job to the E2E workflow: main added
notify-on-failure, this branch added e2e-macos. Keep both, and let the
notification cover the macOS lane too — a lane that can fail silently is
worse than no lane.

Read each job result from an env var and fold the platform list in a
loop, so the step survives `bash -e`: the previous `[ cond ] && var=...`
form exits non-zero the moment a platform passed, aborting the step
before it could report the platforms that failed.
A pipe has no line discipline, so the bare linefeeds a target writes reached
the VT renderer untranslated: each line started in the column the previous
one ended in, and normalized_output() returned a whitespace staircase — the
text the fixing LLM receives under CODEPLAIN_NO_PTY=1.

The normalizer now optionally performs the ONLCR translation itself (NL ->
CR-NL on the raw byte stream, exactly as the PTY's line discipline does),
and the legacy pipe backend opts in.
…lain-tty Phase A)

The codeplain-tty plan's Phase A, client side: a typed version-1 capability
descriptor (protocol_version + the six broker commands), REST support for an
optional platform_test_runtime field on /render_conformance_tests,
/fix_conformance_tests_issue and /render_acceptance_tests, and the
advertisement gate. The gate returns None until the broker and executable
preflight ship, so no request advertises a runtime the client cannot provide
and the API stays on its backward-compatible path.
…ct (Phase B, POSIX)

The deferred language-neutral input driver from the codeplain-tty plan. A
per-execution broker (Unix-domain socket in a fresh 0700 directory, constant-
time token auth, length-framed versioned protocol, every wait and send
bounded) exposes the renderer-owned transcript and the terminal backend's
ordered input queue to a codeplain-tty helper the broker installs per
execution, so the same executable serves a source checkout, an editable
install, and a built wheel. The helper implements wait-for,
wait-until-absent, send-text (typing semantics: newlines become CR),
send-control, send-hex, and size, with exit codes matching the testing-script
conventions (69 = runtime unavailable).

Supporting backend changes: a typed TerminalInputDriver contract replaces the
Optional[object] marker (an attached driver suppresses the spawn-time VEOF,
which getpass/curses TCSAFLUSH would discard anyway); runtime resize lands on
the POSIX backend under the reader bundle's ownership lock (TIOCSWINSZ on the
master also raises SIGWINCH); the pipe backend resizes only its renderer; the
ConPTY backend states resize as unimplemented rather than lying; and the
no-input timeout note now describes the spawn-time EOF as the best effort it
is.

End-to-end tests drive real interactive targets on the PTY backend through
the installed helper: the motivating getpass/TCSAFLUSH reproduction, plain
input(), Ctrl-D as EOF, and a SIGWINCH-observed resize. Windows named-pipe
transport and ConPTY resize are tracked as pending; the capability is simply
not advertised there.
…lain-tty Phase C)

execute_script() gains an explicit platform_test_runtime option — never a
process-global switch. A runtime execution gets a per-execution broker, the
helper prepended to PATH from the broker's own bin directory, and a scoped
environment with caller-supplied CODEPLAIN_TTY_* values stripped before the
broker's own are added; a broker that cannot start is an environment error
(exit 69) and the script never runs. Unit-test and environment-preparation
executions are unchanged.

Only the conformance action asks for the runtime (acceptance tests extend and
execute the same suite), and only when the cached preflight passed: one real
round trip that starts a broker, installs the helper, and completes a
wait-for through the socket exactly the way a generated test will. The same
preflight gates the capability advertisement on the three conformance/
acceptance API calls, so the API is told about a runtime only after this
machine has proven it can provide it.

The acceptance-gate reproduction now passes through the real execution path:
a conformance-style script drives a getpass child (whose TCSAFLUSH discards
the spawn-time VEOF) through codeplain-tty instead of hanging to the
120-second timeout.
…y Phase E, client side)

The last net behind the API's response validation: at module completion,
before the build is copied anywhere, the build folder is walked and any
reference to codeplain-tty, the codeplain_tty module, or the CODEPLAIN_TTY_
environment prefix fails the render with a clear message. Vendor and build
directories are skipped; internal conformance/acceptance tests live outside
the build folder, so any hit is a violation by definition.
The transcript is rendered per line with trailing whitespace stripped, so a
needle quoting a prompt verbatim — 'Master password: ' — could never match as
written; every such wait burned its full timeout and stacked past the script
budget. Deterministically reproduced from the cli-password-manager failure in
the pty-with-codeplain-tty benchmark run (17 of the render's conformance
failures were 120-second timeouts): the generated tests drove the CLI through
codeplain-tty exactly as instructed, and hung only on the trailing space in
'wait-for "Master password: "'.

The broker now strips end-of-line whitespace from wait-for and
wait-until-absent needles before matching; a needle that is empty after the
normalization is a usage error.
The server's internal conformance-fix loop can spend its whole attempt budget
on one functionality; that outcome now arrives as a structured 400 (error_code
ConformanceTestsFixExhausted) instead of a raw 500. The client maps it to a
typed exception carrying the server's message, so the render fails with 'Could
not fix conformance tests issue ... Please review and rewrite the
specification' rather than an opaque HTTPError. Unknown future codes still
fall through unchanged.

Found via the bookshelf-api failure in pty-with-codeplain-tty-retry2: the
customers module's fix loop exhausted 10 server-side attempts, Flask 500'd,
and the client died with no explanation in codeplain.log.
A wait-for matched anywhere in the cumulative transcript, so the second of
two sequential interactive targets in one test file matched the first
target's stale prompt instantly, typed into a terminal nobody was reading
yet, had those bytes discarded by the new target's TCSAFLUSH, and hung the
whole script to its 120-second timeout. Deterministically reproduced with two
getpass children with realistic (Argon2id-like) delays between prompts —
exactly the cli-password-manager failure in pty-with-codeplain-tty-retry2
(FRID 2 exhausted; the trailing-space fix had removed the earlier failure
mode, leaving this one).

Each successful wait-for now consumes the transcript through its match, and
later waits (including wait-until-absent) look only beyond that cursor; the
transcript re-renders as the screen changes, so the cursor is clamped rather
than trusted exactly.
A/B arm ab-no-analyze-plus-tty-no-prompts gave up on python_tui at FRID 1
inside the unit-test fix loop and told the user only:

    ERROR codeplain: None

ExitWithError printed whatever payload the failing action handed over, and
actions reach that state by three routes: an encoded RenderError dict, a plain
string, or nothing at all. The last route printed "None"; the first printed a
raw dict repr, which is what the conformance-fix-exhausted path has been
showing all along:

    ERROR codeplain: {'error': {'message': "The renderer was unable to ...",
                                'type': None, 'details': None}}

Unwraps the reason at the log site and falls back to last_error_message, the
same message the returned payload already carried — so the line a user reads
is never less informative than the error the renderer returns.
…toring one

The conformance-fix loop reports exhaustion as a typed, actionable error
(105b215 server-side, d6777d1 client-side). The two client-side loops that can
also give up did not.

RenderFunctionalRequirement gives up when a functionality's unit tests still
fail after MAX_CODE_GENERATION_RETRIES re-renders. It set last_error_message
but returned no payload, which is the `ERROR codeplain: None` seen when
ab-no-analyze-plus-tty-no-prompts abandoned python_tui at FRID 1. It now returns
an encoded RenderError typed UNIT_TESTS_FIX_EXHAUSTED, carrying the frid and a
message that mirrors the conformance wording: what could not be produced, for
which functionality, and that the specification needs review.

RefactorCode already returned a payload, but built its message without an
f-string prefix, so the user was shown literal
`{MAX_REFACTORING_ITERATIONS}` and `{render_context.frid_context.frid}`.

No API change: unlike conformance, this loop is entirely client-side, so a REST
error code would be dead code.
Every loss in the retry5-era benchmarks was a fix loop spending its whole
budget re-patching one file against one unchanging failure: cli-password-manager
20 conformance attempts on vault_cli FRID 2, loglens the same on loglens_cli
FRID 2, python_tui 16+ unit-test attempts on FRID 1. The loop could not tell it
was stuck, and the only externally visible outcome was "did the render abort" —
a rare binary event, too coarse to compare configurations against.

Adds per-(module, FRID, loop) accounting for both loops. Each script run is
recorded with a fingerprint of its failure output, normalised past the tokens
that differ between two runs of the same failure (renderer temp paths,
durations, addresses) while leaving genuinely different failures apart.
Consecutive identical fingerprints are counted, and three in a row are reported
as they happen rather than at exhaustion.

Counts are emitted as a greppable line per FRID:

    [fix-loop] module=vault_cli frid=2 conformance=20 conformance_failed=20 max_repeat=20

on FRID completion, and for the whole render on both the completed and the
failed path — the FRID that exhausted its budget never reaches
FinishFunctionalRequirement, and its numbers are the ones worth having.

This is measurement, not behaviour: nothing here changes what the renderer
produces, so results stay comparable across the change. It makes
iterations-to-convergence observable per FRID, which is what a rare binary
outcome could not give: a near-continuous measure that says something after a
single run.

Switching strategy on detection (instrument/diagnose/delete rather than
re-patch) is deliberately not included — that would change renderer output and
break comparability with the runs in flight.
The exit summary reaches the terminal through Rich's print, which never touches
logging, so codeplain.log simply stopped at whatever happened to be logged last.
An artifact that ends mid-render is indistinguishable from a process that died
silently — originally noted for hycu's ConflictingRequirements abort, and it
just blocked a real diagnosis: cli-password-manager in
codeplain-tty-capability-run2 rendered to completion (22 functionalities,
48m54s) yet delivered a build with no CLI entry point, and the captured log
ended at 01:29:57 mid-render with no record of how it finished.

Adds a trailer written through the codeplain logger — so it lands in the log
file — on every exit path, since print_exit_summary is called from a finally:

    [render-trailer] outcome=completed render_id=... functionalities=22
        render_time_s=2934 generated_code=- spec=vault_cli...plain
    [render-trailer] error=<reason>          (failed renders only)

Handlers are flushed explicitly; a process exiting immediately after would
otherwise defeat the point of writing it.

The trailer doubles as a probe: because it is written unconditionally, a
captured log *without* one proves the file was truncated rather than merely
uninformative — which is the open question behind the missing entry point.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant