Split _subprocess_execution.py along its stated seams (#117) - #158
Conversation
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 8 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (24)
WalkthroughSplit subprocess execution into orchestration, stdin lifecycle, timeout handling, and observability modules. Update cancellation and timeout cleanup, add regression coverage, document module boundaries, package the new modules, and pin the Rust toolchain to 1.85.0. ChangesSubprocess execution refactor
Ancillary maintenance records
Sequence Diagram(s)sequenceDiagram
participant SafeCmd
participant _execute_subprocess
participant _spawn_subprocess
participant _spawn_stdin_writer
participant _wait_for_exit_code
participant _cancel_pending_consumers
SafeCmd->>_execute_subprocess: run command
_execute_subprocess->>_spawn_subprocess: spawn process and configure streams
_execute_subprocess->>_spawn_stdin_writer: start writer when stdin data exists
_execute_subprocess->>_wait_for_exit_code: await process exit
_wait_for_exit_code->>_cancel_pending_consumers: cancel and drain consumers
_execute_subprocess-->>SafeCmd: return result or propagate exception
Possibly related PRs
Suggested labels: Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 16❌ Failed checks (1 warning, 15 inconclusive)
✅ Passed checks (4 passed)
📋 Issue PlannerBuilt with CodeRabbit's Coding Plans for faster development and fewer bugs. View plan used: ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
7f9860a to
dde3af4
Compare
dde3af4 to
340b4a0
Compare
340b4a0 to
4b20e10
Compare
4b20e10 to
c837ebd
Compare
c837ebd to
9160949
Compare
|
@coderabbitai Have the following failed checks now been resolved? If further work is required, please provide an AI agent prompt for the remaining work to be done to address these failures. Do not treat warnings as optional or aspirational. Where a change is out of scope for this PR, propose a GitHub issue unless one exists already. (Treat documentation and validation coverage as in scope). Developer Documentation |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
echo '== PR changed files =='
gh pr view 158 --repo leynos/cuprum --json files --jq '.files[].path'
echo
echo '== Candidate documentation files =='
fd -a -t f -i 'developers-guide.md' -i 'cuprum-design.md' -i '*adr*' -i '*design*' docs .github 2>/dev/null || true
echo
echo '== Documentation references to subprocess module boundaries =='
rg -n -i -C 3 \
'subprocess_execution|subprocess_stdin|subprocess_timeout|stdin.*timeout|timeout.*stdin|architecture decision|ADR' \
docs 2>/dev/null || trueLength of output: 53513
PR
No further work, AI-agent prompt, or follow-up GitHub issue is required for this check. |
d060b82 to
0b66bd8
Compare
|
@coderabbitai resume |
✅ Action performedReviews resumed. |
0b66bd8 to
0727f90
Compare
|
@coderabbitai resume |
✅ Action performedReviews resumed. |
Address the third round of review feedback. - `_handle_stream_timeout` now delegates to the shared `_cancel_stdin_writer(stdin_task)` helper instead of duplicating the inline cancel-and-tolerant-gather logic, keeping the stdin-task lifecycle identical across the timeout and cancellation paths. - Debugging plan: re-wrap the H1 claim so every prose line stays within 80 columns, name the skipped `leta workspace add` indexing step in inline code (replacing the vague "workspace-indexing prerequisite" wording), and name the generated argument `ctx_tags` in the H1 ordering rationale so it explicitly identifies why H1 is the cheapest decisive check. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Complete the `_wait_for_exit_code` timeout-cleanup finding for the stream path. - Restore the `consumers` annotation to `tuple[asyncio.Task[None] | asyncio.Task[str | None], ...]` so the parameter accepts a blocking cleanup task as well as stdout/stderr readers. - On `TimeoutError`, cancel any consumer still pending after `_terminate_process` before draining it, so a reader wedged on a pipe that never reached EOF cannot make timeout handling hang. Finished readers keep their captured output. Factor the guarded cancel loop into `_cancel_pending_consumers` and reuse it from both the timeout and cancellation branches, keeping them consistent (cancelling an already-done task is a no-op, so behaviour is unchanged). - Add a focused regression test that drives `_wait_for_exit_code` into its timeout-cleanup branch with a blocking consumer and asserts the consumer is cancelled and drained while the original `TimeoutError` propagates. The direct-mode stdin handling and its blocked-writer regression test are unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Address the fifth round of review feedback (test-only). - Add `test_wait_for_exit_code_cancels_pending_consumers_on_cancellation`, which runs `_wait_for_exit_code` in a task, cancels it while a consumer is still pending, and asserts that `asyncio.CancelledError` propagates and the consumer is cancelled and drained. This covers the cancellation cleanup branch distinctly from the existing timeout test. - Give the `_TimeoutWaitProcess.wait` assertion a diagnostic message spelling out the process-double invariant (terminate/kill must record `returncode` before `_exited` is set); wait/return behaviour is unchanged. - Raise `ValueError` rather than `RuntimeError` from the property test's consumer helper when its outcome is "raise"; delay and return behaviour are unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The `_wait_for_exit_code` cancellation branch was only exercised at the unit level (a direct call with a fake process) and via direct-mode (capture=False) cancellation tests; no test cancelled an in-flight streamed run. Add `test_streamed_run_cancellation_cleans_up_task`, which cancels a running `command.run(output=RunOutputOptions(capture=True))` mid-flight. Output capture routes execution through `_run_subprocess_with_streams`, so this drives the `CancelledError` cleanup with real stdout/stderr consumer tasks and asserts the run tears down within a bounded time rather than deadlocking on a pending reader. It complements the unit test `test_wait_for_exit_code_cancels_pending_consumers_on_cancellation`, which asserts the precise consumer cancel/drain state. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Address the CodeScene "Lines of Code in a Single File" finding on `test_safe_cmd_run.py` with a cohesive test-module extraction. - Add `cuprum/unittests/test_safe_cmd_stdin.py` holding the stdin writer lifecycle regression group moved verbatim (assertions, commands, payload size, timeout values, and async/sync coverage unchanged): `test_stdin_input_with_timeout_escalation`, `test_direct_timeout_with_blocked_stdin_writer_does_not_hang`, and `test_stdin_input_cancellation_cleans_up_task`. The module carries only the imports and local helpers (`_execute_async`/`_execute_sync`, a local `python_builder` fixture) those tests need; `collections.abc` is imported under `TYPE_CHECKING` since it is annotation-only here. - Remove those tests from `test_safe_cmd_run.py`, retaining `test_streamed_run_cancellation_cleans_up_task` (stream-consumer cleanup via captured execution) and the shared runtime helpers/imports still used there. - Refresh the maturin wheel-build snapshot for the new module. No production subprocess code changed and no CodeScene suppression added; timeout, blocked-drain, cancellation, and execution-strategy coverage are preserved. `test_safe_cmd_run.py` drops from 635 to 565 non-blank, non-comment lines. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Address the seventh round of review feedback plus the failing Testing / Concurrency checks. Production: - Narrow the `consumers` annotation on `_cancel_pending_consumers` and `_wait_for_exit_code` to `tuple[asyncio.Task[str | None], ...]`, matching what `_spawn_stream_consumers` actually returns; stdin remains handled separately by `_cancel_stdin_writer`. - Merge the now-identical `except TimeoutError` and `except asyncio.CancelledError` branches of `_wait_for_exit_code` into a single `except (TimeoutError, asyncio.CancelledError)` clause; termination, consumer cancellation, tolerant gather, and bare re-raise are unchanged. - Replace the bare `RuntimeError` raised by `_require_timeout` with a package-scoped `_SubprocessInvariantError(RuntimeError)`, so the impossible- state guard is distinguishable from unrelated runtime failures while staying catchable as a `RuntimeError` (message and chaining preserved). Tests: - Cancellation regressions now assert real propagation with `pytest.raises(asyncio.CancelledError)` and `task.cancelled()` instead of suppressing `CancelledError`, in both the stdin (`run()`) and streamed-run tests, so they fail if `run()` ever swallows cancellation. - The stdin cancellation test now uses a child that never reads stdin plus a 1 MiB payload, so it exercises cleanup of a writer genuinely blocked in `drain()`. - The observe `stdin_error` test provokes a real EPIPE (child closes stdin + 1 MiB payload) instead of monkeypatching `_write_stdin`. - Type `_execute_async`/`_execute_sync` kwargs with a `_RunKwargs` TypedDict (no `Any`) and narrow `execution_strategy` to `Literal["async", "sync"]`; correct the module docstring (timeouts cover both strategies, cancellation only `run()`). - `fail_consumer` and the two `blocking_consumer` doubles are retyped/retargeted (`ValueError`; `-> str | None`) to match the above. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The rebase onto main (CQRS refactor #118) left `from pathlib import Path` ahead of the plain `import` statements in test_safe_cmd_run.py after the stdin-test-removal conflict resolution. Reorder it below the stdlib imports to satisfy ruff's isort rule. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The rebase onto main (#116/#157) introduced ADR-006 "Split cuprum/context.py into a context package", colliding with this branch's ADR-006 "Subprocess execution module boundaries". Main owns 006, so renumber the subprocess ADR to 007: rename the file and update every reference (contents.md, cuprum-design.md, developers-guide.md, and the ADR title). Both ADRs are retained. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…#117) Address review feedback on the subprocess execution split: - _run_subprocess_with_streams: if `await stdin_task` raises an unexpected exception (or a cancellation lands on that await), cancel and drain the stdout/stderr consumer tasks before the error propagates, mirroring the timeout and cancellation cleanup paths, so the consumers are not abandoned. - Replace fixed-`asyncio.sleep` synchronisation hacks with deterministic readiness signals before cancellation: - test_streamed_run_cancellation_cleans_up_task waits for an observed stdout line via an observe hook. - test_stdin_input_cancellation_cleans_up_task waits for the stdin writer to begin (and wedge in drain) via a wrapped `_write_stdin`. - test_wait_for_exit_code_cancels_pending_consumers_on_cancellation awaits a new `wait_started` event on the `_TimeoutWaitProcess` double. - test_observe_emits_stdin_error_event_when_process_closes_stdin_early sizes its stdin payload from the probed pipe capacity so drain() is guaranteed to block, replacing the assumed ~64 KiB pipe-buffer race. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The H1 plan's Table 2 and prediction claimed a bounded, representative `_TAGS` timing measurement, but the documented tooling only replays the whole property test (generation plus observation construction) with no isolated sampling harness or acceptance threshold, and no such harness exists in the repo. Remove the unsupported "time representative samples" step and reconcile the prediction so the documented falsification is reproducible as written. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…117) Add a regression test for the streamed (capture) path: inject a failing stdin writer so `await stdin_task` raises, and assert the stdout/stderr consumer tasks are cancelled and drained rather than orphaned. The assertion runs inside the running loop (before asyncio.run tears it down and cancels leftovers) so it genuinely fails without the reconcile fix, closing the coverage gap flagged in review for the fix in 10d28c2. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
test_safe_cmd_run.py was 828 lines, over the 400-line CodeScene "Lines of Code in a Single File" limit. Move tests into focused modules by concern, without changing any test behaviour, assertions, names, or async/sync coverage (the collected node-id set is unchanged): - test_safe_cmd_stdin.py (extended): stdin injection tests (text/bytes feeding, configured encoding, capture-disabled, early-close, and the forbidden-command vs stdin-encoding ordering contract) alongside the existing stdin writer lifecycle regressions. Adds a local `execution_strategy` fixture. - test_safe_cmd_context.py (new): allowlist enforcement and before/after hook integration (FIFO/LIFO order, hook arguments, cancellation skipping after hooks). - test_safe_cmd_streams.py (new): captured-stream (`capture=True`) cleanup on cancellation and stdin-writer failure, preserving the monkeypatch targets. - test_safe_cmd_run.py (slimmed to 366 lines): general execution/output/env/ cwd/timeout coverage plus the non-cooperative-kill escalation test. Regenerate the native-wheel snapshot to package the two new test modules. No production code changed; no CodeScene suppression added. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Resolve the CodeScene duplication finding in test_safe_cmd_stdin.py: the
near-identical test_input_text_feeds_stdin and test_input_bytes_feeds_raw_stdin
shared their whole control flow, differing only in the child script, the
StdinInput payload, and the expected stdout. Replace them with a single
parametrized test_input_feeds_stdin ("text" and "raw-bytes" cases), preserving
both payloads, scripts, expected output, and the run()/run_sync() coverage via
the existing execution_strategy fixture. test_input_text_uses_configured_encoding
stays a separate test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The coverage job pinned Rust 1.92.0 while the MSRV pin (rust-toolchain.toml), the lint-test job, and the typecheck-test job all use 1.85.0. Build coverage with the same toolchain as the rest of CI so the native extension and test suite are exercised under the MSRV. Only the toolchain version changes; the setup-rust action pin and all other CI configuration are unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#117) - _subprocess_stdin._emit_stdin_error: record the failing exception's traceback via `_LOGGER.error(..., exc_info=exc)` and drop the now-redundant inline exception format argument, preserving the context fields and the `stdin_error` observation event. (Ruff LOG004 forbids `.exception()` in this standalone helper, so the explicit `exc_info=exc` form is used.) Update the `test_write_stdin_observes_error_events` log assertion to check `caplog.text`, since the exception detail now lives in the traceback rather than the message. - Add `tests/helpers/execution.py` with the shared `_RunKwargs` TypedDict and `ExecuteFn` alias; import them in test_safe_cmd_stdin.py and test_safe_cmd_context.py instead of duplicating the definitions. - test_safe_cmd_stdin.py: parameterise `test_input_feeds_stdin` with a frozen `_StdinFeedCase` dataclass (3 params); drop the unused `python_builder`/ `execution_strategy` fixtures from `test_input_text_and_input_bytes_conflict`; switch the four string-parametrized tests to the `execution_strategy` fixture; add diagnostic messages to the bare assertions. - test_safe_cmd_context.py: add diagnostic messages to every bare assertion. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- contents.md: add the missing ADR-005 and ADR-006 entries to the ADR index, between ADR-004 and ADR-007, in the established reference-link format. - cuprum-design.md: move the "8.1.5 Subprocess execution module boundaries" subsection to after Figure 3's mermaid block so the figure caption and diagram stay together and 8.1.5 closes section 8.1. - developers-guide.md: replace the duplicated lifecycle-boundary preface with a pointer to cuprum-design.md §8.1.5 and ADR-007, and move the maintainer placement guidance under its own "Subprocess execution module boundaries" heading. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The `stdin_error` transition emitted by `_emit_stdin_error` was surfaced to metrics (a counter) and logs, but the tracing hook grouped it with `plan`/`stdin` and ignored it, so a stdin write/close failure left no trace, and the event carried no stable operation/error-type fields (operation was dropped entirely; error type was only embedded in the free-text `note`). - events.py / _pipeline_types.py: add optional `operation` and `error_type` fields to the public `ExecEvent` and internal `_EventDetails`, wired through `_StageObservation.emit`. - _subprocess_stdin._emit_stdin_error: populate `operation` (write/close) and `error_type` on the emitted `stdin_error` event. (Logging already records the exception traceback via `exc_info` and these fields via `extra`.) - tracing_adapter: record `stdin_error` as a `cuprum.stdin_error` span event (correlated by exec_id) carrying operation/error_type/note, leaving the span open and unmarked since the failure is non-fatal. Consolidate the near-identical output and stdin-error span-event handlers into one `_record_span_event`, keeping the module under the 400-line limit. - Tests: update the stdin_error observe expectations, extend the shared event factory to forward the new fields, and add a tracing regression test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
901ae4e to
b328f2c
Compare
…117) The tracing adapter recently consolidated its two near-identical span-event handlers into one `_record_span_event` and added a `stdin_error` phase, leaving the developers' guide out of date. Update the "Tracing adapter span lifecycle" section to standardise and document the patterns: - Phase-dispatch policy: every ExecEvent phase falls into one of four categories (span lifecycle, span event, deliberately ignored, unhandled); new phases slot into this policy rather than an ad-hoc side path. - One span-event recorder: stdout/stderr/stdin_error all route through `_record_span_event`, which copies whichever of line/operation/error_type/note are set onto a `cuprum.<phase>` event; new recording phases extend the shared field set instead of adding a bespoke method. - Non-fatal events (stdin_error) are recorded but leave the span open and unmarked; only `exit` ends the span. - `record_output` gates stdout/stderr but not stdin_error, so a stdin failure stays diagnosable when line output recording is off. Docs-only; no code change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 4
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
cuprum/_subprocess_execution.py (1)
175-198: 🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy liftKeep stream consumers owned by the timeout path.
_wait_for_exit_code(..., consumers=consumers)drains the consumer tasks onTimeoutError, then_handle_stream_timeout()gathers those same tasks again and maps the cancelled results toNone, which drops captured stdout/stderr fromTimeoutExpired. Move consumer cancellation/drain into the timeout branch here and let_wait_for_exit_code()wait for the exit code only.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@cuprum/_subprocess_execution.py` around lines 175 - 198, Move stream-consumer cancellation and draining out of _wait_for_exit_code and into the TimeoutError branch of _run_subprocess_with_streams, ensuring consumers are drained exactly once before _handle_stream_timeout processes them. Update _wait_for_exit_code to wait only for the process exit code, and preserve captured stdout/stderr when constructing TimeoutExpired.cuprum/unittests/test_observe.py (1)
333-376: 🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy liftPipe-capacity race still unresolved.
This is the outstanding finding already recorded in the PR summary: the test still sizes the payload from
_drain_blocking_payload_size()but never synchronizes the child'ssys.stdin.close()with the writer'sdrain(). Correctness depends entirely on the child closing stdin before the parent's buffered write exhausts the probed capacity — nothing here guarantees that ordering. Deterministic synchronization (a control channel/marker the child signals after closing stdin, or a test seam that lets the harness confirm the writer has actually blocked before the child proceeds) is still needed rather than relying on size alone.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@cuprum/unittests/test_observe.py` around lines 333 - 376, Make test_observe_emits_stdin_error_event_when_process_closes_stdin_early deterministic by synchronizing the child’s sys.stdin.close() with the parent writer reaching its blocked drain state. Add a control channel or explicit marker/seam so the child signals after closing stdin and the harness confirms the writer is blocked before allowing the child to proceed; do not rely solely on _drain_blocking_payload_size().
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@cuprum/unittests/test_observe.py`:
- Around line 314-330: Move _drain_blocking_payload_size from
cuprum/unittests/test_observe.py lines 314-330 into the shared test helper
module, preserving its dynamic pipe-capacity probing. In
cuprum/unittests/test_safe_cmd_stdin.py lines 261-289 and 292-347, import and
use this helper for the stdin payload in
test_direct_timeout_with_blocked_stdin_writer_does_not_hang and
test_stdin_input_cancellation_cleans_up_task, replacing both hardcoded 1 MiB
allocations.
- Around line 6-17: Update test_observe.py so the pipe-capacity logic using
fcntl and F_GETPIPE_SZ is only imported and executed on supported platforms,
allowing test collection elsewhere. Replace the payload-size-based early-close
trigger with an explicit synchronization seam that deterministically coordinates
the close path, while preserving the existing test behavior and assertions.
In `@cuprum/unittests/test_safe_cmd_context.py`:
- Around line 191-229: Update
test_run_does_not_invoke_after_hooks_on_cancellation to replace the fixed
asyncio.sleep readiness guess with the established sh.observe start-phase hook
and an asyncio.Event. Await the event before calling task.cancel(), ensuring
cancellation occurs only after the subprocess has actually started while
preserving the existing cancellation and after-hook assertions.
In `@docs/debugging/debugging-plan-2026-07-18-hypothesis-too-slow.md`:
- Around line 39-41: Insert a comma after “0.098 seconds” in the sentence
beginning “Sampling 100 `_TAGS` examples” to separate the two independent
clauses, while leaving the surrounding investigation results unchanged.
---
Outside diff comments:
In `@cuprum/_subprocess_execution.py`:
- Around line 175-198: Move stream-consumer cancellation and draining out of
_wait_for_exit_code and into the TimeoutError branch of
_run_subprocess_with_streams, ensuring consumers are drained exactly once before
_handle_stream_timeout processes them. Update _wait_for_exit_code to wait only
for the process exit code, and preserve captured stdout/stderr when constructing
TimeoutExpired.
In `@cuprum/unittests/test_observe.py`:
- Around line 333-376: Make
test_observe_emits_stdin_error_event_when_process_closes_stdin_early
deterministic by synchronizing the child’s sys.stdin.close() with the parent
writer reaching its blocked drain state. Add a control channel or explicit
marker/seam so the child signals after closing stdin and the harness confirms
the writer is blocked before allowing the child to proceed; do not rely solely
on _drain_blocking_payload_size().
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 3885c852-f415-4619-805c-2de8f9cc1c88
📒 Files selected for processing (24)
.github/workflows/ci.ymlcuprum/_pipeline_types.pycuprum/_subprocess_execution.pycuprum/_subprocess_stdin.pycuprum/_subprocess_timeout.pycuprum/adapters/tracing_adapter.pycuprum/events.pycuprum/sh.pycuprum/unittests/__snapshots__/test_maturin_build.ambrcuprum/unittests/_adapter_test_support.pycuprum/unittests/test_observe.pycuprum/unittests/test_safe_cmd_context.pycuprum/unittests/test_safe_cmd_run.pycuprum/unittests/test_safe_cmd_stdin.pycuprum/unittests/test_safe_cmd_streams.pycuprum/unittests/test_subprocess_timeout.pycuprum/unittests/test_tracing_adapter.pydocs/adr-007-subprocess-execution-module-boundaries.mddocs/contents.mddocs/cuprum-design.mddocs/debugging/debugging-plan-2026-07-18-hypothesis-too-slow.mddocs/developers-guide.mdrust/rust-toolchain.tomltests/helpers/execution.py
🔗 Linked repositories identified
CodeRabbit considers these linked repositories for cross-repo context during reviews:
leynos/shared-actions(auto-detected)leynos/pylint-pypy-shim(auto-detected)leynos/whitaker(auto-detected)
| import fcntl | ||
| import logging | ||
| import os | ||
| import sys | ||
| import typing as typ | ||
| from pathlib import Path | ||
|
|
||
| import pytest | ||
|
|
||
| from cuprum import _subprocess_execution, sh | ||
| from cuprum import sh | ||
| from cuprum._pipeline_internals import _EventDetails | ||
| from cuprum._subprocess_execution import _write_stdin | ||
| from cuprum._subprocess_stdin import _write_stdin |
There was a problem hiding this comment.
🩺 Stability & Availability | 🔴 Critical | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== test_observe outline ==\n'
ast-grep outline cuprum/unittests/test_observe.py --view expanded || true
printf '\n== relevant lines ==\n'
sed -n '1,120p' cuprum/unittests/test_observe.py
printf '\n---\n'
sed -n '300,470p' cuprum/unittests/test_observe.py
printf '\n== sibling platform skip patterns ==\n'
rg -n "win32|skip\\(" cuprum/unittests/test_safe_cmd_run.py cuprum/unittests -g 'test_*.py' || trueRepository: leynos/cuprum
Length of output: 11230
🏁 Script executed (no clone):
#!/bin/bash
python3 - <<'PY'
import sys
print(sys.platform)
PY
Length of output: 131
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf 'FILE EXISTS?\n'
test -f cuprum/unittests/test_observe.py && echo yes
printf '\nLINES AROUND IMPORTS\n'
nl -ba cuprum/unittests/test_observe.py | sed -n '1,40p'
printf '\nLINES AROUND HELPER AND TEST\n'
nl -ba cuprum/unittests/test_observe.py | sed -n '300,390p'
printf '\nPLATFORM SKIPS IN UNIT TESTS\n'
rg -n "win32|sys\.platform|pytest\.skip" cuprum/unittests/test_*.py || trueRepository: leynos/cuprum
Length of output: 230
Guard the pipe-capacity probe behind a platform check.
- Move
fcntlbehind a lazy import or platform guard; the module-level import andF_GETPIPE_SZcall make this test Linux-only and can break collection on other platforms. - Replace the payload-size heuristic with an explicit synchronisation seam for the early-close path.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@cuprum/unittests/test_observe.py` around lines 6 - 17, Update test_observe.py
so the pipe-capacity logic using fcntl and F_GETPIPE_SZ is only imported and
executed on supported platforms, allowing test collection elsewhere. Replace the
payload-size-based early-close trigger with an explicit synchronization seam
that deterministically coordinates the close path, while preserving the existing
test behavior and assertions.
| def _drain_blocking_payload_size() -> int: | ||
| """Return a stdin payload size that reliably blocks the writer's drain(). | ||
|
|
||
| The early-close ``stdin_error`` only surfaces when ``drain()`` actually | ||
| blocks, which requires a payload larger than the OS pipe buffer plus the | ||
| asyncio transport write high-water mark. Probe the real pipe capacity so | ||
| the assertion does not rely on an assumed ~64 KiB buffer. | ||
| """ | ||
| read_fd, write_fd = os.pipe() | ||
| try: | ||
| pipe_capacity = fcntl.fcntl(write_fd, fcntl.F_GETPIPE_SZ) | ||
| finally: | ||
| os.close(read_fd) | ||
| os.close(write_fd) | ||
| # Exceed the probed pipe capacity and the default 64 KiB transport | ||
| # high-water mark with a full extra MiB of headroom. | ||
| return pipe_capacity + (1 << 20) |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win
Extract the pipe-blocking payload sizing into one shared helper. test_observe.py computes a reliable "drain-blocking" payload size by probing the real OS pipe capacity, while test_safe_cmd_stdin.py duplicates the same intent twice with a hardcoded 1 MiB literal and a comment assuming a "~64 KiB pipe buffer". The dynamic probe is strictly more correct; reuse it everywhere instead of letting the assumption drift.
cuprum/unittests/test_observe.py#L314-L330: move_drain_blocking_payload_size()into a shared test helper module (e.g.tests/helpers/execution.py, which already hosts_RunKwargs/ExecuteFn).cuprum/unittests/test_safe_cmd_stdin.py#L261-L289: replace theb"x" * (1024 * 1024)literal intest_direct_timeout_with_blocked_stdin_writer_does_not_hangwith the shared helper's computed size.cuprum/unittests/test_safe_cmd_stdin.py#L292-L347: replace theb"x" * (1024 * 1024)literal intest_stdin_input_cancellation_cleans_up_taskwith the shared helper's computed size.
📍 Affects 2 files
cuprum/unittests/test_observe.py#L314-L330(this comment)cuprum/unittests/test_safe_cmd_stdin.py#L261-L289cuprum/unittests/test_safe_cmd_stdin.py#L292-L347
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@cuprum/unittests/test_observe.py` around lines 314 - 330, Move
_drain_blocking_payload_size from cuprum/unittests/test_observe.py lines 314-330
into the shared test helper module, preserving its dynamic pipe-capacity
probing. In cuprum/unittests/test_safe_cmd_stdin.py lines 261-289 and 292-347,
import and use this helper for the stdin payload in
test_direct_timeout_with_blocked_stdin_writer_does_not_hang and
test_stdin_input_cancellation_cleans_up_task, replacing both hardcoded 1 MiB
allocations.
| def test_run_does_not_invoke_after_hooks_on_cancellation( | ||
| python_builder: cabc.Callable[..., SafeCmd], | ||
| ) -> None: | ||
| """run() does not invoke after hooks when task is cancelled.""" | ||
| from cuprum.context import ScopeConfig, scoped | ||
|
|
||
| after_called = False | ||
|
|
||
| def after_hook(cmd: SafeCmd, result: CommandResult) -> None: | ||
| """Record that the after hook was invoked.""" | ||
| nonlocal after_called | ||
| _, _ = cmd, result | ||
| after_called = True | ||
|
|
||
| # Use a long-running command that we can cancel | ||
| command = python_builder("-c", "import time; time.sleep(10)") | ||
|
|
||
| async def orchestrate() -> None: | ||
| """Start the command under a scope, then cancel it.""" | ||
| with scoped( | ||
| ScopeConfig( | ||
| allowlist=frozenset([command.program]), after_hooks=(after_hook,) | ||
| ) | ||
| ): | ||
| task = asyncio.create_task( | ||
| command.run( | ||
| output=RunOutputOptions(capture=False), | ||
| context=ExecutionContext(cancel_grace=0.1), | ||
| ), | ||
| ) | ||
| await asyncio.sleep(0.1) # Let the process start | ||
| task.cancel() | ||
| with pytest.raises(asyncio.CancelledError): | ||
| await task | ||
|
|
||
| asyncio.run(orchestrate()) | ||
| assert after_called is False, ( | ||
| "after hooks must not run when the command is cancelled" | ||
| ) |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Replace the fixed sleep with a readiness signal before cancelling.
await asyncio.sleep(0.1) # Let the process start is a scheduler guess, not proof the subprocess has actually started before task.cancel() fires. Under CI load this can cancel before the process is even spawned, silently defeating the regression this test targets. Sibling files in this same PR (test_safe_cmd_streams.py) already fix the identical pattern with an sh.observe hook that sets an asyncio.Event on the "start" phase.
🔧 Proposed fix using the established readiness-signal pattern
+ from cuprum.events import ExecEvent
+
command = python_builder("-c", "import time; time.sleep(10)")
+ started = asyncio.Event()
+
+ def on_start(ev: ExecEvent) -> None:
+ """Signal once the subprocess has actually started."""
+ if ev.phase == "start":
+ started.set()
async def orchestrate() -> None:
"""Start the command under a scope, then cancel it."""
- with scoped(
- ScopeConfig(
- allowlist=frozenset([command.program]), after_hooks=(after_hook,)
- )
- ):
+ with (
+ scoped(
+ ScopeConfig(
+ allowlist=frozenset([command.program]), after_hooks=(after_hook,)
+ )
+ ),
+ sh.observe(on_start),
+ ):
task = asyncio.create_task(
command.run(
output=RunOutputOptions(capture=False),
context=ExecutionContext(cancel_grace=0.1),
),
)
- await asyncio.sleep(0.1) # Let the process start
+ await asyncio.wait_for(started.wait(), timeout=2.0)
task.cancel()
with pytest.raises(asyncio.CancelledError):
await taskAs per coding guidelines, "Changes involving shared state, asynchronous execution, ordering, cancellation, locks, transactions, task lifetimes, or parallelism must make the concurrency model explicit and test failure-prone interleavings."
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| def test_run_does_not_invoke_after_hooks_on_cancellation( | |
| python_builder: cabc.Callable[..., SafeCmd], | |
| ) -> None: | |
| """run() does not invoke after hooks when task is cancelled.""" | |
| from cuprum.context import ScopeConfig, scoped | |
| after_called = False | |
| def after_hook(cmd: SafeCmd, result: CommandResult) -> None: | |
| """Record that the after hook was invoked.""" | |
| nonlocal after_called | |
| _, _ = cmd, result | |
| after_called = True | |
| # Use a long-running command that we can cancel | |
| command = python_builder("-c", "import time; time.sleep(10)") | |
| async def orchestrate() -> None: | |
| """Start the command under a scope, then cancel it.""" | |
| with scoped( | |
| ScopeConfig( | |
| allowlist=frozenset([command.program]), after_hooks=(after_hook,) | |
| ) | |
| ): | |
| task = asyncio.create_task( | |
| command.run( | |
| output=RunOutputOptions(capture=False), | |
| context=ExecutionContext(cancel_grace=0.1), | |
| ), | |
| ) | |
| await asyncio.sleep(0.1) # Let the process start | |
| task.cancel() | |
| with pytest.raises(asyncio.CancelledError): | |
| await task | |
| asyncio.run(orchestrate()) | |
| assert after_called is False, ( | |
| "after hooks must not run when the command is cancelled" | |
| ) | |
| def test_run_does_not_invoke_after_hooks_on_cancellation( | |
| python_builder: cabc.Callable[..., SafeCmd], | |
| ) -> None: | |
| """run() does not invoke after hooks when task is cancelled.""" | |
| from cuprum.context import ScopeConfig, scoped | |
| from cuprum.events import ExecEvent | |
| after_called = False | |
| started = asyncio.Event() | |
| def after_hook(cmd: SafeCmd, result: CommandResult) -> None: | |
| """Record that the after hook was invoked.""" | |
| nonlocal after_called | |
| _, _ = cmd, result | |
| after_called = True | |
| def on_start(ev: ExecEvent) -> None: | |
| """Signal once the subprocess has actually started.""" | |
| if ev.phase == "start": | |
| started.set() | |
| command = python_builder("-c", "import time; time.sleep(10)") | |
| async def orchestrate() -> None: | |
| """Start the command under a scope, then cancel it.""" | |
| with ( | |
| scoped( | |
| ScopeConfig( | |
| allowlist=frozenset([command.program]), after_hooks=(after_hook,) | |
| ) | |
| ), | |
| sh.observe(on_start), | |
| ): | |
| task = asyncio.create_task( | |
| command.run( | |
| output=RunOutputOptions(capture=False), | |
| context=ExecutionContext(cancel_grace=0.1), | |
| ), | |
| ) | |
| await asyncio.wait_for(started.wait(), timeout=2.0) | |
| task.cancel() | |
| with pytest.raises(asyncio.CancelledError): | |
| await task | |
| asyncio.run(orchestrate()) | |
| assert after_called is False, ( | |
| "after hooks must not run when the command is cancelled" | |
| ) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@cuprum/unittests/test_safe_cmd_context.py` around lines 191 - 229, Update
test_run_does_not_invoke_after_hooks_on_cancellation to replace the fixed
asyncio.sleep readiness guess with the established sh.observe start-phase hook
and an asyncio.Event. Await the event before calling task.cancel(), ensuring
cancellation occurs only after the subprocess has actually started while
preserving the existing cancellation and after-hook assertions.
Source: Coding guidelines
Summary
This branch splits the 510-line
cuprum/_subprocess_execution.pyalong theseams its own TODO named, removing the
too-many-linessuppression pragmainstead of carrying it.
Closes #117.
_emit_stdin_error,_write_stdin,_spawn_stdin_writer; thecuprum.stdinlogger now lives in the module whose name matches it.match/casebinding while retaining theTimeoutErrorfallback._execute_subprocess,_run_subprocess_with_streams,_spawn_subprocess, stream-consumer wiring).The redundant
_resolve_timeoutre-export is dropped from__all__;cuprum/sh.pyimports it fromcuprum._subprocess_context.Review walkthrough
_require_timeoutconsolidation, and the requested structural timeout match.
_write_stdinat its new home.and ADR-005; the unrelated duplicate ExecPlan artefact section was removed.
Validation
make check-fmt: passmake lint: passmake typecheck: passmake test: pass (723 passed, 50 skipped; Rust nextest 4/4 passed)make markdownlint: passmake nixie: passNotes
Related: #75 tracks property
tests for
_handle_subprocess_timeout, which now has a single, separatelyimportable home. The wheel-build snapshot reflects the new file list. The
observed Hypothesis health-check failure was replayed successfully with its
supplied seed, so no suppression or speculative test change was made.
References