Skip to content

test(extensions): pin what a capped provider run returns and stops - #4924

Merged
huangruiteng merged 1 commit into
loopx-project:mainfrom
hahahahahayesyeseys:codex/test-capped-process-runtime
Sep 23, 2026
Merged

huangruiteng merged 1 commit into
loopx-project:mainfrom
hahahahahayesyeseys:codex/test-capped-process-runtime

Conversation

@hahahahahayesyeseys

Copy link
Copy Markdown
Contributor

Goal And Delivered Outcome

  • Goal/source and gap: loopx/extensions/process_runtime.py::run_capped_process is the guard that keeps an
    extension provider from (a) handing back an unbounded response and (b) leaving its process group running.
    The file's only two tests both drove the overflow/timeout path and asserted that descendants died. Nothing
    pinned what a caller actually reads back: that a normal response survives intact, that the capture really
    stays bounded against a writer that never stops, that a provider exiting nonzero is not a runtime failure,
    and that a stderr flood is not reported as a stdout flood.
  • Observable before → after, with the decisive row: drop the output bound in the product code
    (destination.extend(chunk[:remaining]) → destination.extend(chunk)) and the whole existing suite stays
    green; with this PR the same mutation is caught by exactly one test. Same for the other three mutations
    below. Validation row regression_parity carries the table.
  • Issue/task and intended base: none; this is a focused coverage gap in the shipped extension runtime.
    Related to PR test(lark): pin the manager context window's bounds and order #4900 (same shape: pin a bounded surface that had no success-path test).

Scope And Continuation

  • Completed scope and remaining work: five tests added to the existing module-owned test file; no product code
    changed. Deliberately not covered here, and named so the next person does not re-derive them: the
    Windows taskkill tree-kill branch and the unsupported-platform fallback (both already marked
    # pragma: no cover - exercised on Windows hosts), env/cwd pass-through, and which failure_kind wins
    when stdout and stderr overflow at the same time — record_limit keeps the first kind under a lock, and the
    race is not deterministic enough to pin honestly from a unit test.
  • Slice boundary / successor: complete within this scope. A successor would be a Windows-host run of this
    file, which needs a runner rather than a test change.

Validation

  • Tested revision: f5db9ad46
  • Run state: finished
  • Input classes: synthetic
Check kind Result Public-safe evidence / limitation
unit passed python -m pytest tests/extensions/test_process_runtime.py → 7 passed in 2.6s, repeated 3× for flake; python -W error::SyntaxWarning clean. The five additions cover: response returned byte-for-byte (including a trailing newline and non-ASCII payload via stdin echo), provider exit(3) reported as returncode=3 with failure_kind=None, a response of exactly output_limit_bytes accepted, a never-ending 64 KiB-per-round writer refused with len(stdout) == limit + 1 and under a wall-clock guard, and a 256 KiB stderr flood reported as stderr_too_large with stdout == b"".
regression_parity passed Five mutations applied one at a time to loopx/extensions/process_runtime.py, each verified to have modified the file, then reverted: (A) swap the two overflow_kind values → 3 failed (test_a_spewing_provider_on_stderr_is_its_own_failure, test_a_spewing_provider_is_stopped_instead_of_drained, pre-existing test_output_overflow_terminates_provider_descendants); (B) extend with the whole chunk → 1 failed, the new spew test only; (D) report nonzero provider exit as a failure → 1 failed, the new declining-provider test only; (E) drop the +1 in the remaining-bytes arithmetic → 3 failed; (C) remove the return after record_limit → 7 passed, not caught. (C) was then read through and is an equivalent mutant: the reader keeps draining a closed pipe until EOF, the captured length stays bounded and the main loop still kills on the same event, so no caller-visible outcome changes. Reported as uncaught rather than papered over.
unit (regression around the change) passed python -m pytest tests/extensions tests/canary → 978 passed in 80s.
static passed ruff check tests/extensions/test_process_runtime.py clean; the file was ruff format --check clean before this change and stays clean (one of my own long lines was reformatted, no other file touched); git diff --check clean; loopx check --scan-path tests/extensions/test_process_runtime.py → public boundary scan clean.
real_backend / integration not_applicable No product code, schema, receipt or state contract changed; the tests spawn real OS subprocesses via the shipped function, so no mock substitute stands in for the behavior under test.
  • Coverage and gaps: every added test drives the real run_capped_process against a real child process, so the
    pins land on the caller-observable tuple (returncode, stdout, failure_kind) rather than on internals.
    Remaining untested paths are the two platform branches and the concurrent-overflow race named above; the
    POSIX process-group behavior itself was already covered and is unchanged.

Frontend / Visual Evidence

  • UI impact: none
  • Before:
  • After:
  • States and viewports shown:
  • Source data: none
  • Attention review: N/A, tests only.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Refactoring (no functional changes)
  • Documentation update
  • Test update

LoopX Area

  • Control plane (goals, todos, quota, scheduler, registry, runtime)
  • Benchmark boundary (adapters, runners, verifiers, scoring, evidence)
  • Capability or extension (providers, adapters, skills)
  • Public docs or presentation surface (README, protocols, dashboard)
  • Build, packaging, installer, or CI
  • Host or runtime integration

Technical Direction

  • Direction / acceptance reference, when applicable: N/A.

Shared-authority RFC fixture impact

  • N/A; no migration or shared-authority claim.

Boundary Checklist

  • Neither the diff nor this PR body/comments/attachments disclose private state, credentials, raw traces
    or verifier output, internal links, or local machine paths (including .loopx/, .codex/goals/, and live
    ACTIVE_GOAL_STATE.md).
  • I did not duplicate maintainer-owned benchmark work unless a maintainer split out a public issue for it.
  • I kept the change scoped to the linked issue/task.
  • I completed the visual evidence section for UI changes, or marked UI impact none.
  • Every commit includes a DCO Signed-off-by trailer (git commit -s).

Why this branch is based on an older main

The branch root is 04ba65ac1 and both loopx/extensions/process_runtime.py and
tests/extensions/test_process_runtime.py are byte-identical between that commit and current main
(verified with git rev-parse <rev>:<path>), so the diff carries only the new tests and the merge review reads
the same either way. Environment used for the runs above: CPython 3.12 with the project installed as
pip install -e ".[test]", Node 22.23.2 on PATH for the sibling suites that shell out.

run_capped_process guards both the bytes a provider can push back and the
process group it leaves behind, but the only two tests here covered failure
paths. Nothing pinned the success contract, that the capture really stays
bounded against an endless writer, or which stream overflowed.

Signed-off-by: hahahahahayesyeseys <95999512+hahahahahayesyeseys@users.noreply.github.com>

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

动机

评审精确 head f5db9ad464398c8ca873afa92a6d7b277c87e537。扩展 provider 的 run_capped_process 是已上线的输出上限和子进程终止边界,但原有两项测试主要覆盖 overflow/timeout 时后代进程退出,没有固定正常响应会原样返回、精确上限可接受、非零 provider 退出码与 runtime failure 的区别,以及 stderr 溢出的归因。

改动思路

只在现有模块所属的 tests/extensions/test_process_runtime.py 增加五项真实子进程测试,断言调用者实际读到的 returncode、stdout 与 failure_kind,同时用持续输出的子进程检验上限和及时停止;不新增产品代码、测试框架或公开示例。

具体改动

新增 UTF-8/换行原样回显、provider exit(3)、恰好达到 byte limit、无界 stdout、stderr flood 五个场景。无界输出场景要求只保留 limit + 1 字节并在 20 秒内结束;stderr 场景要求 stderr_too_large 而不是 stdout failure。调用链检查确认 extension runtime 本来就分别处理 failure_kind 与退出码;作者提供的移除切片上限 mutation 反例与断言直接对应。同作者近期 #4900 守护的是 Lark manager context,非此边界的重复测试批次。

对主干的风险

只改一份测试文件(+95 行),不改变生产行为、状态或权限。当前 head 的本模块 7 项、相邻 extension runtime 63 项、Ruff 与 diff 检查通过;PR base 与当前主干在被测产品文件及原测试文件上的 blob 相同,旧基线未引入额外差异。Windows taskkill 分支和两路同时溢出的非确定性优先级仍未验证,不能从本次 POSIX 测试推断它们通过。

我的整体评价

APPROVE。 这是对真实 extension provider 资源边界有持续价值的薄回归,不是只复述现有行为的独立 smoke,也没有把一个场景拆成多 PR。现有测试文件是合适的维护位置;未见需要伴随的产品重构。

English verdict: APPROVE - exact head f5db9ad adds durable real-subprocess coverage for bounded output, exact bytes, exit semantics and stderr attribution; 70 focused/adjacent tests, Ruff and diff checks passed. Windows-only behavior remains untested.

@huangruiteng

Copy link
Copy Markdown
Collaborator

Exact-head readiness update for f5db9ad464398c8ca873afa92a6d7b277c87e537: the existing APPROVED review is valid and there are no unresolved review threads, but loopx pr-review --check-merge-readiness reports ready=false because GitHub merge state is BEHIND (merge_state_requires_update). CI was not consulted by this check.

Please update the branch against main; that creates a new head requiring fresh exact-head review and readiness qualification. No merge is recommended on this head. This check adds no new code-review verdict or refactor request.

@huangruiteng
huangruiteng merged commit 797a9c5 into loopx-project:main Sep 23, 2026
21 of 22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants