Skip to content

feat(server): suspend idle Pi runtimes and notify clients - #199

Open
oxwen11 wants to merge 13 commits into
mainfrom
feat/pi-runtime-idle-timeout
Open

oxwen11 wants to merge 13 commits into
mainfrom
feat/pi-runtime-idle-timeout

Conversation

@oxwen11

@oxwen11 oxwen11 commented Sep 6, 2026

Copy link
Copy Markdown
Owner

Summary

  • After a held Pi runtime sits at phase idle for PIE_SESSION_RUNTIME_IDLE_MS (default 5 minutes; 0 disables), the session suspends it: kill the process without sealing the session.
  • Publishes new wire event session.runtime.stopped (reason: idle). Session stays queryable; the next prompt re-ensureRuntimes as today.
  • App chat runtime clears live turn/request UI on that event but stays ready (not a terminal error like session.crashed). EventBus subscription keeps flowing — no forced client re-attach; process comes back on the next prompt.

Why not client ChatManager LRU

Client unsubscribe does not free Pi child processes. This puts the control on the server where the cost is.

Test plan

  • packages/server session tests: suspendRuntime + short idle timeout
  • apps/app chat test: session.runtime.stopped → ready, no error
  • Manual: start a session, wait idle timeout (set env low), confirm process gone + UI still usable, send prompt and confirm resume

Kill held Pi processes after an idle timeout without sealing the session,
publish session.runtime.stopped, and let the next prompt ensureRuntime again.
UI clears live turn state but stays ready (not a terminal error).
@github-actions

github-actions Bot commented Sep 6, 2026 •

Copy link
Copy Markdown

React Doctor found no new issues. 🎉

Reviewed by React Doctor for commit 1b5734f.

Use TestClock under @effect/vitest so the idle fiber actually fires, split
toContain calls, and allow the stopped verb in the event naming invariant.
@pkg-pr-new

pkg-pr-new Bot commented Sep 6, 2026 •

Copy link
Copy Markdown
npx https://pkg.pr.new/oxwen11/pie/@getpie/cli@199

commit: 1b5734f

@oxwen11

oxwen11 commented Sep 20, 2026

Copy link
Copy Markdown
Owner Author

reviewer: CI not passed at current head dd59dc7 — required check Check (Code check) is FAILURE: https://github.com/oxwen11/pie/actions/runs/34013754642/job/101433922610 (run confirmed at this exact head). Per .agents/rules/review-and-merge-pr.md, no code review or verification runs until required CI actually passes.

…timeout

# Conflicts:
#	packages/server/src/harness/session-fold.ts
@oxwen11

oxwen11 commented Sep 21, 2026

Copy link
Copy Markdown
Owner Author

reviewer: CI not passed at current head 72887913c1bedc182e755bddb354c481a520a2f8 (prior CI comment was for dd59dc7) — required check Check is QUEUED (react-doctor and both previews green). Not reviewing or verifying until all required checks actually pass at this head.

@oxwen11

oxwen11 commented Sep 22, 2026

Copy link
Copy Markdown
Owner Author

reviewer: CI not passed at current head 72887913c1bedc182e755bddb354c481a520a2f8 (update of the prior QUEUED comment at this head — the check has now executed and failed):

Per .agents/rules/review-and-merge-pr.md: no code review or functional verification at a head where required CI has failed. Fix the failing check and push a new head; the next reviewer run starts again at CI for the new head.

@oxwen11

oxwen11 commented Sep 30, 2026

Copy link
Copy Markdown
Owner Author

Review-and-merge gate record — not merged

  • Head: 960f2d66f53e09315998a41321819a8c6aaa8b11.
  • Current target main and trusted rules commit: d7f4b7a95588e5de2aeefbe790e65111e89195f6.
  • Policy: trusted PR workflow, with review, security and acceptance rules from that base (not PR-proposed or retired rules).
  • Context: feat(server): suspend idle Pi runtimes and notify clients. Same-repository, author oxwen11, open, non-draft.

Required CI observed for this head

Blocking gate and conclusion

  • Comparison to current main shows the head is behind by 8 commits; historical green checks do not satisfy the current-base gate.
  • Required CI has not all passed at this head (see failed or queued/in-progress results above).

Stopped before code review and independent verification; not merged. These are gate observations, not an implementation-quality verdict. No branch update, conflict resolution, CI rerun, protection bypass or fix-and-merge was performed.

Revisit only after a relevant change resolves the recorded blocker: an author-provided head containing current main with no conflict where necessary, and every required check present, executed and successful for the candidate version. The next eligible version starts again at CI; unchanged blockers need no duplicate review/comment.

@oxwen11

oxwen11 commented Sep 30, 2026

Copy link
Copy Markdown
Owner Author

CI 失败定位与修复方向

  • Head:960f2d66f53e09315998a41321819a8c6aaa8b11
  • 当前 main / 可信规则 commit:8f7546548ca95b7167c2261ebe87f4425806cf65
  • 失败检查日志

本次 Check 失败明确来自 format:check:packages/server/src/harness/runtime-idle.ts 不符合格式;日志中的类型检查与测试通过不能覆盖格式失败。先按仓库 formatter 修此文件并检查 diff,再更新到当前 main,运行新 CI。随后仍需验证 idle 真正释放子进程、下次 prompt 恢复、流不断开和非 idle 不误杀;格式通过不等于功能验收完成。

这是对既有阻塞新增的日志定位,不是代码审查通过。未本地重现或验证修复方案,未修改分支、重跑 CI 或合并。修复/环境恢复并得到满足当前 base 的全绿候选后,重新从 CI → 审查 → 独立验收开始。

@oxwen11

oxwen11 commented Sep 30, 2026

Copy link
Copy Markdown
Owner Author

修复提交与独立审查交接

  • 新 head:aea30e9789360b3afc6630f4f4c1929bc6c033ca
  • 集成的 main / 可信规则:8f7546548ca95b7167c2261ebe87f4425806cf65

已修复唯一已定位的格式错误:runtime-idle.ts 函数签名换行,未改变行为。锁定安装后,同一 oxfmt --check 修前失败、修后通过。GitHub 原生 rebase 报冲突后没有强推或重写作者历史;本地模拟整合 main 无文本冲突,使用保留历史的 merge 同步到当前基线并普通 push。

这是实现者的修复自检记录,不是独立验收或批准。已普通推送并触发新 CI;不复用旧 head 的全绿状态。下一步:新 required CI 全部成功后,由独立审查者在干净 worktree 审查完整 PR 并验收。UI 产品路径仍需 Web/Desktop 截图与视频(适用时)。本轮没有合并。

@oxwen11

oxwen11 commented Sep 30, 2026

Copy link
Copy Markdown
Owner Author

独立审查发现新的 P1:空闲释放与新 prompt 存在竞态

  • Head:aea30e9789360b3afc6630f4f4c1929bc6c033ca
  • Base / 可信规则:8f7546548ca95b7167c2261ebe87f4425806cf65
  • 协调端已确认这个 head 的所有 required CI SUCCESS;独立审查进程在另一干净 worktree 工作,没有改源码。

独立审查定位 packages/server/src/harness/session.ts 的 suspendRuntime:旧 runtime 关闭完成前先清除所有权,允许并发 prompt 获取 replacement 并开始新 turn;旧关闭随后无条件发布 session.runtime.stopped,把新 turn 的状态重置为 idle、清掉 activeTurn/buffer,客户端显示 ready,而 replacement 仍在运行。

审查者用隔离的内存生命周期诊断暂停旧 runtime.close 并交错新 prompt,实际断言确认 running/new-turn → idle、activeTurn=null、replacement 仍被持有。它是可运行的竞态重现,不是实际模型或 UI 验收。

结论:独立审查不通过,不合并。需要协调旧 runtime 关闭、新 runtime 获取及 stop 事件发布,使旧资源结束不能覆盖新 turn。后续修复须把该交错变成回归测试,再由独立审查者重新检查。此前格式修复有效,但不构成整个功能通过。

证据边界:独立审查环境无法访问 GitHub,CI 由协调端查验;真实子进程 suspend/resume、Web/Desktop 证据尚未完成。主工作区及审查源码均未改动。

@oxwen11

oxwen11 commented Sep 30, 2026

Copy link
Copy Markdown
Owner Author

P1 修复已提交,等待新版本独立复核

新 head:81ceff43a1b4fc95dad90d1bd60859486ab3065e;base/可信规则 8f7546548ca95b7167c2261ebe87f4425806cf65。

沿用现有 Deferred 所有权机制,增加 suspension ticket:旧 runtime 关闭及 stopped 事件发布完成前,新 acquisition 和 release 等待。不改公共 API、事件格式或存储结构。

实现者证据:确定性回归 suspendRuntime finishes stopping before a replacement turn starts 修前失败(idle != running)、修后通过;五个相关文件共 38 项测试通过,无类型错误,覆盖并发获取、等待方中断、release、重复 suspend、idle timer、close defect。server Turbo typecheck、oxlint、oxfmt、diff 检查及提交钩子通过。

这是修复自检,不是独立批准。新 CI 与独立复核必须针对此 head;真实子进程和 UI 验收仍未完成,没有合并。

@oxwen11

oxwen11 commented Sep 30, 2026

Copy link
Copy Markdown
Owner Author

Independent re-review of the suspension repair — still blocked

Reviewed head 81ceff43a1b4fc95dad90d1bd60859486ab3065e against base/trusted rules 8f7546548ca95b7167c2261ebe87f4425806cf65, after all required checks for that candidate succeeded. This review began before #415 advanced main; it is not acceptance against the newer base.

The previous replacement-turn reset finding is resolved by source inspection: acquisition waits through shutdown and stopped-event publication, and release waits on the same suspension ticket. The reviewer did not independently execute the added tests: locked installation encountered sandbox network restrictions, so Vitest never ran. Earlier implementer tests remain separate evidence.

Remaining P1: idle shutdown can interrupt prompt admission

  • packages/server/src/harness/session-service.ts:411–412 returns an existing held runtime through manager.peek without reserving its use or canceling idle shutdown.
  • packages/server/src/harness/pi/runtime.ts:219–237 asynchronously waits for the Pi prompt response before emitting session.turn.started.
  • Meanwhile packages/server/src/harness/session.ts:491–495 can see the unchanged idle generation and suspend that runtime. A prompt arriving near the timeout boundary can therefore fail or be aborted while admission is underway.
  • There is also a publication window: folding the running phase precedes event publication/logging and afterPublish cancellation (session.ts:232–239), while suspension does not atomically validate current idle eligibility.

This is a source-traced finding, independently reported and cross-checked by the coordinator, not a claimed live reproduction. The ticket protects later acquisitions but does not reserve work that already obtained the runtime.

Required repair: coordinate prompt admission/runtime use with idle shutdown ownership and atomically validate eligibility before shutdown. A phase-only check does not cover admission. Add deterministic deadline/admission and turn-start-publication interleaving regressions. Then obtain new CI, independent re-review and real-process/UI acceptance; no self-approval or merge is granted by a repair.

No additional concrete security finding was established. Real child cleanup, same-session resume/history, interruption/release, disabled timeout, running/requires-action protection and applicable Web/Desktop proof remain acceptance obligations.

@oxwen11

oxwen11 commented Sep 30, 2026

Copy link
Copy Markdown
Owner Author

auto-merge: no

head 81ceff43a1b4fc95dad90d1bd60859486ab3065e — required CI green (Check, react-doctor, both Publish @getpie/cli preview, Continuous Releases, React Doctor), MERGEABLE (BEHIND).

out of .agents/rules/auto-merge-pr.md allowed groups; hits exclusions:

  • feature / new functional product behavior: idle Pi runtime suspend (packages/server/src/harness/runtime-idle.ts, session.suspendRuntime, idle timer/PIE_SESSION_RUNTIME_IDLE_MS)
  • contract/RPC/wire: new session.runtime.stopped event in packages/contract/src/domain.ts + harness fold/framework
  • client control flow: chat.ts treats session.runtime.stopped as idle/ready (not terminal) — handler/phase change, not presentation
  • not docs-tooling, lint-format, or ui-presentation

Din review-first: no squash-merge.

@oxwen11

oxwen11 commented Sep 30, 2026

Copy link
Copy Markdown
Owner Author

Deadline/admission repair pushed — not yet independently accepted

Follow-up to the independent P1 finding.

  • Previous head: 81ceff43a1b4fc95dad90d1bd60859486ab3065e.
  • Repair commit: 0cd6c976.
  • Current head after conflict-free, history-preserving integration of main: 7db38eec2d121d6c01af725bd58dded8172fd640.
  • Current base/trusted rules: a9709fcc6d513f6786dd6c6ce44b13ba8a8e71c3; GitHub compare confirms behind_by=0. Ordinary fast-forward push; no force push or protection change.

Implementation and regression evidence

Prompt lookup/acquisition/admission now holds a scoped runtime-use reservation. Idle shutdown validates reservation count, idle phase and timer generation before claiming ownership under the existing event-fold/publication lock. Shutdown still runs outside that lock, and the prior suspension ticket remains held through stopped-event publication. Scope finalization releases reservations on success, failure and interruption.

The implementation worker demonstrated both deterministic regressions failing before the fix:

  • A prompt held across the idle deadline caused its native runtime to close.
  • A held turn-start publication allowed the runtime to be removed/closed despite its running phase.

After correction, the coordinator reran the focused checks, then reran them again after main integration:

pnpm --filter @getpie/server test test/harness/session.test.ts test/harness/session-manager.test.ts test/harness/session-service.test.ts test/harness/event-manifest.test.ts test/harness/types/session-envelope.test.ts test/harness/pi/runtime.test.ts

6 files / 91 tests passed; no type errors. Coverage includes both new races, concurrent reservation ownership/release, interrupted admission waiters, prior suspension/reacquisition/release cases, running/requires-action protection and disabled timeout. Server typecheck, explicit scoped oxlint, scoped oxfmt check and diff checks passed. Worktree is clean.

This is implementation evidence, not independent acceptance. New required CI must finish successfully before a new independent full-PR review. Previous findings cannot be considered independently resolved solely by this repair record. Real-process cleanup/resume/history and applicable UI acceptance remain outstanding; no merge or deferred auto-merge has been armed.

@oxwen11

oxwen11 commented Sep 30, 2026

Copy link
Copy Markdown
Owner Author

auto-merge: no

head 7db38eec2d121d6c01af725bd58dded8172fd640 — required CI green (Check, react-doctor, Publish @getpie/cli preview, Continuous Releases, React Doctor), MERGEABLE (BEHIND). Prior no was on 81ceff43a1b4fc95dad90d1bd60859486ab3065e; re-reviewed this updated head.

out of .agents/rules/auto-merge-pr.md allowed groups; hits exclusions:

  • feature / new functional product behavior: idle Pi runtime suspend (packages/server/src/harness/runtime-idle.ts, idle timer / PIE_SESSION_RUNTIME_IDLE_MS)
  • contract/RPC/wire: new session.runtime.stopped event + client chat runtime handling
  • non-presentational server/harness control flow; not docs-tooling / lint-format / ui-presentation

fail closed; no merge / no update-branch.

@oxwen11

oxwen11 commented Sep 30, 2026

Copy link
Copy Markdown
Owner Author

Current-base progress — waiting at required CI

After independently verified #412 merged, this branch was updated without resolving conflicts to base/trusted rules e668269e7b1788a4a4483768b8ce0b9ad8e079a3. Current head: 4ed45c5c5bf2dcc10b5d0ebb3dc87a256860f23f.

Both preview checks and react-doctor succeeded; required Check is still queued at this observation. No old-head review/acceptance is carried forward. Current-head full review and independent E2E have not started because the CI-first gate has not passed. No merge, protection bypass or deferred auto-merge armed. Resume at successful required CI, then independent review and affected real-runtime acceptance; no routine human approval is being requested.

@oxwen11

oxwen11 commented Oct 1, 2026

Copy link
Copy Markdown
Owner Author

auto-merge: no

head 4ed45c5c5bf2dcc10b5d0ebb3dc87a256860f23f — required CI green (Check, react-doctor, Publish @getpie/cli preview, Continuous Releases, React Doctor), MERGEABLE (BEHIND). Prior nos were on 81ceff43a1b4fc95dad90d1bd60859486ab3065e and 7db38eec2d121d6c01af725bd58dded8172fd640; re-reviewed this updated head.

out of .agents/rules/auto-merge-pr.md allowed groups; hits exclusions:

  • feature / new functional product behavior: idle Pi runtime suspend (packages/server/src/harness/runtime-idle.ts, session.runtime.stopped wire event in packages/contract, admission/reserveAdmission + session-manager/service control flow, client chat handling in apps/app).
  • non-presentational server/runtime change with RPC/schema/wire + Effect flow — excluded from lint-format and ui-presentation.
  • tests change product control-flow coverage alongside the feature (not standalone hygiene).

rules from 451a89407c6b via /workspace/pie-auto-merge-rule

@oxwen11

oxwen11 commented Oct 1, 2026

Copy link
Copy Markdown
Owner Author

Independent current-head acceptance — P1 queued-message loss; not merged

Head d0b86799c16f72b272ee7104fbade3ca4e801144, base/trusted rules 4a338cb0f5e1b22d24ca1dea46ce1ee6a50c2cd3. Required CI passed before review. Clean pinned reviewer worktree, frozen install and Turbo builds of server/Desktop/Verify; full 14-file diff and lifecycle/service/client callers reviewed. No source changes or fix-and-merge.

Previous findings

The suspension ticket and scoped prompt-admission reservation address the two previously reported ownership/admission races. Independently executed 91 server tests across 6 files (including those regressions) and 45 chat tests; all passed. Full React Doctor scan: 292 files, 100/100. This resolves those specific findings, not the entire PR.

New P1: idle suspension silently destroys queued user input after Stop

session.ts:448–460 treats phase === "idle" with no active admission as disposable, without checking retained steering/follow-up messages. But a canceled turn deliberately leaves Pi's queue intact. The new suspension kills that process; session-fold.ts:266–275 and chat.ts:245–250 then clear the queue, losing user input that was neither executed nor persisted.

The trusted session-chat recipe explicitly requires Stop to leave the queue unchanged. Phase-idle is not equivalent to having no outstanding user work.

Actual Electron reproduction, not a mock:

  1. Set the supported verification override PIE_SESSION_RUNTIME_IDLE_MS=5000 (same code as the 5-minute default).
  2. In the isolated sample Project, start a real GLM-5.3 turn executing Bash sleep 20.
  3. While it runs, send follow-up QUEUED-DRAFT-9274: after the current task, reply QUEUE-COMPLETED-9274.
  4. The queued row, Steer, Edit and Remove are visible. Click Stop generating. Immediately afterward the queued row remains, as expected.
  5. The canceled turn ends at 05:05:29.870Z. At 05:05:34.917Z, session.runtime.stopped reason=idle fires. Without another action, the queued row and all its controls disappear.
  6. The queued marker is absent from the isolated native transcript; there is no assistant result for it. It was lost, not completed.

Attached screenshots show the same stopped session with the queued text and then without it. The 76.934-second, 60fps VP8 recording captures sending, queuing, stopping and the loss. The visible aborted-operation notice is caused by the intentional Stop; it is not the new finding.

Required author follow-up: preserve pending user input across idle handling, or keep runtimes with pending work ineligible for idle disposal. Cover retained follow-up/steering after cancellation and subsequent user actions with deterministic regressions and real runtime proof. Any new persistence design still needs its normal design authorization. After an author update, restart new-head CI → full review → acceptance.

Passed checks and evidence boundaries

  • Web happy path passed: real GLM-5.3/Bash turn ran 31.7 seconds despite the 5-second idle setting. Its process exited 5.035 seconds after completion. A different child resumed the same Session/native identity and correctly recalled LIME-4821; it exited 5.019 seconds after that turn. Unsent composer text survived idle; reload retained both replies. Screenshots and the 188.8-second 60fps recording are attached.
  • Desktop failed on the separate queue path above. Used the actual existing Electron renderer; its pinned target stayed unchanged through both Doctor runs. Anonymous ticket 401 / authenticated ticket 200. No replacement browser/tab, fake provider, RPC-created session or edited application state was used.
  • Both recordings passed full ffmpeg decode. No browser errors were reported. UI acceptance stopped at the defect; no claim of complete Desktop happy-path acceptance or packaged-app validation.
  • Verification used isolated Pie/native-agent state and sample Projects. Owned processes/ports were confirmed stopped, temporary credential copies removed, screenshots/videos retained outside git. The occupied user daemon was not adopted or stopped.

Conclusion: blocked on reproducible user-input loss. No merge or repair attempted.

queued-after-stop

queued-lost-after-idle

recording-001.webm

idle-before-send

idle-resumed-reply

recording-001.webm

@oxwen11

oxwen11 commented Oct 1, 2026

Copy link
Copy Markdown
Owner Author

Queue update after verified #423 merge — waiting at CI

Head 1b5734f178d155aa28b5a162c830e7b8bd88ed87 now contains main/trusted-rules 0920277492d381b108b520b341c449bcc3e3dbd8 (behind_by=0 verified). GitHub performed a conflict-free merge update guarded by the previous exact head; no conflict resolution or source repair.

Required CI for this new head is queued/in progress at this observation, so no old-head review or acceptance is carried forward. The independently reproduced queued-message-loss blocker remains unresolved: #199 (comment)

No merge or deferred auto-merge. Continue only after the applicable blocker is resolved and the new version passes required CI → review → independent acceptance.

@oxwen11

oxwen11 commented Oct 5, 2026

Copy link
Copy Markdown
Owner Author

auto-merge: no

head 1b5734f178d155aa28b5a162c830e7b8bd88ed87 — required CI green (Check, react-doctor, Publish @getpie/cli preview, Continuous Releases, React Doctor), MERGEABLE (BEHIND). Prior no was on 4ed45c5c5bf2dcc10b5d0ebb3dc87a256860f23f; re-reviewed this updated head.

Out of the allowed groups; hits exclusions:

  • feature / new functional product behavior: idle Pi runtime suspend (packages/server/src/harness/runtime-idle.ts, session.ts +181/-16, session-manager.ts, session-service.ts) and client handling in apps/app/src/features/chat/runtime/chat.ts.
  • RPC/schema/wire change: new event in packages/contract/src/domain.ts and the harness event framework/fold.
  • tests cover the new product control flow alongside the feature.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants