fix: geny-executor 2.64.8~2.65.2 이식 — 턴이 멈추던 두 경로 포함 (2.69.0)

CHANGELOG.md

@@ -4,6 +4,47 @@ All notable changes to `xgen-agent-runtime` are recorded here. The format
 follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and
 this project adheres to [Semantic Versioning](https://semver.org/).
 
+## [2.69.0] — 2026-08-11
+
+### Fixed (ported from geny-executor 2.64.8 ~ 2.65.2)
+XGEN 은 2.64.7 에서 갈라져 독자적으로 가지만, **같은 뿌리에서 온 결함은 여기에도
+그대로 있다.** 해당 파일들을 전수 대조한 결과 갈라진 이후의 차이는 포매팅뿐이었고,
+아래 넷은 전부 미이식 상태였다. 재현 테스트까지 함께 가져왔다.
+
+**끝난 CLI 가 제 stdout 으로 턴을 붙잡던 문제 (2.65.2)**
+파이프의 EOF 는 *마지막* writer 가 닫을 때 온다. CLI 는 MCP 서버를 자식으로
+띄우고 그 자식들이 stdout 을 물려받으므로, 하나라도 CLI 보다 오래 살면
+`readline()` 이 영영 오지 않을 바이트를 기다린다. `proc.wait()` 로도 못 푼다 —
+asyncio 는 자식 종료 **와** 모든 파이프 해제를 둘 다 봐야 완료로 치는데, 그
+두 번째 조건을 누수된 FD 가 정확히 막는다. 완전한 답을 만들어 놓고 턴이 멈춘다.
+
+- 읽기 루프가 `proc.returncode` 를 폴링한다(자식 감시자가 채우므로 파이프와
+  무관). 종료가 관측되면 예산이 `exit_drain_grace_s`(기본 5초)로 줄고, 첫
+  조용한 순간에 스트림이 끝난다. 이미 버퍼에 있는 바이트는 그대로 전달된다.
+- `_reap()` 이 종료 상태 대기도 같은 방식으로 묶고, 누수 경로에서 프로세스
+  **그룹**을 죽여 FD 를 해제한다. `_kill_tree(force=True)` 는 직계 자식이 이미
+  거둬졌다고 일찍 돌아가지 않는다 — 남은 생존자가 바로 문제의 원인이다.
+
+**죽은 핫스페어가 이후 모든 턴을 세우던 문제 (2.65.1)**
+`returncode` 는 장부일 뿐이다 — 이벤트 루프가 자식을 거두기 전까지 None 이라,
+이미 죽은 프로세스가 영원히 "건강한 스페어" 로 읽힌다. 턴은 죽은 파이프에
+프롬프트를 건네고 기다린다. `_process_alive()` 가 `kill(pid, 0)` 으로 커널에
+직접 묻는다.
+
+**노트를 지워도 색인 행이 남던 문제 (2.65.0)**
+쓰기에는 자동 벡터 훅이 있었는데 삭제에는 없었다. 지워진 노트의 벡터가 계속
+검색에 잡히고 본문은 못 찾는다. 부팅 시 전방 스캔으로도 못 잡는다 — 존재하는
+파일을 훑는 루프는 없는 파일을 방문하지 않아서, 쌓이기만 한다.
+`attach_vector_remover` 를 인덱서와 대칭으로 두고, 실패는 로그만 남기고
+계속한다(마크다운 삭제는 이미 일어났고 그쪽이 정본이다).
+
+### Changed
+- **`DocGuide` 가 `path` 를 받는다 (2.64.8 이식, XGEN 토픽 집합에 맞춤)** —
+  작업 중인 문서를 넘기면 그 포맷의 토픽만 돌려준다. `.docx` 를 다루는 중에
+  슬라이드 토픽이 목록에 섞여 있으면 에이전트는 그 파일에 쓸 수 없는 도구를
+  읽고 시도한다. `fmt=` 를 모르는 구버전 엔진에서는 조용히 예전 동작으로
+  돌아간다 — 포맷 스코핑은 편의이지 계약이 아니다.
+
 ## [2.64.7] — 2026-08-06
 
 ### Added

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "xgen-agent-runtime"
-version = "2.68.1"
+version = "2.69.0"
 description = "Harness-engineered agent pipeline library with 21-stage dual-abstraction architecture, built on the Anthropic API"
 readme = "README.md"
 license = "Apache-2.0"

src/xgen_agent_runtime/llm_client/_cli_runtime.py

@@ -62,7 +62,6 @@
 
 logger = logging.getLogger(__name__)
 
-
 # ── CLI stdout stream limit ────────────────────────────────────────────
 # The CLI emits one stream-json event per line, and tool_result contents
 # ride INSIDE those lines — a DocXmlRead (200K chars), a big file Read, or
@@ -72,6 +71,12 @@
 # discarded, so the line is unrecoverable). Delegated heavy work makes
 # large tool results routine, so the default is deliberately generous:
 # 32 MiB (a cap, not an allocation — memory is used only per actual line).
+#: How often the stream reader looks up from the pipe to check whether
+#: the child is still alive. Only ever costs a wakeup when no line has
+#: arrived, so a busy stream never pays it.
+_EXIT_POLL_S = 0.25
+
+
 def _cli_stream_limit() -> int:
     raw = os.environ.get("GENY_CLI_STREAM_LIMIT", "").strip()
     try:
@@ -221,6 +226,22 @@ class CLIProcessRunner:
     cwd: Optional[str] = None
     timeout_s: float = 300.0
     kill_grace_s: float = 2.0
+    #: How long to keep reading stdout AFTER the child has exited.
+    #:
+    #: A pipe reaches EOF when the last writer closes it — which is not
+    #: the same event as "the child exited". The CLI spawns MCP servers
+    #: as its own children, and they inherit its stdout; one that
+    #: outlives it (or ignores the parent-death signal) holds the write
+    #: end open, so ``readline()`` blocks on a pipe nothing will ever
+    #: write to again. Waiting on EOF alone therefore parks the turn for
+    #: the FULL ``timeout_s`` — 2026-08-10 in production that was a dead
+    #: CLI (rc=0, output complete) and a turn that hung until a
+    #: host-side stall guard abandoned it minutes later.
+    #:
+    #: After exit, anything still in flight is bytes already written to
+    #: the pipe buffer, which drain immediately. This grace is generous
+    #: for that and short enough that a leaked FD costs seconds.
+    exit_drain_grace_s: float = 5.0
 
     def __post_init__(self) -> None:
         if not self.binary:
@@ -301,9 +322,15 @@ async def stream(
         stderr_task = asyncio.create_task(_collect_stderr(proc.stderr, stderr_buf))
 
         try:
-            async for line in _aiter_lines(proc.stdout, timeout_s=self.timeout_s, start_t=t0):
+            async for line in _aiter_lines(
+                proc.stdout,
+                timeout_s=self.timeout_s,
+                start_t=t0,
+                proc=proc,
+                drain_grace_s=self.exit_drain_grace_s,
+            ):
                 yield line
-            rc = await proc.wait()
+            rc = await self._reap(proc)
         except CLITimeout:
             await self._kill_tree(proc)
             raise
@@ -355,10 +382,48 @@ async def _communicate(
             timeout=self.timeout_s,
         )
 
+    # ------------------------------------------------------------- reap
+    async def _reap(self, proc: asyncio.subprocess.Process) -> int:
+        """Exit status, without betting the turn on ``proc.wait()``.
+
+        ``wait()`` completes only when the child has exited AND every
+        pipe has disconnected. A survivor holding the inherited stdout
+        satisfies the first and blocks the second forever, so awaiting
+        it here would re-introduce exactly the hang the read loop just
+        escaped.
+
+        So: give ``wait()`` a short window, and if it does not land,
+        take the returncode the child watcher already recorded and kill
+        the process group — which is also what finally releases the
+        leaked descriptor.
+        """
+        try:
+            return await asyncio.wait_for(proc.wait(), timeout=self.kill_grace_s)
+        except asyncio.TimeoutError:
+            pass
+        rc = proc.returncode
+        if rc is None:
+            # Still genuinely running — the caller's ladder owns it.
+            return -1
+        logger.warning(
+            "CLI exited rc=%s but a pipe stayed open — killing the process group to release it",
+            rc,
+        )
+        await self._kill_tree(proc, force=True)
+        return rc
+
     # ------------------------------------------------------------- kill
-    async def _kill_tree(self, proc: asyncio.subprocess.Process) -> None:
-        """Send SIGTERM, wait grace, then SIGKILL the process group."""
-        if proc.returncode is not None:
+    async def _kill_tree(self, proc: asyncio.subprocess.Process, *, force: bool = False) -> None:
+        """Send SIGTERM, wait grace, then SIGKILL the process group.
+
+        ``force`` keeps going when the direct child is already reaped:
+        the group can still hold survivors it spawned — an MCP server
+        sitting on the inherited stdout — and those are precisely what
+        needs signalling. Without it the early return below reads "the
+        child is gone, nothing to kill", which is exactly wrong for the
+        case that leaks.
+        """
+        if proc.returncode is not None and not force:
             return
         try:
             if sys.platform != "win32":
@@ -508,35 +573,94 @@ async def _aiter_lines(
     *,
     timeout_s: float,
     start_t: float,
+    proc: Optional[asyncio.subprocess.Process] = None,
+    drain_grace_s: float = 5.0,
 ) -> AsyncIterator[bytes]:
+    """Yield stdout lines until EOF — or until the child is gone and quiet.
+
+    Once the child has exited, EOF is no longer guaranteed to arrive at
+    all: an inherited write end can keep the pipe open indefinitely. So
+    the read budget collapses to ``drain_grace_s`` from the moment the
+    exit is observed, and the iterator ends on the first quiet moment
+    instead of parking on ``timeout_s``. Bytes already in the pipe are
+    still delivered — draining a buffer is instant next to that grace.
+    """
     if stream is None:
         return
-    while True:
-        elapsed = time.monotonic() - start_t
-        remaining = timeout_s - elapsed
-        if remaining <= 0:
-            raise CLITimeout(f"stream timeout after {timeout_s:.1f}s")
-        try:
-            line = await asyncio.wait_for(stream.readline(), timeout=remaining)
-        except asyncio.TimeoutError as e:
-            raise CLITimeout(f"stream readline timeout after {timeout_s:.1f}s") from e
-        except ValueError as e:
-            # asyncio raises ValueError("Separator is found, but chunk is
-            # longer than limit") when ONE line exceeds the StreamReader
-            # limit — and discards the buffered bytes, so the line is
-            # unrecoverable. With the 32 MiB default this is near
-            # impossible; if it still happens, losing ONE event beats
-            # killing the whole delegated turn. Log loudly and continue.
-            logger.warning(
-                "CLI stream line exceeded the %d-byte limit — skipping one "
-                "event and continuing (%s)",
-                _cli_stream_limit(),
-                e,
+    read_task: Optional[asyncio.Task[bytes]] = None
+    died_at: Optional[float] = None
+    try:
+        while True:
+            now = time.monotonic()
+            remaining = timeout_s - (now - start_t)
+            if remaining <= 0:
+                raise CLITimeout(f"stream timeout after {timeout_s:.1f}s")
+
+            # One read task, carried across iterations: a readline that
+            # loses the race must NOT be discarded, or the bytes it is
+            # mid-way through consuming are lost.
+            if read_task is None:
+                read_task = asyncio.ensure_future(stream.readline())
+
+            if died_at is None and proc is not None and proc.returncode is not None:
+                died_at = now
+            if died_at is not None:
+                grace_left = drain_grace_s - (now - died_at)
+                if grace_left <= 0:
+                    logger.warning(
+                        "CLI exited but stdout stayed open for %.1fs — ending "
+                        "the stream (an inherited pipe write end is still "
+                        "held; the answer is complete, the FD is not).",
+                        drain_grace_s,
+                    )
+                    return
+                budget = min(remaining, grace_left)
+            else:
+                # Poll rather than await the child: ``proc.wait()`` cannot
+                # be the death signal here, because asyncio only completes
+                # it once every pipe has ALSO disconnected — which is the
+                # exact condition a leaked stdout FD prevents. The
+                # returncode, by contrast, is set the moment the child
+                # watcher reaps, pipes or no pipes.
+                budget = min(remaining, _EXIT_POLL_S)
+
+            done, _pending = await asyncio.wait(
+                {read_task}, timeout=budget, return_when=asyncio.FIRST_COMPLETED
             )
-            continue
-        if not line:
-            return
-        yield line
+
+            if read_task not in done:
+                if died_at is not None or budget < remaining:
+                    # Either draining after exit, or just a poll tick with
+                    # budget left — go round again.
+                    continue
+                raise CLITimeout(f"stream readline timeout after {timeout_s:.1f}s")
+
+            try:
+                line = read_task.result()
+            except ValueError as e:
+                # asyncio raises ValueError("Separator is found, but chunk is
+                # longer than limit") when ONE line exceeds the StreamReader
+                # limit — and discards the buffered bytes, so the line is
+                # unrecoverable. With the 32 MiB default this is near
+                # impossible; if it still happens, losing ONE event beats
+                # killing the whole delegated turn. Log loudly and continue.
+                logger.warning(
+                    "CLI stream line exceeded the %d-byte limit — skipping one "
+                    "event and continuing (%s)",
+                    _cli_stream_limit(),
+                    e,
+                )
+                read_task = None
+                continue
+            finally:
+                if read_task is not None and read_task.done():
+                    read_task = None
+            if not line:
+                return
+            yield line
+    finally:
+        if read_task is not None and not read_task.done():
+            read_task.cancel()
 
 
 async def _drain_stdin(

src/xgen_agent_runtime/llm_client/claude_code.py

@@ -113,6 +113,29 @@ def _classify_cli_result(result: CLIResult, *, cli_version: str = "") -> APIErro
     )
 
 
+def _process_alive(proc: Any) -> bool:
+    """Is this child still running, really?
+
+    ``asyncio.subprocess.Process.returncode`` is bookkeeping: it stays None
+    until the event loop reaps the child. A process that already exited can
+    therefore look alive indefinitely. Ask the kernel instead.
+    """
+    if getattr(proc, "returncode", None) is not None:
+        return False
+    pid = getattr(proc, "pid", None)
+    if not pid:
+        return False
+    try:
+        os.kill(pid, 0)
+    except ProcessLookupError:
+        return False
+    except PermissionError:
+        return True  # exists, owned by someone else — still alive
+    except OSError:
+        return False
+    return True
+
+
 class ClaudeCodeCLIClient(BaseClient):
     """Subprocess-backed Claude Code client."""
 
@@ -276,7 +299,20 @@ def _take_spare(self, argv: List[str]) -> Optional[Any]:
         The spare is only valid for an IDENTICAL argv (model, MCP config,
         session resume flags, permissions — everything). Any drift means
         the prewarmed process was booted with stale config: discard it
-        and spawn fresh. Also discards a spare that died while idle.
+        and spawn fresh.
+
+        It must also still be ALIVE, and ``returncode`` is not enough to
+        know that. It is only set once asyncio has reaped the child; a
+        process that exited without the transport noticing keeps
+        ``returncode is None`` forever, so a corpse reads as a healthy
+        spare. The turn then hands its prompt to a dead pipe and waits —
+        which is exactly what happened in production: the CLI started,
+        listed its tools, exited, and every subsequent turn stalled until a
+        watchdog abandoned it. Turning the prewarm off made the same
+        session answer in 11 s.
+
+        ``kill(pid, 0)`` costs a syscall and answers the question the
+        bookkeeping cannot.
         """
         spare = self._spare
         if spare is None:
@@ -286,7 +322,7 @@ def _take_spare(self, argv: List[str]) -> Optional[Any]:
         if expire_task is not None:
             expire_task.cancel()
         proc = spare["proc"]
-        if spare["argv"] != list(argv) or proc.returncode is not None:
+        if spare["argv"] != list(argv) or not _process_alive(proc):
             self._discard_spare_proc(spare)
             return None
         return proc