Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,7 @@ Rule of thumb: if you build and operate your own agent in production, use a trac
## Features

- **Automatic session health** — Opens with factual failure, repetition, follow-up and last-recorded call-state summaries. Missing/running/unknown results have evidence links; no human labels or model calls required. Manual notes and transfers are opt-in and never hide automatic facts. [Scope and offline checks](docs/diagnostics.md#automatic-session-health).
- **Codex background-process evidence** — Connect explicit `exec_command` process IDs to later `write_stdin` results, with launch/poll/exit source links. Ambiguous IDs or polling sequences stay unknown; process completion never rewrites historical tool-call states or proves a task passed. [Association limits](docs/diagnostics.md#codex-background-process-evidence).
- **Modification/check chronology** — Distinguish checks before an edit, checks overlapping it and later outcomes. A passed earlier check or a successful output pipeline is not post-change validation; ambiguous command fragments remain unknown. [Recognition and coverage limits](docs/diagnostics.md#modification-and-verification-chronology).

- **Per-turn ledger** — In the session summary, from two user turns on: one row per user turn with wall-clock time, tokens (input + output + cache) and cost, bars scaled to the session maximum, tool-call counts inline (error counts in the row tooltip), click to jump. Answers "why did this take 40 minutes / cost $3" without reading the transcript.
Expand Down
1 change: 1 addition & 0 deletions README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ LangSmith、Langfuse 这类观测平台面向的是*你自己写的* agent:接
## 功能特性

- **自动会话体检** — 默认自动整理失败、重复操作、后续候选和调用最后记录状态;执行中、未知及未记录结果可追溯证据。不依赖人工标注或模型调用,笔记与迁移改为可选,不影响自动事实展示。[口径与离线验证](docs/diagnostics.md#自动体检无需人工标注)。
- **Codex 后台进程证据** — 用明确进程 ID 关联启动、`write_stdin` 轮询和退出结果,可逐步跳转原始证据。ID 重用、轮询交叠和冲突保持未知;不改写历史工具状态,不把进程退出当作任务通过。[关联边界](docs/diagnostics.md#codex-后台进程证据)。
- **修改—检查时序** — 区分修改前成功的检查、与修改重叠的检查及后续最新结果;不把先前通过或管道整体成功当作修改后的验证。复杂命令片段执行状态保持未知。[识别边界](docs/diagnostics.md#修改与验证的先后顺序)。

- **有证据的失败事件(React UI)** — 将同一调用所在用户轮次、同工具、完整同参数的待复查失败分组,重复最多的操作优先展示;可跳转首末及每条原始证据。同参成功切断分组,缺少参数不合并。执行成功采用明确零退出码或 OMP 原生完成证据;事件不等于根因或任务失败。本地规则,无 LLM。[合成演示与判定边界](docs/diagnostics.md#中文使用指南)。
Expand Down
12 changes: 6 additions & 6 deletions claims.json
Original file line number Diff line number Diff line change
Expand Up @@ -102,16 +102,16 @@
},
{
"id": "test-count",
"claim": "290 tests pass on Node's built-in test runner, the count docs/ROADMAP.md records for `npm test`.",
"value": "290",
"metric": "passing node:test cases (# tests 290 / # pass 290 / # fail 0)",
"method": "npm test → node --test test/*.test.js, run in the claims job after npm ci, and the TAP summary is asserted. The roadmap sentence ('290 tests on Node's built-in runner (`npm test`, 2026-09-23)') is verified by the run, not read back from the prose.",
"claim": "313 tests pass on Node's built-in test runner, the count docs/ROADMAP.md records for `npm test`.",
"value": "313",
"metric": "passing node:test cases (# tests 313 / # pass 313 / # fail 0)",
"method": "npm test → node --test test/*.test.js, run in the claims job after npm ci, and the TAP summary is asserted. The roadmap sentence ('313 tests on Node's built-in runner (`npm test`, 2026-09-23)') is verified by the run, not read back from the prose.",
"repro": "npm test 2>&1 | grep -E '^# (tests|pass|fail)'",
"evidence": "docs/ROADMAP.md",
"as_of": "2026-09-13",
"check": {
"cmd": "npm test 2>&1 | grep -E '^# (tests|pass|fail)'",
"expect": { "contains": ["# tests 290", "# pass 290", "# fail 0"] },
"expect": { "contains": ["# tests 313", "# pass 313", "# fail 0"] },
"timeout": 120
}
},
Expand Down Expand Up @@ -238,7 +238,7 @@
"check": {
"cmd": "node scripts/claims-receipts.mjs tests-node-only",
"expect": {
"equals": "20 files in test/ · 14 distinct requires: 11 node builtins, 3 relative, 0 third-party"
"equals": "21 files in test/ · 15 distinct requires: 11 node builtins, 4 relative, 0 third-party"
},
"timeout": 60
}
Expand Down
4 changes: 2 additions & 2 deletions docs/ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
- **Session browser** with tool-call inspection, trace/waterfall view, spawn tracking and message timeline
- **Prompt tooling** — extraction (noise filtered), template clustering with outcome attribution, Claude-powered rewrites, and a prompt library that installs entries as native slash commands
- **Global search** across all platforms, insights dashboard, incremental session backup
- **React + Vite frontend** served by an Express backend; 290 tests on Node's built-in runner (`npm test`, 2026-09-23), CI on Node 22
- **React + Vite frontend** served by an Express backend; 313 tests on Node's built-in runner (`npm test`, 2026-09-23), CI on Node 22
- **Evidence-backed failure events and local review** with full-result invalidation, evidence navigation and narrow-screen session layout

## Current priorities
Expand All @@ -21,7 +21,7 @@ External grounding: official guidance emphasizes [executable verification](https
| --- | --- | --- |
| P0 | Make the new workflow immediately testable | A demo-only entry opens a clearly synthetic case: 7 pending records in 2 events, all evidence accessible, local review does not rewrite automatic results. Preserve the existing default demo and samples. |
| P1 | Current-session review portability implemented | Preview-only import and explicit plaintext download; exact identity/evidence matching, no overwrites, bounded schema and partial-failure reporting. Validate with synthetic migration and publish after CI; whole-history backup and path remapping remain out of scope. |
| P2 | Execution/verification evidence, not another statistics dashboard | Automatic health and modification/check chronology: distinguish before/overlap/after and latest outcomes, show unsupported command coverage, keep optional notes independent. Deterministic transformations and frozen-log regressions verify association rules, not file coverage, human time saved or universal accuracy. Next investigate explicit per-step check outcomes and task/child linkage before relaxing ambiguous shell assumptions. |
| P2 | Execution/verification evidence, not another statistics dashboard | Automatic health and modification/check chronology distinguish before/overlap/after and unknown outcomes. Codex process IDs now connect launch/poll/exit evidence without rewriting history or original check start. Next investigate explicit task/child linkage and per-step outcomes; reject ambiguous associations rather than relaxing shell assumptions. Deterministic tests and frozen-log checks are not file coverage, human time saved or universal accuracy. |
| P3 | Make releases reproducible for contributors | Keep clean-install tests, generated fixtures, documentation claims and release/package verification aligned. Add browser regression automation when it can run deterministically without personal logs. |

No launch dates or star-count targets are promised. Progress is gated on these observable outcomes. Physical-device/keyboard coverage and complex Trace/analytics layouts remain separate work, not implied by the session-screen checks.
Expand Down
12 changes: 12 additions & 0 deletions docs/diagnostics-verification.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,3 +118,15 @@ Browser checks used synthetic fixtures only:
- Call/result jumps resolve to actual source records without moving outer page navigation. At 390px the chronology panel has no horizontal overflow; static demo uses zero backend API requests.

Use `node scripts/demo-verification.cjs` to replay the temporal example and `node scripts/demo-follow-up.cjs` for missing-result updates. The public hosted demo exposes the earlier-test/later-edit example under automatic health. Physical-device keyboards and every complex view are not certified. Existing lint findings (91 warnings / 159 infos) and bundle-size warnings remain.

## Codex background-process acceptance

This increment associates explicit Codex process IDs from launch result headers with later `write_stdin.session_id` calls. It does not rewrite the historical per-call state, failure counts, recovery or human review data. Process outcomes are a separate evidence chain; the original launch index is retained when chronology uses a final polling result.

The full suite has **313 passing tests**. The **23 process-evidence tests** cover success/failure/running/missing results, output-text false positives, malformed/contradictory headers, noncanonical identifiers, duplicate calls/results, reused process IDs, overlap, post-terminal polls, input-sending calls, source references, qualified tool names and multiple launches in one message. Forty fixed transformations verify terminal-result append without inferring automatic recovery. The generated hosted sample is asserted equal to the real Codex parser output.

The frozen real corpus was re-read under content hashes: all previous `diagnoseSession` and `summarizeSessionHealth` outputs match v1.21.0 exactly. In its Codex subset, 60 launch headers and 53 polling calls yielded **53 uniquely linked polls**, **48 successful process exits**, **1 failed exit**, and **11 launches whose last evidence is running**. Every process association and terminal result was checked against the recorded process ID and header boundary. No real commands, process IDs or detailed logs are published. These results do not establish live runtime state, test coverage, general accuracy or time savings.

Synthetic browser checks validate the process summary, unlinked poll count, source-result navigation, a test launched before an edit remaining overlapping, a later failed check remaining visible and live completion changing terminal process count from 2 to 3 without changing historical failure events. At 390px the process panel has no horizontal overflow. `node scripts/demo-process-evidence.cjs` reproduces the example locally without executing transcript commands.

Unknown/conflicting chains are not silently certified; callers who need live job control or cross-session task association still need stronger runtime evidence. Input is represented as a boolean in the process summary, though original call evidence remains accessible. Existing large-bundle and lint findings remain unchanged.
35 changes: 35 additions & 0 deletions docs/diagnostics.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,30 @@ Try `node scripts/demo-follow-up.cjs`: the fixture includes running, unknown and

![Synthetic automatic session health](../screenshots/automatic-health.png)

## Codex background-process evidence

The **Codex 后台进程证据** section links a recorded `exec_command` launch header (`Process running with session ID ...`) to later `write_stdin` calls with that exact numeric `session_id`. It shows launch and polling source records, whether input was sent, and an unambiguous terminal exit result where available.

This is a separate unit from per-tool-call state: the historical launch still returned a running/unknown record, while the process may have finished through a later poll. Polling is not a second launch, and none of these links changes failure events, automatic retry recovery or saved review fingerprints.

- Only supported Codex wrapper headers **before** `Output:` / `Final output:` are used, on exact `exec_command` / `write_stdin` tool names or their `functions.`-qualified forms. Matching text printed in stdout or read from a file is not process metadata.
- Launch and poll call IDs must be unique in the currently viewed transcript. Process IDs must be nonnegative safe integers; string IDs and missing launch history are not guessed.
- Reused process IDs, duplicate launch/poll records, mismatched returned IDs, missing or invalid polling results, overlapping polls or polls after a terminal result make completion ambiguous. The original records remain accessible, but no final result is adopted from the conflicting chain.
- A last running record is not live process monitoring. Input-sending polls are marked without showing input text in the process summary. The raw call remains available if you choose to inspect it.
- A recognized verification command can use its linked final process result, but retains its **original launch position**. A check launched before an edit and finished afterward still overlaps the edit; it is not post-edit verification. Input-fed processes and compound shell fragments keep an unknown check outcome even when the process exits zero.

Use the hosted demo's **Codex** tab and select the `[Synthetic demo] Background process evidence` session. It contains two launches: one finishes through a poll, one has only a running record, plus an unlinked poll. The previous default demo remains unchanged.

For live updates from a source checkout:

```sh
node scripts/demo-process-evidence.cjs
```

Select the Codex synthetic session. Three launches initially include a successful exit, a failed exit and a pending process; one poll has no matching launch. Enter `c` to append the missing completion and watch the process evidence update, or `q` to stop and clean up. No logged commands are executed.

![Synthetic Codex process evidence](../screenshots/codex-process-evidence.png)

## Modification and verification chronology

The **修改—检查时序** section asks a narrower, evidence-based question than “did the task pass?”:
Expand Down Expand Up @@ -208,6 +232,7 @@ node --test test/review-transfer.test.js
node --test test/follow-up-evidence.test.js
node --test test/session-health.test.js
node --test test/verification-chronology.test.js
node --test test/codex-process-evidence.test.js
npm test
npm run build:ui
npm run lint
Expand Down Expand Up @@ -287,6 +312,16 @@ The local frozen regression set contained 30 sessions and 9,076 tool results. Gr

本轮的 373 个真实冻结事件只使用**内存中的合成测试标记**检验隔离与失效,没有替你判断真实事件,也没有把这些测试标记写成真实复核。完整证据见 [本机复核验收](diagnostics-verification.md)。

## Codex 后台进程证据

展开“Codex 后台进程证据”,可以沿着 `exec_command` 的包装头进程 ID,找到同一会话内 `write_stdin.session_id` 对应的轮询与最终退出结果。启动、每次轮询和结果均可跳转。

它与“每条工具调用的最后记录状态”分开统计:历史启动结果不被改写,轮询也不是另一次启动。重复进程 ID、重复调用/结果、返回 ID 不符、交叠轮询、终止后继续轮询或缺少可靠结果时,完成状态保持未知。只认包装头,不把 stdout 中的示例文本当证据,不跨会话猜测。

若这个进程运行的是可识别检查,时序视图可采用唯一关联的终止结果,但仍保留原始启动位置:修改前启动、修改后才返回的测试仍是“重叠”,不算修改后验证。发送过输入或执行复合命令的检查结果仍保持未知;整个进程退出零码不代表任务通过。

在线 Demo 的 Codex 页有明确标记的合成案例;源码可运行 `node scripts/demo-process-evidence.cjs`,输入 `c` 为待完成进程追加结果,`q` 退出。当前只支持上述 Codex 输出契约,不宣称 OMP/Claude 后台任务链也已覆盖。

## 后续相关操作

失败事件下若有候选,可展开“后续相关操作”,查看匹配依据、五类状态、调用/结果位置、参数摘要及原始结果。**它不自动关闭事件,不代表任务已通过。**
Expand Down
25 changes: 25 additions & 0 deletions docs/releases/v1.22.0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# v1.22.0 — Trace Codex background completion to its source

Codex can return from `exec_command` while a process is still running, then deliver its exit result through `write_stdin`. AgentXRay now links that lifecycle using explicit recorded process IDs rather than leaving the launch isolated from its later result.

## New

- **Launch → poll → exit evidence:** source links for the original call, initial result, every associated poll and unique final exit result.
- **Ambiguity stays visible:** reused IDs, duplicate records, overlapping polls, mismatched IDs or polls after terminal output do not certify completion. Unknown/unlinked polls are counted.
- **Verification timing keeps the original start:** a test launched before a modification remains overlapping even if its exit result arrives after the modification. Later polling is not a new test run.
- **No historical rewrites:** per-call health, failure events, recovery decisions and review fingerprints are unchanged. Process outcomes are shown separately. Input-fed and compound commands cannot become clean test successes from an exit-zero wrapper.
- **Synthetic demos:** select the new Codex process-evidence sample in the hosted demo, or run `node scripts/demo-process-evidence.cjs` from a checkout for live completion updates.

## Validation

313 Node tests pass, including 23 process-evidence tests and 40 deterministic terminal transformations. The prior frozen 15-session/2,024-result regression preserves all existing health and diagnostic output. Its Codex subset links 53 polls across 60 starts, finding 48 successful exits, 1 failed exit and 11 starts with only a running record. These are association results, not task correctness or live process monitoring.

Browser checks cover result navigation, failed later checks, pre-edit launch overlap, unlinked polling, live completion and narrow screens. Normal/demo builds, lint and claims checks pass with existing lint findings unchanged.

Only the recognized Codex wrapper contract is supported. Quoted/output text is not metadata; no cross-session or OMP/Claude lifecycle guessing, no automatic commands, no new runtime dependencies or uploads.

[Usage and limits](https://github.com/alloevil/AgentXRay/blob/master/docs/diagnostics.md#codex-background-process-evidence) · [Verification receipt](https://github.com/alloevil/AgentXRay/blob/master/docs/diagnostics-verification.md)

```sh
npx @alloevil/agent-xray@1.22.0
```
Loading
Loading