Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@ Rule of thumb: if you build and operate your own agent in production, use a trac
- **Global search** — One search box across all seven platforms at once, multi-keyword AND matching, colored platform badges per hit — including prompts recovered from sessions that Claude Code's cleanup already deleted
- **Session insights** — Aggregate analytics dashboard with tool stats, error clustering and daily trends
- **Evidence-backed failure events (React UI)** — Groups unresolved failures by the same tool, complete arguments and call's user turn, with repeated operations first, first/last evidence jumps and every original result retained. Successful results split groups; missing arguments stay separate. Execution success requires an explicit zero exit code or OMP-native completion evidence. These are review groups, not root-cause diagnoses or proof of task failure. Local rules, no LLM. [Try the synthetic walkthrough and read the boundaries](docs/diagnostics.md).
- **Follow-up evidence candidates** — See later calls differing only in `i`, or same-turn modifications of the same explicitly identified file. Each has a result status, matching rationale and evidence jump; candidates never automatically resolve the failure. [Matching boundaries](docs/diagnostics.md#follow-up-evidence-candidates).
- **Local review queue** — Record follow-up, expected-failure or alternative-verification notes in your browser. Evidence changes invalidate the old review; manual labels never rewrite automatic outcomes. No account or review backend. [Review workflow and storage limits](docs/diagnostics.md#local-review-workflow).
- **Review portability** — Preview and download current-session review notes, then import only exact evidence matches into empty local slots. Existing notes are never overwritten; stale/unmatched records are skipped. JSON files are unencrypted and contain your written notes, not automatically copied logs. [Transfer limits](docs/diagnostics.md#transfer-reviews-between-browsers).
- **Narrow-screen session workflow** — Below 768px, switch between the session list and full-width content without losing the current review draft; platform tabs scroll horizontally, and evidence jumps keep navigation visible. Desktop retains the two-column layout. [Scope and tested viewports](docs/diagnostics.md#narrow-screen-session-workflow).
Expand Down
1 change: 1 addition & 0 deletions README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ LangSmith、Langfuse 这类观测平台面向的是*你自己写的* agent:接
## 功能特性

- **有证据的失败事件(React UI)** — 将同一调用所在用户轮次、同工具、完整同参数的待复查失败分组,重复最多的操作优先展示;可跳转首末及每条原始证据。同参成功切断分组,缺少参数不合并。执行成功采用明确零退出码或 OMP 原生完成证据;事件不等于根因或任务失败。本地规则,无 LLM。[合成演示与判定边界](docs/diagnostics.md#中文使用指南)。
- **后续相关操作** — 展示仅 `i` 参数不同的调用,以及同轮次、明确同文件的后续修改;标明关系依据、五类结果状态并可跳转证据。候选不自动关闭事件、不代表原问题已修复。[匹配边界](docs/diagnostics.md#后续相关操作)。
- **本机复核队列** — 用必填依据标记“需跟进”“预期失败”“其他验证已通过”,仅存当前浏览器;新证据使旧标记失效,人工判断不改写自动结果。无需账号或复核后端。[使用方式与存储边界](docs/diagnostics.md#本机复核闭环)。
- **复核迁移** — 预览并导出当前会话的有效复核,导入只接受完整证据匹配且本地为空的记录;不覆盖已有笔记,跳过过期或不匹配记录。明文 JSON 含手写依据,不自动复制日志。[迁移边界](docs/diagnostics.md#迁移复核记录)。
- **窄屏会话复核** — 小于 768px 时切换“会话列表 / 返回内容”,正文获得完整宽度,切换列表不清空当前复核草稿;平台栏横向滚动,证据跳转保留顶部导航,桌面继续双栏。[验收范围](docs/diagnostics.md#窄屏操作)。
Expand Down
12 changes: 6 additions & 6 deletions claims.json
Original file line number Diff line number Diff line change
Expand Up @@ -102,16 +102,16 @@
},
{
"id": "test-count",
"claim": "233 tests pass on Node's built-in test runner, the count docs/ROADMAP.md records for `npm test`.",
"value": "233",
"metric": "passing node:test cases (# tests 233 / # pass 233 / # fail 0)",
"method": "npm test → node --test test/*.test.js, run in the claims job after npm ci, and the TAP summary is asserted. The roadmap sentence ('233 tests on Node's built-in runner (`npm test`, 2026-09-23)') is verified by the run, not read back from the prose.",
"claim": "255 tests pass on Node's built-in test runner, the count docs/ROADMAP.md records for `npm test`.",
"value": "255",
"metric": "passing node:test cases (# tests 255 / # pass 255 / # fail 0)",
"method": "npm test → node --test test/*.test.js, run in the claims job after npm ci, and the TAP summary is asserted. The roadmap sentence ('255 tests on Node's built-in runner (`npm test`, 2026-09-23)') is verified by the run, not read back from the prose.",
"repro": "npm test 2>&1 | grep -E '^# (tests|pass|fail)'",
"evidence": "docs/ROADMAP.md",
"as_of": "2026-09-13",
"check": {
"cmd": "npm test 2>&1 | grep -E '^# (tests|pass|fail)'",
"expect": { "contains": ["# tests 233", "# pass 233", "# fail 0"] },
"expect": { "contains": ["# tests 255", "# pass 255", "# fail 0"] },
"timeout": 120
}
},
Expand Down Expand Up @@ -238,7 +238,7 @@
"check": {
"cmd": "node scripts/claims-receipts.mjs tests-node-only",
"expect": {
"equals": "17 files in test/ · 14 distinct requires: 11 node builtins, 3 relative, 0 third-party"
"equals": "18 files in test/ · 14 distinct requires: 11 node builtins, 3 relative, 0 third-party"
},
"timeout": 60
}
Expand Down
4 changes: 2 additions & 2 deletions docs/ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
- **Session browser** with tool-call inspection, trace/waterfall view, spawn tracking and message timeline
- **Prompt tooling** — extraction (noise filtered), template clustering with outcome attribution, Claude-powered rewrites, and a prompt library that installs entries as native slash commands
- **Global search** across all platforms, insights dashboard, incremental session backup
- **React + Vite frontend** served by an Express backend; 233 tests on Node's built-in runner (`npm test`, 2026-09-23), CI on Node 22
- **React + Vite frontend** served by an Express backend; 255 tests on Node's built-in runner (`npm test`, 2026-09-23), CI on Node 22
- **Evidence-backed failure events and local review** with full-result invalidation, evidence navigation and narrow-screen session layout

## Current priorities
Expand All @@ -19,7 +19,7 @@ The product direction is **review the coding-agent sessions you already have, wi
| --- | --- | --- |
| P0 | Make the new workflow immediately testable | A demo-only entry opens a clearly synthetic case: 7 pending records in 2 events, all evidence accessible, local review does not rewrite automatic results. Preserve the existing default demo and samples. |
| P1 | Current-session review portability implemented | Preview-only import and explicit plaintext download; exact identity/evidence matching, no overwrites, bounded schema and partial-failure reporting. Validate with synthetic migration and publish after CI; whole-history backup and path remapping remain out of scope. |
| P2 | Validate daily usefulness with the maintainer's own sessions | Record reviewed/follow-up/expected/alternative-verification counts and timed review tasks using a fixed rubric. Keep measurements local, separate unknowns and stale labels, and publish only consented aggregate evidence. Do not infer precision or time saved from event compression. |
| P2 | Explain what happened after a failure, then measure usefulness | Show only-i and same-turn/same-file follow-up candidates with explicit status and evidence, never automatic recovery. Validate unchanged diagnostic outcomes on frozen logs. Then measure owner review tasks using a fixed rubric; separate unknowns and stale labels, and do not infer precision or time saved from candidate counts. |
| P3 | Make releases reproducible for contributors | Keep clean-install tests, generated fixtures, documentation claims and release/package verification aligned. Add browser regression automation when it can run deterministically without personal logs. |

No launch dates or star-count targets are promised. Progress is gated on these observable outcomes. Physical-device/keyboard coverage and complex Trace/analytics layouts remain separate work, not implied by the session-screen checks.
Expand Down
12 changes: 12 additions & 0 deletions docs/diagnostics-verification.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,3 +83,15 @@ Verified using synthetic notes and isolated browser contexts only:
- Browser migration sent zero backend write requests; automatic failure/recovery counts remained unchanged. At 360px, the transfer panel had no horizontal overflow.

This does not verify real-device file pickers, simultaneous cross-tab transaction safety or actual productivity gains. The file is unencrypted human text, not signed evidence; only current exact matches can be restored. Detailed local run artifacts remain ignored and are not published.

## Follow-up evidence acceptance

The follow-up increment adds evidence-only relationships, not automatic recovery. There are 22 new tests in `test/follow-up-evidence.test.js`, 125 combined diagnostic/review tests, and 255 tests in the full suite. The ordinary and hosted-demo UI builds pass; earlier counts above are historical release receipts.

Synthetic tests cover all five candidate states, only-top-level-`i` differences, complete argument comparison, exact tools, same-turn edit/write paths, explicit relative-path directories, missing/ambiguous fields, pre-existing parallel calls, missing results, call-turn vs result-turn, deduplication, full candidate output in review fingerprints, and unchanged exact-match recovery. The scripted demo `node scripts/demo-follow-up.cjs` exercises seven only-`i` candidates, one same-file candidate and excluded file/turn counterexamples. The browser checked all five labels, five-item initial display plus loading the rest, original result jumps, no page-navigation movement, stale review on newly appended candidate and no narrow-screen overflow.

A fixed, previously inspected set of 15 real sessions (five each from OMP, Codex and Claude Code) with 2,024 tool results was re-read by frozen-length/hash verification. Automatic results stayed at **142 failures, 141 pending records, 76 events and 1 recovered record**, including unchanged event members. An independent relation enumeration agreed with every candidate. It found **3 only-`i` candidates in OMP: one failure, one running result and one success**; Codex/Claude had none under these strict rules. No same-file relation qualified in this set: relevant observed paths lacked explicit absolute context or the calls belonged to later user turns. The code did not weaken criteria to inflate matches.

Storage keys stayed stable. Only the three events with added candidates acquired new review fingerprints; no-candidate events retained their previous fingerprint contract. No real review labels were persisted. Raw logs, parameters and detailed sample identifiers remain private; the regression is not an independent benchmark, global accuracy estimate or time-saving measurement.

The hosted synthetic sample now includes one later successful edit of the same absolute file path with different arguments. It remains **2 events / 7 pending records / 1 recovered record**. Browser checks verified the relation caveat, source result, desktop/mobile rendering and zero static-demo API requests.
44 changes: 44 additions & 0 deletions docs/diagnostics.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,37 @@ Run `npm start` from the built checkout. Select a session and open its **消息*

Start with repeated operations, open the latest failure, and inspect the surrounding calls. A card is a reason to review the transcript—not an instruction to rerun a possibly destructive command. Equivalent commands, alternate verification and external fixes still require your judgment.

## Follow-up evidence candidates

When a failure event has matching later operations, expand **后续相关操作**. These are leads for your review, **not automatic recovery or proof that the original task passed**. Each candidate shows its relationship, call/result message positions, call's user turn, tool outcome and source evidence, arguments, and a jump to the original result. Five are shown initially; the rest can be loaded without filtering out unsuccessful results.

Two relationships are supported:

| Relationship | Required evidence | Not inferred |
| --- | --- | --- |
| Only top-level `i` differs | Same exact tool, equal non-empty remaining arguments, but different full arguments. Object key order is ignored; values and array order are retained. May cross user turns within this transcript. | That `i` is semantically irrelevant, or that the later call fixes the earlier problem. |
| Same-file modification | Explicit `edit`, `Edit`, `write`, `Write` or `MultiEdit` calls, same call-origin user turn, changed parameters, and the same literal `path`/`file_path` plus recorded working-directory fields. | Paths hidden in shell commands/patches, different turns, symlinks, path normalization, or implicit directory changes. |

Both require a recorded result and a call started **after the event's last pending failure**. Results from pre-existing parallel calls, orphan results, missing arguments and calls without results are excluded. Same-file matching requires absolute paths, or identical explicit absolute `cwd`/`workdir`/`working_directory` for relative paths; ambiguous fields are rejected. Different directory field names are not treated as aliases.

Status labels distinguish **success (tool result), failure, running, cancelled/stopped and unknown**. Source fields are shown, not inferred from optimistic prose. A running background job is not success. Platform adapters and automatic failure/recovery matching remain unchanged, including preserving `i` in automatic retry comparison. A same-file edit that returned success may have changed something else entirely.

New or changed candidate evidence requires human review again: the event's storage key remains stable, but the review fingerprint includes complete candidate results and arguments. Existing notes become stale only for affected events; events without candidates retain their old fingerprint contract. Review-transfer files still contain only hashes and handwritten notes, not raw candidate logs.

### Try all five states locally

From a built source checkout:

```sh
node scripts/demo-follow-up.cjs
```

Select its OMP `[Synthetic]` session. The first bash failure has seven only-`i` candidates covering the five outcome states; an edit failure has one same-file candidate. An unrelated file and a different user turn are excluded. **Three automatic failure events remain**, even when a candidate is successful. Save a review and type `n` in the terminal to append another candidate: that review becomes stale, without changing automatic failure/recovery counts. Type `q` or Ctrl-C to stop the isolated demo.

The hosted **Try diagnostics** sample also includes a successful same-file edit with changed arguments. Its existing seven pending failure records and two events do not disappear. Both demos are synthetic; no logged command is executed.

![Synthetic follow-up evidence](../screenshots/follow-up-evidence.png)

## Local review workflow

The automatic result and your judgment are separate. The header always reports automatic events and pending failure records. The review filters show which of those events you have inspected:
Expand Down Expand Up @@ -122,6 +153,7 @@ These checks cover navigation, session messages and the review workflow—not co
node --test test/diagnostic-events.test.js test/diagnostics.test.js test/omp-outcome.test.js
node --test test/diagnostic-reviews.test.js
node --test test/review-transfer.test.js
node --test test/follow-up-evidence.test.js
npm test
npm run build:ui
npm run lint
Expand Down Expand Up @@ -181,6 +213,18 @@ The local frozen regression set contained 30 sessions and 9,076 tool results. Gr

本轮的 373 个真实冻结事件只使用**内存中的合成测试标记**检验隔离与失效,没有替你判断真实事件,也没有把这些测试标记写成真实复核。完整证据见 [本机复核验收](diagnostics-verification.md)。

## 后续相关操作

失败事件下若有候选,可展开“后续相关操作”,查看匹配依据、五类状态、调用/结果位置、参数摘要及原始结果。**它不自动关闭事件,不代表任务已通过。**

- “仅 i 不同”:同工具,只有顶层 `i` 不同,其余完整参数相同,可跨当前会话内的用户轮次;不等于认定 `i` 无语义影响。
- “同文件修改”:仅限明确 edit/write/MultiEdit 工具、同调用轮次、相同记录路径及显式工作目录;允许不同修改参数,不推断它们是同一修复。
- 都必须在事件最后一次失败之后发起且已有结果。此前已启动的并行调用、无结果、缺参数、孤立结果不参与。
- 相对路径必须有相同的显式绝对工作目录;没有就不猜。不同字段名、隐式 cwd、软链、跨轮次修改、命令里的文件名都不自动关联。
- 成功、失败、执行中、取消、未知均有标签和字段依据。候选新增或变化会使受影响旧复核过期;不会改变自动失败/恢复计数,也不会把原始日志加入迁移文件。

源码运行 `node scripts/demo-follow-up.cjs` 可体验五类状态、无关文件反例和实时追加候选。页面里保存复核后,终端输入 `n` 验证重审;`q` 退出。在线 Demo 的合成诊断案例也有一条同文件成功修改,但仍保留原来的失败事件。

## 窄屏操作

- 小于 768px 时,点击“会话列表”进入导航,选择当前或其他会话后返回正文;“返回内容”或 Escape 也可关闭列表。桌面仍是双栏。
Expand Down
Loading
Loading