Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions claims.json
Original file line number Diff line number Diff line change
Expand Up @@ -102,16 +102,16 @@
},
{
"id": "test-count",
"claim": "214 tests pass on Node's built-in test runner, the count docs/ROADMAP.md records for `npm test`.",
"value": "214",
"metric": "passing node:test cases (# tests 214 / # pass 214 / # fail 0)",
"method": "npm test → node --test test/*.test.js, run in the claims job after npm ci, and the TAP summary is asserted. The roadmap sentence ('214 tests on Node's built-in runner (`npm test`, 2026-09-23)') is verified by the run, not read back from the prose.",
"claim": "216 tests pass on Node's built-in test runner, the count docs/ROADMAP.md records for `npm test`.",
"value": "216",
"metric": "passing node:test cases (# tests 216 / # pass 216 / # fail 0)",
"method": "npm test → node --test test/*.test.js, run in the claims job after npm ci, and the TAP summary is asserted. The roadmap sentence ('216 tests on Node's built-in runner (`npm test`, 2026-09-23)') is verified by the run, not read back from the prose.",
"repro": "npm test 2>&1 | grep -E '^# (tests|pass|fail)'",
"evidence": "docs/ROADMAP.md",
"as_of": "2026-09-13",
"check": {
"cmd": "npm test 2>&1 | grep -E '^# (tests|pass|fail)'",
"expect": { "contains": ["# tests 214", "# pass 214", "# fail 0"] },
"expect": { "contains": ["# tests 216", "# pass 216", "# fail 0"] },
"timeout": 120
}
},
Expand Down
19 changes: 16 additions & 3 deletions docs/ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,28 @@
- **Session browser** with tool-call inspection, trace/waterfall view, spawn tracking and message timeline
- **Prompt tooling** — extraction (noise filtered), template clustering with outcome attribution, Claude-powered rewrites, and a prompt library that installs entries as native slash commands
- **Global search** across all platforms, insights dashboard, incremental session backup
- **React + Vite frontend** served by an Express backend; 214 tests on Node's built-in runner (`npm test`, 2026-09-23), CI on Node 22
- **React + Vite frontend** served by an Express backend; 216 tests on Node's built-in runner (`npm test`, 2026-09-23), CI on Node 22
- **Evidence-backed failure events and local review** with full-result invalidation, evidence navigation and narrow-screen session layout

## Roadmap
## Current priorities

The product direction is **review the coding-agent sessions you already have, with evidence**. First prove that a useful review can be completed locally; then make it easy to try and share. Stars, downloads and screenshots alone do not establish usefulness. This order replaces the earlier feature-first ordering below.

| Priority | Outcome | Acceptance, not a promise |
| --- | --- | --- |
| P0 | Make the new workflow immediately testable | A demo-only entry opens a clearly synthetic case: 7 pending records in 2 events, all evidence accessible, local review does not rewrite automatic results. Preserve the existing default demo and samples. |
| P1 | Make review work portable without exposing logs | Design explicit user-controlled review export/import, validate schema and evidence fingerprints, show a preview of exactly what leaves the browser, and reject stale/mismatched records. No automatic upload or raw-log export by default. |
| P2 | Validate daily usefulness with the maintainer's own sessions | Record reviewed/follow-up/expected/alternative-verification counts and timed review tasks using a fixed rubric. Keep measurements local, separate unknowns and stale labels, and publish only consented aggregate evidence. Do not infer precision or time saved from event compression. |
| P3 | Make releases reproducible for contributors | Keep clean-install tests, generated fixtures, documentation claims and release/package verification aligned. Add browser regression automation when it can run deterministically without personal logs. |

No launch dates or star-count targets are promised. Progress is gated on these observable outcomes. Physical-device/keyboard coverage and complex Trace/analytics layouts remain separate work, not implied by the session-screen checks.

## Existing backlog

- [x] **Publish to npm** so `npx agent-xray` works without cloning — automated from GitHub releases (v1.14.0+)
- [x] **Hosted live demo** on GitHub Pages with clearly-labeled synthetic session data, so people can try the UI before installing
- [x] **Gemini CLI adapter** (`~/.gemini/tmp`) — shipped (#5)
- [ ] **More platform adapters** — opencode and Aider are the most-requested log formats; the per-platform module layout in `lib/platforms/` is designed for this
- [ ] **More platform adapters** — opencode and Aider are candidates, not a measured demand ranking. Require representative logs and parser acceptance tests before prioritizing them over the review workflow.
- [x] **Session export & sharing** — render a session (with tool calls) to a standalone Markdown/HTML file you can attach to a bug report or blog post (#6)
- [ ] **Cost & token analytics** — per-session and per-day token spend, broken down by model, building on the token counts we already parse for the summary panel
- [ ] **Watch mode ergonomics** — highlight the currently-active session and surface "agent is waiting for input" state in the sidebar
Expand Down
10 changes: 10 additions & 0 deletions docs/diagnostics-verification.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,3 +57,13 @@ The preceding private, already-inspected regression set had 30 sessions and 9,07
- Existing aggregate statistics retain their original error-field semantics and may differ from diagnostic counts.
- No physical iOS/Android device, Safari or software-keyboard certification is claimed. Other complex pages are not fully covered by the narrow-screen checks.
- Existing lint findings and the Vite large-chunk warning remain; no claim of zero technical debt is made.

## Release and hosted walkthrough follow-up

The feature release is available as [v1.18.0](https://github.com/alloevil/AgentXRay/releases/tag/v1.18.0). Its [Publish run](https://github.com/alloevil/AgentXRay/actions/runs/35880348489) succeeded with a package tarball and provenance attached. An isolated installation from `https://registry.npmjs.org` returned CLI version `1.18.0`; the installed server served a synthetic OMP failure with normalized status and the bundled review UI. The initial registry query preceded npm processing and returned 404; success was recorded only after the registry and installation became available.

A separate hosted-demo increment adds the explicitly synthetic `0199demo-diagnostics` session and a **Try diagnostics** entry, leaving the default Claude demo and previous samples intact. Raw parser output and bundled messages are asserted equal by tests; 216 total Node tests now pass (two additional hosted-fixture tests). The existing 214-test count above records the release acceptance, not the updated suite size.

The hosted sample has 8 historical failures: one has a matching successful retry, leaving 7 pending records grouped into 2 events (6 edit failures and 1 nested search failure). A background start stays running. Six individual edit results, evidence jumps and browser-local review were verified; no backend API requests were made during the static walkthrough. Fixture regeneration produced identical bytes twice.

The demo-only guide is height-limited on short screens. Expanded guide checks leave about 102px for messages at 360×640 and 224px at 360×800; at 740×360 landscape only 35px remains, so portrait use is recommended. This does not change the normal local application's layout or imply physical-device certification.
4 changes: 4 additions & 0 deletions docs/diagnostics.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ This guide covers the React UI. The synthetic terminal walkthrough uses a source

## Try it without sharing your logs

For a no-install tour, open the [hosted demo](https://alloevil.github.io/AgentXRay/) and click **体验失败复核 / Try diagnostics**. Its synthetic sample has 7 pending failure records in 2 events (6 repeated edits and 1 search failure), plus an earlier automatically recovered test failure. Open the first/last/all evidence and record a browser-local review note. This is a static sample: no live results are appended and no actual tool commands run. The terminal walkthrough below is a separate, larger fixture for live updates.

From the repository root, after installing the root and frontend dependencies:

```sh
Expand Down Expand Up @@ -108,6 +110,8 @@ The local frozen regression set contained 30 sessions and 9,076 tool results. Gr

**目标:先找到值得复查的重复操作,再追溯证据,而不是把几百条失败强行解释成几个根因。**

免安装体验:[在线 Demo](https://alloevil.github.io/AgentXRay/),点击“体验失败复核 / Try diagnostics”。这个独立合成案例将 7 条待复查记录聚为 2 个事件(6 次同参 edit 失败、1 次搜索失败),并展示一次早先测试失败的自动恢复。可查看全部证据、填写本浏览器的人工复核;它是静态示例,不追加真实结果。下面的本机终端演示则使用更大的合成日志来验证实时变化。

从源码仓库运行 `npm run build:ui`,再运行 `node scripts/demo-diagnostics.cjs`。打开终端打印的地址,选择 OMP 的 `[Synthetic]` 会话:

- 初始为 **72 条待复查记录 → 7 个事件**,66 条同参 edit 失败集中在第一张卡片;默认展示 5 个事件,可以继续加载。
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
{"type":"session","id":"0199demo-diagnostics","timestamp":"2026-09-23T08:00:00.000Z","cwd":"/demo/diagnostics"}
{"type":"title","title":"[Synthetic] Failure review: 7 records → 2 events"}
{"type":"message","id":"demo-review-user","timestamp":"2026-09-23T08:00:01.000Z","message":{"role":"user","content":[{"type":"text","text":"[Synthetic demo] Review repeated edit failures and a search failure. Inspect the evidence before recording a human review. These logs are fictional; no tool commands were executed."}]}}
{"type":"message","id":"demo-test-first-call","timestamp":"2026-09-23T08:00:02.000Z","message":{"role":"assistant","content":[{"type":"toolCall","id":"demo-test-first","name":"bash","arguments":{"command":"npm test","cwd":"/demo/diagnostics"}}]}}
{"type":"message","id":"demo-test-first-result","timestamp":"2026-09-23T08:00:03.000Z","message":{"role":"toolResult","toolCallId":"demo-test-first","toolName":"bash","isError":true,"details":{"exitCode":1,"wallTimeMs":100},"content":[{"type":"text","text":"Synthetic test failure: expected 2, got 1."}]}}
{"type":"message","id":"demo-test-retry-call","timestamp":"2026-09-23T08:00:04.000Z","message":{"role":"assistant","content":[{"type":"toolCall","id":"demo-test-retry","name":"bash","arguments":{"command":"npm test","cwd":"/demo/diagnostics"}}]}}
{"type":"message","id":"demo-test-retry-result","timestamp":"2026-09-23T08:00:05.000Z","message":{"role":"toolResult","toolCallId":"demo-test-retry","toolName":"bash","isError":false,"details":{"wallTimeMs":100},"content":[{"type":"text","text":"Synthetic test retry passed. This does not verify the later edits."}]}}
{"type":"message","id":"demo-edit-1-call","timestamp":"2026-09-23T08:00:10.000Z","message":{"role":"assistant","content":[{"type":"toolCall","id":"demo-edit-1","name":"edit","arguments":{"path":"/demo/diagnostics/config.ts","oldText":"timeout: 10","newText":"timeout: 30"}}]}}
{"type":"message","id":"demo-edit-1-result","timestamp":"2026-09-23T08:00:11.000Z","message":{"role":"toolResult","toolCallId":"demo-edit-1","toolName":"edit","isError":true,"content":[{"type":"text","text":"Synthetic edit failure 1: exact old text not found. Inspect the file before repeating this operation."}]}}
{"type":"message","id":"demo-edit-2-call","timestamp":"2026-09-23T08:00:20.000Z","message":{"role":"assistant","content":[{"type":"toolCall","id":"demo-edit-2","name":"edit","arguments":{"path":"/demo/diagnostics/config.ts","oldText":"timeout: 10","newText":"timeout: 30"}}]}}
{"type":"message","id":"demo-edit-2-result","timestamp":"2026-09-23T08:00:21.000Z","message":{"role":"toolResult","toolCallId":"demo-edit-2","toolName":"edit","isError":true,"content":[{"type":"text","text":"Synthetic edit failure 2: exact old text still not found."}]}}
{"type":"message","id":"demo-edit-3-call","timestamp":"2026-09-23T08:00:30.000Z","message":{"role":"assistant","content":[{"type":"toolCall","id":"demo-edit-3","name":"edit","arguments":{"path":"/demo/diagnostics/config.ts","oldText":"timeout: 10","newText":"timeout: 30"}}]}}
{"type":"message","id":"demo-edit-3-result","timestamp":"2026-09-23T08:00:31.000Z","message":{"role":"toolResult","toolCallId":"demo-edit-3","toolName":"edit","isError":true,"content":[{"type":"text","text":"Synthetic edit failure 3: no matching text; arguments have not changed."}]}}
{"type":"message","id":"demo-edit-4-call","timestamp":"2026-09-23T08:00:40.000Z","message":{"role":"assistant","content":[{"type":"toolCall","id":"demo-edit-4","name":"edit","arguments":{"path":"/demo/diagnostics/config.ts","oldText":"timeout: 10","newText":"timeout: 30"}}]}}
{"type":"message","id":"demo-edit-4-result","timestamp":"2026-09-23T08:00:41.000Z","message":{"role":"toolResult","toolCallId":"demo-edit-4","toolName":"edit","isError":true,"content":[{"type":"text","text":"Synthetic edit failure 4: exact match missing; no edit was applied."}]}}
{"type":"message","id":"demo-edit-5-call","timestamp":"2026-09-23T08:00:50.000Z","message":{"role":"assistant","content":[{"type":"toolCall","id":"demo-edit-5","name":"edit","arguments":{"path":"/demo/diagnostics/config.ts","oldText":"timeout: 10","newText":"timeout: 30"}}]}}
{"type":"message","id":"demo-edit-5-result","timestamp":"2026-09-23T08:00:51.000Z","message":{"role":"toolResult","toolCallId":"demo-edit-5","toolName":"edit","isError":true,"content":[{"type":"text","text":"Synthetic edit failure 5: expected old text absent."}]}}
{"type":"message","id":"demo-edit-6-call","timestamp":"2026-09-23T08:01:00.000Z","message":{"role":"assistant","content":[{"type":"toolCall","id":"demo-edit-6","name":"edit","arguments":{"path":"/demo/diagnostics/config.ts","oldText":"timeout: 10","newText":"timeout: 30"}}]}}
{"type":"message","id":"demo-edit-6-result","timestamp":"2026-09-23T08:01:01.000Z","message":{"role":"toolResult","toolCallId":"demo-edit-6","toolName":"edit","isError":true,"content":[{"type":"text","text":"Synthetic edit failure 6: unchanged operation failed again. Grouping retains all six results; it does not prove a shared root cause."}]}}
{"type":"message","id":"demo-search-call","timestamp":"2026-09-23T08:01:05.000Z","message":{"role":"assistant","content":[{"type":"toolCall","id":"demo-search","name":"web_search","arguments":{"query":"synthetic timeout configuration documentation"}}]}}
{"type":"message","id":"demo-search-result","timestamp":"2026-09-23T08:01:06.000Z","message":{"role":"toolResult","toolCallId":"demo-search","toolName":"web_search","isError":false,"details":{"error":"Synthetic provider failure: no search result available."},"content":[{"type":"text","text":"Synthetic search failed. OMP details.error records the failure even though the outer isError is false."}]}}
{"type":"message","id":"demo-background-call","timestamp":"2026-09-23T08:01:10.000Z","message":{"role":"assistant","content":[{"type":"toolCall","id":"demo-background","name":"bash","arguments":{"command":"npm run verify-config","cwd":"/demo/diagnostics","async":true}}]}}
{"type":"message","id":"demo-background-result","timestamp":"2026-09-23T08:01:11.000Z","message":{"role":"toolResult","toolCallId":"demo-background","toolName":"bash","isError":false,"details":{"async":{"state":"running","jobId":"synthetic-background"}},"content":[{"type":"text","text":"Synthetic verification was backgrounded. There is no completion result in this sample."}]}}
{"type":"message","id":"demo-review-summary","timestamp":"2026-09-23T08:01:12.000Z","message":{"role":"assistant","content":[{"type":"text","text":"Synthetic review exercise: inspect the six edit results and the search error, then record a follow-up note with evidence. The earlier test success does not validate the edits; a running background job is not a success. This static sample does not append live results."}]}}
27 changes: 25 additions & 2 deletions frontend/src/App.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ export default function App() {
useVersionPoller();
const view = useAppStore((s) => s.view);
const setView = useAppStore((s) => s.setView);
const platform = useAppStore((state) => state.platform);
const selectedSession = useAppStore((state) => state.selectedSessionId);
const [navigationOpen, setNavigationOpen] = useState(false);
const navigationToggle = useRef<HTMLButtonElement>(null);
const navigationPanel = useRef<HTMLDivElement>(null);
Expand Down Expand Up @@ -69,8 +71,20 @@ export default function App() {
<TooltipProvider delayDuration={300}>
<div className="flex h-full flex-col overflow-hidden">
{DEMO ? (
<div className="flex shrink-0 flex-wrap items-center justify-center gap-2 border-b border-[#e3b341]/40 bg-[#e3b341]/15 px-3 py-1.5 text-center text-xs text-[#e3b341]">
<span>🧪 Demo mode — synthetic sample data (not real user sessions). Inspect your own agent logs:</span>
<div className="flex max-h-[24dvh] shrink-0 flex-wrap items-center justify-center gap-2 overflow-y-auto border-b border-[#e3b341]/40 bg-[#e3b341]/15 px-3 py-1.5 text-center text-xs text-[#e3b341] md:max-h-none [@media(max-height:500px)]:max-h-10">
<span>🧪 Demo — synthetic data, not real sessions.</span>
<button type="button"
className="min-h-9 rounded border border-[#e3b341]/60 bg-black/20 px-2 py-1 font-medium hover:bg-black/40"
onClick={() => {
const store = useAppStore.getState();
store.setPlatform('omp');
store.setSelectedSessionId('0199demo-diagnostics');
store.setSessionView('messages');
store.setView('sessions');
returnToContent();
}}>
体验失败复核 / Try diagnostics
</button>
<button
type="button"
title="Copy install command"
Expand All @@ -93,6 +107,15 @@ export default function App() {
>
GitHub
</a>
{platform === 'omp' && selectedSession === '0199demo-diagnostics' && view === 'sessions' ? (
<details className="w-full max-w-3xl text-left leading-5" data-testid="diagnostics-demo-guide">
<summary className="cursor-pointer text-center">演示步骤 / Guide: 7 failure records → 2 events</summary>
<p>6 次同参 edit 失败 + 1 次搜索失败。点击首条、末条或全部证据,再写一条人工复核依据。早先测试通过只恢复了那次测试,不验证后续修改。</p>
<p>Inspect the first/last/all evidence, then record a human review. Notes stay in this browser; the automatic result does not change. This static sample does not append live results.</p>
<a href="https://github.com/alloevil/AgentXRay/blob/master/docs/diagnostics.md" target="_blank" rel="noreferrer"
className="underline underline-offset-2">本机使用与判定边界 / Local walkthrough and limits</a>
</details>
) : null}
</div>
) : null}
<PlatformBar />
Expand Down
Loading
Loading