feat(core): shared agent-transcript layer core/sessions#774
Merged
Conversation
Extract the transcript parsing that missed.py, missed_mcp.py, and corrections.py each reimplemented into one normalized Event stream behind a HarnessAdapter interface: - events.py: Event / ToolUse / ToolResult model, interrupt marker, and iter_deduped_usage (one API message repeats its usage object per content block; summing raw lines overcounts ~2.6x). - adapters/claude_code.py: munged-cwd discovery under ~/.claude/projects/ plus line normalization; field names verified against real transcripts. Sidechain lines stay in the stream and are flagged, as are isMeta and isCompactSummary lines. - adapters/base.py: discover() + normalize() contract with shared iter_events(); a raw-string prefilter keeps the cheap substring gates the miners relied on, so huge lines are still skipped before json.loads. - cursor.py: per-file byte-offset + mtime cursors with iter_new_events for incremental scans (append-only files resume at a line boundary; truncated files restart; a trailing partial line waits). Tested but not yet consumed; session decision mining is the first consumer. The three distill miners now consume the shared layer; their public APIs, report shapes, and window semantics are unchanged. Dispatch now keys on the parsed event shape instead of raw substrings, which also fixes the rare dropped result when a tool output contained the literal text tool_use. Validation: distill suite green unchanged (569), new adapter/cursor tests (19), full unit suite 6943 passed. Dogfood: all three miner reports are byte-identical pre/post refactor on a 342-file 499 MB snapshot of this repo's real transcripts.
|
✅ Health: 7.6 (unchanged) 📋 At a glance 🚨 Change risk: high (riskier than 83% of this repo's commits · raw 9.6/10)
🩹 Review priority (files here with the most recent bug-fix history — defects cluster, so review these first)
🔎 More signals (1)🔥 Hotspots touched (4)
1 more
📊 Full report · ⭐ Star Repowise · 📥 Install bot · Last updated 2026-07-11 06:31 UTC |
swati510
approved these changes
Jul 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
New package
core/sessions/: one normalized transcript stream for every consumer. The three distill miners (missed savings, wasteful re-reads, correction mining) each carried a private copy of Claude Code JSONL parsing: munged-cwd discovery, substring gating, tool_use/tool_result pairing, timestamp handling. That parsing now lives once behind a harness-agnostic interface, so the next transcript consumer starts from Events instead of raw JSONL, and supporting another coding agent means one new adapter file.events.py: theEvent/ToolUse/ToolResultmodel, the user-interrupt marker, anditer_deduped_usage. One API message spans several transcript lines and repeats its usage object on each; summing raw lines overcounts roughly 2.6x, so usage is deduped by message id.adapters/base.py: theHarnessAdaptercontract (discover()+normalize()) with sharediter_events(). Consumers pass a raw-string prefilter so multi-hundred-KB lines are still skipped before any JSON parsing, matching the miners' existing perf posture.adapters/claude_code.py: first adapter. Field handling verified against real transcripts on this machine: sidechain (subagent) lines stay in the stream and are flagged,isMeta/isCompactSummarylines are flagged so prompt-derived signals can skip them, and the top-leveltoolUseResult(dict on success, exit-code string on failure) rides onToolResult.payload.cursor.py: per-file byte-offset + mtime cursors withiter_new_events, for incremental scans over append-only transcripts. Truncated or replaced files restart from zero; a trailing line without a newline is a write in progress and waits for the next pass. Ships tested but unconsumed; the miners keep their windowed rescan semantics.The three miners are now consumers, not parsers. Public APIs, report shapes, window semantics, and test seams are all unchanged. One incidental fix: dispatch now keys on the parsed event shape rather than raw substrings, so a tool result whose output happened to contain the literal text
tool_useis no longer dropped.Validation