Skip to content

feat(core): shared agent-transcript layer core/sessions#774

Merged
RaghavChamadiya merged 1 commit into
mainfrom
feat/sessions-transcript-layer
Jul 11, 2026
Merged

feat(core): shared agent-transcript layer core/sessions#774
RaghavChamadiya merged 1 commit into
mainfrom
feat/sessions-transcript-layer

Conversation

@RaghavChamadiya

Copy link
Copy Markdown
Member

What changed

New package core/sessions/: one normalized transcript stream for every consumer. The three distill miners (missed savings, wasteful re-reads, correction mining) each carried a private copy of Claude Code JSONL parsing: munged-cwd discovery, substring gating, tool_use/tool_result pairing, timestamp handling. That parsing now lives once behind a harness-agnostic interface, so the next transcript consumer starts from Events instead of raw JSONL, and supporting another coding agent means one new adapter file.

  • events.py: the Event / ToolUse / ToolResult model, the user-interrupt marker, and iter_deduped_usage. One API message spans several transcript lines and repeats its usage object on each; summing raw lines overcounts roughly 2.6x, so usage is deduped by message id.
  • adapters/base.py: the HarnessAdapter contract (discover() + normalize()) with shared iter_events(). Consumers pass a raw-string prefilter so multi-hundred-KB lines are still skipped before any JSON parsing, matching the miners' existing perf posture.
  • adapters/claude_code.py: first adapter. Field handling verified against real transcripts on this machine: sidechain (subagent) lines stay in the stream and are flagged, isMeta / isCompactSummary lines are flagged so prompt-derived signals can skip them, and the top-level toolUseResult (dict on success, exit-code string on failure) rides on ToolResult.payload.
  • cursor.py: per-file byte-offset + mtime cursors with iter_new_events, for incremental scans over append-only transcripts. Truncated or replaced files restart from zero; a trailing line without a newline is a write in progress and waits for the next pass. Ships tested but unconsumed; the miners keep their windowed rescan semantics.
  • A README in the package documents the contract, the schema gotchas, and how to add a harness.

The three miners are now consumers, not parsers. Public APIs, report shapes, window semantics, and test seams are all unchanged. One incidental fix: dispatch now keys on the parsed event shape rather than raw substrings, so a tool result whose output happened to contain the literal text tool_use is no longer dropped.

Validation

  • Existing distill suite green with zero test edits (569 tests), plus 19 new adapter/cursor tests over a checked-in real-shaped JSONL fixture (split usage blocks, sidechain, compact summary, interrupt marker, malformed lines).
  • Full unit suite: 6,943 passed.
  • Dogfood equivalence: snapshotted this repo's real transcript directory (342 files, 499 MB), ran all three scans on main and on this branch with a pinned clock; the JSON reports are byte-identical.

Extract the transcript parsing that missed.py, missed_mcp.py, and
corrections.py each reimplemented into one normalized Event stream behind
a HarnessAdapter interface:

- events.py: Event / ToolUse / ToolResult model, interrupt marker, and
  iter_deduped_usage (one API message repeats its usage object per content
  block; summing raw lines overcounts ~2.6x).
- adapters/claude_code.py: munged-cwd discovery under ~/.claude/projects/
  plus line normalization; field names verified against real transcripts.
  Sidechain lines stay in the stream and are flagged, as are isMeta and
  isCompactSummary lines.
- adapters/base.py: discover() + normalize() contract with shared
  iter_events(); a raw-string prefilter keeps the cheap substring gates
  the miners relied on, so huge lines are still skipped before json.loads.
- cursor.py: per-file byte-offset + mtime cursors with iter_new_events for
  incremental scans (append-only files resume at a line boundary; truncated
  files restart; a trailing partial line waits). Tested but not yet
  consumed; session decision mining is the first consumer.

The three distill miners now consume the shared layer; their public APIs,
report shapes, and window semantics are unchanged. Dispatch now keys on the
parsed event shape instead of raw substrings, which also fixes the rare
dropped result when a tool output contained the literal text tool_use.

Validation: distill suite green unchanged (569), new adapter/cursor tests
(19), full unit suite 6943 passed. Dogfood: all three miner reports are
byte-identical pre/post refactor on a 342-file 499 MB snapshot of this
repo's real transcripts.
@repowise-bot

repowise-bot Bot commented Jul 11, 2026

Copy link
Copy Markdown

✅ Health: 7.6 (unchanged)

📋 At a glance
4 hotspots touched · 4 new findings introduced · 1 file with recent fix history. Scoped to packages.

🚨 Change risk: high (riskier than 83% of this repo's commits · raw 9.6/10)
This change's risk is driven by:

  • more lines added than baseline
  • more scattered than baseline

🩹 Review priority (files here with the most recent bug-fix history — defects cluster, so review these first)

🔎 More signals (1)

🔥 Hotspots touched (4)

  • .../distill/missed.py — 2 commits/90d, 9 dependents · primary owner: Raghav Chamadiya (100%)
  • .../distill/session_model.py — 2 commits/90d, 3 dependents · primary owner: Raghav Chamadiya (100%)
  • .../distill/corrections.py — 1 commits/90d, 2 dependents · primary owner: Raghav Chamadiya (100%)
1 more
  • .../distill/missed_mcp.py — 1 commits/90d, 3 dependents · primary owner: Raghav Chamadiya (100%)

📊 Full report · ⭐ Star Repowise · 📥 Install bot · Last updated 2026-07-11 06:31 UTC
Silence on a single PR with [skip repowise] in the title · Per-repo toggle on repowise.dev/settings?tab=bot

@RaghavChamadiya RaghavChamadiya merged commit 02dbf88 into main Jul 11, 2026
7 checks passed
@RaghavChamadiya RaghavChamadiya deleted the feat/sessions-transcript-layer branch July 11, 2026 06:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants