diff --git a/.github/workflows/validate.yml b/.github/workflows/validate.yml index ccf7251..a3cde06 100644 --- a/.github/workflows/validate.yml +++ b/.github/workflows/validate.yml @@ -29,5 +29,14 @@ jobs: - name: Validate session event projection run: python3 scripts/validate-session-events.py + - name: Validate session digest caching + run: python3 scripts/validate-session-digest.py + + - name: Validate session candidate filters + run: python3 scripts/validate-session-candidates.py + - name: Compile validator scripts - run: python3 -m py_compile scripts/validate-skill-package.py scripts/validate-fixtures.py scripts/validate-trigger-matrix.py scripts/validate-session-events.py scripts/session-events.py + run: python3 -m py_compile scripts/validate-skill-package.py scripts/validate-fixtures.py scripts/validate-trigger-matrix.py scripts/validate-session-events.py scripts/validate-session-digest.py scripts/validate-session-candidates.py scripts/validate-handoff.py + + - name: Compile packaged runtime scripts + run: python3 -m py_compile skills/agent-session-resume/scripts/session-events.py skills/agent-session-resume/scripts/session-candidates.py skills/agent-session-resume/scripts/session-digest.py skills/agent-session-resume/scripts/skill-provenance.py diff --git a/.gitignore b/.gitignore index 43ae0e2..992d43b 100644 --- a/.gitignore +++ b/.gitignore @@ -1,2 +1,5 @@ __pycache__/ *.py[cod] + +# digest sidecars produced by session-digest.py +*.digest.json diff --git a/README.md b/README.md index e28d29c..3830e59 100644 --- a/README.md +++ b/README.md @@ -34,8 +34,15 @@ skills/ codex.md cursor.md opencode.md + scripts/ + session-candidates.py + session-digest.py + session-events.py + skill-provenance.py ``` +The runtime helper scripts live inside the skill folder so they ship with every install (for example `~/.claude/skills/agent-session-resume/scripts/`). They are python3-stdlib-only and runnable from any working directory; see the [cookbook](docs/Cookbook.md) for usage, including session candidate time-window/cwd filters and digest sidecar caching. Repo-root `scripts/` holds the CI-only validators. + ## Install The quickest way to install is the [`skills`](https://github.com/vercel-labs/skills) CLI (`npx skills`). It reads the canonical `skills/agent-session-resume` folder from this repo and installs it for whichever agent you target — no manual cloning or copying. @@ -170,6 +177,9 @@ Run the package and fixture validators: python3 scripts/validate-skill-package.py python3 scripts/validate-fixtures.py python3 scripts/validate-trigger-matrix.py +python3 scripts/validate-session-events.py +python3 scripts/validate-session-digest.py +python3 scripts/validate-session-candidates.py claude plugin validate . claude plugin validate .claude-plugin/plugin.json claude plugin validate .claude-plugin/marketplace.json diff --git a/docs/Cookbook.md b/docs/Cookbook.md index 5b7a999..a729ec6 100644 --- a/docs/Cookbook.md +++ b/docs/Cookbook.md @@ -349,15 +349,17 @@ Continue from ./handoff.md. ## Helper Scripts -The repo includes small local helpers for discovery and digesting. They are optional: use them when a transcript store is noisy or a source file is too large to read directly. +The skill package ships small local helpers for discovery and digesting in `skills/agent-session-resume/scripts/`. Because they live inside the skill folder, they are installed alongside `SKILL.md` (for example under `~/.claude/skills/agent-session-resume/scripts/` or `~/.codex/skills/agent-session-resume/scripts/`), so an installed skill can run them instead of re-deriving long jq pipelines. They are python3-stdlib-only, take absolute or relative paths as arguments, and can run from any working directory. They are optional: use them when a transcript store is noisy or a source file is too large to read directly. + +In the examples below, `$SKILL_SCRIPTS` is wherever the scripts live: `skills/agent-session-resume/scripts` in this repo, or `/agent-session-resume/scripts` for an installed skill. ### Find Candidate Sessions Use `session-candidates.py` to shortlist likely transcripts before opening transcript bodies. ```bash -python3 scripts/session-candidates.py --platform codex --cwd "$PWD" --format tsv -python3 scripts/session-candidates.py --platform claude-code --cwd "$PWD" --format tsv +python3 "$SKILL_SCRIPTS/session-candidates.py" --platform codex --cwd "$PWD" --format tsv +python3 "$SKILL_SCRIPTS/session-candidates.py" --platform claude-code --cwd "$PWD" --format tsv ``` For Codex, the helper reads `session_index.jsonl` first and resolves candidate IDs to transcript files. For Claude Code, it derives the likely `~/.claude/projects/` directory from the current cwd before falling back to broader project scans. @@ -365,7 +367,20 @@ For Codex, the helper reads `session_index.jsonl` first and resolves candidate I Use `--topic` when the user gave a title or theme: ```bash -python3 scripts/session-candidates.py --platform codex --cwd "$PWD" --topic "checkout retry" +python3 "$SKILL_SCRIPTS/session-candidates.py" --platform codex --cwd "$PWD" --topic "checkout retry" +``` + +Use `--since` / `--until` to answer time-window asks such as "my Codex threads from the past week" without hand-rolling date enumeration. Both accept an ISO date or datetime (`2026-06-01`, `2026-06-01T12:00:00Z`) or a relative window (`7d`, `12h`, `30m`, `2w`): + +```bash +python3 "$SKILL_SCRIPTS/session-candidates.py" --platform codex --since 7d --format tsv +python3 "$SKILL_SCRIPTS/session-candidates.py" --platform claude-code --since 2026-06-01 --until 2026-06-08 +``` + +`--cwd ` filters as well as ranks: only sessions whose recorded workspace matches the given path exactly, or as a parent/child directory, are kept. Omit `--cwd` to rank by the current directory without filtering. Filters compose, so "this repo, past week" is: + +```bash +python3 "$SKILL_SCRIPTS/session-candidates.py" --platform codex --cwd "$PWD" --since 7d ``` ### Create A Compact Evidence Digest @@ -373,11 +388,27 @@ python3 scripts/session-candidates.py --platform codex --cwd "$PWD" --topic "che Use `session-digest.py` to produce a bounded orientation digest from transcript, export, handoff, or artifact files: ```bash -python3 scripts/session-digest.py path/to/session.jsonl path/to/handoff.md +python3 "$SKILL_SCRIPTS/session-digest.py" path/to/session.jsonl path/to/handoff.md ``` The digest is an orientation aid, not a replacement for evidence review. After digesting, still inspect the relevant transcript slices, tool outputs, changed files, git state, and verification results before continuing work. +#### Digest Caching + +`session-digest.py` caches each digest in a sidecar file named `.digest.json` written next to the source file. The sidecar stores the file size, a SHA-256 of the digested bytes, and the last processed byte offset. On rerun: + +- unchanged file (size and hash match) - the sidecar is reused wholesale and nothing is re-read beyond the hash check; +- append-only growth (the previously digested prefix is unchanged) - only the appended tail is processed and merged into the cached digest; this incremental behavior is the default for Codex and Claude Code JSONL transcripts; +- prefix changed (rewritten, truncated, or compacted file) - the digest is recomputed from scratch. + +Flags: + +- `--sidecar-dir ` writes sidecars into a separate directory instead of next to the transcript (useful for read-only stores). +- `--no-sidecar` disables cache reads and writes entirely. +- `--no-incremental` disables append-only tail processing; unchanged files still get whole-sidecar cache hits. + +If the transcript directory is not writable, the digest still prints and the sidecar is skipped with a notice on stderr. Cache notices (`cache hit`, `incremental update`, `cache invalidated`) go to stderr so stdout stays a clean digest. + ## Benchmarking Improvements When proposing changes to the skill, adapters, fixtures, or helper scripts, describe the behavior being improved and the benchmark target. Use the standard areas in [Benchmarking](Benchmarking.md), especially session selection, discovery effort, token usage proxy, resume accuracy, evidence quality, safety/redaction, robustness, trigger behavior, and reviewer clarity. diff --git a/scripts/session-candidates.py b/scripts/session-candidates.py deleted file mode 100755 index 8f222f7..0000000 --- a/scripts/session-candidates.py +++ /dev/null @@ -1,189 +0,0 @@ -#!/usr/bin/env python3 -"""List likely agent-session transcripts without dumping transcript bodies.""" - -from __future__ import annotations - -import argparse -import json -import os -from pathlib import Path -from typing import Any - - -def read_jsonl(path: Path) -> list[dict[str, Any]]: - rows: list[dict[str, Any]] = [] - try: - with path.open(encoding="utf-8") as handle: - for line in handle: - line = line.strip() - if not line: - continue - try: - rows.append(json.loads(line)) - except json.JSONDecodeError: - continue - except OSError: - return [] - return rows - - -def first_codex_cwd(path: Path) -> str: - for row in read_jsonl(path): - if row.get("type") == "session_meta": - payload = row.get("payload") or {} - return str(payload.get("cwd") or "") - return "" - - -def first_claude_cwd(path: Path) -> str: - for row in read_jsonl(path): - cwd = row.get("cwd") - if row.get("type") == "user" and cwd: - return str(cwd) - return "" - - -def first_claude_title(path: Path) -> str: - seen: set[str] = set() - for row in read_jsonl(path): - if row.get("type") != "ai-title": - continue - title = str(row.get("aiTitle") or "") - if title and title not in seen: - return title - seen.add(title) - return "" - - -def score_candidate(cwd: str, title: str, target_cwd: str, topic: str) -> tuple[int, list[str]]: - score = 0 - signals: list[str] = [] - if target_cwd and cwd == target_cwd: - score += 100 - signals.append("exact-cwd") - elif target_cwd and (cwd.startswith(target_cwd + os.sep) or target_cwd.startswith(cwd + os.sep)): - score += 60 - signals.append("parent-child-cwd") - if topic and topic.lower() in title.lower(): - score += 30 - signals.append("title-match") - return score, signals - - -def find_codex_transcript(codex_home: Path, session_id: str) -> Path | None: - for directory in (codex_home / "sessions", codex_home / "archived_sessions"): - if not directory.exists(): - continue - matches = sorted(directory.rglob(f"*{session_id}*.jsonl")) - if matches: - return matches[-1] - return None - - -def codex_candidates(codex_home: Path, target_cwd: str, topic: str, limit: int) -> list[dict[str, Any]]: - index = codex_home / "session_index.jsonl" - candidates: list[dict[str, Any]] = [] - rows = read_jsonl(index) if index.exists() else [] - for row in rows: - session_id = str(row.get("id") or "") - title = str(row.get("thread_name") or "") - if topic and topic.lower() not in title.lower(): - continue - path = find_codex_transcript(codex_home, session_id) if session_id else None - cwd = first_codex_cwd(path) if path else "" - score, signals = score_candidate(cwd, title, target_cwd, topic) - candidates.append( - { - "platform": "codex", - "id": session_id, - "title": title, - "updated_at": row.get("updated_at") or "", - "cwd": cwd, - "path": str(path) if path else "", - "score": score, - "signals": signals, - } - ) - candidates.sort(key=lambda item: (item["score"], item.get("updated_at", ""), item.get("path", "")), reverse=True) - return candidates[:limit] - - -def encode_claude_project_path(cwd: str) -> str: - return cwd.replace("/", "-") - - -def claude_candidates(claude_home: Path, target_cwd: str, topic: str, limit: int) -> list[dict[str, Any]]: - projects = claude_home / "projects" - project_dirs: list[Path] = [] - if target_cwd: - derived = projects / encode_claude_project_path(target_cwd) - if derived.exists(): - project_dirs.append(derived) - if not project_dirs and projects.exists(): - project_dirs = sorted(projects.iterdir()) - - candidates: list[dict[str, Any]] = [] - for project_dir in project_dirs: - for path in sorted(project_dir.glob("*.jsonl")): - title = first_claude_title(path) - if topic and topic.lower() not in title.lower(): - continue - cwd = first_claude_cwd(path) - score, signals = score_candidate(cwd, title, target_cwd, topic) - candidates.append( - { - "platform": "claude-code", - "id": path.stem, - "title": title, - "updated_at": str(path.stat().st_mtime_ns), - "cwd": cwd, - "path": str(path), - "score": score, - "signals": signals, - } - ) - candidates.sort(key=lambda item: (item["score"], item.get("updated_at", ""), item.get("path", "")), reverse=True) - return candidates[:limit] - - -def print_tsv(candidates: list[dict[str, Any]]) -> None: - print("score\tplatform\tupdated_at\tcwd\ttitle\tpath") - for candidate in candidates: - print( - "\t".join( - [ - str(candidate["score"]), - candidate["platform"], - str(candidate["updated_at"]), - candidate["cwd"], - candidate["title"], - candidate["path"], - ] - ) - ) - - -def main() -> None: - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("--platform", choices=["codex", "claude-code"], required=True) - parser.add_argument("--cwd", default=os.getcwd(), help="Workspace path to match; defaults to the current directory.") - parser.add_argument("--topic", default="", help="Optional title/topic filter.") - parser.add_argument("--limit", type=int, default=10) - parser.add_argument("--format", choices=["json", "tsv"], default="json") - parser.add_argument("--codex-home", default=os.environ.get("CODEX_HOME", str(Path.home() / ".codex"))) - parser.add_argument("--claude-home", default=str(Path.home() / ".claude")) - args = parser.parse_args() - - if args.platform == "codex": - candidates = codex_candidates(Path(args.codex_home), args.cwd, args.topic, args.limit) - else: - candidates = claude_candidates(Path(args.claude_home), args.cwd, args.topic, args.limit) - - if args.format == "tsv": - print_tsv(candidates) - else: - print(json.dumps(candidates, indent=2)) - - -if __name__ == "__main__": - main() diff --git a/scripts/validate-session-candidates.py b/scripts/validate-session-candidates.py new file mode 100644 index 0000000..3a7c8a1 --- /dev/null +++ b/scripts/validate-session-candidates.py @@ -0,0 +1,198 @@ +#!/usr/bin/env python3 +"""Validate the session candidates helper, including time-window and cwd filters.""" + +from __future__ import annotations + +import json +import os +import re +import subprocess +import sys +import tempfile +from datetime import datetime, timedelta, timezone +from pathlib import Path + + +ROOT = Path(__file__).resolve().parents[1] +SCRIPT = ROOT / "skills" / "agent-session-resume" / "scripts" / "session-candidates.py" + +NOW = datetime.now(timezone.utc) +RECENT = (NOW - timedelta(days=1)).isoformat() +OLD = (NOW - timedelta(days=30)).isoformat() +OLD_EPOCH = (NOW - timedelta(days=30)).timestamp() + +ISO_UTC_SECONDS_RE = re.compile(r"^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z$") +ALLOWED_SOURCES = {"index", "mtime"} + + +def fail(message: str) -> None: + print(f"session candidates validation failed: {message}", file=sys.stderr) + raise SystemExit(1) + + +def run_candidates(*args: str) -> list[dict]: + result = subprocess.run( + [sys.executable, str(SCRIPT), *args], + check=True, + text=True, + capture_output=True, + ) + return json.loads(result.stdout) + + +def write_jsonl(path: Path, rows: list[dict]) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text("".join(json.dumps(row) + "\n" for row in rows), encoding="utf-8") + + +def codex_session(session_id: str, cwd: str) -> list[dict]: + return [ + {"type": "session_meta", "payload": {"id": session_id, "cwd": cwd, "timestamp": RECENT, "originator": "codex"}}, + {"type": "event_msg", "timestamp": RECENT, "payload": {"type": "user_message", "message": f"work in {cwd}"}}, + ] + + +def build_codex_home(home: Path, repo_cwd: str, other_cwd: str) -> None: + write_jsonl( + home / "session_index.jsonl", + [ + {"id": "recent-repo", "thread_name": "checkout retry fix", "updated_at": RECENT}, + {"id": "old-repo", "thread_name": "stale repo thread", "updated_at": OLD}, + {"id": "recent-other", "thread_name": "other repo work", "updated_at": RECENT}, + ], + ) + write_jsonl(home / "sessions" / "rollout-recent-repo.jsonl", codex_session("recent-repo", repo_cwd)) + write_jsonl(home / "sessions" / "rollout-old-repo.jsonl", codex_session("old-repo", repo_cwd)) + write_jsonl(home / "sessions" / "rollout-recent-other.jsonl", codex_session("recent-other", other_cwd)) + # Unindexed transcripts (e.g. Codex Desktop sub-threads) live only on disk, + # under dated sessions/YYYY/MM/DD directories, and never get index entries. + dated = home / "sessions" / "2026" / "06" / "09" + write_jsonl(dated / "rollout-sub-recent.jsonl", codex_session("sub-recent", repo_cwd)) + write_jsonl(dated / "rollout-sub-old.jsonl", codex_session("sub-old", repo_cwd)) + os.utime(dated / "rollout-sub-old.jsonl", (OLD_EPOCH, OLD_EPOCH)) + + +def check_normalized(candidates: list[dict], label: str) -> None: + for candidate in candidates: + updated_at = str(candidate.get("updated_at") or "") + if not ISO_UTC_SECONDS_RE.fullmatch(updated_at): + fail(f"{label}: updated_at must be ISO-8601 UTC seconds (e.g. 2026-06-10T00:15:30Z), got {updated_at!r}") + if candidate.get("source") not in ALLOWED_SOURCES: + fail(f"{label}: source must be one of {sorted(ALLOWED_SOURCES)}, got {candidate.get('source')!r}") + + +def claude_session(cwd: str, title: str) -> list[dict]: + return [ + {"type": "ai-title", "aiTitle": title, "timestamp": RECENT}, + {"type": "user", "cwd": cwd, "timestamp": RECENT, "message": {"role": "user", "content": f"work in {cwd}"}}, + ] + + +def build_claude_home(home: Path, repo_cwd: str, other_cwd: str) -> None: + repo_dir = home / "projects" / repo_cwd.replace("/", "-") + other_dir = home / "projects" / other_cwd.replace("/", "-") + write_jsonl(repo_dir / "recent-repo.jsonl", claude_session(repo_cwd, "checkout retry fix")) + write_jsonl(other_dir / "recent-other.jsonl", claude_session(other_cwd, "other repo work")) + write_jsonl(other_dir / "old-other.jsonl", claude_session(other_cwd, "stale other thread")) + old_epoch = (NOW - timedelta(days=30)).timestamp() + os.utime(other_dir / "old-other.jsonl", (old_epoch, old_epoch)) + + +def ids(candidates: list[dict]) -> set[str]: + return {candidate["id"] for candidate in candidates} + + +def validate_codex(tmp: Path) -> None: + home = tmp / "codex-home" + repo_cwd = str(tmp / "work" / "repo") + other_cwd = str(tmp / "elsewhere" / "proj") + build_codex_home(home, repo_cwd, other_cwd) + base = ("--platform", "codex", "--codex-home", str(home)) + + unfiltered = run_candidates(*base) + if ids(unfiltered) != {"recent-repo", "old-repo", "recent-other", "rollout-sub-recent", "rollout-sub-old"}: + fail("codex: unfiltered run should list indexed sessions plus unindexed mtime-fallback sessions") + check_normalized(unfiltered, "codex") + sources = {candidate["id"]: candidate.get("source") for candidate in unfiltered} + for indexed_id in ("recent-repo", "old-repo", "recent-other"): + if sources[indexed_id] != "index": + fail(f"codex: indexed session {indexed_id} should carry source=index") + for fallback_id in ("rollout-sub-recent", "rollout-sub-old"): + if sources[fallback_id] != "mtime": + fail(f"codex: unindexed session {fallback_id} should carry source=mtime") + + if ids(run_candidates(*base, "--since", "7d")) != {"recent-repo", "recent-other", "rollout-sub-recent"}: + fail("codex: --since 7d should drop the 30-day-old sessions but keep the unindexed recent one") + + until = (NOW - timedelta(days=7)).date().isoformat() + if ids(run_candidates(*base, "--until", until)) != {"old-repo", "rollout-sub-old"}: + fail("codex: --until should keep only sessions before the cutoff, including unindexed ones") + + since_iso = (NOW - timedelta(days=7)).date().isoformat() + if ids(run_candidates(*base, "--since", since_iso)) != {"recent-repo", "recent-other", "rollout-sub-recent"}: + fail("codex: ISO --since should drop the 30-day-old sessions") + + if ids(run_candidates(*base, "--cwd", repo_cwd)) != {"recent-repo", "old-repo", "rollout-sub-recent", "rollout-sub-old"}: + fail("codex: --cwd should keep only matching-workspace sessions (fallback cwd comes from the first-line peek)") + + child = str(Path(repo_cwd) / "packages" / "api") + if ids(run_candidates(*base, "--cwd", child)) != {"recent-repo", "old-repo", "rollout-sub-recent", "rollout-sub-old"}: + fail("codex: --cwd should match parent/child workspaces") + + if ids(run_candidates(*base, "--since", "7d", "--cwd", repo_cwd)) != {"recent-repo", "rollout-sub-recent"}: + fail("codex: combined --since and --cwd should intersect") + + if ids(run_candidates(*base, "--topic", "checkout")) != {"recent-repo"}: + fail("codex: --topic should match indexed titles only (untitled fallback rows cannot match)") + + +def validate_claude(tmp: Path) -> None: + home = tmp / "claude-home" + repo_cwd = str(tmp / "work" / "repo") + other_cwd = str(tmp / "work" / "other") + build_claude_home(home, repo_cwd, other_cwd) + base = ("--platform", "claude-code", "--claude-home", str(home)) + + unfiltered = run_candidates(*base) + if ids(unfiltered) != {"recent-repo", "recent-other", "old-other"}: + fail("claude: unfiltered run should list all sessions") + check_normalized(unfiltered, "claude") + if any(candidate.get("source") != "mtime" for candidate in unfiltered): + fail("claude: updated_at is mtime-derived, so every row should carry source=mtime") + + if ids(run_candidates(*base, "--since", "7d")) != {"recent-repo", "recent-other"}: + fail("claude: --since 7d should drop the 30-day-old session") + + if ids(run_candidates(*base, "--cwd", repo_cwd)) != {"recent-repo"}: + fail("claude: --cwd should keep only matching-workspace sessions") + + parent = str(tmp / "work") + if ids(run_candidates(*base, "--cwd", parent)) != {"recent-repo", "recent-other", "old-other"}: + fail("claude: --cwd parent path should match child workspaces") + + if ids(run_candidates(*base, "--cwd", parent, "--until", "7d")) != {"old-other"}: + fail("claude: combined --cwd and --until should intersect") + + +def validate_bad_value() -> None: + result = subprocess.run( + [sys.executable, str(SCRIPT), "--platform", "codex", "--since", "lastweek"], + text=True, + capture_output=True, + ) + if result.returncode == 0: + fail("invalid --since value should exit non-zero") + if "invalid --since/--until value" not in result.stderr: + fail("invalid --since value should explain the accepted formats") + + +def main() -> None: + with tempfile.TemporaryDirectory(prefix="asr-candidates-") as tmp: + validate_codex(Path(tmp)) + validate_claude(Path(tmp)) + validate_bad_value() + print("validated session candidate filters") + + +if __name__ == "__main__": + main() diff --git a/scripts/validate-session-digest.py b/scripts/validate-session-digest.py new file mode 100644 index 0000000..746f961 --- /dev/null +++ b/scripts/validate-session-digest.py @@ -0,0 +1,172 @@ +#!/usr/bin/env python3 +"""Validate the session digest helper, including sidecar cache behavior.""" + +from __future__ import annotations + +import json +import os +import shutil +import subprocess +import sys +import tempfile +from pathlib import Path + + +ROOT = Path(__file__).resolve().parents[1] +SCRIPT = ROOT / "skills" / "agent-session-resume" / "scripts" / "session-digest.py" +CODEX_FIXTURE = ROOT / "tests" / "fixtures" / "codex-noisy-jsonl" / "transcript.jsonl" +CLAUDE_FIXTURE = ROOT / "tests" / "fixtures" / "claude-noisy-jsonl" / "transcript.jsonl" + +APPENDED_ROW = { + "timestamp": "2030-01-01T00:00:00Z", + "type": "event_msg", + "payload": {"type": "agent_message", "message": "appended-tail-marker next step pending"}, +} + + +def fail(message: str) -> None: + print(f"session digest validation failed: {message}", file=sys.stderr) + raise SystemExit(1) + + +def run_digest(*args: str) -> tuple[str, str]: + result = subprocess.run( + [sys.executable, str(SCRIPT), *args], + check=True, + text=True, + capture_output=True, + ) + return result.stdout, result.stderr + + +def sidecar_for(path: Path) -> Path: + return path.with_name(path.name + ".digest.json") + + +def read_sidecar(path: Path) -> dict: + sidecar = sidecar_for(path) + if not sidecar.exists(): + fail(f"expected sidecar at {sidecar}") + return json.loads(sidecar.read_text(encoding="utf-8")) + + +def validate_cache_hit(workdir: Path) -> None: + transcript = workdir / "transcript.jsonl" + shutil.copyfile(CODEX_FIXTURE, transcript) + + first_out, first_err = run_digest(str(transcript)) + if "cache hit" in first_err: + fail("first run should not report a cache hit") + record = read_sidecar(transcript) + for key in ("cache_version", "source", "size", "sha256", "offset", "digest"): + if key not in record: + fail(f"sidecar missing key {key!r}") + if record["size"] != transcript.stat().st_size or record["offset"] != record["size"]: + fail("sidecar size/offset should match the transcript size") + + second_out, second_err = run_digest(str(transcript)) + if "cache hit" not in second_err: + fail("second run on unchanged file should report a cache hit") + if second_out != first_out: + fail("cache-hit output should match the fresh digest output") + + +def validate_append_only_tail(workdir: Path) -> None: + transcript = workdir / "transcript.jsonl" + shutil.copyfile(CODEX_FIXTURE, transcript) + run_digest(str(transcript)) + old_offset = read_sidecar(transcript)["offset"] + + with transcript.open("a", encoding="utf-8") as handle: + handle.write(json.dumps(APPENDED_ROW) + "\n") + + out, err = run_digest(str(transcript)) + if "incremental update" not in err: + fail("appended file should trigger incremental update") + if "appended-tail-marker" not in out: + fail("incremental digest should include the appended event") + + record = read_sidecar(transcript) + if record["offset"] <= old_offset: + fail("sidecar offset should advance after incremental update") + + # Incremental output must match a full recompute of the grown file. + fresh_dir = workdir / "fresh" + fresh_dir.mkdir() + fresh = fresh_dir / "transcript.jsonl" + shutil.copyfile(transcript, fresh) + fresh_out, _ = run_digest(str(fresh)) + if out.replace(str(transcript), "X") != fresh_out.replace(str(fresh), "X"): + fail("incremental digest output should match full recompute") + + +def validate_prefix_change_invalidation(workdir: Path) -> None: + transcript = workdir / "transcript.jsonl" + shutil.copyfile(CODEX_FIXTURE, transcript) + run_digest(str(transcript)) + + data = transcript.read_bytes() + transcript.write_bytes(data.replace(b"session_meta", b"session_munge", 1)) + _, err = run_digest(str(transcript)) + if "cache invalidated" not in err and "recomputing" not in err: + fail("changed prefix should invalidate the cache") + if "cache hit" in err or "incremental update" in err: + fail("changed prefix must not reuse the cache") + + +def validate_no_sidecar(workdir: Path) -> None: + transcript = workdir / "transcript.jsonl" + shutil.copyfile(CLAUDE_FIXTURE, transcript) + run_digest(str(transcript), "--no-sidecar") + if sidecar_for(transcript).exists(): + fail("--no-sidecar should not write a sidecar") + + +def validate_sidecar_dir(workdir: Path) -> None: + transcript = workdir / "transcript.jsonl" + shutil.copyfile(CLAUDE_FIXTURE, transcript) + cache_dir = workdir / "cache" + run_digest(str(transcript), "--sidecar-dir", str(cache_dir)) + if sidecar_for(transcript).exists(): + fail("--sidecar-dir should not write next to the transcript") + if not (cache_dir / "transcript.jsonl.digest.json").exists(): + fail("--sidecar-dir should hold the sidecar") + _, err = run_digest(str(transcript), "--sidecar-dir", str(cache_dir)) + if "cache hit" not in err: + fail("--sidecar-dir rerun should report a cache hit") + + +def validate_unwritable_dir(workdir: Path) -> None: + if os.name != "posix" or os.geteuid() == 0: + return + locked = workdir / "locked" + locked.mkdir() + transcript = locked / "transcript.jsonl" + shutil.copyfile(CODEX_FIXTURE, transcript) + locked.chmod(0o555) + try: + out, err = run_digest(str(transcript)) + if "skipping sidecar" not in err: + fail("read-only directory should skip the sidecar with a notice") + if "# Session Digest" not in out: + fail("digest should still print when the sidecar is skipped") + finally: + locked.chmod(0o755) + + +def main() -> None: + for check in ( + validate_cache_hit, + validate_append_only_tail, + validate_prefix_change_invalidation, + validate_no_sidecar, + validate_sidecar_dir, + validate_unwritable_dir, + ): + with tempfile.TemporaryDirectory(prefix="asr-digest-") as tmp: + check(Path(tmp)) + print("validated session digest caching") + + +if __name__ == "__main__": + main() diff --git a/scripts/validate-session-events.py b/scripts/validate-session-events.py index 7ecb09b..96c1cf0 100755 --- a/scripts/validate-session-events.py +++ b/scripts/validate-session-events.py @@ -10,7 +10,7 @@ ROOT = Path(__file__).resolve().parents[1] -SCRIPT = ROOT / "scripts" / "session-events.py" +SCRIPT = ROOT / "skills" / "agent-session-resume" / "scripts" / "session-events.py" CODEX_FIXTURE = ROOT / "tests" / "fixtures" / "codex-noisy-jsonl" / "transcript.jsonl" CLAUDE_FIXTURE = ROOT / "tests" / "fixtures" / "claude-noisy-jsonl" / "transcript.jsonl" diff --git a/scripts/validate-skill-package.py b/scripts/validate-skill-package.py index 5d1ec55..680e7f1 100644 --- a/scripts/validate-skill-package.py +++ b/scripts/validate-skill-package.py @@ -16,6 +16,7 @@ REFERENCES = SKILL_DIR / "references" MARKETPLACE_JSON = ROOT / ".claude-plugin" / "marketplace.json" CLAUDE_PLUGIN_MANIFEST = ROOT / ".claude-plugin" / "plugin.json" +SCRIPTS_DIR = SKILL_DIR / "scripts" REQUIRED_REFERENCES = { "claude-code.md": "Claude Code", "codex.md": "Codex", @@ -23,6 +24,12 @@ "antigravity.md": "Antigravity", "opencode.md": "OpenCode", } +REQUIRED_SCRIPTS = ( + "session-events.py", + "session-candidates.py", + "session-digest.py", + "skill-provenance.py", +) def fail(message: str) -> None: @@ -100,6 +107,15 @@ def validate_references() -> None: fail(f"SKILL.md does not link {filename}") +def validate_scripts() -> None: + if not SCRIPTS_DIR.is_dir(): + fail(f"missing directory: {SCRIPTS_DIR.relative_to(ROOT)}") + for filename in REQUIRED_SCRIPTS: + text = read_required(SCRIPTS_DIR / filename) + if not text.startswith("#!/usr/bin/env python3"): + fail(f"scripts/{filename} must start with a python3 shebang") + + def validate_openai_yaml() -> None: text = read_required(OPENAI_YAML) required_snippets = ( @@ -163,6 +179,7 @@ def main() -> None: validate_skill_md() validate_references() + validate_scripts() validate_openai_yaml() validate_claude_marketplace() validate_claude_plugin_manifest() diff --git a/skills/agent-session-resume/scripts/session-candidates.py b/skills/agent-session-resume/scripts/session-candidates.py new file mode 100755 index 0000000..34101cc --- /dev/null +++ b/skills/agent-session-resume/scripts/session-candidates.py @@ -0,0 +1,435 @@ +#!/usr/bin/env python3 +"""List likely agent-session transcripts without dumping transcript bodies. + +Supports time-window filters (`--since`, `--until`, ISO dates or relative +values such as `7d`) and a `--cwd` filter that keeps only sessions whose +workspace matches the given path exactly or as a parent/child directory. + +Every row's `updated_at` is normalized to ISO-8601 UTC with seconds precision +(e.g. 2026-06-10T00:15:30Z) on both platforms, and carries a `source` marker: +`index` (platform session index) or `mtime` (transcript file mtime). For codex, +an mtime fallback sweep surfaces in-window transcripts the session index never +recorded (e.g. Codex Desktop sub-threads). +""" + +from __future__ import annotations + +import argparse +import json +import os +import re +import sys +from datetime import datetime, timedelta, timezone +from pathlib import Path +from typing import Any + + +RELATIVE_RE = re.compile(r"^(?P\d+)(?P[smhdw])$") +RELATIVE_UNITS = { + "s": "seconds", + "m": "minutes", + "h": "hours", + "d": "days", + "w": "weeks", +} + +# Where a candidate's updated_at came from: a platform session index ("index") +# or the transcript file's mtime ("mtime"). +SOURCE_INDEX = "index" +SOURCE_MTIME = "mtime" + +# Cap for the optional first-line cwd peek on unindexed codex transcripts, so +# the mtime fallback never reads more than this many bytes per file. +MAX_CWD_PEEK_BYTES = 65536 + + +def parse_when(raw: str) -> float: + """Parse an ISO date/datetime or a relative window like 7d into epoch seconds.""" + value = raw.strip() + match = RELATIVE_RE.match(value) + if match: + delta = timedelta(**{RELATIVE_UNITS[match.group("unit")]: int(match.group("value"))}) + return (datetime.now(timezone.utc) - delta).timestamp() + try: + parsed = datetime.fromisoformat(value.replace("Z", "+00:00")) + except ValueError: + raise SystemExit(f"invalid --since/--until value: {raw!r} (use ISO date/datetime or relative like 7d, 12h)") + if parsed.tzinfo is None: + parsed = parsed.astimezone() + return parsed.timestamp() + + +def parse_iso_epoch(raw: str) -> float | None: + try: + parsed = datetime.fromisoformat(str(raw).replace("Z", "+00:00")) + except ValueError: + return None + if parsed.tzinfo is None: + parsed = parsed.astimezone() + return parsed.timestamp() + + +def iso_utc(epoch: float) -> str: + """Render an epoch as ISO-8601 UTC with seconds precision (the one updated_at format both platforms emit).""" + return datetime.fromtimestamp(epoch, tz=timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ") + + +def normalize_updated_at(raw: str, path: Path | None) -> str: + """Normalize index timestamps (fractional ISO, plain ISO, or epoch nanoseconds) to iso_utc; fall back to file mtime.""" + value = str(raw or "").strip() + if value.isdigit(): + return iso_utc(int(value) / 1_000_000_000) + if value: + epoch = parse_iso_epoch(value) + if epoch is not None: + return iso_utc(epoch) + if path is not None: + try: + return iso_utc(path.stat().st_mtime) + except OSError: + return "" + return "" + + +def candidate_epoch(candidate: dict[str, Any]) -> float | None: + updated_at = str(candidate.get("updated_at") or "") + if updated_at.isdigit(): + return int(updated_at) / 1_000_000_000 + epoch = parse_iso_epoch(updated_at) if updated_at else None + if epoch is not None: + return epoch + path = candidate.get("path") or "" + if path: + try: + return Path(path).stat().st_mtime + except OSError: + return None + return None + + +def cwd_related(candidate_cwd: str, filter_cwd: str) -> bool: + if not candidate_cwd: + return False + candidate_norm = os.path.normpath(candidate_cwd) + filter_norm = os.path.normpath(filter_cwd) + return ( + candidate_norm == filter_norm + or candidate_norm.startswith(filter_norm + os.sep) + or filter_norm.startswith(candidate_norm + os.sep) + ) + + +def apply_filters( + candidates: list[dict[str, Any]], + since_epoch: float | None, + until_epoch: float | None, + filter_cwd: str | None, +) -> list[dict[str, Any]]: + kept: list[dict[str, Any]] = [] + for candidate in candidates: + if filter_cwd is not None and not cwd_related(candidate.get("cwd") or "", filter_cwd): + continue + if since_epoch is not None or until_epoch is not None: + epoch = candidate_epoch(candidate) + if epoch is None: + continue + if since_epoch is not None and epoch < since_epoch: + continue + if until_epoch is not None and epoch > until_epoch: + continue + kept.append(candidate) + return kept + + +def read_jsonl(path: Path) -> list[dict[str, Any]]: + rows: list[dict[str, Any]] = [] + try: + with path.open(encoding="utf-8") as handle: + for line in handle: + line = line.strip() + if not line: + continue + try: + rows.append(json.loads(line)) + except json.JSONDecodeError: + continue + except OSError: + return [] + return rows + + +def first_codex_cwd(path: Path) -> str: + for row in read_jsonl(path): + if row.get("type") == "session_meta": + payload = row.get("payload") or {} + return str(payload.get("cwd") or "") + return "" + + +def first_claude_cwd(path: Path) -> str: + for row in read_jsonl(path): + cwd = row.get("cwd") + if row.get("type") == "user" and cwd: + return str(cwd) + return "" + + +def first_claude_title(path: Path) -> str: + seen: set[str] = set() + for row in read_jsonl(path): + if row.get("type") != "ai-title": + continue + title = str(row.get("aiTitle") or "") + if title and title not in seen: + return title + seen.add(title) + return "" + + +def score_candidate(cwd: str, title: str, target_cwd: str, topic: str) -> tuple[int, list[str]]: + score = 0 + signals: list[str] = [] + if target_cwd and cwd == target_cwd: + score += 100 + signals.append("exact-cwd") + elif target_cwd and (cwd.startswith(target_cwd + os.sep) or target_cwd.startswith(cwd + os.sep)): + score += 60 + signals.append("parent-child-cwd") + if topic and topic.lower() in title.lower(): + score += 30 + signals.append("title-match") + return score, signals + + +def find_codex_transcript(codex_home: Path, session_id: str) -> Path | None: + for directory in (codex_home / "sessions", codex_home / "archived_sessions"): + if not directory.exists(): + continue + matches = sorted(directory.rglob(f"*{session_id}*.jsonl")) + if matches: + return matches[-1] + return None + + +def peek_codex_cwd(path: Path) -> str: + """Read at most MAX_CWD_PEEK_BYTES of the first line to recover session_meta cwd; empty string when not cheap/parseable.""" + try: + with path.open(encoding="utf-8", errors="replace") as handle: + line = handle.readline(MAX_CWD_PEEK_BYTES).strip() + except OSError: + return "" + if not line or not line.endswith("}"): + return "" + try: + row = json.loads(line) + except json.JSONDecodeError: + return "" + if row.get("type") != "session_meta": + return "" + payload = row.get("payload") or {} + return str(payload.get("cwd") or "") + + +def codex_mtime_fallback( + codex_home: Path, + known_paths: set[str], + target_cwd: str, + since_epoch: float | None, + until_epoch: float | None, +) -> list[dict[str, Any]]: + """Surface in-window transcripts the session index never recorded (e.g. Codex Desktop sub-threads). + + Cost is bounded: os.walk + stat per file, plus a size-capped first-line cwd + peek for files that pass the time window. + """ + sessions_dir = codex_home / "sessions" + if not sessions_dir.exists(): + return [] + fallback: list[dict[str, Any]] = [] + for dirpath, _dirnames, filenames in os.walk(sessions_dir): + for name in sorted(filenames): + if not name.endswith(".jsonl"): + continue + path = Path(dirpath) / name + if str(path) in known_paths: + continue + try: + mtime = path.stat().st_mtime + except OSError: + continue + if since_epoch is not None and mtime < since_epoch: + continue + if until_epoch is not None and mtime > until_epoch: + continue + cwd = peek_codex_cwd(path) + score, signals = score_candidate(cwd, "", target_cwd, "") + fallback.append( + { + "platform": "codex", + "id": path.stem, + "title": "", + "updated_at": iso_utc(mtime), + "cwd": cwd, + "path": str(path), + "score": score, + "signals": signals, + "source": SOURCE_MTIME, + } + ) + return fallback + + +def codex_candidates( + codex_home: Path, + target_cwd: str, + topic: str, + since_epoch: float | None, + until_epoch: float | None, +) -> list[dict[str, Any]]: + index = codex_home / "session_index.jsonl" + candidates: list[dict[str, Any]] = [] + rows = read_jsonl(index) if index.exists() else [] + for row in rows: + session_id = str(row.get("id") or "") + title = str(row.get("thread_name") or "") + if topic and topic.lower() not in title.lower(): + continue + path = find_codex_transcript(codex_home, session_id) if session_id else None + cwd = first_codex_cwd(path) if path else "" + score, signals = score_candidate(cwd, title, target_cwd, topic) + candidates.append( + { + "platform": "codex", + "id": session_id, + "title": title, + "updated_at": normalize_updated_at(str(row.get("updated_at") or ""), path), + "cwd": cwd, + "path": str(path) if path else "", + "score": score, + "signals": signals, + "source": SOURCE_INDEX, + } + ) + # The index silently omits unindexed transcripts (Codex Desktop sub-threads + # with parent_thread_id never get index entries); sweep sessions/ by mtime so + # those still surface. Skipped under --topic: unindexed files have no title + # to match, so the topic filter would drop every fallback row anyway. + if not topic: + known_paths = {candidate["path"] for candidate in candidates if candidate["path"]} + candidates.extend(codex_mtime_fallback(codex_home, known_paths, target_cwd, since_epoch, until_epoch)) + return candidates + + +def encode_claude_project_path(cwd: str) -> str: + return cwd.replace("/", "-") + + +def claude_candidates(claude_home: Path, target_cwd: str, topic: str, scan_all_projects: bool) -> list[dict[str, Any]]: + projects = claude_home / "projects" + project_dirs: list[Path] = [] + if target_cwd and not scan_all_projects: + derived = projects / encode_claude_project_path(target_cwd) + if derived.exists(): + project_dirs.append(derived) + if not project_dirs and projects.exists(): + project_dirs = sorted(projects.iterdir()) + + candidates: list[dict[str, Any]] = [] + for project_dir in project_dirs: + if not project_dir.is_dir(): + continue + for path in sorted(project_dir.glob("*.jsonl")): + title = first_claude_title(path) + if topic and topic.lower() not in title.lower(): + continue + cwd = first_claude_cwd(path) + score, signals = score_candidate(cwd, title, target_cwd, topic) + candidates.append( + { + "platform": "claude-code", + "id": path.stem, + "title": title, + "updated_at": iso_utc(path.stat().st_mtime), + "cwd": cwd, + "path": str(path), + "score": score, + "signals": signals, + "source": SOURCE_MTIME, + } + ) + return candidates + + +def print_tsv(candidates: list[dict[str, Any]]) -> None: + print("score\tplatform\tupdated_at\tsource\tcwd\ttitle\tpath") + for candidate in candidates: + print( + "\t".join( + [ + str(candidate["score"]), + candidate["platform"], + str(candidate["updated_at"]), + str(candidate.get("source") or ""), + candidate["cwd"], + candidate["title"], + candidate["path"], + ] + ) + ) + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) + parser.add_argument("--platform", choices=["codex", "claude-code"], required=True) + parser.add_argument( + "--cwd", + default=None, + help=( + "Workspace path filter; keeps sessions whose cwd matches exactly or as a parent/child path " + "and boosts ranking. When omitted, the current directory is used for ranking only." + ), + ) + parser.add_argument("--topic", default="", help="Optional title/topic filter.") + parser.add_argument( + "--since", + default=None, + help="Keep sessions updated at/after this ISO date/datetime or relative window (e.g. 7d, 12h).", + ) + parser.add_argument( + "--until", + default=None, + help="Keep sessions updated at/before this ISO date/datetime or relative window (e.g. 1d).", + ) + parser.add_argument("--limit", type=int, default=10) + parser.add_argument("--format", choices=["json", "tsv"], default="json") + parser.add_argument("--codex-home", default=os.environ.get("CODEX_HOME", str(Path.home() / ".codex"))) + parser.add_argument("--claude-home", default=str(Path.home() / ".claude")) + args = parser.parse_args() + + filter_cwd = os.path.abspath(args.cwd) if args.cwd is not None else None + target_cwd = filter_cwd or os.getcwd() + since_epoch = parse_when(args.since) if args.since else None + until_epoch = parse_when(args.until) if args.until else None + if since_epoch is not None and until_epoch is not None and since_epoch > until_epoch: + print("session-candidates: --since is later than --until; no sessions can match", file=sys.stderr) + + if args.platform == "codex": + candidates = codex_candidates(Path(args.codex_home), target_cwd, args.topic, since_epoch, until_epoch) + else: + # A --cwd filter accepts parent/child workspaces, so the single derived + # project directory is too narrow; scan all project directories instead. + scan_all_projects = filter_cwd is not None + candidates = claude_candidates(Path(args.claude_home), target_cwd, args.topic, scan_all_projects) + + candidates = apply_filters(candidates, since_epoch, until_epoch, filter_cwd) + candidates.sort(key=lambda item: (item["score"], item.get("updated_at", ""), item.get("path", "")), reverse=True) + candidates = candidates[: args.limit] + + if args.format == "tsv": + print_tsv(candidates) + else: + print(json.dumps(candidates, indent=2)) + + +if __name__ == "__main__": + main() diff --git a/scripts/session-digest.py b/skills/agent-session-resume/scripts/session-digest.py similarity index 55% rename from scripts/session-digest.py rename to skills/agent-session-resume/scripts/session-digest.py index 7b30489..21f6da0 100755 --- a/scripts/session-digest.py +++ b/skills/agent-session-resume/scripts/session-digest.py @@ -1,11 +1,20 @@ #!/usr/bin/env python3 -"""Create a compact evidence digest from agent-session files.""" +"""Create a compact evidence digest from agent-session files. + +Digests are cached in a sidecar file (`.digest.json`) written next +to the source file by default. On rerun the sidecar is reused when the source +is unchanged, and only the appended tail is processed when the source has +grown append-only (the default incremental behavior). If the previously +digested prefix changed, the digest is recomputed from scratch. +""" from __future__ import annotations import argparse +import hashlib import json import re +import sys from pathlib import Path from typing import Any @@ -13,6 +22,13 @@ MAX_PREVIEW = 500 KEYWORDS = ("todo", "not done", "partially done", "failed", "error", "next", "pause", "stop here") SIDECAR_RE = re.compile(r"Full output saved to:\s*(?P[^\s]+)") +CACHE_VERSION = 1 +CACHE_SUFFIX = ".digest.json" +JSONL_PLATFORMS = {"codex", "claude-code"} + + +def notice(message: str) -> None: + print(f"session-digest: {message}", file=sys.stderr) def preview(text: Any, limit: int = MAX_PREVIEW) -> str: @@ -23,17 +39,25 @@ def preview(text: Any, limit: int = MAX_PREVIEW) -> str: return value[: limit - 1] + "…" -def read_jsonl(path: Path) -> list[dict[str, Any]]: +def parse_jsonl_bytes(data: bytes) -> list[dict[str, Any]]: rows: list[dict[str, Any]] = [] - with path.open(encoding="utf-8") as handle: - for line in handle: - line = line.strip() - if not line: - continue - rows.append(json.loads(line)) + for line in data.decode("utf-8", errors="replace").splitlines(): + line = line.strip() + if not line: + continue + try: + row = json.loads(line) + except json.JSONDecodeError: + continue + if isinstance(row, dict): + rows.append(row) return rows +def read_jsonl(path: Path) -> list[dict[str, Any]]: + return parse_jsonl_bytes(path.read_bytes()) + + def is_codex(rows: list[dict[str, Any]]) -> bool: return any(row.get("type") == "session_meta" for row in rows) @@ -138,13 +162,13 @@ def digest_codex(path: Path, rows: list[dict[str, Any]]) -> dict[str, Any]: return {"platform": "codex", "path": str(path), "meta": meta, "timeline": timeline, "tool_calls": tool_calls, "evidence": evidence} -def digest_claude(path: Path, rows: list[dict[str, Any]]) -> dict[str, Any]: +def digest_claude(path: Path, rows: list[dict[str, Any]], seen_titles: set[str] | None = None) -> dict[str, Any]: meta: dict[str, Any] = {"titles": []} timeline: list[str] = [] tool_calls: list[str] = [] evidence: list[str] = [] sidecars: dict[str, dict[str, Any]] = {} - seen_titles: set[str] = set() + seen_titles = set(seen_titles or ()) for row in rows: row_type = row.get("type") timestamp = row.get("timestamp") or "" @@ -209,6 +233,150 @@ def digest_file(path: Path) -> dict[str, Any]: return digest_text(path) +def digest_jsonl_rows(path: Path, platform: str, rows: list[dict[str, Any]], seen_titles: set[str]) -> dict[str, Any]: + if platform == "codex": + return digest_codex(path, rows) + return digest_claude(path, rows, seen_titles=seen_titles) + + +def merge_digests(base: dict[str, Any], tail: dict[str, Any]) -> dict[str, Any]: + merged = dict(base) + for key in ("timeline", "tool_calls", "evidence"): + merged[key] = list(base.get(key) or []) + list(tail.get(key) or []) + + base_meta = dict(base.get("meta") or {}) + for key, value in (tail.get("meta") or {}).items(): + if key == "titles": + titles = list(base_meta.get("titles") or []) + titles.extend(title for title in value if title not in titles) + base_meta["titles"] = titles + elif key not in base_meta and value: + base_meta[key] = value + merged["meta"] = base_meta + + if "sidecars" in base or "sidecars" in tail: + by_path: dict[str, dict[str, Any]] = {} + for record in list(base.get("sidecars") or []) + list(tail.get("sidecars") or []): + by_path[str(record.get("path"))] = record + merged["sidecars"] = list(by_path.values()) + return merged + + +def sha256_prefix(path: Path, length: int) -> str: + hasher = hashlib.sha256() + remaining = length + with path.open("rb") as handle: + while remaining > 0: + chunk = handle.read(min(1024 * 1024, remaining)) + if not chunk: + break + hasher.update(chunk) + remaining -= len(chunk) + return hasher.hexdigest() + + +def last_complete_newline_offset(path: Path, size: int) -> int: + if size == 0: + return 0 + offset = 0 + with path.open("rb") as handle: + while True: + chunk = handle.read(min(1024 * 1024, size - offset)) + if not chunk: + break + idx = chunk.rfind(b"\n") + if idx != -1: + offset += idx + 1 + else: + offset += len(chunk) + return offset + + +def cache_sidecar_path(path: Path, sidecar_dir: Path | None) -> Path: + name = path.name + CACHE_SUFFIX + if sidecar_dir is not None: + return sidecar_dir / name + return path.with_name(name) + + +def load_cache(path: Path, cache_path: Path) -> dict[str, Any] | None: + try: + record = json.loads(cache_path.read_text(encoding="utf-8")) + except (OSError, json.JSONDecodeError, UnicodeDecodeError): + return None + if not isinstance(record, dict): + return None + if record.get("cache_version") != CACHE_VERSION: + return None + if record.get("source") != str(path.resolve()): + return None + if not isinstance(record.get("digest"), dict): + return None + if not isinstance(record.get("size"), int) or not isinstance(record.get("offset"), int): + return None + if not isinstance(record.get("sha256"), str) or not record["sha256"]: + return None + return record + + +def write_cache(path: Path, cache_path: Path, digest: dict[str, Any]) -> None: + size = path.stat().st_size + offset = last_complete_newline_offset(path, size) + record = { + "cache_version": CACHE_VERSION, + "source": str(path.resolve()), + "size": size, + "sha256": sha256_prefix(path, offset), + "offset": offset, + "digest": digest, + } + try: + cache_path.parent.mkdir(parents=True, exist_ok=True) + cache_path.write_text(json.dumps(record, ensure_ascii=False), encoding="utf-8") + except OSError as exc: + notice(f"skipping sidecar for {path} (not writable: {exc})") + + +def digest_file_cached(path: Path, sidecar_dir: Path | None, use_sidecar: bool, incremental: bool) -> dict[str, Any]: + if not use_sidecar: + return digest_file(path) + + cache_path = cache_sidecar_path(path, sidecar_dir) + cached = load_cache(path, cache_path) if cache_path.exists() else None + size = path.stat().st_size + + if cached is not None: + if size == cached["size"] and sha256_prefix(path, size) == cached["sha256"]: + notice(f"cache hit for {path} (reused {cache_path})") + digest = cached["digest"] + digest["path"] = str(path) + return digest + + platform = (cached["digest"] or {}).get("platform") + if ( + incremental + and platform in JSONL_PLATFORMS + and sha256_prefix(path, cached["offset"]) == cached["sha256"] + ): + notice(f"incremental update for {path} (processing bytes {cached['offset']}..{size})") + with path.open("rb") as handle: + handle.seek(cached["offset"]) + tail_bytes = handle.read() + base = cached["digest"] + base["path"] = str(path) + seen_titles = set((base.get("meta") or {}).get("titles") or []) + tail = digest_jsonl_rows(path, platform, parse_jsonl_bytes(tail_bytes), seen_titles) + digest = merge_digests(base, tail) + write_cache(path, cache_path, digest) + return digest + + notice(f"cache invalidated for {path} (source changed); recomputing") + + digest = digest_file(path) + write_cache(path, cache_path, digest) + return digest + + def print_digest(digests: list[dict[str, Any]]) -> None: print("# Session Digest") for digest in digests: @@ -239,10 +407,27 @@ def print_digest(digests: list[dict[str, Any]]) -> None: def main() -> None: - parser = argparse.ArgumentParser(description=__doc__) + parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) parser.add_argument("files", nargs="+", type=Path) + parser.add_argument( + "--sidecar-dir", + type=Path, + default=None, + help=f"Directory for `{CACHE_SUFFIX}` cache sidecars; defaults to the transcript's own directory.", + ) + parser.add_argument("--no-sidecar", action="store_true", help="Disable reading and writing digest cache sidecars.") + parser.add_argument( + "--no-incremental", + action="store_true", + help="Disable incremental append-only reuse; unchanged files still get whole-sidecar cache hits.", + ) args = parser.parse_args() - print_digest([digest_file(path) for path in args.files]) + print_digest( + [ + digest_file_cached(path, args.sidecar_dir, not args.no_sidecar, not args.no_incremental) + for path in args.files + ] + ) if __name__ == "__main__": diff --git a/scripts/session-events.py b/skills/agent-session-resume/scripts/session-events.py similarity index 100% rename from scripts/session-events.py rename to skills/agent-session-resume/scripts/session-events.py diff --git a/scripts/skill-provenance.py b/skills/agent-session-resume/scripts/skill-provenance.py similarity index 100% rename from scripts/skill-provenance.py rename to skills/agent-session-resume/scripts/skill-provenance.py diff --git a/tests/README.md b/tests/README.md index 173e976..478d50b 100644 --- a/tests/README.md +++ b/tests/README.md @@ -8,12 +8,15 @@ Run structural validation: python3 scripts/validate-skill-package.py python3 scripts/validate-fixtures.py python3 scripts/validate-trigger-matrix.py +python3 scripts/validate-session-events.py +python3 scripts/validate-session-digest.py +python3 scripts/validate-session-candidates.py claude plugin validate . claude plugin validate .claude-plugin/plugin.json claude plugin validate .claude-plugin/marketplace.json ``` -The package validator checks the installable skill shape and the optional Claude plugin wrapper that points at the same canonical skill folder. The fixture validator checks that every supported platform has a scenario, that each source and expected-output file exists, that expected outputs include the required resume sections, task classifications, and fixture evidence references, and that optional source/expected cues are present. The trigger matrix validator checks prompt coverage for should-trigger and should-not-trigger cases. +The package validator checks the installable skill shape (including the packaged runtime scripts in `skills/agent-session-resume/scripts/`) and the optional Claude plugin wrapper that points at the same canonical skill folder. The session digest validator covers sidecar cache hits, append-only incremental updates, and prefix-change invalidation. The session candidates validator covers the `--since`/`--until` time-window and `--cwd` workspace filters. The fixture validator checks that every supported platform has a scenario, that each source and expected-output file exists, that expected outputs include the required resume sections, task classifications, and fixture evidence references, and that optional source/expected cues are present. The trigger matrix validator checks prompt coverage for should-trigger and should-not-trigger cases. For benchmark areas and issue/PR evaluation fields, see [`docs/Benchmarking.md`](../docs/Benchmarking.md).