Data-loss: session close path can wipe a large transcript (fix + tests available on fork)

[dwillitzer]

Summary

A close/clear path that swaps in a fresh 0-message session and saves with status=Closed triggers save()'s shrink detection (messages.len() < persist_state.messages_len) and checkpoints a full snapshot — overwriting the real transcript snapshot with an empty stub and deleting the journal. The later .bak rotation then captures the already-wiped state, so both copies end up empty and --resume shows an empty session.

Hit this in production 2026-08-05 on a 12-hour session (686 messages, 1.5MB snapshot). Daemon log forensics:

10:55:06  SESSION_PERSISTENCE save_mode=append    messages=686  snapshot_bytes=1,539,899
10:55:38  SESSION_PERSISTENCE save_mode=snapshot  messages=0    snapshot_bytes=600  status=Closed   <- wipe
10:55:41  SESSION_PERSISTENCE load_done           messages=0                                        <- resume sees nothing

Fix (ready on my fork — PR creation appears restricted on this repo)

Branch: https://github.com/dwillitzer/jcode/tree/fix/session-pre-wipe-guard (single commit, cherry-picks cleanly onto origin/master)

checkpoint_snapshot() gains a guard_snapshot_shrink() pre-step: when the in-memory transcript is empty but the on-disk snapshot exceeds 4KB, snapshot + journal are first copied to timestamped .pre-wipe-<ts>.json.bak / .jsonl.bak siblings.

• Never blocks the save — legitimate compaction shrinks are unaffected (guard requires empty; metadata stubs are under threshold)
• Backup names avoid *.json so session scanners don't surface phantoms
• Emits SESSION_PERSISTENCE phase=pre_wipe_backup for observability; backup failure logs and proceeds (fail-open)
• Recovery: cp session_X.pre-wipe-<ts>.json.bak session_X.json

Includes two regression tests (wipe path produces byte-identical restorable backup with full recovery round-trip; small/compaction checkpoints produce no backups).

Related fixes on the same fork, happy to PR any/all if you enable PRs or prefer patches:

Branch	What
fix/swarm-schema-anyof	Top-level anyOf in the swarm tool schema breaks strict providers (Anthropic API rejects it); flattened to a plain object schema
fix/swarm-worker-provider-pin	Swarm worker spawns drop the openai-compatible:<profile> prefix, so workers lose the auth profile and fail; pins the full prefix through spawn (with tests)
feat/prompt-cache-override	supports_prompt_cache config override so openai-compatible profiles fronting Anthropic models emit cache_control breakpoints (verified ~92% cached-token hit rate through a claude-family proxy)
fix/mcp-tool-name-sanitize	MCP tool names with characters outside [a-zA-Z0-9_-] are rejected by provider APIs; sanitizes at registration. Also prefers configured judge models over hardcoded defaults

All branches are single-purpose, based on current master (02439b4), and cherry-pick cleanly. If you'd rather receive these differently (git bundles, mailed patches, enabling PRs), say the word.

[wellsashley00]

⚙️✨ 本地开发一键依赖太香了

[harveysamuel99]

希望能出一版架构简图 😄

[harveysamuel99]

🤝 有没有 Docker 一键启动方案

[wellsashley00]

契约测试有没有上 👍😄

[bauerpaul874801]

🛠️🙌 API 设计很直觉，上手快

[1jehuang]

Thanks for the detailed report. Because this can cause data loss, it needs a focused reproduction and an audit of snapshot/journal atomicity and close ordering before a fix is merged. Recovery and interrupted-close tests should be required as part of the fix.

--- — Jcode agent (automated triage), on behalf of @1jehuang

[1jehuang]

The immediate safety fix is now in PR #820. Empty checkpoints can no longer replace an existing non-empty persisted transcript. Before refusing the save, snapshots larger than 4 KiB and their journals are copied to timestamped pre-wipe recovery files. Normal non-empty compaction remains allowed.

Verification: three focused persistence tests pass, covering byte-identical snapshot/journal recovery, destructive-checkpoint refusal, normal non-empty shrink, and small metadata stubs.

--- — Jcode agent (automated triage), on behalf of @1jehuang