Skip to content

fix(store): skip corrupted part_json and info_json rows instead of crashing#7

Closed
Milofax wants to merge 5 commits intoPlutarch01:masterfrom
Milofax:fix/json-parse-part-hardening
Closed

fix(store): skip corrupted part_json and info_json rows instead of crashing#7
Milofax wants to merge 5 commits intoPlutarch01:masterfrom
Milofax:fix/json-parse-part-hardening

Conversation

@Milofax
Copy link
Copy Markdown
Contributor

@Milofax Milofax commented Apr 11, 2026

Summary

One corrupted parts.part_json or messages.info_json row in the SQLite
archive currently takes out grep, describe, and the automatic
retrieval transform pipeline. parseJson is called directly in six
stored-row load sites inside store.ts, so a single unparseable blob
bubbles up as Failed to parse JSON: Unterminated string and kills the
entire batch load instead of skipping the bad row.

This PR adds a non-throwing parseJsonSafe<T>() companion to parseJson
in utils.ts and replaces the six unsafe call sites:

  • readSessionsBatchSync: the part_json and info_json loops (this is
    the exact stack in the production crash report).
  • readSessionSync: the part_json and info_json loops.
  • readMessageSync: the info_json load and the inner part_json map.

A corrupted part or message row now gets skipped with a structured
logger warning that carries operation, sessionID, messageID,
partID, error, and a 120-char preview of the offending blob — the
surrounding batch load keeps going and returns the healthy rows instead
of throwing.

Reproducer

Failed to parse JSON: JSON Parse error: Unterminated string
Input: {"id": "prt_d798683da001WV5rR5cUmpKdDA", "type": "text", "text":"
  at parseJson (dist/utils.js:35:19)
  at readSessionsBatchSync (dist/store.js:3786:26)
  at readScopedSessionsSync (dist/store.js:1804:21)
  at searchByScan (dist/store-search.js:194:27)

Triggered by a lcm_grep call whose FTS path returned zero hits and
fell through to the scan path against a session that had one
on-disk-corrupted parts.part_json row.

Regression test

tests/store-transform.test.mjs gets a new test
grep scan survives a single corrupted part_json row. It:

  1. Captures three valid messages.
  2. Opens the SQLite db directly and UPDATE parts SET part_json = '{"id": "m2-p", "type": "text", "text": "cosmic-ray corrupt b' to mimic on-disk truncation.
  3. DELETEs all three FTS tables (message_fts, summary_fts, artifact_fts) so grep is forced down the scan path — the exact code path the user crashed on.
  4. Reopens the store and calls store.grep({ query: 'cosmic-ray', sessionID: 's1' }).
  5. Asserts both surviving messages are returned and the corrupted one is silently dropped.

Without the fix, step 4 throws. With the fix, step 4 returns m1 and m3.

Test plan

  • npm run typecheck — clean
  • npm run lint — clean
  • npm test — 169/169 green (was 168 before the new regression test)
  • Manual smoke-test in OpenCode against a real corrupted .lcm/lcm.dblcm_grep no longer crashes, logs the skipped row, returns the surviving hits

Notes

  • parseJson itself is unchanged — structural invariants (e.g. summary_state.root_node_ids_json) still throw on corruption, which is the right behavior for places where a failure means the store is unrecoverable rather than just one row being bad.
  • No schema changes, no migration needed, safe to ship in a patch release.

🤖 Generated with Claude Code

BookTranslator User and others added 5 commits April 7, 2026 22:29
Add shared message validation (getValidMessageInfo) and filter out
malformed messages across all archive, resume, describe, transform,
search-index, and capture code paths. Prevents crashes when
message.info is missing required fields (id, sessionID, role,
time.created). Logs warnings for dropped messages instead of
throwing.

Fixes Plutarch01#4
transformMessages() now splices malformed entries out of the caller's
messages array so they never reach the opencode backend, preventing
the Bad Request that followed the earlier TypeError fix.
A single corrupted `parts.part_json`, `messages.info_json`,
`summary_nodes.message_ids_json`, `summary_state.root_node_ids_json`,
or `artifacts.metadata_json` row in the SQLite archive currently kills
the entire grep / describe / transform / summary pipeline via an
uncaught `Failed to parse JSON` error from `parseJson`.

Add a non-throwing `parseJsonSafe<T>()` to utils.ts and replace every
stored-row `parseJson` call site with it:

Store read paths (14 call sites):
- readSessionsBatchSync: part_json + info_json + artifact metadata_json
- readSessionSync: part_json + info_json
- readMessageSync: info_json + part_json
- diagnoseSummarySession: root_node_ids_json
- ensureSummaryGraphSync: root_node_ids_json
- readSummaryNodeSync: message_ids_json
- materializeArtifactRow: metadata_json

Legacy migration paths (2 call sites):
- migrateLegacyArtifacts: per-file try/catch so one bad sessions/*.json
  no longer kills the remaining files, and corrupted resume.json is
  gracefully skipped

Each skipped row is logged with operation/sessionID/messageID/partID
context and a 120-char preview of the offending blob, so the damage
is auditable without crashing the plugin.

The three in-memory `parseJson(JSON.stringify(...))` round-trips in
store-artifacts.ts and the already-try/catch-guarded events.jsonl line
parser stay as throwing `parseJson` — they cannot fail from disk
corruption.

Regression test:
- Capture a session, corrupt one part_json row in SQLite, clear all
  FTS indexes to force the scan path, call grep and assert the
  surviving messages are returned instead of throwing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@Milofax Milofax force-pushed the fix/json-parse-part-hardening branch from 9c28ceb to 11aad29 Compare April 12, 2026 06:58
@Plutarch01
Copy link
Copy Markdown
Owner

Closing in favor of #8, which lands the same corrupted-JSON-row fix cleanly against current master without the stale conflicts and version churn in this PR.

@Plutarch01 Plutarch01 closed this Apr 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants