Skip to content

bug: read verbs still create events.db-shm when a crash-left events.db-wal exists (residual from #80) #81

Description

@earfman

What happened

status, replay, and doctor's EventStore consistency reads create an events.db-shm sidecar when the store has a crash-left events.db-wal. This is the residual half of #80: the immutable=1 branch closed the leak on clean stores, but the mode=ro fallback — taken exactly when a WAL exists — still mutates the tree on read, because SQLite creates the -shm to map the WAL. So reads are only filesystem-non-mutating on stores that were closed cleanly, and tree-byte-identity checks still break in precisely the crash-recovery case _readonly_query's docstring calls out ("preserving crash-left WAL reads").

Expected

Read verbs leave the .loop file set byte-identical, including when a previous writer crashed with an uncheckpointed WAL.

Reproduction

On main @ 0a888bc (post-#80). Seed a crash-left WAL by killing a writer that still holds an open WAL connection, then run a read verb:

import os, sys, subprocess, tempfile
from pathlib import Path
sys.path.insert(0, ".")  # repo root

ws = Path(tempfile.mkdtemp()); loop_dir = ws / ".loop"; loop_dir.mkdir()
db = loop_dir / "events.db"

# writer crashes (os._exit) with a WAL connection still open -> committed frames stay in events.db-wal
subprocess.run([sys.executable, "-c", f"""
import os, sys, sqlite3, json, uuid
sys.path.insert(0, {str(Path.cwd())!r})
from loop.events import SQLiteEventStore
s = SQLiteEventStore({str(db)!r})
s.append("run-1", "contract_opened", {{"workspace": "x"}}, actor="agent:test")
conn = sqlite3.connect({str(db)!r}, isolation_level=None)
conn.execute("PRAGMA journal_mode=WAL")
conn.execute("INSERT INTO events VALUES (?,?,?,?,?,?,?,?,?,?)",
    ("run-1", 2, str(uuid.uuid4()), "run_paused", "agent:test", None, None,
     "2026-07-24T00:00:00Z", json.dumps({{"iteration_id": "it-1", "reason": "crash"}}), "[]"))
os._exit(0)  # crash: no close, WAL never checkpointed
"""])
(Path(str(db) + "-shm")).unlink(missing_ok=True)  # canonical crash-left state: db + wal

print("BEFORE:", sorted(p.name for p in loop_dir.iterdir()))
from loop.runtime import status_report
status_report(ws)
print("AFTER: ", sorted(p.name for p in loop_dir.iterdir()))

Output:

BEFORE: ['events.db', 'events.db-wal']
AFTER:  ['events.db', 'events.db-shm', 'events.db-wal']   # -shm created by a READ

_readonly_query picks ?mode=ro here, and status_report returns correct data (event_count=2, WAL frames included) — the read itself is fine; it just isn't non-mutating.

Coverage note: scripts/test_doctor_eventstore.py::test_doctor_event_store_reads_do_not_leave_wal_or_shm_sidecars builds its store with a clean close, so no WAL exists and only the immutable=1 branch is exercised — which is why the suite stays green over this. A regression test that seeds a crash-left events.db-wal (as above) would lock the fix in.

Possible directions, from the #80 thread: checkpoint-on-open under a controlled recovery; validate the WAL then reopen with immutable=1; or read from a temp copy.

Environment

Gate output (if relevant)

python3 -m pytest -q scripts passes green on main — that's the gap this issue is about.

— flagged via Coretexa (adversarial PR verification); follow-up to my comment on #80

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions