feat(kernel): SQLite concurrency posture — WAL, busy_timeout, OperationalError → WorkspaceBusy (#80) - #95
Merged
Conversation
…onalError → WorkspaceBusy (#80) Closes the last untranslated adapter exception before background ingest makes lock contention real. - WAL journal mode, set in `initialize()` rather than on every connection: switching to WAL writes the file header, and a connect-time pragma would leave a 4 KB mark on any stranger's file merely inspected by `format_version` — breaking the invariant that `open` creates nothing when it refuses. - `busy_timeout`, 5 s by default, on every connection. Keyword-only with a default so the class still satisfies `MetadataStoreFactory`. - One translation site, `_translated`, that every entry point routes through: `IntegrityError` → `ConstraintViolated`, `SQLITE_BUSY`/`SQLITE_LOCKED` → `WorkspaceBusy`, everything else → `WorkspaceCorrupt`. `unit_of_work` now catches `DatabaseError` rather than only `IntegrityError`, closing a hole where a non-constraint failure inside a transaction escaped raw. - Contention is told from damage by SQLite's result code, not by message text. - WAL sidecars are part of the workspace layout: `_undo_init` and both examples' cleanup now account for them. No migration: FORMAT_VERSION stays 10, VERSION stays 0.0.1.dev0, no openapi.json drift, no dependency change.
JArmandoAnaya
added a commit
that referenced
this pull request
Aug 21, 2026
…onalError → WorkspaceBusy (#80) (#95) Closes the last untranslated adapter exception before background ingest makes lock contention real. - WAL journal mode, set in `initialize()` rather than on every connection: switching to WAL writes the file header, and a connect-time pragma would leave a 4 KB mark on any stranger's file merely inspected by `format_version` — breaking the invariant that `open` creates nothing when it refuses. - `busy_timeout`, 5 s by default, on every connection. Keyword-only with a default so the class still satisfies `MetadataStoreFactory`. - One translation site, `_translated`, that every entry point routes through: `IntegrityError` → `ConstraintViolated`, `SQLITE_BUSY`/`SQLITE_LOCKED` → `WorkspaceBusy`, everything else → `WorkspaceCorrupt`. `unit_of_work` now catches `DatabaseError` rather than only `IntegrityError`, closing a hole where a non-constraint failure inside a transaction escaped raw. - Contention is told from damage by SQLite's result code, not by message text. - WAL sidecars are part of the workspace layout: `_undo_init` and both examples' cleanup now account for them. No migration: FORMAT_VERSION stays 10, VERSION stays 0.0.1.dev0, no openapi.json drift, no dependency change.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #80.
First task of M3. No new capability — this closes the adapter's last untranslated exception before #28 runs ingest in a background task writing through the kernel while request handlers read and write the same file.
What lands
WAL journal mode, set in
initialize()rather than on every connection. This is the one non-obvious decision here and the test suite is what forced it: switching a database to WAL writes its header, growing an empty file to a full page.WorkspaceService.openreadsformat_versionbefore it has decided the file is a workspace, so a connect-time pragma left a 4 KB mark on any stranger's file merely inspected — breaking the invariant thatopencreates nothing when it refuses (test_open_never_creates_a_schema_in_a_stray_databasecaught it).initialize()is exactly where the caller has already established the file is ours to write to. WAL persists in the header, so setting it once is what it takes, and re-running it there is how a pre-WAL workspace converts on its next open.busy_timeout, 5 s by default, on every connection including the one that runs migrations. Keyword-only with a default, soSqliteMetadataStorestill satisfiesMetadataStoreFactory = Callable[[Path], MetadataStore]and stays usable as a bare class reference; a caller wanting a different wait passespartial(SqliteMetadataStore, busy_timeout_ms=…).One translation site.
_translatedis now the adapter's entire exception vocabulary and every entry point routes through it:IntegrityErrorConstraintViolatedOperationalErrorwith aSQLITE_BUSY/SQLITE_LOCKEDresult codeWorkspaceBusy(new)DatabaseErrorWorkspaceCorruptunit_of_worknow catchesDatabaseErrorrather than onlyIntegrityError— that was a real hole: a non-constraint failure raised inside a transaction escaped raw. Contention is told from damage by SQLite's result code (sqlite_errorname), not by message text, so a reworded SQLite release cannot silently reroute a lock intoWorkspaceCorrupt.WAL sidecars are part of the workspace layout.
visionset.db-waland-shmexist while a workspace is open; a cleanclose()checkpoints and removes them, but a killed process leaves them._undo_initand both examples'_clear_previous_runnow account for them, anddocs/workspaces.mdcarries the warning it predicted for itself — copying onlyvisionset.dbfrom an open workspace loses committed data.Two decisions worth stating
One new error, not two. Non-lock
OperationalError(unable to open database file, disk I/O, disk full) falls through toWorkspaceCorrupt, whose docstring is widened to own it: "present but unusable, whatever the cause". A separateWorkspaceUnavailablewould still have no caller — the same objection that keptWorkspaceBusyitself out until #28/#31 supplied one.No
BEGIN IMMEDIATE.docs/workspaces.mdlisted it as the next hardening, and it is now declined rather than pending, with the reason recorded:unit_of_work()serves reads and writes alike with no read-only variant, so an immediate transaction would take the write lock for every read and serialize exactly the concurrency WAL was adopted for. It stays off unless the unit of work grows a read-only form. An advisory lock file remains open.Tests
New
tests/kernel/test_concurrency.py— the suite's first threaded tests. Sequenced onthreading.Event, never sleeps; every thread joined with a timeout and asserted dead. Ten tests: WAL on and persisted, WAL not applied by a bareformat_versionread, no sidecars after close, the busy timeout present on every connection and configurable, foreign keys surviving the listener rewrite, a reader proceeding while a write transaction is held open (the WAL payoff), and contention arriving asWorkspaceBusywith no SQLAlchemy in its module path. Plus aSQLITE_CANTOPENcase intest_metadata_store.pyproving the other half of theOperationalErrorsplit.Ran the concurrency file 20× consecutively: 0 failures.
Checks
ruff format --check,ruff check,mypy src,lint-imports(2 contracts kept) all clean. 896 tests pass, up from 885. Both examples run end to end. Noopenapi.jsondrift.No migration —
FORMAT_VERSIONstays 10,VERSIONstays0.0.1.dev0, no new service / model / event / dependency.