Per-table write workers — eliminate main-thread bottleneck

[tamashi095]

Currently 8 decode workers feed one main thread that writes to 11 SQLite files sequentially. The main thread is the bottleneck.

Proposed architecture: 11 write workers (one per table), each owning its own SQLite connection. Decode workers stream records directly to the appropriate table worker via postMessage, using Bun's zero-copy string fast path (~648ns for any size string).

Flow:

Decode worker → JSON.stringify(records) → postMessage(string) → Table worker → JSON.parse → stmt.run()

Benefits:

• Parallel I/O across 11 files (no single-writer bottleneck)
• Main thread only dispatches, doesn't parse or write
• Zero-copy string transfer via Bun fast path

Considerations:

• 11 extra workers = 11 extra threads
• JSON.stringify/parse overhead per batch (mitigated by batching)
• Only for SQLite (Postgres could use the same pattern with connection-per-worker)

Ref: Option B from the per-table SQLite design discussion.