bug: served export of a wide populated Vector table dies with a bare stream error — no served route to get stored vectors out

[ragnorc]

What happened

A served omnigraph export --type <T> of a table whose rows carry a populated 3072-dim Vector column dies after ~2.5 s with a transport-level stream error and no engine error message, so the CLI cannot say why:

error decoding response body
error reading a body from connection
stream error received: unexpected internal error encountered

The table is TranscriptSegment in our crm graph: 12,139 rows, embedding: Vector(3072)? @embed("text") @index, every row populated (embedded offline). Per-type export of every other type in the same graph works, including small vector-bearing tables (Account, 80 rows with a Vector(3072) profile embedding; Note, 55 rows). The size suggests the server trips a scan budget mid-stream rather than a wire limit: 12,139 × 3072 × 4 B ≈ 149 MB of vector data, right at the 150 MiB ordered_scan_memory_bytes constant that the 0.10 CLI reported as a hard error on the same table (resource limit exceeded for ordered_scan_memory_bytes … limit 157286400, see #705 for the same full-width-scan shape on merge). On 0.11.0 the served path surfaces nothing — the HTTP stream is just reset.

Combined with the fact that a served read cannot project a Vector property at all (return { $s } returns non-Blob/non-Vector properties, and $s.embedding is not projectable), there is no served route to get stored vectors out of a graph. The only ways are a direct --store read of the storage root — which is unsafe against a graph a server is actively writing to, because the CLI's open-time recovery sweep can roll back an in-flight publish — or keeping the offline omnigraph embed artifact that produced them. For a rebuild (export → new root → load) that is a real gap: we recovered the 12,139 vectors only because the embed output file was still on a laptop.

Expected: either the export streams the wide table (paged/chunked scan, like the per-type row count already suggests), or it fails with the engine's resource-limit error so the operator knows to page.

Steps to reproduce

1. A graph with a node type carrying embedding: Vector(3072)? @embed("text") @index and ~12k rows with the vector populated (ours: crm.TranscriptSegment, ~66 KB of JSON per row, ~800 MB total). Any comparable width × row count should do; the 80-row vector table exports fine.
2. Serve it with omnigraph-server --cluster <root> (ours: mr-cluster, 0.11.0, single instance).
3. omnigraph export --server <name> --graph crm --type TranscriptSegment > out.jsonl
4. Observed: exits after ~2.5 s with the stream error above, out.jsonl empty. Expected: 12,139 rows, or an engine error naming the limit.

Workaround we use: page the rows out through the server one parent at a time (match { $t: MeetingTranscript { slug: … } $g segmentOfTranscript $t } return { $g.slug, $g.text, … }, no vector), then restore embedding by slug from the offline embed artifact before loading the new root.

Version

omnigraph 0.11.0 on both sides (CLI release binary; server image modernrelay/omnigraph-server:v0.11.0)

Environment

Server: Linux x86_64 on EC2, S3 storage root (s3://…/clusters/mr-cluster-v9), internal schema v9. Client: macOS (Darwin 25.5) arm64.

Logs / output

$ time omnigraph export --server modernrelay --graph crm --type TranscriptSegment > TranscriptSegment.jsonl
Error:
   0: error decoding response body
   1: error reading a body from connection
   2: stream error received: unexpected internal error encountered

Location:
   crates/omnigraph-cli/src/client.rs:1320
omnigraph export … 0.01s user 0.01s system 0% cpu 2.484 total
$ wc -l TranscriptSegment.jsonl
0

[ragnorc]

Reproduced on main (80406fee), local cluster, Seg { embedding: Vector(3072)? @index }, 13,000 rows.

Symptom. A served export --type Seg fails immediately with error decoding response body / unexpected EOF during chunk size line and writes 0 rows. The server logs nothing, even with RUST_LOG=info,omnigraph=debug. A direct export --store file://… --type Seg fails with the typed resource limit exceeded for ordered_scan_memory_bytes: actual 157286401, limit 157286400.

Threshold. Much lower than the 150 MiB budget suggests. I exported after each 1,000-row load: 6,000 rows export, 7,000 rows fail. That's about 80 MB of raw vector data, so the sort reserves roughly twice the data size and does not spill.

Mechanism.

• Export orders every table by id (ExportRowOrder in crates/omnigraph/src/db/omnigraph/export.rs, scan_stream_bounded(… ordering …)), so the sort buffers the full table width, vector column included.
• On the served route the 200 headers are already sent before the scan fails. server_export turns the engine error into io::Error::other(error.to_string()) in ExportFrame::Terminal, and ExportBodyStream yields it as a body error (crates/omnigraph-server/src/export_transport.rs). hyper can only abort the chunked body at that point, so the engine message never reaches the client and nothing is logged.

Correction to the report. On main a served query can project a Vector property. return { $s.slug, $s.embedding } works on both engines through --server: all 13,000 rows with vectors come back, with or without order { $s.slug }, and 1,000-row slug-range pages also work. So there is a served route to get vectors out, and the defect is specific to export: its full-width ordered scan, and an error that is invisible once streaming has started.