Skip to content

docs(rfc-0033): record Phase 3 as implemented #950

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ragnorc merged 1 commit into from
Oct 12, 2026
Merged

docs(rfc-0033): record Phase 3 as implemented #950

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 26 additions & 10 deletions docs/rfcs/0033-blob-management.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ implementation: partial
authors:
- OmniGraph maintainers
created: 2026-08-09
updated: 2026-10-10
updated: 2026-10-11
discussion: null
supersedes: []
superseded_by: []
Expand All @@ -27,10 +27,11 @@ and Lance 10.0.0 as the required implementation substrate.
**Audience:** engine, compiler, server, CLI, security, storage, maintenance,
and documentation maintainers.
**Current rollout:** Phases 0, 1, 2A (HTTP read delivery), and 2B (CLI read
delivery) are implemented on the v0.10 development line. Phase 3 (mutation) is
planned against current main in §13, with its decisions in the 2026-10-08
decision-log entry. Phase 4 (measured optimization) and the production delivery
telemetry named in §11 remain open.
delivery) are implemented on the v0.10 development line. Phase 3 (mutation:
put and clear through the engine, HTTP and the CLI) is implemented for v0.13.1,
with its decisions in the 2026-10-08 and 2026-10-10 decision-log entries.
Phase 4 (measured optimization) and the production delivery telemetry named in
§11 remain open.

This RFC is the required successor to an earlier reverted blob-delivery
experiment. The relevant evidence and decisions are restated here; no untracked
Expand Down Expand Up @@ -1267,14 +1268,15 @@ They exist for three reasons:
- a mutation or load holds its batches in memory until its one publication;
- Lance writes a Blob value from an in-memory Arrow array.

The ceilings are not a setting. The evidence behind the figure is
A setting cannot raise the ceilings: `write_max_bytes` (§4.3) only lowers them,
for a session or a process, and 32 MiB is its maximum. The evidence behind the figure is
[RFC 0023's cost gate](0023-key-conflict-fencing.md#114-cost-gate): paired
peak-RSS overhead of bounded fenced transactions under the old combined 32 MiB
accounting, for the workloads that gate names. It is not a workload-independent
process-memory bound, and it does not qualify the split envelope of §4.3, up to
32 MiB of payload plus 32 MiB of framing; qualifying that remains open work for
the same gate. A setting without such a measured multiplier would promise a
bound it cannot keep. Going beyond the ceilings means keeping large data out of
the same gate. A setting that raised the ceilings without such a measured
multiplier would promise a bound it cannot keep. Going beyond the ceilings means keeping large data out of
the write path's memory. There are three separate limits, each with its own dependency
and trigger:

Expand Down Expand Up @@ -1673,6 +1675,8 @@ correctness gate.

### Phase 3 — mutation

Implemented: 3A in #908, 3B in #922 and 3C in #928.

- **Served-read prerequisite:** met. The routed `GraphRequest` lease and the
read observer are held by the GET/HEAD response through EOF; the bounded
transport tests of §12.4 that assert it for Blob land with 3B. The schema-token
Expand All @@ -1698,8 +1702,12 @@ correctness gate.

- Batched reads (`read_blobs`) landed for materializing rewrites, export, the
change-feed baseline, change images and entity reads (§4), and schema apply
stopped rewriting Blob columns (§8.4). Remaining: benchmark. Tuning is
optional and may not weaken the batched contract or change logical behavior.
stopped rewriting Blob columns (§8.4). Remaining: benchmark. The active
benchmark catalog runs GQ operations only, and Blob reads and export are not
GQ operations, so that benchmark waits for a harness that can drive them;
until then `export.rs::export_reads_each_batch_of_managed_blobs_in_one_read_per_column`
pins the read count. Tuning is optional and may not weaken the batched
contract or change logical behavior.
- Retain the exact empty/null/neighbor compaction guard across every future Lance
dependency bump.
- Consider descriptor-preserving unchanged-cell updates only with an ownership
Expand Down Expand Up @@ -1930,6 +1938,14 @@ publisher architecture.

## Decision log

- 2026-10-11: Phase 3 is implemented: the engine writes (#908), `PUT` and
`DELETE /blob` (#922) and `blob put` / `blob clear` (#928). The Phase 4
benchmark waits for a harness that can drive Blob reads and export, which are
not GQ operations. Superseded: the header's "Phase 3 (mutation) is planned
against current main in §13"; and §10.1's "The ceilings are not a setting."
and "A setting without such a measured multiplier would promise a bound it
cannot keep.", since `write_max_bytes` now lowers the ceilings but cannot
raise them.
- 2026-10-10: Step 3-pre landed as part of the `write_max_bytes` session
setting rather than on its own. Row data and logical Blob payloads get
independent allowances of `write_max_bytes` each, 32 MiB by default and at
Expand Down
Loading