Skip to content

feat(stream): bound the batch member and observation lists - #2546

Merged
oceanwaves630 merged 2 commits into
mainfrom
code/batch-cap
Sep 10, 2026
Merged

feat(stream): bound the batch member and observation lists#2546
oceanwaves630 merged 2 commits into
mainfrom
code/batch-cap

Conversation

@oceanwaves630

@oceanwaves630 oceanwaves630 commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

Implements ADR-174 Amendment 7 (#2544).

stream.batch refused an empty ops list and named the writer it would take for a batch that writes nothing. It did not refuse a large one, and in atomic mode the whole list executes inside one BEGIN IMMEDIATE, so the statement count inside a single writer hold was whatever the caller sent. The only ceiling was the daemon's 8 MiB frame, a transport limit.

What changes

  • At most 1000 members in ops, at most 100 entries in observed. Over either bound the call refuses with invalid_input naming both the cap and the count sent.
  • The check runs at admission: before any member is parsed into an action, before any note plan is prepared, before a writer is requested.
  • The bound is on the list, so it applies in both modes. Per-member mode already runs one member per transaction, and is capped anyway so one input is refused the same way whatever the mode.
  • stream.batch(help=true) names both caps.

Measured writer hold

Taken with the store's own open-transaction registry (tx_registry::oldest) sampled from a second task, so preparation before the writer is not counted as hold. File-backed store, one dedicated run each, pinned toolchain:

members peak writer hold whole dispatch
1000 (the cap) 101.6 ms, 105.4 ms 115.9 ms, 120.0 ms
186,000 (what the 8 MiB frame admits for a minimal member) 36.72 s, 31.06 s 38.99 s, 33.34 s

Two independent runs of each, on separate gate passes, so the numbers are not one sample. Roughly 300 times the hold, from one caller, against a store that admits one writer. The second row is measured with the cap constant raised, which is the only way to observe the behaviour the amendment exists to prevent; the raise is a one-line patch applied and reverted around that run, with the tree proven clean afterwards.

Acceptance, six arms, all green:

  • a list at the member cap commits, and a list at the member cap with the observation list at its own cap commits — the arms that keep the bound from silently drifting below its stated number;
  • one member over refuses naming 1001 and 1000, with the stream head unchanged and the row counts unchanged as the control that the refusal preceded the writer rather than rolling back after it;
  • one observation over refuses on its own number, with a member list well inside its bound so the two caps are told apart;
  • per-member mode refuses the same list the same way, with nothing written and no member values returned;
  • a list of individually-invalid members still refuses on the cap, which is what proves the check runs before members are interpreted.

Mutations, both predicted before running and both confirmed: with the member cap raised above the count the over-cap arm sends, that arm goes red (rc=101) and restores green; with the check moved after member preparation, the precedence arm goes red (rc=101) and restores green. Each revert proved by a clean tree.

Gate at the branch head 8247c8d57, pinned toolchain: cargo fmt --all -- --check clean, cargo clippy -p khive-pack-kg --all-targets -- -D warnings clean, cargo test -p khive-pack-kg --no-fail-fast 249 + 308 + 8 + 11 + 5 + 3 + 1 passed, one ignored (the measurement arm), zero failed.

One note on arm 1: its 1000 members are appends to a single stream, which is what §5 and Amendment 1 admit (appends to one stream take consecutive numbers). The repeated-target rule governs duplicate write (kind, key) pairs, so it does not fire here, and a red in that arm would be about the cap rather than about member shape.

Closes #2543.

stream.batch refused an empty ops list and named the writer it would take
for a batch that writes nothing, and did not refuse a large one. In atomic
mode the whole list runs inside one BEGIN IMMEDIATE, so the statement count
inside a single writer hold was whatever the caller sent, ceilinged only by
the 8 MiB frame.

The verb now admits at most 1000 members and at most 100 observed entries,
refusing with invalid_input that names both the cap and the count sent. The
check runs at admission, before any member is parsed into an action, any
note plan is prepared, or a writer is requested, and it applies to the list
in both modes so one input is refused the same way whatever the mode.

Six acceptance arms: a list at the cap commits, a list at the cap with the
observation list at its own cap commits, one member over refuses naming
1001 and 1000 with the stream head unchanged as the control that the
refusal preceded the writer, one observation over refuses on its own
number, per-member mode refuses the same list the same way, and a list of
individually-invalid members still refuses on the cap, which is what proves
the check runs before members are interpreted. Help names both caps.

The ignored cap_measure_writer_hold arm measures the hold with the store's
own open-transaction registry rather than wall time around the dispatch, so
preparation before the writer is not counted as hold.

Implements ADR-174 Amendment 7.
@oceanwaves630
oceanwaves630 merged commit a6abad9 into main Sep 10, 2026
28 checks passed
@oceanwaves630
oceanwaves630 deleted the code/batch-cap branch September 10, 2026 22:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

stream.batch bounds the empty member list but not the large one

2 participants