Skip to content

stream.batch bounds the empty member list but not the large one #2543

Description

@oceanwaves630

stream.batch refuses an empty ops list with an explicit reason about taking the writer for a batch that writes nothing. It does not refuse a large one.

In atomic mode every member and every observed entry executes inside one BEGIN IMMEDIATE, so the statement count inside a single writer hold grows with whatever the caller sends:

  • one paired SELECT id, version per observed entry, plus one clock read when any observation carries a deadline;
  • per member either a prepared note plan with its row guards and one ledger INSERT, or a keyed write's prepared plan and one SELECT version, updated_at.

The only ceiling today is the daemon frame limit, MAX_FRAME_BYTES = 8 MiB in crates/khive-runtime/src/daemon.rs. That is a transport limit rather than a decision about writer hold time, and it admits a member count on the order of a hundred thousand. SQLite admits one writer, so a single large call extends every competing writer's wait.

Comparable paths in the tree name their own numbers instead: web manifest ingest caps at 10,000 entities and 50,000 edges, and blob GC processes at most 128 rows per unit. Both are recorded in ADR-091 Amendment 11.

Per-member mode is unaffected: it runs one member per transaction.

Proposed fix: a cap on ops and on observed in atomic mode, refused with the count and the limit, mirroring the existing empty-list refusal. The number is a contract decision and belongs in ADR-174's stream section.

The gap is recorded, named and not resolved, in ADR-091 Amendment 18 (#2445).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions