Skip to content

memory: the snapshot bound ladder is inconsistent, and the CLI duplicates the byte cap #38

Description

@0thernet

Follow-up to #37. With the join no longer quadratic, work is the last limit to bind rather than the first, which exposes the other bounds. Measured while scaling algal memory query over real curated facts.

The ladder, in the order it actually binds

# bound value where
1 CLI file load 262,144 B main.rs, two call sites
2 snapshot canonical bytes 262,144 B memory.rs query()
3 facts list 2,048 memory.rs query()
4 result rows 256 Limits::max_rows
5 join bindings 4,096 Limits::max_bindings
6 work 250,000 Limits::max_work

Two problems.

The CLI duplicates the byte cap. load(&snapshot, 262_144) in main.rs gates the file before query() ever sees it, so raising the library's limit alone changes nothing — the CLI rejects with document bytes first. Two constants govern one thing.

The facts-list bound is unreachable. At the ~170 canonical bytes a typical fact occupies, 2,048 facts is roughly 348 KB, well past the 262,144-byte snapshot cap. Bound 2 always fires before bound 3, so list(&snapshot["facts"], 2048) never actually limits anything.

Measurements

At the current ceiling, 1,429 real facts and 260,549 bytes: 167 ms, ~16 MB peak RSS, 4,628 work, which is 1% of the work budget.

Raising bounds 1 through 4 in a throwaway worktree, then scaling by relabelled copies of the same real structure:

facts bytes ms work % work result
1,533 280 KB — 4,992 1% 138 rows
3,066 565 KB 376 9,984 3% 276 rows
6,132 1.1 MB 753 19,968 7% 552 rows
12,264 2.3 MB 1,469 39,936 15% 1,104 rows
24,528 4.6 MB — — — Datalog join bindings (bound 5)

Work doubles exactly per doubling of facts, confirming the post-#37 linearity. At 12,264 facts the engine is at 15% of the work budget and roughly 1.5 s.

What I am not proposing

Specific numbers. These are resource-posture decisions and AGENTS.md is explicit that every count and byte size gets a bound, so raising them is yours to choose rather than mine. Two things do look like plain defects regardless of the numbers chosen:

  1. the byte cap living in two places, so the library's constant is not authoritative
  2. a facts-list bound that cannot be reached

Happy to send a PR for either, or for a coherent ladder once you pick the targets.

Found while building algal-bio, which uses algal memory over aging-biology curation.

🤖 Generated with Claude Code

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions