v0.1.7: every created pack row reserves the full pack limit — 212 MB of Store for 74 MB of content on history-stride1 (#219)

[yifanxuaaa]

Status: measured on main at eb2c06e2 while validating the fix for #228. A space/layout consequence of the #219 pack work, not a correctness defect: the canonical content is unchanged (157/157 history roots identical to eight archived runs). Filed for a ruling, because the remedy is a product change.

What was measured

Same corpus, same history arm (LAYERFS_HISTORY_ADVISORY=1 LAYERFS_HISTORY_DEPTH_LIMIT=7), two builds —
stage-6-history-209-merged-20260920T225623Z/binary-archive/merged-main and today's tree:

	Sep-20 archived	today
history-stride1 Store file	88,944,640 B	212,779,008 B (du 208 MB — not sparse)
object_packs rows	855	784
sum(length(data))	73,044,275 B	205,520,896 B
bytes per pack: avg / min / max	85,432 / 180 / 262,136	262,144 / 262,144 / 262,144

Every pack row written today is exactly PACK_LIMIT = 262,144 B — the minimum equals the maximum — while the
archived build wrote each pack at its assembled length (min 180 B). The same corpus and the same representation
counts (inserted 97,788 both; prefix_records 78,548 vs 78,537; full_records 19,240 vs 19,251;
trials 70,562 both).

Why

A created pack row is inserted as a zero-filled BLOB at the pack's capacity and then appended in place:

// sqlite/write.rs:95
"INSERT INTO object_packs (pack_id, data, save_id) VALUES (?1, zeroblob(?2), (SELECT save_id FROM temp.layerfs_read_scope))"

and pack/placement.rs:52-54 declares the policy:

"Bytes the pack row allocates, which is what a created row is zero-filled to. Every lane but Singleton allocates
its full pack limit."

That is the #219 work (a5c54df16 "reserve the pack directory and append in place, −21.2%"), and it buys real time:
on the two runs above, delta.statements 75,990 → 2,602, delta.pack_appends 84,492 → 11,168,
delta.profile_sql_ns 2,542,546,634 → 1,193,735,816, delta.profile_place_ns 372,000,611 → 15,010,825.

The cost is confined to lanes whose packs are sparsely filled. At stride1 it is ~130 MB of reserved-but-unused
capacity (784 × 262 KB ≈ 206 MB against ~73 MB of content). It costs nothing where packs fill: the
pipeline-namespace-100000 row's geometry (canonical 513,684,532 / pack bodies 571,473,920 / ratio 1.112) is
identical in all nine of its runs from 09-21 23:55 to now.

What this is not

• Not the v0.1.7: the whole-file lane's chain charge counts its group, and history-stride10 no longer completes (#190) #228 charge fix. That change moves 11 records on this lane and touches no stored byte's class;
  the store is identical with and without the fix's sibling diagnostic (see below).
• Not the payload probe. 4d8e6e2ab's payload_is_incompressible path was forced off in a diagnostic build
  (always frame, the pre-change behaviour) and the Store was identical: 212,815,872 B, object_packs 205,926,400.

Options, for the ruling

1. Reserve less for sparsely-filled lanes. capacity is per lane already (Singleton has its own limit); the
   whole-file compact lane could reserve used + one group target at creation and extend as it appends.
2. Grow the row instead of pre-allocating it. Insert at the first write's assembled length and extend the BLOB
   when an append would not fit — SQLite grows a BLOB in place where the page has room.
3. Keep it and account for it. The space is a deliberate trade; if it stays, the row's published store figure
   should distinguish allocated from live bytes, because sum(length(data)) and the file size now overstate the
   content by ~2.4× on this lane, and v0.1.7: retained-history read optimizations and remaining qualification gaps #190's store-size axis reads those numbers.
4. Do nothing on this lane, fix the accounting. Different from 3 only in scope: the compaction path
   (sqlite/cleanup.rs) already treats object_packs as the unit it removes.

Nothing here is implemented; all four are product changes and need a ruling. Option 2 is the one that keeps the
write-path win intact on the lanes that benefit while removing the reservation from the lanes that do not.

Evidence

docs/roadmap/0.1/0.1.7/evidence/issue228-stride10-chain-charge-20260922T124500Z/report.md §5c and
raw/pack-allocation.tsv; the two Stores are on disk (/tmp/hs1-fixed and .../stage-6-history-209-merged-20260920T225623Z/runs/merged-history-stride1/raw/).

[yifanxuaaa]

The new #231 performance child #237 explicitly treats this issue as a storage-space regression guard. Faster native Init must retain matched physical Store bytes, pack utilization and readback, not merely equal canonical roots. #229 remains open for its own sparse-pack ruling; the new child does not waive or close it.