docs: proposal of journaling

[kamil-kielbasa]

What changed

New document with proposal of internal mechanism called journaling.

Why

Required logic by higher abstration layers.

How tested

None.

PR Checklist

• Tests pass on native_sim
• Coverage targets met (line >= 80%, branch >= 70%)
• Code formatted with ./scripts/format.sh
• Documentation updated for any API/architecture/config changes
• CHANGELOG entry added under [Unreleased]
• No new compiler warnings (-Werror -Wextra -Wshadow)
• Secure tests include \brief, \details, \expected docblocks
  (and, where applicable, the optional \oracle, \trace,
  \precondition tags — see Test Strategy)
• Forensic scan passes on secure flash image (scripts/scan_flash.py)

[wmilek]

Thanks for writing this up — the on-flash format and the crash analysis in §5.6/§7 are thorough, and the API split in §8 is something we want regardless. But on the core question: I don't think the journal belongs at the UBI layer, and I don't think we need it there.

Linux deliberately doesn't do this

UBI has no journal. Its atomicity primitive is copy-on-write + sqnum + the copy_flag/data_crc pair in the VID header: ubi_eba_atomic_leb_change() writes a fresh PEB, and at attach ubi_compare_lebs() resolves an interrupted change by verifying the CRC of the higher-sqnum copy before letting it win. No log, no replay.

The journal in that stack lives in UBIFS, one layer up — and it journals the index, not the data. Nodes are written into buds and stay there permanently; commit only rewrites the wandering-tree index to point at them. §1's table states this correctly ("data stays in the journal, commit only updates the index"), but the proposed design does the opposite: records are replayed into their target LEBs and the journal is erased. That's jbd2 in ext3/4's data=journal mode, which double-writes every byte and is the mode nobody enables by default. Worth naming the difference explicitly in §1, because the UBIFS row currently reads as precedent for the design when it's precedent against it.

For our client it's cost without benefit

blob_db never rewrites an already-programmed region. It erases a whole PEB, writes a header, then appends slot records at increasing offsets — exactly ubi_leb_write_at(), i.e. Phase 1, which we already have. On a journaled volume it would get worse three ways:

• Reads. blob_db_get() reads a whole bucket LEB, so merged reads (§5.4) turn an O(1) read into O(live records targeting that LEB). Our L1 contract requires get to be O(1) flash reads independent of database size — that can't hold on a journaled volume.
• Flushes. ubi_leb_unmap() flushing (§5.3) means every bucket compaction forces a full journal flush, rebuilding unrelated targets. One compaction does five erases (two master writes + scratch/bucket/scratch).
• RAM. The index (journal_leb_count × MAX_RECORDS_PER_LEB × 12) plus the attach-time record scan is per-write RAM and a mount-time index rebuild, placed in the layer least able to bound it.

Structurally: blob_db is itself an append-only log with latest-wins and per-bucket compaction, and its index is id mod N — no wandering tree, so nothing to defer. UBIFS journals because path-copy to the root is expensive; we don't have that problem, so a journal underneath us is a second journal doing a job the first one already did.

The one case that does justify it: secure mode

ubi_leb_write_at() returns -ENOSYS on the secure backend, because a LEB payload is a single AEAD-authenticated blob. That means blob_db currently cannot run on CONFIG_UBI_SECURE at all. The journal is the only mechanism that makes an offset write expressible under AEAD — and that, rather than "100 small writes burn 100 PEBs", is the argument I'd lead with. On plain flash Phase 1 alone already solves the problem stated in §1, which leaves Phases 3–4 looking unmotivated.

Concretely

1. Land Phases 1 and 2 on their own — you already note they're independently useful and testable. Phase 1 is ubi_leb_write_at on feature/leb-partial-update. §8's flip to Linux naming (ubi_leb_change = atomic replace) is the right call, but it's a breaking rename that deserves its own PR; converging it would let us point west.yml back at an upstream tag instead of a fork branch.
2. Document journal_leb_count = 0 as a first-class choice, with a "when not to journal" rule: the client already logs and compacts, the client reads whole LEBs, or the client erases frequently. blob_db is all three.
3. Promote secure mode to the primary motivation in §1, and move §5.3's unmap-flush amplification and §5.4's read-cost-depends-on-write-history into §9 as contract-level caveats rather than just costs — any client with a documented O(1)-read contract needs to know it can't sit on a journaled volume.

Happy to be wrong on the framing if there's a plain-flash client that genuinely rewrites already-programmed regions — that would change the picture, and it'd be worth naming in §1.

---

Generated by Claude Code