Skip to content

feat(cli): add an export command that dumps the kv layer as import-co… - #85

Merged
cevheri merged 1 commit into
libredb:mainfrom
harish18092002:feat/cli-export-command
Sep 1, 2026
Merged

feat(cli): add an export command that dumps the kv layer as import-co…#85
cevheri merged 1 commit into
libredb:mainfrom
harish18092002:feat/cli-export-command

Conversation

@harish18092002

Copy link
Copy Markdown
Contributor

What and why

Closes #60.

The CLI could bulk-load a JSON object with import but had no way to produce one, so
the workflow was a one-way door. docs/CLI.md carried a placeholder admitting the gap
("A first-class export command is on the roadmap"), and the workaround it offered was
not usable in practice: scan requires a prefix, so there was no way to dump a whole
database from the CLI at all — you had to know every namespace prefix up front and
reassemble the pieces by hand.

This adds libredb export <path> <file.json>, the counterpart to import. It writes the
key-value layer as a flat JSON object of string values — the exact shape import
consumes — so export -> import round-trips without a second backup format.

How it works

This does not touch src/core.ts. The change is confined to the CLI edge
(src/cli/run.ts) plus docs and tests; no kernel, storage, lens, or public API surface
is modified, and the import format is unchanged.

export is a read command and reuses the existing read path verbatim: withReadDb opens
through readonlyFileSystem(), which implements no lock() at all (the seam's lock? is
optional), so a dump takes no lock, creates no <path>.lock, and leaves the database
byte-identical — it can read a file a live writer holds open. The whole dump is read in
one kernel transaction, mirroring import's one-transaction write, so the output is a
single consistent snapshot.

Three details are load-bearing and worth a reviewer's eye:

  • Scan bounds. The kv lens cannot express "scan everything": prefix("") throws, and
    range() takes string bounds, so no argument can encode an upper bound above the whole
    keyspace. Export therefore reads tx.getRange directly — the same kernel API import
    already writes through. The bound is [<empty>, 0xF5): no valid UTF-8 encoding begins
    above 0xF4 (the lead byte of U+10FFFF), so 0xF5 is above every key a lens or CLI
    command can write. The start is the empty key, which is itself legal and sorts *b
    the reserved namespace — which is why reserved keys are excluded by predicate, not by
    lower bound.
  • Reserved namespace. import refuses reserved keys, so emitting them would produce a
    file its own counterpart rejects. Export filters with the published isReservedKe contract rather than a hardcoded prefix, so it stays correct if the reserved namespace grows. Consequence, documented: a restored file holds every row but no catalog en inspect` lists nothing until a lens re-registers. A byte-exact copy is still a file copy.
  • Well-formedness. Import writes through the kernel directly and validates
    well-formedness itself; export reads through the kernel directly and carries the same
    obligation. It decodes with a fatal TextDecoder, so a database holding raw non-
    bytes (only reachable through the kernel API directly) is refused rather than dumped with
    replacement characters that would import back as different data. All escaping is
    JSON.stringify's — no JSON text is built by hand.

The dump object is built with Object.fromEntries, not object[key] = value: assignment
hits the inherited __proto__ setter and defines no own property, so a legal
libredb set app.libredb __proto__ v key would silently vanish from the dump. There is a
test for it.

Behaviour left to convention, since the issue does not specify it: the output file
overwritten like a shell redirect, parent directories are not created (clean ENOENT,
exit 1), and output is 2-space indented with a trailing newline because a dump is a
humans read and diff. No new flags.

Checklist

  • bun run gate passes locally (typecheck, format, lint, knip, build, size, test).
  • Tests are added or updated to cover the change (coverage is held at 100%).
  • A changeset is added for user-facing changes (bun run changeset).
  • Docs updated if behavior or the public API changed.
  • English only, no emoji, Conventional Commit PR title.
    Two points a reviewer will want flagged, which I would add as a PR comment rather than bury in the description:
    The changeset is minor, per CONTRIBUTING.md:76 ("new runtime capabilities are minor"). The counter-evidence is that the entire CLI shipped as a patch in 0.1.3 — I read that as pre-dating the written policy, but it is a one-word edit if you disagree.

The fatal TextDecoder slightly exceeds the issue's letter. The scope bullet only asoduce lone surrogates, which non-fatal decoding plus JSON.stringify alreadysatisfies. It is also imperfect by necessity: a key whose first byte is at or above 0xF5 falls outside the scanned range and is silently omitted rather than refused, because a half-open range cannot reach "all keys". Removing decodeText is a self-contained revert if you'd ra.

@cevheri
cevheri merged commit 1d8d665 into libredb:main Sep 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[P3][cli] Add an export command (JSON dump) as the import counterpart

2 participants