feat(storage): upgrade format 8 and 9 graphs to format 14

[azimafroozeh]

What & why

This PR adds a direct, offline upgrade from storage formats 8 and 9, written by release 0.11.x, to format 14. It extends the format 13 route from PR 873.

• Preserve branches, commit ids and history without rewriting graph data.
• Validate the root schema against main's pinned datasets, archive it for interrupted runs, and retire a leftover schema-apply lock once main is fenced.
• Add release-binary compatibility journeys covering schema evolution, prior cleanup, and graphs upgraded from v0.10.0 through v0.11.0.

Backing issue / RFC

• Implements RFC 0064: Explicit storage upgrades, with the format 8/9 route documented in this change.

Checklist

• Change is focused (extend the existing offline upgrade to formats 8 and 9).
• Tests added/updated (conversion, refusal, interruption and cross-version journeys).
• Public docs updated (upgrade guide, CLI reference and release notes).
• Reviewed against architectural invariants (explicit conversion, source validation and refusal during incomplete upgrades).

Local verification

Recorded on the implementation tree after review fixes; not rerun after applying the patch to current main:

• cargo test -p omnigraph-catalog --lib: 182 passed.
• cargo test -p omnigraph-engine --lib --features failpoints db::upgrade::tests: 50 passed.
• python3 scripts/check-storage-upgrade-ci.py --self-test: 13 passed.
• cargo fmt --all -- --check: passed.
• cargo test --workspace --exclude omnigraph-gqt --locked --test crossversion_upgrade --features "omnigraph-engine/failpoints,omnigraph-cluster/failpoints" -- --test-threads=1: four tests executed successfully, including the genuine format 13 journey. Nine returned at skip checks. All four journeys requiring v0.11.0 remain unverified because that binary was unavailable.
• cargo test --workspace --exclude omnigraph-gqt --exclude omnigraph-dst --locked --features omnigraph-engine/failpoints,omnigraph-cluster/failpoints --no-fail-fast: not rerun after review fixes. Earlier validation retained four sandbox failures: three CLI session-cache tests and blob::tests::external_blob_file_policy_rejects_special_files.
• cargo clippy --workspace --all-targets -q --message-format=short: passed before review fixes; not rerun on the final tree.
• ./target/debug/omnigraph-vocabulary-guard check --surface rust-string --base 4b1ae43fa9442007e353d3139a226d24bb5dc8a3 --inventory tools/omnigraph-vocabulary-guard/graph-vocabulary-inventory.tsv --openapi openapi.json: failed on unclassified occurrences and stale inventory entries at the merge base. The same command with --surface openapi passed; --surface user-docs is unsupported by this version.

Notes for reviewers

• Stop all readers and writers and retain a verified whole-root backup. Run omnigraph upgrade <graph> --check, then omnigraph upgrade <graph>. Cluster-managed graphs still require export and rebuild.
• Every pre-upgrade 0.11.x commit uses the schema live at upgrade, matching 0.11.x's single-schema model. Historical schemas are not reconstructed.
• If 0.11.x previously ran cleanup or schema apply --allow-data-loss, the first cleanup after upgrading must use --keep 1 --confirm alone. The upgrade cannot recover previously deleted dataset versions.

[ragnorc]

@azimafroozeh Storage Upgrade Compatibility has been red on main since this PR. The same three journeys fail on this PR's own run, its merge-queue run, main's push runs for #876 and #877, the v0.12.0 tag run, and every open PR (#871, #878). It's a required context, so per .github/branch-protection.json this is stop-the-line: nothing can merge until it's fixed or reverted. Two separate causes:

1. genuine_v0_10_0_to_stamp_8_… and genuine_v0_10_0_to_stamp_9_by_default_…: test fixture, not engine.
journey_fixture (crates/omnigraph-cli/tests/crossversion_upgrade.rs) appends the edge row
{"edge":"Cites","id":"citation-1",…} with a top-level id. The 0.11.0 binary accepts that; the 0.10.0 binary refuses it before any upgrade code runs:

old binary `predecessor load` failed
Error: line 6: unknown top-level graph batch field 'id'

Possible fix: give the 0.10.0 journeys a fixture without the edge id, or with it inside data as at line 1144, and check the later edge assertions don't depend on citation-1.

2. genuine_v0_11_0_storage_upgrade_after_predecessor_cleanup: a retention question.
After the 0.11.0 cleanup --keep 1, the upgrade, and the new cleanup --keep 1, the final cleanup --older-than 7d still fails the trace for node:Doc and edge:Cites:

the collector's trace did not finish; nothing is deleted for this table: linear version 1, pinned by a retained `__manifest` version, is absent from the listing; … linear version 4 …

The predecessor's cleanup reclaimed those linear versions, but the upgraded history still has __manifest versions pinning them. --older-than 7d retains them because they're minutes old, even after --keep 1. Is the expected outcome that --keep 1 drops the pre-upgrade versions so a later --older-than passes? If so, either retention doesn't reach the upgraded history rows, or the upgrade should mark reclaimed pins. Or is the assertion wrong? This is your design call.

Happy to send the fixture fix for (1) as a separate PR if that helps. (2) needs your decision. Reverting #876 is the other way to get main green, but I'd rather not lose the upgrade route.