Nothing prevents the next non-additive sector-data change landing without a lens

[LKSNDRTMLKV]

Reported from the consumer side: odal-node/dpp-engine#81.

dpp-domain 0.15.0 made TextileData.gtin a required field (Gtin, no Option, no #[serde(default)]) and, separately, renamed the wire key backing country_of_origin from countryOfManufacturing to countryOfOrigin. The rename is still visible in the crate's own docs: SectorData's doc comment (src/domain/sector/data/sector_data.rs) shows an example with the old key —

{ "sector": "textile", "fibreComposition": [...], "countryOfManufacturing": "BD" }

right next to a struct that now requires countryOfOrigin. Neither change was additive, and neither shipped with any way for a consumer to keep reading documents written under the old shape.

Why this matters more than a normal breaking change

A struct used only in memory can break its callers at compile time — cheap, loud, fixed once. Passport and every SectorData variant are not that: they are the literal on-disk shape of every previously-published digital product passport a node has ever stored. A non-additive change to one of these structs doesn't just require consumers to update their code, it makes every already-written document of that shape permanently undeserialisable the moment a node upgrades its dpp-domain pin — a runtime failure against data, discovered per-request, with no compile-time signal anywhere. Downstream (odal-node/dpp-engine#81), this took out reads for 244 of 276 passports in a database the instant the node was upgraded, because those documents predated the requirement being added.

The crate already has the right instinct — for a different axis

SectorData's hand-written Deserialize deliberately treats an unrecognised sector tag as Other { sector, data } rather than failing, specifically so that "a passport for a sector added to the catalog after this crate was released survives a round trip unchanged — which is the property that makes adding a product group a data change rather than a release" (verbatim from that impl's doc comment). That's exactly the right principle. It just stops at the sector boundary: a known sector's struct gaining a required field, or renaming a key, gets none of that protection — it fails exactly as hard as the crate's own design explicitly tries to avoid for the sector-tag case.

What might close this (not a decision, for discussion)

1. A written contract: any change to a struct that is part of a persisted document (Passport, any SectorData variant) must be additive — Option<T> + #[serde(default)], or a rename that accepts the old key as an alias — unless it ships alongside an explicit transform (see chore(deps): bump actions/checkout from 4 to 7 #2) and a schema_version bump consumers can key off.
2. Expose the transform mechanism from this crate, since it already owns SectorCatalog and schema_versions per sector (src/catalog/). Something like SectorData::upgrade(from_version: &str, value: serde_json::Value) -> Result<serde_json::Value, _> would give every consumer — not just dpp-engine — one canonical place to keep old documents readable, instead of each reimplementing it.
3. A compatibility test in this repo: freeze one fixture per sector per version actually listed in that sector's schema_versions, and assert it still deserialises on every change. SectorCatalog already declares which versions are supposed to be valid — this repo is in the best position to enforce that its own structs still parse everything they claim to support, rather than leaving that entirely to downstream consumers to discover.

dpp-engine has added a version of (3) scoped to what's actually in one node's database (odal-node/dpp-engine#81) — it can only catch the next incompatible change, not retroactively protect data already written under an old shape. That half of the fix has to live here.

[LKSNDRTMLKV]

Partial progress, recorded rather than closing — this asks for more than one change and I can only verify some of it.

Landed:

• The upcast lens registry, so a stored record written against an older sector schema is readable at a newer one, with honest derived/lens_chain/lossy provenance rather than a silent identity.
• from_stored upcasting as far as the registered lenses reach, instead of refusing when no exact path exists.
• The envelope's rule is now stated on Passport::schema_version: additive-only, permanently, with no exception path — because a lens transforms one sector's sub-object while an envelope field is shared by every sector's stored documents.
• Consumer-side, the persisted document is no longer overwritten wholesale, so keys the struct does not model survive a read-modify-write.

Not verified as done: whether the stale countryOfManufacturing example in SectorData's doc comment was corrected, and whether anything prevents the next non-additive change to a known sector's struct from landing without a lens.

That second one is the substance of this issue and I have not confirmed it. Leaving open.