Skip to content

fix: a metadata update must not rebase over a restore - #9361

Open
wkalt wants to merge 1 commit into
mainfrom
ticket/ent-2602/restore-conflicts-with-metadata-update
Open

wkalt wants to merge 1 commit into
mainfrom
ticket/ent-2602/restore-conflicts-with-metadata-update

Conversation

@wkalt

@wkalt wkalt commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

A restore replaces the manifest with an older one, schema and field metadata included. An UpdateConfig that carries schema or field metadata updates was allowed to rebase over it in either commit order, which reinstates metadata the restore discarded. A caller that reads a column's metadata, merges into it and writes it back is the exposed case: the value it computed describes a schema the table no longer has.

The same pair is already rejected for Merge, with the same reasoning, and Overwrite already conflicts with a metadata update for the same reason a restore should. This adds Restore to the existing symmetric guard beside the merge one, as an incompatible conflict, because re-reading cannot make the update applicable.

Plain config upserts are unaffected and still rebase over a restore. Only schema and field metadata updates change, and a caller racing a restore now sees a conflict instead of a silently reinstated value. Previously such a rebase could also fail deep in manifest building with Field with id N does not exist when the restore had removed the column.

The transaction format doc lists restore as trumping every other operation, so its compatibility section is updated alongside.

A restore rewinds schema and field metadata with the rest of the manifest.
An UpdateConfig carrying schema or field metadata updates rebased over it
reinstated metadata the restore discarded, in either commit order, while
the same pair is already rejected for Merge and for Overwrite.
@github-actions github-actions Bot added A-format On-disk format: protos and format spec docs format-change A change to the format spec, which requires a vote. Remove if minor (e.g. fixing typo). bug Something isn't working labels Sep 17, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Important

Format specification vote

This PR modifies the Lance format specification, so it requires 3 binding +1 votes from PMC members (excluding the proposer) and a minimum 72-hour voting period, weekends excluded, before it can merge. Vote by approving this PR (+1) or requesting changes (−1, a veto). See the voting process.

Status: ❌ Blocked — 0 of 3 required approvals

Approvals (this commit) none (0/3)
Vetoes none
Voting period ends Tue 2026-09-22 23:04 UTC (16:04 PDT)

Updated automatically by the format-spec vote gate, which re-checks every 15 minutes — just voted? Re-check now (press Run workflow; leave the input blank to re-check every open format PR). A PMC member may apply the format-waived label to waive the vote for a trivial edit (typo, wording, formatting).

@lance-gatekeeper lance-gatekeeper Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gate recommendation: request changes.

The stale-metadata race is real, and the resolver guard closes it. The current PR cannot be accepted as a combined specification-and-implementation change: establish the restore/metadata compatibility contract in a spec-only vote first, then land the resolver and regression test in a follow-up based on that approved contract.

other operation. Here are the operations that conflict with Restore:

- UpdateMemWalState
- UpdateConfig (only if it updates schema or field metadata, which a restore rewinds)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This format-spec contract is bundled with its implementation, but docs/src/format/AGENTS.md requires format-spec changes to be isolated for the PMC vote and implementation to follow separately. Please make this PR spec-only and move the resolver/test changes to a follow-up, or first merge a separate spec PR and then rebase this implementation without this doc hunk. That keeps the voted contract independent of implementation details.

@lance-gatekeeper lance-gatekeeper Bot added the K-changes Latest Gatekeeper recommendation requests changes. label Sep 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-format On-disk format: protos and format spec docs bug Something isn't working format-change A change to the format spec, which requires a vote. Remove if minor (e.g. fixing typo). K-changes Latest Gatekeeper recommendation requests changes.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant