fix: make table updates retryable and cleanup-safe

[zhjwpku]

Table updates currently have inconsistent retry and cleanup behavior between
standalone updates and explicit transactions.

This change:

• Registers standalone updates in their temporary transaction before Apply, so
  commit retries can replay the update instead of losing it.
• Replays explicit transaction updates when refreshed metadata changes, while
  keeping standalone retries replayable.
• Makes Apply failures terminal for explicit transactions and adds explicit
  Abort cleanup.
• Separates known failure cleanup from CommitStateUnknown, preserving staged
  files when the catalog outcome is uncertain.
• Tracks staged snapshot files before writing them and retries failed cleanup
  best effort.
• Cleans no-op snapshot updates without reporting a commit.
• Defers ExpireSnapshots physical cleanup until the final transaction commit
  and uses reachable cleanup for explicit transactions.
• Keeps the existing Apply() APIs and removes unnecessary freeze/deep-copy
  lifecycle machinery.

The implementation also adds coverage for standalone retry, transaction retry,
no-op updates, Apply failures, Abort, unknown commit outcomes, staged-file
cleanup, ExpireSnapshots retry, and finalization/reporting behavior.

[Copilot]

Pull request overview

This PR tightens the lifecycle semantics for PendingUpdate/Transaction so updates have deterministic ownership and always reach a terminal finalized state across both standalone commits and explicit transactions, including transaction commit retries.

Changes:

• Switch several update factories to return std::shared_ptr<...> and make PendingUpdate enable_shared_from_this so transactions can retain/finalize updates safely.
• Ensure updates are finalized exactly once (including no-op commits and apply failures) and prevent reuse by introducing finalized_ plus a centralized Transaction::FinalizeUpdates(...).
• Add a commit-retry lifecycle marker (committing_) guarded by RAII to avoid premature finalization during retry reapplication; expand tests to cover these paths.

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
src/iceberg/update/row_delta.h	Factory now returns shared_ptr for shared ownership.
src/iceberg/update/row_delta.cc	Factory implementation updated to shared_ptr.
src/iceberg/update/rewrite_files.h	Factory now returns shared_ptr; doc updated.
src/iceberg/update/rewrite_files.cc	Factory implementation updated to shared_ptr.
src/iceberg/update/replace_partitions.h	Factory now returns shared_ptr.
src/iceberg/update/replace_partitions.cc	Factory implementation updated to shared_ptr.
src/iceberg/update/merge_append.h	Factory now returns shared_ptr.
src/iceberg/update/merge_append.cc	Factory implementation updated to shared_ptr.
src/iceberg/update/fast_append.h	Factory now returns shared_ptr.
src/iceberg/update/fast_append.cc	Factory implementation updated to shared_ptr.
src/iceberg/update/delete_files.h	Factory now returns shared_ptr.
src/iceberg/update/delete_files.cc	Factory implementation updated to shared_ptr.
src/iceberg/update/pending_update.h	Require shared ownership for commit; enable shared_from_this.
src/iceberg/update/pending_update.cc	Scoped temporary transaction binding; eager finalization on apply failures outside commit retries.
src/iceberg/transaction.h	Add FinalizeUpdates, finalized_, and committing_ to support deterministic finalization and retry safety.
src/iceberg/transaction.cc	Centralize finalization; detach temporary transactions; RAII retry lifecycle marker.
src/iceberg/test/transaction_test.cc	New tests for standalone bindings, shared-ownership enforcement, no-op commit finalization, apply-failure finalization, retry reapplication, and exception lifecycle restoration.
src/iceberg/test/replace_partitions_test.cc	Update helper return type to shared_ptr.
src/iceberg/test/merging_snapshot_update_test.cc	Update test-only update factories/return types to shared_ptr.
src/iceberg/test/fast_append_test.cc	New test asserting staged-file cleanup on transaction apply failure.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[wgtmac]

I simplified the patch and kept the changes needed for correct retry and cleanup:

• Standalone updates are registered in their temporary transaction before Apply. This prevents a retry from replaying an empty update list and returning success without the update.
• Explicit transactions still allow only one pending operation. An Apply failure marks the transaction failed, cleans staged files best effort, and rejects further use.
• Explicit transactions replay updates only when refreshed metadata has changed. Standalone updates still re-apply on each retry. A replay failure stops the outer retry loop.
• Known commit failures and Abort clean staged files. CommitStateUnknown preserves all files because the catalog may already have committed them.
• Snapshot updates register file paths before writing. Failed deletions remain tracked, so Abort can retry them later.
• Snapshot updates that become no-ops clean their temporary files and do not emit a commit report.
• ExpireSnapshots performs physical deletion only after the final transaction succeeds. Explicit transactions use reachable cleanup against the final metadata, so files referenced by later updates are preserved.
• I removed the freeze/deep-copy machinery, EnsureMutable(), and redundant lifecycle state. Callers are instead required not to modify an update or its inputs after the first Commit.
• I kept the existing Apply() names. Cleanup and finalization are internal transaction hooks.
• I also removed tests for unsupported callback re-entry, restored identity assertions, and fixed validation tests that created duplicate snapshot metadata.