Repository navigation
Commit 844e67b
v5.19: Address all five critique findings (early-acquire lock, pagination, fail-fast poll, route validation, docs)
P1 — Non-adhoc early-acquire lock repair (production-critical):
- Introduce _main_row_acquired / _acquired_from_status / _entered_bucket_execution flags
- After CAS succeeds, record prior status in _acquired_from_status
- Set _entered_bucket_execution = True just before first destructive bucket work in
each of Cases 1, 2a, and 2b/3 (before backup creation or delete_data)
- In the outer except block: if _main_row_acquired and not _entered_bucket_execution,
run a targeted UPDATE that restores only status + update_time and leaves
range_start, range_end, last_backup_table, job_instance_id untouched
- In the no-new-data branch: same targeted restore so the row is not left in Running
when there is simply nothing new to process
- Per-bucket write-ahead, same-run restore, and _recover_from_backup_if_needed
are entirely unchanged once _entered_bucket_execution is True
P2 — Parameter discovery pagination:
- _discover_dataflow_parameters now loops via continuationToken / continuationUri
across all pages before deciding support/non-support
- Early exit as soon as both RangeStart and RangeEnd are confirmed on any page
- Logs page count and total parameter count for operator diagnostics
- DataflowNotParametricError handling and transient-failure logic unchanged
P2 — Exact CI/CD job-instance poll fail-fast:
- Add module-level _DeterministicPollError(status_code, request_id, api_error_code)
- Exact-instance poll raises _DeterministicPollError for HTTP 400/403/404 instead
of returning None; 5xx and network errors still return None (transient)
- wait_for_refresh_completion adds a _HARD_POLL_GRACE=3 consecutive-error counter;
after 3 hard errors raises RuntimeError with full diagnostics rather than
keeping the bucket active until the full timeout expires
P3 — route_mode validation in refresh_dataflow:
- Add the same allowlist check at the top of refresh_dataflow's try block so direct
callers get the same fail-fast contract as execute_incremental_refresh
P3 — Documentation drift:
- Remove all stale sys.exit wording from notebook markdown and README feature
lists and failure-flow descriptions; replace with accurate exception-raise language
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>1 parent 8bb6b29 commit 844e67b
2 files changed
Lines changed: 264 additions & 63 deletions
0 commit comments