Skip to content

Classify sqlserver transience and self-heal append reconnects #139

Description

@victorchutw

Implements ADR-0067. Part of the SQL Server destination cycle (map #106, sliced from #113).

Blocked by #137, #138

Context

The sqlserver connector is the first to classify any failure Transient (ADR-0048's engine stops being provably idle). The connector slices landed every site Terminal; this slice is the flip — ADR-0067's classification and the append self-heal are one decision and land together, because a transient marking without the reconnect would re-invoke a dead client.

The contract

  • The family (connection-loss-or-deadlock): Io-kind errors of the connection kinds (ConnectionRefused, ConnectionReset, ConnectionAborted, BrokenPipe, TimedOut, UnexpectedEof) ∪ server errors of class ≥ 20 (connection-fatal; the spike's mid-bulk session kill surfaces as error 596, class 21, at finalize()) ∪ deadlock victim 1205 (batch rolled back whole, connection survives). Everything outside stays Terminal: 18456 auth, certificate rejection (InvalidData), duplicate key 2627, bulk type mismatches, incompatible_destination_table.
  • Per-unit application:
    • begin(mode) applies the family in every mode — each call opens a fresh connection and commits nothing (connect, TLS, auth, Accept Family validation, full refresh's BEGIN TRAN+DELETE, merge's stage creation all included); a deadlocked full-refresh DELETE is genuinely transient. No Azure server-code allowlist (4060/40613) — the environment is on-premises.
    • write_chunk under full refresh and merge: Terminal always, without consulting the family — mid-transaction failure sheds every prior chunk; no session can accept a one-chunk re-attempt (the DuckDB structural argument).
    • write_chunk under append: applies the family. A 1205 re-attempts the same chunk on the surviving connection. Connection loss is transient because the writer self-heals: it marks the client dead, and the next re-attempt performs a pure reconnect (no Accept Family re-validation — session-begin work belongs to begin) before re-sending the same chunk; a failed reconnect classifies under the same family and consumes the same unit's ADR-0049 attempt budget, so reconnection is bounded without new configuration.
    • commit is never engine-retried (ADR-0048, structural) — the terminal COMMIT of full refresh and merge surfaces failures unchanged.
  • Standing choices: LOCK_TIMEOUT stays at the server default of infinite wait (1222 unreachable, deliberately unclassified); no jitter (ADR-0049's revisit condition is not triggered at parallelism 1); retry defaults (3 attempts / 200 ms / 5000 ms) untouched — restart-length outage tuning is destination-guide guidance in the docs slice, not a defaults change.
  • Zero new failure codes, knobs, or glossary terms.

Acceptance criteria

  1. Ungated table-driven classification tests: every family member classifies Transient at begin (all three modes) and at append write_chunk; every named Terminal error classifies Terminal; family members at full-refresh/merge write_chunk still classify Terminal.
  2. Gated self-heal test: mid-append, kill the session from a second connection (KILL <spid> via a helper connection or docker exec … sqlcmd); the failed chunk re-attempts through a reconnect and the load completes — final row count proves no lost and no duplicated rows (the killed batch was atomic).
  3. Gated begin-retry visibility test: a transient-shaped begin failure (e.g. a refused port with retry configured) consumes the configured attempts and reports them through the existing retry surface before failing.
  4. The report surface is unchanged in shape — retry facts appear exactly as the engine already reports them.
  5. CHANGELOG.md Unreleased entry: append survives connection loss via bounded reconnect; deadlocks and connection-fatal errors now classify transient on re-attemptable units.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    ready-for-agentFully specified and ready for an AFK agent

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions