Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CONTEXT.md
Original file line number Diff line number Diff line change
Expand Up @@ -222,6 +222,10 @@ _Avoid_: Good row, valid row, kept record
A temporary data object prepared so a destination can load records through its native batch loading path.
_Avoid_: Temp file, intermediate file, upload

**Created Shape**:
The destination table a load creates for a dataset schema: one column per dataset field, in field order, each with the exact-fit destination column type and the field's nullability.
_Avoid_: Table template, default table, auto schema

**Accept Family**:
The set of existing destination column shapes a dataset field may write into without silent value degradation, plus explicitly opted-in exceptions.
_Avoid_: Compatible types, type whitelist, column tolerance
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,5 @@ Every dataset field type maps to the narrowest SQL Server column type that holds
Two mappings are judgment calls rather than forced moves. `utf8` becomes `NVARCHAR(MAX)` because the dataset type is unbounded text and the mapping must stay true under chunked streaming: the writer sees one chunk at a time and can never know a global maximum length, and an append or merge table created from early data must not become invalid when later loads carry longer values. The cost — LOB storage and no plain index on the column — is schema-design territory that belongs to the table's consumers, not the loader. Instant timestamps land as UTC-normalized `DATETIME2(6)` rather than `DATETIMEOFFSET(6)` because the maintainer's production environment stores datetimes exclusively in the `DATETIME` family, so `DATETIME2` is what every downstream consumer already reads; the price, stated plainly, is that wall-clock and instant columns are indistinguishable by destination type — the offset discipline of ADR-0043 lives only in the load definition and the report. Nothing of the instant is lost beyond that marker: ADR-0043 already normalizes instants to UTC at parse without retaining the original offset, so a `DATETIMEOFFSET` column would have carried a constant `+00:00`. Precision is 6, not SQL Server's default 7, because the dataset contract is microseconds (ADR-0043): the column type states the real contract instead of implying 100-nanosecond precision that never arrives.

Because every mapping is exact, a value that satisfies the dataset schema can fail destination typing only in two enumerated slivers: `NVARCHAR` values longer than 32,767 characters — the Tiberius bulk-path client guard measured by the spike (encoded UCS-2 length ≤ 65,535 bytes), a driver-path limit rather than a server one — and timestamps whose value lies outside SQL Server's year range of 0001–9999, reachable because the strict four-digit-year parse menu admits year 0000 and instant UTC normalization can carry a boundary value across the range edge (DuckDB's ±290k-year range never sees either case). These land as write failures, never Rejected Records: a Rejected Record is a record that violates the chosen schema or load rules, and these records violate neither — they exceed one destination path's representable range. Freezing a driver cap into record-level semantics would let the same record load on DuckDB, reject on SQL Server, and load again if the write path ever changes — semantics drifting with implementation. Failure codes, retry classification, and any client-side preflight are write-semantics and error-classification decisions (#109, #112); what an existing table's columns are allowed to look like on append or merge — including production tables whose datetime columns are legacy `DATETIME` — is likewise a named #109 question, not part of this mapping. Non-finite `float64` values are absent from the sliver list because ADR-0063 removes them at parse. `decimal(38,38)` is likewise not a third sliver: the released tiberius 0.12.3 panicked on scale 38 and would have described a scale-38 fraction as the invalid `numeric(39,38)`, but that was a dependency defect, not a representability limit — the patched revision ADR-0069 pins (#156) accepts scale 38 and declares such a value as `numeric(38,38)`, so the exact-fit `DECIMAL(p,s)` mapping holds across the complete declaration range. Alternatives rejected: `NVARCHAR(4000)` (an arbitrary bound that trades the measured 32,767 cap for a lower, server-side failure surface); length sized from observed data (unknowable under streaming, invalidates evolving append targets); pinning a collation (asserts an opinion the database already answers); `DATETIMEOFFSET(6)` for instants (a self-describing marker, but it buys two bytes per row of type distinction the report already records, against the grain of the environment this tool actually loads into); `DATETIME2(7)` (implied false precision); legacy `DATETIME` as the created type (~3.33 ms rounding against the never-round ethos of ADR-0044, and a 1753 floor below the menu's range); and tightening the parse menu to 0001–9999 (welds one destination's range into the shared contract every destination pays for).

Range check activated (2026-09-03, #134). The bulk-row encoder refuses a timestamp outside 0001-01-01..=9999-12-31 itself, as `destination_write_failed` naming the record, the dataset field, and the value, instead of letting the value reach the driver. This is forced, not chosen: the pinned tiberius revision's `Date::new` asserts on a day count with bits above 24 — which is what a pre-0001 day becomes once cast to `u32` — rather than returning an error, and a year-0000 value is reachable both through the four-digit-year menu and through instant normalization (`0001-01-01T00:00:00+01:00`), so without the check this sliver would surface as a panic, which is not the write failure this ADR assigns it. The upper bound is the same rule applied symmetrically: the driver would let years 10000–45941 travel to the server and only assert again from day 2^24, and one range with one message states this ADR's sliver rather than the driver's accident, while naming the offending record where a server-side rejection of the bulk batch would not. The check is the conversion reporting what it cannot produce — not a preflight pass, and not record semantics: the record remains a Surviving Record of the dataset schema, and the failure is a chunk write failure like any other. The `NVARCHAR` sliver stays unpreflighted, because tiberius reports an over-long value as an error at `send()`. No exit condition is needed: should tiberius ever return an error instead of asserting, the check becomes redundant and stays correct.
6 changes: 6 additions & 0 deletions src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,12 @@ mod dispatch;
mod rejection;
mod retry;
mod schema;
// The SQL Server mapping module has no live consumer until the first
// sqlserver write session (#136); until then its items are reached only by
// tests and would trip dead_code in non-test builds. Drop the attribute
// with that consumer.
#[cfg_attr(not(test), allow(dead_code))]
mod sqlserver;

use connector::{
destination_connector, resolved_source_format, source_connector, DestinationWriteFacts,
Expand Down
Loading