Resolve Tiberius decimal(38,38) bulk-row limitation

[victorchutw]

Problem

Issue #134 requires every declared decimal(p,s) value to become a Tiberius Numeric at the declared scale. Data Spark's accepted schema contract permits decimal(38,38) (0 <= s <= p <= 38), but Tiberius 0.12.3 panics for that valid declaration:

thread panicked at tiberius-0.12.3/src/tds/numeric.rs:35:9:
assertion failed: scale < 38

The same assertion remains on the Tiberius upstream main branch as of 2026-09-02. Upstream PR tiberius-rs/tiberius#436 now proposes accepting scale 38, but its current patch leaves Numeric::precision() returning 39 for values with no integral digits at scale 38. The optional rust_decimal and bigdecimal conversions ultimately call the same constructor, so enabling either feature does not restore the complete scale-38 invariant.

This blocks #134. Silently narrowing the schema contract would conflict with ADR-0044, and treating the value as a write failure would conflict with ADR-0062's exhaustive two-sliver list. ADR-0069 already defines the applicable response ladder: use a pinned revision on a maintainer-controlled fork when this connector needs an upstream fix that has not been released.

Reproduction

Construct an Arrow Decimal128(38, 38) containing scaled value -1, then encode it with:

Numeric::new_with_scale(-1, 38)

Tiberius 0.12.3 panics before a bulk row can be emitted. Merely changing the constructor assertion is incomplete: the current zero-integral-part precision calculation returns 1 + scale, which would describe the value as the invalid NUMERIC(39,38) when Tiberius emits parameter metadata without destination type context.

Decision

Preserve Data Spark's complete declared Decimal contract and fix Tiberius through the fork path anticipated by ADR-0069.

• decimal(38,38) remains a valid Declared Type and maps to SQL Server DECIMAL(38,38) verbatim.
• The Tiberius behavior is a dependency defect, not a Rejected Record condition or a third SQL Server representability sliver.
• Base a public, maintainer-controlled fork on upstream tag v0.12.3 and carry one focused Numeric patch.
• Accept scale <= 38; continue rejecting scales above 38.
• Correct the zero-integral-part precision invariant to max(scale, 1). Thus 0.05 is described as NUMERIC(2,2), and a scale-38 fractional value is described as NUMERIC(38,38). Preserve integral_digits + scale when integral digits exist.
• Pin Data Spark to the fork with an immutable full commit SHA in Cargo.toml and Cargo.lock; do not track a branch.
• Keep the fork as a patch carrier rather than continuously syncing upstream main. It carries its tests, but Data Spark does not maintain a separate fork CI system; every new revision must pass the required local/live verification before adoption.
• Return to crates.io after an upstream release contains the complete fix and passes the same Data Spark probes.

Unsafe construction of Tiberius's private Numeric fields is not acceptable.

Upstream coordination

Start with a factual comment on tiberius-rs/tiberius#436 showing the precision-39 reproduction and the complete invariant. If its author or an upstream maintainer requests a split, open a focused upstream issue and PR. Record a public upstream trace before closing this issue, but do not wait for upstream merge or release.

Scope split with #134

This issue owns the dependency resolution: the fork patch and tests, immutable dependency revision, ADR updates, upstream trace, and a direct Data Spark dependency smoke test.

Issue #134 continues to own the production Dataset Schema-to-SQL-Server-row seam. Its row-encoding tests must include the focused Arrow Decimal128(38,38) scaled value -1 to Tiberius Numeric value -1, scale 38 regression. #134 remains blocked until this issue closes, then moves back to ready-for-agent.

Acceptance criteria

1. A maintainer-controlled fork based on upstream v0.12.3 contains a focused Numeric patch with no unrelated dependency or driver changes.
2. Fork unit tests cover scale 38 construction, corrected precision metadata, encode/decode, relevant positive/negative/zero boundaries, and continued rejection of scale 39.
3. A live Tiberius test round-trips a DECIMAL(38,38) value through SQL Server, including parameter metadata, without panic, rounding, or value change.
4. Data Spark pins the fork by full commit SHA, regenerates Cargo.lock, and has a direct dependency smoke test proving scaled value -1 at scale 38 is deterministic and reports precision 38.
5. ADR-0069 records the activated fork path, base and pinned revision, minimal patch, upstream trace, verification posture, and crates.io exit condition. ADR-0062 records that patched Tiberius preserves the decimal(38,38) exact-fit mapping and does not add a representability sliver. ADR-0044 and CONTEXT.md remain unchanged.
6. The upstream coordination trace described above is linked from this issue; upstream merge or release is not required.
7. Generate SQL Server DDL and bulk rows from the dataset schema #134 contains the production-seam regression criterion, remains blocked while this issue is open, and is moved from blocked to ready-for-agent when this issue completes.
8. No CHANGELOG entry is added: SQL Server has no live load mode yet, so the dependency correction is not independently user-observable.

Out of scope

• Implementing the production DDL or row-encoding seam owned by Generate SQL Server DDL and bulk rows from the dataset schema #134.
• Narrowing Data Spark's Decimal declaration contract.
• Adding a SQL Server write-failure sliver for decimal(38,38).
• Maintaining a separate CI platform for the fork.
• Waiting for an upstream merge or release.

[victorchutw]

Upstream coordination trace (AC6):

• Fork: https://github.com/victorchutw/tiberius — branch data-spark/numeric-scale-38, base upstream c34fab2 ("Bump v0.12.3": the v0.12.3 tag plus its own manifest version bump; tree byte-identical to the published 0.12.3 sources, whereas the tag commit itself still declares 0.12.2).
• Patch commit (the pinned revision): victorchutw/tiberius@65c1a32 — src/tds/numeric.rs only, plus unit and live tests.
• Upstream comment with the precision-39 reproduction on PR #436's head (3945747) and the complete max(scale, 1) invariant: harden: malformed-input hardening, non-panicking encoders, unit coverage tiberius-rs/tiberius#436 (comment)

Verification of the fork revision before adoption (Data Spark feature set tds73,rustls,chrono, SQL Server 2022 CU26 container, tokio and async-std runtimes): lib unit tests 128 passed; bulk 84 passed; query 162 passed; deadlocks 1 passed; login_errors 1 passed; custom-cert 2 failed on UnsupportedCertVersion — those two tests require upstream's custom-certificate container image and fail identically without the patch, so they are environmental. The numeric/decimal tests also pass with rust_decimal,bigdecimal enabled (20 passed). Clippy adds no warning inside the patched line ranges.