fix: tolerate malformed UTF-16 row values

[t8y2]

Summary

• decode NCHAR, NVARCHAR, and NTEXT row values lossily so unpaired UTF-16 surrogates become U+FFFD instead of terminating the query stream
• keep odd UTF-16 byte lengths as protocol errors, including an explicit NTEXT guard
• preserve strict decoding for XML and non-Unicode codepages
• add raw TDS frame regression tests covering lone surrogates, valid BMP and surrogate-pair text, nulls, and malformed lengths

Rationale

ColumnData::String stores a Rust String, which cannot represent unpaired UTF-16 surrogate code units. The current strict conversion returns Error::Utf16 while decoding the row, before callers can inspect or skip the affected value. Replacing only malformed surrogate sequences keeps the result stream readable without relaxing TDS framing validation.

Fixes #325

Testing

• cargo fmt --all -- --check
• cargo clippy --lib --features tds73 -- -D warnings
• cargo clippy --features=all
• cargo test --lib --features tds73 (125 passed)
• cargo test --lib --features=all (128 passed)
• cargo test --lib --no-default-features (106 passed)

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 8ff71c09-caf7-49e1-9ee2-199d59d9d5b1

📥 Commits

Reviewing files that changed from the base of the PR and between a6b4fcd and bb6b74b.

📒 Files selected for processing (3)

• src/tds/codec/column_data.rs
• src/tds/codec/column_data/string.rs
• src/tds/codec/column_data/text.rs

Included review availability: Your plan provides up to 2 included reviews per hour; 1 remains after this review.

---

Summary by CodeRabbit

• Bug Fixes

  • Improved handling of malformed Unicode in NVARCHAR, NCHAR, and NTEXT values by replacing invalid sequences with �.
  • Added validation for invalid odd-length NTEXT data, returning a protocol error instead of decoding incorrectly.
  • Preserved correct handling of valid Unicode, NULL values, code-page text, and strict XML decoding.

• Tests

  • Added coverage for malformed, valid, NULL, and invalid-length text values.

Walkthrough

NCHAR, NVARCHAR, and NTEXT now replace malformed UTF-16 sequences with U+FFFD. NTEXT rejects odd byte lengths with Error::Protocol. Valid Unicode and NULL values retain their existing behavior. BigVarChar continues to use collation code-page decoding. XML retains strict UTF-16 errors. Tests cover these decoding paths and regression cases.

Merge Risk: ⚪ Minimal · up to bb6b7

The change replaces malformed Unicode surrogate values while preserving protocol-length validation, and no actionable merge-blocking risk remains beyond normal checks and review.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely describes the main change: tolerating malformed UTF-16 row values.
Description check	✅ Passed	The description directly explains the decoding changes, error handling, scope, rationale, and regression tests.
Linked Issues check	✅ Passed	The changes satisfy issue #325 by replacing malformed UTF-16 sequences and preserving readable row streams.
Out of Scope Changes check	✅ Passed	The implementation and tests remain within the linked issue scope of tolerant UTF-16 row decoding.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

✨ Simplify code

• Create PR with simplified code

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}