Conversation
Keep Arrow field metadata through literal rendering and reuse the existing SQL type lookup for casts, including array elements and structs. Share BigQuery type checks and exclude JSON/GEOGRAPHY from ORDER BY. Render JSON through the common value dispatch and cast path. Type empty BigQuery arrays explicitly; nonempty arrays reuse their element casts. Cover reported schemas, nested values, nulls, empty fixtures, and other adapters' mappings. Validate generated fixtures with live BigQuery dry runs. Signed-off-by: Fredrik Fornwall <fredrik@fornwall.net>
fornwall
force-pushed
the
bigquery-data-type
branch
from
September 12, 2026 09:58
aeb3917 to
ed226a1
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Unit-test fixtures become SQL rows whose types must match the referenced columns. The renderer previously worked from Arrow data types alone: BigQuery
GEOGRAPHYandJSONboth look like strings. Fixture casts also reused the type mapping fordbt seed, which choosesDATETIMEfor timestamp values even when the referenced column isTIMESTAMP.For example, this model returns
GEOGRAPHY,TIMESTAMP, andJSONcolumns:A unit test supplies the matching expected row:
Before this fix, the expected fixture row used these casts (simplified SQL):
The
STRINGvalues are incompatible with the model'sGEOGRAPHYandJSONcolumns. The timestamp cast fails because its UTC offset is invalid forDATETIME.With this fix, the same fixture uses:
The renderer preserves the reported BigQuery type from field metadata, including nested arrays and structs and
NUMERIC/BIGNUMERICvalues. Empty arrays get explicit element types.JSONandGEOGRAPHYare excluded fromORDER BY, which BigQuery does not support for these types. The fix reuses the existing type lookup and rendering paths; other adapters keep their current mappings.Related: #13804, #14271, #14625, #15708, #15882