Skip to content

fix(bigquery): preserve reported types in unit test fixtures - #16294

Open
fornwall wants to merge 1 commit into
dbt-labs:mainfrom
fornwall:bigquery-data-type
Open

fornwall wants to merge 1 commit into
dbt-labs:mainfrom
fornwall:bigquery-data-type

Conversation

@fornwall

@fornwall fornwall commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Unit-test fixtures become SQL rows whose types must match the referenced columns. The renderer previously worked from Arrow data types alone: BigQuery GEOGRAPHY and JSON both look like strings. Fixture casts also reused the type mapping for dbt seed, which chooses DATETIME for timestamp values even when the referenced column is TIMESTAMP.

For example, this model returns GEOGRAPHY, TIMESTAMP, and JSON columns:

-- models/fixture_types.sql
SELECT
  ST_GEOGPOINT(100, -37) AS location,
  TIMESTAMP '2026-01-01 00:00:00+00' AS created_at,
  PARSE_JSON('{"segmentCode":"HORECA"}') AS payload;

A unit test supplies the matching expected row:

# models/fixture_types.yml
unit_tests:
  - name: preserves_column_types
    model: fixture_types
    given: []
    expect:
      rows:
        - location: "ST_GEOGPOINT(100, -37)"
          created_at: "2026-01-01 00:00:00+00"
          payload: '{"segmentCode":"HORECA"}'

Before this fix, the expected fixture row used these casts (simplified SQL):

SELECT
  CAST('ST_GEOGPOINT(100, -37)' AS STRING) AS location,
  CAST('2026-01-01 00:00:00+00' AS DATETIME) AS created_at,
  CAST('{"segmentCode":"HORECA"}' AS STRING) AS payload;

The STRING values are incompatible with the model's GEOGRAPHY and JSON columns. The timestamp cast fails because its UTC offset is invalid for DATETIME.

With this fix, the same fixture uses:

SELECT
  CAST(ST_GEOGPOINT(100, -37) AS GEOGRAPHY) AS location,
  CAST('2026-01-01 00:00:00+00' AS TIMESTAMP) AS created_at,
  CAST(PARSE_JSON('{"segmentCode":"HORECA"}') AS JSON) AS payload;

The renderer preserves the reported BigQuery type from field metadata, including nested arrays and structs and NUMERIC/BIGNUMERIC values. Empty arrays get explicit element types. JSON and GEOGRAPHY are excluded from ORDER BY, which BigQuery does not support for these types. The fix reuses the existing type lookup and rendering paths; other adapters keep their current mappings.

Related: #13804, #14271, #14625, #15708, #15882

@fornwall
fornwall requested a review from a team as a code owner September 12, 2026 09:42
@cla-bot cla-bot Bot added the cla:yes label Sep 12, 2026
Keep Arrow field metadata through literal rendering and reuse the existing
SQL type lookup for casts, including array elements and structs. Share
BigQuery type checks and exclude JSON/GEOGRAPHY from ORDER BY.

Render JSON through the common value dispatch and cast path. Type empty
BigQuery arrays explicitly; nonempty arrays reuse their element casts.

Cover reported schemas, nested values, nulls, empty fixtures, and other
adapters' mappings. Validate generated fixtures with live BigQuery dry runs.

Signed-off-by: Fredrik Fornwall <fredrik@fornwall.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant