Problem
BIGQUERY preserves bigquery.Value rows and RowIterator.Schema while reading a query, but immediately converts every cell to display text through formatBigQueryValue. The returned Result therefore contains only Rows []Row.
Consequences for export formats include:
NULL, booleans, integers, and floats become JSON strings in JSONL output;
- JSON, ARRAY, and RECORD values become quoted JSON text;
- nested RECORD field names are lost because nested values are positional;
- native type distinctions such as NUMERIC versus BIGNUMERIC do not survive downstream;
- the display spelling may remain useful for STRING, BYTES, temporal, geography, and interval values, but their BigQuery type identity is absent.
This limitation is recorded in the internal/mycli/feature/bigquery package comment and was called out as tracked debt when BIGQUERY moved behind the feature seam in #781, originating from #778.
Issue #738 added the exclusive TypedRows payload for buffered results, but that payload intentionally contains Spanner ResultSetMetadata and []*spanner.Row. It is not a provider-neutral typed-result contract. Presentation rows without a TypedRows payload are currently replayed as synthetic Spanner STRING columns, so the existing machinery cannot preserve BigQuery values by itself.
Design choices
A. JSONL semantic fidelity with documented surrogate mappings
Recommended first step.
Add a feature-local conversion for the BigQuery types that can be represented faithfully enough by the existing Spanner-oriented typed pipeline. Preserve the current table-output bytes, keep SQL export disabled, and define explicit fallback behavior for unsupported or non-equivalent BigQuery types.
This option should not preclude option B later.
Potentially faithful mappings include STRING, BYTES, INT64, FLOAT64, BOOL, TIMESTAMP, DATE, NUMERIC, JSON, ARRAY, and STRUCT. The design must explicitly decide how to handle BIGNUMERIC, TIME, DATETIME, GEOGRAPHY, INTERVAL, RANGE, and any future BigQuery-only type without presenting a misleading native type.
Acceptance coverage must include:
- null versus the literal string
"NULL";
- BOOL, INT64, and FLOAT64 values;
- JSON values;
- NUMERIC and BIGNUMERIC distinction;
- repeated values;
- nested RECORD values with field names;
- the fallback policy for unsupported types;
- empty-result headers;
- byte-level CSV and JSONL output;
- byte-identical existing table output.
B. Provider-neutral typed buffered results
Generalize TypedRows and its rendering hooks so buffered results can carry provider-specific schema and values without first translating them into Spanner rows.
This is the structural end state if multiple non-Spanner producers need typed buffered results, but BIGQUERY alone does not justify the abstraction and migration cost. The trigger for this option should be a second concrete consumer that needs provider-neutral typed results.
Constraints
This belongs near the table/output architecture tracked by the #29 issue family, but remains a standalone BIGQUERY design decision rather than a child issue.
Decision needed
Choose whether to pursue option A as a bounded first step, defer until option B has a second consumer, or retain display-only exports explicitly.
Problem
BIGQUERYpreservesbigquery.Valuerows andRowIterator.Schemawhile reading a query, but immediately converts every cell to display text throughformatBigQueryValue. The returnedResulttherefore contains onlyRows []Row.Consequences for export formats include:
NULL, booleans, integers, and floats become JSON strings in JSONL output;This limitation is recorded in the
internal/mycli/feature/bigquerypackage comment and was called out as tracked debt when BIGQUERY moved behind the feature seam in #781, originating from #778.Issue #738 added the exclusive
TypedRowspayload for buffered results, but that payload intentionally contains SpannerResultSetMetadataand[]*spanner.Row. It is not a provider-neutral typed-result contract. Presentation rows without aTypedRowspayload are currently replayed as synthetic Spanner STRING columns, so the existing machinery cannot preserve BigQuery values by itself.Design choices
A. JSONL semantic fidelity with documented surrogate mappings
Recommended first step.
Add a feature-local conversion for the BigQuery types that can be represented faithfully enough by the existing Spanner-oriented typed pipeline. Preserve the current table-output bytes, keep SQL export disabled, and define explicit fallback behavior for unsupported or non-equivalent BigQuery types.
This option should not preclude option B later.
Potentially faithful mappings include STRING, BYTES, INT64, FLOAT64, BOOL, TIMESTAMP, DATE, NUMERIC, JSON, ARRAY, and STRUCT. The design must explicitly decide how to handle BIGNUMERIC, TIME, DATETIME, GEOGRAPHY, INTERVAL, RANGE, and any future BigQuery-only type without presenting a misleading native type.
Acceptance coverage must include:
"NULL";B. Provider-neutral typed buffered results
Generalize
TypedRowsand its rendering hooks so buffered results can carry provider-specific schema and values without first translating them into Spanner rows.This is the structural end state if multiple non-Spanner producers need typed buffered results, but BIGQUERY alone does not justify the abstraction and migration cost. The trigger for this option should be a second concrete consumer that needs provider-neutral typed results.
Constraints
Resultbody-payload invariant established by Result-type decomposition: TypedRows preserving raw typed rows for buffered results #738.This belongs near the table/output architecture tracked by the #29 issue family, but remains a standalone BIGQUERY design decision rather than a child issue.
Decision needed
Choose whether to pursue option A as a bounded first step, defer until option B has a second consumer, or retain display-only exports explicitly.