Conversation
GizmoSQL is an Arrow Flight SQL server backed by DuckDB. This adds `AdapterType::GizmoSQL` / `Backend::GizmoSQL`, treated as a DuckDB-dialect adapter throughout (dialect, keywords, type rendering, metadata adapter, get_relation, constraint support) and inheriting the `dbt-duckdb` macro package through the dispatch chain (`gizmosql` -> `duckdb` -> `default`), the way lake compute does. A small `dbt-gizmosql` macro package overrides what a remote DuckDB needs: seeds use the default batched insert loader instead of `COPY ... FROM` a local file, grants warn, and the seed binding char is `%s`. Driver loading mirrors Exasol: the `adbc_driver_gizmosql` library is loaded from the system search path (entrypoint `AdbcDriverGizmosqlInit`), not the CDN. Auth builds a `grpc+tls://` (or `grpc+tcp://`) URI plus basic auth, and supports `tls_skip_verify` and `auth_type: external` (OAuth/SSO). The profile schema and key aliases match the Python dbt-gizmosql 1.x adapter so existing profiles work unchanged; `database` (the DuckDB catalog on the server) is required and is set as the session's current catalog on connect. Also registers `get_binding_char` and `convert_datetimes_to_strs` as pure adapter methods for the time-machine layer; the DuckDB non-fast seed path hit a debug assertion without them. Verified against a local GizmoSQL 1.38.4 server with driver 2.0.13: `dbt build` on jaffle-shop-classic (3 seeds, 5 models, 20 tests) plus incremental, snapshot, full refresh, docs generate, and show. The DuckDB macro tests now run for both DuckDB and GizmoSQL; multi-adapter, contract wrapper, package-chain, auth, and driver (env-gated) tests added. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017ZP2QGdMHXGr5VeCMjPgfQ
|
Thanks for your pull request, and welcome to our community! We require contributors to sign our Contributor License Agreement and we don't seem to have your signature on file. Check out this article for more information on why we have a CLA. In order for us to review and merge your code, please submit the Individual Contributor License Agreement form attached above above. If you have questions about the CLA, or if you believe you've received this message in error, please reach out through a comment on this PR. CLA has not been signed by users: @prmoore77 |
There was a problem hiding this comment.
🟡 Changes recommended
There are blocking correctness/security issues in GizmoSQL auth handling and interactive profile setup that can lead to unintended credential usage or prevent configuring documented auth modes.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds a new GizmoSQL community adapter (type: gizmosql) to dbt Core v2, treating it as a DuckDB-dialect, Arrow Flight SQL–backed system using an ADBC driver loaded from the system library path.
Changes:
- Introduces
AdapterType::GizmoSQL/Backend::GizmoSQLand wires it through dialect, relation, SQL typing, metadata, and ADBC engine integration. - Adds GizmoSQL auth/profile schema support plus interactive/headless profile setup and adapter config aliases to match the Python adapter.
- Adds a
dbt-gizmosqlinternal macro package that inheritsdbt-duckdb, plus tests ensuring macro dispatch/inheritance parity.
File summaries
| File | Description |
|---|---|
| crates/dbt-tasks-sa/src/sql/dialect.rs | Maps GizmoSQL to DuckDB sqlparser dialect. |
| crates/dbt-schemas/src/schemas/relations/base.rs | Includes GizmoSQL in relation time filtering behavior. |
| crates/dbt-schemas/src/schemas/profiles.rs | Adds DbConfig::GizmoSQL, config fields, and target context conversion. |
| crates/dbt-profile-schemas/src/lib.rs | Exposes GizmoSQL profile setup module and helper. |
| crates/dbt-profile-schemas/src/headless.rs | Enables GizmoSQL in headless profile generation. |
| crates/dbt-profile-schemas/src/gizmosql_config.rs | Implements interactive GizmoSQL profile prompts. |
| crates/dbt-metricflow/src/lib.rs | Accepts gizmosql as DuckDB dialect in metricflow parsing. |
| crates/dbt-loader/tests/macros/multi_adapter.rs | Adds multi-adapter macro dispatch test coverage for GizmoSQL. |
| crates/dbt-loader/tests/macros/duckdb.rs | Runs DuckDB macro tests for both DuckDB and GizmoSQL. |
| crates/dbt-loader/tests/macros/contract_wrappers.rs | Includes GizmoSQL in contract wrapper macro tests. |
| crates/dbt-loader/tests/macros/aggregated_tests.rs | Adds assertion that GizmoSQL renders batched unique like DuckDB. |
| crates/dbt-loader/src/load_packages.rs | Ensures GizmoSQL loads dbt-duckdb via internal package chain. |
| crates/dbt-loader/src/dbt_macro_assets/dbt-gizmosql/macros/adapters.sql | Adds GizmoSQL macro overrides (seeds/grants/binding char). |
| crates/dbt-loader/src/dbt_macro_assets/dbt-gizmosql/dbt_project.yml | Defines the dbt-gizmosql internal macro package. |
| crates/dbt-jinja/minijinja/src/dispatch_object.rs | Adds GizmoSQL → DuckDB dispatch prefix and internal package inheritance. |
| crates/dbt-init/src/profile_setup.rs | Adds GizmoSQL to interactive dbt init adapter list and setup routing. |
| crates/dbt-df-providers/src/seed_io.rs | Aligns GizmoSQL seed column-name inference with DuckDB/LakeCompute. |
| crates/dbt-common/src/adapter.rs | Maps GizmoSQL to DuckDB dialect. |
| crates/dbt-auth/src/lib.rs | Registers GizmoSQL auth implementation. |
| crates/dbt-auth/src/gizmosql/mod.rs | Implements GizmoSQL auth → ADBC database options translation + tests. |
| crates/dbt-adbc/tests/driver/mod.rs | Adds env-gated GizmoSQL driver execution tests. |
| crates/dbt-adbc/src/repl.rs | Enables gizmosql backend selection in the ADBC REPL. |
| crates/dbt-adbc/src/install.rs | Excludes GizmoSQL from CDN install path (system-only). |
| crates/dbt-adbc/src/driver.rs | Adds Backend::GizmoSQL with system-library driver name/entrypoint. |
| crates/dbt-adapter/src/time_machine/semantic.rs | Marks new adapter methods as pure for time_machine semantics. |
| crates/dbt-adapter/src/sql_types.rs | Includes GizmoSQL in type metadata/comment handling and size limits. |
| crates/dbt-adapter/src/relation/factory.rs | Enables static relation construction for GizmoSQL. |
| crates/dbt-adapter/src/metadata/get_relation.rs | Routes GizmoSQL relation lookup through DuckDB metadata path. |
| crates/dbt-adapter/src/engine/adbc.rs | Sets current catalog for GizmoSQL connections based on database. |
| crates/dbt-adapter/src/column/column_builder.rs | Constructs columns for GizmoSQL and aligns it with postgres-like path. |
| crates/dbt-adapter/src/catalog_relation.rs | Uses DuckDB defaults/attrs for GizmoSQL catalog relations. |
| crates/dbt-adapter/src/adapter/mod.rs | Adds GizmoSQL to various adapter behavior match arms. |
| crates/dbt-adapter/src/adapter/adapter_impl.rs | Reuses DuckDB metadata adapter and adds GizmoSQL feature/behavior arms. |
| crates/dbt-adapter/src/adapter/adapter_factory.rs | Maps AdapterType::GizmoSQL to Backend::GizmoSQL. |
| crates/dbt-adapter-sql/src/types/tests.rs | Treats GizmoSQL like DuckDB in type test scaffolding. |
| crates/dbt-adapter-sql/src/types/mod.rs | Uses DuckDB-like rendering/units for GizmoSQL SQL types. |
| crates/dbt-adapter-sql/src/statements.rs | Includes GizmoSQL in update-statement detection. |
| crates/dbt-adapter-sql/src/keywords.rs | Reuses DuckDB keyword sets for GizmoSQL. |
| crates/dbt-adapter-sql/src/ident.rs | Aligns GizmoSQL identifier quoting/validity rules with DuckDB-like backends. |
| crates/dbt-adapter-core/src/lib.rs | Adds AdapterType::GizmoSQL and quote-char mapping + tests. |
| crates/dbt-adapter-core/src/config_aliases.rs | Adds GizmoSQL profile key alias map matching Python adapter. |
| .changes/unreleased/Features-20260911-095735.yaml | Adds changelog entry for the new GizmoSQL adapter. |
Review details
- Files reviewed: 42/42 changed files
- Comments generated: 3
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
The GizmoSQL ADBC driver owns the `gizmosql://` scheme (TLS by default, `?transport=tcp` for plaintext) and rewrites it to the underlying Flight SQL URI itself. Building that instead of `grpc+tls://` / `grpc+tcp://` keeps the transport a driver concern, so it can change without touching dbt. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017ZP2QGdMHXGr5VeCMjPgfQ
|
Thanks for your pull request, and welcome to our community! We require contributors to sign our Contributor License Agreement and we don't seem to have your signature on file. Check out this article for more information on why we have a CLA. In order for us to review and merge your code, please submit the Individual Contributor License Agreement form attached above above. If you have questions about the CLA, or if you believe you've received this message in error, please reach out through a comment on this PR. CLA has not been signed by users: @prmoore77 |
…s range - `auth_type: external` no longer sends profile `username`/`password` as Flight SQL basic auth; the OAuth flow supplies its own credentials. If either is present it is ignored with a warning. Test added. - Interactive/headless profile setup gains an `auth_type` selector (password or external); `username`/`password` are only asked for password auth, and `external_root` is an optional field. - `threads` given as an integer is range-checked with `u16::try_from` instead of `as u16`, matching the string path's error. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017ZP2QGdMHXGr5VeCMjPgfQ
There was a problem hiding this comment.
🔵 Needs a closer look
It introduces a new adapter/backend and threads that behavior through many core crates (auth, profiles, macro dispatch, ADBC engine/driver), so it needs final human review for cross-cutting correctness and compatibility.
Review details
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
crates/dbt-adapter/src/engine/adbc.rs:608
new_connection_with_configadds GizmoSQL-specific connection setup using anif self.adapter_type == ...chain; this is the same target-specific control-flow pattern used just above for ClickHouse and is easier to extend/scan when handled via a singlematch self.adapter_typedispatch.
- Files reviewed: 42/42 changed files
- Comments generated: 0 new
- Review effort level: Lite
Copilot suggested folding the ClickHouse and GizmoSQL connection-option branches in `new_connection_with_config` into one `match self.adapter_type`, matching the adapters guidance to dispatch on adapter type rather than chain `if` checks. No behavior change. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017ZP2QGdMHXGr5VeCMjPgfQ
There was a problem hiding this comment.
🟡 Changes recommended
TargetContext construction for GizmoSQL does not currently validate required connection fields (notably host), which can allow invalid profiles to proceed until later failures.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
- Files reviewed: 42/42 changed files
- Comments generated: 2
- Review effort level: Lite
… comment - `TargetContext` construction for GizmoSQL now fails at profile load when `host` is missing or empty, with the same "field is required" error as `database`, instead of leaving `target.host` unset until the connection fails. `GizmoSQLTargetEnv.host` is a `String` accordingly. - The note about GizmoSQL not batching multi-statement SQL now sits on the splitter arm it describes, not on the DuckDB/LakeCompute batching arm. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017ZP2QGdMHXGr5VeCMjPgfQ
There was a problem hiding this comment.
🟡 Changes recommended
GizmoSQL columns are currently built via a path that hard-codes AdapterType::Postgres in ColumnBuilder, which can cause GizmoSQL columns to be tagged/parsed with Postgres rules instead of GizmoSQL/DuckDB behavior.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
- Files reviewed: 42/42 changed files
- Comments generated: 1
- Review effort level: Lite
`ColumnBuilder::build_postgres_like` hard-coded `AdapterType::Postgres` when constructing the `Column`, so GizmoSQL columns built from Arrow fields were tagged as Postgres while `build_from_parts` tagged them as GizmoSQL. The builder now takes the adapter type to tag the column with. Postgres, Salesforce, DuckDB and lake compute keep the Postgres tag they always had; only GizmoSQL passes its own type. Numeric precision/scale still follow the Postgres rules for all of them (integers get scale 0). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017ZP2QGdMHXGr5VeCMjPgfQ
There was a problem hiding this comment.
🔵 Needs a closer look
A couple of required GizmoSQL fields are not validated against empty-string values (notably username and database), which can allow invalid profiles to pass and fail later in confusing ways.
Review details
Suppressed comments (3)
Previously missed (3) — in code that hasn't changed since the last review.
crates/dbt-auth/src/gizmosql/mod.rs:156
usernameis treated as present even when it is an empty string, so a profile likeusername: ""will pass validation (unlessauth_type: external) and then attempt to authenticate with an empty username. Consider trimming/filtering empty usernames so the existing "requires 'username'" check also rejects empty values.
crates/dbt-schemas/src/schemas/profiles.rs:2293databaseis required for GizmoSQL, but an explicitly empty string (e.g.,database: "") will currently passok_or_else(missing("database"))and become the session catalog, which is likely invalid and can lead to confusing runtime behavior. Treat empty strings the same as missing for consistency with thehostcheck above.
crates/dbt-adapter/src/adapter/adapter_impl.rs:314- This match duplicates the same
DuckDBMetadataAdapter::new(engine)construction for DuckDB, LakeCompute, and GizmoSQL. These can be combined into a single pattern arm to reduce repetition and keep future edits consistent.
- Files reviewed: 42/42 changed files
- Comments generated: 0 new
- Review effort level: Lite
… metadata arms - Treat an empty `username` the same as a missing one in GizmoSQL auth, so `username: ""` fails the required-username check (unless `auth_type: external`) instead of attempting basic auth with an empty name. - Treat an empty `database` the same as a missing one when building the GizmoSQL target context, matching the existing `host` check. - Collapse the identical DuckDB / LakeCompute / GizmoSQL metadata adapter arms into one. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Addressed the three suppressed Copilot comments in 5d2134e: an empty |
There was a problem hiding this comment.
🔵 Needs a closer look
It introduces a new adapter type and threads it through many cross-cutting code paths (auth/ADBC, macro dispatch, profiles, adapter logic), which warrants final human review despite targeted tests.
Review details
- Files reviewed: 42/42 changed files
- Comments generated: 0 new
- Review effort level: Lite
Summary
Adds a GizmoSQL community adapter (
type: gizmosql). GizmoSQL is an Arrow Flight SQL server backed by DuckDB, maintained by GizmoData; the Pythondbt-gizmosqladapter has served dbt Core 1.x users and this brings it to dbt Core v2.Contributed on behalf of GizmoData, following the community adapter guide and the shape of the Exasol contribution.
Design
AdapterType::GizmoSQL/Backend::GizmoSQLare handled alongside DuckDB everywhere the code matches on adapter type: dialect and keywords, type rendering,DuckDBMetadataAdapter,duckdb_get_relation, column builder, constraint support, catalog relation defaults.dbt-duckdbmacro package through the dispatch chain (gizmosql→duckdb→default), exactly aslakecomputedoes. A smalldbt-gizmosqlpackage overrides only what a remote DuckDB needs: seeds use the default batchedinsert ... valuesloader (the server cannot read the client's seed file), grants warn, andget_binding_charreturns%s.adbc_driver_gizmosql(entrypointAdbcDriverGizmosqlInit), not the CDN, like Exasol. The driver is the gizmosql-adbc Go driver (a thin layer over the upstream Flight SQL ADBC driver); shared libraries for linux/macos/windows × amd64/arm64 are published on its releases page.dbt-authbuilds agizmosql://host:portURI (the driver's own scheme: TLS by default,?transport=tcpwhenuse_encryption: false) plus basic auth, withtls_skip_verifyandauth_type: external(OAuth/SSO browser flow) options. Profile keys and aliases match the Python adapter's credentials so existingprofiles.ymltargets work unchanged.database(the DuckDB catalog on the server) is required, since it cannot be derived offline; it is set as the session's current catalog on connect.NON_EXPERIMENTAL_ADAPTERS; runs withDBT_ALLOW_EXPERIMENTAL_ADAPTERS=true.get_binding_charandconvert_datetimes_to_strsas pure adapter methods intime_machine::semantic; the DuckDB non-fast seed path hit the debug assertion without them.Profile example
Testing
cargo clippy --workspace --all-targets -- -D warnings,cargo fmt --all --check, andcargo nextest runfor every touched crate pass.dbt-loaderDuckDB macro tests now run for bothDuckDBandGizmoSQL; added tests for the auth module, the package inheritance chain, multi-adapterrun_hooksresolution, and contract wrappers. Thedbt-adbcdriver test is env-gated onADBC_GIZMOSQL_URI.dbt buildon jaffle-shop-classic (3 seeds, 5 models, 20 tests) plus an incremental model (two runs), acheck-strategy snapshot (two runs),--full-refresh,docs generate, andshow, all clean.CI
For dbt Labs CI, a
gizmodata/gizmosqlDocker image is available (the server is a single binary; thegizmosqlPyPI package can also start one as a subprocess). Happy to provide a hosted instance instead if that is easier — please ping @prmoore77.Not in scope for this first cut
Python models and the DuckDB
externalmaterialization.🤖 Generated with Claude Code
https://claude.ai/code/session_017ZP2QGdMHXGr5VeCMjPgfQ