Conversation
Add the `athena` variant of `DbConfig` so Athena profiles parse and reach
the experimental-adapter gate and `dbt-auth`, which already carry the
Athena pieces (`AdapterType::Athena`, `Backend::Athena`, `AthenaAuth`).
`AthenaDbConfig` declares every field of dbt-athena's `AthenaCredentials`,
including the ones `AthenaAuth` rejects today: the auth layer reads the
profile through `to_mapping()`, so an undeclared field would be dropped
silently instead of raising the "not yet supported" error. `catalog` is
accepted as an alias of `database`, matching dbt-athena.
`connection_keys_for(Athena)` lists the honoured, non-secret keys.
`TargetContext::Athena` exposes `region_name`, `s3_staging_dir`,
`work_group`, `aws_profile_name` and the common fields, with `database`
defaulting to `awsdatacatalog` as in `AthenaAuth`; threads parsing goes
through `threads_from_profile`.
The relation factory builds the generic `RelationStatic` for Athena, as
for every other adapter, so `dbt parse` and `dbt ls` stop at a
configuration error naming the missing `dbt-athena` macro package instead
of a `todo!` panic.
Athena stays outside `NON_EXPERIMENTAL_ADAPTERS`. The remaining
`todo!("Athena")` arms in `dbt-adapter` and the macro package are left
for follow-ups.
Resolves dbt-labs#16252
fpiped
force-pushed
the
feat/athena-profile-schema
branch
from
September 10, 2026 08:14
75ee2dc to
391a3c8
Compare
|
Thanks for your pull request, and welcome to our community! We require contributors to sign our Contributor License Agreement and we don't seem to have your signature on file. Check out this article for more information on why we have a CLA. In order for us to review and merge your code, please submit the Individual Contributor License Agreement form attached above above. If you have questions about the CLA, or if you believe you've received this message in error, please reach out through a comment on this PR. CLA has not been signed by users: @fpiped |
Author
|
@cla-bot check |
|
The cla-bot has been summoned, and re-checked this pull request! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves #16252
Related: #13822
Problem
type: athenainprofiles.ymlfails at parse time (InvalidConfig (dbt1005): Failed to parse profiles.yml: unknown variant 'athena'), before the experimental-adapter gate and beforedbt-authare reached. The Athena pieces already in the repo (AdapterType::Athena,Backend::Athena, the ADBC option keys incrates/dbt-adbc/src/athena.rs,AthenaAuthincrates/dbt-auth/src/athena/mod.rs) are therefore unreachable from a profile:DbConfighas noAthenavariant (crates/dbt-schemas/src/schemas/profiles.rs,// Athena,).Solution
The profile-schema slice, same shape as ClickHouse "Part 2" (
b468efcd6), plus one arm in the relation factory:AthenaDbConfig, with the field set of dbt-athena'sAthenaCredentials(connections.py) and thecatalogalias fordatabase. Every dbt-athena field is declared, including the onesAthenaAuthcurrently rejects (assume_role_*,s3_data_dir,poll_interval, ...): the auth layer reads the profile throughto_mapping(), so a field absent from the struct would be dropped silently instead of producing the existing "not yet supported" error. A test pins this.DbConfig::Athenawired throughadapter_type,get_unique_field(s3_staging_dir, as in dbt-athena),get_database/get_database_or_default(defaults toawsdatacatalog, the defaultAthenaAuthapplies),get_schema,get_threads/set_threads,to_yaml_value.connection_keys_for(AdapterType::Athena): dbt-athena's_connection_keysrestricted to whatAthenaAuthhonours today;aws_secret_access_keyandaws_session_tokenstay out, sodbt debugnever prints them (covered by the existingconnection_keys_never_expose_a_credential).TargetContext::Athena/AthenaTargetEnv(region_name,s3_staging_dir,work_group,aws_profile_nameplus the commondatabase/schema/type/threads). Threads parsing goes through a smallthreads_from_profilehelper instead of a fourth inline copy of the samematch.relation/factory.rs: Athena joins the genericRelationStaticarm, sodbt parse/dbt lsstop at a configuration error instead oftodo!("Athena").dbt-test-utils::profiles: schema / database overrides for Athena, as for the other adapters.Athena is not added to
NON_EXPERIMENTAL_ADAPTERS, so it stays behindDBT_ALLOW_EXPERIMENTAL_ADAPTERS. The remainingtodo!("Athena")arms indbt-adapter(adapter_impl.rsmetadata adapter,column_builder.rs,sql_types.rs) and thedbt-athenamacro package underdbt_macro_assetsare the next slices and are not touched here.Behaviour with this branch, debug build, against the profile below:
dbt debug,DBT_ALLOW_EXPERIMENTAL_ADAPTERS=falseThe 'athena' adapter is not yet supported by dbt. Supported adapters: snowflake, ...(gate reached; before:unknown variant 'athena')dbt debug, gate openAuthenticationFailed: Driver not found: adbc_driver_athena(profile →AthenaAuth→ driver load)dbt debug,catalog: my_catalogdbt debug,assume_role_arn: ...Athena profile field 'assume_role_arn' is not yet supported by the dbt Athena backend(theAthenaAuthrejection now fires)dbt debug, noschemaAthena requires 'schema' in profile configurationdbt parse/dbt ls, gate openMissing dbt_project.yml in embedded package 'dbt-athena'(before this branch: panic atrelation/factory.rs:28)Not in scope, and a dbt Labs step: the Athena ADBC driver is not in
INSTALLABLE_DRIVERS, so loading falls back toLoadStrategy::System(a locally builtadbc_driver_athenafromdbt-labs/athena).Checks run locally:
cargo fmt --check,cargo nextest run -p dbt-schemas -E 'test(/profiles::tests/)'(36 passed),cargo check -p dbt-test-utils -p dbt-profile-schemas -p dbt-init -p dbt-loader -p dbt-compilation,cargo clippy --all-targets --all-featuresondbt-schemas,dbt-test-utilsanddbt-adapter,cargo build -p dbt-sa-clifor the table above.Checklist
type: athenaprofile type, gated as experimental; flagging for Product/DX review.