From bc7f5e28c9139203acce47500a9a66c6beb1f88c Mon Sep 17 00:00:00 2001 From: Tobias Mayer Date: Thu, 5 Feb 2026 11:18:09 +0100 Subject: [PATCH 1/2] Add web identity authentication documentation for AWS operators Document the new `web_identity` option for OIDC-based authentication using the AWS AssumeRoleWithWebIdentity API. This enables cross-cloud authentication with tokens from Azure, Google Cloud, or custom OIDC identity providers. Co-Authored-By: Claude Opus 4.5 --- .../docs/integrations/amazon/index.mdx | 32 +++++++++++++ src/partials/operators/AWSIAMOptions.mdx | 48 +++++++++++++++++++ 2 files changed, 80 insertions(+) diff --git a/src/content/docs/integrations/amazon/index.mdx b/src/content/docs/integrations/amazon/index.mdx index fc6ba01f0..628f1dcf2 100644 --- a/src/content/docs/integrations/amazon/index.mdx +++ b/src/content/docs/integrations/amazon/index.mdx @@ -31,6 +31,38 @@ load_s3 "s3://my-bucket/data.json", aws_iam={ } ``` +### Web identity authentication + +For cross-cloud authentication, you can use OIDC tokens from external identity +providers like Azure, Google Cloud, or custom OIDC endpoints. This approach +uses the AWS `AssumeRoleWithWebIdentity` API to exchange an OIDC token for +temporary AWS credentials. + +```tql +load_s3 "s3://my-bucket/data.json", aws_iam={ + region: "us-east-1", + assume_role: "arn:aws:iam::123456789012:role/CrossCloudRole", + web_identity: { + token_file: "/var/run/secrets/tokens/aws-token" + } +} +``` + +The `web_identity` option supports three token sources: + +- **`token_file`**: Path to a file containing the JWT token. This is standard + for Kubernetes workload identity where the platform mounts a projected service + account token. + +- **`token_endpoint`**: HTTP endpoint that returns a token. Use this for Azure + IMDS or similar metadata services. + +- **`token`**: Direct token value for testing or when tokens come from another + source. + +Credentials automatically refresh before expiration, making this suitable for +long-running pipelines. + ### Default credential chain If no `aws_iam` parameter is specified, operators use AWS's [default credentials diff --git a/src/partials/operators/AWSIAMOptions.mdx b/src/partials/operators/AWSIAMOptions.mdx index 1ee0cc001..e8a75009b 100644 --- a/src/partials/operators/AWSIAMOptions.mdx +++ b/src/partials/operators/AWSIAMOptions.mdx @@ -12,6 +12,7 @@ the operator uses the default AWS credential chain. assume_role: string, // ARN of IAM role to assume. session_name: string, // session name for role assumption. external_id: string, // external ID for role assumption. + web_identity: record, // OIDC web identity token configuration. } ``` @@ -22,3 +23,50 @@ neither is specified, the operator uses the default AWS credential chain: 2. Shared credentials file (`~/.aws/credentials`) 3. IAM role for Amazon EC2 or ECS task role 4. Instance metadata service + +#### Web identity authentication + +The `web_identity` option enables OIDC-based authentication using the AWS +`AssumeRoleWithWebIdentity` API. This lets you authenticate with AWS resources +using OpenID Connect tokens from external identity providers like Azure, Google +Cloud, or custom OIDC endpoints. + +When `web_identity` is specified, you must also provide `assume_role` with the +ARN of the IAM role configured to trust your identity provider. + +The `web_identity` record accepts the following fields: + +```tql +{ + token_file: string, // path to file containing the JWT token. + token_endpoint: string, // HTTP endpoint URL to fetch tokens from. + token: string, // direct token value. + headers: record, // HTTP headers for token endpoint requests. + token_path: string, // JSON path to extract token from response. +} +``` + +Exactly one of `token_file`, `token_endpoint`, or `token` must be specified: + +- `token_file`: Path to a file containing the JWT token. This is the standard + approach for Kubernetes workload identity (EKS, AKS, GKE) where the platform + mounts a token file into the pod. + +- `token_endpoint`: HTTP endpoint URL that returns a token. Use this for Azure + IMDS (`http://169.254.169.254/metadata/identity/oauth2/token?...`) or similar + metadata services. + +- `token`: Direct token value as a string. Useful for testing or when the token + is available from another source. + +The following fields are only valid when using `token_endpoint`: + +- `headers`: HTTP headers to include in the token request. For Azure IMDS, you + typically need `{Metadata: "true"}`. + +- `token_path`: JSON path to extract the token from the endpoint response. + Defaults to `.access_token`. Set to `null` for endpoints that return the + token as plain text. + +Credentials are automatically refreshed before expiration, with exponential +backoff retry logic for transient failures. From 136ca7436a260a45a3c19b06bd18b6e1a26ba0d2 Mon Sep 17 00:00:00 2001 From: Tobias Mayer Date: Fri, 6 Feb 2026 07:20:51 +0100 Subject: [PATCH 2/2] Update docs for nested token_endpoint structure Reflect API change where token_endpoint is now a record with url, headers, and path fields instead of a flat structure. Co-Authored-By: Claude Opus 4.5 --- .../docs/integrations/amazon/index.mdx | 9 ++--- src/partials/operators/AWSIAMOptions.mdx | 36 ++++++++++--------- 2 files changed, 24 insertions(+), 21 deletions(-) diff --git a/src/content/docs/integrations/amazon/index.mdx b/src/content/docs/integrations/amazon/index.mdx index 628f1dcf2..5dd037c84 100644 --- a/src/content/docs/integrations/amazon/index.mdx +++ b/src/content/docs/integrations/amazon/index.mdx @@ -51,11 +51,12 @@ load_s3 "s3://my-bucket/data.json", aws_iam={ The `web_identity` option supports three token sources: - **`token_file`**: Path to a file containing the JWT token. This is standard - for Kubernetes workload identity where the platform mounts a projected service - account token. + for Kubernetes workload identity where the platform mounts a service account + token into the pod. -- **`token_endpoint`**: HTTP endpoint that returns a token. Use this for Azure - IMDS or similar metadata services. +- **`token_endpoint`**: Configuration for fetching tokens from an HTTP endpoint. + Contains `url`, optional `headers`, and optional `path` for JSON extraction. + Use this for Azure IMDS or similar metadata services. - **`token`**: Direct token value for testing or when tokens come from another source. diff --git a/src/partials/operators/AWSIAMOptions.mdx b/src/partials/operators/AWSIAMOptions.mdx index e8a75009b..80ea606c3 100644 --- a/src/partials/operators/AWSIAMOptions.mdx +++ b/src/partials/operators/AWSIAMOptions.mdx @@ -38,11 +38,13 @@ The `web_identity` record accepts the following fields: ```tql { - token_file: string, // path to file containing the JWT token. - token_endpoint: string, // HTTP endpoint URL to fetch tokens from. - token: string, // direct token value. - headers: record, // HTTP headers for token endpoint requests. - token_path: string, // JSON path to extract token from response. + token_file: string, // path to file containing the JWT token. + token_endpoint: { // HTTP endpoint configuration. + url: string, // endpoint URL to fetch tokens from. + headers: record, // HTTP headers for the request. + path: string, // JSON path to extract token from response. + }, + token: string, // direct token value. } ``` @@ -52,21 +54,21 @@ Exactly one of `token_file`, `token_endpoint`, or `token` must be specified: approach for Kubernetes workload identity (EKS, AKS, GKE) where the platform mounts a token file into the pod. -- `token_endpoint`: HTTP endpoint URL that returns a token. Use this for Azure - IMDS (`http://169.254.169.254/metadata/identity/oauth2/token?...`) or similar - metadata services. +- `token_endpoint`: Configuration for fetching tokens from an HTTP endpoint. + Use this for Azure IMDS or similar metadata services. The nested record + contains: + - `url` (required): The HTTP endpoint URL that returns a token, such as + `http://169.254.169.254/metadata/identity/oauth2/token?...` for Azure IMDS. -- `token`: Direct token value as a string. Useful for testing or when the token - is available from another source. - -The following fields are only valid when using `token_endpoint`: + - `headers`: HTTP headers to include in the token request. For Azure IMDS, + you typically need `{Metadata: "true"}`. -- `headers`: HTTP headers to include in the token request. For Azure IMDS, you - typically need `{Metadata: "true"}`. + - `path`: JSON path to extract the token from the endpoint response. Defaults + to `.access_token`. Set to `null` for endpoints that return the token as + plain text. -- `token_path`: JSON path to extract the token from the endpoint response. - Defaults to `.access_token`. Set to `null` for endpoints that return the - token as plain text. +- `token`: Direct token value as a string. Useful for testing or when the token + is available from another source. Credentials are automatically refreshed before expiration, with exponential backoff retry logic for transient failures.