-
Notifications
You must be signed in to change notification settings - Fork 26
Add AWS benchmarking authentication documentation #1154
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
liyu-ma
wants to merge
2
commits into
main
Choose a base branch
from
maliyu/aws-benchmarking-auth-doc
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| # AWS Benchmarking Authentication | ||
|
|
||
| This document describes how Telescope pipelines authenticate to AWS for benchmark runs, covering the current OIDC-based approach and the infrastructure setup required. | ||
|
|
||
| ## Overview | ||
|
|
||
| Telescope uses **OIDC (OpenID Connect) federation** to authenticate Azure DevOps (ADO) pipelines to AWS. Instead of storing long-lived static IAM access keys, the pipeline obtains short-lived temporary credentials at runtime through the AWS Toolkit for Azure DevOps extension. | ||
|
|
||
| ### Authentication Flow | ||
|
|
||
| ``` | ||
| ADO Pipeline | ||
| └─ AWSShellScript@1 task (AWS Toolkit extension) | ||
| ├─ ADO issues an OIDC token via the service connection | ||
| ├─ AWS Toolkit exchanges the token with AWS STS (AssumeRoleWithWebIdentity) | ||
| └─ Injects AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN | ||
| └─ Pipeline exports these as pipeline variables | ||
| └─ Shared "AWS Login" step configures the AWS CLI | ||
| ``` | ||
|
|
||
| ### Key Files | ||
|
|
||
| | File | Purpose | | ||
| |------|---------| | ||
| | `steps/cloud/aws/login.yml` | Central AWS auth template used by all AWS pipelines | | ||
| | `jobs/competitive-test.yml` | Job template that passes `credential_type` to login | | ||
| | `steps/setup-tests.yml` | Passes `credential_type` from job to cloud login | | ||
|
|
||
| ## Credential Types | ||
|
|
||
| The `credential_type` parameter controls how AWS credentials are obtained. It flows from the pipeline → `jobs/competitive-test.yml` → `steps/setup-tests.yml` → `steps/cloud/aws/login.yml`. | ||
|
|
||
| ### `service_connection` (required for OIDC) | ||
|
|
||
| Uses the **AWS Toolkit for Azure DevOps** extension (`AWSShellScript@1` task) with an ADO service connection. This is required for OIDC authentication. | ||
|
|
||
| - The service connection is referenced via the `$(AWS_SERVICE_CONNECTION)` pipeline variable | ||
| - The extension automatically handles the OIDC token exchange and returns temporary credentials (access key, secret key, session token) | ||
|
|
||
| ```yaml | ||
| # Pipeline usage | ||
| credential_type: service_connection | ||
| ``` | ||
|
|
||
| > **Note:** The `variable_group` credential type stores static IAM access keys in an ADO variable group, so it does not support OIDC. | ||
|
|
||
| ## Pipeline Configuration | ||
|
|
||
| The ADO service connection `AWS-for-Telescope-OIDC` has been configured with a role in the AWS benchmarking account. To use OIDC authentication, pipelines need to set the following variables: | ||
|
|
||
| ### `AWS_SERVICE_CONNECTION` | ||
|
|
||
| Set this pipeline variable to `AWS-for-Telescope-OIDC`: | ||
|
|
||
| ```yaml | ||
| variables: | ||
| AWS_SERVICE_CONNECTION: AWS-for-Telescope-OIDC | ||
| ``` | ||
|
|
||
| ### `aws.rolecredential.maxduration` | ||
|
|
||
| Controls the credential lifetime in seconds. The AWS IAM role is configured with a maximum session duration of `43200` (12 hours). Pipelines must set `aws.rolecredential.maxduration` to a value less than or equal to `43200` that covers the full duration of the benchmark run: | ||
|
|
||
| ```yaml | ||
| variables: | ||
| aws.rolecredential.maxduration: 43200 | ||
| ``` | ||
|
liyu-ma marked this conversation as resolved.
|
||
|
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.