feat(aws-kinesis): cache assumed-role credentials to avoid STS throttling - #3909
Draft
mdkhan-tw wants to merge 1 commit into
Draft
feat(aws-kinesis): cache assumed-role credentials to avoid STS throttling#3909mdkhan-tw wants to merge 1 commit into
mdkhan-tw wants to merge 1 commit into
Conversation
…ling Under high TPS, aws-kinesis called STS AssumeRole on every request (twice: intermediary role + target role), causing IAM AssumeRole throttling errors. Add an in-memory TTL cache in the shared AWS sts lib (used only by aws-kinesis), keyed by role ARN + external id + region: - TTL derived from the actual STS credential Expiration minus a 5-minute safety buffer, falling back to 55 minutes when STS omits an expiration. - In-flight refresh de-duplication so a burst of concurrent cache misses collapses into a single STS refresh (avoids thundering herd). This reduces STS calls from two per request to at most two per role per TTL window. assumeRole's public signature is unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Adds an in-memory, TTL-based cache for STS assumed-role credentials in the aws-kinesis STS helper to reduce AssumeRole call volume and prevent throttling under high throughput, with tests covering cache reuse and concurrency behavior.
Changes:
- Implemented assumed-role credential caching keyed by role ARN + external ID + region, with TTL derived from STS expiration and an expiry safety buffer.
- Added in-flight refresh de-duplication to collapse concurrent cache misses into a single refresh.
- Extended Jest tests to reset module-level cache between cases and verify reuse/expiry/concurrency behaviors.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| packages/destination-actions/src/lib/AWS/sts.ts | Introduces TTL cache + in-flight refresh de-duplication for assumeRole, and returns expiration metadata from STS calls. |
| packages/destination-actions/src/lib/AWS/test/index.test.ts | Adds cache-reset hook and new test cases validating caching and concurrency semantics. |
Comment on lines
+59
to
+63
| // In-memory cache of assumed-role credentials, keyed by role ARN + external id + region. | ||
| // Under high TPS, calling STS AssumeRole on every request causes IAM throttling. Caching the | ||
| // assumed credentials until shortly before they expire keeps STS calls to (at most) one refresh | ||
| // per role per TTL window instead of one (well, two - intermediary + target) per request. | ||
| const assumedRoleCache = new Map<string, AssumedRoleCacheEntry>() |
Comment on lines
+76
to
+77
| const buildAssumedRoleCacheKey = (roleArn: string, externalId: string, region: string): string => | ||
| `${roleArn}|${externalId}|${region}` |
Comment on lines
+79
to
+83
| // Exposed for tests to reset the in-memory caches between cases. | ||
| export const __clearAssumedRoleCacheForTests = (): void => { | ||
| assumedRoleCache.clear() | ||
| inflightRoleRefreshes.clear() | ||
| } |
Comment on lines
+204
to
+211
| const ttl = target.expiration | ||
| ? target.expiration.getTime() - Date.now() - CREDENTIALS_EXPIRY_BUFFER_MS | ||
| : DEFAULT_CREDENTIALS_TTL_MS | ||
|
|
||
| return { | ||
| credentials: target.credentials, | ||
| expiresAt: Date.now() + Math.max(ttl, 0) | ||
| } |
| expect(second).toEqual(first) | ||
| }) | ||
|
|
||
| it('does not share cache entries across different roles/regions', async () => { |
mdkhan-tw
marked this pull request as draft
July 29, 2026 11:13
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Under high TPS, the
actions-aws-kinesisdestination calls STSAssumeRoleon every request — and eachassumeRole()makes two STS calls (intermediary role → target role). This causes IAMAssumeRolethrottling errors.Fix
Adds an in-memory TTL cache in the shared AWS
stslib (lib/AWS/sts.ts, which is used only by aws-kinesis — the s3 destinations have their own copies), keyed byroleArn | externalId | region:Expirationminus a 5-minute safety buffer, so credentials are never handed out about to expire mid-use. Falls back to 55 minutes if STS omits an expiration.Net effect: STS calls drop from two per request to at most two per role per TTL window.
assumeRole's public signature is unchanged, sosend()andtestAuthentication()need no changes.Testing
Extended
src/lib/AWS/__test__/index.test.ts:__clearAssumedRoleCacheForTests) inbeforeEachsince the cache is module-level.Local run (jest 27, via package-local binary):
(7 sts lib tests + 17 aws-kinesis destination tests)
🤖 Generated with Claude Code