Skip to content

6.7 - Share SEP-10 challenge single-use claim across instances via Redis - #175

Open
Otfrugger wants to merge 1 commit into
determined-001:mainfrom
Otfrugger:fix/sep10-challenge-shared-store
Open

6.7 - Share SEP-10 challenge single-use claim across instances via Redis#175
Otfrugger wants to merge 1 commit into
determined-001:mainfrom
Otfrugger:fix/sep10-challenge-shared-store

Conversation

@Otfrugger

Copy link
Copy Markdown
Contributor

closes #160

Summary

apps/api/src/services/challenge.ts tracked redeemed SEP-10 challenges in an in-process Map. With more than one API instance and no shared store, "single-use" only held per process: the same signed challenge could be redeemed once on each instance inside its ~15-minute validity window, minting a session per instance from one signature.

1. New port

packages/core/src/ports/index.ts gains UsedChallengeStore:

claim(hash: string, expiresAt: number): Promise<boolean>

Atomically marks a challenge hash as used; returns false if another caller already claimed it. The port's own doc comment explains the exploit it closes.

2. Two implementations

  • MemoryUsedChallengeStore (still in challenge.ts) — the default. Same behavior as before the change (check + set inside one synchronous call, no await in between, so no interleaving within one process), just moved behind the port instead of being private state on ChallengeService.
  • RedisUsedChallengeStore (new: apps/api/src/services/redis-used-challenge-store.ts) — SET key value NX PX <ttl>, so two instances racing the same nonce cannot both win. TTL mirrors the challenge's own validity window (expiresAt), so a claim never outlives the challenge it guards.

3. Selection

container.ts branches on env.redisUrl when constructing ChallengeService, the same REDIS_URL check the rate limiter already uses in index.ts.

4. The atomicity fix itself

ChallengeService.verify() previously did if (this.used.has(hash)) throw; ...; this.used.set(hash, ...) — a check-then-set, which is exactly the race this issue is about once the store is shared: two instances could both pass the has() check, both independently verify the (same, valid) signature, and both then set(). The fix folds this into a single claim() call made once, after signature verification succeeds, so there's no window between checking and claiming. The old early has() pre-check (before doing the Horizon signer fetch) is gone — it was a non-atomic fast-path optimization that would have been misleading once real logic lived in a shared store; correctness now depends only on the one atomic claim() call.

Test

Added to apps/api/test/challenge.test.ts: two ChallengeService instances (same server keypair, different ChallengeService objects) constructed with one shared MemoryUsedChallengeStore — models a two-instance deployment. The same signed challenge succeeds on the first verify() and is rejected with /already been used/ on the second.

I used MemoryUsedChallengeStore rather than a live Redis connection for this, since no Redis server is available in this environment (and I found no existing test in this repo that spins one up either — the rate limiter's own RedisStore has no test coverage today). MemoryUsedChallengeStore.claim() implements the exact same UsedChallengeStore contract RedisUsedChallengeStore does, so the test exercises the real race-closing logic in ChallengeService, just not the Redis wire protocol itself. If you'd like, I can also add a RedisUsedChallengeStore-specific test gated behind a live Redis (e.g. via redis-memory-server or a CI service container) as a follow-up.

Also updated

  • docs/MAINNET.md — the existing REDIS_URL caveat now also mentions SEP-10 challenge sharing.

Verification

  • Read through challenge.ts, container.ts, rate-limit.ts/redis-store.ts (for the existing REDIS_URL pattern to mirror) and the redis package's SET ... NX PX API before writing this.
  • Byte-scanned every changed/new file for stray control bytes before committing.
  • Could not run: no pnpm/node_modules install is available in this environment, so I could not run pnpm typecheck or pnpm test. Please run those before merging, along with the new challenge.test.ts case specifically.

Done-when checklist

  • With REDIS_URL set, a challenge redeemed on one instance is rejected on another — covered by the port contract + the new test against MemoryUsedChallengeStore; not yet verified against a live Redis (see caveat above).
  • With REDIS_URL unset, behavior is exactly as today — MemoryUsedChallengeStore is a straight extraction of the previous Map logic with the same single-process, no-interleaving guarantee.

…ia Redis

apps/api/src/services/challenge.ts tracked redeemed SEP-10 challenges in an
in-process Map. With more than one API instance and no shared store,
single-use only held per process: the same signed challenge could be
redeemed once on each instance inside its ~15-minute validity window,
minting a session per instance from one signature.

Added packages/core/src/ports/index.ts's UsedChallengeStore port (claim()
atomically marks a challenge hash used, returning false if another caller
already claimed it). ChallengeService now delegates to it instead of owning
a Map directly:
- MemoryUsedChallengeStore (challenge.ts) is the default, same behavior as
  before, just moved behind the port.
- RedisUsedChallengeStore (redis-used-challenge-store.ts) uses SET NX PX so
  two instances racing the same nonce cannot both win, with a TTL matching
  the challenge's own validity window.

container.ts selects between them via REDIS_URL, the same branch the rate
limiter already uses. verify() now calls claim() once, after signature
verification succeeds, rather than a separate has()-then-set() — the
earlier code's own check-then-set was exactly the race this issue is about,
so folding it into one atomic call was necessary, not just a port seam.

Added a test modeling two ChallengeService instances sharing one store: the
same signed challenge succeeds on the first and is rejected on the second.
@drips-wave

drips-wave Bot commented Aug 28, 2026

Copy link
Copy Markdown

@Otfrugger Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@vercel

vercel Bot commented Aug 28, 2026

Copy link
Copy Markdown

@Otfrugger is attempting to deploy a commit to the determined's projects Team on Vercel.

A member of the Team first needs to authorize it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6.7 - SEP-10 challenge nonces are single-use per process, not globally

1 participant