Skip to content

Add rate limiting and cost guardrails for Bedrock AI calls #7

Description

@llinsss

Problem

/api/v1/hint and /api/v1/story both default use_bedrock=True and have zero rate limiting. A misbehaving or retry-happy frontend (or a malicious caller, given auth isn't in place yet) can trigger unbounded Bedrock invocations with no cap, risking a surprise AWS bill.

Scope

  • Add per-student and/or per-API-key rate limiting on Bedrock-backed endpoints (e.g. token bucket, N calls per minute).
  • Add a global daily/monthly Bedrock call budget with a hard cutover to template-only fallback once exceeded (log when this happens).
  • Expose current usage via an admin endpoint (e.g. GET /api/v1/admin/bedrock-usage).
  • Make limits configurable via environment variables, with sane documented defaults.

Acceptance criteria

  • Exceeding the per-student rate limit returns a 429 (or falls back to template mode, your call — document the choice).
  • Exceeding the global budget forces template-only behavior for all requests until the window resets (test simulates budget exhaustion).
  • Usage counters are observable via the admin endpoint.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

GrantFox OSSIssue tracked in GrantFox OSSMaybe RewardedIssue may be eligible for a GrantFox rewardThird CampaignCampaign: Third CampaignapiAPI surface changescost-controlCost/rate limiting guardrails

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions