Skip to content

Add audio pronunciation (TTS) for words and hints #4

Description

@llinsss

Problem

The entire hint and story system (agent/hint_generator.py, agent/story_mode.py) is text-only. The target users are children learning to read — many can't decode the hint text itself, which undermines the product's core purpose.

Scope

  • Integrate a TTS provider (e.g. Amazon Polly, since the project already uses AWS/boto3 for Bedrock) to generate audio for: the target word, each hint level, and generated stories.
  • Add an endpoint (or extend existing ones) to return an audio URL/stream alongside text, e.g. POST /api/v1/hint response gains an audio_url field.
  • Cache generated audio (don't regenerate on every request for the same word/hint) — a local file cache keyed by content hash is sufficient.
  • Fall back gracefully (text-only) if the TTS provider is unavailable or unconfigured, mirroring the existing Bedrock fallback pattern.

Acceptance criteria

  • /api/v1/hint and /api/v1/story responses include an audio reference when TTS is configured.
  • Requesting the same word/hint twice reuses cached audio (verified by mocking the TTS client and asserting it's called once).
  • With TTS unconfigured/erroring, endpoints still return valid text-only responses (test mirrors the existing test_bedrock_story_fallback_on_error pattern).

Metadata

Metadata

Assignees

No one assigned

    Labels

    GrantFox OSSIssue tracked in GrantFox OSSMaybe RewardedIssue may be eligible for a GrantFox rewardOfficial Campaign | FWC26Campaign: Official Campaign | FWC26accessibilityAccessibility improvementsenhancementNew feature or requestttsText-to-speech/audio

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions