Skip to content

Add AWS Bedrock provider support#169

Open
Amama-Fatima wants to merge 59 commits into
NitroRCr:masterfrom
Amama-Fatima:aws-bedrock
Open

Add AWS Bedrock provider support#169
Amama-Fatima wants to merge 59 commits into
NitroRCr:masterfrom
Amama-Fatima:aws-bedrock

Conversation

@Amama-Fatima

Copy link
Copy Markdown

Summary

This PR adds AWS Bedrock as an AI provider in AIaW.

It introduces an AI SDK-compatible aws-bedrock provider that calls Bedrock models through the AWS SDK, converts AIaW/AI SDK prompts into Bedrock-specific request formats, normalizes Bedrock responses back into AI SDK response parts, and streams responses into the format expected by the existing chat UI.

Demo video: AWS Bedrock support demo

Branch Notes

All AWS Bedrock model integration work is contained in this AWS Bedrock feature branch.

I have not integrated these changes into my fork’s main branch. My fork’s main branch has been kept up to date with the original repository, and I have regularly merged the latest main branch changes into this AWS Bedrock branch to keep the feature branch current with upstream.

What Changed

  • Added a new AWS Bedrock provider under src/utils/aws-bedrock.
  • Added AWS credential settings for:
    • AWS Region
    • Access Key ID
    • Secret Access Key
  • Added dynamic Bedrock model discovery using:
    • ListFoundationModelsCommand
    • ListInferenceProfilesCommand
  • Filters Bedrock model lists to active, text-capable, supported chat models.
  • Supports Bedrock inference profile IDs such as us.*, eu.*, apac.*, and global.*.
  • Added request, response, and stream adapters for multiple Bedrock model families:
    • Anthropic Claude
    • Meta Llama
    • Mistral
    • Amazon Nova
    • AI21 Jamba
  • Added support for both non-streaming and streaming generation paths.
  • Added Bedrock Converse API usage where needed for tool-calling capable models.
  • Added model-family-specific tool-call handling.
  • Added Bedrock error handling so AWS service errors surface clearer messages in the app.

Tool Calling Support

This PR includes tool/plugin calling support for supported Bedrock model families.

The implementation handles provider-specific differences, including:

  • Claude tool schema conversion and streamed tool-use events.
  • Nova Converse API tool calls and tool results.
  • Mistral and Jamba Converse API tool calls.
  • Llama prompt-based tool-call instructions and response parsing.
  • Mapping sanitized Bedrock tool names back to AIaW plugin tool names.
  • Preserving tool-call history in chat context so follow-up turns continue correctly after tool execution.

Tool-Calling Limitations

Some Bedrock models require a different Bedrock runtime/tool-use path than the one implemented in this PR. For those models, normal chat can still be used with tools/plugins disabled, but tool calling is intentionally blocked for now with a clear user-facing message.

This mainly affects some newer or nonstandard Mistral models such as Devstral, Ministral, Voxtral, and some Small/Instruct variants. Meta Llama tool usage is currently prompt-based rather than native Bedrock toolConfig, so behavior may vary by model.

Support for these model-specific tool-calling paths can be added in a future follow-up after their Bedrock runtime behavior is implemented and tested.

Notes

Some Bedrock models are intentionally filtered out because they are legacy, unavailable, non-chat, or unreliable for this integration.

Cohere-specific files are present from earlier work, but Cohere Bedrock models are not currently exposed in the supported model list or active routing. Cohere support can be added in the future.

Verification

AWS Bedrock documentation was referenced throughout the integration to account for model-family-specific request formats, streaming behavior, Converse API usage, inference profiles, and tool-calling differences.

I tested the supported Bedrock models and their tool-calling behavior using my own external MCP servers, as shown in the demo video. This helped validate real plugin/tool execution flows beyond simple text generation.

  • Ran npm run build
  • Quasar SPA build completed successfully

Amama-Fatima and others added 30 commits January 2, 2026 01:24
…r mistral models. nova model tool calling still needs fixes
…extracted helper functions to separate files for much neater code
…ova and extracted helper functions to separate files for much neater code
…plicity mentioning that tool calling feature for this model not added yet
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants