Skip to content

llm: add Amazon Bedrock and Mantle inference - #79

Open
5herlocked wants to merge 1 commit into
laude-institute:mainfrom
5herlocked:main
Open

llm: add Amazon Bedrock and Mantle inference#79
5herlocked wants to merge 1 commit into
laude-institute:mainfrom
5herlocked:main

Conversation

@5herlocked

@5herlocked 5herlocked commented Aug 27, 2026

Copy link
Copy Markdown

Summary

Adds Amazon Bedrock as an inference provider for Headlong.

  • Claude models use Bedrock Runtime's InvokeModel API.
  • OpenAI openai.gpt-* models use the Bedrock Mantle Responses API, including streaming.
  • Authentication supports Bedrock API keys, standard AWS credentials, and refreshable AWS_PROFILE / credential_process credentials.
  • The installer detects Bedrock credentials, persists the selected Region and authentication configuration, and propagates credentials safely through shellm and its Docker sandbox.
  • Documentation and provider-specific regression tests are included.

Verification

  • Live Claude and Mantle calls passed with both bearer-token and SigV4 authentication.
  • Live profile-only calls passed using refreshed credentials from AWS CLI.
  • All 32 shell test suites pass.

@nickjalbert nickjalbert left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your work here! The live verification, provider tests, installer changes, and credential handling show that the integration was tested carefully.

Headlong now has a written provider policy, merged in PR #78. The implementation in this PR falls outside the core provider boundary because it adds SigV4 signing, AWS profile refresh, credential_process, installer behavior, and sandbox credential handling to the main completion path.

Headlong now supports integrations like Bedrock through an external adapter. Operators set LLM_PROVIDER=adapter and LLM_ADAPTER=/path/to/executable. The adapter receives the request through stdin and documented flags, then writes the response to stdout. bin/llm continues to manage the deadline, child process cleanup, health marker, and usage ledger.

Bedrock adapter can use the AWS CLI or an SDK and can manage AWS credentials without adding those dependencies to core. The adapter can support both Claude through InvokeModel and OpenAI models through Mantle. It can also keep the credential refresh behavior you implemented without reimplementing SigV4 in Bash.

I would welcome a reworked PR that moves the Bedrock implementation behind the adapter interface.

@5herlocked

Copy link
Copy Markdown
Author

Will do - makes sense to move providers into an adapter pattern.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants