llm: add Amazon Bedrock and Mantle inference - #79
Conversation
nickjalbert
left a comment
There was a problem hiding this comment.
Thanks for your work here! The live verification, provider tests, installer changes, and credential handling show that the integration was tested carefully.
Headlong now has a written provider policy, merged in PR #78. The implementation in this PR falls outside the core provider boundary because it adds SigV4 signing, AWS profile refresh, credential_process, installer behavior, and sandbox credential handling to the main completion path.
Headlong now supports integrations like Bedrock through an external adapter. Operators set LLM_PROVIDER=adapter and LLM_ADAPTER=/path/to/executable. The adapter receives the request through stdin and documented flags, then writes the response to stdout. bin/llm continues to manage the deadline, child process cleanup, health marker, and usage ledger.
Bedrock adapter can use the AWS CLI or an SDK and can manage AWS credentials without adding those dependencies to core. The adapter can support both Claude through InvokeModel and OpenAI models through Mantle. It can also keep the credential refresh behavior you implemented without reimplementing SigV4 in Bash.
I would welcome a reworked PR that moves the Bedrock implementation behind the adapter interface.
|
Will do - makes sense to move providers into an adapter pattern. |
Summary
Adds Amazon Bedrock as an inference provider for Headlong.
InvokeModelAPI.openai.gpt-*models use the Bedrock Mantle Responses API, including streaming.AWS_PROFILE/credential_processcredentials.Verification