Skip to content

fix: Bedrock routing for OpenAI-compatible endpoints#2219

Open
paylm wants to merge 2 commits intoWei-Shaw:mainfrom
vikadata:fix/upstream-bedrock-compat
Open

fix: Bedrock routing for OpenAI-compatible endpoints#2219
paylm wants to merge 2 commits intoWei-Shaw:mainfrom
vikadata:fix/upstream-bedrock-compat

Conversation

@paylm
Copy link
Copy Markdown

@paylm paylm commented May 6, 2026

Problem

Bedrock accounts can be selected for Anthropic-platform groups, but OpenAI-compatible routes such as /v1/chat/completions and /v1/responses were forwarding through the generic Anthropic upstream builder.

That path does not use Bedrock Runtime signing or API-key handling. When a Bedrock account is selected on /v1/chat/completions, the request can be sent as if it were a normal Anthropic API request, causing upstream authentication failures such as:

Authentication failed (401): x-api-key header is required

This is especially visible after the platform-based routing split for /v1/chat/completions, where non-OpenAI groups route to GatewayHandler.ChatCompletions and can schedule Bedrock accounts.

Fix

  • Detect account.IsBedrock() inside GatewayService.ForwardAsChatCompletions and GatewayService.ForwardAsResponses.
  • Route converted Anthropic request bodies through Bedrock Runtime request construction instead of the generic Anthropic upstream request builder.
  • Preserve Bedrock SigV4 and Bedrock API Key modes:
    • SigV4 accounts use NewBedrockSignerFromAccount.
    • Bedrock API Key accounts use Authorization: Bearer <api_key>.
  • Adapt Bedrock EventStream chunks back into Anthropic SSE so the existing Responses and Chat Completions conversion code can be reused.
  • Skip normal Anthropic model normalization for Bedrock accounts and let the Bedrock resolver produce the AWS model id.

Tests

  • Added coverage for Bedrock API Key accounts through /v1/chat/completions and /v1/responses, asserting requests go to bedrock-runtime with Authorization: Bearer and no x-api-key.
  • Not run locally in this sandbox: no go/gofmt binary is available, and Docker daemon access is denied.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 6, 2026

Thank you for your contribution! Before we can merge this PR, we need you to sign our Contributor License Agreement (CLA).

To sign, please reply with the following comment:

I have read the CLA Document and I hereby sign the CLA

You only need to sign once — it will be valid for all your future contributions to this project.


I have read the CLA Document and I hereby sign the CLA


You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

@paylm paylm changed the title Fix Bedrock routing for OpenAI-compatible endpoints fix: Bedrock routing for OpenAI-compatible endpoints May 6, 2026
@paylm
Copy link
Copy Markdown
Author

paylm commented May 6, 2026

Thank you for your contribution! Before we can merge this PR, we need you to sign our Contributor License Agreement (CLA).

To sign, please reply with the following comment:

I have read the CLA Document and I hereby sign the CLA

You only need to sign once — it will be valid for all your future contributions to this project.

I have read the CLA Document and I hereby sign the CLA

You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

I have read the CLA Document and I hereby sign the CLA

1 similar comment
@paylm
Copy link
Copy Markdown
Author

paylm commented May 6, 2026

Thank you for your contribution! Before we can merge this PR, we need you to sign our Contributor License Agreement (CLA).

To sign, please reply with the following comment:

I have read the CLA Document and I hereby sign the CLA

You only need to sign once — it will be valid for all your future contributions to this project.

I have read the CLA Document and I hereby sign the CLA

You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

I have read the CLA Document and I hereby sign the CLA

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant