You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by "shreemaan-abhishek (via GitHub)" <gi...@apache.org> on 2026/04/17 09:01:11 UTC
[PR] feat(ai-proxy): support aws bedrock [apisix]
shreemaan-abhishek opened a new pull request, #13249:
URL: https://github.com/apache/apisix/pull/13249
### Description
support `aws-bedrock` in the `ai-proxy` plugin.
### Checklist
- [x] I have explained the need for this PR and the problem it solves
- [x] I have explained the changes or the new features added to this PR
- [x] I have added tests corresponding to this change
- [ ] I have updated the documentation to reflect this change
- [ ] I have verified that this change is backward compatible (If not, please discuss on the [APISIX mailing list](https://github.com/apache/apisix/tree/master#community) first)
<!--
Note
1. Mark the PR as draft until it's ready to be reviewed.
2. Always add/update tests for any changes unless you have a good reason.
3. Always update the documentation to reflect the changes made in the PR.
4. Make a new commit to resolve conversations instead of `push -f`.
5. To resolve merge conflicts, merge master instead of rebasing.
6. Use "request review" to notify the reviewer after making changes.
7. Only a reviewer can mark a conversation as resolved.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ai-proxy): support aws bedrock [apisix]
Posted by "shreemaan-abhishek (via GitHub)" <gi...@apache.org>.
shreemaan-abhishek commented on code in PR #13249:
URL: https://github.com/apache/apisix/pull/13249#discussion_r3099141239
##########
apisix/plugins/ai-protocols/bedrock-converse.lua:
##########
@@ -0,0 +1,283 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Bedrock Converse protocol adapter (client-side).
+-- Handles detection and response parsing for the Amazon Bedrock
+-- Converse API format. Non-streaming only in this phase.
+
+local core = require("apisix.core")
+local string_sub = string.sub
+local type = type
+local ipairs = ipairs
+local table = table
+
+local _M = {}
+
+
+--- Detect whether the request matches the Bedrock Converse API format.
+-- Uses URI suffix (/converse) and body (valid JSON table with messages).
+function _M.matches(body, ctx)
+ local uri = ctx.var and ctx.var.uri
+ return uri and string_sub(uri, -9) == "/converse"
+ and type(body) == "table" and type(body.messages) == "table"
Review Comment:
nah this is unnecessary
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ai-proxy): support aws bedrock [apisix]
Posted by "nic-6443 (via GitHub)" <gi...@apache.org>.
nic-6443 commented on code in PR #13249:
URL: https://github.com/apache/apisix/pull/13249#discussion_r3128792063
##########
apisix/plugins/ai-providers/bedrock.lua:
##########
@@ -0,0 +1,81 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+local core = require("apisix.core")
+local str_fmt = string.format
+local ngx_escape_uri = ngx.escape_uri
+
+local host_template = "bedrock-runtime.%s.amazonaws.com"
+local chat_path_template = "/model/%s/converse"
+
+local function get_host(region)
+ return str_fmt(host_template, region)
+end
+
+
+local function get_region(instance_conf)
+ return core.table.try_read_attr(instance_conf, "provider_conf", "region")
+end
+
+
+local function get_node(instance_conf)
+ return {
+ scheme = "https",
+ host = get_host(get_region(instance_conf)),
+ port = 443,
+ }
+end
+
+
+-- Map override.request_body fields to Bedrock Converse API format.
+-- Bedrock uses inferenceConfig.maxTokens (camelCase, nested) for max output tokens.
+local function rewrite_chat_request_body(body, override, force)
Review Comment:
Since the bedrock provider may support other AI protocols in the future, such as openai-chat and anthropic-message, naming needs to be done according to the protocol here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ai-proxy): support aws bedrock [apisix]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #13249:
URL: https://github.com/apache/apisix/pull/13249#discussion_r3099067800
##########
apisix/plugins/ai-proxy/schema.lua:
##########
@@ -133,19 +157,54 @@ local ai_instance_schema = {
}
},
required = {"name", "provider", "auth", "weight"},
- ["if"] = {
- properties = { provider = { enum = { "vertex-ai" } } },
- },
- ["then"] = {
- properties = {
- provider_conf = provider_vertex_ai_schema,
+ allOf = {
+ {
+ ["if"] = {
+ properties = { provider = { enum = { "vertex-ai" } } },
+ },
+ ["then"] = {
+ properties = {
+ provider_conf = provider_vertex_ai_schema,
+ },
+ anyOf = {
+ { required = { "provider_conf" } },
+ { required = { "override" } },
+ },
+ },
+ },
+ {
+ ["if"] = {
+ properties = { provider = { enum = { "bedrock" } } },
+ },
+ ["then"] = {
+ properties = {
+ provider_conf = provider_bedrock_schema,
+ },
+ anyOf = {
+ { required = { "provider_conf" } },
+ { required = { "override" } },
+ },
Review Comment:
For provider="bedrock", the schema currently requires either provider_conf or override to be present. This prevents a valid configuration where region is supplied via AWS_REGION (or the provider default) and only auth/options are configured.
If the implementation supports env/default region (see bedrock.lua), consider relaxing this anyOf so bedrock can be configured without provider_conf/override (while still requiring options.model when override.endpoint is not set).
```suggestion
```
##########
apisix/plugins/ai-protocols/bedrock-converse.lua:
##########
@@ -0,0 +1,283 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Bedrock Converse protocol adapter (client-side).
+-- Handles detection and response parsing for the Amazon Bedrock
+-- Converse API format. Non-streaming only in this phase.
+
+local core = require("apisix.core")
+local string_sub = string.sub
+local type = type
+local ipairs = ipairs
+local table = table
+
+local _M = {}
+
+
+--- Detect whether the request matches the Bedrock Converse API format.
+-- Uses URI suffix (/converse) and body (valid JSON table with messages).
+function _M.matches(body, ctx)
+ local uri = ctx.var and ctx.var.uri
+ return uri and string_sub(uri, -9) == "/converse"
+ and type(body) == "table" and type(body.messages) == "table"
Review Comment:
matches() treats any request whose URI ends with "/converse" and has a messages array as Bedrock Converse. This will also match OpenAI Chat-style bodies (messages with string content) if a route happens to use a "/converse" suffix, causing protocol mis-detection and confusing "provider ... does not support bedrock-converse" errors.
Consider tightening detection to Bedrock-specific structure (e.g., messages[*].content is an array of {text=...} blocks and/or system is an array of {text=...}).
```suggestion
local function is_bedrock_text_block_array(value)
if type(value) ~= "table" then
return false
end
for _, block in ipairs(value) do
if type(block) == "table" and type(block.text) == "string" then
return true
end
end
return false
end
local function has_bedrock_converse_shape(body)
if type(body) ~= "table" or type(body.messages) ~= "table" then
return false
end
if is_bedrock_text_block_array(body.system) then
return true
end
for _, message in ipairs(body.messages) do
if type(message) == "table"
and is_bedrock_text_block_array(message.content)
then
return true
end
end
return false
end
--- Detect whether the request matches the Bedrock Converse API format.
-- Uses URI suffix (/converse) and Bedrock-specific content block structure.
function _M.matches(body, ctx)
local uri = ctx.var and ctx.var.uri
return uri and string_sub(uri, -9) == "/converse"
and has_bedrock_converse_shape(body)
```
##########
apisix/plugins/ai-protocols/bedrock-converse.lua:
##########
@@ -0,0 +1,283 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Bedrock Converse protocol adapter (client-side).
+-- Handles detection and response parsing for the Amazon Bedrock
+-- Converse API format. Non-streaming only in this phase.
+
+local core = require("apisix.core")
+local string_sub = string.sub
+local type = type
+local ipairs = ipairs
+local table = table
+
+local _M = {}
+
+
+--- Detect whether the request matches the Bedrock Converse API format.
+-- Uses URI suffix (/converse) and body (valid JSON table with messages).
+function _M.matches(body, ctx)
+ local uri = ctx.var and ctx.var.uri
+ return uri and string_sub(uri, -9) == "/converse"
+ and type(body) == "table" and type(body.messages) == "table"
+end
+
+
+--- Check whether the request is a streaming request.
+-- Streaming is not supported in this phase.
+function _M.is_streaming(body)
+ return false
+end
+
+
+--- Prepare the outgoing request body for the target provider.
+-- Remove fields Bedrock doesn't accept.
+function _M.prepare_outgoing_request(body)
+ body.stream = nil
+end
+
+
+--- Extract token usage from a non-streaming Bedrock response.
+-- Bedrock format: res_body.usage.inputTokens / outputTokens / totalTokens
+function _M.extract_usage(res_body)
+ if type(res_body) ~= "table" or type(res_body.usage) ~= "table" then
+ return nil, nil
+ end
+ local raw = res_body.usage
+ return {
+ prompt_tokens = raw.inputTokens or 0,
+ completion_tokens = raw.outputTokens or 0,
+ total_tokens = raw.totalTokens
+ or (raw.inputTokens or 0) + (raw.outputTokens or 0),
+ }, raw
+end
+
+
+--- Extract response text from a Bedrock Converse response.
+-- Bedrock format: res_body.output.message.content[].text
+function _M.extract_response_text(res_body)
+ if type(res_body) ~= "table" then
+ return nil
+ end
+ local message = res_body.output and res_body.output.message
+ if type(message) ~= "table" or type(message.content) ~= "table" then
+ return nil
+ end
+ local texts = {}
+ for _, block in ipairs(message.content) do
+ if type(block) == "table" and type(block.text) == "string" then
+ core.table.insert(texts, block.text)
+ end
+ end
+ if #texts > 0 then
+ return table.concat(texts, " ")
+ end
+ return nil
+end
+
+
+--- Extract all text content from a request body for moderation.
+function _M.extract_request_content(body)
+ local contents = {}
+ if type(body.system) == "table" then
+ for _, block in ipairs(body.system) do
+ if type(block) == "table" and type(block.text) == "string" then
+ core.table.insert(contents, block.text)
+ end
+ end
+ end
+ if type(body.messages) == "table" then
+ for _, message in ipairs(body.messages) do
+ if type(message) ~= "table" then
+ goto CONTINUE_MESSAGE
+ end
+ if type(message.content) == "table" then
+ for _, block in ipairs(message.content) do
+ if type(block) == "table" and type(block.text) == "string" then
+ core.table.insert(contents, block.text)
+ end
+ end
+ end
+ ::CONTINUE_MESSAGE::
+ end
+ end
+ return contents
+end
+
+
+--- Get messages in canonical {role, content} format.
+-- Bedrock content blocks [{text: "..."}] are flattened to plain text.
+function _M.get_messages(body)
+ local messages = {}
+ if type(body.system) == "table" then
+ local texts = {}
+ for _, block in ipairs(body.system) do
+ if type(block) == "table" and type(block.text) == "string" then
+ core.table.insert(texts, block.text)
+ end
+ end
+ if #texts > 0 then
+ core.table.insert(messages, {
+ role = "system",
+ content = table.concat(texts, " "),
+ })
+ end
+ end
+ if type(body.messages) == "table" then
+ for _, message in ipairs(body.messages) do
+ if type(message) ~= "table" then
+ goto CONTINUE
+ end
+ if type(message.content) == "table" then
+ local texts = {}
+ for _, block in ipairs(message.content) do
+ if type(block) == "table" and type(block.text) == "string" then
+ core.table.insert(texts, block.text)
+ end
+ end
+ if #texts > 0 then
+ core.table.insert(messages, {
+ role = message.role,
+ content = table.concat(texts, " "),
+ })
+ end
+ end
+ ::CONTINUE::
+ end
+ end
+ return messages
+end
+
+
+--- Prepend messages to the request body.
+-- System messages go to body.system; user/assistant messages go to body.messages.
+function _M.prepend_messages(body, msgs)
+ if not msgs or #msgs == 0 then return end
+
+ local new_system_blocks = {}
+ local new_chat_messages = {}
+ for _, msg in ipairs(msgs) do
+ if msg.role == "system" then
+ core.table.insert(new_system_blocks, {text = msg.content})
+ else
+ core.table.insert(new_chat_messages, {
+ role = msg.role,
+ content = {{text = msg.content}},
+ })
+ end
+ end
+
+ if #new_system_blocks > 0 then
+ if not body.system then
+ body.system = {}
+ end
+ local merged_system = {}
+ for _, block in ipairs(new_system_blocks) do
+ core.table.insert(merged_system, block)
+ end
+ for _, block in ipairs(body.system) do
+ core.table.insert(merged_system, block)
+ end
+ body.system = merged_system
+ end
+
+ if #new_chat_messages > 0 then
+ if not body.messages then
+ body.messages = {}
+ end
+ local merged_messages = {}
+ for _, msg in ipairs(new_chat_messages) do
+ core.table.insert(merged_messages, msg)
+ end
+ for _, msg in ipairs(body.messages) do
+ core.table.insert(merged_messages, msg)
+ end
+ body.messages = merged_messages
+ end
+end
+
+
+--- Append messages to the request body.
+-- System messages go to body.system; user/assistant messages go to body.messages.
+function _M.append_messages(body, msgs)
+ if not msgs or #msgs == 0 then return end
+
+ for _, msg in ipairs(msgs) do
+ if msg.role == "system" then
+ if not body.system then
+ body.system = {}
+ end
+ core.table.insert(body.system, {text = msg.content})
+ else
+ if not body.messages then
+ body.messages = {}
+ end
+ core.table.insert(body.messages, {
+ role = msg.role,
+ content = {{text = msg.content}},
+ })
+ end
Review Comment:
append_messages() has the same assumption as prepend_messages(): if body.system/body.messages are present but not tables, core.table.insert(...) will error. Normalizing these fields to tables before inserting would prevent request-time exceptions on malformed input.
##########
t/plugin/ai-proxy-bedrock.t:
##########
@@ -0,0 +1,490 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+use t::APISIX 'no_plan';
+
+log_level("info");
+repeat_each(1);
+no_long_string();
+no_shuffle();
+no_root_location();
+
+
+add_block_preprocessor(sub {
+ my ($block) = @_;
+
+ if (!defined $block->request) {
+ $block->set_value("request", "GET /t");
+ }
+
+ my $main_config = $block->main_config // <<_EOC_;
+ env AWS_EC2_METADATA_DISABLED=true;
+ env AWS_REGION=us-east-1;
+_EOC_
+ $block->set_value("main_config", $main_config);
+
+ my $user_yaml_config = <<_EOC_;
+plugins:
+ - ai-proxy-multi
+ - prometheus
+_EOC_
+ $block->set_value("extra_yaml_config", $user_yaml_config);
+
+ my $http_config = $block->http_config // <<_EOC_;
+ server {
+ server_name bedrock;
+ listen 6724;
+
+ default_type 'application/json';
+
+ location ~ ^/model/.+/converse\$ {
+ content_by_lua_block {
+ local json = require("cjson.safe")
+
+ if ngx.req.get_method() ~= "POST" then
+ ngx.status = 400
+ ngx.say("Unsupported request method: ", ngx.req.get_method())
+ return
+ end
+
+ -- Check SigV4 auth headers
+ local auth_header = ngx.req.get_headers()["authorization"]
+ local amz_date = ngx.req.get_headers()["x-amz-date"]
+ if not auth_header or not amz_date then
+ ngx.status = 403
+ ngx.say(json.encode({
+ message = "Missing Authentication Token"
+ }))
+ return
+ end
+
+ -- Capture session token if provided so tests can assert
+ -- that auth.aws.session_token was propagated as
+ -- x-amz-security-token.
+ local session_token = ngx.req.get_headers()["x-amz-security-token"]
+
+ ngx.req.read_body()
+ local body_data = ngx.req.get_body_data()
+ local body, err = json.decode(body_data)
+
+ if not body then
+ ngx.status = 400
+ ngx.say(json.encode({ message = "Invalid JSON: " .. (err or "") }))
+ return
+ end
+
+ -- Verify model is NOT in the body (remove_model = true)
+ if body.model then
+ ngx.status = 400
+ ngx.say(json.encode({
+ message = "model field should not be in request body"
+ }))
+ return
+ end
+
+ -- Verify request has messages
+ if not body.messages or #body.messages < 1 then
+ ngx.status = 400
+ ngx.say(json.encode({ message = "messages is required" }))
+ return
+ end
+
+ -- Extract text from first user message
+ local first_content = ""
+ for _, msg in ipairs(body.messages) do
+ if msg.role == "user" and msg.content then
+ for _, block in ipairs(msg.content) do
+ if block.text then
+ first_content = block.text
+ break
+ end
+ end
+ break
+ end
+ end
+
Review Comment:
The variable first_content is computed but never used. Removing it (and the loop that populates it) would reduce noise in the test stub and make the intent of the mock handler clearer.
```suggestion
```
##########
apisix/plugins/ai-protocols/bedrock-converse.lua:
##########
@@ -0,0 +1,283 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Bedrock Converse protocol adapter (client-side).
+-- Handles detection and response parsing for the Amazon Bedrock
+-- Converse API format. Non-streaming only in this phase.
+
+local core = require("apisix.core")
+local string_sub = string.sub
+local type = type
+local ipairs = ipairs
+local table = table
+
+local _M = {}
+
+
+--- Detect whether the request matches the Bedrock Converse API format.
+-- Uses URI suffix (/converse) and body (valid JSON table with messages).
+function _M.matches(body, ctx)
+ local uri = ctx.var and ctx.var.uri
+ return uri and string_sub(uri, -9) == "/converse"
+ and type(body) == "table" and type(body.messages) == "table"
+end
+
+
+--- Check whether the request is a streaming request.
+-- Streaming is not supported in this phase.
+function _M.is_streaming(body)
+ return false
+end
+
+
+--- Prepare the outgoing request body for the target provider.
+-- Remove fields Bedrock doesn't accept.
+function _M.prepare_outgoing_request(body)
+ body.stream = nil
+end
+
+
+--- Extract token usage from a non-streaming Bedrock response.
+-- Bedrock format: res_body.usage.inputTokens / outputTokens / totalTokens
+function _M.extract_usage(res_body)
+ if type(res_body) ~= "table" or type(res_body.usage) ~= "table" then
+ return nil, nil
+ end
+ local raw = res_body.usage
+ return {
+ prompt_tokens = raw.inputTokens or 0,
+ completion_tokens = raw.outputTokens or 0,
+ total_tokens = raw.totalTokens
+ or (raw.inputTokens or 0) + (raw.outputTokens or 0),
+ }, raw
+end
+
+
+--- Extract response text from a Bedrock Converse response.
+-- Bedrock format: res_body.output.message.content[].text
+function _M.extract_response_text(res_body)
+ if type(res_body) ~= "table" then
+ return nil
+ end
+ local message = res_body.output and res_body.output.message
+ if type(message) ~= "table" or type(message.content) ~= "table" then
+ return nil
+ end
+ local texts = {}
+ for _, block in ipairs(message.content) do
+ if type(block) == "table" and type(block.text) == "string" then
+ core.table.insert(texts, block.text)
+ end
+ end
+ if #texts > 0 then
+ return table.concat(texts, " ")
+ end
+ return nil
+end
+
+
+--- Extract all text content from a request body for moderation.
+function _M.extract_request_content(body)
+ local contents = {}
+ if type(body.system) == "table" then
+ for _, block in ipairs(body.system) do
+ if type(block) == "table" and type(block.text) == "string" then
+ core.table.insert(contents, block.text)
+ end
+ end
+ end
+ if type(body.messages) == "table" then
+ for _, message in ipairs(body.messages) do
+ if type(message) ~= "table" then
+ goto CONTINUE_MESSAGE
+ end
+ if type(message.content) == "table" then
+ for _, block in ipairs(message.content) do
+ if type(block) == "table" and type(block.text) == "string" then
+ core.table.insert(contents, block.text)
+ end
+ end
+ end
+ ::CONTINUE_MESSAGE::
+ end
+ end
+ return contents
+end
+
+
+--- Get messages in canonical {role, content} format.
+-- Bedrock content blocks [{text: "..."}] are flattened to plain text.
+function _M.get_messages(body)
+ local messages = {}
+ if type(body.system) == "table" then
+ local texts = {}
+ for _, block in ipairs(body.system) do
+ if type(block) == "table" and type(block.text) == "string" then
+ core.table.insert(texts, block.text)
+ end
+ end
+ if #texts > 0 then
+ core.table.insert(messages, {
+ role = "system",
+ content = table.concat(texts, " "),
+ })
+ end
+ end
+ if type(body.messages) == "table" then
+ for _, message in ipairs(body.messages) do
+ if type(message) ~= "table" then
+ goto CONTINUE
+ end
+ if type(message.content) == "table" then
+ local texts = {}
+ for _, block in ipairs(message.content) do
+ if type(block) == "table" and type(block.text) == "string" then
+ core.table.insert(texts, block.text)
+ end
+ end
+ if #texts > 0 then
+ core.table.insert(messages, {
+ role = message.role,
+ content = table.concat(texts, " "),
+ })
+ end
+ end
+ ::CONTINUE::
+ end
+ end
+ return messages
+end
+
+
+--- Prepend messages to the request body.
+-- System messages go to body.system; user/assistant messages go to body.messages.
+function _M.prepend_messages(body, msgs)
+ if not msgs or #msgs == 0 then return end
+
+ local new_system_blocks = {}
+ local new_chat_messages = {}
+ for _, msg in ipairs(msgs) do
+ if msg.role == "system" then
+ core.table.insert(new_system_blocks, {text = msg.content})
+ else
+ core.table.insert(new_chat_messages, {
+ role = msg.role,
+ content = {{text = msg.content}},
+ })
+ end
+ end
+
+ if #new_system_blocks > 0 then
+ if not body.system then
+ body.system = {}
+ end
+ local merged_system = {}
+ for _, block in ipairs(new_system_blocks) do
+ core.table.insert(merged_system, block)
+ end
+ for _, block in ipairs(body.system) do
+ core.table.insert(merged_system, block)
+ end
+ body.system = merged_system
Review Comment:
prepend_messages() assumes body.system and body.messages are tables when they already exist. If a client sends an invalid Bedrock body where system/messages are present but not arrays, this will raise (ipairs/insert on non-table) instead of failing gracefully.
Consider normalizing these fields before use (e.g., if type(body.system) ~= "table" then body.system = {} end; same for body.messages) to avoid runtime errors.
##########
apisix/plugins/ai-transport/auth-aws.lua:
##########
@@ -0,0 +1,134 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- AWS SigV4 signing helper for AI providers.
+-- Signs outgoing HTTP requests using AWS Signature Version 4.
+
+require("resty.aws.config") -- reads env vars before init
+local aws = require("resty.aws")
+local core = require("apisix.core")
+local signer = require("resty.aws.request.sign")
+local ngx_escape_uri = ngx.escape_uri
+
+local aws_instance
+
+
+-- Encode a URL path for AWS SigV4 canonical URI.
+-- AWS SigV4 requires each path segment to be URI-encoded twice. This function
+-- applies a single ngx.escape_uri pass per segment; the second encoding pass
+-- comes from the caller's input, which is expected to already be URL-encoded
+-- (e.g. bedrock.lua escapes the model ID before building the path, turning
+-- raw ":" into "%3A"). Running ngx.escape_uri here then escapes the "%" to
+-- "%25", yielding the "%253A" required by the canonical URI.
+local function encode_path_for_canonical_uri(path)
+ local segments = {}
+ for segment in path:gmatch("[^/]+") do
+ -- Encodes any unreserved chars and re-escapes "%" from the upstream
+ -- encoding pass, producing the double-encoded form SigV4 expects.
+ segments[#segments + 1] = ngx_escape_uri(segment)
Review Comment:
encode_path_for_canonical_uri() assumes params.path is already URL-encoded once (so a single escape pass yields the required double-encoding). When users configure bedrock via override.endpoint with a raw path (common, and also used in the tests), segments containing ':' or other reserved characters will only be encoded once in canonicalURI, producing an invalid SigV4 signature.
Consider deriving canonicalURI by normalizing each segment from its decoded form (e.g., ngx.unescape_uri(segment)) and then applying URI-encoding twice, so both pre-encoded and raw paths sign correctly.
```suggestion
local ngx_unescape_uri = ngx.unescape_uri
local aws_instance
-- Encode a URL path for AWS SigV4 canonical URI.
-- AWS SigV4 requires each path segment to be URI-encoded twice. Normalize
-- each segment from its decoded form first so both raw paths (for example
-- "model:invoke") and already encoded paths (for example "model%3Ainvoke")
-- produce the same canonical URI segment ("%253A" for ":").
local function encode_path_for_canonical_uri(path)
local segments = {}
for segment in path:gmatch("[^/]+") do
local normalized_segment = ngx_unescape_uri(segment)
segments[#segments + 1] = ngx_escape_uri(ngx_escape_uri(normalized_segment))
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ai-proxy): support aws bedrock [apisix]
Posted by "nic-6443 (via GitHub)" <gi...@apache.org>.
nic-6443 commented on code in PR #13249:
URL: https://github.com/apache/apisix/pull/13249#discussion_r3128787028
##########
apisix/plugins/ai-providers/bedrock.lua:
##########
@@ -0,0 +1,81 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+local core = require("apisix.core")
+local str_fmt = string.format
+local ngx_escape_uri = ngx.escape_uri
+
+local host_template = "bedrock-runtime.%s.amazonaws.com"
+local chat_path_template = "/model/%s/converse"
+
+local function get_host(region)
+ return str_fmt(host_template, region)
+end
+
+
+local function get_region(instance_conf)
+ return core.table.try_read_attr(instance_conf, "provider_conf", "region")
+end
+
+
+local function get_node(instance_conf)
+ return {
+ scheme = "https",
+ host = get_host(get_region(instance_conf)),
+ port = 443,
+ }
+end
+
+
+-- Map override.request_body fields to Bedrock Converse API format.
+-- Bedrock uses inferenceConfig.maxTokens (camelCase, nested) for max output tokens.
+local function rewrite_chat_request_body(body, override, force)
Review Comment:
```suggestion
local function rewrite_converse_request_body(body, override, force)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ai-proxy): support aws bedrock [apisix]
Posted by "elsodiop (via GitHub)" <gi...@apache.org>.
elsodiop commented on PR #13249:
URL: https://github.com/apache/apisix/pull/13249#issuecomment-5059804671
Hello,
ai-proxy accepts $secret:// references in auth.aws, but the Bedrock provider passes access_key_id directly to the SigV4 signer without resolving the secret first. The signer therefore builds the credential scope using the literal secret URI.
I created a Bedrock toute with:
- name: ai-proxy
enable: true
config:
provider: bedrock
auth:
aws:
access_key_id: "$secret://aws/xxxxxxxx/secret-name/access-key-id"
secret_access_key: "$secret://aws/xxxx/secret-name/secret-access-key"
provider_conf:
region: us-east-1
options:
model: anthropic.claude-3-5-sonnet-20240620-v1:0
When i test my route i get {"message":"Credential must have exactly 5 slash-delimited elements, e.g. keyid/date/region/service/term, got '$secret://aws/xxxx/secret-name/access-key-id/20260723/us-east-1/bedrock/aws4_request'"}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ai-proxy): support aws bedrock [apisix]
Posted by "nic-6443 (via GitHub)" <gi...@apache.org>.
nic-6443 commented on code in PR #13249:
URL: https://github.com/apache/apisix/pull/13249#discussion_r3114891024
##########
apisix/plugins/ai-protocols/bedrock-converse.lua:
##########
@@ -0,0 +1,283 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Bedrock Converse protocol adapter (client-side).
+-- Handles detection and response parsing for the Amazon Bedrock
+-- Converse API format. Non-streaming only in this phase.
+
+local core = require("apisix.core")
+local string_sub = string.sub
+local type = type
+local ipairs = ipairs
+local table = table
+
+local _M = {}
+
+
+--- Detect whether the request matches the Bedrock Converse API format.
+-- Uses URI suffix (/converse) and body (valid JSON table with messages).
+function _M.matches(body, ctx)
+ local uri = ctx.var and ctx.var.uri
+ return uri and string_sub(uri, -9) == "/converse"
+ and type(body) == "table" and type(body.messages) == "table"
+end
+
+
+--- Check whether the request is a streaming request.
+-- Streaming is not supported in this phase.
+function _M.is_streaming(body)
+ return false
+end
+
+
+--- Prepare the outgoing request body for the target provider.
+-- Remove fields Bedrock doesn't accept.
+function _M.prepare_outgoing_request(body)
+ body.stream = nil
Review Comment:
This should be a TODO item, not that Bedrock doesn't accept it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ai-proxy): support aws bedrock [apisix]
Posted by "shreemaan-abhishek (via GitHub)" <gi...@apache.org>.
shreemaan-abhishek merged PR #13249:
URL: https://github.com/apache/apisix/pull/13249
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ai-proxy): support aws bedrock [apisix]
Posted by "membphis (via GitHub)" <gi...@apache.org>.
membphis commented on code in PR #13249:
URL: https://github.com/apache/apisix/pull/13249#discussion_r3108746580
##########
apisix/plugins/ai-providers/base.lua:
##########
@@ -187,6 +187,20 @@ function _M.build_request(self, ctx, conf, request_body, opts)
request_body.model = nil
end
+ -- AWS SigV4 signing (must be last — signs the finalized body)
+ if auth.aws then
+ local auth_aws = require("apisix.plugins.ai-transport.auth-aws")
+ local region = opts.conf and opts.conf.region
+ if not region then
+ return nil, "missing region for AWS SigV4 signing "
+ .. "(provider_conf.region required for bedrock)"
+ end
+ local sign_err = auth_aws.sign_request(params, auth.aws, region)
+ if sign_err then
+ return nil, "failed to sign AWS request: " .. sign_err
+ end
+ end
Review Comment:
If we can follow this way, it is much better @shreemaan-abhishek
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ai-proxy): support aws bedrock [apisix]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #13249:
URL: https://github.com/apache/apisix/pull/13249#discussion_r3108077216
##########
docs/en/latest/plugins/ai-proxy-multi.md:
##########
@@ -58,10 +70,11 @@ In addition, the Plugin also supports logging LLM request information in the acc
| balancer.key | string | False | | | Used when `type` is `chash`. When `hash_on` is set to `header` or `cookie`, `key` is required. When `hash_on` is set to `consumer`, `key` is not required as the consumer name will be used as the key automatically. |
| instances | array[object] | True | | | LLM instance configurations. |
| instances.name | string | True | | | Name of the LLM service instance. |
-| instances.provider | string | True | | [openai, deepseek, azure-openai, aimlapi, anthropic, openrouter, gemini, vertex-ai, openai-compatible] | LLM service provider. When set to `openai`, the Plugin will proxy the request to `api.openai.com`. When set to `deepseek`, the Plugin will proxy the request to `api.deepseek.com`. When set to `aimlapi`, the Plugin uses the OpenAI-compatible driver and proxies the request to `api.aimlapi.com` by default. When set to `anthropic`, the Plugin will proxy the request to `api.anthropic.com` by default. When set to `openrouter`, the Plugin uses the OpenAI-compatible driver and proxies the request to `openrouter.ai` by default. When set to `gemini`, the Plugin uses the OpenAI-compatible driver and proxies the request to `generativelanguage.googleapis.com` by default. When set to `vertex-ai`, the Plugin will proxy the request to `aiplatform.googleapis.com` by default and requires `provide
r_conf` or `override`. When set to `openai-compatible`, the Plugin will proxy the request to the custom endpoint configured in `override`. |
-| instances.provider_conf | object | False | | | Configuration for the specific provider. Required when `provider` is set to `vertex-ai` and `override` is not configured. |
+| instances.provider | string | True | | [openai, deepseek, azure-openai, aimlapi, anthropic, openrouter, gemini, vertex-ai, bedrock, openai-compatible] | LLM service provider. When set to `openai`, the Plugin will proxy the request to `api.openai.com`. When set to `deepseek`, the Plugin will proxy the request to `api.deepseek.com`. When set to `aimlapi`, the Plugin uses the OpenAI-compatible driver and proxies the request to `api.aimlapi.com` by default. When set to `anthropic`, the Plugin will proxy the request to `api.anthropic.com` by default. When set to `openrouter`, the Plugin uses the OpenAI-compatible driver and proxies the request to `openrouter.ai` by default. When set to `gemini`, the Plugin uses the OpenAI-compatible driver and proxies the request to `generativelanguage.googleapis.com` by default. When set to `vertex-ai`, the Plugin will proxy the request to `aiplatform.googleapis.com` by default and requires
`provider_conf` or `override`. When set to `bedrock`, the Plugin will proxy the request to the AWS Bedrock Converse API (`bedrock-runtime.<region>.amazonaws.com`) and signs the request with AWS SigV4. When set to `openai-compatible`, the Plugin will proxy the request to the custom endpoint configured in `override`. |
+| instances.provider_conf | object | False | | | Configuration for the specific provider. Required when `provider` is set to `vertex-ai` and `override` is not configured. Required when `provider` is set to `bedrock`. |
| instances.provider_conf.project_id | string | True | | | Google Cloud Project ID. |
| instances.provider_conf.region | string | True | | | Google Cloud Region. |
+| instances.provider_conf.region | string | True | | minLength = 1 | AWS region used to construct the Bedrock endpoint and to sign the request with SigV4. Required when `provider` is set to `bedrock`. |
| instances.priority | integer | False | 0 | | Priority of the LLM instance in load balancing. `priority` takes precedence over `weight`. |
| instances.weight | string | True | 0 | greater or equal to 0 | Weight of the LLM instance in load balancing. |
| instances.auth | object | True | | | Authentication configurations. |
Review Comment:
The attributes table now lists `instances.provider_conf.region` twice (Google Cloud Region vs AWS Region). Duplicate keys in the same table are ambiguous and make it hard to tell which providers each row applies to. Please consolidate into a single row with provider-specific semantics/requirements, or split provider-specific config into separate subsections.
```suggestion
| instances.provider_conf.region | string | True | | minLength = 1 | Provider-specific region. For `vertex-ai`, this is the Google Cloud region. For `bedrock`, this is the AWS region used to construct the Bedrock endpoint and sign the request with SigV4. Required when `provider` is set to `bedrock`, and required for `vertex-ai` when `provider_conf` is used. |
| instances.priority | integer | False | 0 | | Priority of the LLM instance in load balancing. `priority` takes precedence over `weight`. |
| instances.weight | string | True | 0 | greater or equal to 0 | Weight of the LLM instance in load balancing. |
| instances.auth | object | True | | | Authentication configurations. |
```
##########
docs/en/latest/plugins/ai-proxy.md:
##########
@@ -47,25 +47,42 @@ In addition, the Plugin also supports logging LLM request information in the acc
| `messages.role` | String | True | Role of the message (`system`, `user`, `assistant`).|
| `messages.content` | String | True | Content of the message. |
+### Bedrock Converse Request Format
+
+When `provider` is set to `bedrock`, the Plugin expects requests in the [Bedrock Converse API](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html) format. The request URI must end with `/converse` and the body must contain a `messages` array.
+
+| Name | Type | Required | Description |
+| ------------------ | ------ | -------- | ---------------------------------------------------------------------------------------------------- |
+| `messages` | Array | True | An array of message objects. |
+| `messages.role` | String | True | Role of the message (`user`, `assistant`). |
+| `messages.content` | Array | True | An array of content blocks. Each block contains a `text` field (e.g., `[{"text": "What is 1+1?"}]`). |
+| `system` | Array | False | Optional system prompt blocks (e.g., `[{"text": "You are a helpful assistant."}]`). |
+| `inferenceConfig` | Object | False | Optional inference parameters such as `maxTokens`, `temperature`, `topP`, etc. |
+
## Attributes
| Name | Type | Required | Default | Valid values | Description |
|--------------------|--------|----------|---------|------------------------------------------|-------------|
-| provider | string | True | | [openai, deepseek, azure-openai, aimlapi, anthropic, openrouter, gemini, vertex-ai, openai-compatible] | LLM service provider. When set to `openai`, the Plugin will proxy the request to `https://api.openai.com/chat/completions`. When set to `deepseek`, the Plugin will proxy the request to `https://api.deepseek.com/chat/completions`. When set to `aimlapi`, the Plugin uses the OpenAI-compatible driver and proxies the request to `https://api.aimlapi.com/v1/chat/completions` by default. When set to `anthropic`, the Plugin will proxy the request to `https://api.anthropic.com/v1/chat/completions` by default. When set to `openrouter`, the Plugin uses the OpenAI-compatible driver and proxies the request to `https://openrouter.ai/api/v1/chat/completions` by default. When set to `gemini`, the Plugin uses the OpenAI-compatible driver and proxies the request to `https://generativelanguage.googleapis.com/v1beta/openai/chat/completions` by defa
ult. When set to `vertex-ai`, the Plugin will proxy the request to `https://aiplatform.googleapis.com` by default and requires `provider_conf` or `override`. When set to `openai-compatible`, the Plugin will proxy the request to the custom endpoint configured in `override`. |
-| provider_conf | object | False | | | Configuration for the specific provider. Required when `provider` is set to `vertex-ai` and `override` is not configured. |
+| provider | string | True | | [openai, deepseek, azure-openai, aimlapi, anthropic, openrouter, gemini, vertex-ai, bedrock, openai-compatible] | LLM service provider. When set to `openai`, the Plugin will proxy the request to `https://api.openai.com/chat/completions`. When set to `deepseek`, the Plugin will proxy the request to `https://api.deepseek.com/chat/completions`. When set to `aimlapi`, the Plugin uses the OpenAI-compatible driver and proxies the request to `https://api.aimlapi.com/v1/chat/completions` by default. When set to `anthropic`, the Plugin will proxy the request to `https://api.anthropic.com/v1/chat/completions` by default. When set to `openrouter`, the Plugin uses the OpenAI-compatible driver and proxies the request to `https://openrouter.ai/api/v1/chat/completions` by default. When set to `gemini`, the Plugin uses the OpenAI-compatible driver and proxies the request to `https://generativelanguage.googleapis.com/v1beta/openai/chat/completions
` by default. When set to `vertex-ai`, the Plugin will proxy the request to `https://aiplatform.googleapis.com` by default and requires `provider_conf` or `override`. When set to `bedrock`, the Plugin will proxy the request to the AWS Bedrock Converse API (`https://bedrock-runtime.<region>.amazonaws.com`) and signs the request with AWS SigV4. When set to `openai-compatible`, the Plugin will proxy the request to the custom endpoint configured in `override`. |
+| provider_conf | object | False | | | Configuration for the specific provider. Required when `provider` is set to `vertex-ai` and `override` is not configured. Required when `provider` is set to `bedrock`. |
| provider_conf.project_id | string | True | | | Google Cloud Project ID. |
| provider_conf.region | string | True | | | Google Cloud Region. |
+| provider_conf.region | string | True | | minLength = 1 | AWS region used to construct the Bedrock endpoint and to sign the request with SigV4. Required when `provider` is set to `bedrock`. |
Review Comment:
The attributes table now lists `provider_conf.region` twice (once for Google Cloud Region and once for AWS Region). This is ambiguous for readers and makes it unclear which providers each row applies to. Please consolidate into a single `provider_conf.region` row that explains the per-provider meaning/requirements, or split provider-specific configuration into separate subsections to avoid duplicate keys.
```suggestion
| provider_conf.region | string | True | | minLength = 1 | Provider-specific region. For `vertex-ai`, this is the Google Cloud region. For `bedrock`, this is the AWS region used to construct the Bedrock endpoint and sign the request with SigV4. Required when `provider` is set to `bedrock`. |
```
##########
apisix/plugins/ai-providers/base.lua:
##########
@@ -187,6 +187,20 @@ function _M.build_request(self, ctx, conf, request_body, opts)
request_body.model = nil
end
+ -- AWS SigV4 signing (must be last — signs the finalized body)
+ if auth.aws then
+ local auth_aws = require("apisix.plugins.ai-transport.auth-aws")
+ local region = opts.conf and opts.conf.region
+ if not region then
+ return nil, "missing region for AWS SigV4 signing "
+ .. "(provider_conf.region required for bedrock)"
+ end
+ local sign_err = auth_aws.sign_request(params, auth.aws, region)
+ if sign_err then
+ return nil, "failed to sign AWS request: " .. sign_err
+ end
+ end
Review Comment:
AWS SigV4 signing is triggered solely by the presence of `auth.aws`, regardless of which provider is selected. Because the schema doesn’t currently restrict `auth.aws` to Bedrock-only, a misconfigured instance could end up SigV4-signing requests to non-AWS providers (adding/overwriting `authorization`, `host`, etc.). Consider gating this behind `self`/provider type (e.g., only sign when provider is `bedrock`) and/or tightening the schema so `auth.aws` is only allowed/required for Bedrock.
##########
apisix/plugins/ai-proxy/schema.lua:
##########
@@ -120,9 +145,16 @@ local ai_instance_schema = {
properties = {
endpoint = {
type = "string",
- description = "To be specified to override the endpoint of the AI Instance",
+ description = "Override the endpoint of the AI Instance. "
+ .. "Typically used for custom hosts (e.g., AWS "
+ .. "PrivateLink, reverse proxies) — provide only the "
+ .. "scheme + host so the plugin computes the path. "
+ .. "If you include a path with reserved characters "
+ .. "(e.g., Bedrock inference profile ARNs containing "
+ .. "':' or '/'), they must be URL-encoded.",
Review Comment:
The schema description for `override.endpoint` says to “provide only the scheme + host so the plugin computes the path”, but `ai-providers/base.lua` explicitly uses `parsed_url.path` when a path is present (and the Bedrock tests/examples set a full `/model/.../converse` path). This description is misleading; please update it to reflect that `override.endpoint` may include a path (and query), and clarify when the plugin computes vs honors the provided path.
```suggestion
.. "PrivateLink, reverse proxies). You may provide "
.. "only the scheme + host, in which case the plugin "
.. "computes the provider-specific path, or provide "
.. "a full endpoint including path and query, in "
.. "which case the plugin uses the supplied path/query. "
.. "If your custom path or query contains reserved "
.. "characters (e.g., Bedrock inference profile ARNs "
.. "containing ':' or '/'), they must be URL-encoded.",
```
##########
apisix/plugins/ai-proxy/schema.lua:
##########
@@ -133,19 +165,51 @@ local ai_instance_schema = {
}
},
required = {"name", "provider", "auth", "weight"},
- ["if"] = {
- properties = { provider = { enum = { "vertex-ai" } } },
- },
- ["then"] = {
- properties = {
- provider_conf = provider_vertex_ai_schema,
+ allOf = {
+ {
+ ["if"] = {
+ properties = { provider = { enum = { "vertex-ai" } } },
+ },
+ ["then"] = {
+ properties = {
+ provider_conf = provider_vertex_ai_schema,
+ },
+ anyOf = {
+ { required = { "provider_conf" } },
+ { required = { "override" } },
+ },
+ },
+ },
+ {
+ ["if"] = {
+ properties = { provider = { enum = { "bedrock" } } },
+ },
+ ["then"] = {
+ properties = {
+ provider_conf = provider_bedrock_schema,
+ },
+ required = { "provider_conf" },
Review Comment:
For `provider=bedrock`, the schema currently enforces `provider_conf.region` and (optionally) `options.model`, but it does not require `auth.aws`. Given the provider relies on SigV4 signing, it’s better to fail fast at config time by requiring `auth.aws` (and potentially disallowing it for non-AWS providers) to avoid unsigned requests reaching Bedrock and returning confusing upstream errors.
```suggestion
auth = {
required = { "aws" },
},
},
required = { "provider_conf", "auth" },
```
##########
t/plugin/ai-proxy-bedrock.t:
##########
@@ -0,0 +1,553 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+use t::APISIX 'no_plan';
+
+log_level("info");
+repeat_each(1);
+no_long_string();
+no_shuffle();
+no_root_location();
+
+
+add_block_preprocessor(sub {
+ my ($block) = @_;
+
+ if (!defined $block->request) {
+ $block->set_value("request", "GET /t");
+ }
+
+ my $main_config = $block->main_config // <<_EOC_;
+ env AWS_EC2_METADATA_DISABLED=true;
+_EOC_
+ $block->set_value("main_config", $main_config);
+
+ my $user_yaml_config = <<_EOC_;
+plugins:
+ - ai-proxy-multi
+ - prometheus
+_EOC_
+ $block->set_value("extra_yaml_config", $user_yaml_config);
+
+ my $http_config = $block->http_config // <<_EOC_;
+ server {
+ server_name bedrock;
+ listen 6724;
+
+ default_type 'application/json';
+
+ location ~ ^/model/.+/converse\$ {
+ content_by_lua_block {
+ local json = require("cjson.safe")
+
+ -- Log the raw request URI so tests can assert the exact
+ -- path shape (e.g., that ARN model IDs are URL-encoded
+ -- as a single path segment).
+ ngx.log(ngx.WARN, "[test] received uri: ", ngx.var.request_uri)
+
+ if ngx.req.get_method() ~= "POST" then
+ ngx.status = 400
+ ngx.say("Unsupported request method: ", ngx.req.get_method())
+ return
+ end
+
+ -- Check SigV4 auth headers
+ local auth_header = ngx.req.get_headers()["authorization"]
+ local amz_date = ngx.req.get_headers()["x-amz-date"]
+ if not auth_header or not amz_date then
+ ngx.status = 403
+ ngx.say(json.encode({
+ message = "Missing Authentication Token"
+ }))
+ return
+ end
+
+ -- Validate Authorization header structure (SigV4):
+ -- AWS4-HMAC-SHA256 Credential=<AK>/<DATE>/<REGION>/<SERVICE>/aws4_request,
+ -- SignedHeaders=<list>, Signature=<64 hex>
+ if not auth_header:match("^AWS4%-HMAC%-SHA256 ") then
+ ngx.status = 403
+ ngx.say(json.encode({
+ message = "Authorization header missing AWS4-HMAC-SHA256 algorithm prefix"
+ }))
+ return
+ end
+
+ if not auth_header:match("Credential=[^,]+") then
+ ngx.status = 403
+ ngx.say(json.encode({
+ message = "Authorization header missing Credential component"
+ }))
+ return
+ end
+
+ -- Strict credential scope: access_key/<date>/us-east-1/bedrock/aws4_request
+ if not auth_header:match(
+ "Credential=AKIAIOSFODNN7EXAMPLE/%d%d%d%d%d%d%d%d/us%-east%-1/bedrock/aws4_request"
+ ) then
+ ngx.status = 403
+ ngx.say(json.encode({
+ message = "Authorization Credential scope does not match expected "
+ .. "AKIAIOSFODNN7EXAMPLE/<DATE>/us-east-1/bedrock/aws4_request"
+ }))
+ return
+ end
+
+ if not auth_header:match("SignedHeaders=[^,]+") then
+ ngx.status = 403
+ ngx.say(json.encode({
+ message = "Authorization header missing SignedHeaders component"
+ }))
+ return
+ end
+
+ -- Lua patterns don't support {n} quantifiers, so match
+ -- exactly 64 hex chars by repeating %x sixty-four times.
+ local hex64 = string.rep("%x", 64)
+ if not auth_header:match("Signature=" .. hex64) then
+ ngx.status = 403
+ ngx.say(json.encode({
+ message = "Authorization Signature is missing or not 64 hex chars"
+ }))
+ return
+ end
+
+ -- Validate X-Amz-Date format: YYYYMMDDTHHMMSSZ
+ if not amz_date:match("^%d%d%d%d%d%d%d%dT%d%d%d%d%d%dZ\$") then
+ ngx.status = 403
+ ngx.say(json.encode({
+ message = "X-Amz-Date header does not match YYYYMMDDTHHMMSSZ format"
+ }))
+ return
+ end
+
+ -- Capture session token if provided so tests can assert
+ -- that auth.aws.session_token was propagated as
+ -- x-amz-security-token.
+ local session_token = ngx.req.get_headers()["x-amz-security-token"]
+
+ ngx.req.read_body()
+ local body_data = ngx.req.get_body_data()
+ local body, err = json.decode(body_data)
+
+ if not body then
+ ngx.status = 400
+ ngx.say(json.encode({ message = "Invalid JSON: " .. (err or "") }))
+ return
+ end
+
+ -- Verify model is NOT in the body (remove_model = true)
+ if body.model then
+ ngx.status = 400
+ ngx.say(json.encode({
+ message = "model field should not be in request body"
+ }))
+ return
+ end
+
+ -- Verify request has messages
+ if not body.messages or #body.messages < 1 then
+ ngx.status = 400
+ ngx.say(json.encode({ message = "messages is required" }))
+ return
+ end
+
+ -- Return Bedrock Converse response
+ local assistant_text = "1 + 1 = 2."
+ if session_token then
+ assistant_text = assistant_text
+ .. " session_token_seen=" .. session_token
+ end
+ ngx.status = 200
+ ngx.say(json.encode({
+ output = {
+ message = {
+ role = "assistant",
+ content = {{text = assistant_text}}
+ }
+ },
+ stopReason = "end_turn",
+ usage = {
+ inputTokens = 10,
+ outputTokens = 8,
+ totalTokens = 18
+ }
+ }))
+ }
+ }
+ }
+_EOC_
+
+ $block->set_value("http_config", $http_config);
+});
+
+run_tests();
+
+__DATA__
+
+=== TEST 1: set route with bedrock provider
+--- config
+ location /t {
+ content_by_lua_block {
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/routes/1',
+ ngx.HTTP_PUT,
+ [[{
+ "uri": "/ai/converse",
+ "plugins": {
+ "ai-proxy-multi": {
+ "instances": [
+ {
+ "name": "bedrock",
+ "provider": "bedrock",
Review Comment:
This test suite exercises Bedrock via `ai-proxy-multi` only. Since the PR claims Bedrock support for both `ai-proxy` and `ai-proxy-multi`, it would be good to add at least one analogous happy-path + schema-validation case for the single-instance `ai-proxy` plugin (especially around required `provider_conf.region` and `auth.aws`).
##########
docs/zh/latest/plugins/ai-proxy-multi.md:
##########
@@ -58,10 +70,11 @@ description: ai-proxy-multi 插件通过负载均衡、重试、故障转移和
| balancer.key | string | 否 | | | 当 `type` 为 `chash` 时使用。当 `hash_on` 设置为 `header` 或 `cookie` 时，需要 `key`。当 `hash_on` 设置为 `consumer` 时，不需要 `key`，因为消费者名称将自动用作键。 |
| instances | array[object] | 是 | | | LLM 实例配置。 |
| instances.name | string | 是 | | | LLM 服务实例的名称。 |
-| instances.provider | string | 是 | | [openai, deepseek, azure-openai, aimlapi, anthropic, openrouter, gemini, vertex-ai, openai-compatible] | LLM 服务提供商。设置为 `openai` 时，插件将代理请求到 `api.openai.com`。设置为 `deepseek` 时，插件将代理请求到 `api.deepseek.com`。设置为 `aimlapi` 时，插件使用 OpenAI 兼容驱动程序，默认将请求代理到 `api.aimlapi.com`。设置为 `anthropic` 时，插件使用 OpenAI 兼容驱动程序，默认将请求代理到 `api.anthropic.com`。设置为 `openrouter` 时，插件使用 OpenAI 兼容驱动程序，默认将请求代理到 `openrouter.ai`。设置为 `gemini` 时，插件使用 OpenAI 兼容驱动程序，默认将请求代理到 `generativelanguage.googleapis.com`。设置为 `vertex-ai` 时，插件默认将请求代理到 `aiplatform.googleapis.com`，且需要配置 `provider_conf` 或 `override`。设置为 `openai-
compatible` 时，插件将代理请求到在 `override` 中配置的自定义端点。 |
-| instances.provider_conf | object | 否 | | | 特定提供商的配置。当 `provider` 设置为 `vertex-ai` 且未配置 `override` 时必填。 |
+| instances.provider | string | 是 | | [openai, deepseek, azure-openai, aimlapi, anthropic, openrouter, gemini, vertex-ai, bedrock, openai-compatible] | LLM 服务提供商。设置为 `openai` 时，插件将代理请求到 `api.openai.com`。设置为 `deepseek` 时，插件将代理请求到 `api.deepseek.com`。设置为 `aimlapi` 时，插件使用 OpenAI 兼容驱动程序，默认将请求代理到 `api.aimlapi.com`。设置为 `anthropic` 时，插件使用 OpenAI 兼容驱动程序，默认将请求代理到 `api.anthropic.com`。设置为 `openrouter` 时，插件使用 OpenAI 兼容驱动程序，默认将请求代理到 `openrouter.ai`。设置为 `gemini` 时，插件使用 OpenAI 兼容驱动程序，默认将请求代理到 `generativelanguage.googleapis.com`。设置为 `vertex-ai` 时，插件默认将请求代理到 `aiplatform.googleapis.com`，且需要配置 `provider_conf` 或 `override`。设置为
`bedrock` 时，插件将代理请求到 AWS Bedrock Converse API（`bedrock-runtime.<region>.amazonaws.com`），并使用 AWS SigV4 对请求进行签名。设置为 `openai-compatible` 时，插件将代理请求到在 `override` 中配置的自定义端点。 |
+| instances.provider_conf | object | 否 | | | 特定提供商的配置。当 `provider` 设置为 `vertex-ai` 且未配置 `override` 时必填。当 `provider` 设置为 `bedrock` 时必填。 |
| instances.provider_conf.project_id | string | 是 | | | Google Cloud 项目 ID。 |
| instances.provider_conf.region | string | 是 | | | Google Cloud 区域。 |
+| instances.provider_conf.region | string | 是 | | minLength = 1 | 用于构造 Bedrock 端点并使用 SigV4 对请求进行签名的 AWS 区域。当 `provider` 设置为 `bedrock` 时必填。 |
Review Comment:
属性表中 `instances.provider_conf.region` 现在出现了两次（一次表示 Google Cloud Region，一次表示 AWS Region）。同一表格内重复键会导致含义不清晰。建议合并为同一行并在描述中注明不同 provider 的语义/必选条件，或将 provider-specific 配置拆分到单独小节。
```suggestion
| instances.provider_conf.region | string | 是 | | minLength = 1 | 区域配置：当 `provider` 设置为 `vertex-ai` 时表示 Google Cloud 区域；当 `provider` 设置为 `bedrock` 时表示 AWS 区域，用于构造 Bedrock 端点并使用 SigV4 对请求进行签名，且该场景下必填。 |
```
##########
docs/zh/latest/plugins/ai-proxy.md:
##########
@@ -47,25 +47,42 @@ description: ai-proxy 插件通过将插件配置转换为所需的请求格式
| `messages.role` | String | 是 | 消息的角色（`system`、`user`、`assistant`）。|
| `messages.content` | String | 是 | 消息的内容。 |
+### Bedrock Converse 请求格式
+
+当 `provider` 设置为 `bedrock` 时，插件期望请求采用 [Bedrock Converse API](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html) 格式。请求 URI 必须以 `/converse` 结尾，且请求体必须包含 `messages` 数组。
+
+| 名称 | 类型 | 必选项 | 描述 |
+| ------------------ | ------ | -------- | ---------------------------------------------------------------------------------------------------- |
+| `messages` | Array | 是 | 消息对象数组。 |
+| `messages.role` | String | 是 | 消息的角色（`user`、`assistant`）。 |
+| `messages.content` | Array | 是 | 内容块数组。每个块包含一个 `text` 字段（例如 `[{"text": "What is 1+1?"}]`）。 |
+| `system` | Array | 否 | 可选的系统提示块（例如 `[{"text": "You are a helpful assistant."}]`）。 |
+| `inferenceConfig` | Object | 否 | 可选的推理参数，如 `maxTokens`、`temperature`、`topP` 等。 |
+
## 属性
| 名称 | 类型 | 必选项 | 默认值 | 有效值 | 描述 |
|--------------------|--------|----------|---------|------------------------------------------|-------------|
-| provider | string | 是 | | [openai, deepseek, azure-openai, aimlapi, anthropic, openrouter, gemini, vertex-ai, openai-compatible] | LLM 服务提供商。当设置为 `openai` 时，插件将代理请求到 `https://api.openai.com/chat/completions`。当设置为 `deepseek` 时，插件将代理请求到 `https://api.deepseek.com/chat/completions`。当设置为 `aimlapi` 时，插件使用 OpenAI 兼容驱动程序，默认将请求代理到 `https://api.aimlapi.com/v1/chat/completions`。当设置为 `anthropic` 时，插件将代理请求到 `https://api.anthropic.com/v1/chat/completions`。当设置为 `openrouter` 时，插件使用 OpenAI 兼容驱动程序，默认将请求代理到 `https://openrouter.ai/api/v1/chat/completions`。当设置为 `gemini` 时，插件使用 OpenAI 兼容驱动程序，默认将请求代理到 `https://generativelanguage.googleapis.com/v1beta/openai/chat/completions`。当设置为 `vertex-ai` 时，插件默认将请求
代理到 `https://aiplatform.googleapis.com`，且需要配置 `provider_conf` 或 `override`。当设置为 `openai-compatible` 时，插件将代理请求到在 `override` 中配置的自定义端点。 |
-| provider_conf | object | 否 | | | 特定提供商的配置。当 `provider` 设置为 `vertex-ai` 且未配置 `override` 时必填。 |
+| provider | string | 是 | | [openai, deepseek, azure-openai, aimlapi, anthropic, openrouter, gemini, vertex-ai, bedrock, openai-compatible] | LLM 服务提供商。当设置为 `openai` 时，插件将代理请求到 `https://api.openai.com/chat/completions`。当设置为 `deepseek` 时，插件将代理请求到 `https://api.deepseek.com/chat/completions`。当设置为 `aimlapi` 时，插件使用 OpenAI 兼容驱动程序，默认将请求代理到 `https://api.aimlapi.com/v1/chat/completions`。当设置为 `anthropic` 时，插件将代理请求到 `https://api.anthropic.com/v1/chat/completions`。当设置为 `openrouter` 时，插件使用 OpenAI 兼容驱动程序，默认将请求代理到 `https://openrouter.ai/api/v1/chat/completions`。当设置为 `gemini` 时，插件使用 OpenAI 兼容驱动程序，默认将请求代理到 `https://generativelanguage.googleapis.com/v1beta/openai/chat/completions`。当设置为 `vertex-ai` 时，插件默认
将请求代理到 `https://aiplatform.googleapis.com`，且需要配置 `provider_conf` 或 `override`。当设置为 `bedrock` 时，插件将代理请求到 AWS Bedrock Converse API（`https://bedrock-runtime.<region>.amazonaws.com`），并使用 AWS SigV4 对请求进行签名。当设置为 `openai-compatible` 时，插件将代理请求到在 `override` 中配置的自定义端点。 |
+| provider_conf | object | 否 | | | 特定提供商的配置。当 `provider` 设置为 `vertex-ai` 且未配置 `override` 时必填。当 `provider` 设置为 `bedrock` 时必填。 |
| provider_conf.project_id | string | 是 | | | Google Cloud 项目 ID。 |
| provider_conf.region | string | 是 | | | Google Cloud 区域。 |
+| provider_conf.region | string | 是 | | minLength = 1 | 用于构造 Bedrock 端点并使用 SigV4 对请求进行签名的 AWS 区域。当 `provider` 设置为 `bedrock` 时必填。 |
Review Comment:
属性表中 `provider_conf.region` 现在出现了两次（一次表示 Google Cloud Region，一次表示 AWS Region）。这会让读者难以区分该字段在不同 provider 下的含义/必选条件。建议合并为同一行并在描述中注明不同 provider 的语义与要求，或将 provider-specific 配置拆分到单独小节，避免重复键。
```suggestion
| provider_conf.region | string | 是 | | minLength = 1 | 区域配置：当 `provider` 设置为 `vertex-ai` 时，表示 Google Cloud 区域；当 `provider` 设置为 `bedrock` 时，表示 AWS 区域，用于构造 Bedrock 端点并使用 SigV4 对请求进行签名。在 `bedrock` 场景下该字段必填且不能为空。 |
```
##########
docs/en/latest/plugins/ai-proxy-multi.md:
##########
@@ -7,7 +7,7 @@ keywords:
- ai-proxy-multi
- AI
- LLM
-description: The ai-proxy-multi Plugin extends the capabilities of ai-proxy with load balancing, retries, fallbacks, and health chekcs, simplifying the integration with OpenAI, DeepSeek, Azure, AIMLAPI, Anthropic, OpenRouter, Gemini, Vertex AI, and other OpenAI-compatible APIs.
+description: The ai-proxy-multi Plugin extends the capabilities of ai-proxy with load balancing, retries, fallbacks, and health chekcs, simplifying the integration with OpenAI, DeepSeek, Azure, AIMLAPI, Anthropic, OpenRouter, Gemini, Vertex AI, Amazon Bedrock, and other OpenAI-compatible APIs.
Review Comment:
Typo in the front-matter description: “health chekcs” should be “health checks”.
```suggestion
description: The ai-proxy-multi Plugin extends the capabilities of ai-proxy with load balancing, retries, fallbacks, and health checks, simplifying the integration with OpenAI, DeepSeek, Azure, AIMLAPI, Anthropic, OpenRouter, Gemini, Vertex AI, Amazon Bedrock, and other OpenAI-compatible APIs.
```
##########
apisix/plugins/ai-proxy/schema.lua:
##########
@@ -200,10 +264,14 @@ _M.ai_proxy_schema = {
description = "To be specified to override the endpoint of the AI Instance",
},
},
+ required = { "endpoint" },
},
},
required = {"provider", "auth"},
- encrypt_fields = {"auth.header", "auth.query", "auth.gcp.service_account_json"},
+ encrypt_fields = {
+ "auth.header", "auth.query", "auth.gcp.service_account_json",
+ "auth.aws.secret_access_key", "auth.aws.session_token",
+ },
Review Comment:
For the single-instance `ai-proxy` schema, Bedrock-specific requirements described in this PR (e.g., `provider_conf.region` and `auth.aws` being required for `provider=bedrock`) are not enforced here—only the encrypt_fields list was updated. This allows misconfigured Bedrock routes to pass schema validation and fail later at runtime during SigV4 signing/path construction. Please add Bedrock-specific conditional requirements to `ai_proxy_schema` (similar to the `instances` schema in this file).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ai-proxy): support aws bedrock [apisix]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #13249:
URL: https://github.com/apache/apisix/pull/13249#discussion_r3135476583
##########
apisix/plugins/ai-proxy/schema.lua:
##########
@@ -348,7 +359,39 @@ _M.ai_proxy_multi_schema = {
"instances.auth.header",
"instances.auth.query",
"instances.auth.gcp.service_account_json",
+ "instances.auth.aws.secret_access_key",
+ "instances.auth.aws.session_token",
},
}
+function _M.validate_provider_requirements(conf)
+ local provider = conf.provider
+ local has_override = conf.override and conf.override.endpoint
+
+ if provider == "vertex-ai" then
+ local pc = conf.provider_conf
+ local has_provider_conf = pc and pc.project_id and pc.region
+ if not has_provider_conf and not has_override then
+ return false, "vertex-ai requires either provider_conf "
+ .. "(project_id + region) or override.endpoint"
+ end
+ end
+
+ if provider == "bedrock" then
+ if not (conf.provider_conf and conf.provider_conf.region) then
+ return false, "bedrock requires provider_conf.region"
+ end
+ if not (conf.auth and conf.auth.aws) then
+ return false, "bedrock requires auth.aws"
+ end
+ -- options.model is intentionally NOT required: clients may pass `model`
+ -- in the request body for per-request model selection. The bedrock
+ -- provider falls back to options.model only when body.model is absent.
+ -- If neither is set at request time, build_request returns 400.
Review Comment:
The Bedrock-specific comment here says the provider “falls back to options.model only when body.model is absent”, but the current model selection logic in apisix/plugins/ai-proxy/base.lua sets `llm_model` to `options.model` whenever it’s configured and only uses `body.model` when `options.model` is missing. Please either update this comment to match the actual precedence (route config wins), or change the precedence if per-request model override is intended.
```suggestion
-- in the request body when no route-level model is configured. If
-- options.model is set, it takes precedence; body.model is only used
-- when options.model is absent. If neither is set at request time,
-- build_request returns 400.
```
##########
apisix/plugins/ai-proxy/schema.lua:
##########
@@ -116,7 +129,16 @@ local override_schema = {
properties = {
endpoint = {
type = "string",
- description = "To be specified to override the endpoint of the AI Instance",
+ description = "Override the endpoint of the AI Instance. "
+ .. "Typically used for custom hosts (e.g., AWS "
+ .. "PrivateLink, reverse proxies). You may provide "
+ .. "only the scheme + host, in which case the plugin "
+ .. "computes the provider-specific path, or provide "
+ .. "a full endpoint including path and query, in "
+ .. "which case the plugin uses the supplied path/query. "
+ .. "If your custom path or query contains reserved "
+ .. "characters (e.g., Bedrock inference profile ARNs "
+ .. "containing ':' or '/'), they must be URL-encoded.",
},
Review Comment:
In validate_provider_requirements(), `has_override` treats any non-nil `override.endpoint` as satisfying provider requirements. In Lua, an empty string is truthy, so `override.endpoint: ""` would pass validation even though it can’t be parsed into a usable URL at runtime. Consider adding `minLength = 1` to `override.endpoint` in the schema (and/or checking `endpoint ~= ""` when computing `has_override`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ai-proxy): support aws bedrock [apisix]
Posted by "elsodiop (via GitHub)" <gi...@apache.org>.
elsodiop commented on PR #13249:
URL: https://github.com/apache/apisix/pull/13249#issuecomment-5067917694
Thanks @shreemaan-abhishek for your feedback.
I resolve my problem as you mention with iam user credentials.
Am using the APISIX 3.17 version.
Best regards
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ai-proxy): support aws bedrock [apisix]
Posted by "shreemaan-abhishek (via GitHub)" <gi...@apache.org>.
shreemaan-abhishek commented on PR #13249:
URL: https://github.com/apache/apisix/pull/13249#issuecomment-5060994453
Hello @elsodiop
The bedrock provider doesn't do its own secret resolution, and neither does any other plugin: since #13312 the whole plugin conf is resolved centrally in plugin.filter (apisix/plugin.lua:97) before any plugin handler runs, so auth.aws is already plain text when it reaches the SigV4 signer.
What your error shows is that the reference failed to resolve: secret.lua falls back to the literal string when a fetch fails (refs[k] = fetch(v, use_cache) or v), so the raw URI ends up in the credential scope. Could you confirm
- your APISIX version
- and that the /apisix/secrets/aws/<id> resource exists and its credentials can read that secret?
Separately, this did surface a real problem: when resolution fails on the cached path there is no error log at all, so the misconfiguration is invisible. I'll open a fix for that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ai-proxy): support aws bedrock [apisix]
Posted by "nic-6443 (via GitHub)" <gi...@apache.org>.
nic-6443 commented on code in PR #13249:
URL: https://github.com/apache/apisix/pull/13249#discussion_r3114883826
##########
apisix/plugins/ai-proxy/schema.lua:
##########
@@ -234,7 +308,58 @@ _M.ai_proxy_schema = {
override = override_schema,
},
required = {"provider", "auth"},
- encrypt_fields = {"auth.header", "auth.query", "auth.gcp.service_account_json"},
+ allOf = {
+ {
+ ["if"] = {
+ properties = { provider = { enum = { "vertex-ai" } } },
+ },
+ ["then"] = {
+ properties = {
+ provider_conf = provider_vertex_ai_schema,
+ },
+ anyOf = {
+ { required = { "provider_conf" } },
+ { required = { "override" } },
+ },
+ },
+ },
+ {
+ ["if"] = {
+ properties = { provider = { enum = { "bedrock" } } },
+ },
+ ["then"] = {
+ properties = {
+ provider_conf = provider_bedrock_schema,
+ auth = {
+ required = { "aws" },
+ },
+ },
+ required = { "provider_conf", "auth" },
+ },
+ },
+ {
+ ["if"] = {
+ properties = { provider = { enum = { "bedrock" } } },
+ required = { "provider" },
+ ["not"] = {
+ required = { "override" },
+ properties = {
+ override = { required = { "endpoint" } },
+ },
+ },
+ },
+ ["then"] = {
+ properties = {
+ options = { required = { "model" } },
+ },
+ required = { "options" },
+ },
+ },
+ },
Review Comment:
I think the logic of this if-else block is overly complicated. It would be sufficient to throw an error directly when a request is made if the necessary auth configuration is missing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ai-proxy): support aws bedrock [apisix]
Posted by "nic-6443 (via GitHub)" <gi...@apache.org>.
nic-6443 commented on code in PR #13249:
URL: https://github.com/apache/apisix/pull/13249#discussion_r3128813902
##########
apisix/plugins/ai-proxy/schema.lua:
##########
@@ -348,7 +359,42 @@ _M.ai_proxy_multi_schema = {
"instances.auth.header",
"instances.auth.query",
"instances.auth.gcp.service_account_json",
+ "instances.auth.aws.secret_access_key",
+ "instances.auth.aws.session_token",
},
}
+function _M.validate_provider_requirements(conf)
+ local provider = conf.provider
+ local has_override = conf.override and conf.override.endpoint
+
+ if provider == "vertex-ai" then
+ local pc = conf.provider_conf
+ local has_provider_conf = pc and pc.project_id and pc.region
+ if not has_provider_conf and not has_override then
+ return false, "vertex-ai requires either provider_conf "
+ .. "(project_id + region) or override.endpoint"
+ end
+ end
+
+ if provider == "bedrock" then
+ if not (conf.provider_conf and conf.provider_conf.region) then
+ return false, "bedrock requires provider_conf.region"
+ end
+ if not (conf.auth and conf.auth.aws) then
+ return false, "bedrock requires auth.aws"
+ end
+ if not has_override then
+ local has_model = conf.options and conf.options.model
+ if not has_model then
+ return false, "bedrock requires options.model when "
+ .. "override.endpoint is not set"
+ end
+ end
Review Comment:
In this case, users need to configure a separate plugin for each model supported by bedrock, which feels extremely inconvenient to use. We need to investigate how other AI gateways handle this situation. For example, downstream requests should be allowed to specify the model in a certain way.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ai-proxy): support aws bedrock [apisix]
Posted by "shreemaan-abhishek (via GitHub)" <gi...@apache.org>.
shreemaan-abhishek commented on code in PR #13249:
URL: https://github.com/apache/apisix/pull/13249#discussion_r3135449698
##########
apisix/plugins/ai-proxy/schema.lua:
##########
@@ -348,7 +359,42 @@ _M.ai_proxy_multi_schema = {
"instances.auth.header",
"instances.auth.query",
"instances.auth.gcp.service_account_json",
+ "instances.auth.aws.secret_access_key",
+ "instances.auth.aws.session_token",
},
}
+function _M.validate_provider_requirements(conf)
+ local provider = conf.provider
+ local has_override = conf.override and conf.override.endpoint
+
+ if provider == "vertex-ai" then
+ local pc = conf.provider_conf
+ local has_provider_conf = pc and pc.project_id and pc.region
+ if not has_provider_conf and not has_override then
+ return false, "vertex-ai requires either provider_conf "
+ .. "(project_id + region) or override.endpoint"
+ end
+ end
+
+ if provider == "bedrock" then
+ if not (conf.provider_conf and conf.provider_conf.region) then
+ return false, "bedrock requires provider_conf.region"
+ end
+ if not (conf.auth and conf.auth.aws) then
+ return false, "bedrock requires auth.aws"
+ end
+ if not has_override then
+ local has_model = conf.options and conf.options.model
+ if not has_model then
+ return false, "bedrock requires options.model when "
+ .. "override.endpoint is not set"
+ end
+ end
Review Comment:
we can specify the model in request body, I have added the logic for this now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ai-proxy): support aws bedrock [apisix]
Posted by "shreemaan-abhishek (via GitHub)" <gi...@apache.org>.
shreemaan-abhishek commented on code in PR #13249:
URL: https://github.com/apache/apisix/pull/13249#discussion_r3106364390
##########
apisix/plugins/ai-protocols/bedrock-converse.lua:
##########
@@ -0,0 +1,283 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Bedrock Converse protocol adapter (client-side).
+-- Handles detection and response parsing for the Amazon Bedrock
+-- Converse API format. Non-streaming only in this phase.
+
+local core = require("apisix.core")
+local string_sub = string.sub
+local type = type
+local ipairs = ipairs
+local table = table
+
+local _M = {}
+
+
+--- Detect whether the request matches the Bedrock Converse API format.
+-- Uses URI suffix (/converse) and body (valid JSON table with messages).
+function _M.matches(body, ctx)
+ local uri = ctx.var and ctx.var.uri
+ return uri and string_sub(uri, -9) == "/converse"
+ and type(body) == "table" and type(body.messages) == "table"
Review Comment:
A user sending an image-only or tool-only message (valid Bedrock requests) would have content blocks that don't
contain text, and the tightened matches() would return false. Their request would then fall through to
openai-chat, producing the same "provider does not support openai-chat" error this change is trying to avoid —
except now it'd hit valid users instead of misconfigured ones.
The mis-detection scenario this addresses requires a user to (1) explicitly route to a /converse URI on the
bedrock provider, and (2) send a body using OpenAI's string-content format. That's a user error, and the current
error message ("provider bedrock does not support openai-chat protocol") clearly tells them their body doesn't
match Bedrock format.
If we wanted to tighten without breaking multimodal/tool use, the safer check would be "content is an array of
objects" (distinguishes from OpenAI's string content). But since /converse URI is an explicit opt-in for Bedrock,
I think the current behavior is appropriate.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ai-proxy): support aws bedrock [apisix]
Posted by "shreemaan-abhishek (via GitHub)" <gi...@apache.org>.
shreemaan-abhishek commented on code in PR #13249:
URL: https://github.com/apache/apisix/pull/13249#discussion_r3108856044
##########
apisix/plugins/ai-providers/base.lua:
##########
@@ -187,6 +187,20 @@ function _M.build_request(self, ctx, conf, request_body, opts)
request_body.model = nil
end
+ -- AWS SigV4 signing (must be last — signs the finalized body)
+ if auth.aws then
+ local auth_aws = require("apisix.plugins.ai-transport.auth-aws")
+ local region = opts.conf and opts.conf.region
+ if not region then
+ return nil, "missing region for AWS SigV4 signing "
+ .. "(provider_conf.region required for bedrock)"
+ end
+ local sign_err = auth_aws.sign_request(params, auth.aws, region)
+ if sign_err then
+ return nil, "failed to sign AWS request: " .. sign_err
+ end
+ end
Review Comment:
already done:
<img width="2070" height="860" alt="image" src="https://github.com/user-attachments/assets/ce18cb6c-0152-4822-9c38-b81ecd6b3fb1" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ai-proxy): support aws bedrock [apisix]
Posted by "nic-6443 (via GitHub)" <gi...@apache.org>.
nic-6443 commented on code in PR #13249:
URL: https://github.com/apache/apisix/pull/13249#discussion_r3128823904
##########
t/fixtures/bedrock/converse-basic.json:
##########
@@ -0,0 +1,14 @@
+{
+ "output": {
+ "message": {
+ "role": "assistant",
+ "content": [{"text": "1 + 1 = 2."}]
+ }
+ },
+ "stopReason": "end_turn",
+ "usage": {
+ "inputTokens": 10,
+ "outputTokens": 8,
+ "totalTokens": 18
+ }
+}
Review Comment:
From now on, we will no longer create fake data for testing; instead, we will directly record real responses from the Bedrock API.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org