You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by "Revolyssup (via GitHub)" <gi...@apache.org> on 2025/07/09 07:02:15 UTC
[PR] fix(consumer): missed consumer update due to wrong version in cache [apisix]
Revolyssup opened a new pull request, #12413:
URL: https://github.com/apache/apisix/pull/12413
### Description
This PR introduced a critical bug https://github.com/apache/apisix/pull/11840 due to which consumers are not updated until a subsequent update in credentials due to credential version being used even for checking consumer update.
More context on this comment: https://github.com/apache/apisix/pull/11840/files#r2193898420
### Checklist
- [ ] I have explained the need for this PR and the problem it solves
- [ ] I have explained the changes or the new features added to this PR
- [ ] I have added tests corresponding to this change
- [ ] I have updated the documentation to reflect this change
- [ ] I have verified that this change is backward compatible (If not, please discuss on the [APISIX mailing list](https://github.com/apache/apisix/tree/master#community) first)
<!--
Note
1. Mark the PR as draft until it's ready to be reviewed.
2. Always add/update tests for any changes unless you have a good reason.
3. Always update the documentation to reflect the changes made in the PR.
4. Make a new commit to resolve conversations instead of `push -f`.
5. To resolve merge conflicts, merge master instead of rebasing.
6. Use "request review" to notify the reviewer after making changes.
7. Only a reviewer can mark a conversation as resolved.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(consumer): missed consumer update due to wrong version in cache [apisix]
Posted by "bzp2010 (via GitHub)" <gi...@apache.org>.
bzp2010 commented on PR #12413:
URL: https://github.com/apache/apisix/pull/12413#issuecomment-3052257666
Truth be told, I didn't understand your modifications. Can you provide more insight? 🤔
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(consumer): missed consumer update due to wrong version in cache [apisix]
Posted by "bzp2010 (via GitHub)" <gi...@apache.org>.
bzp2010 commented on PR #12413:
URL: https://github.com/apache/apisix/pull/12413#issuecomment-3052300912
Thanks for the explanation, I seem to have run into some GitHub bug that prevents the web page from pinpointing the exact comment you pointed out.
Anyway, I've seen it. I will check the code again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(consumer): missed consumer update due to wrong version in cache [apisix]
Posted by "membphis (via GitHub)" <gi...@apache.org>.
membphis commented on code in PR #12413:
URL: https://github.com/apache/apisix/pull/12413#discussion_r2196337693
##########
apisix/consumer.lua:
##########
@@ -159,17 +120,50 @@ function plugin_consumer()
conf_version = consumers.conf_version
}
end
-
- local consumer = consumers_id_lrucache(val.value.id .. name,
- val.modifiedIndex, construct_consumer_data, val, config)
- if consumer == nil then
- goto CONTINUE
+ -- if the val is a Consumer, clone it to the local consumer;
+ -- if the val is a Credential, to get the Consumer by consumer_name and then clone
+ -- it to the local consumer.
+ local consumer
Review Comment:
I don't prefer current code style, it not easy to read
We can put those changing code to one or two seperate functions, and check the return value if it failed
Reducing the nesting depth can make the code more readable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(consumer): missed consumer update due to wrong version in cache [apisix]
Posted by "nic-6443 (via GitHub)" <gi...@apache.org>.
nic-6443 commented on code in PR #12413:
URL: https://github.com/apache/apisix/pull/12413#discussion_r2194348033
##########
apisix/consumer.lua:
##########
@@ -159,17 +120,49 @@ function plugin_consumer()
conf_version = consumers.conf_version
}
end
-
- local consumer = consumers_id_lrucache(val.value.id .. name,
- val.modifiedIndex, construct_consumer_data, val, config)
- if consumer == nil then
- goto CONTINUE
+ -- if the val is a Consumer, clone it to the local consumer;
+ -- if the val is a Credential, to get the Consumer by consumer_name and then clone
+ -- it to the local consumer.
+ local consumer
+ if is_credential_etcd_key(val.key) then
+ local consumer_name = get_consumer_name_from_credential_etcd_key(val.key)
+ local the_consumer = consumers:get(consumer_name)
+ if the_consumer and the_consumer.value then
+ consumer = consumers_id_lrucache(val.value.id .. name, val.modifiedIndex..the_consumer.modifiedIndex,
Review Comment:
why not use `val.key` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(consumer): missed consumer update due to wrong version in cache [apisix]
Posted by "nic-6443 (via GitHub)" <gi...@apache.org>.
nic-6443 commented on code in PR #12413:
URL: https://github.com/apache/apisix/pull/12413#discussion_r2194335193
##########
apisix/consumer.lua:
##########
@@ -159,17 +120,49 @@ function plugin_consumer()
conf_version = consumers.conf_version
}
end
-
- local consumer = consumers_id_lrucache(val.value.id .. name,
- val.modifiedIndex, construct_consumer_data, val, config)
- if consumer == nil then
- goto CONTINUE
+ -- if the val is a Consumer, clone it to the local consumer;
+ -- if the val is a Credential, to get the Consumer by consumer_name and then clone
+ -- it to the local consumer.
+ local consumer
+ if is_credential_etcd_key(val.key) then
+ local consumer_name = get_consumer_name_from_credential_etcd_key(val.key)
+ local the_consumer = consumers:get(consumer_name)
+ if the_consumer and the_consumer.value then
+ consumer = consumers_id_lrucache(the_consumer.value, val.modifiedIndex+the_consumer.modifiedIndex,
Review Comment:
we should concatenate those `modifiedIndex` in string way.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(consumer): missed consumer update due to wrong version in cache [apisix]
Posted by "Baoyuantop (via GitHub)" <gi...@apache.org>.
Baoyuantop commented on PR #12413:
URL: https://github.com/apache/apisix/pull/12413#issuecomment-3051644867
Will this pr fixed https://github.com/apache/apisix/issues/12393?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(consumer): missed consumer update due to wrong version in cache [apisix]
Posted by "nic-6443 (via GitHub)" <gi...@apache.org>.
nic-6443 commented on code in PR #12413:
URL: https://github.com/apache/apisix/pull/12413#discussion_r2194344762
##########
t/admin/consumer-credentials.t:
##########
@@ -0,0 +1,165 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+use t::APISIX 'no_plan';
+
+repeat_each(1);
+no_long_string();
+no_root_location();
+no_shuffle();
+log_level("info");
+
+run_tests;
+
+__DATA__
+
+=== TEST 1: Verify consumer plugin update takes effect immediately
+--- extra_yaml_config
+nginx_config:
+ worker_processes: 1
+--- config
+ location /t {
+ content_by_lua_block {
+ local t = require("lib.test_admin").test
+ local json = require("toolkit.json")
+ local http = require("resty.http")
+
+ -- 1. Create route with key-auth plugin
+ local code, body = t('/apisix/admin/routes/1',
+ ngx.HTTP_PUT,
+ [[{
+ "uri": "/anything",
+ "upstream": {
+ "type": "roundrobin",
+ "nodes": {
+ "httpbin.org:80": 1
+ }
+ },
+ "plugins": {
+ "key-auth": {
+ "query": "apikey"
+ }
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say("Route creation failed: " .. body)
+ return
+ end
+
+ -- 2. Create consumer jack
+ code, body = t('/apisix/admin/consumers',
+ ngx.HTTP_PUT,
+ [[{
+ "username": "jack"
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say("Consumer creation failed: " .. body)
+ return
+ end
+
+ -- 3. Create credentials for jack
+ code, body = t('/apisix/admin/consumers/jack/credentials/auth-one',
+ ngx.HTTP_PUT,
+ [[{
+ "plugins": {
+ "key-auth": {
+ "key": "auth-one"
+ }
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say("Credential creation failed: " .. body)
+ return
+ end
+ ngx.sleep(0.5) -- wait for etcd to sync
+ -- 4. Verify valid request succeeds
+ local httpc = http.new()
+ local res, err = httpc:request_uri(
+ "http://127.0.0.1:"..ngx.var.server_port.."/anything?apikey=auth-one",
+ { method = "GET" }
+ )
+ if not res then
+ ngx.say("Request failed: ", err)
+ return
+ end
+
+ if res.status ~= 200 then
+ ngx.say("Unexpected status: ", res.status)
+ ngx.say(res.body)
+ return
+ end
+
+ -- 5. Update consumer with fault-injection plugin
+ code, body = t('/apisix/admin/consumers/jack',
+ ngx.HTTP_PUT,
+ [[{
+ "username": "jack",
+ "plugins": {
+ "fault-injection": {
+ "abort": {
+ "http_status": 400,
+ "body": "abort"
+ }
+ }
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say("Consumer update failed: " .. body)
+ return
+ end
+
+ -- 6. Verify all requests return 400
+ for i = 1, 5 do
+ local res, err = httpc:request_uri(
+ "http://127.0.0.1:"..ngx.var.server_port.."/anything?apikey=auth-one",
+ { method = "GET" }
+ )
+ if not res then
+ ngx.say(i, ": Request failed: ", err)
+ return
+ end
+
+ if res.status ~= 400 then
+ ngx.say(i, ": Expected 400 but got ", res.status)
+ return
+ end
+
+ if res.body ~= "abort" then
+ ngx.say(i, ": Unexpected response body: ", res.body)
+ return
+ end
+ end
+
+ ngx.say("All requests aborted as expected")
+ }
+ }
+--- request
+GET /t
+--- response_body
+All requests aborted as expected
+--- error_log
Review Comment:
What does this assertion mean?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(consumer): missed consumer update due to wrong version in cache [apisix]
Posted by "Revolyssup (via GitHub)" <gi...@apache.org>.
Revolyssup commented on code in PR #12413:
URL: https://github.com/apache/apisix/pull/12413#discussion_r2194337246
##########
apisix/consumer.lua:
##########
@@ -159,17 +120,49 @@ function plugin_consumer()
conf_version = consumers.conf_version
}
end
-
- local consumer = consumers_id_lrucache(val.value.id .. name,
- val.modifiedIndex, construct_consumer_data, val, config)
- if consumer == nil then
- goto CONTINUE
+ -- if the val is a Consumer, clone it to the local consumer;
+ -- if the val is a Credential, to get the Consumer by consumer_name and then clone
+ -- it to the local consumer.
+ local consumer
+ if is_credential_etcd_key(val.key) then
+ local consumer_name = get_consumer_name_from_credential_etcd_key(val.key)
+ local the_consumer = consumers:get(consumer_name)
+ if the_consumer and the_consumer.value then
+ consumer = consumers_id_lrucache(the_consumer.value, val.modifiedIndex+the_consumer.modifiedIndex,
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(consumer): missed consumer update due to wrong version in cache [apisix]
Posted by "Revolyssup (via GitHub)" <gi...@apache.org>.
Revolyssup merged PR #12413:
URL: https://github.com/apache/apisix/pull/12413
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(consumer): missed consumer update due to wrong version in cache [apisix]
Posted by "bzp2010 (via GitHub)" <gi...@apache.org>.
bzp2010 commented on PR #12413:
URL: https://github.com/apache/apisix/pull/12413#issuecomment-3052256620
To tell you the truth, I didn't understand your modifications. Can you provide more insight? 🤔
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(consumer): missed consumer update due to wrong version in cache [apisix]
Posted by "Revolyssup (via GitHub)" <gi...@apache.org>.
Revolyssup commented on PR #12413:
URL: https://github.com/apache/apisix/pull/12413#issuecomment-3052267132
> * change is backward compatible (If not, please discuss on the [APISIX mailing list](https://github.com/apache/apisix/tree/master#community) first)
As this comment explains https://github.com/apache/apisix/pull/11840/files#r2193898420
The `version` being used in the cache was that of credential modified index. This means until the credential is modified, we will use stale value from cache.
But in the scenario, where the underlying consumer has been modified, we need to recalculate and not use the old value. Therefore in my changes, I have used `credential.modified..consumer.modified` as the version, so that even when underlying consumer changes, a resync is triggered.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(consumer): missed consumer update due to wrong version in cache [apisix]
Posted by "nic-6443 (via GitHub)" <gi...@apache.org>.
nic-6443 commented on code in PR #12413:
URL: https://github.com/apache/apisix/pull/12413#discussion_r2194339155
##########
apisix/consumer.lua:
##########
@@ -159,17 +120,49 @@ function plugin_consumer()
conf_version = consumers.conf_version
}
end
-
- local consumer = consumers_id_lrucache(val.value.id .. name,
- val.modifiedIndex, construct_consumer_data, val, config)
- if consumer == nil then
- goto CONTINUE
+ -- if the val is a Consumer, clone it to the local consumer;
+ -- if the val is a Credential, to get the Consumer by consumer_name and then clone
+ -- it to the local consumer.
+ local consumer
+ if is_credential_etcd_key(val.key) then
+ local consumer_name = get_consumer_name_from_credential_etcd_key(val.key)
+ local the_consumer = consumers:get(consumer_name)
+ if the_consumer and the_consumer.value then
+ consumer = consumers_id_lrucache(the_consumer.value, val.modifiedIndex+the_consumer.modifiedIndex,
Review Comment:
lru key should be `val.key`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(consumer): missed consumer update due to wrong version in cache [apisix]
Posted by "Revolyssup (via GitHub)" <gi...@apache.org>.
Revolyssup commented on code in PR #12413:
URL: https://github.com/apache/apisix/pull/12413#discussion_r2194348765
##########
apisix/consumer.lua:
##########
@@ -159,17 +120,49 @@ function plugin_consumer()
conf_version = consumers.conf_version
}
end
-
- local consumer = consumers_id_lrucache(val.value.id .. name,
- val.modifiedIndex, construct_consumer_data, val, config)
- if consumer == nil then
- goto CONTINUE
+ -- if the val is a Consumer, clone it to the local consumer;
+ -- if the val is a Credential, to get the Consumer by consumer_name and then clone
+ -- it to the local consumer.
+ local consumer
+ if is_credential_etcd_key(val.key) then
+ local consumer_name = get_consumer_name_from_credential_etcd_key(val.key)
+ local the_consumer = consumers:get(consumer_name)
+ if the_consumer and the_consumer.value then
+ consumer = consumers_id_lrucache(val.value.id .. name, val.modifiedIndex..the_consumer.modifiedIndex,
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(consumer): missed consumer update due to wrong version in cache [apisix]
Posted by "Revolyssup (via GitHub)" <gi...@apache.org>.
Revolyssup commented on PR #12413:
URL: https://github.com/apache/apisix/pull/12413#issuecomment-3051654665
> Will this pr fixed #12393?
Yes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(consumer): missed consumer update due to wrong version in cache [apisix]
Posted by "nic-6443 (via GitHub)" <gi...@apache.org>.
nic-6443 commented on code in PR #12413:
URL: https://github.com/apache/apisix/pull/12413#discussion_r2194341422
##########
apisix/consumer.lua:
##########
@@ -159,17 +120,49 @@ function plugin_consumer()
conf_version = consumers.conf_version
}
end
-
- local consumer = consumers_id_lrucache(val.value.id .. name,
- val.modifiedIndex, construct_consumer_data, val, config)
- if consumer == nil then
- goto CONTINUE
+ -- if the val is a Consumer, clone it to the local consumer;
+ -- if the val is a Credential, to get the Consumer by consumer_name and then clone
+ -- it to the local consumer.
+ local consumer
+ if is_credential_etcd_key(val.key) then
+ local consumer_name = get_consumer_name_from_credential_etcd_key(val.key)
+ local the_consumer = consumers:get(consumer_name)
+ if the_consumer and the_consumer.value then
+ consumer = consumers_id_lrucache(the_consumer.value, val.modifiedIndex+the_consumer.modifiedIndex,
+ function (val, the_consumer)
+ consumer = core.table.clone(the_consumer.value)
+ consumer.modifiedIndex = the_consumer.modifiedIndex
+ consumer.credential_id = get_credential_id_from_etcd_key(val.key)
+ return consumer
+ end, val, the_consumer)
+ else
+ -- Normally wouldn't get here:
+ -- it should belong to a consumer for any credential.
+ core.log.error("failed to get the consumer for the credential,",
+ " a wild credential has appeared!",
+ " credential key: ", val.key, ", consumer name: ", consumer_name)
+ goto CONTINUE
+ end
+ else
+ consumer = consumers_id_lrucache(val.value, val.modifiedIndex,
Review Comment:
```suggestion
consumer = consumers_id_lrucache(val.key, val.modifiedIndex,
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org