You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by GitBox <gi...@apache.org> on 2022/03/09 14:35:34 UTC
[GitHub] [apisix] kingmouse-yx opened a new issue #6564: bug: The jwt-auth has an exception in the case of concurrency
kingmouse-yx opened a new issue #6564:
URL: https://github.com/apache/apisix/issues/6564
### Issue description
When I authenticated with jwt-auth, I found that there will be problems when using the same token to make multiple requests at the same time.
My jwt-auth configuration is as follows:
```json
{
"username": "consumer_test",
"plugins": {
"jwt-auth": {
"base64_secret": true,
"disable": false,
"exp": 86400,
"key": "consumer_test_key"
}
}
}
```
My routing configuration is as follows:
```json
{
"uri": "/get04",
"name": "测试jwt",
"methods": [
"GET",
"POST",
"PUT",
"DELETE",
"PATCH",
"HEAD",
"OPTIONS",
"CONNECT",
"TRACE"
],
"plugins": {
"jwt-auth": {
"disable": false
},
"proxy-rewrite": {
"uri": "/get"
}
},
"upstream_id": "396931816625275591",
"status": 1
}
```
After I asked http://127.0.0.1:9080/apisix/plugin/jwt/sign?key=consumer_test and got a token, I did concurrency testing by using jmter. I set up two concurrent, which lasted 10 seconds.However I found that most of the requests were in failure status. The result of the response is 401. Then I checked the log and found a lot of information about authentication failure.
```text
run_plugin(): jwt-auth exits with http status code 401
```
But it's normal when I click manually through postman.
### Environment
- apisix version (cmd: `apisix version`): `2.9`
- OS (cmd: `uname -a`): `Darwin kingmouse.lan 21.3.0 Darwin Kernel Version 21.3.0: Wed Jan 5 21:37:58 PST 2022; root:xnu-8019.80.24~20/RELEASE_ARM64_T8101 arm64`
- OpenResty / Nginx version (cmd: `nginx -V` or `openresty -V`): `openresty/1.19.3.1`
- etcd version, if have (cmd: run `curl http://127.0.0.1:9090/v1/server_info` to get the info from server-info API): `3.4.0`
- apisix-dashboard version, if have: ` 2.10.1`
- the plugin runner version, if the issue is about a plugin runner (cmd: depended on the kind of runner):
- luarocks version, if the issue is about installation (cmd: `luarocks --version`):
### Steps to reproduce
1.add upstream
2.add consumer
3.add route, enable jwt-auth plugin
4.use jmeter test
### Actual result
There will be problems when using the same token to make multiple requests at the same time.
### Error log
```log
2022/03/09 13:43:14 [warn] 42#42: *53410 [lua] plugin.lua:658: run_plugin(): jwt-auth exits with http status code 401, client: 172.18.0.1, server: _, request: "POST /get04 HTTP/1.1", host: "192.168.18.215:9080"
```
### Expected result
The same token can initiate multiple requests at the same time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] kingmouse-yx commented on issue #6564: bug: The jwt-auth has an exception in the case of concurrency
Posted by GitBox <gi...@apache.org>.
kingmouse-yx commented on issue #6564:
URL: https://github.com/apache/apisix/issues/6564#issuecomment-1063987354
@soulbird
I don't quite agree with this statement.
Because I often encounter invalid tokens. The information it returns is only a part of the token.
In order to verify my statement, I added a log in jwt-auth.
```lua
function _M.rewrite(conf, ctx)
local jwt_token, err = fetch_jwt_token(ctx)
-- The following two lines of code were added by me.
local uuid = uuid.generate_v4()
core.log.warn("uuid：",uuid,"jwt_token: ",tostring(jwt_token))
if not jwt_token then
if err and err:sub(1, #"no cookie") ~= "no cookie" then
core.log.error("failed to fetch JWT token: ", err)
end
return 401, {message = "Missing JWT token in request"}
end
local jwt_obj = jwt:load_jwt(jwt_token)
core.log.info("jwt object: ", core.json.delay_encode(jwt_obj))
if not jwt_obj.valid then
return 401, {message = jwt_obj.reason}
end
local user_key = jwt_obj.payload and jwt_obj.payload.key
if not user_key then
return 401, {message = "missing user key in JWT token"}
end
local consumer_conf = consumer_mod.plugin(plugin_name)
if not consumer_conf then
return 401, {message = "Missing related consumer"}
end
local consumers = lrucache("consumers_key", consumer_conf.conf_version,
create_consume_cache, consumer_conf)
local consumer = consumers[user_key]
if not consumer then
return 401, {message = "Invalid user key in JWT token"}
end
core.log.info("consumer: ", core.json.delay_encode(consumer))
local _, auth_secret = algorithm_handler(consumer)
jwt_obj = jwt:verify_jwt_obj(auth_secret, jwt_obj)
core.log.info("jwt object: ", core.json.delay_encode(jwt_obj))
if not jwt_obj.verified then
-- The following line of code was added by me.
core.log.warn("uuid：",uuid,"jwt_obj.reason: ",tostring(jwt_obj.reason))
return 401, {message = jwt_obj.reason}
end
consumer_mod.attach_consumer(ctx, consumer, consumer_conf)
core.log.info("hit jwt-auth rewrite")
end
```
After testing, I found many similar logs in my log.
```log
2022/03/10 11:49:42 [warn] 43#43: *1220630 [lua] jwt-auth.lua:258: phase_func(): uuid：4b958404-45e8-4596-a1b0-58c423673705jwt_token: eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJleHAiOjE2NDY5OTkwNjEsImtleSI6InRlc3QwMSJ9.kB25Qzxf7gB5IJ8MYgf9EOWKHaC8BtIWdOYk-Oc39YQ, client: 172.18.0.1, server: _, request: "POST /get04 HTTP/1.1", host: "127.0.0.1:9080"
2022/03/10 11:49:42 [warn] 43#43: *1220630 [lua] jwt-auth.lua:300: phase_func(): uuid：4b958404-45e8-4596-a1b0-58c423673705jwt_obj.reason: signature mismatch: kB25Qzxf7gB5IJ8MYgf9EOWKHaC8BtIWdOYk-Oc39YQ, client: 172.18.0.1, server: _, request: "POST /get04 HTTP/1.1", host: "127.0.0.1:9080"
```
Obviously, this is the result of one request, and apisik received the complete token
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander closed issue #6564: bug: The jwt-auth has an exception in the case of concurrency
Posted by GitBox <gi...@apache.org>.
spacewander closed issue #6564:
URL: https://github.com/apache/apisix/issues/6564
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] soulbird commented on issue #6564: bug: The jwt-auth has an exception in the case of concurrency
Posted by GitBox <gi...@apache.org>.
soulbird commented on issue #6564:
URL: https://github.com/apache/apisix/issues/6564#issuecomment-1064744495
I still can't reproduce your problem, please give me your contact information email or wechat if it is convenient, I would like to communicate with you by voice for more information. Then I will sync the result here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] soulbird commented on issue #6564: bug: The jwt-auth has an exception in the case of concurrency
Posted by GitBox <gi...@apache.org>.
soulbird commented on issue #6564:
URL: https://github.com/apache/apisix/issues/6564#issuecomment-1069800524
https://github.com/apache/apisix-dashboard/issues/2176#issuecomment-944127485
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] soulbird commented on issue #6564: bug: The jwt-auth has an exception in the case of concurrency
Posted by GitBox <gi...@apache.org>.
soulbird commented on issue #6564:
URL: https://github.com/apache/apisix/issues/6564#issuecomment-1063945225
I checked your configuration of jmeter, that http request header config like this:
```xml
<HeaderManager guiclass="HeaderPanel" testclass="HeaderManager" testname="HTTP信息头管理器" enabled="true">
<collectionProp name="HeaderManager.headers">
<elementProp name="" elementType="Header">
<stringProp name="Header.name">Authorization</stringProp>
<stringProp name="Header.value">eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJleHAiOjE2NDY5OTA5OTQsImtleSI6InRlc3QwMSJ9.J-faXYzdL0AMgLGO4IH9P5uLyaFlG0Mic8HZ3vlAKt8</stringProp>
</elementProp>
</collectionProp>
</HeaderManager>
```
The token is `eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJleHAiOjE2NDY5OTA5OTQsImtleSI6InRlc3QwMSJ9.J-faXYzdL0AMgLGO4IH9P5uLyaFlG0Mic8HZ3vlAKt8`, and what you sent to apisix is `J-faXYzdL0AMgLGO4IH9P5uLyaFlG0Mic8HZ3vlAKt8`. You can see, they don't match. So, you can confirm the following points
1、The token you send to apisix is what you expect
2、Confirm that your network environment is normal during the stress test
3、Any other means you can think of
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] kingmouse-yx commented on issue #6564: bug: The jwt-auth has an exception in the case of concurrency
Posted by GitBox <gi...@apache.org>.
kingmouse-yx commented on issue #6564:
URL: https://github.com/apache/apisix/issues/6564#issuecomment-1063863720
@soulbird
First of all, thank you very much.
I tested it again just now and the issue still exists. When jmeter responds to 401, the response message is:
```json
{"message":"signature mismatch: J-faXYzdL0AMgLGO4IH9P5uLyaFlG0Mic8HZ3vlAKt8"}
```
The normal message is:
```json
{
"tag": {
"certStatus": "1",
"certTp": "01"
},
"data": {
"a_name": "xxx",
"a_age": "29"
},
"certId": "341126197709218366",
"userid": "xxxxxxxxxxxx01",
"realname": "互联网-lua测试",
"caseId": "11111111999999990000009909"
}
```
Here is my report in jmeter. Nearly half of the requests failed.
| Label | # 样本 | 平均值 | 中位数 | 90% 百分位 | 95% 百分位 | 99% 百分位 | 最小值 | 最大值 | 异常 % | 吞吐量 | 接收 KB/sec | 发送 KB/sec |
| -------- | ------ | ------ | ------ | ---------- | ---------- | ---------- | ------ | ------ | ------ | --------- | ----------- | ----------- |
| HTTP请求 | 6304 | 2 | 3 | 4 | 5 | 7 | 1 | 49 | 49.67% | 633.88638 | 226.68 | 199.33 |
| 总体 | 6304 | 2 | 3 | 4 | 5 | 7 | 1 | 49 | 49.67% | 633.88638 | 226.68 | 199.33 |
This is my jmeter configuration：
[HTTP请求-jwt-000009.jmx.zip](https://github.com/apache/apisix/files/8222192/HTTP.-jwt-000009.jmx.zip)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on issue #6564: bug: The jwt-auth has an exception in the case of concurrency
Posted by GitBox <gi...@apache.org>.
spacewander commented on issue #6564:
URL: https://github.com/apache/apisix/issues/6564#issuecomment-1069803908
Move the discussion to https://github.com/apache/apisix-dashboard/issues/2383
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] soulbird commented on issue #6564: bug: The jwt-auth has an exception in the case of concurrency
Posted by GitBox <gi...@apache.org>.
soulbird commented on issue #6564:
URL: https://github.com/apache/apisix/issues/6564#issuecomment-1063695040
I have not been able to reproduce your problem with jmeter(set up two concurrent, which lasted 10 seconds). I also try to reproduce it with wrk like this:
```shell
wrk -t2 -c2 -d10s curl http://127.0.0.1:9080/get04 -H "Authorization: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2NDY5Njc4MDAsImtleSI6ImNvbnN1bWVyX3Rlc3Rfa2V5In0.nMnGGAhOILFRTX1j6et-oLCjznZ11U7r9swftbDjVDE"
```
or
```shell
wrk -t2 -c2 -d20s curl http://127.0.0.1:9080/get04 -H "Authorization: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2NDY5Njc4MDAsImtleSI6ImNvbnN1bWVyX3Rlc3Rfa2V5In0.nMnGGAhOILFRTX1j6et-oLCjznZ11U7r9swftbDjVDE"
```
The results are all normal, and Apache APISIX works fine too. So, can you provide the response body when responding to 401 in jmeter? Better if you can provide configuration of jmeter, I will try reproduce again
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] kingmouse-yx commented on issue #6564: bug: The jwt-auth has an exception in the case of concurrency
Posted by GitBox <gi...@apache.org>.
kingmouse-yx commented on issue #6564:
URL: https://github.com/apache/apisix/issues/6564#issuecomment-1064827433
> I still can't reproduce your problem, please give me your contact information email or wechat if it is convenient, I would like to communicate with you by voice for more information. Then I will sync the result here
my email：`kingmousejiao@gmail.com`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on issue #6564: bug: The jwt-auth has an exception in the case of concurrency
Posted by GitBox <gi...@apache.org>.
spacewander commented on issue #6564:
URL: https://github.com/apache/apisix/issues/6564#issuecomment-1063552295
@soulbird
Please help us to confirm it. You can try to reproduce it with wrk.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] kingmouse-yx commented on issue #6564: bug: The jwt-auth has an exception in the case of concurrency
Posted by GitBox <gi...@apache.org>.
kingmouse-yx commented on issue #6564:
URL: https://github.com/apache/apisix/issues/6564#issuecomment-1064227632
@soulbird
Hello, I found that this method may lead to the failure of token verification: `algorithm_handler(consumer)`
```lua
local _, auth_secret = algorithm_handler(consumer)
```
I printed the value of auth-secret. Although it is a string of error code, it is still very available.
I found that this value in the log can be divided into the following two types:
- 1
```txt
???????j??gs?,(?L0=?2OG?
?v+??
```
- 2
```txt
%#?A???o?FF???}ާ???? L#
```
Among them, the number of times counted by 1 is just the number of successful verification, and the number of times counted by 2 is just the number of failed verification.
I think this may be the reason why the inspection failed.
```log
2022/03/10 15:42:05 [warn] 42#42: *1330147 [lua] jwt-auth.lua:295: phase_func(): uuid：faeff528-cc07-4b0d-9f7c-170221999fd6 auth_secret: ???????j??gs?,(?L0=?2OG?
?v+??, client: 172.18.0.1, server: _, request: "POST /get04 HTTP/1.1", host: "127.0.0.1:9080"
2022/03/10 15:42:05 [warn] 43#43: *1330146 [lua] jwt-auth.lua:295: phase_func(): uuid：36eed0fd-06a4-41b6-8f46-a87d6b024bdb auth_secret: %#?A???o?FF???}ާ???? L#, client: 172.18.0.1, server: _, request: "POST /get04 HTTP/1.1", host: "127.0.0.1:9080"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] kingmouse-yx commented on issue #6564: bug: The jwt-auth has an exception in the case of concurrency
Posted by GitBox <gi...@apache.org>.
kingmouse-yx commented on issue #6564:
URL: https://github.com/apache/apisix/issues/6564#issuecomment-1064289505
@soulbird
Hello, finally I found the location of this issue: lrucache
```lua
local consumers = lrucache("consumers_key", consumer_conf.conf_version,
create_consume_cache, consumer_conf)
local consumer = consumers[user_key]
-- The following two lines of code were added by me.
core.log.warn("user_key ",user_key)
core.log.warn("consumer: ",core.json.encode(consumer,true))
```
In the log, I found the value of user-key is fixed, but the value of consumer is different(`secret value`). The log is as follows:
```log
2022/03/10 16:40:17 [warn] 43#43: *1465247 [lua] jwt-auth.lua:295: phase_func(): consumer: {"auth_conf":{"base64_secret":false,"key":"test01","algorithm":"HS256","exp":86400,"secret":"VIN\/yaVQeSk5VcQ\/Tln9OMKViC7xUNWgxQue6UM4NcQ=","disable":false},"create_time":1646723289,"update_time":1646929454,"id":"test01","username":"test01","consumer_name":"test01","plugins":{"jwt-auth":"table: 0x4afa5e83de10"}}, client: 172.18.0.1, server: _, request: "POST /get04 HTTP/1.1", host: "127.0.0.1:9080"
2022/03/10 16:40:17 [warn] 42#42: *1465245 [lua] jwt-auth.lua:295: phase_func(): consumer: {"auth_conf":{"base64_secret":false,"key":"test01","algorithm":"HS256","exp":86400,"secret":"YStztFps1YK+XmnoPJepSSySkQ5mMljZ9OSex3gMAdk=","disable":false},"create_time":1646723289,"update_time":1646929454,"id":"test01","username":"test01","consumer_name":"test01","plugins":{"jwt-auth":"table: 0x4afa5e47ab48"}}, client: 172.18.0.1, server: _, request: "POST /get04 HTTP/1.1", host: "127.0.0.1:9080"
```
> Note: in this test, I set base64_secret'value to false
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org