You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by GitBox <gi...@apache.org> on 2022/04/12 07:09:16 UTC
[GitHub] [apisix] johndudo opened a new issue, #6829: help request: plugin limit-req works not correctly when burst&rate are larger than 3000.
johndudo opened a new issue, #6829:
URL: https://github.com/apache/apisix/issues/6829
### Description
when burst&rate are lower than 2000, limit-req works as expectation mostly; however, it works not correctly when burst&rate are larger than 3000.
test client: wrk
proxy-server: apisix 2.12.1, 2.13.1
upstream: nginx
### Environment
- APISIX version (run `apisix version`): 2.12.1
- Operating system (run `uname -a`): Linux dycolg_lgs01 3.10.0-693.el7.x86_64 #1 SMP Tue Aug 22 21:09:27 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
- OpenResty / Nginx version (run `openresty -V` or `nginx -V`):
- etcd version, if relevant (run `curl http://127.0.0.1:9090/v1/server_info`): 3.14.3
- APISIX Dashboard version, if relevant:
- Plugin runner version, for issues related to plugin runners:
- LuaRocks version, for installation issues (run `luarocks --version`):
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] johndudo commented on issue #6829: help request: plugin limit-req works not correctly when burst&rate are larger than 3000.
Posted by GitBox <gi...@apache.org>.
johndudo commented on issue #6829:
URL: https://github.com/apache/apisix/issues/6829#issuecomment-1097839511
> > I'm so confused about this and did not know what to do
>
> I think you should read the documentation first, rather than testing directly.
The following is the detail info about test case and config
### (1) "limit-req" test case on nginx (not pass apisix)
nginx located on VM: 4C8G, CentOS7
Linux dycolg_lgs03 3.10.0-693.el7.x86_64 #1 SMP Tue Aug 22 21:09:27 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux,
nginx version: nginx/1.20.2
nginx ip: 10.118.22.217
nginx config:
limit_req_zone $binary_remote_addr zone=mylimit:10m rate=3000r/s;
server {
listen 80;
server_name localhost;
#access_log /var/log/nginx/host.access.log main;
location / {
root /usr/share/nginx/html;
index index.html index.htm;
}
#error_page 404 /404.html;
# redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
location = /test/performance.html {
limit_req zone=mylimit burst=5000;
root /usr/share/nginx/html;
}
}
- wrk test result
[tomcat@TYCOSUTOMAPI02 wrk-4.1.0]$ ./wrk -t 4 -c 100 -d 30s http://10.118.22.217/test/performance.html --latency
Running 30s test @ http://10.118.22.217/test/performance.html
4 threads and 100 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 33.10ms 2.03ms 52.30ms 91.23%
Req/Sec 753.50 35.50 0.98k 93.17%
Latency Distribution
50% 33.29ms
75% 33.79ms
90% 34.27ms
99% 39.28ms
90041 requests in 30.02s, 130.95MB read
Requests/sec: 2999.16
Transfer/sec: 4.36MB
- ab test result
[tomcat@TYCOSUTOMAPI02 wrk-4.1.0]$ ab -n 90000 -c 100 http://10.118.22.217/test/performance.html
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 10.118.22.217 (be patient)
Completed 9000 requests
Completed 18000 requests
Completed 27000 requests
Completed 36000 requests
Completed 45000 requests
Completed 54000 requests
Completed 63000 requests
Completed 72000 requests
Completed 81000 requests
Completed 90000 requests
Finished 90000 requests
Server Software: nginx/1.20.2
Server Hostname: 10.118.22.217
Server Port: 80
Document Path: /test/performance.html
Document Length: 1286 bytes
Concurrency Level: 100
Time taken for tests: 31.472 seconds
Complete requests: 90000
Failed requests: 0
Write errors: 0
Total transferred: 136803040 bytes
HTML transferred: 115742572 bytes
Requests per second: 2859.66 [#/sec] (mean)
Time per request: 34.969 [ms] (mean)
Time per request: 0.350 [ms] (mean, across all concurrent requests)
Transfer rate: 4244.91 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 3 24.0 1 1025
Processing: 1 32 13.0 32 451
Waiting: 1 32 12.9 32 451
Total: 3 35 28.0 33 1214
Percentage of the requests served within a certain time (ms)
50% 33
66% 34
75% 34
80% 35
90% 40
95% 46
98% 64
99% 82
100% 1214 (longest request)
test case shows the following result:
limit-req worked as expect on nginx. both 'wrk' and 'ab' test results showed that the QPS was lower than 3000.
### (2) "limit-req" test case on apisix(apisix proxy nginx)
upstream nginx: the nginx on 10.118.22.217 above, remove the limit-req config on nginx;
apisix located on VM: 4C8G, CentOS7,Linux dycolg_lgs01 3.10.0-693.el7.x86_64 #1 SMP Tue Aug 22 21:09:27 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
apisix version: 2.12.1
apisix ip: 10.118.22.215
etcd :3.14.3
apisix route config:
{
"uri": "/test/performance.html",
"name": "performance-test-base-line",
"methods": [
"GET",
"POST",
"PUT",
"DELETE",
"PATCH",
"HEAD",
"OPTIONS",
"CONNECT",
"TRACE"
],
"plugins": {
"limit-req": {
"burst": 5000,
"disable": false,
"key": "remote_addr",
"key_type": "var",
"nodelay": false,
"rate": 3000,
"rejected_code": 503
},
"prometheus": {
"disable": false,
"prefer_name": true
}
},
"upstream": {
"nodes": [
{
"host": "10.118.22.217",
"port": 80,
"weight": 1
}
],
"timeout": {
"connect": 6,
"send": 6,
"read": 6
},
"type": "roundrobin",
"scheme": "http",
"pass_host": "pass",
"keepalive_pool": {
"idle_timeout": 60,
"requests": 1000,
"size": 320
}
},
"status": 1
}
- wrk test result
[tomcat@TYCOSUTOMAPI02 wrk-4.1.0]$ ./wrk -t 4 -c 100 -d 30s http://10.118.22.215:9080/test/performance.html --latency
Running 30s test @ http://10.118.22.215:9080/test/performance.html
4 threads and 100 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 22.30ms 9.23ms 113.32ms 64.98%
Req/Sec 1.13k 374.14 2.85k 82.92%
Latency Distribution
50% 22.99ms
75% 29.96ms
90% 32.35ms
99% 42.06ms
134939 requests in 30.02s, 198.30MB read
Requests/sec: 4494.34
Transfer/sec: 6.60MB
- ab test result
[tomcat@TYCOSUTOMAPI02 wrk-4.1.0]$ ab -n 134939 -c 100 http://10.118.22.215:9080/test/performance.html
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 10.118.22.215 (be patient)
Completed 13493 requests
Completed 26986 requests
Completed 40479 requests
Completed 53972 requests
Completed 67465 requests
Completed 80958 requests
Completed 94451 requests
Completed 107944 requests
Completed 121437 requests
Completed 134930 requests
Finished 134939 requests
Server Software: APISIX/2.12.1
Server Hostname: 10.118.22.215
Server Port: 9080
Document Path: /test/performance.html
Document Length: 1286 bytes
Concurrency Level: 100
Time taken for tests: 39.961 seconds
Complete requests: 134939
Failed requests: 0
Write errors: 0
Total transferred: 207267840 bytes
HTML transferred: 173532840 bytes
Requests per second: 3376.73 [#/sec] (mean)
Time per request: 29.614 [ms] (mean)
Time per request: 0.296 [ms] (mean, across all concurrent requests)
Transfer rate: 5065.13 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 1.1 1 41
Processing: 3 28 8.5 28 208
Waiting: 3 28 8.5 28 207
Total: 4 30 8.4 30 208
Percentage of the requests served within a certain time (ms)
50% 30
66% 31
75% 33
80% 33
90% 37
95% 41
98% 46
99% 52
100% 208 (longest request)
test case shows the following result:
limit-req did not worked as expect on apisix. both 'wrk' and 'ab' test results showed that the QPS was larger than 3000, the expect result was lower than 3000.
maybe there is something wrong with my config or something I missed, I am not sure...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on issue #6829: help request: plugin limit-req works not correctly when burst&rate are larger than 3000.
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on issue #6829:
URL: https://github.com/apache/apisix/issues/6829#issuecomment-1098175226
IMO, we should observe if any request is delayed and if any request is rejected. This has nothing to do with QPS.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on issue #6829: help request: plugin limit-req works not correctly when burst&rate are larger than 3000.
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on issue #6829:
URL: https://github.com/apache/apisix/issues/6829#issuecomment-1096791401
> I'm so confused about this and did not know what to do
I think you should read the documentation first, rather than testing directly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] johndudo commented on issue #6829: help request: plugin limit-req works not correctly when burst&rate are larger than 3000.
Posted by GitBox <gi...@apache.org>.
johndudo commented on issue #6829:
URL: https://github.com/apache/apisix/issues/6829#issuecomment-1096600077
> @johndudo
>
> FYI: http://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req
>
> Please read the docs, the limit-req plugin works just as the nginx limit_req module.
yeah, I know; I've checked and test the similar config on nginx, nginx worked correctly as expect, but apisix did not worked as expect when burst&rate are larger than 3000. I'm so confused about this and did not know what to do. so Please help me.
Thanks a lot.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on issue #6829: help request: plugin limit-req works not correctly when burst&rate are larger than 3000.
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on issue #6829:
URL: https://github.com/apache/apisix/issues/6829#issuecomment-1098752063
> Is there any right way or right tools that can be used to verify that apisix works as nginx when using the following same config
I remember that Apache JMeter can analyze the request results.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] help request: plugin limit-req works not correctly when burst&rate are larger than 3000. [apisix]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #6829: help request: plugin limit-req works not correctly when burst&rate are larger than 3000.
URL: https://github.com/apache/apisix/issues/6829
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on issue #6829: help request: plugin limit-req works not correctly when burst&rate are larger than 3000.
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on issue #6829:
URL: https://github.com/apache/apisix/issues/6829#issuecomment-1096789437
I don't agree with the conclusion of your test. You can't use wrk's QPS results as a test result for limit-req.
IMO, you need to verify this with the response's status code distribution.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tokers commented on issue #6829: help request: plugin limit-req works not correctly when burst&rate are larger than 3000.
Posted by GitBox <gi...@apache.org>.
tokers commented on issue #6829:
URL: https://github.com/apache/apisix/issues/6829#issuecomment-1096453233
@johndudo
FYI: http://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req
Please read the docs, the limit-req plugin works just as the nginx limit_req module.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] help request: plugin limit-req works not correctly when burst&rate are larger than 3000. [apisix]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #6829:
URL: https://github.com/apache/apisix/issues/6829#issuecomment-1936959739
This issue has been closed due to lack of activity. If you think that is incorrect, or the issue requires additional review, you can revive the issue at any time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] johndudo commented on issue #6829: help request: plugin limit-req works not correctly when burst&rate are larger than 3000.
Posted by GitBox <gi...@apache.org>.
johndudo commented on issue #6829:
URL: https://github.com/apache/apisix/issues/6829#issuecomment-1098657940
> IMO, we should observe if any request is delayed and if any request is rejected. This has nothing to do with QPS.
that's true. if any request is delayed or if any request is rejected. That has nothing to do with QPS.
The rejected requests should be sorted to Non-2xx or 3xx responses on the test result( both wrk and ab).
Is there any right way or right tools that can be used to verify that apisix works as nginx when using the following same config?
burst 5000, rate 3000, nodelay false.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] johndudo commented on issue #6829: help request: plugin limit-req works not correctly when burst&rate are larger than 3000.
Posted by GitBox <gi...@apache.org>.
johndudo commented on issue #6829:
URL: https://github.com/apache/apisix/issues/6829#issuecomment-1118219989
i tried but gave up using jmeter, did anyone else meet the same case result when using apache ab or wrk?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] kingluo commented on issue #6829: help request: plugin limit-req works not correctly when burst&rate are larger than 3000.
Posted by "kingluo (via GitHub)" <gi...@apache.org>.
kingluo commented on issue #6829:
URL: https://github.com/apache/apisix/issues/6829#issuecomment-1424009479
@johndudo I suggest you calculate the rate from the upstream itself, which is the most real and accurate.
In this demo, I use the same APISIX instance as a fake upstream and calculate RPS there:
https://gist.github.com/kingluo/6b33e732948b511b2e524910a229dcc2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] help request: plugin limit-req works not correctly when burst&rate are larger than 3000. [apisix]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #6829:
URL: https://github.com/apache/apisix/issues/6829#issuecomment-1911786440
This issue has been marked as stale due to 350 days of inactivity. It will be closed in 2 weeks if no further activity occurs. If this issue is still relevant, please simply write any comment. Even if closed, you can still revive the issue at any time or discuss it on the dev@apisix.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on issue #6829: help request: plugin limit-req works not correctly when burst&rate are larger than 3000.
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on issue #6829:
URL: https://github.com/apache/apisix/issues/6829#issuecomment-1098173841
As I said above, you're not testing the right way, so I can't help you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] johndudo commented on issue #6829: help request: plugin limit-req works not correctly when burst&rate are larger than 3000.
Posted by GitBox <gi...@apache.org>.
johndudo commented on issue #6829:
URL: https://github.com/apache/apisix/issues/6829#issuecomment-1096259789
(1) work normal case
plugin settting:
"limit-req": {
"burst": 5000,
"disable": false,
"key": "server_addr",
"key_type": "var",
"nodelay": false,
"rate": 1000,
"rejected_code": 503
}
**running result:**
[tomcat@TYCOSUTOMAPI02 wrk-4.1.0]$ ./wrk -t 4 -c 100 -d 10s http://XXXX:9080/test/getjson --latency
Running 10s test @ http://XXXX:9080/test/getjson
4 threads and 100 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 97.34ms 8.15ms 113.64ms 97.39%
Req/Sec 256.52 18.60 480.00 93.50%
Latency Distribution
50% 98.67ms
75% 99.84ms
90% 100.82ms
99% 103.51ms
10227 requests in 10.01s, 14.85MB read
Requests/sec: 1021.84
Transfer/sec: 1.48MB
期望QPS 1000, 实际 1021.84
(2) work wrong case
plugin settting:
"limit-req": {
"burst": 5000,
"disable": false,
"key": "server_addr",
"key_type": "var",
"nodelay": false,
"rate": 3000,
"rejected_code": 503
}
**running result:**
[tomcat@TYCOSUTOMAPI02 wrk-4.1.0]$ ./wrk -t 4 -c 100 -d 10s http://XXXX:9080/test/getjson --latency
Running 10s test @ http://XXXX:9080/test/getjson
4 threads and 100 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 17.60ms 12.86ms 133.63ms 83.90%
Req/Sec 1.50k 703.36 3.01k 67.92%
Latency Distribution
50% 13.10ms
75% 27.00ms
90% 31.39ms
99% 70.52ms
59767 requests in 10.01s, 86.81MB read
Requests/sec: 5970.67
Transfer/sec: 8.67MB
期望QPS 3000, 实际 5970.67
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org