You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@brpc.apache.org by GitBox <gi...@apache.org> on 2020/12/29 12:38:48 UTC
[GitHub] [incubator-brpc] zhuqingrui opened a new issue #1319: 最大限流bug
zhuqingrui opened a new issue #1319:
URL: https://github.com/apache/incubator-brpc/issues/1319
版本：rpc_revision : "0.9.7|master|a9890174|2020-01-22T11:38:39+08:00"
最大并发设置：2000
压测qps：3000 服务返回结果正常
压测qps：5000，服务先偶现拒绝，后全部拒绝
停掉5000压测，改用qps100，服务请求不通
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org
For additional commands, e-mail: dev-help@brpc.apache.org
[GitHub] [incubator-brpc] zhuqingrui commented on issue #1319: 最大限流bug
Posted by GitBox <gi...@apache.org>.
zhuqingrui commented on issue #1319:
URL: https://github.com/apache/incubator-brpc/issues/1319#issuecomment-752315585
嗯，我之前改小了并发设置，max_concurrency=20，限流生效了;就是大并发下（最大并发参数设置不当，2000），再次请求的时候服务没有恢复处理请求这个是什么原因呢，我去看下源码还有错误码
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org
For additional commands, e-mail: dev-help@brpc.apache.org
[GitHub] [incubator-brpc] zhuqingrui commented on issue #1319: 最大限流bug
Posted by GitBox <gi...@apache.org>.
zhuqingrui commented on issue #1319:
URL: https://github.com/apache/incubator-brpc/issues/1319#issuecomment-754526371
我设置成max_concurrency=200都不行，不加这个参数，服务是可以恢复正常的，客户端接收到的错误是service faild, err_code: 2004 reason: [E2004][127.0.0.1:9053][E2004]Reached server's max_concurrency=200 [R1][E2004]；看服务端 不停报，src/bthread/task_group.cpp:672] _remote_rq is full, capacity=2048，监控上并发度还为concurrency: 200，是一些状态变量没法恢复吗，不设置这个参数，是可以恢复正常的
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org
For additional commands, e-mail: dev-help@brpc.apache.org
[GitHub] [incubator-brpc] jamesge commented on issue #1319: 最大限流bug
Posted by GitBox <gi...@apache.org>.
jamesge commented on issue #1319:
URL: https://github.com/apache/incubator-brpc/issues/1319#issuecomment-752103288
“最大并发2000”具体是怎么设置的？max_concurrency并不是qps，是同时进行中的请求，几千的qps下并发一般不会超过100，可以用延时（秒）*qps估算下
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org
For additional commands, e-mail: dev-help@brpc.apache.org
[GitHub] [incubator-brpc] cdjingit commented on issue #1319: 最大限流bug
Posted by GitBox <gi...@apache.org>.
cdjingit commented on issue #1319:
URL: https://github.com/apache/incubator-brpc/issues/1319#issuecomment-754603575
> 我设置成max_concurrency=200都不行，不加这个参数，服务是可以恢复正常的，客户端接收到的错误是service faild, err_code: 2004 reason: [E2004][127.0.0.1:9053][E2004]Reached server's max_concurrency=200 [R1][E2004]；看服务端 不停报，src/bthread/task_group.cpp:672] _remote_rq is full, capacity=2048，监控上并发度还为concurrency: 200，是一些状态变量没法恢复吗，不设置这个参数，是可以恢复正常的
看你描述max_concurrency=200对于你们服务来说还是大了。
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org
For additional commands, e-mail: dev-help@brpc.apache.org
[GitHub] [incubator-brpc] cdjingit commented on issue #1319: 最大限流bug
Posted by GitBox <gi...@apache.org>.
cdjingit commented on issue #1319:
URL: https://github.com/apache/incubator-brpc/issues/1319#issuecomment-755305176
done->Run()没有调用导致泄漏了把，出现问题的时候，worker_usage, bthread_count这些bvar是多少? cpu不是唯一指标。
另外确认，max_concurrency不设置反而不会触发你说的问题？
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org
For additional commands, e-mail: dev-help@brpc.apache.org
[GitHub] [incubator-brpc] zhuqingrui commented on issue #1319: 最大限流bug
Posted by GitBox <gi...@apache.org>.
zhuqingrui commented on issue #1319:
URL: https://github.com/apache/incubator-brpc/issues/1319#issuecomment-752061554
@jamesge
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org
For additional commands, e-mail: dev-help@brpc.apache.org
[GitHub] [incubator-brpc] hiberabyss commented on issue #1319: 最大限流bug
Posted by GitBox <gi...@apache.org>.
hiberabyss commented on issue #1319:
URL: https://github.com/apache/incubator-brpc/issues/1319#issuecomment-977580740
在压测环境里把 max_concurrency 设高了，然后并发数高的话，最后也是整个服务会卡死掉，就不会接收新的请求
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org
For additional commands, e-mail: dev-help@brpc.apache.org
[GitHub] [incubator-brpc] cdjingit commented on issue #1319: 最大限流bug
Posted by GitBox <gi...@apache.org>.
cdjingit commented on issue #1319:
URL: https://github.com/apache/incubator-brpc/issues/1319#issuecomment-754400878
> 嗯，我之前改小了并发设置，max_concurrency=20，限流生效了;就是大并发下（最大并发参数设置不当，2000），再次请求的时候服务没有恢复处理请求这个是什么原因呢，我去看下源码还有错误码
最大并发设置过大，服务已经压垮了，没法恢复是正常的。
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org
For additional commands, e-mail: dev-help@brpc.apache.org
[GitHub] [incubator-brpc] zhuqingrui commented on issue #1319: 最大限流bug
Posted by GitBox <gi...@apache.org>.
zhuqingrui commented on issue #1319:
URL: https://github.com/apache/incubator-brpc/issues/1319#issuecomment-754727509
brpc框架认为是大了，但是机器cpu利用率只有40%；另外当我把请求停掉的时候，空跑着的服务的server._concurrency的值一直是200，当我再单次请求的时候，会增加成201，而被限流；另外发现task_group的队列也是一直满着的，没有进行消费呢
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org
For additional commands, e-mail: dev-help@brpc.apache.org
[GitHub] [incubator-brpc] zhuqingrui commented on issue #1319: 最大限流bug
Posted by GitBox <gi...@apache.org>.
zhuqingrui commented on issue #1319:
URL: https://github.com/apache/incubator-brpc/issues/1319#issuecomment-755855550
线上服务，高并发跑了一两年了的😄
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org
For additional commands, e-mail: dev-help@brpc.apache.org