You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@brpc.apache.org by "vinllen (via GitHub)" <gi...@apache.org> on 2023/08/02 11:50:09 UTC

[I] bthread_creation_max_latency延迟过大 (brpc)

vinllen opened a new issue, #2342:
URL: https://github.com/apache/brpc/issues/2342

   **Describe the bug (描述bug)**
   非BUG，问题定位。
   开启了FLAGS_show_bthread_creation_in_vars，目前显示bthread创建时间（max延迟）过大，max达到100-150ms，P9999看起来最大在10ms，正常不到1ms，但是本身bthread usage使用率不到一半，CPU/内存负载并不高，请问有什么情况下会导致这个bthread创建时间过大？
   
   **To Reproduce (复现方法)**
   
   
   **Expected behavior (期望行为)**
   
   
   **Versions (各种版本)**
   OS:
   Compiler:
   brpc:
   protobuf:
   
   **Additional context/screenshots (更多上下文/截图)**
   
   ![image](https://github.com/apache/brpc/assets/6987398/083ac11c-f57d-4b5f-b6de-071099343a65)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org
For additional commands, e-mail: dev-help@brpc.apache.org

Re: [I] bthread_creation_max_latency延迟过大 (brpc)

Posted by "wwbmmm (via GitHub)" <gi...@apache.org>.
wwbmmm commented on issue #2342:
URL: https://github.com/apache/brpc/issues/2342#issuecomment-1680492769

   > 本身bthread usage使用率不到一半，CPU/内存负载并不高
   
   这个显示的是一段时间的平均值。有可能瞬时worker usage较高，bthread得不到及时调度，所以出现了一些长尾


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org
For additional commands, e-mail: dev-help@brpc.apache.org

Re: [I] bthread_creation_max_latency延迟过大 (brpc)

Posted by "wwbmmm (via GitHub)" <gi...@apache.org>.
wwbmmm commented on issue #2342:
URL: https://github.com/apache/brpc/issues/2342#issuecomment-1761439320

   > 我们这边生产集群也有这个现象。 针对大佬说的瞬时并发，我试了一下用rpc_replay设置相同qps减少瞬时值影响，确实可以降低长尾，瞬时并发较高应该是bthread调度长尾的一个原因。但是长尾依旧比较长（bthread_creation_latency平均200us，三九线4500us），大佬还有啥别的想法吗？ 例如client的bthread调度要比server的bthread调度差之类的（因为我们这边没有下游的服务，在bthread_creation_qps相同的情况下，bthread_creation_latency一般都会明显比有下游的服务要好得多，平均几us）。
   
   client的bthread调度和server的bthread调度是一样的。我自己起了个client访问server，client的bthread_creation_latency也是很低的。你这个可能和client端的其它业务逻辑有关


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org
For additional commands, e-mail: dev-help@brpc.apache.org

Re: [I] bthread_creation_max_latency延迟过大 (brpc)

Posted by "FancyJan (via GitHub)" <gi...@apache.org>.
FancyJan commented on issue #2342:
URL: https://github.com/apache/brpc/issues/2342#issuecomment-1735199190

   > > 本身bthread usage使用率不到一半，CPU/内存负载并不高
   > 
   > 这个显示的是一段时间的平均值。有可能瞬时worker usage较高，bthread得不到及时调度，所以出现了一些长尾
   
   我们这边生产集群也有这个现象。
   针对大佬说的瞬时并发，我试了一下用rpc_replay设置相同qps减少瞬时值影响，确实可以降低长尾，瞬时并发较高应该是bthread调度长尾的一个原因。但是长尾依旧比较长（bthread_creation_latency平均200us，三九线4500us），大佬还有啥别的想法吗？
   例如client的bthread调度要比server的bthread调度差之类的（因为我们这边没有下游的服务，在bthread_creation_qps相同的情况下，bthread_creation_latency一般都会明显比有下游的服务要好得多，平均几us）。


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org
For additional commands, e-mail: dev-help@brpc.apache.org