[GitHub] [incubator-brpc] promoon commented on issue #697: 客户端出现大量timeout

[GitBox <gi...@apache.org>]

promoon commented on issue #697: 客户端出现大量timeout
URL: https://github.com/apache/incubator-brpc/issues/697#issuecomment-507983880
 
 
   > 已经把bvar的线程移动到main函数之后执行，以保证fork之后才执行bvar线程．
   > 
   > 目前进行业务压测，超时问题应该是bthread_worker_usage达到上限导致,跟预期服务承载能力不相符，目前还在排查．
   > ![1755025561](https://user-images.githubusercontent.com/8772285/55847307-5cf25080-5b7b-11e9-8b62-20c5aae2a2d8.jpg)
   > ![376294209](https://user-images.githubusercontent.com/8772285/55847315-61b70480-5b7b-11e9-9f03-238b22bb9d5f.jpg)
   > 
   > 但是随着qps继续上涨，服务会直接卡死，访问前端页面(http://_:6104,http://_:6104/vars)也不行，压测全部报超时error
   > 
   > Search request Error1: 1008: [E1008]Reached timeout=3500ms @_:6104 Query : 爱情练习题 Search request Error1: 1008: [E1008]Reached timeout=3500ms @_:6104 Query : 爱情两部曲
   > Search request Error1: 1008: [E1008]Reached timeout=3500ms @_:6104 Query : 爱情两好三坏 Search request Error1: 1008: [E1008]Reached timeout=3500ms @_:6104 Query : 爱情领域
   > 
   > 且服务cpu占用基本为0,服务不能自己恢复正常,只能重启解决．
   > 但是netstat查看监听端口正常,处于LINSEN状态，并且使用telnet访问此端口也能连接上.
   > 最后发现是服务的线程block住，导致的问题.
   > ![Screenshot from 2019-04-10 14:16:22](https://user-images.githubusercontent.com/8772285/55856000-989d1280-5b9b-11e9-89d2-060053fac326.png)
   > 
   > 我的理解是这样子的:由于我的服务还会使用brpc client往下游发送数据，而造成bthread资源不够,所以整体block住,不知道对不对.
   > 
   > 请问这种情况如何避免?
   
   我们目前在用的bvar也遇到了同样的core dump，请问你是如何将bvar线程移到main函数之后的？

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org
For additional commands, e-mail: dev-help@brpc.apache.org