You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@servicecomb.apache.org by GitBox <gi...@apache.org> on 2022/01/22 02:21:13 UTC
[GitHub] [servicecomb-java-chassis] doom15 opened a new issue #2705: 多下游实例，高TPS下的性能问题
doom15 opened a new issue #2705:
URL: https://github.com/apache/servicecomb-java-chassis/issues/2705
https://github.com/apache/servicecomb-java-chassis/blob/ad53a38dd7a18cfe76705f16f8772859bdf890c4/handlers/handler-loadbalance/src/main/java/org/apache/servicecomb/loadbalance/filter/IsolationDiscoveryFilter.java#L109-L119
此处可能有性能问题，请分析：
1. 此处的child是new出来的，但是在其他的discovery filter中都是发现使用map的computeIfAbsent。
2. 由于1中的DiscoveryTreeNode是新建的，在后续ServerDiscoveryFilter的filter中child肯定为空。则调用createDiscoveryTreeNode，新建全部ServiceCombServer对象，即每次请求都会新建所有的ServiceCombServer对象。
我们的生产环境遇到该问题如下：
1. 下游1500左右的实例，每个节点每秒对下游发起500次请求，若下游服务响应较慢时，则出现此现象。
2. 此时内存中ServiceCombServer对象在内存中存在500万左右的实例，占用内存，一直g在c，且CPU占用高，严重时可能发生OOM。
3. 使用arthas监控，每10S新建ServiceCombServer对象60万次。查看metrics日志有较多connection的请求在排队和使用，总数约4W
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@servicecomb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-java-chassis] doom15 commented on issue #2705: 多下游实例，高TPS下的性能问题
Posted by GitBox <gi...@apache.org>.
doom15 commented on issue #2705:
URL: https://github.com/apache/servicecomb-java-chassis/issues/2705#issuecomment-1055233392
@lbc97 @liubao68 能合到1.3.x的分支吗？ 我们切2.0估计还没那么快
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@servicecomb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-java-chassis] liubao68 commented on issue #2705: 多下游实例，高TPS下的性能问题
Posted by GitBox <gi...@apache.org>.
liubao68 commented on issue #2705:
URL: https://github.com/apache/servicecomb-java-chassis/issues/2705#issuecomment-1042897959
https://github.com/apache/servicecomb-java-chassis/pull/2720
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@servicecomb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-java-chassis] liubao68 closed issue #2705: 多下游实例，高TPS下的性能问题
Posted by GitBox <gi...@apache.org>.
liubao68 closed issue #2705:
URL: https://github.com/apache/servicecomb-java-chassis/issues/2705
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@servicecomb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org