[GitHub] [dubbo] shenhuaxin removed a comment on issue #9257: 连接Zookeeper , ConnLoss 导致内存泄露的问题

[GitBox <gi...@apache.org>]

shenhuaxin removed a comment on issue #9257:
URL: https://github.com/apache/dubbo/issues/9257#issuecomment-967791219


   使用 tc 命令模拟网络异常， 将zk服务端的包延时设置为了5秒钟， 并将jvm的堆设置小一点， 成功复现了一次。
   将日志级别调整为debug , 发现在重连阶段，产生了大量的 WatchEvent 事件。（3500条）
   ```java
   2021-11-12 21:26:12 | siac |  | main-EventThread | DEBUG | o.apache.curator.framework.recipes.cache.TreeCache |  | TreeCache.java:341 | process: WatchedEvent state:Disconnected type:None path:null
   ```
   Event事件打印完毕之后就是打印ConnLoss日志
   ```java
   2021-11-12 21:26:13 | siac | evcard-goods-back | Curator-Framework-0 | ERROR | o.a.curator.framework.imps.CuratorFrameworkImpl |  | CuratorFrameworkImpl.java:557 | Background operation retry gave up
   org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss
   	at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
   	at org.apache.curator.framework.imps.CuratorFrameworkImpl.checkBackgroundRetry(CuratorFrameworkImpl.java:728)
   	at org.apache.curator.framework.imps.CuratorFrameworkImpl.performBackgroundOperation(CuratorFrameworkImpl.java:857)
   	at org.apache.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop(CuratorFrameworkImpl.java:809)
   	at org.apache.curator.framework.imps.CuratorFrameworkImpl.access$300(CuratorFrameworkImpl.java:64)
   	at org.apache.curator.framework.imps.CuratorFrameworkImpl$4.call(CuratorFrameworkImpl.java:267)
   	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
   	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
   	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   	at java.lang.Thread.run(Thread.java:748)
   ```
   
   上述两个步骤不断的执行，EventThread 中的 waitingEvents队列的消费速度跟不上生产速度，最终导致了OOM。
   特别是在cpu压力大的情况下，很容易复现这个问题。
   
   
   感觉和这篇文章提到的大量注册Watcher比较类似。
   https://www.codeleading.com/article/37613832153/
   https://issues.apache.org/jira/browse/ZOOKEEPER-706
   
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org