You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by GitBox <gi...@apache.org> on 2021/01/06 02:48:20 UTC

[GitHub] [rocketmq-client-go] NAND86 opened a new issue #587: [Master] consumeMessageOrderly goroutine leaks & CPU100%

NAND86 opened a new issue #587:
URL: https://github.com/apache/rocketmq-client-go/issues/587


   **BUG REPORT**  
   
   1. Please describe the issue you observed:
   
   CPU升至200%（200、300都有可能），疯狂打印日志"the message queue not be able to consume, because it's dropped."
   同一消费者组多个 顺序消费者 ，A先启动，过几秒再启动B，A几乎必现这个问题。
   现象应该类似于 #575 的第一个问题。
   
   2. Please tell us about your environment:
    
        - What is your OS? linux
        - What is your client version? 2.0.0
        - What is your RocketMQ version? 4.3.2
   
   3. Other information (e.g. detailed explanation, logs, related issues, suggestions on how to fix, etc):
   
   每次balance的时候，如果分到新的队列，会向通道写入一个PullRequest（dc.prCh<-pr）。
   
   在另一边，Start的时候会创建协程循环从dc.prCh读取pr，每读到一个就会开启协程处理这个PullRequest（pc.pullMessage(&pr)）。pullMessage处理时，首先会开启一个协程循环去submitToConsume（顺序消费者是consumeMessageOrderly）。
   
   注意看这个循环的退出条件。
   https://github.com/apache/rocketmq-client-go/blob/f07155bab3e072ed76fb86d3d1e5772f9c198897/consumer/push_consumer.go#L458-L470
   
   如果A一开始被分了10个队列，在B启动后会重新分配队列，A会分走5个给B。此时A会在被分走的队列上WithDropped，在pullMessage这个协程中，如果发现队列已经Drop，协程会退出。但是在上面的循环协程中，这个循环只有shutdown才会退出。所以它仍然会调用consumeMessageOrderly，每次调用都会发现队列已经被drop，于是打印日志后返回，继续死循环。所以会造成协程泄露和cpu100%的现象。
   
   这个问题还有另外一个现象，如果订阅的topic一直没有消息，协程会卡在consumeMessageOrderly 内部 的takeMessages中，一直for循环等待msgCache不为空。这就是 #580 的问题，但是 #580 协程泄露这么明显主要是异常测试中同一消费者组内的顺序消费者频繁启停，导致一直存活的客户端一直新增队列删除队列。
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq-client-go] ShannonDing closed issue #587: [Master] consumeMessageOrderly goroutine leaks & CPU100%

Posted by GitBox <gi...@apache.org>.
ShannonDing closed issue #587:
URL: https://github.com/apache/rocketmq-client-go/issues/587


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq-client-go] ShannonDing commented on issue #587: [Master] consumeMessageOrderly goroutine leaks & CPU100%

Posted by GitBox <gi...@apache.org>.
ShannonDing commented on issue #587:
URL: https://github.com/apache/rocketmq-client-go/issues/587#issuecomment-755043141


   link #574 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org