[GitHub] [rocketmq] cserwen opened a new issue #3787: [POP] When consumption is slow, it is easy to cause repeated consumption due to failure

[GitBox <gi...@apache.org>]

cserwen opened a new issue #3787:
URL: https://github.com/apache/rocketmq/issues/3787


   **BUG REPORT**
   
   ### Please describe the issue you observed:
   
   - What did you do (The steps to reproduce)?
   
   > Use pop model to consume and the params is as fllows:
   ```bash
   #client
   consumeThreadMin: 1
   consumeThreadMax: 1
   popBatchNums: 32 (default value)
   popThresholdForQueue: 96 (default value)
   
   # broker
   enablePopBufferMerge: true
   enablePopLog: false
   Others is all default value
   
   ```
   > And the message listener is:
   ``` java
   msgs.forEach(System.out::println);
   try {
       Thread.sleep(100);
   } catch (Exception e) {
       e.printStackTrace();
   }
   return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
   ```
   > **So the consume qps is 10 for each client.**
   - What did you expect to see?
   
   > It should only be consumed once normally.
   
   - What did you see instead?
   
   > There are many messages that are considered to have failed to be consumed but are actually consumed successfully. So these messages are written to the retry queue and thus consumed again
   
   ### Please tell us about your environment:
   > 5.0.0-alpha
   ### Other information (e.g. detailed explanation, logs, related issues, suggestions how to fix, etc):
   
   This problem occurs when ck messages are written from buffer to store. When the ck message stays in the buffer for more than 8.5 seconds, the corresponding ack message can only be written to the store.
   https://github.com/apache/rocketmq/blob/52548056cc85381bf774e913c168ef8f1bd810ee/broker/src/main/java/org/apache/rocketmq/broker/processor/PopBufferMergeService.java#L494
   So the ack will be written to the topic firstly. The ck msg will be written when the time in the buffer exceeds 10s.
   https://github.com/apache/rocketmq/blob/52548056cc85381bf774e913c168ef8f1bd810ee/broker/src/main/java/org/apache/rocketmq/broker/processor/PopBufferMergeService.java#L236
   Because the ack and ck messages are in the same delay level, the previous ones will reach the `reviveTopic` firstly. This will cause the ack message to be delivered to `reviveTopic` earlier than the ck message.So `reviveService` thinks that the message has not been acked.
   
   **If we can support second-level delay messages, this problem can be easily solved.** Because the ck msg can be delivered earlier.
   https://github.com/apache/rocketmq/blob/52548056cc85381bf774e913c168ef8f1bd810ee/broker/src/main/java/org/apache/rocketmq/broker/processor/PopMessageProcessor.java#L662


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] lizhiboo commented on issue #3787: [POP] When consumption is slow, it is easy to cause repeated consumption due to failure

[GitBox <gi...@apache.org>]

lizhiboo commented on issue #3787:
URL: https://github.com/apache/rocketmq/issues/3787#issuecomment-1033468592


   @cserwen if we set the corresponding ack message timeout > 10s, that is longer than ck message timeout. Is this problem will be solved? BTW, i don't understand the adjust - 1500, do u know in which case it works?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] cserwen commented on issue #3787: [POP] When consumption is slow, it is easy to cause repeated consumption due to failure

[GitBox <gi...@apache.org>]

cserwen commented on issue #3787:
URL: https://github.com/apache/rocketmq/issues/3787#issuecomment-1035998737


   > @cserwen if we set the corresponding ack message timeout > 10s, that is longer than ck message timeout. Is this problem will be solved? BTW, i don't understand the adjust - 1500, do u know in which case it works?
   
   @lizhiboo  
   设置这个ack消息的超时时间超过10s，我不太明白这个怎么避免这个问题的发生:joy:
   
   关于这个 - 1.5s的判断，在添加ack消息时会进行判断， 如果ack消息对应的ck消息在buffer中待的时间超过了`brokerController.getBrokerConfig().getPopCkStayBufferTime() - 1500`，这个时间，ack消息就必须写入延时Topic中，即这个时候ck消息不能被ack了，等reviveService去处理这部分ack消息。 我认为这1.5s应该是为了不想要加锁，提前禁止ck消息被ack？


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org