You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@rocketmq.apache.org by "tongtaodragon (via GitHub)" <gi...@apache.org> on 2024/06/24 13:06:01 UTC
[I] [Bug] The consumer minOffSet keeps no change before restart container [rocketmq]
tongtaodragon opened a new issue, #8327:
URL: https://github.com/apache/rocketmq/issues/8327
### Before Creating the Bug Report
- [X] I found a bug, not just asking a question, which should be created in [GitHub Discussions](https://github.com/apache/rocketmq/discussions).
- [X] I have searched the [GitHub Issues](https://github.com/apache/rocketmq/issues) and [GitHub Discussions](https://github.com/apache/rocketmq/discussions) of this repository and believe that this is not a duplicate.
- [X] I have confirmed that this bug belongs to the current repository, not other repositories of RocketMQ.
### Runtime platform environment
Redhat 8.0
### RocketMQ version
Broker: 4.8.0
Client SDK: 4.9.3
### JDK Version
Orace JDK 1.8.0_121
### Describe the Bug
In our product environment, we met this issue for several times but can't reproduce always. It happened during upgrading. We have several container instances in environment. During upgrade we adopt rolling upgrade. When it happened, the consumer minOffset of one queue in one broker always keeps fixed value, but the consumer still can consumer later messages successfully. And the number of cumulative messages grow bigger. After we restart the problematic consumer instance, this issue disappeared.
1. In broker log, we didn't find any warn or error messages related to this.
2. In client log, we didn't find any warn or error messages related to this. Since the number of cumulative messages is larger than 2000 which is larger than maxSpan so in client log, some warn flow control log message logged.
3. We compared the cleanExpireMessage thread between problematic and normal instance, cleanExpiredMessage thread work well. We found an old issue which is related to cleanExpiredMessage thread, it will result in this issue, but it was fixed in SDK 4.9.3.
### Steps to Reproduce
We doubt one consume thread exit abnormally and doesn't set ConsumeStartTimeStamp. Then we reproduce it using below hack method.
In ConsumeMessageConcurrentlyService.java, we change code.
1. Add one member as below
private static int count = 0;
2. Add code in methos run of class ConsumeRequest as below
@Override
public void run() {
if (this.processQueue.isDropped()) {
log.info("xxxx");
return;
}
// Add below code
if (count == 0) {
count++;
return;
}
1. Recompile rocketmq-client jar and install
2. Produce some messages in queue
3. Start one consumer instance, then we can found issue happened.
### What Did You Expect to See?
After some time, this queue will be normal, minOffset keep grow normally and cumulative messages be normal. We don't want to restart consumer instance since it will consume old messages again.
### What Did You See Instead?
minOffset keep a fixed value and no grow, the cumulative messages grow bigger.
### Additional Context
In our case
1. clustering consume
2. DefaultPushConsume
3. Concurrently consume
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] The consumer minOffSet keeps no change before restart container [rocketmq]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #8327:
URL: https://github.com/apache/rocketmq/issues/8327#issuecomment-3016176463
This issue was closed because it has been inactive for 3 days since being marked as stale.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] The consumer minOffSet keeps no change before restart container [rocketmq]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #8327:
URL: https://github.com/apache/rocketmq/issues/8327#issuecomment-3006585172
This issue is stale because it has been open for 365 days with no activity. It will be closed in 3 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] The consumer minOffSet keeps no change before restart container [rocketmq]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #8327: [Bug] The consumer minOffSet keeps no change before restart container
URL: https://github.com/apache/rocketmq/issues/8327
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org