You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by GitBox <gi...@apache.org> on 2020/10/16 02:19:36 UTC
[GitHub] [rocketmq] xiaobing395845388 opened a new issue #2357: too many %RETRY%topic message in the commitlog
xiaobing395845388 opened a new issue #2357:
URL: https://github.com/apache/rocketmq/issues/2357
The issue tracker is **ONLY** used for bug report(feature request need to follow [RIP process](https://github.com/apache/rocketmq/wiki/RocketMQ-Improvement-Proposal)). Keep in mind, please check whether there is an existing same report before your raise a new one.
Alternately (especially if your communication is not a bug report), you can send mail to our [mailing lists](http://rocketmq.apache.org/about/contact/). We welcome any friendly suggestions, bug fixes, collaboration and other improvements.
Please ensure that your bug report is clear and that it is complete. Otherwise, we may be unable to understand it or to reproduce it, either of which would prevent us from fixing the bug. We strongly recommend the report(bug report or feature request) could include some hints as the following:
**BUG REPORT**
1. Please describe the issue you observed:
in our product production environment ,there are three brokers in dledger mode,we found that the dledger data in store dir increased quickly,ten minutes increased one GB,we edit the commitlog and found that more then 4/5 of messages were "%retry%" message which consumer consume message failed. The retry message were always those five or six message which consumer consume message failed
2. Please tell us about your environment:
three brokers、dledger mode、concurrently consumer
3. Other information (e.g. detailed explanation, logs, related issues, suggestions how to fix, etc):
**FEATURE REQUEST**
1. Please describe the feature you are requesting.
2. Provide any additional detail on your proposed use case for this feature.
2. Indicate the importance of this issue to you (blocker, must-have, should-have, nice-to-have). Are you currently using any workarounds to address this issue?
4. If there are some sub-tasks using -[] for each subtask and create a corresponding issue to map to the sub task:
- [sub-task1-issue-number](example_sub_issue1_link_here): sub-task1 description here,
- [sub-task2-issue-number](example_sub_issue2_link_here): sub-task2 description here,
- ...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] xiaobing395845388 removed a comment on issue #2357: too many %RETRY%topic message in the commitlog
Posted by GitBox <gi...@apache.org>.
xiaobing395845388 removed a comment on issue #2357:
URL: https://github.com/apache/rocketmq/issues/2357#issuecomment-721463160
> > > > > > This problem can be repetition in test environment by consumer consume failed and return reconsume_later status, then shutdown broker0 master .mq version is 4.5.2
> > > > >
> > > > >
> > > > > This is normal if consumer return reconsume_later status and the message send to retry topic even in master-slave mode.
> > > >
> > > >
> > > > you can test on it in my described envionment,you can found that one message will be retried times then before
> > >
> > >
> > > OK, I will try it
> >
> >
> > Hi. Is there any progress?
>
> Need more info of this issue.
> What do you exactly mean of too many %RETRY%topic message in the commitlog?
>
> By the way, if the consumer always return RECONSUME_LATER, it will retry 16 times by default.
you can query in the console by the message id, one message will retry more then 16 times, times then 16 times,
also you can vim the dledger commitlog and search the retry message
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] xiaobing395845388 edited a comment on issue #2357: too many %RETRY%topic message in the commitlog
Posted by GitBox <gi...@apache.org>.
xiaobing395845388 edited a comment on issue #2357:
URL: https://github.com/apache/rocketmq/issues/2357#issuecomment-716259647
> > This problem can be repetition in test environment by consumer consume failed and return reconsume_later status, then shutdown broker0 master .mq version is 4.5.2
>
> This is normal if consumer return reconsume_later status and the message send to retry topic even in master-slave mode.
you can test on it in my described envionment,you can found that one message will be retried times then before
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] xiaobing395845388 commented on issue #2357: Delay messages and retry messages will be consumed repeatedly when master changed
Posted by GitBox <gi...@apache.org>.
xiaobing395845388 commented on issue #2357:
URL: https://github.com/apache/rocketmq/issues/2357#issuecomment-751673049
> Sorry for the late reply, I submitted a PR to fix this issue, you can have a try.
> #2518
I will try right now,thanks !
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] xiangwangcheng edited a comment on issue #2357: too many %RETRY%topic message in the commitlog
Posted by GitBox <gi...@apache.org>.
xiangwangcheng edited a comment on issue #2357:
URL: https://github.com/apache/rocketmq/issues/2357#issuecomment-721041982
> > > > > This problem can be repetition in test environment by consumer consume failed and return reconsume_later status, then shutdown broker0 master .mq version is 4.5.2
> > > >
> > > >
> > > > This is normal if consumer return reconsume_later status and the message send to retry topic even in master-slave mode.
> > >
> > >
> > > you can test on it in my described envionment,you can found that one message will be retried times then before
> >
> >
> > OK, I will try it
>
> Hi. Is there any progress?
Need more info of this issue.
What do you exactly mean of too many %RETRY%topic message in the commitlog?
By the way, if the consumer always return RECONSUME_LATER, it will retry 16 times by default.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] RongtongJin commented on issue #2357: too many %RETRY%topic message in the commitlog
Posted by GitBox <gi...@apache.org>.
RongtongJin commented on issue #2357:
URL: https://github.com/apache/rocketmq/issues/2357#issuecomment-713528085
> This problem can be repetition in test environment by consumer consume failed and return reconsume_later status, then shutdown broker0 master .mq version is 4.5.2
This is normal if consumer return reconsume_later status and the message send to retry topic even in master-slave mode.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] xiaobing395845388 commented on issue #2357: too many %RETRY%topic message in the commitlog
Posted by GitBox <gi...@apache.org>.
xiaobing395845388 commented on issue #2357:
URL: https://github.com/apache/rocketmq/issues/2357#issuecomment-711831361
> Does retry topic are really not sent successfully in your production?
Retry topic are sent successfully
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] xiaobing395845388 edited a comment on issue #2357: too many %RETRY%topic message in the commitlog
Posted by GitBox <gi...@apache.org>.
xiaobing395845388 edited a comment on issue #2357:
URL: https://github.com/apache/rocketmq/issues/2357#issuecomment-712740641
@vongosling @RongtongJin any ideas?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] xiaobing395845388 edited a comment on issue #2357: too many %RETRY%topic message in the commitlog
Posted by GitBox <gi...@apache.org>.
xiaobing395845388 edited a comment on issue #2357:
URL: https://github.com/apache/rocketmq/issues/2357#issuecomment-721465688
> > > > > > > This problem can be repetition in test environment by consumer consume failed and return reconsume_later status, then shutdown broker0 master .mq version is 4.5.2
> > > > > >
> > > > > >
> > > > > > This is normal if consumer return reconsume_later status and the message send to retry topic even in master-slave mode.
> > > > >
> > > > >
> > > > > you can test on it in my described envionment,you can found that one message will be retried times then before
> > > >
> > > >
> > > > OK, I will try it
> > >
> > >
> > > Hi. Is there any progress?
> >
> >
> > Need more info of this issue.
> > What do you exactly mean of too many %RETRY%topic message in the commitlog?
> > By the way, if the consumer always return RECONSUME_LATER, it will retry 16 times by default.
>
you can query in the console by the message id, one message will retry more then 16 times, times then 16 times
normally it retrys by delaytimelevel total 16 times, but actually it retry times at the same moment
also you can vim the dledger commitlog and search the retry message
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] xiaobing395845388 edited a comment on issue #2357: too many %RETRY%topic message in the commitlog
Posted by GitBox <gi...@apache.org>.
xiaobing395845388 edited a comment on issue #2357:
URL: https://github.com/apache/rocketmq/issues/2357#issuecomment-721463160
> > > > > > This problem can be repetition in test environment by consumer consume failed and return reconsume_later status, then shutdown broker0 master .mq version is 4.5.2
> > > > >
> > > > >
> > > > > This is normal if consumer return reconsume_later status and the message send to retry topic even in master-slave mode.
> > > >
> > > >
> > > > you can test on it in my described envionment,you can found that one message will be retried times then before
> > >
> > >
> > > OK, I will try it
> >
> >
> > Hi. Is there any progress?
>
> Need more info of this issue.
> What do you exactly mean of too many %RETRY%topic message in the commitlog?
>
> By the way, if the consumer always return RECONSUME_LATER, it will retry 16 times by default.
you can query in the console by the message id, one message will retry more then 16 times, times then 16 times,
also you can vim the dledger commitlog and search the retry message
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] xiaobing395845388 commented on issue #2357: too many %RETRY%topic message in the commitlog
Posted by GitBox <gi...@apache.org>.
xiaobing395845388 commented on issue #2357:
URL: https://github.com/apache/rocketmq/issues/2357#issuecomment-712740641
@vongosling any ideas?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] vongosling commented on issue #2357: too many %RETRY%topic message in the commitlog
Posted by GitBox <gi...@apache.org>.
vongosling commented on issue #2357:
URL: https://github.com/apache/rocketmq/issues/2357#issuecomment-711458697
Does retry topic are really not sent successfully in your production?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] xiaobing395845388 commented on issue #2357: too many %RETRY%topic message in the commitlog
Posted by GitBox <gi...@apache.org>.
xiaobing395845388 commented on issue #2357:
URL: https://github.com/apache/rocketmq/issues/2357#issuecomment-721463160
> > > > > > This problem can be repetition in test environment by consumer consume failed and return reconsume_later status, then shutdown broker0 master .mq version is 4.5.2
> > > > >
> > > > >
> > > > > This is normal if consumer return reconsume_later status and the message send to retry topic even in master-slave mode.
> > > >
> > > >
> > > > you can test on it in my described envionment,you can found that one message will be retried times then before
> > >
> > >
> > > OK, I will try it
> >
> >
> > Hi. Is there any progress?
>
> Need more info of this issue.
> What do you exactly mean of too many %RETRY%topic message in the commitlog?
>
> By the way, if the consumer always return RECONSUME_LATER, it will retry 16 times by default.
you can query in the console by the message id, one message will retry more then 16 times, times then 16 times,
also you can vim the dledger commitlog and serch the retry message
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] xiaobing395845388 commented on issue #2357: too many %RETRY%topic message in the commitlog
Posted by GitBox <gi...@apache.org>.
xiaobing395845388 commented on issue #2357:
URL: https://github.com/apache/rocketmq/issues/2357#issuecomment-720916363
> > > > This problem can be repetition in test environment by consumer consume failed and return reconsume_later status, then shutdown broker0 master .mq version is 4.5.2
> > >
> > >
> > > This is normal if consumer return reconsume_later status and the message send to retry topic even in master-slave mode.
> >
> >
> > you can test on it in my described envionment,you can found that one message will be retried times then before
>
> OK, I will try it
Hi. Is there any progress?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] vongosling edited a comment on issue #2357: too many %RETRY%topic message in the commitlog
Posted by GitBox <gi...@apache.org>.
vongosling edited a comment on issue #2357:
URL: https://github.com/apache/rocketmq/issues/2357#issuecomment-711458697
Is the retry topic really not successfully sent in your production?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] xiangwangcheng commented on issue #2357: too many %RETRY%topic message in the commitlog
Posted by GitBox <gi...@apache.org>.
xiangwangcheng commented on issue #2357:
URL: https://github.com/apache/rocketmq/issues/2357#issuecomment-721041982
> > > > > This problem can be repetition in test environment by consumer consume failed and return reconsume_later status, then shutdown broker0 master .mq version is 4.5.2
> > > >
> > > >
> > > > This is normal if consumer return reconsume_later status and the message send to retry topic even in master-slave mode.
> > >
> > >
> > > you can test on it in my described envionment,you can found that one message will be retried times then before
> >
> >
> > OK, I will try it
>
> Hi. Is there any progress?
Need more info of this issue.
What do you exactly mean of too many %RETRY%topic message in the commitlog?
By the way, if the consumer always return RECONSUME_LATER, it will retry 16 times by default.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] xiaobing395845388 closed issue #2357: Delay messages and retry messages will be consumed repeatedly when master changed
Posted by GitBox <gi...@apache.org>.
xiaobing395845388 closed issue #2357:
URL: https://github.com/apache/rocketmq/issues/2357
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] xiaobing395845388 commented on issue #2357: too many %RETRY%topic message in the commitlog
Posted by GitBox <gi...@apache.org>.
xiaobing395845388 commented on issue #2357:
URL: https://github.com/apache/rocketmq/issues/2357#issuecomment-721465688
> > > > > > > This problem can be repetition in test environment by consumer consume failed and return reconsume_later status, then shutdown broker0 master .mq version is 4.5.2
> > > > > >
> > > > > >
> > > > > > This is normal if consumer return reconsume_later status and the message send to retry topic even in master-slave mode.
> > > > >
> > > > >
> > > > > you can test on it in my described envionment,you can found that one message will be retried times then before
> > > >
> > > >
> > > > OK, I will try it
> > >
> > >
> > > Hi. Is there any progress?
> >
> >
> > Need more info of this issue.
> > What do you exactly mean of too many %RETRY%topic message in the commitlog?
> > By the way, if the consumer always return RECONSUME_LATER, it will retry 16 times by default.
>
> you can query in the console by the message id, one message will retry more then 16 times, times then 16 times
> normally it retrys by delaytimelevel total 16 times, but actually it retry times at the same moment
> also you can vim the dledger commitlog and search the retry message
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] RongtongJin commented on issue #2357: too many %RETRY%topic message in the commitlog
Posted by GitBox <gi...@apache.org>.
RongtongJin commented on issue #2357:
URL: https://github.com/apache/rocketmq/issues/2357#issuecomment-716262209
> > > This problem can be repetition in test environment by consumer consume failed and return reconsume_later status, then shutdown broker0 master .mq version is 4.5.2
> >
> >
> > This is normal if consumer return reconsume_later status and the message send to retry topic even in master-slave mode.
>
> you can test on it in my described envionment,you can found that one message will be retried times then before
OK, I will try it
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] xiaobing395845388 commented on issue #2357: too many %RETRY%topic message in the commitlog
Posted by GitBox <gi...@apache.org>.
xiaobing395845388 commented on issue #2357:
URL: https://github.com/apache/rocketmq/issues/2357#issuecomment-709967060
I want to make some supplements to above ,it occurd after restart one of broker and lead a new broker
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] RongtongJin edited a comment on issue #2357: delay messages and retry messages will be consumed repeatedly when master changed
Posted by GitBox <gi...@apache.org>.
RongtongJin edited a comment on issue #2357:
URL: https://github.com/apache/rocketmq/issues/2357#issuecomment-751425098
Sorry for the late reply, I submitted a PR to fix this issue, you can have a try.
https://github.com/apache/rocketmq/pull/2518
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] xiaobing395845388 commented on issue #2357: too many %RETRY%topic message in the commitlog
Posted by GitBox <gi...@apache.org>.
xiaobing395845388 commented on issue #2357:
URL: https://github.com/apache/rocketmq/issues/2357#issuecomment-711465181
This problem can be repetition in test environment by consumer consume failed and return reconsume_later status, then shutdown broker0 master .mq version is 4.5.2
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] xiaobing395845388 commented on issue #2357: too many %RETRY%topic message in the commitlog
Posted by GitBox <gi...@apache.org>.
xiaobing395845388 commented on issue #2357:
URL: https://github.com/apache/rocketmq/issues/2357#issuecomment-716259647
> > This problem can be repetition in test environment by consumer consume failed and return reconsume_later status, then shutdown broker0 master .mq version is 4.5.2
>
> This is normal if consumer return reconsume_later status and the message send to retry topic even in master-slave mode.
you can test on it in my described envionment,you can found that one message will retry times then before
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] RongtongJin commented on issue #2357: delay messages and retry messages will be consumed repeatedly when master changed
Posted by GitBox <gi...@apache.org>.
RongtongJin commented on issue #2357:
URL: https://github.com/apache/rocketmq/issues/2357#issuecomment-751425098
Sorry for the late reply, I submitted a PR to fix this issue, you can have a try
https://github.com/apache/rocketmq/pull/2518
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] xiaobing395845388 commented on issue #2357: Delay messages and retry messages will be consumed repeatedly when master changed
Posted by GitBox <gi...@apache.org>.
xiaobing395845388 commented on issue #2357:
URL: https://github.com/apache/rocketmq/issues/2357#issuecomment-789674926
> > Sorry for the late reply, I submitted a PR to fix this issue, you can have a try.
> > #2518
>
> I will try right now,thanks !
Had resolved our problem in product env, thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org