You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@rocketmq.apache.org by GitBox <gi...@apache.org> on 2022/11/01 12:09:59 UTC

[GitHub] [rocketmq] cdmaji opened a new issue, #5445: 线上环境出现的一次resetoffset的bug

cdmaji opened a new issue, #5445:
URL: https://github.com/apache/rocketmq/issues/5445

   版本：v4.2.0+
   问题描述：通过resetOffsetByTime重置位置点，发现重置位置点不能准确定位到目标的位置。
   相关代码：org.apache.rocketmq.store.ConsumeQueue getOffsetInQueueByTime()
   org.apache.rocketmq.store.CommitLog putMessages(final MessageExtBatch messageExtBatch)
   关于生产消息的时候，如果是batch生产消息，在存储和落盘的时候这一批消息的BornTimestamp是一致的：
   ![image](https://user-images.githubusercontent.com/13196820/199229249-3f254779-9fe8-4824-abcc-cf7af9fb1212.png)
   对于消息整体流程来说是合理的，因为一批的消息确实应该是一个时间。但是在重置位置点的时候，实际是按照BornTimestamp去做的二分查找：
   ![image](https://user-images.githubusercontent.com/13196820/199229336-a6142a2a-438c-4069-969b-712fed8da730.png)
   这样会导致概率性的出现重置位置点不准确的问题（比如大部分的需求都是重置到这一批消息的第一条消息，但是按照二分算法，和消息条数有关系完全有可能处于中间的某条）。在我们实际线上环境中，业务方会比较纠结这个问题，总是重置不到理想位置。
   针对这个问题，其实有两种解决思路：
   1、修改broker端存储BornTimestamp；
   2、修改二分算法，利用二分结果再去左右匹配直到匹配到左边第一条；
   
   个人比较建议第二种方式，虽然性能上略有损耗，但是比较resetOffset是一个运维工具，没有必要去改变存储，而且如果修改BornTimestamp后也不太能判断N条消息是否为一批消息。


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] xiaoyifang commented on issue #5445: 线上环境出现的一次resetoffset的bug

Posted by GitBox <gi...@apache.org>.
xiaoyifang commented on issue #5445:
URL: https://github.com/apache/rocketmq/issues/5445#issuecomment-1302858171

   seems a duplicate of https://github.com/apache/rocketmq/issues/4890


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] 线上环境出现的一次resetoffset的bug [rocketmq]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #5445: 线上环境出现的一次resetoffset的bug
URL: https://github.com/apache/rocketmq/issues/5445


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] xiaoyifang commented on issue #5445: 线上环境出现的一次resetoffset的bug

Posted by GitBox <gi...@apache.org>.
xiaoyifang commented on issue #5445:
URL: https://github.com/apache/rocketmq/issues/5445#issuecomment-1302147238

   这种效果和命中相同的纪录后，向左继续查找1个不相等的数据，结果上是等效的。


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] 线上环境出现的一次resetoffset的bug [rocketmq]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #5445:
URL: https://github.com/apache/rocketmq/issues/5445#issuecomment-1802940883

   This issue was closed because it has been inactive for 3 days since being marked as stale.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] xiaoyifang commented on issue #5445: 线上环境出现的一次resetoffset的bug

Posted by GitBox <gi...@apache.org>.
xiaoyifang commented on issue #5445:
URL: https://github.com/apache/rocketmq/issues/5445#issuecomment-1301805747

   what about this way?
   when invoke `resetOffsetByTime timestamp`
   in the backend actually execute   `resetOffsetByTime timestamp-0.5`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] nowinkeyy commented on issue #5445: 线上环境出现的一次resetoffset的bug

Posted by GitBox <gi...@apache.org>.
nowinkeyy commented on issue #5445:
URL: https://github.com/apache/rocketmq/issues/5445#issuecomment-1298424225

   Nice catch! Are you willing to submit a PR to fix it? 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] cdmaji commented on issue #5445: 线上环境出现的一次resetoffset的bug

Posted by GitBox <gi...@apache.org>.
cdmaji commented on issue #5445:
URL: https://github.com/apache/rocketmq/issues/5445#issuecomment-1299447050

   Yes of course.I will try to fix it with the 2th solution.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] lizhimins commented on issue #5445: 线上环境出现的一次resetoffset的bug

Posted by GitBox <gi...@apache.org>.
lizhimins commented on issue #5445:
URL: https://github.com/apache/rocketmq/issues/5445#issuecomment-1299909311

   在复杂的生产环境下，比如说对应的时间点有 ha 切换，时间可能会回退的更小，所以依赖这个时间也不是很准确。我们总是希望通过幂等的处理逻辑来更好的支持重新消费，比如说希望重置到 2:00，这时候一般会选择 1:55 作为起始时间戳。
   当然这里提出的二分后向左扫描一下也很有意义的，欢迎提交 pr 来改进这个细节～


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] cdmaji commented on issue #5445: 线上环境出现的一次resetoffset的bug

Posted by GitBox <gi...@apache.org>.
cdmaji commented on issue #5445:
URL: https://github.com/apache/rocketmq/issues/5445#issuecomment-1301870864

   > what about this way? when invoke `resetOffsetByTime timestamp` in the backend actually execute `resetOffsetByTime timestamp-0.5` ![image](https://user-images.githubusercontent.com/105986/199682934-8ae279a6-7099-4b52-b3e2-4777467a234c.png)
   
   这种方式和使用命令时指定timestamp往前往后推N秒效果一样，其实并没有解决问题。


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] 线上环境出现的一次resetoffset的bug [rocketmq]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #5445:
URL: https://github.com/apache/rocketmq/issues/5445#issuecomment-1793586848

   This issue is stale because it has been open for 365 days with no activity. It will be closed in 3 days if no further activity occurs.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org