You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by GitBox <gi...@apache.org> on 2022/09/22 07:36:38 UTC

[GitHub] [rocketmq] Knowden opened a new issue, #5162: There may be a bug in DefaultMessageStore#getMessage

Knowden opened a new issue, #5162:
URL: https://github.com/apache/rocketmq/issues/5162

   `maxFilterMessageCount` is calculated as `maxMsgNums * ConsumeQueue.CQ_STORE_UNIT_SIZE` and it's seems like meaning the maxFilterMessageSize.
   
   ```java
   final int maxFilterMessageCount = Math.max(16000, maxMsgNums * ConsumeQueue.CQ_STORE_UNIT_SIZE);
   ```
   
   and it's used to judge wether this pull has scanned enough message as follow
   ```java
   if (cqUnit.getQueueOffset() - offset > maxFilterMessageCount) {
       break;
   }
   ```
   the cqUnit.queueOffset is logic offset of consumer queue which is calculated as follow
   ```java
   long queueOffset = (sbr.getStartOffset() + sbr.getByteBuffer().position() -  relativePos) / CQ_STORE_UNIT_SIZE;
   CqUnit cqUnit = new CqUnit(queueOffset,
           sbr.getByteBuffer().getLong(),
           sbr.getByteBuffer().getInt(),
           sbr.getByteBuffer().getLong());
   ```
   Is there something going wrong? or if I made some mistake when reading source code and submitting issue, please figure it out thanks
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [rocketmq] francisoliverlee commented on issue #5162: There may be a bug in DefaultMessageStore#getMessage

Posted by GitBox <gi...@apache.org>.
francisoliverlee commented on issue #5162:
URL: https://github.com/apache/rocketmq/issues/5162#issuecomment-1255312960

   For some pulled messages for maxMsgNums could be filter-out. so `Message Count ` is not a good way to limit.
   you can check code on 4.9.x
   https://github.com/apache/rocketmq/blob/c88686f13e6f4acef1f00c6d3e18589683ec3368/store/src/main/java/org/apache/rocketmq/store/DefaultMessageStore.java#L623
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [rocketmq] Knowden commented on issue #5162: There may be a bug in DefaultMessageStore#getMessage

Posted by GitBox <gi...@apache.org>.
Knowden commented on issue #5162:
URL: https://github.com/apache/rocketmq/issues/5162#issuecomment-1255751166

   https://github.com/apache/rocketmq/pull/5171


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [rocketmq] Knowden commented on issue #5162: There may be a bug in DefaultMessageStore#getMessage

Posted by GitBox <gi...@apache.org>.
Knowden commented on issue #5162:
URL: https://github.com/apache/rocketmq/issues/5162#issuecomment-1255741566

   > For some pulled messages for maxMsgNums could be filter-out. so `Message Count ` is not a good way to limit. Change Count-Limit to Bytes-Limit, you can check code on 4.9.x.
   > 
   > https://github.com/apache/rocketmq/blob/c88686f13e6f4acef1f00c6d3e18589683ec3368/store/src/main/java/org/apache/rocketmq/store/DefaultMessageStore.java#L623
   > 
   > but it seems need some updates on
   > 
   > ```java
   > if ((cqUnit.getQueueOffset() - offset) *  CQ_STORE_UNIT_SIZE  > maxFilterMessageCount) {
   >     break;
   > }
   > ```
   ok. I will submit a PR to fix this later on.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [rocketmq] HScarb commented on issue #5162: There may be a bug in DefaultMessageStore#getMessage

Posted by GitBox <gi...@apache.org>.
HScarb commented on issue #5162:
URL: https://github.com/apache/rocketmq/issues/5162#issuecomment-1255729563

   IMO you're right.
   `maxFilterMessageCount` is physical offset of `ConsumeQueue` file, while `cqUnit.getQueueOffset() - offset` is logical offset


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [rocketmq] ni-ze commented on issue #5162: There may be a bug in DefaultMessageStore#getMessage

Posted by GitBox <gi...@apache.org>.
ni-ze commented on issue #5162:
URL: https://github.com/apache/rocketmq/issues/5162#issuecomment-1257396488

   LGTM~


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [rocketmq] RongtongJin closed issue #5162: There may be a bug in DefaultMessageStore#getMessage

Posted by GitBox <gi...@apache.org>.
RongtongJin closed issue #5162: There may be a bug in DefaultMessageStore#getMessage
URL: https://github.com/apache/rocketmq/issues/5162


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org