You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by GitBox <gi...@apache.org> on 2022/09/22 07:36:38 UTC
[GitHub] [rocketmq] Knowden opened a new issue, #5162: There may be a bug in DefaultMessageStore#getMessage
Knowden opened a new issue, #5162:
URL: https://github.com/apache/rocketmq/issues/5162
`maxFilterMessageCount` is calculated as `maxMsgNums * ConsumeQueue.CQ_STORE_UNIT_SIZE` and it's seems like meaning the maxFilterMessageSize.
```java
final int maxFilterMessageCount = Math.max(16000, maxMsgNums * ConsumeQueue.CQ_STORE_UNIT_SIZE);
```
and it's used to judge wether this pull has scanned enough message as follow
```java
if (cqUnit.getQueueOffset() - offset > maxFilterMessageCount) {
break;
}
```
the cqUnit.queueOffset is logic offset of consumer queue which is calculated as follow
```java
long queueOffset = (sbr.getStartOffset() + sbr.getByteBuffer().position() - relativePos) / CQ_STORE_UNIT_SIZE;
CqUnit cqUnit = new CqUnit(queueOffset,
sbr.getByteBuffer().getLong(),
sbr.getByteBuffer().getInt(),
sbr.getByteBuffer().getLong());
```
Is there something going wrong? or if I made some mistake when reading source code and submitting issue, please figure it out thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] francisoliverlee commented on issue #5162: There may be a bug in DefaultMessageStore#getMessage
Posted by GitBox <gi...@apache.org>.
francisoliverlee commented on issue #5162:
URL: https://github.com/apache/rocketmq/issues/5162#issuecomment-1255312960
For some pulled messages for maxMsgNums could be filter-out. so `Message Count ` is not a good way to limit.
you can check code on 4.9.x
https://github.com/apache/rocketmq/blob/c88686f13e6f4acef1f00c6d3e18589683ec3368/store/src/main/java/org/apache/rocketmq/store/DefaultMessageStore.java#L623
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] Knowden commented on issue #5162: There may be a bug in DefaultMessageStore#getMessage
Posted by GitBox <gi...@apache.org>.
Knowden commented on issue #5162:
URL: https://github.com/apache/rocketmq/issues/5162#issuecomment-1255751166
https://github.com/apache/rocketmq/pull/5171
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] Knowden commented on issue #5162: There may be a bug in DefaultMessageStore#getMessage
Posted by GitBox <gi...@apache.org>.
Knowden commented on issue #5162:
URL: https://github.com/apache/rocketmq/issues/5162#issuecomment-1255741566
> For some pulled messages for maxMsgNums could be filter-out. so `Message Count ` is not a good way to limit. Change Count-Limit to Bytes-Limit, you can check code on 4.9.x.
>
> https://github.com/apache/rocketmq/blob/c88686f13e6f4acef1f00c6d3e18589683ec3368/store/src/main/java/org/apache/rocketmq/store/DefaultMessageStore.java#L623
>
> but it seems need some updates on
>
> ```java
> if ((cqUnit.getQueueOffset() - offset) * CQ_STORE_UNIT_SIZE > maxFilterMessageCount) {
> break;
> }
> ```
ok. I will submit a PR to fix this later on.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] HScarb commented on issue #5162: There may be a bug in DefaultMessageStore#getMessage
Posted by GitBox <gi...@apache.org>.
HScarb commented on issue #5162:
URL: https://github.com/apache/rocketmq/issues/5162#issuecomment-1255729563
IMO you're right.
`maxFilterMessageCount` is physical offset of `ConsumeQueue` file, while `cqUnit.getQueueOffset() - offset` is logical offset
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] ni-ze commented on issue #5162: There may be a bug in DefaultMessageStore#getMessage
Posted by GitBox <gi...@apache.org>.
ni-ze commented on issue #5162:
URL: https://github.com/apache/rocketmq/issues/5162#issuecomment-1257396488
LGTM~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] RongtongJin closed issue #5162: There may be a bug in DefaultMessageStore#getMessage
Posted by GitBox <gi...@apache.org>.
RongtongJin closed issue #5162: There may be a bug in DefaultMessageStore#getMessage
URL: https://github.com/apache/rocketmq/issues/5162
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org