You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@rocketmq.apache.org by "AYue-94 (via GitHub)" <gi...@apache.org> on 2024/02/29 03:22:44 UTC
[I] [Bug] Tiered Store Query Message Async return different view each time [rocketmq]
AYue-94 opened a new issue, #7872:
URL: https://github.com/apache/rocketmq/issues/7872
### Before Creating the Bug Report
- [X] I found a bug, not just asking a question, which should be created in [GitHub Discussions](https://github.com/apache/rocketmq/discussions).
- [X] I have searched the [GitHub Issues](https://github.com/apache/rocketmq/issues) and [GitHub Discussions](https://github.com/apache/rocketmq/discussions) of this repository and believe that this is not a duplicate.
- [X] I have confirmed that this bug belongs to the current repository, not other repositories of RocketMQ.
### Runtime platform environment
macos
### RocketMQ version
5.2.0
### JDK Version
jdk8
### Describe the Bug
query message with key in tiered store return different result each time.
broker log:
```
# query result is ok
2024-02-29 11:01:58 INFO pool-2-thread-8 - MessageFetcher#queryMessageAsync, query result: 32, topic: TopicTest, topicId: 2, key: abcdefg, maxCount: 32, timestamp: 0-1709175717912
# query result is wrong
2024-02-29 11:02:01 INFO pool-2-thread-7 - MessageFetcher#queryMessageAsync, query result: 31, topic: TopicTest, topicId: 2, key: abcdefg, maxCount: 32, timestamp: 0-1709175721000
```
so client receive first time 32 msgs, and second time 31 msgs.
### Steps to Reproduce
1. broker
start broker with tiered store, and tieredStorageLevel = FORCE.
```
brokerClusterName=DefaultCluster
brokerName=broker-a
brokerId=0
deleteWhen=04
fileReservedTime=48
brokerRole=ASYNC_MASTER
flushDiskType=ASYNC_FLUSH
namesrvAddr=127.0.0.1:9876
messageStorePlugIn=org.apache.rocketmq.tieredstore.TieredMessageStore
tieredMetadataServiceProvider=org.apache.rocketmq.tieredstore.metadata.TieredMetadataManager
# custom segment use such as aliyunoss
tieredBackendServiceProvider=org.apache.rocketmq.tieredstore.provider.aliyun.AliyunFileSegment
objectStoreAccessKey=xxx
objectStoreSecretKey=xxx
objectStoreBucket=xxx
objectStoreEndpoint=xx
# force read from tiered store
tieredStorageLevel=FORCE
```
2. client
use org.apache.rocketmq.example.quickstart.Producer produce msgs , topic=TopicTest, key = abcdefg.
3. wait tiered buffer flush
for about 60s later...
4. client query message
```java
DefaultMQProducer producer = new DefaultMQProducer();
producer.setNamesrvAddr("127.0.0.1:9876");
producer.start();
while (true) {
QueryResult result = producer.queryMessage("TopicTest", "abcdefg", 32, 0, System.currentTimeMillis());
System.out.println(result.getMessageList().size());
if (result.getMessageList().size() < 32) {
System.out.println("error!!!");
break;
}
}
```
and you will see client log:
```
32
32
32
31
error!!!
```
### What Did You Expect to See?
query message async from tiered store should return consistent view.
### What Did You See Instead?
query message async from tiered store return different view each time.
### Additional Context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] Tiered Store Query Message Async return different view each time [rocketmq]
Posted by "lizhimins (via GitHub)" <gi...@apache.org>.
lizhimins closed issue #7872: [Bug] Tiered Store Query Message Async return different view each time
URL: https://github.com/apache/rocketmq/issues/7872
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] Tiered Store Query Message Async return different view each time [rocketmq]
Posted by "AYue-94 (via GitHub)" <gi...@apache.org>.
AYue-94 commented on issue #7872:
URL: https://github.com/apache/rocketmq/issues/7872#issuecomment-1970330171
when segment#read0 use different thread, QueryMessageResult#addMessage will encounter thread safety issues.
focus here
https://github.com/apache/rocketmq/blob/develop/tieredstore/src/main/java/org/apache/rocketmq/tieredstore/TieredMessageFetcher.java#L568
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] Tiered Store Query Message Async return different view each time [rocketmq]
Posted by "lizhimins (via GitHub)" <gi...@apache.org>.
lizhimins commented on issue #7872:
URL: https://github.com/apache/rocketmq/issues/7872#issuecomment-1971077463
Good catch
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] Tiered Store Query Message Async return different view each time [rocketmq]
Posted by "AYue-94 (via GitHub)" <gi...@apache.org>.
AYue-94 commented on issue #7872:
URL: https://github.com/apache/rocketmq/issues/7872#issuecomment-1970329447
when segment#read0 use different thread, QueryMessageResult#addMessage will encounter thread safety issues.
focus here
https://github.com/apache/rocketmq/blob/develop/tieredstore/src/main/java/org/apache/rocketmq/tieredstore/TieredMessageFetcher.java#L568
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] Tiered Store Query Message Async return different view each time [rocketmq]
Posted by "AYue-94 (via GitHub)" <gi...@apache.org>.
AYue-94 commented on issue #7872:
URL: https://github.com/apache/rocketmq/issues/7872#issuecomment-1970330724
i will try to fix it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org