You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@rocketmq.apache.org by "tju-yxq (via GitHub)" <gi...@apache.org> on 2026/08/12 18:07:34 UTC
[PR] fix: keep newest topic query messages across queues [rocketmq-dashboard]
tju-yxq opened a new pull request, #2043:
URL: https://github.com/apache/rocketmq-dashboard/pull/2043
## Summary
This PR fixes #1828 with a stronger topic-query ordering guarantee than simply sorting the records already collected from the first scanned queues.
`RocketMQMessageProvider.queryByTopic(...)` used to append messages in queue scan order and stop once the default topic-query limit was reached. That means a busy earlier queue could fill the limit before later queues were scanned, so newer records from later queues could be omitted entirely.
Changes:
- Keep a bounded min-heap of the newest topic-query records while scanning queues.
- Scan all returned message queues instead of stopping as soon as the first queues fill the result limit.
- Return the bounded candidates sorted by `storeTime` descending.
- Add focused tests for:
- newest-first ordering across two queues;
- preserving newer messages from later queues even when an earlier queue fills the default limit.
## Verification
```bash
cd server
mvn -DskipTests=false "-Dtest=RocketMQMessageProviderTest#queryByTopicSortsMessagesAcrossQueuesNewestFirst+queryByTopicKeepsNewestMessagesWhenEarlierQueuesFillTheDefaultLimit+queryByTopicStopsWhenPullOffsetDoesNotAdvance" test
```
Result: `BUILD SUCCESS`; `Tests run: 3, Failures: 0, Errors: 0, Skipped: 0`.
## Evidence boundary
- Verified by code inspection and mocked unit tests around `DefaultMQPullConsumer`.
- Not verified with a live multi-queue RocketMQ broker or browser UI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: keep newest topic query messages across queues [rocketmq-dashboard]
Posted by "RockteMQ-AI (via GitHub)" <gi...@apache.org>.
RockteMQ-AI commented on code in PR #2043:
URL: https://github.com/apache/rocketmq-dashboard/pull/2043#discussion_r3769322174
##########
server/src/main/java/org/apache/rocketmq/studio/provider/apache/RocketMQMessageProvider.java:
##########
@@ -191,22 +192,20 @@ private List<MessageRecordVO> queryByKey(DefaultMQAdminExt adminExt, String topi
*/
private List<MessageRecordVO> queryByTopic(String endpoint, String topic, String tag, long begin, long end, int limit) {
DefaultMQPullConsumer consumer = newPullConsumer("studio-msg-query", endpoint);
- List<MessageRecordVO> result = new ArrayList<>();
+ int resultLimit = Math.min(limit, TOPIC_QUERY_HARD_CAP);
+ PriorityQueue<MessageRecordVO> newestMessages = new PriorityQueue<>(
+ Comparator.comparingLong(MessageRecordVO::getStoreTime));
try {
Review Comment:
Removing the early-exit (`break outer`) means every queue is now fully scanned. For topics with a large number of messages this could be significantly slower than the previous implementation. Consider documenting the trade-off or adding a per-queue offset budget (e.g. cap the number of offsets scanned per queue proportionally to `resultLimit / queues.size()`) so that a single hot queue cannot dominate the scan time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: keep newest topic query messages across queues [rocketmq-dashboard]
Posted by "lizhimins (via GitHub)" <gi...@apache.org>.
lizhimins merged PR #2043:
URL: https://github.com/apache/rocketmq-dashboard/pull/2043
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org