You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@rocketmq.apache.org by "tju-yxq (via GitHub)" <gi...@apache.org> on 2026/09/04 19:07:38 UTC
[PR] perf(dlq): parallelize group stats and debounce search [rocketmq-dashboard]
tju-yxq opened a new pull request, #3168:
URL: https://github.com/apache/rocketmq-dashboard/pull/3168
## What happened
The DLQ group page made its latency worse than necessary in two places.
Backend:
- `RocketMQDLQProvider.listDLQGroups(...)` sliced the current page and then called `adminExt.examineTopicStats(dlqTopic)` for every `%DLQ%` topic serially.
- With the default page size of 20, the page waited for up to 20 sequential admin RPCs. Slow brokers made this especially visible.
Frontend:
- The search input updated `search` state on every keystroke.
- That state is in the group-loading effect's dependency list, so every character started a new backend group-list request. The request-generation guard prevented stale responses from overwriting newer data, but it did not prevent the redundant requests from being sent.
## Change
Backend:
- Load topic statistics for the current page on a bounded daemon thread pool with 8 workers.
- Preserve the current deterministic page order by collecting futures in input order.
- Cap each stats lookup at 5 seconds; cancel timed-out work.
- Preserve row-level degradation: a failed or timed-out stats call only marks that row `UNAVAILABLE`.
- Shut down the stats executor on bean destruction.
Frontend:
- Split visible search input from the search value used for requests.
- Debounce typing by 300 ms.
- Submit immediately when the user presses Enter or clicks the search action.
- Clear pending debounce timers on unmount.
Tests:
- Backend test proves three simulated 150 ms stats calls complete in less than the serial sum, preserve order, and still call stats once per topic.
- Frontend test proves two rapid keystrokes produce only one request and that the request uses the final search value.
## Verification
Focused backend:
```
cd server
mvn -q '-Dtest=RocketMQDLQProviderTest,DLQServiceTest,DLQControllerTest' test
mvn -q checkstyle:check
```
Focused frontend:
```
cd web
npm test -- DLQPage.test.tsx message.test.ts --run
```
Result: 3 files / 28 tests passed.
Lint/build:
```
npm run lint -- --quiet
npm run build
```
Lint reported 0 errors and 10 pre-existing warnings elsewhere. The production build completed successfully.
`git diff --check` passed before commit.
Production/config additions are 82 lines (48 backend, 34 frontend). This is the natural size of the bounded-executor and debounce change; no unrelated refactoring was added.
Closes #3167
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(dlq): parallelize group stats and debounce search [rocketmq-dashboard]
Posted by "lizhimins (via GitHub)" <gi...@apache.org>.
lizhimins closed pull request #3168: perf(dlq): parallelize group stats and debounce search
URL: https://github.com/apache/rocketmq-dashboard/pull/3168
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(dlq): parallelize group stats and debounce search [rocketmq-dashboard]
Posted by "RockteMQ-AI (via GitHub)" <gi...@apache.org>.
RockteMQ-AI commented on code in PR #3168:
URL: https://github.com/apache/rocketmq-dashboard/pull/3168#discussion_r3937761250
##########
server/src/main/java/org/apache/rocketmq/studio/provider/apache/RocketMQDLQProvider.java:
##########
@@ -122,13 +141,38 @@ private PageResult<DLQGroupVO> listDLQGroups(MQAdminExt adminExt, String search,
long offset = Pagination.pageOffset(page, pageSize);
int from = (int) Math.min(offset, dlqTopics.size());
int to = (int) Math.min(offset + pageSize, dlqTopics.size());
- List<DLQGroupVO> groups = dlqTopics.subList(from, to).stream()
- .map(topic -> buildDLQGroup(adminExt,
- topic.substring(MixAll.DLQ_GROUP_TOPIC_PREFIX.length()), topic))
- .toList();
+ List<String> pageTopics = dlqTopics.subList(from, to);
+ List<DLQGroupVO> groups = loadGroupStatsInParallel(adminExt, pageTopics);
return PageResult.of(groups, dlqTopics.size(), page, pageSize);
}
+ private List<DLQGroupVO> loadGroupStatsInParallel(MQAdminExt adminExt, List<String> pageTopics) {
+ List<Future<DLQGroupVO>> futures = new ArrayList<>(pageTopics.size());
+ for (String topic : pageTopics) {
+ String groupName = topic.substring(MixAll.DLQ_GROUP_TOPIC_PREFIX.length());
+ futures.add(statsExecutor.submit(() -> buildDLQGroup(adminExt, groupName, topic)));
+ }
+ List<DLQGroupVO> groups = new ArrayList<>(futures.size());
+ for (Future<DLQGroupVO> future : futures) {
+ groups.add(awaitGroupStats(future));
+ }
+ return groups;
+ }
+
+ private DLQGroupVO awaitGroupStats(Future<DLQGroupVO> future) {
+ try {
+ return future.get(STATS_TIMEOUT_SECONDS, TimeUnit.SECONDS);
+ } catch (TimeoutException e) {
+ future.cancel(true);
+ } catch (InterruptedException e) {
+ future.cancel(true);
+ Thread.currentThread().interrupt();
+ } catch (ExecutionException e) {
+ // buildDLQGroup reports single-topic failures inside the row itself.
Review Comment:
When `awaitGroupStats` falls back (timeout / interrupt / execution exception), the returned `DLQGroupVO` has no `groupName` or `dlqTopic` set — the user will see a blank row in the DLQ table with only `UNAVAILABLE` status.
Consider passing `groupName` and `dlqTopic` into `awaitGroupStats` so the fallback can populate them:
```java
private DLQGroupVO awaitGroupStats(Future<DLQGroupVO> future, String groupName, String dlqTopic) {
try {
return future.get(STATS_TIMEOUT_SECONDS, TimeUnit.SECONDS);
} catch (TimeoutException e) {
future.cancel(true);
} catch (InterruptedException e) {
future.cancel(true);
Thread.currentThread().interrupt();
} catch (ExecutionException e) {
// single-topic failure
}
return DLQGroupVO.builder()
.groupName(groupName).dlqTopic(dlqTopic)
.statsAvailable(false).status("UNAVAILABLE").build();
}
```
And update `loadGroupStatsInParallel` to pass them through.
##########
server/src/main/java/org/apache/rocketmq/studio/provider/apache/RocketMQDLQProvider.java:
##########
@@ -122,13 +141,38 @@ private PageResult<DLQGroupVO> listDLQGroups(MQAdminExt adminExt, String search,
long offset = Pagination.pageOffset(page, pageSize);
int from = (int) Math.min(offset, dlqTopics.size());
int to = (int) Math.min(offset + pageSize, dlqTopics.size());
- List<DLQGroupVO> groups = dlqTopics.subList(from, to).stream()
- .map(topic -> buildDLQGroup(adminExt,
- topic.substring(MixAll.DLQ_GROUP_TOPIC_PREFIX.length()), topic))
- .toList();
+ List<String> pageTopics = dlqTopics.subList(from, to);
+ List<DLQGroupVO> groups = loadGroupStatsInParallel(adminExt, pageTopics);
return PageResult.of(groups, dlqTopics.size(), page, pageSize);
}
+ private List<DLQGroupVO> loadGroupStatsInParallel(MQAdminExt adminExt, List<String> pageTopics) {
+ List<Future<DLQGroupVO>> futures = new ArrayList<>(pageTopics.size());
+ for (String topic : pageTopics) {
+ String groupName = topic.substring(MixAll.DLQ_GROUP_TOPIC_PREFIX.length());
+ futures.add(statsExecutor.submit(() -> buildDLQGroup(adminExt, groupName, topic)));
+ }
+ List<DLQGroupVO> groups = new ArrayList<>(futures.size());
+ for (Future<DLQGroupVO> future : futures) {
+ groups.add(awaitGroupStats(future));
+ }
+ return groups;
+ }
+
+ private DLQGroupVO awaitGroupStats(Future<DLQGroupVO> future) {
+ try {
+ return future.get(STATS_TIMEOUT_SECONDS, TimeUnit.SECONDS);
+ } catch (TimeoutException e) {
+ future.cancel(true);
+ } catch (InterruptedException e) {
+ future.cancel(true);
+ Thread.currentThread().interrupt();
+ } catch (ExecutionException e) {
+ // buildDLQGroup reports single-topic failures inside the row itself.
+ }
Review Comment:
The `ExecutionException` catch block silently swallows the underlying exception. Consider adding a `log.debug` or `log.warn` so operators can diagnose slow/failing brokers:
```java
} catch (ExecutionException e) {
log.warn("Failed to load DLQ stats for topic, row marked UNAVAILABLE", e.getCause());
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org