You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@rocketmq.apache.org by "unbridled-41 (via GitHub)" <gi...@apache.org> on 2026/09/25 05:16:48 UTC
[PR] fix(tencent): reject an incomplete topic consumer page instead of truncating [rocketmq-dashboard]
unbridled-41 opened a new pull request, #5090:
URL: https://github.com/apache/rocketmq-dashboard/pull/5090
## Problem
`TencentInstanceProvider.getTopicConsumers` pages `DescribeTopic` with `CONSUMER_PAGE_SIZE` (100) and ends the scan as soon as a page comes back short:
```java
Long subscriptionCount = response.getSubscriptionCount();
if (data.length < CONSUMER_PAGE_SIZE
|| subscriptionCount != null && fetchedSubscriptions >= subscriptionCount) {
break;
}
```
The short-page test runs first, so a page that returns fewer rows than requested ends the scan **even when the response's own `SubscriptionCount` says more subscriptions exist**. The rows are dropped silently: the method returns what it fetched, and `TopicConsumerPageVO.total` is just `consumers.size()`, so the caller cannot tell the list is incomplete.
## Evidence
Call path: `TopicController.getTopicConsumers` / `MetadataService.getTopicConsumers` → `InstanceProviderRegistry` → `TencentInstanceProvider.getTopicConsumers` (`TencentInstanceProvider.java:396-431`); also `ops/ai/tool/handler/topic/TopicDetailToolHandler`.
Reproduction (stubbed client, in the added test): page 0 returns 40 `SubscriptionData` while `SubscriptionCount = 250`. Observed: the call returns 40 consumers and no error. Expected: HTTP 502 `"Tencent Cloud returned an incomplete topic consumer page"`, which is what the three sibling scans in this class already do.
The siblings call `requireCompletePage(...)` and end their loops on `hasFetchedAll(...) || isUnknownTotalCount(...) && returned < PAGE_SIZE` — `listTopics` (`:222-238`), `listConsumerGroups` (`:434-467`), `getGroupSubscriptions` (`:1004-1027`). `34b03320` ("reject incomplete resource pages", #4531) added that guard to those three loops and missed this fourth one, which has the same shape (offset/limit paging plus a server-reported total). Each sibling has an incomplete-page regression test (`:217`, `:578`, `:776`); `getTopicConsumers` had none.
## Root cause
The loop terminates on the page-length heuristic before consulting the total it already reads, so a short page is accepted as the end of the list instead of being validated against `SubscriptionCount`.
## Fix
Mirror the sibling loops: compute the page offset, call `requireCompletePage("topic consumer", offset, data.length, subscriptionCount)` before continuing, and break only when the reported total is reached (`hasFetchedAll`) or when the total is unknown and the page is short (`isUnknownTotalCount`). The now-redundant `fetchedSubscriptions` accumulator is removed since the completion test is the same `offset + returned` expression the siblings use.
## Priority
**54** = impact 15 (silent data loss in an inventory read: a subscribed group is reported as "not subscribed", which is what an operator or the AI tool acts on) + blast radius 10 (Tencent instances only; topic detail page and `rmq.topic.detail`) + reproducibility 16 (deterministic with a stub; production needs a genuinely short page) + maintenance 13 (closes the residual of an incomplete fix whose guard and tests already exist as a pattern).
`FIX_CONFIDENCE` **92**.
## Tests
New `getTopicConsumersShouldRejectIncompletePageWhenTotalCountRequiresMoreTest` **fails before the fix** (`Expecting code to raise a throwable`) and passes after.
```
$ mvn -o test -Dtest='org.apache.rocketmq.studio.provider.tencent.*Test'
Tests run: 85, Failures: 0, Errors: 0, Skipped: 0
BUILD SUCCESS
```
The three pre-existing `getTopicConsumers` tests (map, read every page, stop on a full page whose count is reached) still pass, which is what pins the unchanged behaviour for complete pages.
## Risk
Low. A 502 now surfaces where a truncated list was returned before; that is the same contract the topic-list, consumer-group-list and subscription scans already enforce for the same API family, and the error is a `BusinessException` the shared handler already renders. A stub that returns a short page without a `SubscriptionCount` (or with an unknown total) keeps the previous behaviour.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(tencent): reject an incomplete topic consumer page instead of truncating [rocketmq-dashboard]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #5090:
URL: https://github.com/apache/rocketmq-dashboard/pull/5090#issuecomment-5963521527
This pull request has been automatically closed because there has been no activity for 7 days. If you would like to continue working on this, please reopen the PR with updated changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(tencent): reject an incomplete topic consumer page instead of truncating [rocketmq-dashboard]
Posted by "unbridled-41 (via GitHub)" <gi...@apache.org>.
unbridled-41 commented on PR #5090:
URL: https://github.com/apache/rocketmq-dashboard/pull/5090#issuecomment-5827779243
### Issue linkage — none filed for this change
Unlike the other three PRs from this batch, I did not open an issue for this one, because the change
contradicts an expectation the repository has already recorded for this exact method and I do not
want to file a report asserting a defect the project may have specified on purpose.
- #1583 ("Paginate Tencent Topic subscriptions") is about `TencentInstanceProvider#getTopicConsumers`
— the method this PR changes — and lists, as expected behaviour, fetching pages "until one of these
conditions is met: the response page is empty or shorter than the requested limit; the collected
count reaches `SubscriptionCount`; …". The current code implements exactly that: a short page ends
the scan before the count is consulted. The merged fix for #1583 is where this loop came from.
- #4530 / #4531 ("reject incomplete resource pages") then applied the fail-closed total check to
three *other* inventories — `listTopics`, `listConsumerGroups` and the group subscription read —
and left this one out.
So this PR removes a stop condition that #1583 recorded for the same method, in favour of the
stricter contract the sibling scans now follow. Both readings are defensible; I cannot tell which one
is wanted, so I am leaving the decision here instead of asserting a defect. If the short-page stop is
intended for this path, this PR should be closed rather than reviewed.
The behaviour itself is real, not speculative — the description has the measurement (a page of 40
rows against `SubscriptionCount = 250` returns 40 consumers with no error). The open question is only
whether it should fail closed or stay as specified in #1583.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(tencent): reject an incomplete topic consumer page instead of truncating [rocketmq-dashboard]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #5090: fix(tencent): reject an incomplete topic consumer page instead of truncating
URL: https://github.com/apache/rocketmq-dashboard/pull/5090
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org