You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Shekharrajak (via GitHub)" <gi...@apache.org> on 2026/10/27 08:08:59 UTC
[PR] KAFKA-20803: Index batched read summary responses once [kafka]
Shekharrajak opened a new pull request, #23974:
URL: https://github.com/apache/kafka/pull/23974
Follow-up to #22835 for [KAFKA-20803](https://issues.apache.org/jira/browse/KAFKA-20803).
- **What:** Remove repeated scans of batched `READ_SHARE_GROUP_STATE_SUMMARY` responses.
- **How:** Build one shared `PartitionResultIndex` per batch and look up each handler's result by topic ID and partition.
- **Impact:** Reduce demultiplexing from O(N²) to average O(N), using O(N) temporary indexing memory while preserving summary fields, errors, and retries.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20803: Demultiplex batched persister responses once (Index batched read summary) [kafka]
Posted by "Shekharrajak (via GitHub)" <gi...@apache.org>.
Shekharrajak commented on code in PR #23974:
URL: https://github.com/apache/kafka/pull/23974#discussion_r4386924230
##########
server-common/src/main/java/org/apache/kafka/server/share/persister/PersisterStateManager.java:
##########
@@ -1232,52 +1244,44 @@ protected void handleRequestResponse(ClientResponse response) {
switch (clientResponseError) {
case NONE:
- ReadShareGroupStateSummaryResponse combinedResponse = (ReadShareGroupStateSummaryResponse) response.responseBody();
- for (ReadShareGroupStateSummaryResponseData.ReadStateSummaryResult readStateSummaryResult : combinedResponse.data().results()) {
- if (readStateSummaryResult.topicId().equals(partitionKey().topicId())) {
- Optional<ReadShareGroupStateSummaryResponseData.PartitionResult> partitionStateData =
- readStateSummaryResult.partitions().stream().filter(partitionResult -> partitionResult.partition() == partitionKey().partition())
- .findFirst();
-
- if (partitionStateData.isPresent()) {
- Errors error = Errors.forCode(partitionStateData.get().errorCode());
- String errorMessage = partitionStateData.get().errorMessage();
- if (errorMessage == null || errorMessage.isEmpty()) {
- errorMessage = error.message();
- }
-
- switch (error) {
- case NONE:
- readStateSummaryBackoff.resetAttempts();
- ReadShareGroupStateSummaryResponseData.ReadStateSummaryResult result = ReadShareGroupStateSummaryResponse.toResponseReadStateSummaryResult(
- partitionKey().topicId(),
- List.of(partitionStateData.get())
- );
- this.result.complete(new ReadShareGroupStateSummaryResponse(new ReadShareGroupStateSummaryResponseData()
- .setResults(List.of(result))));
- return;
+ ReadShareGroupStateSummaryResponseData.PartitionResult partitionResult = lookupPartitionResult(response);
Review Comment:
similar to https://github.com/apache/kafka/pull/23971
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20803: Index batched read summary responses once [kafka]
Posted by "Shekharrajak (via GitHub)" <gi...@apache.org>.
Shekharrajak commented on PR #23974:
URL: https://github.com/apache/kafka/pull/23974#issuecomment-6436844372
Validation:
- All 99 `PersisterStateManagerTest` cases passed. The new 16- and 256-partition cases failed against unchanged trunk and now pass, verifying one partition-key access per result and one index build per batch.
- Tests cover cross-topic routing, summary payload preservation, missing partitions, and null responses.
- Passed `./gradlew :server-common:spotlessApply :server-common:spotlessCheck :server-common:checkstyleMain :server-common:checkstyleTest :server-common:test --tests org.apache.kafka.server.share.persister.PersisterStateManagerTest`.
- Passed `git diff --check upstream/trunk...HEAD`.
Prepared with Codex assistance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org