You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "AndrewJSchofield (via GitHub)" <gi...@apache.org> on 2025/10/21 09:09:44 UTC
[PR] KAFKA-19767: Add partitions to share fetch buffer together [kafka]
AndrewJSchofield opened a new pull request, #20737:
URL: https://github.com/apache/kafka/pull/20737
For a ShareFetchResponse containing data from multiple partitions, the code used to add the records for each partition separately to the share fetch buffer, signalling completion for each. This small change adds them all in the same operation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19767: Add partitions to share fetch buffer together [kafka]
Posted by "AndrewJSchofield (via GitHub)" <gi...@apache.org>.
AndrewJSchofield commented on code in PR #20737:
URL: https://github.com/apache/kafka/pull/20737#discussion_r2484913260
##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/ShareConsumeRequestManager.java:
##########
@@ -815,21 +816,26 @@ private void handleShareFetchSuccess(Node fetchTarget,
}
}
- ShareCompletedFetch completedFetch = new ShareCompletedFetch(
+ completedFetches.add(
+ new ShareCompletedFetch(
logContext,
BufferSupplier.create(),
fetchTarget.id(),
tip,
partitionData,
shareFetchMetricsAggregator,
- requestVersion);
- shareFetchBuffer.add(completedFetch);
+ requestVersion)
+ );
if (!partitionData.acquiredRecords().isEmpty()) {
fetchMoreRecords = false;
}
}
+ if (!completedFetches.isEmpty()) {
+ shareFetchBuffer.add(completedFetches);
Review Comment:
It's more to do with correctness. I'm going back into the callback stuff shortly, so I'll bear in mind your comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19767: Add partitions to share fetch buffer together [kafka]
Posted by "AndrewJSchofield (via GitHub)" <gi...@apache.org>.
AndrewJSchofield merged PR #20737:
URL: https://github.com/apache/kafka/pull/20737
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19767: Add partitions to share fetch buffer together [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #20737:
URL: https://github.com/apache/kafka/pull/20737#discussion_r2483974740
##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/ShareConsumeRequestManager.java:
##########
@@ -815,21 +816,26 @@ private void handleShareFetchSuccess(Node fetchTarget,
}
}
- ShareCompletedFetch completedFetch = new ShareCompletedFetch(
+ completedFetches.add(
+ new ShareCompletedFetch(
logContext,
BufferSupplier.create(),
fetchTarget.id(),
tip,
partitionData,
shareFetchMetricsAggregator,
- requestVersion);
- shareFetchBuffer.add(completedFetch);
+ requestVersion)
+ );
if (!partitionData.acquiredRecords().isEmpty()) {
fetchMoreRecords = false;
}
}
+ if (!completedFetches.isEmpty()) {
+ shareFetchBuffer.add(completedFetches);
Review Comment:
Is this primarily a performance optimization for handling many partitions? If so, should we reduce the `ShareAcknowledgementCommitCallbackEvent` by batching partitions ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org