You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "lianetm (via GitHub)" <gi...@apache.org> on 2026/03/16 21:14:42 UTC
[PR] KAFKA-20321: Mark lost partitions before callbacks for Consumer and Streams [kafka]
lianetm opened a new pull request, #21781:
URL: https://github.com/apache/kafka/pull/21781
Mark partitions lost to avoid returning buffered records in the case of a race after the callback completes (app thread moves onto polling, while background moves onto clearing the lost assignment)
Same fix applied for Consumer and Stream managers (Share has no callbacks so this does not apply)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20321: Mark lost partitions before callbacks for Consumer and Streams [kafka]
Posted by "DL1231 (via GitHub)" <gi...@apache.org>.
DL1231 commented on code in PR #21781:
URL: https://github.com/apache/kafka/pull/21781#discussion_r2944114140
##########
clients/src/test/java/org/apache/kafka/clients/consumer/internals/ConsumerMembershipManagerTest.java:
##########
@@ -272,6 +272,72 @@ public void testFencingWhenStateIsStable() {
verify(subscriptionState).assignFromSubscribed(Collections.emptySet());
}
+ @Test
+ public void testTransitionToFencedMarksPendingRevocationBeforeSignalingPartitionsLost() {
Review Comment:
Thanks for the patch! Overall the fix looks good to me. Just noticed there are a few tests failing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20321: Mark lost partitions before callbacks for Consumer and Streams [kafka]
Posted by "lianetm (via GitHub)" <gi...@apache.org>.
lianetm merged PR #21781:
URL: https://github.com/apache/kafka/pull/21781
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20321: Mark lost partitions before callbacks for Consumer and Streams [kafka]
Posted by "nileshkumar3 (via GitHub)" <gi...@apache.org>.
nileshkumar3 commented on code in PR #21781:
URL: https://github.com/apache/kafka/pull/21781#discussion_r2943970027
##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerMembershipManager.java:
##########
@@ -300,6 +300,7 @@ protected CompletableFuture<Void> signalMemberLeavingGroup() {
*/
@Override
protected CompletableFuture<Void> signalPartitionsLost(Set<TopicPartition> partitionsLost) {
+ markPendingRevocationToPauseFetching(partitionsLost);
Review Comment:
Can we have a 1 liner comment like other places.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20321: Mark lost partitions before callbacks for Consumer and Streams [kafka]
Posted by "DL1231 (via GitHub)" <gi...@apache.org>.
DL1231 commented on PR #21781:
URL: https://github.com/apache/kafka/pull/21781#issuecomment-4072039803
@lianetm Thanks for the patch! Overall the fix looks good to me. Just noticed there are a few tests failing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20321: Mark lost partitions before callbacks for Consumer and Streams [kafka]
Posted by "lianetm (via GitHub)" <gi...@apache.org>.
lianetm commented on code in PR #21781:
URL: https://github.com/apache/kafka/pull/21781#discussion_r2945787290
##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerMembershipManager.java:
##########
@@ -300,6 +300,7 @@ protected CompletableFuture<Void> signalMemberLeavingGroup() {
*/
@Override
protected CompletableFuture<Void> signalPartitionsLost(Set<TopicPartition> partitionsLost) {
+ markPendingRevocationToPauseFetching(partitionsLost);
Review Comment:
sure, done
##########
clients/src/test/java/org/apache/kafka/clients/consumer/internals/ConsumerMembershipManagerTest.java:
##########
@@ -272,6 +272,72 @@ public void testFencingWhenStateIsStable() {
verify(subscriptionState).assignFromSubscribed(Collections.emptySet());
}
+ @Test
+ public void testTransitionToFencedMarksPendingRevocationBeforeSignalingPartitionsLost() {
Review Comment:
thanks, I was missing a mock, added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org