You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Szymon Blumczynski (Jira)" <ji...@apache.org> on 2026/08/21 21:40:00 UTC
[jira] [Commented] (KAFKA-18535) Flaky PlaintextConsumerPollTest.testMaxPollIntervalMsDelayInRevocation
[ https://issues.apache.org/jira/browse/KAFKA-18535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18106795#comment-18106795 ]
Szymon Blumczynski commented on KAFKA-18535:
--------------------------------------------
This is a duplicate of KAFKA-20112, which fixed the same test and is resolved.
Root cause: the test used awaitRebalance() as a proxy for "assignment complete", but an assignment does not imply an initialized fetch position. Since the test produced no records, topic-0 could still be in INITIALIZING when the revocation began. Both protocols mark partitions pendingRevocation before invoking onPartitionsRevoked (AbstractMembershipManager.markPendingRevocationToPauseFetching, ConsumerCoordinator.onJoinPrepare), and SubscriptionState.TopicPartitionState.shouldInitialize excludes pendingRevocation partitions, so the consumer.position(tp) call inside the callback could never resolve a position. It spun for the full default.api.timeout.ms and threw the reported "Timeout of 60000ms expired before the position for partition topic-0 could be determined".
That exception was then swallowed by ConsumerRebalanceListenerInvoker.invokePartitionsRevoked (returns rather than throws) and the rebalance continued, so onPartitionsAssigned still fired, awaitRebalance returned, and the assertion saw the untouched -1 sentinel. The still-unpositioned partition resurfaced as the suppressed timeout during close() in tearDown.
Fixed by commit [3d8a90a47b |https://github.com/apache/kafka/commit/3d8a90a47b5ed3c4305d84e5495eca66ed3060d0](PR [#21746|https://github.com/apache/kafka/pull/21746]), which produces a record per topic and replaces both awaitRebalance() calls with awaitNonEmptyRecords(), guaranteeing a valid position before revocation.
Verified on trunk [@05f61b9ff5|https://github.com/apache/kafka/commit/05f61b9ff5ed72f32ef8371f6ae338ab9b8f7ce5]: 30 repeats each of testClassicConsumerMaxPollIntervalMsDelayInRevocation and testAsyncConsumerMaxPollIntervalMsDelayInRevocation via -Pkafka.cluster.test.repeat=30 - 0 occurrences of this failure mode.
Hey [~yangpoan] I believe the ticket can be closed.
> Flaky PlaintextConsumerPollTest.testMaxPollIntervalMsDelayInRevocation
> ----------------------------------------------------------------------
>
> Key: KAFKA-18535
> URL: https://issues.apache.org/jira/browse/KAFKA-18535
> Project: Kafka
> Issue Type: Bug
> Components: clients, consumer
> Reporter: PoAn Yang
> Priority: Major
> Labels: integration-test, kip-848-client-support
>
> This case failed 6 times with a same error since from Jan 10 2025.
>
> {noformat}
> org.opentest4j.AssertionFailedError: expected: <0> but was: <-1>
> at app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
> at app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
> at app//org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
> at app//org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:166)
> at app//org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:161)
> at app//org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:632)
> at app//kafka.api.PlaintextConsumerPollTest.testMaxPollIntervalMsDelayInRevocation(PlaintextConsumerPollTest.scala:128)
> at java.base@23.0.1/java.lang.reflect.Method.invoke(Method.java:580)
> at java.base@23.0.1/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at java.base@23.0.1/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215)
> at java.base@23.0.1/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:197)
> at java.base@23.0.1/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215)
> at java.base@23.0.1/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at java.base@23.0.1/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215)
> at java.base@23.0.1/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215)
> at java.base@23.0.1/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215)
> at java.base@23.0.1/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at java.base@23.0.1/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215)
> at java.base@23.0.1/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:1024)
> at java.base@23.0.1/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:807)
> at java.base@23.0.1/java.util.stream.ReferencePipeline$7$1FlatMap.accept(ReferencePipeline.java:294)
> at java.base@23.0.1/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215)
> at java.base@23.0.1/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215)
> at java.base@23.0.1/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215)
> at java.base@23.0.1/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:1024)
> at java.base@23.0.1/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:570)
> at java.base@23.0.1/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:560)
> at java.base@23.0.1/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at java.base@23.0.1/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at java.base@23.0.1/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:265)
> at java.base@23.0.1/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:636)
> at java.base@23.0.1/java.util.stream.ReferencePipeline$7$1FlatMap.accept(ReferencePipeline.java:294)
> at java.base@23.0.1/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215)
> at java.base@23.0.1/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215)
> at java.base@23.0.1/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215)
> at java.base@23.0.1/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1709)
> at java.base@23.0.1/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:570)
> at java.base@23.0.1/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:560)
> at java.base@23.0.1/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at java.base@23.0.1/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at java.base@23.0.1/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:265)
> at java.base@23.0.1/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:636)
> at java.base@23.0.1/java.util.stream.ReferencePipeline$7$1FlatMap.accept(ReferencePipeline.java:294)
> at java.base@23.0.1/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1709)
> at java.base@23.0.1/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:570)
> at java.base@23.0.1/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:560)
> at java.base@23.0.1/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at java.base@23.0.1/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at java.base@23.0.1/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:265)
> at java.base@23.0.1/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:636)
> at java.base@23.0.1/java.util.ArrayList.forEach(ArrayList.java:1597)
> at java.base@23.0.1/java.util.ArrayList.forEach(ArrayList.java:1597)
> Suppressed: org.apache.kafka.common.KafkaException: Failed to close kafka consumer
> at app//org.apache.kafka.clients.consumer.internals.AsyncKafkaConsumer.close(AsyncKafkaConsumer.java:1364)
> at app//org.apache.kafka.clients.consumer.internals.AsyncKafkaConsumer.close(AsyncKafkaConsumer.java:1251)
> at app//org.apache.kafka.clients.consumer.KafkaConsumer.close(KafkaConsumer.java:1795)
> at app//kafka.api.IntegrationTestHarness.$anonfun$tearDown$3(IntegrationTestHarness.scala:243)
> at app//kafka.api.IntegrationTestHarness.$anonfun$tearDown$3$adapted(IntegrationTestHarness.scala:243)
> at app//scala.collection.IterableOnceOps.foreach(IterableOnce.scala:619)
> at app//scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:617)
> at app//scala.collection.AbstractIterable.foreach(Iterable.scala:935)
> at app//kafka.api.IntegrationTestHarness.tearDown(IntegrationTestHarness.scala:243)
> ... 45 more
> Caused by: org.apache.kafka.common.errors.TimeoutException: Timeout of 60000ms expired before the position for partition topic-0 could be determined{noformat}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)