You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Kirk True (Jira)" <ji...@apache.org> on 2023/12/13 23:46:00 UTC

[jira] [Updated] (KAFKA-16011) Fix PlaintextConsumerTest.testMultiConsumerSessionTimeoutOnClose

     [ https://issues.apache.org/jira/browse/KAFKA-16011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kirk True updated KAFKA-16011:
------------------------------
    Description: 
The integration test {{PlaintextConsumerTest.testMultiConsumerSessionTimeoutOnClose}} is failing when using the {{AsyncKafkaConsumer}}.

The error is:

{code}
org.opentest4j.AssertionFailedError: Did not get valid assignment for partitions HashSet(topic1-2, topic1-4, topic-1, topic-0, topic1-5, topic1-1, topic1-0, topic1-3). Instead, got ArrayBuffer(Set(topic1-0, topic-0, topic-1), Set(), Set(topic1-1, topic1-5))
	at org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:38)
	at org.junit.jupiter.api.Assertions.fail(Assertions.java:134)
	at kafka.api.AbstractConsumerTest.validateGroupAssignment(AbstractConsumerTest.scala:286)
	at kafka.api.PlaintextConsumerTest.runMultiConsumerSessionTimeoutTest(PlaintextConsumerTest.scala:1865)
	at kafka.api.PlaintextConsumerTest.testMultiConsumerSessionTimeoutOnClose(PlaintextConsumerTest.scala:1277)
{code}

The logs include these lines:
 
{code}
[2023-12-13 15:33:33,180] ERROR [Consumer clientId=ConsumerTestConsumer, groupId=my-test] GroupHeartbeatRequest failed due to error: INVALID_REQUEST (org.apache.kafka.clients.consumer.internals.HeartbeatRequestManager:376)
[2023-12-13 15:33:33,180] ERROR [Consumer clientId=ConsumerTestConsumer, groupId=my-test] Member JQ_e0S5FTzKnYyStB3aBrQ with epoch 0 transitioned to FATAL state (org.apache.kafka.clients.consumer.internals.MembershipManagerImpl:456)
[2023-12-13 15:33:33,212] ERROR [daemon-consumer-assignment]: Error due to (kafka.api.AbstractConsumerTest$ConsumerAssignmentPoller:139)
org.apache.kafka.common.errors.InvalidRequestException: RebalanceTimeoutMs must be provided in first request.
[2023-12-13 15:33:39,196] WARN [Consumer clientId=ConsumerTestConsumer, groupId=my-test] consumer poll timeout has expired. This means the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms, which typically implies that the poll loop is spending too much time processing messages. You can address this either by increasing max.poll.interval.ms or by reducing the maximum size of batches returned in poll() with max.poll.records. (org.apache.kafka.clients.consumer.internals.HeartbeatRequestManager:188)
[2023-12-13 15:33:39,200] WARN [Consumer clientId=ConsumerTestConsumer, groupId=my-test] consumer poll timeout has expired. This means the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms, which typically implies that the poll loop is spending too much time processing messages. You can address this either by increasing max.poll.interval.ms or by reducing the maximum size of batches returned in poll() with max.poll.records. (org.apache.kafka.clients.consumer.internals.HeartbeatRequestManager:188)
{code} 

I don't know if that's related or not.

  was:
The integration test {{PlaintextConsumerTest.testMultiConsumerSessionTimeoutOnStopPolling}} is failing when using the {{AsyncKafkaConsumer}}.

The error is:

{code}
org.opentest4j.AssertionFailedError: Did not get valid assignment for partitions [topic1-2, topic1-4, topic-1, topic-0, topic1-5, topic1-1, topic1-0, topic1-3] after one consumer left
	at org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:38)
	at org.junit.jupiter.api.Assertions.fail(Assertions.java:134)
	at kafka.api.AbstractConsumerTest.validateGroupAssignment(AbstractConsumerTest.scala:286)
	at kafka.api.PlaintextConsumerTest.runMultiConsumerSessionTimeoutTest(PlaintextConsumerTest.scala:1883)
	at kafka.api.PlaintextConsumerTest.testMultiConsumerSessionTimeoutOnStopPolling(PlaintextConsumerTest.scala:1281)
{code}

The logs include these lines:
 
{code}
[2023-12-13 15:26:40,736] WARN [Consumer clientId=ConsumerTestConsumer, groupId=my-test] consumer poll timeout has expired. This means the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms, which typically implies that the poll loop is spending too much time processing messages. You can address this either by increasing max.poll.interval.ms or by reducing the maximum size of batches returned in poll() with max.poll.records. (org.apache.kafka.clients.consumer.internals.HeartbeatRequestManager:188)
[2023-12-13 15:26:40,736] WARN [Consumer clientId=ConsumerTestConsumer, groupId=my-test] consumer poll timeout has expired. This means the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms, which typically implies that the poll loop is spending too much time processing messages. You can address this either by increasing max.poll.interval.ms or by reducing the maximum size of batches returned in poll() with max.poll.records. (org.apache.kafka.clients.consumer.internals.HeartbeatRequestManager:188)
[2023-12-13 15:26:40,736] WARN [Consumer clientId=ConsumerTestConsumer, groupId=my-test] consumer poll timeout has expired. This means the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms, which typically implies that the poll loop is spending too much time processing messages. You can address this either by increasing max.poll.interval.ms or by reducing the maximum size of batches returned in poll() with max.poll.records. (org.apache.kafka.clients.consumer.internals.HeartbeatRequestManager:188)
{code} 

I don't know if that's related or not.


> Fix PlaintextConsumerTest.testMultiConsumerSessionTimeoutOnClose
> ----------------------------------------------------------------
>
>                 Key: KAFKA-16011
>                 URL: https://issues.apache.org/jira/browse/KAFKA-16011
>             Project: Kafka
>          Issue Type: Test
>          Components: clients, consumer, unit tests
>    Affects Versions: 3.7.0
>            Reporter: Kirk True
>            Priority: Major
>              Labels: consumer-threading-refactor, kip-848
>
> The integration test {{PlaintextConsumerTest.testMultiConsumerSessionTimeoutOnClose}} is failing when using the {{AsyncKafkaConsumer}}.
> The error is:
> {code}
> org.opentest4j.AssertionFailedError: Did not get valid assignment for partitions HashSet(topic1-2, topic1-4, topic-1, topic-0, topic1-5, topic1-1, topic1-0, topic1-3). Instead, got ArrayBuffer(Set(topic1-0, topic-0, topic-1), Set(), Set(topic1-1, topic1-5))
> 	at org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:38)
> 	at org.junit.jupiter.api.Assertions.fail(Assertions.java:134)
> 	at kafka.api.AbstractConsumerTest.validateGroupAssignment(AbstractConsumerTest.scala:286)
> 	at kafka.api.PlaintextConsumerTest.runMultiConsumerSessionTimeoutTest(PlaintextConsumerTest.scala:1865)
> 	at kafka.api.PlaintextConsumerTest.testMultiConsumerSessionTimeoutOnClose(PlaintextConsumerTest.scala:1277)
> {code}
> The logs include these lines:
>  
> {code}
> [2023-12-13 15:33:33,180] ERROR [Consumer clientId=ConsumerTestConsumer, groupId=my-test] GroupHeartbeatRequest failed due to error: INVALID_REQUEST (org.apache.kafka.clients.consumer.internals.HeartbeatRequestManager:376)
> [2023-12-13 15:33:33,180] ERROR [Consumer clientId=ConsumerTestConsumer, groupId=my-test] Member JQ_e0S5FTzKnYyStB3aBrQ with epoch 0 transitioned to FATAL state (org.apache.kafka.clients.consumer.internals.MembershipManagerImpl:456)
> [2023-12-13 15:33:33,212] ERROR [daemon-consumer-assignment]: Error due to (kafka.api.AbstractConsumerTest$ConsumerAssignmentPoller:139)
> org.apache.kafka.common.errors.InvalidRequestException: RebalanceTimeoutMs must be provided in first request.
> [2023-12-13 15:33:39,196] WARN [Consumer clientId=ConsumerTestConsumer, groupId=my-test] consumer poll timeout has expired. This means the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms, which typically implies that the poll loop is spending too much time processing messages. You can address this either by increasing max.poll.interval.ms or by reducing the maximum size of batches returned in poll() with max.poll.records. (org.apache.kafka.clients.consumer.internals.HeartbeatRequestManager:188)
> [2023-12-13 15:33:39,200] WARN [Consumer clientId=ConsumerTestConsumer, groupId=my-test] consumer poll timeout has expired. This means the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms, which typically implies that the poll loop is spending too much time processing messages. You can address this either by increasing max.poll.interval.ms or by reducing the maximum size of batches returned in poll() with max.poll.records. (org.apache.kafka.clients.consumer.internals.HeartbeatRequestManager:188)
> {code} 
> I don't know if that's related or not.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)