You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "xijiu (via GitHub)" <gi...@apache.org> on 2024/11/13 04:39:50 UTC
[PR] KAFKA-17904: Flaky testMultiConsumerSessionTimeoutOnClose [kafka]
xijiu opened a new pull request, #17789:
URL: https://github.com/apache/kafka/pull/17789
First of all, the reason for the failure of this test is due to TIMEOUT, the method `AbstractConsumerTest#validateGroupAssignment` timeout after waiting for 10 seconds. And it reproduced on my computer.
`AbstractConsumerTest#validateGroupAssignment` is used to check all the consumer's assignments meet expectations, the exception as below:
```
org.opentest4j.AssertionFailedError: Did not get valid assignment for partitions HashSet(topic1-2, topic1-4, topic-1, topic-0, topic1-5, topic1-1, topic1-0, topic1-3). Instead, got ArrayBuffer(Set(topic1-1, topic1-0, topic1-2), Set(topic1-5, topic1-4), Set())
```
I ran this junit test many times on my local computer after I added some logs. Then I found the timeout case is the GroupProtocol.CONSUMER mode. The CONSUMER mode maybe interact with the GroupCoordinator multiple times before reconciliation completed
The frequency of interaction is controlled by configuration `group.consumer.heartbeat.interval.ms` which default value is 5000ms. Those successful unit tests take at least 5 seconds to complete, so maybe we can reduce heartbeat interval.
After I set `group.consumer.heartbeat.interval.ms` to 1000ms, this problem has not occurred again on my computer. And running this unit test has become more faster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-17904: Flaky testMultiConsumerSessionTimeoutOnClose [kafka]
Posted by "xijiu (via GitHub)" <gi...@apache.org>.
xijiu commented on PR #17789:
URL: https://github.com/apache/kafka/pull/17789#issuecomment-2472363186
@lianetm @chia7712 PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-17904: Flaky testMultiConsumerSessionTimeoutOnClose [kafka]
Posted by "lianetm (via GitHub)" <gi...@apache.org>.
lianetm commented on PR #17789:
URL: https://github.com/apache/kafka/pull/17789#issuecomment-2474031016
Hello @xijiu , thanks for taking a look at this! Very interesting finding indeed. I expect this same issue is behind the flakiness on `testMultiConsumerSessionTimeoutOnStopPolling` too right? (Could you maybe validate locally for this one too?)
https://ge.apache.org/scans/tests?search.buildOutcome=failure&search.relativeStartTime=P28D&search.rootProjectNames=kafka&search.tags=trunk&search.timeZoneId=America%2FToronto&tests.container=kafka.api.PlaintextConsumerPollTest&tests.test=testMultiConsumerSessionTimeoutOnStopPolling(String%2C%20String)%5B2%5D
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-17904: Flaky testMultiConsumerSessionTimeoutOnClose [kafka]
Posted by "lianetm (via GitHub)" <gi...@apache.org>.
lianetm commented on PR #17789:
URL: https://github.com/apache/kafka/pull/17789#issuecomment-2479690754
4 unrelated test failures (failures exist in trunk). 3 already tracked. I filed https://issues.apache.org/jira/browse/KAFKA-18025.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-17904: Flaky testMultiConsumerSessionTimeoutOnClose [kafka]
Posted by "xijiu (via GitHub)" <gi...@apache.org>.
xijiu commented on PR #17789:
URL: https://github.com/apache/kafka/pull/17789#issuecomment-2475702480
@lianetm Thanks for reply.
I think this PR will server boths.
Both `testMultiConsumerSessionTimeoutOnClose()` and `testMultiConsumerSessionTimeoutOnStopPolling()` will call the method `runMultiConsumersSessionTimeoutTest()`
```
@ParameterizedTest(name = TestInfoUtils.TestWithParameterizedQuorumAndGroupProtocolNames)
@MethodSource(Array("getTestQuorumAndGroupProtocolParametersAll"))
def testMultiConsumerSessionTimeoutOnStopPolling(quorum: String, groupProtocol: String): Unit = {
runMultiConsumerSessionTimeoutTest(false)
}
@ParameterizedTest(name = TestInfoUtils.TestWithParameterizedQuorumAndGroupProtocolNames)
@MethodSource(Array("getTestQuorumAndGroupProtocolParametersAll"))
def testMultiConsumerSessionTimeoutOnClose(quorum: String, groupProtocol: String): Unit = {
runMultiConsumerSessionTimeoutTest(true)
}
```
And the failure of method `runMultiConsumersSessionTimeoutTest(boolean)` is unrelated to the input parameters.
But to be honest, this flaky test is hard to reproduce on my mac , I ran many many times but only reproduced it once 😁.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-17904: Flaky testMultiConsumerSessionTimeoutOnClose [kafka]
Posted by "lianetm (via GitHub)" <gi...@apache.org>.
lianetm commented on code in PR #17789:
URL: https://github.com/apache/kafka/pull/17789#discussion_r1843913989
##########
core/src/test/scala/integration/kafka/api/PlaintextConsumerPollTest.scala:
##########
@@ -32,6 +34,12 @@ import scala.jdk.CollectionConverters._
@Timeout(600)
class PlaintextConsumerPollTest extends AbstractConsumerTest {
+ override protected def brokerPropertyOverrides(properties: Properties): Unit = {
+ super.brokerPropertyOverrides(properties)
+ properties.setProperty(GroupCoordinatorConfig.CONSUMER_GROUP_HEARTBEAT_INTERVAL_MS_CONFIG, "1000")
Review Comment:
I notice there are several test in this same file overriding the equivalent of this prop for the classic consumer (ConsumerConfig.HEARTBEAT_INTERVAL_MS_CONFIG), but to 500. Should we consider that value instead? With that, we would have those tests running for both consumers with the same config. What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-17904: Flaky testMultiConsumerSessionTimeoutOnClose [kafka]
Posted by "xijiu (via GitHub)" <gi...@apache.org>.
xijiu commented on code in PR #17789:
URL: https://github.com/apache/kafka/pull/17789#discussion_r1843944804
##########
core/src/test/scala/integration/kafka/api/PlaintextConsumerPollTest.scala:
##########
@@ -32,6 +34,12 @@ import scala.jdk.CollectionConverters._
@Timeout(600)
class PlaintextConsumerPollTest extends AbstractConsumerTest {
+ override protected def brokerPropertyOverrides(properties: Properties): Unit = {
+ super.brokerPropertyOverrides(properties)
+ properties.setProperty(GroupCoordinatorConfig.CONSUMER_GROUP_HEARTBEAT_INTERVAL_MS_CONFIG, "1000")
Review Comment:
@lianetm Yeah, you are right, I will fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-17904: Flaky testMultiConsumerSessionTimeoutOnClose [kafka]
Posted by "lianetm (via GitHub)" <gi...@apache.org>.
lianetm merged PR #17789:
URL: https://github.com/apache/kafka/pull/17789
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org