You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "mjsax (via GitHub)" <gi...@apache.org> on 2026/07/14 02:02:57 UTC
[PR] KAFKA-20786: Speed up TimeWindowedKStreamIntegrationTest [kafka]
mjsax opened a new pull request, #22824:
URL: https://github.com/apache/kafka/pull/22824
Currently, the test needs to wait for 45sec for session timeout to expire. With a 60sec test timeout the test is flaky.
This PR let Kafka Streams send a leave group request on close() which reduces the test runtime (for a single parameter run) from about 50sec to 5sec.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20786: Speed up TimeWindowedKStreamIntegrationTest [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #22824:
URL: https://github.com/apache/kafka/pull/22824#discussion_r3609770615
##########
streams/integration-tests/src/test/java/org/apache/kafka/streams/integration/TimeWindowedKStreamIntegrationTest.java:
##########
@@ -390,7 +392,9 @@ public void shouldRestoreAfterJoinRestart(final StrategyType type, final boolean
assertThat(windowedMessages, is(expectResult));
- kafkaStreams.close();
+ // Leave the group on close so the immediate restart below does not have to wait for the
+ // previous member to be evicted via session timeout (~45s) before its rebalance completes.
+ kafkaStreams.close(CloseOptions.groupMembershipOperation(GroupMembershipOperation.LEAVE_GROUP));
Review Comment:
This is a good approach to reduce the elapsed time. We will file a patch for `TimestampedStoreUpgradeIntegrationTest`, which can reduce the elapsed time from 5m to 30s
```
Gradle Test Run :streams:integration-tests:test > Gradle Test Executor 35 > TimestampedStoreUpgradeIntegrationTest > shouldMigrateInMemoryKeyValueStoreToTimestampedKeyValueStoreUsingPapi() PASSED
Gradle Test Run :streams:integration-tests:test > Gradle Test Executor 35 > TimestampedStoreUpgradeIntegrationTest > shouldProxyWindowStoreToTimestampedWindowStoreUsingPapi() PASSED
Gradle Test Run :streams:integration-tests:test > Gradle Test Executor 35 > TimestampedStoreUpgradeIntegrationTest > shouldMigratePersistentKeyValueStoreToTimestampedKeyValueStoreUsingPapi() PASSED
Gradle Test Run :streams:integration-tests:test > Gradle Test Executor 35 > TimestampedStoreUpgradeIntegrationTest > shouldMigratePersistentWindowStoreToTimestampedWindowStoreUsingPapi() PASSED
Gradle Test Run :streams:integration-tests:test > Gradle Test Executor 35 > TimestampedStoreUpgradeIntegrationTest > shouldSuccessfullyDowngradeFromTimestampedToRegularKeyValueStoreAfterCleanup() PASSED
Gradle Test Run :streams:integration-tests:test > Gradle Test Executor 35 > TimestampedStoreUpgradeIntegrationTest > shouldProxyKeyValueStoreToTimestampedKeyValueStoreUsingPapi() PASSED
Gradle Test Run :streams:integration-tests:test > Gradle Test Executor 35 > TimestampedStoreUpgradeIntegrationTest > shouldMigrateInMemoryWindowStoreToTimestampedWindowStoreUsingPapi() PASSED
Gradle Test Run :streams:integration-tests:test > Gradle Test Executor 35 > TimestampedStoreUpgradeIntegrationTest > shouldFailDowngradeFromTimestampedToRegularKeyValueStore() PASSED
[Incubating] Problems report is available at: file:///home/chia7712/project/kafka/build/reports/problems/problems-report.html
Deprecated Gradle features were used in this build, making it incompatible with Gradle 10.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/9.4.1/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD SUCCESSFUL in 34s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20786: Speed up TimeWindowedKStreamIntegrationTest [kafka]
Posted by "Russole (via GitHub)" <gi...@apache.org>.
Russole commented on code in PR #22824:
URL: https://github.com/apache/kafka/pull/22824#discussion_r3855085822
##########
streams/integration-tests/src/test/java/org/apache/kafka/streams/integration/TimeWindowedKStreamIntegrationTest.java:
##########
@@ -390,7 +392,9 @@ public void shouldRestoreAfterJoinRestart(final StrategyType type, final boolean
assertThat(windowedMessages, is(expectResult));
- kafkaStreams.close();
+ // Leave the group on close so the immediate restart below does not have to wait for the
+ // previous member to be evicted via session timeout (~45s) before its rebalance completes.
+ kafkaStreams.close(CloseOptions.groupMembershipOperation(GroupMembershipOperation.LEAVE_GROUP));
Review Comment:
https://github.com/apache/kafka/pull/23272
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20786: Speed up TimeWindowedKStreamIntegrationTest [kafka]
Posted by "Russole (via GitHub)" <gi...@apache.org>.
Russole commented on code in PR #22824:
URL: https://github.com/apache/kafka/pull/22824#discussion_r3839187071
##########
streams/integration-tests/src/test/java/org/apache/kafka/streams/integration/TimeWindowedKStreamIntegrationTest.java:
##########
@@ -390,7 +392,9 @@ public void shouldRestoreAfterJoinRestart(final StrategyType type, final boolean
assertThat(windowedMessages, is(expectResult));
- kafkaStreams.close();
+ // Leave the group on close so the immediate restart below does not have to wait for the
+ // previous member to be evicted via session timeout (~45s) before its rebalance completes.
+ kafkaStreams.close(CloseOptions.groupMembershipOperation(GroupMembershipOperation.LEAVE_GROUP));
Review Comment:
HeadersStoreUpgradeIntegrationTest has the same pattern:
https://github.com/apache/kafka/blob/trunk/streams/integration-tests/src/test/java/org/apache/kafka/streams/integration/HeadersStoreUpgradeIntegrationTest.java
I'll fix it in a follow-up patch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20786: Speed up TimeWindowedKStreamIntegrationTest [kafka]
Posted by "mjsax (via GitHub)" <gi...@apache.org>.
mjsax merged PR #22824:
URL: https://github.com/apache/kafka/pull/22824
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20786: Speed up TimeWindowedKStreamIntegrationTest [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #22824:
URL: https://github.com/apache/kafka/pull/22824#discussion_r3839110279
##########
streams/integration-tests/src/test/java/org/apache/kafka/streams/integration/TimeWindowedKStreamIntegrationTest.java:
##########
@@ -390,7 +392,9 @@ public void shouldRestoreAfterJoinRestart(final StrategyType type, final boolean
assertThat(windowedMessages, is(expectResult));
- kafkaStreams.close();
+ // Leave the group on close so the immediate restart below does not have to wait for the
+ // previous member to be evicted via session timeout (~45s) before its rebalance completes.
+ kafkaStreams.close(CloseOptions.groupMembershipOperation(GroupMembershipOperation.LEAVE_GROUP));
Review Comment:
#23251
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org