You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "m1a2st (via GitHub)" <gi...@apache.org> on 2024/08/22 03:17:38 UTC

[PR] KAFKA-16813: Add global timeout for "@Test" and "@TestTemplate" (For Github Action) [kafka]

m1a2st opened a new pull request, #16957:
URL: https://github.com/apache/kafka/pull/16957

   Jira: https://issues.apache.org/jira/browse/KAFKA-16813
   
   add ClusterTest and ClusterTests timout for 30 seconds.
   
   for github action
   
   ### Committer Checklist (excluded from commit message)
   - [ ] Verify design and implementation 
   - [ ] Verify test coverage and CI build status
   - [ ] Verify documentation (including upgrade notes)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16813: Add global timeout for "@Test" and "@TestTemplate" (For Github Action) [kafka]

Posted by "m1a2st (via GitHub)" <gi...@apache.org>.
m1a2st commented on code in PR #16957:
URL: https://github.com/apache/kafka/pull/16957#discussion_r1726912025


##########
core/src/test/java/kafka/test/annotation/ClusterTest.java:
##########
@@ -34,6 +35,7 @@
 @Target({METHOD})
 @Retention(RUNTIME)
 @TestTemplate
+@Timeout(30)

Review Comment:
   Hello @TaiJuWu,
   In this PR, we're setting a 30-second timeout limit for all new test infrastructure. We’ve already identified and fixed test cases exceeding this limit in a [previous PR](https://github.com/apache/kafka/pull/16796) and addressed these issues in a follow-up [PR](https://github.com/apache/kafka/pull/16810). Moving forward, developers should ensure their tests complete within this 30-second window to avoid prolonged execution times.
   WDYT?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16813: Add global timeout for "@Test" and "@TestTemplate" (For Github Action) [kafka]

Posted by "mumrah (via GitHub)" <gi...@apache.org>.
mumrah commented on PR #16957:
URL: https://github.com/apache/kafka/pull/16957#issuecomment-2361777157

   @m1a2st Is the PR title accurate? Does this add a global timeout for any test (`@Test`) or just our cluster tests? 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16813: Add global timeout for "@Test" and "@TestTemplate" (For Github Action) [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #16957:
URL: https://github.com/apache/kafka/pull/16957#discussion_r1727828772


##########
core/src/test/java/kafka/test/annotation/ClusterTest.java:
##########
@@ -34,6 +35,7 @@
 @Target({METHOD})
 @Retention(RUNTIME)
 @TestTemplate
+@Timeout(30)

Review Comment:
   > Does this work? I know we have tried setting global timeouts in the past with no success.
   
   yes, it works. see the QA (https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka-pr/detail/PR-16957/3/tests) and following error:
   ```
   Error
   java.util.concurrent.TimeoutException: testClientInstanceId(kafka.test.ClusterInstance) timed out after 30 seconds
   Stacktrace
   java.util.concurrent.TimeoutException: testClientInstanceId(kafka.test.ClusterInstance) timed out after 30 seconds
   	at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
   	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
   	at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
   	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
   	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
   	at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:647)
   	at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:272)
   	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
   	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
   	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
   	at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
   	at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
   	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
   	at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
   	at java.util.ArrayList.forEach(ArrayList.java:1259)
   	at java.util.ArrayList.forEach(ArrayList.java:1259)
   	Suppressed: org.apache.kafka.common.errors.InterruptException: java.lang.InterruptedException
   		at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.maybeThrowInterruptException(ConsumerNetworkClient.java:537)
   		at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:298)
   		at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:253)
   		at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:244)
   		at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.awaitMetadataUpdate(ConsumerNetworkClient.java:166)
   		at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:303)
   		at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:264)
   		at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.coordinatorUnknownAndUnreadySync(ConsumerCoordinator.java:451)
   		at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.commitOffsetsSync(ConsumerCoordinator.java:1112)
   		at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.maybeAutoCommitOffsetsSync(ConsumerCoordinator.java:1144)
   		at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.close(ConsumerCoordinator.java:980)
   		at org.apache.kafka.clients.consumer.internals.ClassicKafkaConsumer.lambda$close$3(ClassicKafkaConsumer.java:1140)
   		at org.apache.kafka.common.utils.Utils.swallow(Utils.java:1050)
   		at org.apache.kafka.clients.consumer.internals.ClassicKafkaConsumer.close(ClassicKafkaConsumer.java:1140)
   		at org.apache.kafka.clients.consumer.internals.ClassicKafkaConsumer.close(ClassicKafkaConsumer.java:1109)
   		at org.apache.kafka.clients.consumer.internals.ClassicKafkaConsumer.close(ClassicKafkaConsumer.java:1097)
   		at org.apache.kafka.clients.consumer.KafkaConsumer.close(KafkaConsumer.java:1757)
   		at kafka.admin.ClientTelemetryTest.testClientInstanceId(ClientTelemetryTest.java:116)
   		at java.lang.reflect.Method.invoke(Method.java:498)
   		... 16 more
   	Caused by: java.lang.InterruptedException
   		... 35 more
   ```
   
   BTW, it is not a "true" global timeout since it is applied to the tests using `ClusterTest`. That means the tests extending `KafkaServerTestHarness` are not limited.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16813: Add global timeout for "@Test" and "@TestTemplate" (For Github Action) [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #16957:
URL: https://github.com/apache/kafka/pull/16957#issuecomment-2307970417

   > I think I should add a constant in TestUtils for 30 second, It will be more easy to controll this parameter. WDYT?
   
   there are only 3 annotations needing the `@Timeout(30)`, so it should be fine to add the timeout annotation individually.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16813: Add global timeout for "@TestTemplate" [kafka]

Posted by "m1a2st (via GitHub)" <gi...@apache.org>.
m1a2st commented on PR #16957:
URL: https://github.com/apache/kafka/pull/16957#issuecomment-2361814767

   I consider the "@TestTemplate" is a symbol for these annotations `ClusterTest`, `ClusterTemplate`, `ClusterTests`, but I will fix the title more clearly.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16813: Add global timeout for "@Test" and "@TestTemplate" (For Github Action) [kafka]

Posted by "m1a2st (via GitHub)" <gi...@apache.org>.
m1a2st commented on PR #16957:
URL: https://github.com/apache/kafka/pull/16957#issuecomment-2326888103

   I open these jira tasks to trace these timeout tests
   https://issues.apache.org/jira/browse/KAFKA-17471
   https://issues.apache.org/jira/browse/KAFKA-17472
   https://issues.apache.org/jira/browse/KAFKA-17473


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16813: Add global timeout for "@Test" and "@TestTemplate" (For Github Action) [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #16957:
URL: https://github.com/apache/kafka/pull/16957#issuecomment-2304037720

   @m1a2st please rebase code and fix conflicts


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16813: Add global timeout for "@Test" and "@TestTemplate" (For Github Action) [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #16957:
URL: https://github.com/apache/kafka/pull/16957#issuecomment-2361572165

   @m1a2st Could you please check the failed tests? It seems no cluster tests have timeout error, so maybe we can keep this "60 seconds" WDYT?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16813: Add global timeout for "@Test" and "@TestTemplate" (For Github Action) [kafka]

Posted by "m1a2st (via GitHub)" <gi...@apache.org>.
m1a2st commented on PR #16957:
URL: https://github.com/apache/kafka/pull/16957#issuecomment-2326781500

   I will check these tests which timeout 30 second
   - `ResetConsumerGroupOffsetTest.testResetOffsetsToEarliestOnTopicsAndPartitions`
   - `ResetConsumerGroupOffsetTest.testResetOffsetsAllTopicsAllGroups`
   - `DescribeConsumerGroupTest.testDescribeWithMultiPartitionTopicAndMultipleConsumers`
   - `DescribeConsumerGroupTest.testDescribeWithConsumersWithoutAssignedPartitions`
   - `ZkMigrationIntegrationTest.testMigrateTopicDeletions`
   - `ClientTelemetryTest.testClientInstanceId`
   ---
   This test is timeout for 120s
   - `KafkaAdminClientTest.testClientSideTimeoutAfterFailureToReceiveResponse`
   ---
   This is flaky 
   - [RestoreIntegrationTest.shouldInvokeUserDefinedGlobalStateRestoreListener](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-16957/5/testReport/junit/org.apache.kafka.streams.integration/RestoreIntegrationTest/Build___JDK_21_and_Scala_2_13___shouldInvokeUserDefinedGlobalStateRestoreListener__/)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16813: Add global timeout for "@Test" and "@TestTemplate" (For Github Action) [kafka]

Posted by "mumrah (via GitHub)" <gi...@apache.org>.
mumrah commented on PR #16957:
URL: https://github.com/apache/kafka/pull/16957#issuecomment-2326823233

   @chia7712 I have a separate PR for ZkMigrationIntegrationTest https://github.com/apache/kafka/pull/17004. This test definitely needs to set its own timeouts as it is rather complicated. In retrospect, some parts of this tests probably should have been system tests.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16813: Add global timeout for `@ClusterTemplate`, `@ClusterTest` and `@ClusterTests` [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 merged PR #16957:
URL: https://github.com/apache/kafka/pull/16957


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16813: Add global timeout for "@Test" and "@TestTemplate" (For Github Action) [kafka]

Posted by "mumrah (via GitHub)" <gi...@apache.org>.
mumrah commented on PR #16957:
URL: https://github.com/apache/kafka/pull/16957#issuecomment-2307276425

   @m1a2st can you pull in the latest trunk changes? I have added code to report the JUnit failures in the Github Actions workflows. I want to do a few runs with this change to ensure we haven't introduced a bunch of timeout failures


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16813: Add global timeout for "@Test" and "@TestTemplate" (For Github Action) [kafka]

Posted by "m1a2st (via GitHub)" <gi...@apache.org>.
m1a2st commented on PR #16957:
URL: https://github.com/apache/kafka/pull/16957#issuecomment-2361781955

   @mumrah, Thanks for your reminder, we only add timeout for cluster tests in this PR, I should modify the title.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16813: Add global timeout for "@Test" and "@TestTemplate" (For Github Action) [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #16957:
URL: https://github.com/apache/kafka/pull/16957#issuecomment-2325047599

   @m1a2st Could you please list the timeout tests? Maybe we can file PR to speedup them


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16813: Add global timeout for "@Test" and "@TestTemplate" (For Github Action) [kafka]

Posted by "m1a2st (via GitHub)" <gi...@apache.org>.
m1a2st commented on code in PR #16957:
URL: https://github.com/apache/kafka/pull/16957#discussion_r1728916881


##########
core/src/test/java/kafka/test/annotation/ClusterTest.java:
##########
@@ -34,6 +35,7 @@
 @Target({METHOD})
 @Retention(RUNTIME)
 @TestTemplate
+@Timeout(30)

Review Comment:
   > Is it possible for `@ClusterTests` to override this annotation for longer timeout?
   
   I don't consider I can achieve this using the `@Timeout` annotation alone. Developers should avoid scenarios where they need to modify this setting. If a test genuinely requires more than 30 seconds, the `@Timeout`annotation can be applied at the method level, it will override this annotation.
   ```java
       @Timeout(1)
       @ClusterTests({
               @ClusterTest(types = {Type.ZK}),
               @ClusterTest(types = {Type.KRAFT}, serverProperties = {
                       @ClusterConfigProperty(key = AUTHORIZER_CLASS_NAME_CONFIG, value = STANDARD_AUTHORIZER)
               })
       })
       public void testAclCliWithAdminAPI(ClusterInstance cluster) throws InterruptedException {
           Thread.sleep(2000);
           testAclCli(cluster, adminArgs(cluster.bootstrapServers(), Optional.empty()));
       }
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16813: Add global timeout for "@Test" and "@TestTemplate" (For Github Action) [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #16957:
URL: https://github.com/apache/kafka/pull/16957#issuecomment-2307375843

   @m1a2st could you please add timeout to `@ClusterTemplate`?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16813: Add global timeout for "@Test" and "@TestTemplate" (For Github Action) [kafka]

Posted by "TaiJuWu (via GitHub)" <gi...@apache.org>.
TaiJuWu commented on code in PR #16957:
URL: https://github.com/apache/kafka/pull/16957#discussion_r1726688336


##########
core/src/test/java/kafka/test/annotation/ClusterTest.java:
##########
@@ -34,6 +35,7 @@
 @Target({METHOD})
 @Retention(RUNTIME)
 @TestTemplate
+@Timeout(30)

Review Comment:
   In some case, the timeout is 120 and the other is 30.
   Should we set it to 120 to avoid to make test flaky?
   
   WDYT?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16813: Add global timeout for "@Test" and "@TestTemplate" (For Github Action) [kafka]

Posted by "mumrah (via GitHub)" <gi...@apache.org>.
mumrah commented on code in PR #16957:
URL: https://github.com/apache/kafka/pull/16957#discussion_r1727651045


##########
core/src/test/java/kafka/test/annotation/ClusterTest.java:
##########
@@ -34,6 +35,7 @@
 @Target({METHOD})
 @Retention(RUNTIME)
 @TestTemplate
+@Timeout(30)

Review Comment:
   Does this work? I know we have tried setting global timeouts in the past with no success.
   
   Also, 30 seconds might be too short for `ClusterTest` tests since these are integration tests. I think 120 is more reasonable.
   
   Can you add a new test that just sleeps for 5 minutes to verify the behavior? 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16813: Add global timeout for "@Test" and "@TestTemplate" (For Github Action) [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #16957:
URL: https://github.com/apache/kafka/pull/16957#issuecomment-2308804723

   ```
   | ❌ | LagFetchIntegrationTest | shouldFetchLagsDuringRebalancingWithNoOptimization() | java.lang.AssertionError: 
   Expected: <0L>
        but: was <5L> | 63.29s |
   ```
   I have filed https://issues.apache.org/jira/browse/KAFKA-17418 to fix the markdown


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16813: Add global timeout for "@Test" and "@TestTemplate" (For Github Action) [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #16957:
URL: https://github.com/apache/kafka/pull/16957#issuecomment-2329649435

   @m1a2st Could you please rebase code to trigger QA again?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16813: Add global timeout for "@Test" and "@TestTemplate" (For Github Action) [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #16957:
URL: https://github.com/apache/kafka/pull/16957#issuecomment-2337981828

   @m1a2st Could you please increase the timeout from 30 secs to 60 secs? It seems to me 30 secs is too small. Also, please merge trunk to run CI again


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16813: Add global timeout for "@TestTemplate" [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #16957:
URL: https://github.com/apache/kafka/pull/16957#issuecomment-2361803088

   @m1a2st Do you forgot to add `ClusterTest` and `ClusterTests` to the title?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16813: Add global timeout for "@Test" and "@TestTemplate" (For Github Action) [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #16957:
URL: https://github.com/apache/kafka/pull/16957#discussion_r1758132188


##########
core/src/test/java/kafka/test/annotation/ClusterTest.java:
##########
@@ -34,6 +35,7 @@
 @Target({METHOD})
 @Retention(RUNTIME)
 @TestTemplate
+@Timeout(30)

Review Comment:
   @m1a2st please change it to `30` second :)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16813: Add global timeout for "@Test" and "@TestTemplate" (For Github Action) [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #16957:
URL: https://github.com/apache/kafka/pull/16957#issuecomment-2356919583

   #17117 is merged now, so @m1a2st could you please rebase code?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16813: Add global timeout for "@Test" and "@TestTemplate" (For Github Action) [kafka]

Posted by "TaiJuWu (via GitHub)" <gi...@apache.org>.
TaiJuWu commented on code in PR #16957:
URL: https://github.com/apache/kafka/pull/16957#discussion_r1728049863


##########
core/src/test/java/kafka/test/annotation/ClusterTest.java:
##########
@@ -34,6 +35,7 @@
 @Target({METHOD})
 @Retention(RUNTIME)
 @TestTemplate
+@Timeout(30)

Review Comment:
   > Hello @TaiJuWu, In this PR, we're setting a 30-second timeout limit for all new test infrastructure. We’ve already identified and fixed test cases exceeding this limit in a [previous PR](https://github.com/apache/kafka/pull/16796) and addressed these issues in a follow-up [PR](https://github.com/apache/kafka/pull/16810). Moving forward, developers should ensure their tests complete within this 30-second window to avoid prolonged execution times. WDYT?
   
   Got it. Sound great. Thanks for your explain.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16813: Add global timeout for "@Test" and "@TestTemplate" (For Github Action) [kafka]

Posted by "m1a2st (via GitHub)" <gi...@apache.org>.
m1a2st commented on PR #16957:
URL: https://github.com/apache/kafka/pull/16957#issuecomment-2307955764

   > @m1a2st  can you pull in the latest trunk changes? I have added code to report the JUnit failures in the Github Actions workflows. I want to do a few runs with this change to ensure we haven't introduced a bunch of timeout failures
   
   I rebased it. 
   
   > @m1a2st could you please add timeout to @ClusterTemplate?
   
   Should I add a constant in TestUtils for 30 second, It will be more easy to controll this parameter.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16813: Add global timeout for "@Test" and "@TestTemplate" (For Github Action) [kafka]

Posted by "TaiJuWu (via GitHub)" <gi...@apache.org>.
TaiJuWu commented on code in PR #16957:
URL: https://github.com/apache/kafka/pull/16957#discussion_r1726688336


##########
core/src/test/java/kafka/test/annotation/ClusterTest.java:
##########
@@ -34,6 +35,7 @@
 @Target({METHOD})
 @Retention(RUNTIME)
 @TestTemplate
+@Timeout(30)

Review Comment:
   In some case, the timeout is 120 and the other is 30.
   Should we set it to 120 to avoid to make test flaky?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16813: Add global timeout for "@Test" and "@TestTemplate" (For Github Action) [kafka]

Posted by "mumrah (via GitHub)" <gi...@apache.org>.
mumrah commented on code in PR #16957:
URL: https://github.com/apache/kafka/pull/16957#discussion_r1727999754


##########
core/src/test/java/kafka/test/annotation/ClusterTest.java:
##########
@@ -34,6 +35,7 @@
 @Target({METHOD})
 @Retention(RUNTIME)
 @TestTemplate
+@Timeout(30)

Review Comment:
   Very nice! 
   
   Is it possible for `@ClusterTest`s to override this annotation for longer timeout?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16813: Add global timeout for "@Test" and "@TestTemplate" (For Github Action) [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #16957:
URL: https://github.com/apache/kafka/pull/16957#discussion_r1729196478


##########
core/src/test/java/kafka/test/annotation/ClusterTest.java:
##########
@@ -34,6 +35,7 @@
 @Target({METHOD})
 @Retention(RUNTIME)
 @TestTemplate
+@Timeout(30)

Review Comment:
   > Is it possible for @ClusterTests to override this annotation for longer timeout?
   
   yes, it can override the timeout, but each test case created by `@ClusterTests` have isolated elapsed time. That means we don't need to have "a timeout" to include all test cases of a `@ClusterTests`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16813: Add global timeout for "@Test" and "@TestTemplate" (For Github Action) [kafka]

Posted by "m1a2st (via GitHub)" <gi...@apache.org>.
m1a2st commented on PR #16957:
URL: https://github.com/apache/kafka/pull/16957#issuecomment-2361729960

   Hello @chia7712, 
   The fail tests are:
   - `OffsetsApiIntegrationTest#testAlterSinkConnectorOffsetsDifferentKafkaClusterTargeted` the reason is can't met condition in 30s
   - `PlaintextConsumerSubscriptionTest` the reason is  Failed to close kafka consumer
   
   I think we could keep the 60s timeout for cluster test.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16813: Add global timeout for "@Test" and "@TestTemplate" (For Github Action) [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #16957:
URL: https://github.com/apache/kafka/pull/16957#issuecomment-2326799913

   `testClientSideTimeoutAfterFailureToReceiveResponse` is known flaky (https://issues.apache.org/jira/browse/KAFKA-15916)
   
   `ZkMigrationIntegrationTest` is a bit complicated, so it should have different timeout. The other IT needs to be improved to reduce the elapsed time. Could you please file sub-tasks (jira) for each test class?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org