You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2021/04/16 06:27:34 UTC

[GitHub] [kafka] showuon opened a new pull request #10547: KAFKA-12284: increase request timeout to make tests reliable

showuon opened a new pull request #10547:
URL: https://github.com/apache/kafka/pull/10547


   The `MirrorConnectorsIntegrationTests` recently failed with 
   ```
   TimeoutException: The request timed out.
   ```
   quite frequently. After investigation, the reason is the `createTopic` in the server doesn't complete within `request.timeout.ms` (default to 30 seconds). It makes sense that the server is pretty slow during 2 connector cluster (primary and backup) started at the same time. What I did are:
   1. increase the timeout value to 60 seconds to make the tests reliable.
   2. wait for internal topics created after connector cluster started, and then try to create our test topics.
   
   
   ### Committer Checklist (excluded from commit message)
   - [ ] Verify design and implementation 
   - [ ] Verify test coverage and CI build status
   - [ ] Verify documentation (including upgrade notes)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [kafka] showuon commented on pull request #10547: KAFKA-12284: increase request timeout to make tests reliable

Posted by GitBox <gi...@apache.org>.
showuon commented on pull request #10547:
URL: https://github.com/apache/kafka/pull/10547#issuecomment-824644809


   Done. Let's wait and see :)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [kafka] showuon commented on a change in pull request #10547: KAFKA-12284: increase request timeout to make tests reliable

Posted by GitBox <gi...@apache.org>.
showuon commented on a change in pull request #10547:
URL: https://github.com/apache/kafka/pull/10547#discussion_r621231344



##########
File path: connect/runtime/src/test/java/org/apache/kafka/connect/util/clusters/EmbeddedKafkaCluster.java
##########
@@ -337,18 +337,28 @@ public void createTopic(String topic) {
      * @param topic The name of the topic.
      */
     public void createTopic(String topic, int partitions) {
-        createTopic(topic, partitions, 1, new HashMap<>());
+        createTopic(topic, partitions, 1, new HashMap<>(), new Properties());

Review comment:
       Good to me. I'll work on it tomorrow! Thanks.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [kafka] chia7712 commented on a change in pull request #10547: KAFKA-12284: increase request timeout to make tests reliable

Posted by GitBox <gi...@apache.org>.
chia7712 commented on a change in pull request #10547:
URL: https://github.com/apache/kafka/pull/10547#discussion_r621784059



##########
File path: connect/runtime/src/test/java/org/apache/kafka/connect/util/clusters/EmbeddedKafkaCluster.java
##########
@@ -337,18 +337,28 @@ public void createTopic(String topic) {
      * @param topic The name of the topic.
      */
     public void createTopic(String topic, int partitions) {
-        createTopic(topic, partitions, 1, new HashMap<>());
+        createTopic(topic, partitions, 1, Collections.emptyMap(), new Properties());

Review comment:
       It seems we don't need to pass `new Properties()`, right?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [kafka] chia7712 commented on a change in pull request #10547: KAFKA-12284: increase request timeout to make tests reliable

Posted by GitBox <gi...@apache.org>.
chia7712 commented on a change in pull request #10547:
URL: https://github.com/apache/kafka/pull/10547#discussion_r621167591



##########
File path: connect/runtime/src/test/java/org/apache/kafka/connect/util/clusters/EmbeddedKafkaCluster.java
##########
@@ -337,18 +337,28 @@ public void createTopic(String topic) {
      * @param topic The name of the topic.
      */
     public void createTopic(String topic, int partitions) {
-        createTopic(topic, partitions, 1, new HashMap<>());
+        createTopic(topic, partitions, 1, new HashMap<>(), new Properties());

Review comment:
       How about replace this by `createTopic(topic, partitions, 1, Collections.emptyMap());`?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [kafka] showuon commented on pull request #10547: KAFKA-12284: increase request timeout to make tests reliable

Posted by GitBox <gi...@apache.org>.
showuon commented on pull request #10547:
URL: https://github.com/apache/kafka/pull/10547#issuecomment-821176472


   Failed tests are all un-related. (Not in MirrorMakerIntegrationTest)
   ```
   )
   
       Build / JDK 8 and Scala 2.12 / kafka.server.RaftClusterTest.testCreateClusterAndCreateAndManyTopicsWithManyPartitions()
       Build / JDK 15 and Scala 2.13 / org.apache.kafka.connect.integration.ConnectorClientPolicyIntegrationTest.testCreateWithNotAllowedOverridesForPrincipalPolicy
       Build / JDK 15 and Scala 2.13 / kafka.admin.DeleteOffsetsConsumerGroupCommandIntegrationTest.testDeleteOffsetsNonExistingGroup()
       Build / JDK 15 and Scala 2.13 / kafka.admin.ResetConsumerGroupOffsetTest.testResetOffsetsByDurationToEarliest()
       Build / JDK 15 and Scala 2.13 / kafka.admin.ResetConsumerGroupOffsetTest.testResetOffsetsExistingTopicSelectedGroups()
       Build / JDK 15 and Scala 2.13 / kafka.admin.ResetConsumerGroupOffsetTest.testResetOffsetsToZonedDateTime()
       Build / JDK 15 and Scala 2.13 / kafka.server.RaftClusterTest.testCreateClusterAndCreateAndManyTopicsWithManyPartitions()
       Build / JDK 15 and Scala 2.13 / org.apache.kafka.streams.integration.KTableKTableForeignKeyInnerJoinMultiIntegrationTest.shouldInnerJoinMultiPartitionQueryable
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [kafka] ning2008wisc commented on pull request #10547: KAFKA-12284: increase request timeout to make tests reliable

Posted by GitBox <gi...@apache.org>.
ning2008wisc commented on pull request #10547:
URL: https://github.com/apache/kafka/pull/10547#issuecomment-822535696


   thanks for the PR :) change seems reasonable and lgtm (if consistent test results can be verified by multiple runs). The final review will be done by the committer. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [kafka] showuon commented on pull request #10547: KAFKA-12284: increase request timeout to make tests reliable

Posted by GitBox <gi...@apache.org>.
showuon commented on pull request #10547:
URL: https://github.com/apache/kafka/pull/10547#issuecomment-821069903


   @mimaison @ning2008wisc , I've updated the PR, please help review it. Thank you.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [kafka] showuon commented on pull request #10547: KAFKA-12284: increase request timeout to make tests reliable

Posted by GitBox <gi...@apache.org>.
showuon commented on pull request #10547:
URL: https://github.com/apache/kafka/pull/10547#issuecomment-828180085


   Failed tests are all un-related. (Not in MirrorMakerIntegrationTest)
   ```
       Build / JDK 11 and Scala 2.13 / kafka.network.ConnectionQuotasTest.testListenerConnectionRateLimitWhenActualRateAboveLimit()
       Build / JDK 11 and Scala 2.13 / kafka.server.RaftClusterTest.testCreateClusterAndCreateAndManyTopics()
       Build / JDK 8 and Scala 2.12 / org.apache.kafka.streams.integration.StoreQueryIntegrationTest.shouldQueryOnlyActivePartitionStoresByDefault
       Build / JDK 15 and Scala 2.13 / kafka.api.TransactionsTest.testCommitTransactionTimeout()
       Build / JDK 15 and Scala 2.13 / org.apache.kafka.streams.integration.KTableKTableForeignKeyInnerJoinMultiIntegrationTest.shouldInnerJoinMultiPartitionQueryable
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [kafka] showuon commented on a change in pull request #10547: KAFKA-12284: increase request timeout to make tests reliable

Posted by GitBox <gi...@apache.org>.
showuon commented on a change in pull request #10547:
URL: https://github.com/apache/kafka/pull/10547#discussion_r621757453



##########
File path: connect/runtime/src/test/java/org/apache/kafka/connect/util/clusters/EmbeddedKafkaCluster.java
##########
@@ -337,18 +337,28 @@ public void createTopic(String topic) {
      * @param topic The name of the topic.
      */
     public void createTopic(String topic, int partitions) {
-        createTopic(topic, partitions, 1, new HashMap<>());
+        createTopic(topic, partitions, 1, new HashMap<>(), new Properties());

Review comment:
       Updated. Thanks.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [kafka] chia7712 commented on pull request #10547: KAFKA-12284: increase request timeout to make tests reliable

Posted by GitBox <gi...@apache.org>.
chia7712 commented on pull request #10547:
URL: https://github.com/apache/kafka/pull/10547#issuecomment-824640807


   @showuon Could you merge trunk to trigger QA again?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [kafka] chia7712 merged pull request #10547: KAFKA-12284: increase request timeout to make tests reliable

Posted by GitBox <gi...@apache.org>.
chia7712 merged pull request #10547:
URL: https://github.com/apache/kafka/pull/10547


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [kafka] showuon commented on pull request #10547: KAFKA-12284: increase request timeout to make tests reliable

Posted by GitBox <gi...@apache.org>.
showuon commented on pull request #10547:
URL: https://github.com/apache/kafka/pull/10547#issuecomment-822930347


   Thanks @ning2008wisc ! 
   @mimaison , see if you have any comments. Thanks.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [kafka] showuon commented on pull request #10547: KAFKA-12284: increase request timeout to make tests reliable

Posted by GitBox <gi...@apache.org>.
showuon commented on pull request #10547:
URL: https://github.com/apache/kafka/pull/10547#issuecomment-824789512


   Failed tests are
   ```
       Build / JDK 8 and Scala 2.12 / org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationSSLTest.testReplication()
       Build / JDK 8 and Scala 2.12 / kafka.server.RaftClusterTest.testCreateClusterAndCreateAndManyTopicsWithManyPartitions()
       Build / JDK 15 and Scala 2.13 / org.apache.kafka.connect.integration.RebalanceSourceConnectorsIntegrationTest.testDeleteConnector
       Build / JDK 15 and Scala 2.13 / org.apache.kafka.streams.integration.KTableKTableForeignKeyInnerJoinMultiIntegrationTest.shouldInnerJoinMultiPartitionQueryable
   ```
   
   
   The failed `MirrorConnectorsIntegrationSSLTest.testReplication()` is not request timeout anymore.
   ```
   org.opentest4j.AssertionFailedError: Condition not met within timeout 20000. Offsets not translated downstream to primary cluster. ==> expected: <true> but was: <false>
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [kafka] showuon commented on a change in pull request #10547: KAFKA-12284: increase request timeout to make tests reliable

Posted by GitBox <gi...@apache.org>.
showuon commented on a change in pull request #10547:
URL: https://github.com/apache/kafka/pull/10547#discussion_r621788834



##########
File path: connect/runtime/src/test/java/org/apache/kafka/connect/util/clusters/EmbeddedKafkaCluster.java
##########
@@ -337,18 +337,28 @@ public void createTopic(String topic) {
      * @param topic The name of the topic.
      */
     public void createTopic(String topic, int partitions) {
-        createTopic(topic, partitions, 1, new HashMap<>());
+        createTopic(topic, partitions, 1, Collections.emptyMap(), new Properties());

Review comment:
       yes, you're right! Sorry, I didn't catch your idea. Updated. Thanks.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org