You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2021/04/16 06:27:34 UTC
[GitHub] [kafka] showuon opened a new pull request #10547: KAFKA-12284: increase request timeout to make tests reliable
showuon opened a new pull request #10547:
URL: https://github.com/apache/kafka/pull/10547
The `MirrorConnectorsIntegrationTests` recently failed with
```
TimeoutException: The request timed out.
```
quite frequently. After investigation, the reason is the `createTopic` in the server doesn't complete within `request.timeout.ms` (default to 30 seconds). It makes sense that the server is pretty slow during 2 connector cluster (primary and backup) started at the same time. What I did are:
1. increase the timeout value to 60 seconds to make the tests reliable.
2. wait for internal topics created after connector cluster started, and then try to create our test topics.
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] showuon commented on pull request #10547: KAFKA-12284: increase request timeout to make tests reliable
Posted by GitBox <gi...@apache.org>.
showuon commented on pull request #10547:
URL: https://github.com/apache/kafka/pull/10547#issuecomment-824644809
Done. Let's wait and see :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] showuon commented on a change in pull request #10547: KAFKA-12284: increase request timeout to make tests reliable
Posted by GitBox <gi...@apache.org>.
showuon commented on a change in pull request #10547:
URL: https://github.com/apache/kafka/pull/10547#discussion_r621231344
##########
File path: connect/runtime/src/test/java/org/apache/kafka/connect/util/clusters/EmbeddedKafkaCluster.java
##########
@@ -337,18 +337,28 @@ public void createTopic(String topic) {
* @param topic The name of the topic.
*/
public void createTopic(String topic, int partitions) {
- createTopic(topic, partitions, 1, new HashMap<>());
+ createTopic(topic, partitions, 1, new HashMap<>(), new Properties());
Review comment:
Good to me. I'll work on it tomorrow! Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] chia7712 commented on a change in pull request #10547: KAFKA-12284: increase request timeout to make tests reliable
Posted by GitBox <gi...@apache.org>.
chia7712 commented on a change in pull request #10547:
URL: https://github.com/apache/kafka/pull/10547#discussion_r621784059
##########
File path: connect/runtime/src/test/java/org/apache/kafka/connect/util/clusters/EmbeddedKafkaCluster.java
##########
@@ -337,18 +337,28 @@ public void createTopic(String topic) {
* @param topic The name of the topic.
*/
public void createTopic(String topic, int partitions) {
- createTopic(topic, partitions, 1, new HashMap<>());
+ createTopic(topic, partitions, 1, Collections.emptyMap(), new Properties());
Review comment:
It seems we don't need to pass `new Properties()`, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] chia7712 commented on a change in pull request #10547: KAFKA-12284: increase request timeout to make tests reliable
Posted by GitBox <gi...@apache.org>.
chia7712 commented on a change in pull request #10547:
URL: https://github.com/apache/kafka/pull/10547#discussion_r621167591
##########
File path: connect/runtime/src/test/java/org/apache/kafka/connect/util/clusters/EmbeddedKafkaCluster.java
##########
@@ -337,18 +337,28 @@ public void createTopic(String topic) {
* @param topic The name of the topic.
*/
public void createTopic(String topic, int partitions) {
- createTopic(topic, partitions, 1, new HashMap<>());
+ createTopic(topic, partitions, 1, new HashMap<>(), new Properties());
Review comment:
How about replace this by `createTopic(topic, partitions, 1, Collections.emptyMap());`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] showuon commented on pull request #10547: KAFKA-12284: increase request timeout to make tests reliable
Posted by GitBox <gi...@apache.org>.
showuon commented on pull request #10547:
URL: https://github.com/apache/kafka/pull/10547#issuecomment-821176472
Failed tests are all un-related. (Not in MirrorMakerIntegrationTest)
```
)
Build / JDK 8 and Scala 2.12 / kafka.server.RaftClusterTest.testCreateClusterAndCreateAndManyTopicsWithManyPartitions()
Build / JDK 15 and Scala 2.13 / org.apache.kafka.connect.integration.ConnectorClientPolicyIntegrationTest.testCreateWithNotAllowedOverridesForPrincipalPolicy
Build / JDK 15 and Scala 2.13 / kafka.admin.DeleteOffsetsConsumerGroupCommandIntegrationTest.testDeleteOffsetsNonExistingGroup()
Build / JDK 15 and Scala 2.13 / kafka.admin.ResetConsumerGroupOffsetTest.testResetOffsetsByDurationToEarliest()
Build / JDK 15 and Scala 2.13 / kafka.admin.ResetConsumerGroupOffsetTest.testResetOffsetsExistingTopicSelectedGroups()
Build / JDK 15 and Scala 2.13 / kafka.admin.ResetConsumerGroupOffsetTest.testResetOffsetsToZonedDateTime()
Build / JDK 15 and Scala 2.13 / kafka.server.RaftClusterTest.testCreateClusterAndCreateAndManyTopicsWithManyPartitions()
Build / JDK 15 and Scala 2.13 / org.apache.kafka.streams.integration.KTableKTableForeignKeyInnerJoinMultiIntegrationTest.shouldInnerJoinMultiPartitionQueryable
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] ning2008wisc commented on pull request #10547: KAFKA-12284: increase request timeout to make tests reliable
Posted by GitBox <gi...@apache.org>.
ning2008wisc commented on pull request #10547:
URL: https://github.com/apache/kafka/pull/10547#issuecomment-822535696
thanks for the PR :) change seems reasonable and lgtm (if consistent test results can be verified by multiple runs). The final review will be done by the committer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] showuon commented on pull request #10547: KAFKA-12284: increase request timeout to make tests reliable
Posted by GitBox <gi...@apache.org>.
showuon commented on pull request #10547:
URL: https://github.com/apache/kafka/pull/10547#issuecomment-821069903
@mimaison @ning2008wisc , I've updated the PR, please help review it. Thank you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] showuon commented on pull request #10547: KAFKA-12284: increase request timeout to make tests reliable
Posted by GitBox <gi...@apache.org>.
showuon commented on pull request #10547:
URL: https://github.com/apache/kafka/pull/10547#issuecomment-828180085
Failed tests are all un-related. (Not in MirrorMakerIntegrationTest)
```
Build / JDK 11 and Scala 2.13 / kafka.network.ConnectionQuotasTest.testListenerConnectionRateLimitWhenActualRateAboveLimit()
Build / JDK 11 and Scala 2.13 / kafka.server.RaftClusterTest.testCreateClusterAndCreateAndManyTopics()
Build / JDK 8 and Scala 2.12 / org.apache.kafka.streams.integration.StoreQueryIntegrationTest.shouldQueryOnlyActivePartitionStoresByDefault
Build / JDK 15 and Scala 2.13 / kafka.api.TransactionsTest.testCommitTransactionTimeout()
Build / JDK 15 and Scala 2.13 / org.apache.kafka.streams.integration.KTableKTableForeignKeyInnerJoinMultiIntegrationTest.shouldInnerJoinMultiPartitionQueryable
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] showuon commented on a change in pull request #10547: KAFKA-12284: increase request timeout to make tests reliable
Posted by GitBox <gi...@apache.org>.
showuon commented on a change in pull request #10547:
URL: https://github.com/apache/kafka/pull/10547#discussion_r621757453
##########
File path: connect/runtime/src/test/java/org/apache/kafka/connect/util/clusters/EmbeddedKafkaCluster.java
##########
@@ -337,18 +337,28 @@ public void createTopic(String topic) {
* @param topic The name of the topic.
*/
public void createTopic(String topic, int partitions) {
- createTopic(topic, partitions, 1, new HashMap<>());
+ createTopic(topic, partitions, 1, new HashMap<>(), new Properties());
Review comment:
Updated. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] chia7712 commented on pull request #10547: KAFKA-12284: increase request timeout to make tests reliable
Posted by GitBox <gi...@apache.org>.
chia7712 commented on pull request #10547:
URL: https://github.com/apache/kafka/pull/10547#issuecomment-824640807
@showuon Could you merge trunk to trigger QA again?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] chia7712 merged pull request #10547: KAFKA-12284: increase request timeout to make tests reliable
Posted by GitBox <gi...@apache.org>.
chia7712 merged pull request #10547:
URL: https://github.com/apache/kafka/pull/10547
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] showuon commented on pull request #10547: KAFKA-12284: increase request timeout to make tests reliable
Posted by GitBox <gi...@apache.org>.
showuon commented on pull request #10547:
URL: https://github.com/apache/kafka/pull/10547#issuecomment-822930347
Thanks @ning2008wisc !
@mimaison , see if you have any comments. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] showuon commented on pull request #10547: KAFKA-12284: increase request timeout to make tests reliable
Posted by GitBox <gi...@apache.org>.
showuon commented on pull request #10547:
URL: https://github.com/apache/kafka/pull/10547#issuecomment-824789512
Failed tests are
```
Build / JDK 8 and Scala 2.12 / org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationSSLTest.testReplication()
Build / JDK 8 and Scala 2.12 / kafka.server.RaftClusterTest.testCreateClusterAndCreateAndManyTopicsWithManyPartitions()
Build / JDK 15 and Scala 2.13 / org.apache.kafka.connect.integration.RebalanceSourceConnectorsIntegrationTest.testDeleteConnector
Build / JDK 15 and Scala 2.13 / org.apache.kafka.streams.integration.KTableKTableForeignKeyInnerJoinMultiIntegrationTest.shouldInnerJoinMultiPartitionQueryable
```
The failed `MirrorConnectorsIntegrationSSLTest.testReplication()` is not request timeout anymore.
```
org.opentest4j.AssertionFailedError: Condition not met within timeout 20000. Offsets not translated downstream to primary cluster. ==> expected: <true> but was: <false>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] showuon commented on a change in pull request #10547: KAFKA-12284: increase request timeout to make tests reliable
Posted by GitBox <gi...@apache.org>.
showuon commented on a change in pull request #10547:
URL: https://github.com/apache/kafka/pull/10547#discussion_r621788834
##########
File path: connect/runtime/src/test/java/org/apache/kafka/connect/util/clusters/EmbeddedKafkaCluster.java
##########
@@ -337,18 +337,28 @@ public void createTopic(String topic) {
* @param topic The name of the topic.
*/
public void createTopic(String topic, int partitions) {
- createTopic(topic, partitions, 1, new HashMap<>());
+ createTopic(topic, partitions, 1, Collections.emptyMap(), new Properties());
Review comment:
yes, you're right! Sorry, I didn't catch your idea. Updated. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org