You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by GitBox <gi...@apache.org> on 2020/01/09 20:30:14 UTC
[GitHub] [camel-kafka-connector] valdar opened a new pull request #66: Added
support for testing the Kafka service using Strimzi
valdar opened a new pull request #66: Added support for testing the Kafka service using Strimzi
URL: https://github.com/apache/camel-kafka-connector/pull/66
Also updates the default Kafka service to use the containers from the Strimzi project
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [camel-kafka-connector] orpiske commented on issue #66: Added
support for testing the Kafka service using Strimzi
Posted by GitBox <gi...@apache.org>.
orpiske commented on issue #66: Added support for testing the Kafka service using Strimzi
URL: https://github.com/apache/camel-kafka-connector/pull/66#issuecomment-573402669
> @orpiske I fixed as you suggested and made possible to configure the image at test-run time. I also bumped the used image version to `strimzi/kafka:0.15.0-kafka-2.3.1`. Can you please review it?
I think everything looks alright. I think we should use this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [camel-kafka-connector] orpiske commented on issue #66: Added
support for testing the Kafka service using Strimzi
Posted by GitBox <gi...@apache.org>.
orpiske commented on issue #66: Added support for testing the Kafka service using Strimzi
URL: https://github.com/apache/camel-kafka-connector/pull/66#issuecomment-572748420
I added some comments with the changes needed to bring it up to date with the other changes in the test code I did in the last few days.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [camel-kafka-connector] valdar commented on a change in pull
request #66: Added support for testing the Kafka service using Strimzi
Posted by GitBox <gi...@apache.org>.
valdar commented on a change in pull request #66: Added support for testing the Kafka service using Strimzi
URL: https://github.com/apache/camel-kafka-connector/pull/66#discussion_r365022975
##########
File path: tests/pom.xml
##########
@@ -178,7 +178,7 @@
<artifactId>maven-failsafe-plugin</artifactId>
<configuration>
<!-- This is needed for the AWS Kinesis tests -->
- <argLine>-Dcom.amazonaws.sdk.disableCbor=true</argLine>
+ <argLine>-Dcom.amazonaws.sdk.disableCbor=true -Ditest.strimzi.container.image=${itest.strimzi.container.image} -Ditest.zookeeper.container.image=${itest.zookeeper.container.image}</argLine>
Review comment:
Is this the best way to acieve what I needed to @orpiske ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [camel-kafka-connector] valdar commented on a change in pull
request #66: Added support for testing the Kafka service using Strimzi
Posted by GitBox <gi...@apache.org>.
valdar commented on a change in pull request #66: Added support for testing the Kafka service using Strimzi
URL: https://github.com/apache/camel-kafka-connector/pull/66#discussion_r365586777
##########
File path: tests/src/test/java/org/apache/camel/kafkaconnector/services/kafka/StrimziContainer.java
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.camel.kafkaconnector.services.kafka;
+
+import java.util.function.Consumer;
+import com.github.dockerjava.api.command.CreateContainerCmd;
+import org.testcontainers.containers.GenericContainer;
+import org.testcontainers.containers.Network;
+import org.testcontainers.containers.wait.strategy.Wait;
+
+public class StrimziContainer extends GenericContainer {
+ private static final String STRIMZI_CONTAINER = System.getProperty("itest.strimzi.container.image");
+ private static final int KAFKA_PORT = 9092;
+
+ public StrimziContainer(Network network, String name) {
+ super(STRIMZI_CONTAINER);
+
+ withEnv("LOG_DIR", "/tmp/logs");
+ withExposedPorts(KAFKA_PORT);
Review comment:
These were my conclusions as well, but it wasn't clear to me looking at the 2 docker files which difference resulted in the different behavior.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [camel-kafka-connector] orpiske commented on a change in pull
request #66: Added support for testing the Kafka service using Strimzi
Posted by GitBox <gi...@apache.org>.
orpiske commented on a change in pull request #66: Added support for testing the Kafka service using Strimzi
URL: https://github.com/apache/camel-kafka-connector/pull/66#discussion_r364952012
##########
File path: tests/src/test/java/org/apache/camel/kafkaconnector/services/kafka/StrimziContainer.java
##########
@@ -0,0 +1,68 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.camel.kafkaconnector.services.kafka;
+
+import java.util.function.Consumer;
+import com.github.dockerjava.api.command.CreateContainerCmd;
+import org.testcontainers.containers.GenericContainer;
+import org.testcontainers.containers.Network;
+
+public class StrimziContainer extends GenericContainer {
+ private static final String STRIMZI_CONTAINER = "strimzi/kafka:0.11.4-kafka-2.1.0";
+ private static final int KAFKA_PORT = 9092;
+
+ public StrimziContainer(Network network, String name) {
+ super(STRIMZI_CONTAINER);
+
+ withEnv("LOG_DIR", "/tmp/logs");
+ withEnv("KAFKA_ADVERTISED_LISTENERS", "PLAINTEXT://localhost:9092");
+ withEnv("KAFKA_LISTENERS", "PLAINTEXT://0.0.0.0:9092");
+ withEnv("KAFKA_ZOOKEEPER_CONNECT", "zookeeper:2181");
+ withNetwork(network);
+
+ withCreateContainerCmdModifier(
+ new Consumer<CreateContainerCmd>() {
+ @Override
+ public void accept(CreateContainerCmd createContainerCmd) {
+ createContainerCmd.withHostName(name);
+ createContainerCmd.withName(name);
+ }
+ }
+ );
+
+
+ withCommand("sh", "-c",
+ "bin/kafka-server-start.sh config/server.properties "
+ + "--override listeners=${KAFKA_LISTENERS} "
+ + "--override advertised.listeners=${KAFKA_ADVERTISED_LISTENERS} "
+ + "--override zookeeper.connect=${KAFKA_ZOOKEEPER_CONNECT}");
+ }
Review comment:
I think we'll need a waitingFor(Wait.forListeningPort()) here, so that it waits for the container to go up.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [camel-kafka-connector] valdar commented on issue #66: Added
support for testing the Kafka service using Strimzi
Posted by GitBox <gi...@apache.org>.
valdar commented on issue #66: Added support for testing the Kafka service using Strimzi
URL: https://github.com/apache/camel-kafka-connector/pull/66#issuecomment-572817907
@orpiske I fixed as you suggested and made possible to configure the image at test-run time. I also bumped the used image version to `strimzi/kafka:0.15.0-kafka-2.3.1`. Can you please review it?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [camel-kafka-connector] valdar commented on a change in pull
request #66: Added support for testing the Kafka service using Strimzi
Posted by GitBox <gi...@apache.org>.
valdar commented on a change in pull request #66: Added support for testing the Kafka service using Strimzi
URL: https://github.com/apache/camel-kafka-connector/pull/66#discussion_r365022954
##########
File path: tests/src/test/java/org/apache/camel/kafkaconnector/services/kafka/StrimziContainer.java
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.camel.kafkaconnector.services.kafka;
+
+import java.util.function.Consumer;
+import com.github.dockerjava.api.command.CreateContainerCmd;
+import org.testcontainers.containers.GenericContainer;
+import org.testcontainers.containers.Network;
+import org.testcontainers.containers.wait.strategy.Wait;
+
+public class StrimziContainer extends GenericContainer {
+ private static final String STRIMZI_CONTAINER = System.getProperty("itest.strimzi.container.image");
+ private static final int KAFKA_PORT = 9092;
+
+ public StrimziContainer(Network network, String name) {
+ super(STRIMZI_CONTAINER);
+
+ withEnv("LOG_DIR", "/tmp/logs");
+ withExposedPorts(KAFKA_PORT);
Review comment:
@orpiske I have no idea how it was possible that it was working without this line (i.e. `withExposedPorts(KAFKA_PORT);`) with `strimzi/kafka:0.11.4-kafka-2.1.0` image?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [camel-kafka-connector] orpiske commented on a change in pull
request #66: Added support for testing the Kafka service using Strimzi
Posted by GitBox <gi...@apache.org>.
orpiske commented on a change in pull request #66: Added support for testing the Kafka service using Strimzi
URL: https://github.com/apache/camel-kafka-connector/pull/66#discussion_r364952306
##########
File path: tests/src/test/java/org/apache/camel/kafkaconnector/services/kafka/ZookeeperContainer.java
##########
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.camel.kafkaconnector.services.kafka;
+
+import java.util.function.Consumer;
+import com.github.dockerjava.api.command.CreateContainerCmd;
+import org.testcontainers.containers.GenericContainer;
+import org.testcontainers.containers.Network;
+import org.testcontainers.containers.wait.strategy.Wait;
+
+public class ZookeeperContainer extends GenericContainer {
+ private static final String ZOOKEEPER_CONTAINER = "strimzi/kafka:0.11.4-kafka-2.1.0";
+ private static final int ZOOKEEPER_PORT = 2181;
+
+ public ZookeeperContainer(Network network, String name) {
+ super(ZOOKEEPER_CONTAINER);
+
+ withEnv("LOG_DIR", "/tmp/logs");
+ withExposedPorts(ZOOKEEPER_PORT);
+ withNetwork(network);
+ withCreateContainerCmdModifier(
+ new Consumer<CreateContainerCmd>() {
+ @Override
+ public void accept(CreateContainerCmd createContainerCmd) {
+ createContainerCmd.withHostName(name);
+ createContainerCmd.withName(name);
+ }
+ }
+ );
+
+ withCommand( "sh", "-c",
+ "bin/zookeeper-server-start.sh config/zookeeper.properties");
+ }
Review comment:
I think we'll need a waitingFor(Wait.forListeningPort()) here, so that it waits for the container to go up.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [camel-kafka-connector] valdar merged pull request #66: Added
support for testing the Kafka service using Strimzi
Posted by GitBox <gi...@apache.org>.
valdar merged pull request #66: Added support for testing the Kafka service using Strimzi
URL: https://github.com/apache/camel-kafka-connector/pull/66
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [camel-kafka-connector] orpiske commented on a change in pull
request #66: Added support for testing the Kafka service using Strimzi
Posted by GitBox <gi...@apache.org>.
orpiske commented on a change in pull request #66: Added support for testing the Kafka service using Strimzi
URL: https://github.com/apache/camel-kafka-connector/pull/66#discussion_r364951432
##########
File path: tests/src/test/java/org/apache/camel/kafkaconnector/services/kafka/StrimziService.java
##########
@@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.camel.kafkaconnector.services.kafka;
+
+import org.apache.camel.kafkaconnector.ContainerUtil;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.testcontainers.containers.Network;
+
+public class StrimziService implements KafkaService {
+ private static final Logger LOG = LoggerFactory.getLogger(StrimziService.class);
+ private final ZookeeperContainer zookeeperContainer;
+ private final StrimziContainer strimziContainer;
+
+ public StrimziService() {
+ Network network = Network.newNetwork();
+
+ zookeeperContainer = new ZookeeperContainer(network, "zookeeper");
+ strimziContainer = new StrimziContainer(network, "strimzi");
+ }
+
+ private Integer getKafkaPort() {
+ return strimziContainer.getKafkaPort();
+ }
+
+ @Override
+ public String getBootstrapServers() {
+ return "localhost:" + getKafkaPort();
+ }
+
+ @Override
+ public void initialize() {
+ zookeeperContainer.start();
+ ContainerUtil.waitForInitialization(zookeeperContainer);
+
+ String zookeeperConnect = "zookeeper:" + zookeeperContainer.getZookeeperPort();
+ LOG.info("Apache Zookeeper running at address {}", zookeeperConnect);
+
+ strimziContainer.start();
+ ContainerUtil.waitForInitialization(strimziContainer);
Review comment:
Same thing here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [camel-kafka-connector] orpiske commented on a change in pull
request #66: Added support for testing the Kafka service using Strimzi
Posted by GitBox <gi...@apache.org>.
orpiske commented on a change in pull request #66: Added support for testing the Kafka service using Strimzi
URL: https://github.com/apache/camel-kafka-connector/pull/66#discussion_r365573713
##########
File path: tests/src/test/java/org/apache/camel/kafkaconnector/services/kafka/StrimziContainer.java
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.camel.kafkaconnector.services.kafka;
+
+import java.util.function.Consumer;
+import com.github.dockerjava.api.command.CreateContainerCmd;
+import org.testcontainers.containers.GenericContainer;
+import org.testcontainers.containers.Network;
+import org.testcontainers.containers.wait.strategy.Wait;
+
+public class StrimziContainer extends GenericContainer {
+ private static final String STRIMZI_CONTAINER = System.getProperty("itest.strimzi.container.image");
+ private static final int KAFKA_PORT = 9092;
+
+ public StrimziContainer(Network network, String name) {
+ super(STRIMZI_CONTAINER);
+
+ withEnv("LOG_DIR", "/tmp/logs");
+ withExposedPorts(KAFKA_PORT);
Review comment:
TBH, I think there's more to it than this.
The tests work fine if we remove the withExposedPorts(KAFKA_PORT) and run the tests using ```strimzi/kafka:0.11.4-kafka-2.1.0``` image
```mvn -Ditest.zookeeper.container.image=strimzi/kafka:0.11.4-kafka-2.1.0 -Ditest.strimzi.container.image=strimzi/kafka:0.11.4-kafka-2.1.0 -DskipIntegrationTests=false clean verify package```
However, the same thing using ```strimzi/kafka:0.15.0-kafka-2.3.1``` leads to a failure:
```Caused by: org.testcontainers.containers.ContainerLaunchException: Container startup failed```
Looking at the docker ps output for both, it seems it does not expose the port for the newer image:
```
48ffc42dfae8 strimzi/kafka:0.11.4-kafka-2.1.0 "sh -c 'bin/kafka-se…" About a minute ago Up About a minute 0.0.0.0:9092->9092/tcp, 0.0.0.0:32979->9091/tcp, 0.0.0.0:32978->9404/tcp strimzi
```
```
e30afed7a6bc strimzi/kafka:0.15.0-kafka-2.3.1 "sh -c 'bin/kafka-se…" 55 seconds ago Up 54 seconds strimzi
```
Adding the ```withExposedPorts(KAFKA_PORT);``` makes it work for both the older and the newer strimzi images.
I am guessing that differences on Dockerfiles may also play a part on why it worked without that for the older image.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [camel-kafka-connector] orpiske commented on a change in pull
request #66: Added support for testing the Kafka service using Strimzi
Posted by GitBox <gi...@apache.org>.
orpiske commented on a change in pull request #66: Added support for testing the Kafka service using Strimzi
URL: https://github.com/apache/camel-kafka-connector/pull/66#discussion_r364951178
##########
File path: tests/src/test/java/org/apache/camel/kafkaconnector/services/kafka/StrimziService.java
##########
@@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.camel.kafkaconnector.services.kafka;
+
+import org.apache.camel.kafkaconnector.ContainerUtil;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.testcontainers.containers.Network;
+
+public class StrimziService implements KafkaService {
+ private static final Logger LOG = LoggerFactory.getLogger(StrimziService.class);
+ private final ZookeeperContainer zookeeperContainer;
+ private final StrimziContainer strimziContainer;
+
+ public StrimziService() {
+ Network network = Network.newNetwork();
+
+ zookeeperContainer = new ZookeeperContainer(network, "zookeeper");
+ strimziContainer = new StrimziContainer(network, "strimzi");
+ }
+
+ private Integer getKafkaPort() {
+ return strimziContainer.getKafkaPort();
+ }
+
+ @Override
+ public String getBootstrapServers() {
+ return "localhost:" + getKafkaPort();
+ }
+
+ @Override
+ public void initialize() {
+ zookeeperContainer.start();
+ ContainerUtil.waitForInitialization(zookeeperContainer);
Review comment:
I think we can remove this. It shouldn't be needed anymore.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [camel-kafka-connector] orpiske commented on a change in pull
request #66: Added support for testing the Kafka service using Strimzi
Posted by GitBox <gi...@apache.org>.
orpiske commented on a change in pull request #66: Added support for testing the Kafka service using Strimzi
URL: https://github.com/apache/camel-kafka-connector/pull/66#discussion_r365569972
##########
File path: tests/pom.xml
##########
@@ -178,7 +178,7 @@
<artifactId>maven-failsafe-plugin</artifactId>
<configuration>
<!-- This is needed for the AWS Kinesis tests -->
- <argLine>-Dcom.amazonaws.sdk.disableCbor=true</argLine>
+ <argLine>-Dcom.amazonaws.sdk.disableCbor=true -Ditest.strimzi.container.image=${itest.strimzi.container.image} -Ditest.zookeeper.container.image=${itest.zookeeper.container.image}</argLine>
Review comment:
Yes, I think this does the trick.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [camel-kafka-connector] valdar commented on a change in pull
request #66: Added support for testing the Kafka service using Strimzi
Posted by GitBox <gi...@apache.org>.
valdar commented on a change in pull request #66: Added support for testing the Kafka service using Strimzi
URL: https://github.com/apache/camel-kafka-connector/pull/66#discussion_r365022954
##########
File path: tests/src/test/java/org/apache/camel/kafkaconnector/services/kafka/StrimziContainer.java
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.camel.kafkaconnector.services.kafka;
+
+import java.util.function.Consumer;
+import com.github.dockerjava.api.command.CreateContainerCmd;
+import org.testcontainers.containers.GenericContainer;
+import org.testcontainers.containers.Network;
+import org.testcontainers.containers.wait.strategy.Wait;
+
+public class StrimziContainer extends GenericContainer {
+ private static final String STRIMZI_CONTAINER = System.getProperty("itest.strimzi.container.image");
+ private static final int KAFKA_PORT = 9092;
+
+ public StrimziContainer(Network network, String name) {
+ super(STRIMZI_CONTAINER);
+
+ withEnv("LOG_DIR", "/tmp/logs");
+ withExposedPorts(KAFKA_PORT);
Review comment:
I have no idea how it was possible that it was working without this line (i.e. `withExposedPorts(KAFKA_PORT);`) with `strimzi/kafka:0.11.4-kafka-2.1.0` image @orpiske ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [camel-kafka-connector] valdar commented on a change in pull
request #66: Added support for testing the Kafka service using Strimzi
Posted by GitBox <gi...@apache.org>.
valdar commented on a change in pull request #66: Added support for testing the Kafka service using Strimzi
URL: https://github.com/apache/camel-kafka-connector/pull/66#discussion_r365022954
##########
File path: tests/src/test/java/org/apache/camel/kafkaconnector/services/kafka/StrimziContainer.java
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.camel.kafkaconnector.services.kafka;
+
+import java.util.function.Consumer;
+import com.github.dockerjava.api.command.CreateContainerCmd;
+import org.testcontainers.containers.GenericContainer;
+import org.testcontainers.containers.Network;
+import org.testcontainers.containers.wait.strategy.Wait;
+
+public class StrimziContainer extends GenericContainer {
+ private static final String STRIMZI_CONTAINER = System.getProperty("itest.strimzi.container.image");
+ private static final int KAFKA_PORT = 9092;
+
+ public StrimziContainer(Network network, String name) {
+ super(STRIMZI_CONTAINER);
+
+ withEnv("LOG_DIR", "/tmp/logs");
+ withExposedPorts(KAFKA_PORT);
Review comment:
I have no idea how it was possible that it was warking without this line with `strimzi/kafka:0.11.4-kafka-2.1.0` image @orpiske ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [camel-kafka-connector] valdar commented on a change in pull
request #66: Added support for testing the Kafka service using Strimzi
Posted by GitBox <gi...@apache.org>.
valdar commented on a change in pull request #66: Added support for testing the Kafka service using Strimzi
URL: https://github.com/apache/camel-kafka-connector/pull/66#discussion_r365022954
##########
File path: tests/src/test/java/org/apache/camel/kafkaconnector/services/kafka/StrimziContainer.java
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.camel.kafkaconnector.services.kafka;
+
+import java.util.function.Consumer;
+import com.github.dockerjava.api.command.CreateContainerCmd;
+import org.testcontainers.containers.GenericContainer;
+import org.testcontainers.containers.Network;
+import org.testcontainers.containers.wait.strategy.Wait;
+
+public class StrimziContainer extends GenericContainer {
+ private static final String STRIMZI_CONTAINER = System.getProperty("itest.strimzi.container.image");
+ private static final int KAFKA_PORT = 9092;
+
+ public StrimziContainer(Network network, String name) {
+ super(STRIMZI_CONTAINER);
+
+ withEnv("LOG_DIR", "/tmp/logs");
+ withExposedPorts(KAFKA_PORT);
Review comment:
I have no idea how it was possible that it was warking without this line (i.e. `withExposedPorts(KAFKA_PORT);`) with `strimzi/kafka:0.11.4-kafka-2.1.0` image @orpiske ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [camel-kafka-connector] orpiske commented on a change in pull
request #66: Added support for testing the Kafka service using Strimzi
Posted by GitBox <gi...@apache.org>.
orpiske commented on a change in pull request #66: Added support for testing the Kafka service using Strimzi
URL: https://github.com/apache/camel-kafka-connector/pull/66#discussion_r365570628
##########
File path: tests/src/test/java/org/apache/camel/kafkaconnector/services/kafka/StrimziContainer.java
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.camel.kafkaconnector.services.kafka;
+
+import java.util.function.Consumer;
+import com.github.dockerjava.api.command.CreateContainerCmd;
+import org.testcontainers.containers.GenericContainer;
+import org.testcontainers.containers.Network;
+import org.testcontainers.containers.wait.strategy.Wait;
+
+public class StrimziContainer extends GenericContainer {
+ private static final String STRIMZI_CONTAINER = System.getProperty("itest.strimzi.container.image");
+ private static final int KAFKA_PORT = 9092;
+
+ public StrimziContainer(Network network, String name) {
+ super(STRIMZI_CONTAINER);
+
+ withEnv("LOG_DIR", "/tmp/logs");
+ withExposedPorts(KAFKA_PORT);
Review comment:
It works because of the call to [addFixedExposedPort(KAFKA_PORT, KAFKA_PORT)](https://github.com/apache/camel-kafka-connector/pull/66/commits/75e8953c0f3d224d1c1e9309bd3dc7ebd5d24bfc#diff-98b65a9f0c00948336e48a4237b379e5R65) on the start() method.
Maybe we should add a comment here, but the explanation is that this makes the port mapping from the localhost to inside the container to use the same standard Kafka port (i.e: like running docker -p 9092:9092). The usage of withExposedPort will configure it to use a dynamic port which may require further configuration on the Kafka client.
The addFixedExposedPort is the default behaviour for the Kafka container, which is why I had originally used it here as well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services