You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kafka.apache.org by vv...@apache.org on 2020/06/01 20:32:39 UTC
[kafka] branch 1.1 updated (f1a5730 -> 9b233fc)
This is an automated email from the ASF dual-hosted git repository.
vvcephei pushed a change to branch 1.1
in repository https://gitbox.apache.org/repos/asf/kafka.git.
from f1a5730 MINOR: Upgrade ducktape to 0.7.7 (#8487)
new 8e52ae4 MINOR: Fix Streams EOS system tests by adding clean-up of state dir (#7693)
new e07098c KAFKA-8417: Remove redundant network definition --net=host when starting testing docker containers (#6797)
new 9b233fc KAFKA-8499: ensure java is in PATH for ducker system tests (#6898)
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
tests/docker/Dockerfile | 3 ++-
tests/docker/ducker-ak | 2 +-
tests/kafkatest/tests/streams/streams_eos_test.py | 8 ++++----
3 files changed, 7 insertions(+), 6 deletions(-)
[kafka] 03/03: KAFKA-8499: ensure java is in PATH for ducker system
tests (#6898)
Posted by vv...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
vvcephei pushed a commit to branch 1.1
in repository https://gitbox.apache.org/repos/asf/kafka.git
commit 9b233fc63f5648ae49ce48058949cd82f7520e9e
Author: Lucas Bradstreet <lu...@confluent.io>
AuthorDate: Fri Jun 7 14:23:49 2019 -0700
KAFKA-8499: ensure java is in PATH for ducker system tests (#6898)
Reviewers: Colin P. McCabe <cm...@apache.org>
---
tests/docker/Dockerfile | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/tests/docker/Dockerfile b/tests/docker/Dockerfile
index 3d3ae6c..c243465 100644
--- a/tests/docker/Dockerfile
+++ b/tests/docker/Dockerfile
@@ -37,6 +37,7 @@ RUN pip install -U pip==9.0.3 setuptools && pip install --upgrade cffi virtualen
# Set up ssh
COPY ./ssh-config /root/.ssh/config
RUN ssh-keygen -q -t rsa -N '' -f /root/.ssh/id_rsa && cp -f /root/.ssh/id_rsa.pub /root/.ssh/authorized_keys
+RUN echo 'PermitUserEnvironment yes' >> /etc/ssh/sshd_config
# Install binary test dependencies.
# we use the same versions as in vagrant/base.sh
@@ -59,7 +60,7 @@ RUN apt-get install fuse
RUN cd /opt && git clone -q https://github.com/confluentinc/kibosh.git && cd "/opt/kibosh" && git reset --hard $KIBOSH_VERSION && mkdir "/opt/kibosh/build" && cd "/opt/kibosh/build" && ../configure && make -j 2
# Set up the ducker user.
-RUN useradd -ms /bin/bash ducker && mkdir -p /home/ducker/ && rsync -aiq /root/.ssh/ /home/ducker/.ssh && chown -R ducker /home/ducker/ /mnt/ && echo 'ducker ALL=(ALL) NOPASSWD: ALL' >> /etc/sudoers
+RUN useradd -ms /bin/bash ducker && mkdir -p /home/ducker/ && rsync -aiq /root/.ssh/ /home/ducker/.ssh && chown -R ducker /home/ducker/ /mnt/ && echo "PATH=$(runuser -l ducker -c 'echo $PATH'):$JAVA_HOME/bin" >> /home/ducker/.ssh/environment && echo 'PATH=$PATH:'"$JAVA_HOME/bin" >> /home/ducker/.profile && echo 'ducker ALL=(ALL) NOPASSWD: ALL' >> /etc/sudoers
USER ducker
CMD sudo service ssh start && tail -f /dev/null
[kafka] 02/03: KAFKA-8417: Remove redundant network definition
--net=host when starting testing docker containers (#6797)
Posted by vv...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
vvcephei pushed a commit to branch 1.1
in repository https://gitbox.apache.org/repos/asf/kafka.git
commit e07098cd119d30066cb2d6c987e643fabcb1660b
Author: Konstantine Karantasis <ko...@confluent.io>
AuthorDate: Thu May 23 11:46:10 2019 -0700
KAFKA-8417: Remove redundant network definition --net=host when starting testing docker containers (#6797)
Reviewers: Colin P. McCabe <cm...@apache.org>
---
tests/docker/ducker-ak | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/docker/ducker-ak b/tests/docker/ducker-ak
index f7eae49..1b46f1b 100755
--- a/tests/docker/ducker-ak
+++ b/tests/docker/ducker-ak
@@ -236,7 +236,7 @@ docker_run() {
# and mount FUSE filesystems inside the container. We also need it to
# run iptables inside the container.
must_do -v docker run --privileged \
- -d -t --net=host -h "${node}" --network ducknet \
+ -d -t -h "${node}" --network ducknet \
--memory=${docker_run_memory_limit} --memory-swappiness=1 \
-v "${kafka_dir}:/opt/kafka-dev" --name "${node}" -- "${image_name}"
}
[kafka] 01/03: MINOR: Fix Streams EOS system tests by adding
clean-up of state dir (#7693)
Posted by vv...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
vvcephei pushed a commit to branch 1.1
in repository https://gitbox.apache.org/repos/asf/kafka.git
commit 8e52ae4a428f7781c3f262311c2fab5715136ddf
Author: Bruno Cadonna <br...@confluent.io>
AuthorDate: Thu Nov 21 16:32:31 2019 +0100
MINOR: Fix Streams EOS system tests by adding clean-up of state dir (#7693)
Recently, system tests test_rebalance_[simple|complex] failed
repeatedly with a verfication error. The cause was most probably
the missing clean-up of a state directory of one of the processors.
A node is cleaned up when a service on that node is started and when
a test is torn down.
If the clean-up flag clean_node_enabled of a EOS Streams service is
unset, the clean-up of the node is skipped.
The clean-up flag of processor1 in the EOS tests should stay set before
its first start, so that the node is cleaned before the service is started.
Afterwards for the multiple restarts of processor1 the cleans-up flag should
be unset to re-use the local state.
After the multiple restarts are done, the clean-up flag of processor1 should
again be set to trigger node clean-up during the test teardown.
A dirty node can lead to test failures when tests from Streams EOS tests are
scheduled on the same node, because the state store would not start empty
since it reads the local state that was not cleaned up.
Reviewers: Matthias J. Sax <mj...@apache.org>, Andrew Choi <an...@linkedin.com>, Bill Bejeck <bb...@gmail.com>
---
tests/kafkatest/tests/streams/streams_eos_test.py | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/tests/kafkatest/tests/streams/streams_eos_test.py b/tests/kafkatest/tests/streams/streams_eos_test.py
index 7e5cc26..c56489d 100644
--- a/tests/kafkatest/tests/streams/streams_eos_test.py
+++ b/tests/kafkatest/tests/streams/streams_eos_test.py
@@ -60,9 +60,8 @@ class StreamsEosTest(KafkaTest):
self.driver.start()
- processor1.clean_node_enabled = False
-
self.add_streams(processor1)
+ processor1.clean_node_enabled = False
self.add_streams2(processor1, processor2)
self.add_streams3(processor1, processor2, processor3)
self.stop_streams3(processor2, processor3, processor1)
@@ -70,6 +69,7 @@ class StreamsEosTest(KafkaTest):
self.stop_streams3(processor1, processor3, processor2)
self.stop_streams2(processor1, processor3)
self.stop_streams(processor1)
+ processor1.clean_node_enabled = True
self.driver.stop()
@@ -100,9 +100,8 @@ class StreamsEosTest(KafkaTest):
self.driver.start()
- processor1.clean_node_enabled = False
-
self.add_streams(processor1)
+ processor1.clean_node_enabled = False
self.add_streams2(processor1, processor2)
self.add_streams3(processor1, processor2, processor3)
self.abort_streams(processor2, processor3, processor1)
@@ -112,6 +111,7 @@ class StreamsEosTest(KafkaTest):
self.abort_streams(processor1, processor3, processor2)
self.stop_streams2(processor1, processor3)
self.stop_streams(processor1)
+ processor1.clean_node_enabled = True
self.driver.stop()