You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kafka.apache.org by bb...@apache.org on 2019/11/21 19:21:37 UTC
[kafka] branch 2.4 updated: MINOR: Fix Streams EOS system tests by
adding clean-up of state dir (#7693)
This is an automated email from the ASF dual-hosted git repository.
bbejeck pushed a commit to branch 2.4
in repository https://gitbox.apache.org/repos/asf/kafka.git
The following commit(s) were added to refs/heads/2.4 by this push:
new 0bc40c6 MINOR: Fix Streams EOS system tests by adding clean-up of state dir (#7693)
0bc40c6 is described below
commit 0bc40c63e1b5f8c6385e1202407ef026644a97a8
Author: Bruno Cadonna <br...@confluent.io>
AuthorDate: Thu Nov 21 16:32:31 2019 +0100
MINOR: Fix Streams EOS system tests by adding clean-up of state dir (#7693)
Recently, system tests test_rebalance_[simple|complex] failed
repeatedly with a verfication error. The cause was most probably
the missing clean-up of a state directory of one of the processors.
A node is cleaned up when a service on that node is started and when
a test is torn down.
If the clean-up flag clean_node_enabled of a EOS Streams service is
unset, the clean-up of the node is skipped.
The clean-up flag of processor1 in the EOS tests should stay set before
its first start, so that the node is cleaned before the service is started.
Afterwards for the multiple restarts of processor1 the cleans-up flag should
be unset to re-use the local state.
After the multiple restarts are done, the clean-up flag of processor1 should
again be set to trigger node clean-up during the test teardown.
A dirty node can lead to test failures when tests from Streams EOS tests are
scheduled on the same node, because the state store would not start empty
since it reads the local state that was not cleaned up.
Reviewers: Matthias J. Sax <mj...@apache.org>, Andrew Choi <an...@linkedin.com>, Bill Bejeck <bb...@gmail.com>
---
tests/kafkatest/tests/streams/streams_eos_test.py | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/tests/kafkatest/tests/streams/streams_eos_test.py b/tests/kafkatest/tests/streams/streams_eos_test.py
index 428db9b..117b448 100644
--- a/tests/kafkatest/tests/streams/streams_eos_test.py
+++ b/tests/kafkatest/tests/streams/streams_eos_test.py
@@ -60,9 +60,8 @@ class StreamsEosTest(KafkaTest):
self.driver.start()
- processor1.clean_node_enabled = False
-
self.add_streams(processor1)
+ processor1.clean_node_enabled = False
self.add_streams2(processor1, processor2)
self.add_streams3(processor1, processor2, processor3)
self.stop_streams3(processor2, processor3, processor1)
@@ -70,6 +69,7 @@ class StreamsEosTest(KafkaTest):
self.stop_streams3(processor1, processor3, processor2)
self.stop_streams2(processor1, processor3)
self.stop_streams(processor1)
+ processor1.clean_node_enabled = True
self.driver.stop()
@@ -100,9 +100,8 @@ class StreamsEosTest(KafkaTest):
self.driver.start()
- processor1.clean_node_enabled = False
-
self.add_streams(processor1)
+ processor1.clean_node_enabled = False
self.add_streams2(processor1, processor2)
self.add_streams3(processor1, processor2, processor3)
self.abort_streams(processor2, processor3, processor1)
@@ -112,6 +111,7 @@ class StreamsEosTest(KafkaTest):
self.abort_streams(processor1, processor3, processor2)
self.stop_streams2(processor1, processor3)
self.stop_streams(processor1)
+ processor1.clean_node_enabled = True
self.driver.stop()