You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by gu...@apache.org on 2019/08/03 01:36:36 UTC
[spark] branch branch-2.3 updated: [SPARK-28582][PYTHON] Fix flaky
test DaemonTests.do_termination_test which fail on Python 3.7
This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch branch-2.3
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-2.3 by this push:
new e686178 [SPARK-28582][PYTHON] Fix flaky test DaemonTests.do_termination_test which fail on Python 3.7
e686178 is described below
commit e686178ffe7ceb33ee42558ee0b4fe28c417124d
Author: WeichenXu <we...@databricks.com>
AuthorDate: Sat Aug 3 10:31:15 2019 +0900
[SPARK-28582][PYTHON] Fix flaky test DaemonTests.do_termination_test which fail on Python 3.7
This PR picks up https://github.com/apache/spark/pull/25315 back after removing `Popen.wait` usage which exists in Python 3 only. I saw the last test results wrongly and thought it was passed.
Fix flaky test DaemonTests.do_termination_test which fail on Python 3.7. I add a sleep after the test connection to daemon.
Run test
```
python/run-tests --python-executables=python3.7 --testname "pyspark.tests.test_daemon DaemonTests"
```
**Before**
Fail on test "test_termination_sigterm". And we can see daemon process do not exit.
**After**
Test passed
Closes #25343 from HyukjinKwon/SPARK-28582.
Authored-by: WeichenXu <we...@databricks.com>
Signed-off-by: HyukjinKwon <gu...@apache.org>
(cherry picked from commit b3394db1930b3c9f55438cb27bb2c584bf041f8e)
Signed-off-by: HyukjinKwon <gu...@apache.org>
---
python/pyspark/tests.py | 3 +++
1 file changed, 3 insertions(+)
diff --git a/python/pyspark/tests.py b/python/pyspark/tests.py
index 5ba121f..e1c780f 100644
--- a/python/pyspark/tests.py
+++ b/python/pyspark/tests.py
@@ -1862,6 +1862,9 @@ class DaemonTests(unittest.TestCase):
# daemon should accept connections
self.assertTrue(self.connect(port))
+ # wait worker process spawned from daemon exit.
+ time.sleep(1)
+
# request shutdown
terminator(daemon)
time.sleep(1)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org