You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by gu...@apache.org on 2018/12/06 01:14:53 UTC
spark git commit: [SPARK-26275][PYTHON][ML] Increases timeout for
StreamingLogisticRegressionWithSGDTests.test_training_and_prediction test
Repository: spark
Updated Branches:
refs/heads/master dd518a196 -> ab76900fe
[SPARK-26275][PYTHON][ML] Increases timeout for StreamingLogisticRegressionWithSGDTests.test_training_and_prediction test
## What changes were proposed in this pull request?
Looks this test is flaky
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99704/console
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99569/console
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99644/console
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99548/console
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99454/console
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99609/console
```
======================================================================
FAIL: test_training_and_prediction (pyspark.mllib.tests.test_streaming_algorithms.StreamingLogisticRegressionWithSGDTests)
Test that the model improves on toy data with no. of batches
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/jenkins/workspace/SparkPullRequestBuilder/python/pyspark/mllib/tests/test_streaming_algorithms.py", line 367, in test_training_and_prediction
self._eventually(condition)
File "/home/jenkins/workspace/SparkPullRequestBuilder/python/pyspark/mllib/tests/test_streaming_algorithms.py", line 78, in _eventually
% (timeout, lastValue))
AssertionError: Test failed due to timeout after 30 sec, with last condition returning: Latest errors: 0.67, 0.71, 0.78, 0.7, 0.75, 0.74, 0.73, 0.69, 0.62, 0.71, 0.69, 0.75, 0.72, 0.77, 0.71, 0.74
----------------------------------------------------------------------
Ran 13 tests in 185.051s
FAILED (failures=1, skipped=1)
```
This looks happening after increasing the parallelism in Jenkins to speed up at https://github.com/apache/spark/pull/23111. I am able to reproduce this manually when the resource usage is heavy (with manual decrease of timeout).
## How was this patch tested?
Manually tested by
```
cd python
./run-tests --testnames 'pyspark.mllib.tests.test_streaming_algorithms StreamingLogisticRegressionWithSGDTests.test_training_and_prediction' --python-executables=python
```
Closes #23236 from HyukjinKwon/SPARK-26275.
Authored-by: Hyukjin Kwon <gu...@apache.org>
Signed-off-by: Hyukjin Kwon <gu...@apache.org>
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/ab76900f
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/ab76900f
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/ab76900f
Branch: refs/heads/master
Commit: ab76900fedc05df7080c9b6c81d65a3f260c1c26
Parents: dd518a1
Author: Hyukjin Kwon <gu...@apache.org>
Authored: Thu Dec 6 09:14:46 2018 +0800
Committer: Hyukjin Kwon <gu...@apache.org>
Committed: Thu Dec 6 09:14:46 2018 +0800
----------------------------------------------------------------------
python/pyspark/mllib/tests/test_streaming_algorithms.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/ab76900f/python/pyspark/mllib/tests/test_streaming_algorithms.py
----------------------------------------------------------------------
diff --git a/python/pyspark/mllib/tests/test_streaming_algorithms.py b/python/pyspark/mllib/tests/test_streaming_algorithms.py
index 4bc8904..bf2ad2d 100644
--- a/python/pyspark/mllib/tests/test_streaming_algorithms.py
+++ b/python/pyspark/mllib/tests/test_streaming_algorithms.py
@@ -364,7 +364,7 @@ class StreamingLogisticRegressionWithSGDTests(MLLibStreamingTestCase):
return True
return "Latest errors: " + ", ".join(map(lambda x: str(x), errors))
- self._eventually(condition)
+ self._eventually(condition, timeout=60.0)
class StreamingLinearRegressionWithTests(MLLibStreamingTestCase):
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org