You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Dongjoon Hyun <do...@gmail.com> on 2020/02/01 01:17:51 UTC
Re: new branch-3.0 jenkins job configs are ready to be deployed...
Thank you, Shane.
BTW, we need to enable JDK11 unit run on Python and R. (Currently, it's
only tested in PRBuilder.)
https://issues.apache.org/jira/browse/SPARK-28900
Today, Thomas and I'm hitting Python UT failure on JDK11 environment in
independent PRs.
ERROR [32.750s]: test_parameter_accuracy
(pyspark.mllib.tests.test_streaming_algorithms.StreamingLogisticRegressionWithSGDTests)
----------------------------------------------------------------------
...
File "/home/jenkins/workspace/SparkPullRequestBuilder@2/python/pyspark/mllib/tests/test_streaming_algorithms.py",
line 226, in condition
self.assertAlmostEqual(rel, 0.1, 1)
AssertionError: 0.17619737864096185 != 0.1 within 1 places
Although I'm investigating now, we need Jenkins jobs as a 3rd party
validator from 3.0.0 QA period.
Bests,
Dongjoon
On Fri, Jan 31, 2020 at 11:26 AM Xiao Li <li...@databricks.com> wrote:
> Thank you always, Shane!
>
> Xiao
>
> On Fri, Jan 31, 2020 at 11:19 AM shane knapp ☠ <sk...@berkeley.edu>
> wrote:
>
>> ...whenever i get the word. :)
>>
>> FWIW they will all be identical to the current group of master
>> builds/tests.
>>
>> shane
>> --
>> Shane Knapp
>> Computer Guy / Voice of Reason
>> UC Berkeley EECS Research / RISELab Staff Technical Lead
>> https://rise.cs.berkeley.edu
>>
>> ---------------------------------------------------------------------
>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>
>>
>
> --
> <https://databricks.com/sparkaisummit/north-america>
>
Re: new branch-3.0 jenkins job configs are ready to be deployed...
Posted by Hyukjin Kwon <gu...@gmail.com>.
FWIW, I believe all tests are fixed in PySpark and SparkR with JDK 11. Let
me know if you guys meet some test failures.
2020년 2월 1일 (토) 오전 10:45, Dongjoon Hyun <do...@gmail.com>님이 작성:
> Oops. I found this flaky test fails event in `Hadoop 2.7 with Hive 1.2`.
>
>
> https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/spark-master-test-sbt-hadoop-2.7-hive-1.2/lastCompletedBuild/testReport/pyspark.mllib.tests.test_streaming_algorithms/StreamingLogisticRegressionWithSGDTests/test_parameter_accuracy/
>
> Anyway, I'll file a JIRA issue for this Python flakiness.
>
> Bests,
> Dongjoon.
>
>
> On Fri, Jan 31, 2020 at 5:17 PM Dongjoon Hyun <do...@gmail.com>
> wrote:
>
>> Thank you, Shane.
>>
>> BTW, we need to enable JDK11 unit run on Python and R. (Currently, it's
>> only tested in PRBuilder.)
>>
>> https://issues.apache.org/jira/browse/SPARK-28900
>>
>> Today, Thomas and I'm hitting Python UT failure on JDK11 environment in
>> independent PRs.
>>
>> ERROR [32.750s]: test_parameter_accuracy
>> (pyspark.mllib.tests.test_streaming_algorithms.StreamingLogisticRegressionWithSGDTests)
>> ----------------------------------------------------------------------
>> ...
>> File "/home/jenkins/workspace/SparkPullRequestBuilder@2/python/pyspark/mllib/tests/test_streaming_algorithms.py",
>> line 226, in condition
>> self.assertAlmostEqual(rel, 0.1, 1)
>> AssertionError: 0.17619737864096185 != 0.1 within 1 places
>>
>> Although I'm investigating now, we need Jenkins jobs as a 3rd party
>> validator from 3.0.0 QA period.
>>
>> Bests,
>> Dongjoon
>>
>>
>> On Fri, Jan 31, 2020 at 11:26 AM Xiao Li <li...@databricks.com> wrote:
>>
>>> Thank you always, Shane!
>>>
>>> Xiao
>>>
>>> On Fri, Jan 31, 2020 at 11:19 AM shane knapp ☠ <sk...@berkeley.edu>
>>> wrote:
>>>
>>>> ...whenever i get the word. :)
>>>>
>>>> FWIW they will all be identical to the current group of master
>>>> builds/tests.
>>>>
>>>> shane
>>>> --
>>>> Shane Knapp
>>>> Computer Guy / Voice of Reason
>>>> UC Berkeley EECS Research / RISELab Staff Technical Lead
>>>> https://rise.cs.berkeley.edu
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>>
>>>>
>>>
>>> --
>>> <https://databricks.com/sparkaisummit/north-america>
>>>
>>
Re: new branch-3.0 jenkins job configs are ready to be deployed...
Posted by Dongjoon Hyun <do...@gmail.com>.
Oops. I found this flaky test fails event in `Hadoop 2.7 with Hive 1.2`.
https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/spark-master-test-sbt-hadoop-2.7-hive-1.2/lastCompletedBuild/testReport/pyspark.mllib.tests.test_streaming_algorithms/StreamingLogisticRegressionWithSGDTests/test_parameter_accuracy/
Anyway, I'll file a JIRA issue for this Python flakiness.
Bests,
Dongjoon.
On Fri, Jan 31, 2020 at 5:17 PM Dongjoon Hyun <do...@gmail.com>
wrote:
> Thank you, Shane.
>
> BTW, we need to enable JDK11 unit run on Python and R. (Currently, it's
> only tested in PRBuilder.)
>
> https://issues.apache.org/jira/browse/SPARK-28900
>
> Today, Thomas and I'm hitting Python UT failure on JDK11 environment in
> independent PRs.
>
> ERROR [32.750s]: test_parameter_accuracy
> (pyspark.mllib.tests.test_streaming_algorithms.StreamingLogisticRegressionWithSGDTests)
> ----------------------------------------------------------------------
> ...
> File "/home/jenkins/workspace/SparkPullRequestBuilder@2/python/pyspark/mllib/tests/test_streaming_algorithms.py",
> line 226, in condition
> self.assertAlmostEqual(rel, 0.1, 1)
> AssertionError: 0.17619737864096185 != 0.1 within 1 places
>
> Although I'm investigating now, we need Jenkins jobs as a 3rd party
> validator from 3.0.0 QA period.
>
> Bests,
> Dongjoon
>
>
> On Fri, Jan 31, 2020 at 11:26 AM Xiao Li <li...@databricks.com> wrote:
>
>> Thank you always, Shane!
>>
>> Xiao
>>
>> On Fri, Jan 31, 2020 at 11:19 AM shane knapp ☠ <sk...@berkeley.edu>
>> wrote:
>>
>>> ...whenever i get the word. :)
>>>
>>> FWIW they will all be identical to the current group of master
>>> builds/tests.
>>>
>>> shane
>>> --
>>> Shane Knapp
>>> Computer Guy / Voice of Reason
>>> UC Berkeley EECS Research / RISELab Staff Technical Lead
>>> https://rise.cs.berkeley.edu
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>
>>>
>>
>> --
>> <https://databricks.com/sparkaisummit/north-america>
>>
>