You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Cheolsoo Park <pi...@gmail.com> on 2015/07/12 22:33:42 UTC

pyspark.sql.tests: is test_time_with_timezone a flaky test?

Hi devs,

For some reason, I keep getting this test failure (3 out of 4 builds) in my
PR <https://github.com/apache/spark/pull/7216>-

======================================================================
FAIL: test_time_with_timezone (__main__.SQLTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File
"/home/jenkins/workspace/SparkPullRequestBuilder/python/pyspark/sql/tests.py",
line 718, in test_time_with_timezone
    *self.assertEqual(now, now1)*
AssertionError: datetime.datetime(2015, 7, 12, 13, 18, 46, *504366*) !=
datetime.datetime(2015, 7, 12, 13, 18, 46, *504365*)

Jenkins builds-
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37100/console
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37092/console
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37081/console

I am aware that there was a hot fix for this test case, and I already have
it in the commit log-

commit 05ac023dc8d9004a27c2f06ee875b0ff3743ccdd

Author: Davies Liu <da...@databricks.com>
Date:   Fri Jul 10 13:05:23 2015 -0700
    [HOTFIX] fix flaky test in PySpark SQL

I looked at the test code, and it seems that precision in microseconds is
lost somewhere in a round trip from Python to DataFrame. Can someone please
help me debug this error?

Thanks!
Cheolsoo

Re: pyspark.sql.tests: is test_time_with_timezone a flaky test?

Posted by Cheolsoo Park <pi...@gmail.com>.
Thank you!

On Sun, Jul 12, 2015 at 10:59 PM, Davies Liu <da...@databricks.com> wrote:

> Will be fixed by https://github.com/apache/spark/pull/7363
>
> On Sun, Jul 12, 2015 at 7:45 PM, Davies Liu <da...@databricks.com> wrote:
> > Thanks for reporting this, I'm working on it. It turned out that it's
> > a bug in when run with Python3.4, will sending out a fix soon.
> >
> > On Sun, Jul 12, 2015 at 1:33 PM, Cheolsoo Park <pi...@gmail.com>
> wrote:
> >> Hi devs,
> >>
> >> For some reason, I keep getting this test failure (3 out of 4 builds)
> in my
> >> PR-
> >>
> >> ======================================================================
> >> FAIL: test_time_with_timezone (__main__.SQLTests)
> >> ----------------------------------------------------------------------
> >> Traceback (most recent call last):
> >>   File
> >>
> "/home/jenkins/workspace/SparkPullRequestBuilder/python/pyspark/sql/tests.py",
> >> line 718, in test_time_with_timezone
> >>     self.assertEqual(now, now1)
> >> AssertionError: datetime.datetime(2015, 7, 12, 13, 18, 46, 504366) !=
> >> datetime.datetime(2015, 7, 12, 13, 18, 46, 504365)
> >>
> >> Jenkins builds-
> >>
> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37100/console
> >>
> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37092/console
> >>
> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37081/console
> >>
> >> I am aware that there was a hot fix for this test case, and I already
> have
> >> it in the commit log-
> >>
> >> commit 05ac023dc8d9004a27c2f06ee875b0ff3743ccdd
> >>
> >> Author: Davies Liu <da...@databricks.com>
> >> Date:   Fri Jul 10 13:05:23 2015 -0700
> >>     [HOTFIX] fix flaky test in PySpark SQL
> >>
> >> I looked at the test code, and it seems that precision in microseconds
> is
> >> lost somewhere in a round trip from Python to DataFrame. Can someone
> please
> >> help me debug this error?
> >>
> >> Thanks!
> >> Cheolsoo
> >>
> >>
>

Re: pyspark.sql.tests: is test_time_with_timezone a flaky test?

Posted by Davies Liu <da...@databricks.com>.
Will be fixed by https://github.com/apache/spark/pull/7363

On Sun, Jul 12, 2015 at 7:45 PM, Davies Liu <da...@databricks.com> wrote:
> Thanks for reporting this, I'm working on it. It turned out that it's
> a bug in when run with Python3.4, will sending out a fix soon.
>
> On Sun, Jul 12, 2015 at 1:33 PM, Cheolsoo Park <pi...@gmail.com> wrote:
>> Hi devs,
>>
>> For some reason, I keep getting this test failure (3 out of 4 builds) in my
>> PR-
>>
>> ======================================================================
>> FAIL: test_time_with_timezone (__main__.SQLTests)
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>>   File
>> "/home/jenkins/workspace/SparkPullRequestBuilder/python/pyspark/sql/tests.py",
>> line 718, in test_time_with_timezone
>>     self.assertEqual(now, now1)
>> AssertionError: datetime.datetime(2015, 7, 12, 13, 18, 46, 504366) !=
>> datetime.datetime(2015, 7, 12, 13, 18, 46, 504365)
>>
>> Jenkins builds-
>> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37100/console
>> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37092/console
>> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37081/console
>>
>> I am aware that there was a hot fix for this test case, and I already have
>> it in the commit log-
>>
>> commit 05ac023dc8d9004a27c2f06ee875b0ff3743ccdd
>>
>> Author: Davies Liu <da...@databricks.com>
>> Date:   Fri Jul 10 13:05:23 2015 -0700
>>     [HOTFIX] fix flaky test in PySpark SQL
>>
>> I looked at the test code, and it seems that precision in microseconds is
>> lost somewhere in a round trip from Python to DataFrame. Can someone please
>> help me debug this error?
>>
>> Thanks!
>> Cheolsoo
>>
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org


Re: pyspark.sql.tests: is test_time_with_timezone a flaky test?

Posted by Davies Liu <da...@databricks.com>.
Thanks for reporting this, I'm working on it. It turned out that it's
a bug in when run with Python3.4, will sending out a fix soon.

On Sun, Jul 12, 2015 at 1:33 PM, Cheolsoo Park <pi...@gmail.com> wrote:
> Hi devs,
>
> For some reason, I keep getting this test failure (3 out of 4 builds) in my
> PR-
>
> ======================================================================
> FAIL: test_time_with_timezone (__main__.SQLTests)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File
> "/home/jenkins/workspace/SparkPullRequestBuilder/python/pyspark/sql/tests.py",
> line 718, in test_time_with_timezone
>     self.assertEqual(now, now1)
> AssertionError: datetime.datetime(2015, 7, 12, 13, 18, 46, 504366) !=
> datetime.datetime(2015, 7, 12, 13, 18, 46, 504365)
>
> Jenkins builds-
> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37100/console
> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37092/console
> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37081/console
>
> I am aware that there was a hot fix for this test case, and I already have
> it in the commit log-
>
> commit 05ac023dc8d9004a27c2f06ee875b0ff3743ccdd
>
> Author: Davies Liu <da...@databricks.com>
> Date:   Fri Jul 10 13:05:23 2015 -0700
>     [HOTFIX] fix flaky test in PySpark SQL
>
> I looked at the test code, and it seems that precision in microseconds is
> lost somewhere in a round trip from Python to DataFrame. Can someone please
> help me debug this error?
>
> Thanks!
> Cheolsoo
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org