[jira] [Created] (SPARK-25130) Wrong timestamp returned by toPandas

["Anton Daitche (JIRA)" <ji...@apache.org>]

Anton Daitche created SPARK-25130:
-------------------------------------

             Summary: Wrong timestamp returned by toPandas
                 Key: SPARK-25130
                 URL: https://issues.apache.org/jira/browse/SPARK-25130
             Project: Spark
          Issue Type: Bug
          Components: Input/Output
    Affects Versions: 2.3.1, 2.3.0
         Environment: Tested with version 2.3.1 on OSX and 2.3.0 on Linux.
            Reporter: Anton Daitche


The code snippet
{code:java}
import datetime

df = spark.createDataFrame([(datetime.datetime(1901, 1, 1, 0, 0, 0),)], ["ts"])
print(df.collect())
print(df.toPandas())
 {code}
prints
{code:java}
collect: 1901-01-01 00:00:00
toPandas: 1900-12-31 23:53:00{code}
Hence the toPandas methods seems to convert the timestamp wrongly.

The problem disappears for "1902-01-01 00:00:00" and later dates (I didn't do an exhaustive test though).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org