[jira] [Resolved] (SPARK-31887) Date casting to string is giving wrong value

["Hyukjin Kwon (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-31887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hyukjin Kwon resolved SPARK-31887.
----------------------------------
    Resolution: Cannot Reproduce

I can't reproduce this in the current master. Probably fixed somewhere in the master. It would be good if we can identify and think about porting it back.

> Date casting to string is giving wrong value
> --------------------------------------------
>
>                 Key: SPARK-31887
>                 URL: https://issues.apache.org/jira/browse/SPARK-31887
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core
>    Affects Versions: 2.4.5
>         Environment: The spark is running on cluster mode with Mesos.
>  
> Mesos agents are dockerised running on Ubuntu 18.
>  
> Timezone setting of docker instance: UTC
> Timezone of server hosting docker: America/New_York
> Timezone of driver machine: America/New_York
>            Reporter: Amit Gupta
>            Priority: Major
>
> The code converts the string to date and then write it in csv.
> {code:java}
> val x = Seq(("2020-02-19", "2020-02-19 05:11:00")).toDF("a", "b").select('a.cast("date"), 'b.cast("timestamp"))
> x.show()
> +----------+-------------------+
> |         a|                  b|
> +----------+-------------------+
> |2020-02-19|2020-02-19 05:11:00|
> +----------+-------------------+
> x.write.mode("overwrite").option("header", true).csv("/tmp/test1.csv")
> {code}
>  
> The date written in CSV file is different:
> {code:java}
> > snakebite cat "/tmp/test1.csv/*.csv"
> a,b
> 2020-02-18,2020-02-19T05:11:00.000Z{code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org