You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Wenchen Fan (Jira)" <ji...@apache.org> on 2020/02/07 18:37:00 UTC
[jira] [Assigned] (SPARK-30752) Wrong result of to_utc_timestamp()
on daylight saving day
[ https://issues.apache.org/jira/browse/SPARK-30752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Wenchen Fan reassigned SPARK-30752:
-----------------------------------
Assignee: Maxim Gekk
> Wrong result of to_utc_timestamp() on daylight saving day
> ---------------------------------------------------------
>
> Key: SPARK-30752
> URL: https://issues.apache.org/jira/browse/SPARK-30752
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.4.4, 3.0.0
> Reporter: Maxim Gekk
> Assignee: Maxim Gekk
> Priority: Major
>
> The to_utc_timestamp() function returns wrong result when:
> * JVM system time zone is PST
> * the session local time zone is UTC
> * fromZone is Asia/Hong_Kong
> for the local date '2019-11-03T12:00:00', the result must be '2019-11-03T04:00:00'
> {code}
> scala> import java.util.TimeZone
> import java.util.TimeZone
> scala> import org.apache.spark.sql.catalyst.util.DateTimeUtils._
> import org.apache.spark.sql.catalyst.util.DateTimeUtils._
> scala> import org.apache.spark.sql.functions._
> import org.apache.spark.sql.functions._
> scala> TimeZone.setDefault(getTimeZone("PST"))
> scala> spark.conf.set("spark.sql.session.timeZone", "UTC")
> scala> val df = Seq("2019-11-03T12:00:00").toDF("localTs")
> df: org.apache.spark.sql.DataFrame = [localTs: string]
> scala> df.select(to_utc_timestamp(col("localTs"), "Asia/Hong_Kong")).show
> +-----------------------------------------+
> |to_utc_timestamp(localTs, Asia/Hong_Kong)|
> +-----------------------------------------+
> | 2019-11-03 03:00:00|
> +-----------------------------------------+
> {code}
>
> See https://www.worldtimebuddy.com/?qm=1&lid=8,1819729,100&h=8&date=2019-11-2&sln=21-22
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org