You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sumeet (Jira)" <ji...@apache.org> on 2021/07/09 03:34:00 UTC
[jira] [Created] (SPARK-36065) date_trunc returns incorrect output
Sumeet created SPARK-36065:
------------------------------
Summary: date_trunc returns incorrect output
Key: SPARK-36065
URL: https://issues.apache.org/jira/browse/SPARK-36065
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.3.0
Reporter: Sumeet
Hi,
Running date_trunc on any hour of "1891-10-01" returns incorrect output for "Europe/Bratislava" timezone.
Use the following steps in order to reproduce the issue:
* Run spark-shell using:
{code:java}
TZ="Europe/Bratislava" ./bin/spark-shell --conf spark.driver.extraJavaOptions='-Duser.timezone=Europe/Bratislava' --conf spark.executor.extraJavaOptions='-Duser.timezone=Europe/Bratislava' --conf spark.sql.session.timeZone="Europe/Bratislava"{code}
* Generate test data:
{code:java}
((0 until 9).map(hour => s"1891-10-01 00:0$hour:00") ++ (10 until 24).map(hour => s"1891-10-01 00:$hour:00")).toDF("ts_string").createOrReplaceTempView("temp_ts")
{code}
* Run query:
{code:java}
sql("select ts_string, cast(ts_string as TIMESTAMP) as ts, date_trunc('day', ts_string) from temp_ts").show(false)
{code}
* Output:
{code:java}
+-------------------+-------------------+--------------------------+
|ts_string |ts |date_trunc(day, ts_string)|
+-------------------+-------------------+--------------------------+
|1891-10-01 00:00:00|1891-10-01 00:02:16|1891-10-01 00:02:16 |
|1891-10-01 00:01:00|1891-10-01 00:03:16|1891-10-01 00:02:16 |
|1891-10-01 00:02:00|1891-10-01 00:04:16|1891-10-01 00:02:16 |
|1891-10-01 00:03:00|1891-10-01 00:03:00|1891-10-01 00:02:16 |
|1891-10-01 00:04:00|1891-10-01 00:04:00|1891-10-01 00:02:16 |
|1891-10-01 00:05:00|1891-10-01 00:05:00|1891-10-01 00:02:16 |
|1891-10-01 00:06:00|1891-10-01 00:06:00|1891-10-01 00:02:16 |
|1891-10-01 00:07:00|1891-10-01 00:07:00|1891-10-01 00:02:16 |
|1891-10-01 00:08:00|1891-10-01 00:08:00|1891-10-01 00:02:16 |
|1891-10-01 00:10:00|1891-10-01 00:10:00|1891-10-01 00:02:16 |
|1891-10-01 00:11:00|1891-10-01 00:11:00|1891-10-01 00:02:16 |
|1891-10-01 00:12:00|1891-10-01 00:12:00|1891-10-01 00:02:16 |
|1891-10-01 00:13:00|1891-10-01 00:13:00|1891-10-01 00:02:16 |
|1891-10-01 00:14:00|1891-10-01 00:14:00|1891-10-01 00:02:16 |
|1891-10-01 00:15:00|1891-10-01 00:15:00|1891-10-01 00:02:16 |
|1891-10-01 00:16:00|1891-10-01 00:16:00|1891-10-01 00:02:16 |
|1891-10-01 00:17:00|1891-10-01 00:17:00|1891-10-01 00:02:16 |
|1891-10-01 00:18:00|1891-10-01 00:18:00|1891-10-01 00:02:16 |
|1891-10-01 00:19:00|1891-10-01 00:19:00|1891-10-01 00:02:16 |
|1891-10-01 00:20:00|1891-10-01 00:20:00|1891-10-01 00:02:16 |
+-------------------+-------------------+--------------------------+
only showing top 20 rows
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org