[jira] [Created] (SPARK-33404) "date_trunc" expression returns incorrect results

["Utkarsh Agarwal (Jira)" <ji...@apache.org>]

Utkarsh Agarwal created SPARK-33404:
---------------------------------------

             Summary: "date_trunc" expression returns incorrect results
                 Key: SPARK-33404
                 URL: https://issues.apache.org/jira/browse/SPARK-33404
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 3.0.1, 3.0.0
            Reporter: Utkarsh Agarwal
             Fix For: 3.1.0


`date_trunc` SQL expression returns incorrect results for {{minute}} formatting string.

Context: The {{minute}} formatting string should truncate the timestamps such that the seconds is set to ZERO.

Repro (run the following commands in spark-shell):
{quote}
spark.conf.set("spark.sql.session.timeZone", "America/Los_Angeles")
spark.sql("SELECT date_trunc('minute', '1769-10-17 17:10:02')").show()
{quote}

Spark currently incorrectly returns 
{quote}
1769-10-17 17:10:02
{quote}
against the expected return value of 
{quote}
1769-10-17 17:10:00
{quote}
This happens as {{truncTimestamp}} in package {{org.apache.spark.sql.catalyst.util.DateTimeUtils}} incorrectly assumes that time zone offsets can never have the granularity of a second and thus does not account for time zone adjustment when truncating the timestamp to {{minute}}. 
This assumption is currently used when truncating the timestamps to {{microsecond, millisecond, second, or minute}}. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org