[GitHub] [spark] MaxGekk commented on pull request #32170: [SPARK-35051][SQL] Support add/subtract of a day-time interval to/from a date

[GitBox <gi...@apache.org>]

MaxGekk commented on pull request #32170:
URL: https://github.com/apache/spark/pull/32170#issuecomment-819632794


   > Spark 3.1 truncates timestamp to date?
   
   @cloud-fan Yep. Spark 3.1 as well as master when `spark.sql.legacy.interval.enabled` is set to `true` casts the result to the type of the left parameter.
   
   > I think the new behavior makes more sense
   
   It depends how you get the interval. For instance,
   
   Spark 3.1:
   ```sql
   spark-sql> select date'now' + (date'now' - date'yesterday');
   2021-04-15
   ```
   
   After the changes:
   ```sql
   spark-sql> select date'now' + (date'now' - date'yesterday');
   2021-04-15 00:00:00
   ```
   As you can see, Spark 3.1 returns a date as the result which also makes sense.
   
   > shall we add an item in the migration guide?
   
   Adding ANSI day-time interval to a date wasn't supported in Spark 3.1. Actually, the behavior change is related to subtraction of dates/timestamps which has been already documented in the SQL migration guide:
   ```
     - In Spark 3.2, the dates subtraction expression such as `date1 - date2` returns values of `DayTimeIntervalType`. In Spark 3.1 and earlier, the returned type is `CalendarIntervalType`. To restore the behavior before Spark 3.2, you can set `spark.sql.legacy.interval.enabled` to `true`.
   
     - In Spark 3.2, the timestamps subtraction expression such as `timestamp '2021-03-31 23:48:00' - timestamp '2021-01-01 00:00:00'` returns values of `DayTimeIntervalType`. In Spark 3.1 and earlier, the type of the same expression is `CalendarIntervalType`. To restore the behavior before Spark 3.2, you can set `spark.sql.legacy.interval.enabled` to `true`.
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org