You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2019/07/19 03:45:21 UTC
[GitHub] [spark] HyukjinKwon commented on a change in pull request #25153:
[SPARK-28389][SQL] Use Java 8 API in add_months
HyukjinKwon commented on a change in pull request #25153: [SPARK-28389][SQL] Use Java 8 API in add_months
URL: https://github.com/apache/spark/pull/25153#discussion_r305192591
##########
File path: docs/sql-migration-guide-upgrade.md
##########
@@ -151,6 +151,8 @@ license: |
- Since Spark 3.0, substitution order of nested WITH clauses is changed and an inner CTE definition takes precedence over an outer. In version 2.4 and earlier, `WITH t AS (SELECT 1), t2 AS (WITH t AS (SELECT 2) SELECT * FROM t) SELECT * FROM t2` returns `1` while in version 3.0 it returns `2`. The previous behaviour can be restored by setting `spark.sql.legacy.ctePrecedence.enabled` to `true`.
+ - Since Spark 3.0, the `add_months` function adjusts the resulting date to a last day of month only if it is invalid. For example, `select add_months(DATE'2019-01-31', 1)` results `2019-02-28`. In Spark version 2.4 and earlier, the resulting date is adjusted when it is invalid, or the original date is a last day of months. For example, adding a month to `2019-02-28` resultes in `2019-03-31`.
Review comment:
Hm .. shall we update this migration guide? Actually `select add_months(DATE'2019-01-31', 1)` returns `2019-02-28` in both the current master and the versions before Spark 2.4.x.
I think we should explicitly mention `select add_months(DATE'2019-02-28', 1)` case only:
```
scala> sql("select add_months(DATE'2019-02-28', 1)").show()
+--------------------------------+
|add_months(DATE '2019-02-28', 1)|
+--------------------------------+
| 2019-03-31|
+--------------------------------+
```
```
scala> sql("select add_months(DATE'2019-02-28', 1)").show()
+--------------------------------+
|add_months(DATE '2019-02-28', 1)|
+--------------------------------+
| 2019-03-28|
+--------------------------------+
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org