[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25331: [SPARK-27768][SQL] Support Infinity/NaN-related float/double literals case-insensitively

[GitBox <gi...@apache.org>]

dongjoon-hyun commented on a change in pull request #25331: [SPARK-27768][SQL] Support Infinity/NaN-related float/double literals case-insensitively
URL: https://github.com/apache/spark/pull/25331#discussion_r313190910
 
 

 ##########
 File path: docs/sql-migration-guide-upgrade.md
 ##########
 @@ -161,6 +161,95 @@ license: |
 
   - Since Spark 3.0, Dataset query fails if it contains ambiguous column reference that is caused by self join. A typical example: `val df1 = ...; val df2 = df1.filter(...);`, then `df1.join(df2, df1("a") > df2("a"))` returns an empty result which is quite confusing. This is because Spark cannot resolve Dataset column references that point to tables being self joined, and `df1("a")` is exactly the same as `df2("a")` in Spark. To restore the behavior before Spark 3.0, you can set `spark.sql.analyzer.failAmbiguousSelfJoin` to `false`.
 
+  - Since Spark 3.0, `Cast` function processes string literals such as 'Infinity', '+Infinity', '-Infinity', 'NaN', 'Inf', '+Inf', '-Inf' in case insensitive manner when casting the literals to `Double` or `Float` type to ensure greater compatibility with other database systems. This behaviour change is illustrated in the table below:
+    <table class="table">
+        <tr>
+          <th>
+            <b>Operation</b>
+          </th>
+          <th>
+            <b>Result prior to spark 3.0</b>
 
 Review comment:
   `spark` -> `Spark`.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org