You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2019/02/02 07:29:42 UTC
[GitHub] viirya commented on a change in pull request #23723:
[MINOR][SQL][DOCS] Reformat the tables in SQL migration guide
viirya commented on a change in pull request #23723: [MINOR][SQL][DOCS] Reformat the tables in SQL migration guide
URL: https://github.com/apache/spark/pull/23723#discussion_r253257325
##########
File path: docs/sql-migration-guide-upgrade.md
##########
@@ -100,64 +100,64 @@ displayTitle: Spark SQL Upgrading Guide
## Upgrading From Spark SQL 2.3 to 2.4
- In Spark version 2.3 and earlier, the second parameter to array_contains function is implicitly promoted to the element type of first array type parameter. This type promotion can be lossy and may cause `array_contains` function to return wrong result. This problem has been addressed in 2.4 by employing a safer type promotion mechanism. This can cause some change in behavior and are illustrated in the table below.
- <table class="table">
+ <table class="table">
<tr>
<th>
<b>Query</b>
</th>
<th>
- <b>Result Spark 2.3 or Prior</b>
+ <b>Spark 2.3 or Prior</b>
</th>
<th>
- <b>Result Spark 2.4</b>
+ <b>Spark 2.4</b>
</th>
<th>
<b>Remarks</b>
</th>
</tr>
<tr>
- <th>
- <b>SELECT <br> array_contains(array(1), 1.34D);</b>
- </th>
- <th>
- <b>true</b>
- </th>
- <th>
- <b>false</b>
- </th>
- <th>
- <b>In Spark 2.4, left and right parameters are promoted to array(double) and double type respectively.</b>
- </th>
+ <td>
+ <code>SELECT array_contains(array(1), 1.34D);</code>
+ </td>
+ <td>
+ <code>true</code>
+ </td>
+ <td>
+ <code>false</code>
+ </td>
+ <td>
+ In Spark 2.4, left and right parameters are promoted to array type of double type and double type respectively.
+ </td>
</tr>
<tr>
- <th>
- <b>SELECT <br> array_contains(array(1), '1');</b>
- </th>
- <th>
- <b>true</b>
- </th>
- <th>
- <b>AnalysisException is thrown since integer type can not be promoted to string type in a loss-less manner.</b>
- </th>
- <th>
- <b>Users can use explicit cast</b>
- </th>
+ <td>
+ <code>SELECT array_contains(array(1), '1');</code>
+ </td>
+ <td>
+ <code>true</code>
+ </td>
+ <td>
+ <code>AnalysisException</code> is thrown.
+ </td>
+ <td>
+ Explicit cast can be used in arguments to avoid the exception. In Spark 2.4, <code>AnalysisException</code> is thrown since integer type can not be promoted to string type in a loss-less manner.
+ </td>
</tr>
<tr>
- <th>
- <b>SELECT <br> array_contains(array(1), 'anystring');</b>
- </th>
- <th>
- <b>null</b>
- </th>
- <th>
- <b>AnalysisException is thrown since integer type can not be promoted to string type in a loss-less manner.</b>
- </th>
- <th>
- <b>Users can use explicit cast</b>
- </th>
+ <td>
+ <code>SELECT <br> array_contains(array(1), 'anystring');</code>
Review comment:
`<br>` is remove in above rows. Do we need it here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org