You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/06/14 06:39:14 UTC
[GitHub] [spark] MaxGekk commented on a diff in pull request #36855: [SPARK-39432][SQL] element_at(*, 0) does not return INVALID_ARRAY_INDEX_IN_ELEMENT_AT
MaxGekk commented on code in PR #36855:
URL: https://github.com/apache/spark/pull/36855#discussion_r896426977
##########
sql/core/src/test/resources/sql-tests/results/ansi/array.sql.out:
##########
@@ -191,8 +191,8 @@ select element_at(array(1, 2, 3), 0)
-- !query schema
struct<>
-- !query output
-java.lang.ArrayIndexOutOfBoundsException
-SQL array indices start at 1
+org.apache.spark.SparkArrayIndexOutOfBoundsException
+[INVALID_ARRAY_INDEX_IN_ELEMENT_AT] The index 0 is out of bounds. The array has 3 elements. Use `try_element_at` to tolerate accessing element at invalid index and return NULL instead. If necessary set "spark.sql.ansi.enabled" to "false" to bypass this error.
Review Comment:
The error might confuse users, I guess. Why 0 is out of bound? We should say that indexes can be <0 (counting from the end) and >0 (the first element has index 1).
I would introduce separate error class for the case. WDYT @cloud-fan @srielau ?
##########
sql/core/src/test/resources/sql-tests/results/ansi/try_element_at.sql.out:
##########
@@ -7,8 +7,8 @@ SELECT try_element_at(array(1, 2, 3), 0)
-- !query schema
struct<>
-- !query output
-java.lang.ArrayIndexOutOfBoundsException
-SQL array indices start at 1
+org.apache.spark.SparkArrayIndexOutOfBoundsException
+[INVALID_ARRAY_INDEX_IN_ELEMENT_AT] The index 0 is out of bounds. The array has 3 elements. Use `try_element_at` to tolerate accessing element at invalid index and return NULL instead. If necessary set "spark.sql.ansi.enabled" to "false" to bypass this error.
Review Comment:
> Use `try_element_at` to tolerate
The query uses `try_element_at` already. The recommendation is not useful again.
##########
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CollectionExpressionsSuite.scala:
##########
@@ -2348,7 +2348,7 @@ class CollectionExpressionsSuite extends SparkFunSuite with ExpressionEvalHelper
// SQL array indices start at 1 exception throws for both mode.
expr = ElementAt(array, Literal(0))
- val errMsg = "SQL array indices start at 1"
+ val errMsg = "The index 0 is out of bounds. The array has 3 elements."
Review Comment:
The test `SPARK-33386: element_at ArrayIndexOutOfBoundsException` checks errors only. Should we move it to `QueryExecutionAnsiErrorsSuite` or/and `QueryExecutionErrorsSuite`?
##########
sql/core/src/test/resources/sql-tests/results/ansi/string-functions.sql.out:
##########
@@ -177,8 +177,8 @@ SELECT split_part('11.12.13', '.', 0)
-- !query schema
struct<>
-- !query output
-java.lang.ArrayIndexOutOfBoundsException
-SQL array indices start at 1
+org.apache.spark.SparkArrayIndexOutOfBoundsException
+[INVALID_ARRAY_INDEX_IN_ELEMENT_AT] The index 0 is out of bounds. The array has 3 elements. Use `try_element_at` to tolerate accessing element at invalid index and return NULL instead. If necessary set "spark.sql.ansi.enabled" to "false" to bypass this error.
Review Comment:
Use `try_element_at` to tolerate accessing element ... this might confuse users. The recommendation is not useful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org