You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2023/01/01 18:02:12 UTC
[GitHub] [spark] Daniel-Davies commented on a diff in pull request #38867: [SPARK-41234][SQL][PYTHON] Add `array_insert` function
Daniel-Davies commented on code in PR #38867:
URL: https://github.com/apache/spark/pull/38867#discussion_r1059780745
##########
sql/core/src/test/resources/sql-tests/results/array.sql.out:
##########
@@ -427,6 +427,103 @@ struct<get(array(1, 2, 3), -1):int>
NULL
+-- !query
+select array_insert(array(1, 2, 3), 4, 4)
+-- !query schema
+struct<array_insert(array(1, 2, 3), 4, 4):array<int>>
+-- !query output
+[1,2,3,4]
+
+
+-- !query
+select array_insert(array(2, 3, 4), 0, 1)
+-- !query schema
+struct<array_insert(array(2, 3, 4), 0, 1):array<int>>
+-- !query output
+[1,2,3,4]
+
+
+-- !query
+select array_insert(array(2, 3, 4), 1, 1)
+-- !query schema
+struct<array_insert(array(2, 3, 4), 1, 1):array<int>>
+-- !query output
+[1,2,3,4]
+
+
+-- !query
+select array_insert(array(1, 3, 4), -2, 2)
+-- !query schema
+struct<array_insert(array(1, 3, 4), -2, 2):array<int>>
+-- !query output
+[1,2,3,4]
+
+
+-- !query
+select array_insert(array(1, 2, 3), 4, "4")
+-- !query schema
+struct<>
+-- !query output
+org.apache.spark.sql.AnalysisException
+{
+ "errorClass" : "DATATYPE_MISMATCH.ARRAY_FUNCTION_DIFF_TYPES",
+ "messageParameters" : {
+ "dataType" : "\"ARRAY\"",
+ "functionName" : "`array_insert`",
+ "leftType" : "\"ARRAY<INT>\"",
+ "rightType" : "\"STRING\"",
+ "sqlExpr" : "\"array_insert(array(1, 2, 3), 4, 4)\""
+ },
+ "queryContext" : [ {
+ "objectType" : "",
+ "objectName" : "",
+ "startIndex" : 8,
+ "stopIndex" : 43,
+ "fragment" : "array_insert(array(1, 2, 3), 4, \"4\")"
+ } ]
+}
+
+
+-- !query
+select array_insert(cast(NULL as ARRAY<INT>), 1, 1)
+-- !query schema
+struct<array_insert(NULL, 1, 1):array<int>>
+-- !query output
+NULL
+
+
+-- !query
+select array_insert(array(1, 2, 3, NULL), cast(NULL as INT), 4)
+-- !query schema
+struct<array_insert(array(1, 2, 3, NULL), CAST(NULL AS INT), 4):array<int>>
+-- !query output
+NULL
+
+
+-- !query
+select array_insert(array(1, 2, 3, NULL), 4, cast(NULL as INT))
+-- !query schema
+struct<array_insert(array(1, 2, 3, NULL), 4, CAST(NULL AS INT)):array<int>>
+-- !query output
+[1,2,3,null,null]
+
+
+-- !query
+select array_insert(array(2, 3, null, 4), 6, 5)
+-- !query schema
+struct<array_insert(array(2, 3, NULL, 4), 6, 5):array<int>>
+-- !query output
+[2,3,null,4,null,5]
Review Comment:
There doesn't seem to be a way to specify that the array can have NULL elements in the raw SQL syntax. I've added a null to the base array in the meantime, but please let me know if there is a better way to avoid the following outcome:
`-- !query
select array_insert(array(2, 3, 4), 5, 1)
-- !query schema
struct<array_insert(array(2, 3, 4), 5, 1):array<int>>
-- !query output
[2,3,4,0,1]`
##########
sql/core/src/test/resources/sql-tests/results/array.sql.out:
##########
@@ -427,6 +427,103 @@ struct<get(array(1, 2, 3), -1):int>
NULL
+-- !query
+select array_insert(array(1, 2, 3), 4, 4)
+-- !query schema
+struct<array_insert(array(1, 2, 3), 4, 4):array<int>>
+-- !query output
+[1,2,3,4]
+
+
+-- !query
+select array_insert(array(2, 3, 4), 0, 1)
+-- !query schema
+struct<array_insert(array(2, 3, 4), 0, 1):array<int>>
+-- !query output
+[1,2,3,4]
+
+
+-- !query
+select array_insert(array(2, 3, 4), 1, 1)
+-- !query schema
+struct<array_insert(array(2, 3, 4), 1, 1):array<int>>
+-- !query output
+[1,2,3,4]
+
+
+-- !query
+select array_insert(array(1, 3, 4), -2, 2)
+-- !query schema
+struct<array_insert(array(1, 3, 4), -2, 2):array<int>>
+-- !query output
+[1,2,3,4]
+
+
+-- !query
+select array_insert(array(1, 2, 3), 4, "4")
+-- !query schema
+struct<>
+-- !query output
+org.apache.spark.sql.AnalysisException
+{
+ "errorClass" : "DATATYPE_MISMATCH.ARRAY_FUNCTION_DIFF_TYPES",
+ "messageParameters" : {
+ "dataType" : "\"ARRAY\"",
+ "functionName" : "`array_insert`",
+ "leftType" : "\"ARRAY<INT>\"",
+ "rightType" : "\"STRING\"",
+ "sqlExpr" : "\"array_insert(array(1, 2, 3), 4, 4)\""
+ },
+ "queryContext" : [ {
+ "objectType" : "",
+ "objectName" : "",
+ "startIndex" : 8,
+ "stopIndex" : 43,
+ "fragment" : "array_insert(array(1, 2, 3), 4, \"4\")"
+ } ]
+}
+
+
+-- !query
+select array_insert(cast(NULL as ARRAY<INT>), 1, 1)
+-- !query schema
+struct<array_insert(NULL, 1, 1):array<int>>
+-- !query output
+NULL
+
+
+-- !query
+select array_insert(array(1, 2, 3, NULL), cast(NULL as INT), 4)
+-- !query schema
+struct<array_insert(array(1, 2, 3, NULL), CAST(NULL AS INT), 4):array<int>>
+-- !query output
+NULL
+
+
+-- !query
+select array_insert(array(1, 2, 3, NULL), 4, cast(NULL as INT))
+-- !query schema
+struct<array_insert(array(1, 2, 3, NULL), 4, CAST(NULL AS INT)):array<int>>
+-- !query output
+[1,2,3,null,null]
+
+
+-- !query
+select array_insert(array(2, 3, null, 4), 6, 5)
+-- !query schema
+struct<array_insert(array(2, 3, NULL, 4), 6, 5):array<int>>
+-- !query output
+[2,3,null,4,null,5]
Review Comment:
There doesn't seem to be a way to specify that the array can have NULL elements in the raw SQL syntax. I've added a null to the base array in the meantime, but please let me know if there is a better way to avoid the following outcome:
-- !query
select array_insert(array(2, 3, 4), 5, 1)
-- !query schema
struct<array_insert(array(2, 3, 4), 5, 1):array<int>>
-- !query output
[2,3,4,0,1]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org