[jira] [Created] (SPARK-42401) Incorrect results or NPE when inserting null value using array_insert/array_append

["Bruce Robbins (Jira)" <ji...@apache.org>]

Bruce Robbins created SPARK-42401:
-------------------------------------

             Summary: Incorrect results or NPE when inserting null value using array_insert/array_append
                 Key: SPARK-42401
                 URL: https://issues.apache.org/jira/browse/SPARK-42401
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 3.4.0, 3.5.0
            Reporter: Bruce Robbins


Example:
{noformat}
create or replace temp view v1 as
select * from values
(array(1, 2, 3, 4), 5, 5),
(array(1, 2, 3, 4), 5, null)
as v1(col1,col2,col3);

select array_insert(col1, col2, col3) from v1;
{noformat}
This produces an incorrect result:
{noformat}
[1,2,3,4,5]
[1,2,3,4,0] <== should be [1,2,3,4,null]
{noformat}
A more succint example:
{noformat}
select array_insert(array(1, 2, 3, 4), 5, cast(null as int));
{noformat}
This also produces an incorrect result:
{noformat}
[1,2,3,4,0] <== should be [1,2,3,4,null]
{noformat}
Another example:
{noformat}
create or replace temp view v1 as
select * from values
(array('1', '2', '3', '4'), 5, '5'),
(array('1', '2', '3', '4'), 5, null)
as v1(col1,col2,col3);

select array_insert(col1, col2, col3) from v1;
{noformat}
The above query throws a {{NullPointerException}}:
{noformat}
23/02/10 11:08:05 ERROR SparkSQLDriver: Failed in [select array_insert(col1, col2, col3) from v1]
java.lang.NullPointerException
	at org.apache.spark.sql.catalyst.expressions.codegen.UnsafeWriter.write(UnsafeWriter.java:110)
	at org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection.apply(Unknown Source)
	at org.apache.spark.sql.execution.LocalTableScanExec.$anonfun$unsafeRows$1(LocalTableScanExec.scala:44)
{noformat}
{{array_append}} has the same issue:
{noformat}
spark-sql> select array_append(array(1, 2, 3, 4), cast(null as int));
[1,2,3,4,0] <== should be [1,2,3,4,null]
Time taken: 3.679 seconds, Fetched 1 row(s)
spark-sql> select array_append(array('1', '2', '3', '4'), cast(null as string));
23/02/10 11:13:36 ERROR Executor: Exception in task 0.0 in stage 1.0 (TID 1)
java.lang.NullPointerException
	at org.apache.spark.sql.catalyst.expressions.codegen.UnsafeWriter.write(UnsafeWriter.java:110)
	at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.project_doConsume_0$(Unknown Source)
	at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown Source)
{noformat}




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org