[jira] [Created] (SPARK-42384) Mask function's generated code does not handle null input

["Bruce Robbins (Jira)" <ji...@apache.org>]

Bruce Robbins created SPARK-42384:
-------------------------------------

             Summary: Mask function's generated code does not handle null input
                 Key: SPARK-42384
                 URL: https://issues.apache.org/jira/browse/SPARK-42384
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 3.5.0
            Reporter: Bruce Robbins


Example:
{noformat}
create or replace temp view v1 as
select * from values
(null),
('AbCD123-@$#')
as data(col1);

cache table v1;

select mask(col1) from v1;
{noformat}
This query results in a {{NullPointerException}}:
{noformat}
23/02/07 16:36:06 ERROR Executor: Exception in task 0.0 in stage 3.0 (TID 3)
java.lang.NullPointerException
	at org.apache.spark.sql.catalyst.expressions.codegen.UnsafeWriter.write(UnsafeWriter.java:110)
	at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown Source)
	at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
	at org.apache.spark.sql.execution.WholeStageCodegenExec$$anon$1.hasNext(WholeStageCodegenExec.scala:760)
{noformat}
The generated code calls {{UnsafeWriter.write(0, value_0)}} regardless of whether {{Mask.transformInput}} returns null or not. The {{UnsafeWriter.write}} method for {{UTF8String}} does not expect a null pointer.
{noformat}
/* 031 */     boolean isNull_1 = i.isNullAt(0);
/* 032 */     UTF8String value_1 = isNull_1 ?
/* 033 */     null : (i.getUTF8String(0));
/* 034 */
/* 035 */
/* 036 */
/* 037 */
/* 038 */     UTF8String value_0 = null;
/* 039 */     value_0 = org.apache.spark.sql.catalyst.expressions.Mask.transformInput(value_1, ((UTF8String) references[0] /* literal */), ((UTF8String) references[1] /* literal */), ((UTF8String) references[2] /* literal */), ((UTF8String) references[3] /* literal */));;
/* 040 */     if (false) {
/* 041 */       mutableStateArray_0[0].setNullAt(0);
/* 042 */     } else {
/* 043 */       mutableStateArray_0[0].write(0, value_0);
/* 044 */     }
/* 045 */     return (mutableStateArray_0[0].getRow());
/* 046 */   }
{noformat}

The bug is not exercised by a literal null input value, since there appears to be some optimization that simply replaces the entire function call with a null literal:
{noformat}
spark-sql> explain SELECT mask(NULL);
== Physical Plan ==
*(1) Project [null AS mask(NULL, X, x, n, NULL)#47]
+- *(1) Scan OneRowRelation[]

Time taken: 0.026 seconds, Fetched 1 row(s)
spark-sql> SELECT mask(NULL);
NULL
Time taken: 0.042 seconds, Fetched 1 row(s)
spark-sql> 
{noformat}




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org