[jira] [Created] (SPARK-39220) codegen cause NullPointException

["chenxusheng (Jira)" <ji...@apache.org>]

chenxusheng created SPARK-39220:
-----------------------------------

             Summary: codegen cause NullPointException
                 Key: SPARK-39220
                 URL: https://issues.apache.org/jira/browse/SPARK-39220
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 3.2.1, 2.4.6, 2.3.0
            Reporter: chenxusheng


The following code raises NullPointException

```

SELECT
  fk4c7a8cfc,
  fka54f2a73,
  fk37e266f7
FROM
  be2a04fad4a24848bee641825e5b3466
WHERE
  (
    fk4c7a8cfc is not null
    and fk4c7a8cfc<> ''
  )
LIMIT
  1000

```

However, if so, it is normal

```

SELECT
  fk4c7a8cfc,
  fka54f2a73,
  fk37e266f7
FROM
  be2a04fad4a24848bee641825e5b3466
WHERE
  (
    fk4c7a8cfc is not null
    and '' <> fk4c7a8cfc
  )
LIMIT
  1000

```

I just put the '' in where in front.

The reason for this problem is that the data contains null values.

 

org.apache.spark.sql.catalyst.expressions.codegen.CodegenContext#genEqual

```

def genEqual(dataType: DataType, c1: String, c2: String): String = dataType match {
case BinaryType => s"java.util.Arrays.equals($c1, $c2)"
case FloatType =>
s"((java.lang.Float.isNaN($c1) && java.lang.Float.isNaN($c2)) || $c1 == $c2)"
case DoubleType =>
s"((java.lang.Double.isNaN($c1) && java.lang.Double.isNaN($c2)) || $c1 == $c2)"
case dt: DataType if isPrimitiveType(dt) => s"$c1 == $c2"
case dt: DataType if dt.isInstanceOf[AtomicType] => s"$c1.equals($c2)"
case array: ArrayType => genComp(array, c1, c2) + " == 0"
case struct: StructType => genComp(struct, c1, c2) + " == 0"
case udt: UserDefinedType[_] => genEqual(udt.sqlType, c1, c2)
case NullType => "false"
case _ =>
throw new IllegalArgumentException(
"cannot generate equality code for un-comparable type: " + dataType.catalogString)
}

```

`case dt: DataType if dt.isInstanceOf[AtomicType] => s"$c1.equals($c2)"` Missing null value judgment?



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org