[jira] [Created] (SPARK-37897) Filter with subexpression elimination may cause query failed

["hujiahua (Jira)" <ji...@apache.org>]

hujiahua created SPARK-37897:
--------------------------------

             Summary: Filter with subexpression elimination may cause query failed
                 Key: SPARK-37897
                 URL: https://issues.apache.org/jira/browse/SPARK-37897
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 3.2.0
            Reporter: hujiahua


 

The following test results will fail, the root cause was that the execution order of filter predicates had changed after subexpression elimination. So I think we should keep predicates execution order after subexpression elimination.
{code:java}
test("filter with subexpression elimination may cause query failed.") {
  withSQLConf((SQLConf.WHOLESTAGE_CODEGEN_ENABLED.key, "false")) {
    val df = Seq(-1, 1, 2).toDF("c1")

    //register `plusOne` udf, and the function will failed if input was not a positive number.
    spark.sqlContext.udf.register("plusOne",
      (n: Int) => { if (n >= 0) n + 1 else throw new SparkException("Must be positive number.") })

    val result = df.filter("c1 >= 0 and plusOne(c1) > 1 and plusOne(c1) < 3").collect()
    assert(result.size === 1)
  }
} {code}
 

 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org