[GitHub] HyukjinKwon commented on a change in pull request #23855: [SPARK-26930][SQL] Tests in ParquetFilterSuite don't verify filter class

[GitBox <gi...@apache.org>]

HyukjinKwon commented on a change in pull request #23855: [SPARK-26930][SQL] Tests in ParquetFilterSuite don't verify filter class
URL: https://github.com/apache/spark/pull/23855#discussion_r258871293
 
 

 ##########
 File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala
 ##########
 @@ -114,12 +114,24 @@ class ParquetFilterSuite extends QueryTest with ParquetTest with SharedSQLContex
             new SparkToParquetSchemaConverter(conf).convert(df.schema), pred)
           assert(maybeFilter.isDefined, s"Couldn't generate filter predicate for $pred")
           // Doesn't bother checking type parameters here (e.g. `Eq[Integer]`)
-          maybeFilter.exists(_.getClass === filterClass)
+          assert(flattenPredicateTree(maybeFilter.get).exists(_.getClass === filterClass))
         }
         checker(stripSparkFilter(query), expected)
     }
   }
 
+  private def flattenPredicateTree(filterPredicate: FilterPredicate): Seq[Object] = {
 
 Review comment:
   Hm, but I guess we're a bit unclear if we're going to check nested filters or only top filter.
   
   Hmmm .. I made another take based upon your branch (see https://github.com/nandorKollar/spark/pull/1) by disabling problematic rule(https://github.com/apache/spark/commit/ef77003178eb5cdcb4fe519fc540917656c5d577) explicitly. You could merge that into your branch so that it can be pushed to your PR.
   
   If we find no better way, it should be okay to disable that rule for this test specifically since technically it had to be done together at https://github.com/apache/spark/commit/ef77003178eb5cdcb4fe519fc540917656c5d577.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org