[GitHub] [spark] HyukjinKwon commented on a change in pull request #25008: [SPARK-28213][SQL] Replace ColumnarBatchScan with equivilant from Columnar

[GitBox <gi...@apache.org>]

HyukjinKwon commented on a change in pull request #25008: [SPARK-28213][SQL] Replace ColumnarBatchScan with equivilant from Columnar
URL: https://github.com/apache/spark/pull/25008#discussion_r307569775
 
 

 ##########
 File path: sql/core/src/test/scala/org/apache/spark/sql/test/SQLTestUtils.scala
 ##########
 @@ -392,7 +392,7 @@ private[sql] trait SQLTestUtilsBase
    */
   protected def stripSparkFilter(df: DataFrame): DataFrame = {
     val schema = df.schema
-    val withoutFilters = df.queryExecution.sparkPlan.transform {
+    val withoutFilters = df.queryExecution.executedPlan.transform {
 
 Review comment:
   Seems like the existing columnar logic within each plans have moved to `RowToColumnarExec` and `ColumnarToRowExec` to deduplicate but it's now dependent on `ApplyColumnarRulesAndInsertTransitions` rule, which, now, requires execution preparation (`QueryExecution.preparations`).
   
   However, per the doc, `executedPlan` should be only used for execution ideally. It could have been best to avoid. @revans2 even though it's too late, can you please describe what does this PR fixes in the PR description (presumably by listing each item)?
   
   I have no idea what this PR fixes from reading the PR description and JIRA.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org