[GitHub] [spark] ulysses-you commented on a diff in pull request #38619: [SPARK-41112][SQL] RuntimeFilter should apply ColumnPruning eagerly with in-subquery filter

[GitBox <gi...@apache.org>]

ulysses-you commented on code in PR #38619:
URL: https://github.com/apache/spark/pull/38619#discussion_r1019946997


##########
sql/core/src/test/scala/org/apache/spark/sql/InjectRuntimeFilterSuite.scala:
##########
@@ -257,6 +257,11 @@ class InjectRuntimeFilterSuite extends QueryTest with SQLTestUtils with SharedSp
         val normalizedDisabled = normalizePlan(normalizeExprIds(planDisabled))
         ensureLeftSemiJoinExists(planEnabled)
         assert(normalizedEnabled != normalizedDisabled)
+        val agg = planEnabled.collect {
+          case Join(_, agg: Aggregate, LeftSemi, _, _) => agg
+        }
+        assert(agg.size == 1)
+        assert(agg.head.fastEquals(ColumnPruning(agg.head)))

Review Comment:
   this test can pass without this pr because
   https://github.com/apache/spark/blob/38897b1ce96a1e24629875619c165b4fd1fa2d8f/sql/core/src/main/scala/org/apache/spark/sql/execution/SparkOptimizer.scala#L68-L79
   
   I think it is just a coincidence since we converted the subquery to left semi join ..



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org