[GitHub] [spark] wangyum commented on a change in pull request #31240: [SPARK-34121][SQL] Intersect operator missing rowCount when CBO enabled

[GitBox <gi...@apache.org>]

wangyum commented on a change in pull request #31240:
URL: https://github.com/apache/spark/pull/31240#discussion_r560068147



##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/BasicStatsPlanVisitor.scala
##########
@@ -57,7 +57,15 @@ object BasicStatsPlanVisitor extends LogicalPlanVisitor[Statistics] {
 
   override def visitGlobalLimit(p: GlobalLimit): Statistics = fallback(p)
 
-  override def visitIntersect(p: Intersect): Statistics = fallback(p)
+  override def visitIntersect(p: Intersect): Statistics = {
+    val stats = p.children.map(_.stats)
+    val rowCount = if (stats.exists(_.rowCount.isEmpty)) {
+      None
+    } else {
+      Some(stats.map(_.rowCount.get).min)
+    }
+    Statistics(sizeInBytes = stats.map(_.sizeInBytes).min, rowCount = rowCount)

Review comment:
       We should avoid change the logic, maybe it should be:
   ```scala
   val leftStats = p.left.stats
   val rightStats = p.right.stats
   val leftSize = leftStats.sizeInBytes
   val rightSize = rightStats.sizeInBytes
   if (leftSize < rightSize) {
     Statistics(sizeInBytes = leftSize, rowCount = leftStats.rowCount)
   } else {
     Statistics(sizeInBytes = rightSize, rowCount = rightStats.rowCount)
   }
   ```




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org