[GitHub] [spark] ulysses-you commented on a change in pull request #33711: Fix bug in disable unnecessary bucketed scan

[GitBox <gi...@apache.org>]

ulysses-you commented on a change in pull request #33711:
URL: https://github.com/apache/spark/pull/33711#discussion_r687307744



##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/bucketing/DisableUnnecessaryBucketedScan.scala
##########
@@ -120,7 +120,7 @@ object DisableUnnecessaryBucketedScan extends Rule[SparkPlan] {
 
   private def hasInterestingPartition(plan: SparkPlan): Boolean = {
     plan.requiredChildDistribution.exists {
-      case _: ClusteredDistribution | _: HashClusteredDistribution => true
+      case _: ClusteredDistribution | _: HashClusteredDistribution | AllTuples => true

Review comment:
       It's flaky to fix this issue in an optimizer rule since this rule can be disabled sometime. Is it better that change the outpt partitioning in `FileSourceScanExec` to `UnknownPartitioning` if the bucket is 1 ?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org