[GitHub] [spark] maropu commented on pull request #29473: [SPARK-32656][SQL] Repartition bucketed tables for sort merge join / shuffled hash join if applicable

[GitBox <gi...@apache.org>]

maropu commented on pull request #29473:
URL: https://github.com/apache/spark/pull/29473#issuecomment-679105200


   Thanks for the work, @imback82. Just a question; we cannot set a simpler rule to select which strategy (reapportioning or coalescing) we use when reading buckets? I think it is a bit annoying to set true to `repartitionBucketsInJoin.enabled` or `coalesceBucketsInJoin.enabled` data-by-data. What's a factor to make a difference between the two strategies, e.g., bucket size? For example, if one has too small buckets (this case is extreme though), repartitioning for higher cardinality might not be able to help for better performance.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org