You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Apache Spark (Jira)" <ji...@apache.org> on 2020/07/12 23:00:08 UTC
[jira] [Commented] (SPARK-32286) Coalesce bucketed tables for
shuffled hash join if applicable
[ https://issues.apache.org/jira/browse/SPARK-32286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17156409#comment-17156409 ]
Apache Spark commented on SPARK-32286:
--------------------------------------
User 'c21' has created a pull request for this issue:
https://github.com/apache/spark/pull/29079
> Coalesce bucketed tables for shuffled hash join if applicable
> -------------------------------------------------------------
>
> Key: SPARK-32286
> URL: https://issues.apache.org/jira/browse/SPARK-32286
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.1.0
> Reporter: Cheng Su
> Priority: Trivial
>
> Based on a follow up comment in PR [#28123|https://github.com/apache/spark/pull/28123], where we can coalesce buckets for shuffled hash join as well. The note here is we only coalesce the buckets from shuffled hash join stream side (i.e. the side not building hash map), so we don't need to worry about OOM when coalescing multiple buckets in one task for building hash map.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org