You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Cheng Su (Jira)" <ji...@apache.org> on 2020/07/27 20:12:00 UTC
[jira] [Updated] (SPARK-32286) Coalesce bucketed tables for
shuffled hash join if applicable
[ https://issues.apache.org/jira/browse/SPARK-32286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheng Su updated SPARK-32286:
-----------------------------
Parent: SPARK-32461
Issue Type: Sub-task (was: Improvement)
> Coalesce bucketed tables for shuffled hash join if applicable
> -------------------------------------------------------------
>
> Key: SPARK-32286
> URL: https://issues.apache.org/jira/browse/SPARK-32286
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 3.1.0
> Reporter: Cheng Su
> Assignee: Cheng Su
> Priority: Trivial
> Fix For: 3.1.0
>
>
> Based on a follow up comment in PR [#28123|https://github.com/apache/spark/pull/28123], where we can coalesce buckets for shuffled hash join as well. The note here is we only coalesce the buckets from shuffled hash join stream side (i.e. the side not building hash map), so we don't need to worry about OOM when coalescing multiple buckets in one task for building hash map.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org