You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (Jira)" <ji...@apache.org> on 2022/03/07 02:25:00 UTC
[jira] [Commented] (SPARK-38399) Why doesn't shuffle hash join support build left table for left-outer-join but full-outer-join?
[ https://issues.apache.org/jira/browse/SPARK-38399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17502059#comment-17502059 ]
Hyukjin Kwon commented on SPARK-38399:
--------------------------------------
[~mengdou] let's ask questions to the Spark user/dev mailing list before filing it as an issue. You should be able to get a better answer.
> Why doesn't shuffle hash join support build left table for left-outer-join but full-outer-join?
> -----------------------------------------------------------------------------------------------
>
> Key: SPARK-38399
> URL: https://issues.apache.org/jira/browse/SPARK-38399
> Project: Spark
> Issue Type: Question
> Components: Spark Core, SQL
> Affects Versions: 3.2.0
> Reporter: mengdou
> Priority: Minor
> Attachments: image-2022-03-03-16-34-17-909.png
>
>
> Why doesn't shuffle hash join support building left table for left-outer-join, but it supports building right table for full-outer-join?
> !image-2022-03-03-16-34-17-909.png!
>
> IMO, if left table is the build table, similar to full-outer-table, we can first create a BitSet to record any mismatch of next joins, and iterate all rows from stream table iterator and look up the hash key in the hash relation.
> If no one from stream table can join with the built hash relation, then we iterate the hash relation and get relative value from the BitSet, so we can get the left-outer rows.
>
> Does anyone help me? Thx~
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org