You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "mengdou (Jira)" <ji...@apache.org> on 2022/03/03 08:35:00 UTC
[jira] [Updated] (SPARK-38399) Why doesn't shuffle hash join support build left table for left-outer-join but full-outer-join?
[ https://issues.apache.org/jira/browse/SPARK-38399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
mengdou updated SPARK-38399:
----------------------------
Description:
Why doesn't shuffle hash join support building left table for left-outer-join, but it supports building right table for full-outer-join?
!image-2022-03-03-16-34-17-909.png!
IMO, if left table is the build table, similar to full-outer-table, we can first create a BitSet to record any mismatch of next joins, and iterate all rows from stream table iterator and look up the hash key in the hash relation.
If no one from stream table can join with the built hash relation, then we iterate the hash relation and get relative value from the BitSet, so we can get the left-outer rows.
Does anyone help me? Thx~
was:
Why doesn't shuffle hash join support building left table for left-outer-join, but it supports building right table for full-outer-join?
!image-2022-03-03-16-34-17-909.png!
IMO, if left table is the build table, similar to full-outer-table, we can first create a BitSet to record any mismatch of next joins, and iterate all rows from stream table iterator and look up the hash key in the hash relation.
If no one from stream table can join with the built hash relation, then we iterate the hash relation and get relative value from the BitSet, so we can get the left-outer rows.
Does anyone helps? Thx~
> Why doesn't shuffle hash join support build left table for left-outer-join but full-outer-join?
> -----------------------------------------------------------------------------------------------
>
> Key: SPARK-38399
> URL: https://issues.apache.org/jira/browse/SPARK-38399
> Project: Spark
> Issue Type: Question
> Components: Spark Core, SQL
> Affects Versions: 3.2.0
> Reporter: mengdou
> Priority: Minor
> Attachments: image-2022-03-03-16-34-17-909.png
>
>
> Why doesn't shuffle hash join support building left table for left-outer-join, but it supports building right table for full-outer-join?
> !image-2022-03-03-16-34-17-909.png!
>
> IMO, if left table is the build table, similar to full-outer-table, we can first create a BitSet to record any mismatch of next joins, and iterate all rows from stream table iterator and look up the hash key in the hash relation.
> If no one from stream table can join with the built hash relation, then we iterate the hash relation and get relative value from the BitSet, so we can get the left-outer rows.
>
> Does anyone help me? Thx~
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org