You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Cheng Su (Jira)" <ji...@apache.org> on 2020/07/24 03:17:00 UTC
[jira] [Created] (SPARK-32420) Add handling for unique key in
non-codegen hash join
Cheng Su created SPARK-32420:
--------------------------------
Summary: Add handling for unique key in non-codegen hash join
Key: SPARK-32420
URL: https://issues.apache.org/jira/browse/SPARK-32420
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 3.1.0
Reporter: Cheng Su
`HashRelation` has two separate code paths for unique key look up and non-unique key look up E.g. in its subclass `UnsafeHashedRelation`([https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala#L144-L177]), unique key look up is more efficient as it does not have extra `Iterator[UnsafeRow].hasNext()/next()` overhead per row.
`BroadcastHashJoinExec` has handled unique key vs non-unique key separately in code-gen path ([https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoinExec.scala#L289-L321]). But the non-codegen path for broadcast hash join and shuffled hash join do not separate it yet, so adding the support here.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org