You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "wangmeng (JIRA)" <ji...@apache.org> on 2014/07/22 10:31:40 UTC
[jira] [Updated] (HIVE-7469) skew join keys when two join table
have the same big skew key
[ https://issues.apache.org/jira/browse/HIVE-7469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
wangmeng updated HIVE-7469:
---------------------------
Description:
In https://issues.apache.org/jira/browse/HIVE-964, I have a general idea about how to deal with skew join key ,but there has a case which troubles me:
if the two join tables have the same big skew key on one value :
for example , select * from table A join B on A.id=b.id, both table A and B have a lot of keys on id=1, in this case , if we use map join to deal with the skew key id=1 ,maybe it will OOM.
so ,how to fix this case? Will it rollback to common join ? Thanks.
was:
In https://issues.apache.org/jira/browse/HIVE-964, I have an general idea about how to deal with skew join key ,but there has a case which troubles me:
if the two join tables have the same big skew key on one value :
for example , select * from table A join B on A.id=b.id, both table A and B have a lot of keys on id=1, in this case , if we use map join to deal with the skew key id=1 ,maybe it will OOM.
so ,how to fix this case? Will it rollback to common join ? Thanks.
> skew join keys when two join table have the same big skew key
> -----------------------------------------------------------------
>
> Key: HIVE-7469
> URL: https://issues.apache.org/jira/browse/HIVE-7469
> Project: Hive
> Issue Type: Improvement
> Reporter: wangmeng
>
> In https://issues.apache.org/jira/browse/HIVE-964, I have a general idea about how to deal with skew join key ,but there has a case which troubles me:
> if the two join tables have the same big skew key on one value :
> for example , select * from table A join B on A.id=b.id, both table A and B have a lot of keys on id=1, in this case , if we use map join to deal with the skew key id=1 ,maybe it will OOM.
> so ,how to fix this case? Will it rollback to common join ? Thanks.
--
This message was sent by Atlassian JIRA
(v6.2#6252)