You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Hudson (JIRA)" <ji...@apache.org> on 2013/01/09 11:27:33 UTC
[jira] [Commented] (HIVE-3306) SMBJoin/BucketMapJoin should be
allowed only when join key expression is exactly matches with sort/cluster
key
[ https://issues.apache.org/jira/browse/HIVE-3306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13548023#comment-13548023 ]
Hudson commented on HIVE-3306:
------------------------------
Integrated in Hive-trunk-hadoop2 #54 (See [https://builds.apache.org/job/Hive-trunk-hadoop2/54/])
HIVE-3306 SMBJoin/BucketMapJoin should be allowed only when join key expression is exactly matches
with sort/cluster key (Navis via namit) (Revision 1381669)
Result = ABORTED
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381669
Files :
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/BucketMapJoinOptimizer.java
* /hive/trunk/ql/src/test/queries/clientpositive/bucket_map_join_1.q
* /hive/trunk/ql/src/test/queries/clientpositive/bucket_map_join_2.q
* /hive/trunk/ql/src/test/queries/clientpositive/bucketmapjoin_negative3.q
* /hive/trunk/ql/src/test/results/clientpositive/bucket_map_join_1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucket_map_join_2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin_negative3.q.out
> SMBJoin/BucketMapJoin should be allowed only when join key expression is exactly matches with sort/cluster key
> --------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-3306
> URL: https://issues.apache.org/jira/browse/HIVE-3306
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Minor
> Fix For: 0.10.0
>
> Attachments: HIVE-3306.1.patch.txt
>
>
> CREATE TABLE bucket_small (key int, value string) CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS STORED AS TEXTFILE;
> load data local inpath '/home/navis/apache/oss-hive/data/files/srcsortbucket1outof4.txt' INTO TABLE bucket_small;
> load data local inpath '/home/navis/apache/oss-hive/data/files/srcsortbucket2outof4.txt' INTO TABLE bucket_small;
> CREATE TABLE bucket_big (key int, value string) CLUSTERED BY (key) SORTED BY (key) INTO 4 BUCKETS STORED AS TEXTFILE;
> load data local inpath '/home/navis/apache/oss-hive/data/files/srcsortbucket1outof4.txt' INTO TABLE bucket_big;
> load data local inpath '/home/navis/apache/oss-hive/data/files/srcsortbucket2outof4.txt' INTO TABLE bucket_big;
> load data local inpath '/home/navis/apache/oss-hive/data/files/srcsortbucket3outof4.txt' INTO TABLE bucket_big;
> load data local inpath '/home/navis/apache/oss-hive/data/files/srcsortbucket4outof4.txt' INTO TABLE bucket_big;
> select count(*) FROM bucket_small a JOIN bucket_big b ON a.key + a.key = b.key;
> select /* + MAPJOIN(a) */ count(*) FROM bucket_small a JOIN bucket_big b ON a.key + a.key = b.key;
> returns 116 (same)
> But with BucketMapJoin or SMBJoin, it returns 61. But this should not be allowed cause hash(a.key) != hash(a.key + a.key).
> Bucket context should be utilized only with exact matching join expression with sort/cluster key.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira