You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@hive.apache.org by Timothy Manuel <st...@yahoo.com> on 2015/04/07 07:12:39 UTC

Unable to make Sort Merge Bucket Join work

Hi,


 
I have two large tables which I need to perform an equijoinon. I have bucketed and sorted the two tables on the join key. I have then madethe following specifications when running the join SQL:-


 
SET hive.input.format=org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat;SET hive.auto.convert.sortmerge.join=true;SET hive.optimize.bucketmapjoin=true;SET hive.optimize.bucketmapjoin.sortedmerge=true;SET hive.auto.convert.sortmerge.join.noconditionaltask=true;
However I get this error:-


 
FAILED: SemanticException [Error 10135]: Sort merge bucketedjoin could not be performed. If you really want to perform the operation,either set hive.optimize.bucketmapjoin.sortedmerge=false, or sethive.enforce.sortmergebucketmapjoin=false.


 
What am I doing wrong? The version of Hive is0.13.0.2.1.2.0-402


 
Thanks

Re: Unable to make Sort Merge Bucket Join work

Posted by Timothy Manuel <st...@yahoo.com>.

Can someone please help me on this? I am getting the error :-
FAILED: SemanticException [Error 10135]: Sort merge bucketed join could not be performed. If you really want to perform the operation, either set hive.optimize.bucketmapjoin.sortedmerge=false, or set hive.enforce.sortmergebucketmapjoin=false.

Under what circumstances does this error occur? I have ensured both the tables are bucketed into the same number of buckets and sorted ascending by the same key (which is the join key).
What else can be the reason for the error?


 



     On Tuesday, 7 April 2015 10:47 AM, Timothy Manuel <st...@yahoo.com> wrote:
   

 
Hi,    I have two large tables which I need to perform an equijoinon. I have bucketed and sorted the two tables on the join key. I have then madethe following specifications when running the join SQL:-    SET hive.input.format=org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat;SET hive.auto.convert.sortmerge.join=true;SET hive.optimize.bucketmapjoin=true;SET hive.optimize.bucketmapjoin.sortedmerge=true;SET hive.auto.convert.sortmerge.join.noconditionaltask=true;
However I get this error:-    FAILED: SemanticException [Error 10135]: Sort merge bucketedjoin could not be performed. If you really want to perform the operation,either set hive.optimize.bucketmapjoin.sortedmerge=false, or sethive.enforce.sortmergebucketmapjoin=false.    What am I doing wrong? The version of Hive is0.13.0.2.1.2.0-402    Thanks