[jira] [Created] (HIVE-10283) HIVE-4240 may be causing issue with bucketed tables

["Ryan P (JIRA)" <ji...@apache.org>]

Ryan P created HIVE-10283:
-----------------------------

             Summary: HIVE-4240 may be causing issue with bucketed tables 
                 Key: HIVE-10283
                 URL: https://issues.apache.org/jira/browse/HIVE-10283
             Project: Hive
          Issue Type: Bug
          Components: Hive
            Reporter: Ryan P


I suspect that by removing the reducer, HIVE-4240, may be causing issues. Because of this inserts will not consolidate 'buckets' into single files which is problematic when attempting to use bucketmapjoin.

CREATE TABLE IF NOT EXISTS buckettestinput( 
data string 
) 
ROW FORMAT DELIMITED FIELDS TERMINATED BY ','; 


CREATE TABLE IF NOT EXISTS buckettestoutput1( 
data string 
)CLUSTERED BY(data) 
INTO 2 BUCKETS 
ROW FORMAT DELIMITED FIELDS TERMINATED BY ','; 


CREATE TABLE IF NOT EXISTS buckettestoutput2( 
data string 
)CLUSTERED BY(data) 
INTO 2 BUCKETS 
ROW FORMAT DELIMITED FIELDS TERMINATED BY ','; 

Then I inserted the following data into the "buckettestinput" table 

firstinsert1 
firstinsert2 
firstinsert3 
firstinsert4 
firstinsert5 
firstinsert6 
firstinsert7 
firstinsert8 
secondinsert1 
secondinsert2 
secondinsert3 
secondinsert4 
secondinsert5 
secondinsert6 
secondinsert7 
secondinsert8 

set hive.enforce.bucketing = true; 
set hive.enforce.sorting=true; 

insert into table buckettestoutput1 
select * from buckettestinput where data like 'first%' 

SELECT * 
FROM buckettestoutput1 TABLESAMPLE(BUCKET 1 OUT OF 1 ON data) s; 


insert into table buckettestoutput1 
select * from buckettestinput where data like 'second%' 

check the results of the table sample query. 

for sort merge bucket map join 

set hive.auto.convert.sortmerge.join=true; 
set hive.optimize.bucketmapjoin = true; 
set hive.optimize.bucketmapjoin.sortedmerge = true; 
set hive.auto.convert.sortmerge.join.noconditionaltask=true; 


select * from buckettestoutput1 a join buckettestoutput2 b on (a.data=b.data) 

hive> select * from buckettestoutput1 a join buckettestoutput2 b on (a.data=b.data); 
FAILED: SemanticException [Error 10141]: Bucketed table metadata is not correct. Fix the metadata or don't use bucketed mapjoin, by setting hive.enforce.bucketmapjoin to false. The number of buckets for table buckettestoutput1 is 2, whereas the number of files is 4 




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)