Review Request 24792: RCFile and bzip2 compression not working

[Navis Ryu <na...@nexr.com>]

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24792/
-----------------------------------------------------------

Review request for hive.


Bugs: HIVE-4788
    https://issues.apache.org/jira/browse/HIVE-4788


Repository: hive-git


Description
-------

The issue is that Bzip2 compressed rcfile data is encountering an error when being queried even the most simple query "select *". The issue is easily reproducible using the following. 

Create a table and load the sample data below. 

DDL: create table source_data (a string, b string) row format delimited fields terminated by ',';

Sample data: 
apple,sauce 

Test: 

Do the following and you should receive the error listed below for the rcfile table with bz2 compression. 

create table rc_nobz2 (a string, b string) stored as rcfile; 
insert into table rc_nobz2 select * from source_txt; 

SET io.seqfile.compression.type=BLOCK; 
SET hive.exec.compress.output=true; 
SET mapred.compress.map.output=true; 
SET mapred.output.compress=true; 
SET mapred.output.compression.codec=org.apache.hadoop.io.compress.BZip2Codec; 

create table rc_bz2 (a string, b string) stored as rcfile; 
insert into table rc_bz2 select * from source_txt; 

hive> select * from rc_bz2; 
Failed with exception java.io.IOException:java.io.IOException: Stream is not BZip2 formatted: expected 'h' as first byte but got 'ï¿¿' 
hive> select * from rc_nobz2; 
apple	sauce


Diffs
-----

  ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java 2a27676 
  ql/src/test/queries/clientpositive/rcfile_compress.q PRE-CREATION 
  ql/src/test/results/clientpositive/rcfile_compress.q.out PRE-CREATION 

Diff: https://reviews.apache.org/r/24792/diff/


Testing
-------


Thanks,

Navis Ryu