You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "xuhuan (JIRA)" <ji...@apache.org> on 2015/04/07 03:51:12 UTC
[jira] [Created] (HIVE-10234) Problems using custom inputformat
xuhuan created HIVE-10234:
-----------------------------
Summary: Problems using custom inputformat
Key: HIVE-10234
URL: https://issues.apache.org/jira/browse/HIVE-10234
Project: Hive
Issue Type: Bug
Components: File Formats
Affects Versions: 0.13.1
Environment: CentOS release 6.3
CDH-5.3.1-1.cdh5.3.1.p0.5
hive-0.13.1-cdh5.3.1
Inputformat: https://github.com/twitter/elephant-bird/blob/master/rcfile/src/main/java/com/twitter/elephantbird/mapreduce/input/RCFileProtobufInputFormat.java
Reporter: xuhuan
Priority: Critical
I apply the RCFileProtobufInputFormat from twitter's elephantbird project into hive to handle Protobuf records which are written in a RCFile file format. What I have done is belowing:
1. compile the soure code of RCFileProtobufInputFormat and make a jar
2. add the jar to hive_aux_jar_path and update the config of server and client
3. create table using inputformat "com.twitter.elephantbird.mapreduce.input.RCFileProtobufInputFormat"
Creating table is ok.but when I excute a query like "select * from table test"(there is no data in table test),I got these mistake: FAILED: SemanticException 1:14 Input format must implement InputFormat. Error encountered near token test.
I found some cases in stackoverflow,someones said that's because the custom inputformat class implement the new API(org.apache.hadoop.marpreduce) and RCFileProtobufInputFormat class indeed extends the new API. I'm wondering wheather a custom inputformat must extend the old API(org.apache.hadoop.marpred)? Could this be the reason of my problem?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)