Hive + input format versioning dilemma

[Nerius Landys <nl...@gmail.com>]

Hi, I'm running some Hive jobs on Amazon Elastic MapReduce.

The versions for Hive and Hadoop as reported by the instance:

hadoop@ip-10-64-33-113:~$ hive --version
Hive version 0.7.1.4

hadoop@ip-10-64-33-113:~$ hadoop version
Hadoop 0.20.205

I'm able to specify a table and execute a query using the "STORED AS
INPUTFORMAT" just fine.  I even have my own custom input format
defined.  However, I noticed that the input format must inherit from
the "old" package org.apache.hadoop.mapred.  Anything that extends the
new input format classes in the new package
org.apache.hadoop.mapreduce won't work.

As a specific example of this, I'm able to specify:

STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat'

However the following fails:

STORED AS INPUTFORMAT 'org.apache.hadoop.mapreduce.lib.input.TextInputFormat'

...presumably because the latter TextInputFormat does not inherit from
the "older" org.apache.hadoop.mapred.InputFormat.

The exact error I get when executing a select with the newer input format API is

FAILED: Error in semantic analysis: Line 1:14 Input format must
implement InputFormat foo_table

Is there any way to convince Hive to honor the newer input format API in Hadoop?

Or do I need to keep the older input format implementations around in
my custom input format code?

Thanks in advance.