Hive 0.7 use the old mapred API

[java8964 java8964 <ja...@hotmail.com>]

Hi, 
Our company current is using CDH3 release, which comes with Hive 0.7.1.
Right now, I have the data coming from another team, which also provides the custom InputFormat and RecorderReader, but using the new mapreduce API.
I am trying to build a hive table on these data, and hope I can reuse the existing InputFormat. But I got a problem, that in HIVE 0.7.x, only old API is supported.
I have 3 options now:
1) Upgrade whole hadoop system,which is not a real option as right now.2) Ask the other team to provide the custom InputFormat/RecorderReader under old mapred API. I am working on this, but not sure if it is possible or how long.3) Writable a wrapper class, providing old API based on the new API class. My question is, is the 3rd optional valid? Does anyone do the similar things before?

Thanks
Yong 		 	   		  

Re: Hive 0.7 use the old mapred API

[Edward Capriolo <ed...@gmail.com>]

this is an unfortunate issue with hive we only support mapred api not
mapreduce. If you look at the hbase and cassandra support you see that
sonetimes you can wrap a mapreduce input format by extending the new
interface and implementing the old one. So it is possible to use a newer
api but its ugly code wise. Cleaner would be to produce two distinct input
formats.

On Tuesday, December 4, 2012, Mark Grover <gr...@gmail.com>
wrote:
> Yong,
> In my opinion, #2 wouldn't be that hard to do.
> Of course, upgrading to the new API (albeit by updating Hive) would the
right thing to do long term.
> Mark
> On Mon, Dec 3, 2012 at 7:42 AM, java8964 java8964 <ja...@hotmail.com>
wrote:
>>
>> Hi,
>> Our company current is using CDH3 release, which comes with Hive 0.7.1.
>> Right now, I have the data coming from another team, which also provides
the custom InputFormat and RecorderReader, but using the new mapreduce API.
>> I am trying to build a hive table on these data, and hope I can reuse
the existing InputFormat. But I got a problem, that in HIVE 0.7.x, only old
API is supported.
>> I have 3 options now:
>> 1) Upgrade whole hadoop system,which is not a real option as right now.
>> 2) Ask the other team to provide the custom InputFormat/RecorderReader
under old mapred API. I am working on this, but not sure if it is possible
or how long.
>> 3) Writable a wrapper class, providing old API based on the new API
class. My question is, is the 3rd optional valid? Does anyone do
the similar things before?
>>
>> Thanks
>> Yong
>

Re: Hive 0.7 use the old mapred API

[Mark Grover <gr...@gmail.com>]

Yong,
In my opinion, #2 wouldn't be that hard to do.

Of course, upgrading to the new API (albeit by updating Hive) would the
right thing to do long term.

Mark

On Mon, Dec 3, 2012 at 7:42 AM, java8964 java8964 <ja...@hotmail.com>wrote:

>  Hi,
>
> Our company current is using CDH3 release, which comes with Hive 0.7.1.
>
> Right now, I have the data coming from another team, which also provides
> the custom InputFormat and RecorderReader, but using the new mapreduce API.
>
> I am trying to build a hive table on these data, and hope I can reuse the
> existing InputFormat. But I got a problem, that in HIVE 0.7.x, only old API
> is supported.
>
> I have 3 options now:
>
> 1) Upgrade whole hadoop system,which is not a real option as right now.
> 2) Ask the other team to provide the custom InputFormat/RecorderReader
> under old mapred API. I am working on this, but not sure if it is possible
> or how long.
> 3) Writable a wrapper class, providing old API based on the new API class.
> My question is, is the 3rd optional valid? Does anyone do
> the similar things before?
>
>
> Thanks
>
> Yong
>