You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Gang Luo <lg...@yahoo.com.cn> on 2009/11/13 01:58:03 UTC
Re： How to use comstom mapreduce script
Thanks Zheng. It helps.
-Gang
----- 原始邮件 ----
发件人： Zheng Shao <zs...@gmail.com>
收件人： hive-user@hadoop.apache.org; pyang@facebook.com
发送日期： 2009/11/12 (周四) 7:10:33 下午
主 题： Re: How to use comstom mapreduce script
Yes you can compile it into a jar, and insert the command line "java
/xxx/my.jar" into Hive queries.
http://www.slideshare.net/ragho/hive-user-meeting-august-2009-facebook
Page 72 has an example
IF your map/reduce function is simple, you can probably write a Hive
UDF instead.
In the near future, Hive will support UDTF and you can write a UDTF
for any map/reduce function. This will make it more efficient than the
current way (running "java /xxx/my.jar").
Zheng
2009/11/13 Gang Luo <lg...@yahoo.com.cn>:
> Hi all,
> It is said that we can plug our own map/reduce into hive. I got two
> questions here.
> 1) Can I write this 'script' in Java? If so, should I compile it and make it
> into a jar file first?
> 2) Should I load this script file into HDFS first?
>
> Thanks.
>
> --Gang
>
> ________________________________
> 好玩贺卡等你发，邮箱贺卡全新上线！
--
Yours,
Zheng
___________________________________________________________
好玩贺卡等你发，邮箱贺卡全新上线！
http://card.mail.cn.yahoo.com/
Re: Re： How to use comstom mapreduce script
Posted by Carl Steinbach <ca...@cloudera.com>.
Larry's helper classes would be a useful addition to hive/contrib.
Carl
2009/11/12 Larry Ogrodnek <la...@bizo.com>
> Hello Gang,
>
> I put together some examples and helper classes for writing hive
> map/reduce scripts in java here:
>
> http://dev.bizo.com/2009/10/hive-map-reduce-in-java.html
>
> I hope it is useful.
>
> thanks,
> larry
>
> 2009/11/12 Larry Ogrodnek <la...@bizo.com>:
> > Hello Gang,
> >
> > I put together some examples and helper classes for writing hive
> > map/reduce scripts in java here:
> >
> > http://dev.bizo.com/2009/10/hive-map-reduce-in-java.html
> >
> > I hope it is useful.
> >
> > 2009/11/12 Gang Luo <lg...@yahoo.com.cn>:
> >> Thanks Zheng. It helps.
> >>
> >>
> >> -Gang
> >>
> >>
> >>
> >> ----- 原始邮件 ----
> >> 发件人： Zheng Shao <zs...@gmail.com>
> >> 收件人： hive-user@hadoop.apache.org; pyang@facebook.com
> >> 发送日期： 2009/11/12 (周四) 7:10:33 下午
> >> 主 题： Re: How to use comstom mapreduce script
> >>
> >> Yes you can compile it into a jar, and insert the command line "java
> >> /xxx/my.jar" into Hive queries.
> >>
> >> http://www.slideshare.net/ragho/hive-user-meeting-august-2009-facebook
> >> Page 72 has an example
> >>
> >> IF your map/reduce function is simple, you can probably write a Hive
> >> UDF instead.
> >> In the near future, Hive will support UDTF and you can write a UDTF
> >> for any map/reduce function. This will make it more efficient than the
> >> current way (running "java /xxx/my.jar").
> >>
> >>
> >> Zheng
> >>
> >> 2009/11/13 Gang Luo <lg...@yahoo.com.cn>:
> >>> Hi all,
> >>> It is said that we can plug our own map/reduce into hive. I got two
> >>> questions here.
> >>> 1) Can I write this 'script' in Java? If so, should I compile it and
> make it
> >>> into a jar file first?
> >>> 2) Should I load this script file into HDFS first?
> >>>
> >>> Thanks.
> >>>
> >>> --Gang
> >>>
> >>> ________________________________
> >>> 好玩贺卡等你发，邮箱贺卡全新上线！
> >>
> >>
> >>
> >> --
> >> Yours,
> >> Zheng
> >>
> >>
> >>
> >> ___________________________________________________________
> >> 好玩贺卡等你发，邮箱贺卡全新上线！
> >> http://card.mail.cn.yahoo.com/
> >>
> >
>
Re: Re： How to use comstom mapreduce script
Posted by Larry Ogrodnek <la...@bizo.com>.
Hello Gang,
I put together some examples and helper classes for writing hive
map/reduce scripts in java here:
http://dev.bizo.com/2009/10/hive-map-reduce-in-java.html
I hope it is useful.
thanks,
larry
2009/11/12 Larry Ogrodnek <la...@bizo.com>:
> Hello Gang,
>
> I put together some examples and helper classes for writing hive
> map/reduce scripts in java here:
>
> http://dev.bizo.com/2009/10/hive-map-reduce-in-java.html
>
> I hope it is useful.
>
> 2009/11/12 Gang Luo <lg...@yahoo.com.cn>:
>> Thanks Zheng. It helps.
>>
>>
>> -Gang
>>
>>
>>
>> ----- 原始邮件 ----
>> 发件人： Zheng Shao <zs...@gmail.com>
>> 收件人： hive-user@hadoop.apache.org; pyang@facebook.com
>> 发送日期： 2009/11/12 (周四) 7:10:33 下午
>> 主 题： Re: How to use comstom mapreduce script
>>
>> Yes you can compile it into a jar, and insert the command line "java
>> /xxx/my.jar" into Hive queries.
>>
>> http://www.slideshare.net/ragho/hive-user-meeting-august-2009-facebook
>> Page 72 has an example
>>
>> IF your map/reduce function is simple, you can probably write a Hive
>> UDF instead.
>> In the near future, Hive will support UDTF and you can write a UDTF
>> for any map/reduce function. This will make it more efficient than the
>> current way (running "java /xxx/my.jar").
>>
>>
>> Zheng
>>
>> 2009/11/13 Gang Luo <lg...@yahoo.com.cn>:
>>> Hi all,
>>> It is said that we can plug our own map/reduce into hive. I got two
>>> questions here.
>>> 1) Can I write this 'script' in Java? If so, should I compile it and make it
>>> into a jar file first?
>>> 2) Should I load this script file into HDFS first?
>>>
>>> Thanks.
>>>
>>> --Gang
>>>
>>> ________________________________
>>> 好玩贺卡等你发，邮箱贺卡全新上线！
>>
>>
>>
>> --
>> Yours,
>> Zheng
>>
>>
>>
>> ___________________________________________________________
>> 好玩贺卡等你发，邮箱贺卡全新上线！
>> http://card.mail.cn.yahoo.com/
>>
>