You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Koji Noguchi <kn...@yahoo-inc.com> on 2011/01/18 20:47:55 UTC
Re: How to use combiner in hadoop streaming
(Bcc general@. This is for Hadoop project level announcements and
discussions. http://hadoop.apache.org/mailing_lists.html . Using
mapreduce-user@)
Yeshan,
Which version of hadoop are you using?
Officially, combiner for non-java command was added in 0.21
https://issues.apache.org/jira/browse/HADOOP-4842
but I¹d bet some of those 0.20s have this backported.
Koji
On 1/18/11 12:06 AM, "Harsh J" <qw...@gmail.com> wrote:
> Right now, the Combiner has to be a Java Class. A command is
> unacceptable, as noted here:
> http://wiki.apache.org/hadoop/HadoopStreaming
>
> On Tue, Jan 18, 2011 at 1:30 PM, xieyeshan <xi...@software.ict.ac.cn>
> wrote:
>> hi,all
>>
>> I want to use combiner in hadoop streaming, but combiner is not the same
>> as mapper or reducer in streaming.How could I do this?
>> Thanks!
>>
>>
>> Yours,
>>
>> Yeshan Xie
>>
>> 2011-01-18
>>
>>
>>
>> xieyeshan
>>
>
>
>
> --
> Harsh J
> www.harshj.com
>
Re: Re: How to use combiner in hadoop streaming
Posted by xieyeshan <xi...@software.ict.ac.cn>.
Hadoop 0.21.0's combiner could be cmd and it just solves my problem.Thank all of you.Thanks^_^
2011-01-21
xieyeshan
发件人： Koji Noguchi
发送时间： 2011-01-19 03:48:31
收件人： mapreduce-user@hadoop.apache.org
抄送：
主题： Re: How to use combiner in hadoop streaming
(Bcc general@. This is for Hadoop project level announcements and
discussions. http://hadoop.apache.org/mailing_lists.html . Using
mapreduce-user@)
Yeshan,
Which version of hadoop are you using?
Officially, combiner for non-java command was added in 0.21
https://issues.apache.org/jira/browse/HADOOP-4842
but I¹d bet some of those 0.20s have this backported.
Koji
On 1/18/11 12:06 AM, "Harsh J" <qw...@gmail.com> wrote:
> Right now, the Combiner has to be a Java Class. A command is
> unacceptable, as noted here:
> http://wiki.apache.org/hadoop/HadoopStreaming
>
> On Tue, Jan 18, 2011 at 1:30 PM, xieyeshan <xi...@software.ict.ac.cn>
> wrote:
>> hi,all
>>
>> I want to use combiner in hadoop streaming, but combiner is not the same
>> as mapper or reducer in streaming.How could I do this?
>> Thanks!
>>
>>
>> Yours,
>>
>> Yeshan Xie
>>
>> 2011-01-18
>>
>>
>>
>> xieyeshan
>>
>
>
>
> --
> Harsh J
> www.harshj.com
>