You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by ma qiang <ma...@gmail.com> on 2008/11/11 02:54:27 UTC
how many maps in a map task?
hi all,
I hava a data set stored in hbase, and I run a mapreduce program
to analyze. Now I want to know how many maps in a map task?
I want to use the number of the maps in my program. For
example. There are 100 maps in a map task, and I want to collect all
the values, and analyze these values then output the last result in
the last map.
Generally, there is no ouput in the first 99 maps, only output exists
in the last map. So I should know how many maps in a map task and use
it in my code.
Thanks a lot.
yours qiang.
Re: how many maps in a map task?
Posted by ma qiang <ma...@gmail.com>.
yes,
It need further analyze in reducer.
On Tue, Nov 11, 2008 at 10:28 AM, Mice <mi...@gmail.com> wrote:
> it there a reducer in your program? or you need to output the result
> in map-side?
>
> 2008/11/11 ma qiang <ma...@gmail.com>:
>> hi all,
>> I hava a data set stored in hbase, and I run a mapreduce program
>> to analyze. Now I want to know how many maps in a map task?
>> I want to use the number of the maps in my program. For
>> example. There are 100 maps in a map task, and I want to collect all
>> the values, and analyze these values then output the last result in
>> the last map.
>> Generally, there is no ouput in the first 99 maps, only output exists
>> in the last map. So I should know how many maps in a map task and use
>> it in my code.
>>
>> Thanks a lot.
>>
>> yours qiang.
>>
>
>
>
> --
> http://www.hadoopchina.com
>
Re: how many maps in a map task?
Posted by Mice <mi...@gmail.com>.
it there a reducer in your program? or you need to output the result
in map-side?
2008/11/11 ma qiang <ma...@gmail.com>:
> hi all,
> I hava a data set stored in hbase, and I run a mapreduce program
> to analyze. Now I want to know how many maps in a map task?
> I want to use the number of the maps in my program. For
> example. There are 100 maps in a map task, and I want to collect all
> the values, and analyze these values then output the last result in
> the last map.
> Generally, there is no ouput in the first 99 maps, only output exists
> in the last map. So I should know how many maps in a map task and use
> it in my code.
>
> Thanks a lot.
>
> yours qiang.
>
--
http://www.hadoopchina.com