Filtering while mapping or reducing?

[Afflictedd2 <fl...@gmail.com>]

Hi everyone,

I need to add the frequencies of two elements with the same last letter,
however
I don't want single elements in the output of this map reduce. I don't know
where
the filtering of those elements should either be done in the mapping
function or
the reduce function?

For example, if I have the following pairs:

{"f2B"=>4,
 "f1A"=>3,
 "f2C"=>5,
 "f1B"=>4,
 "f2D"=>7,
 "f1C"=>5,
 "f2E"=>8,
 "f1D"=>7,
 "f1F"=>8,
 "f2A"=>3}

I want the following pairs out

{"B"=>4,
 "A"=>3,
 "C"=>5,
 "B"=>4,
 "D"=>7,
 "C"=>5,
 "E"=>8,
 "D"=>7,
 "F"=>8,
 "A"=>3}

but I don't want E and F in my final result of reduced pairs

{"B"=>8,
 "A"=>6,
 "C"=>10,
 "D"=>14,
}
-- 
View this message in context: http://old.nabble.com/Filtering-while-mapping-or-reducing--tp31364272p31364272.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.

Re: Filtering while mapping or reducing?

[Harsh J <ha...@cloudera.com>]

Hello,

On Sun, Apr 10, 2011 at 8:59 PM, Afflictedd2 <fl...@gmail.com> wrote:
> For example, if I have the following pairs:
>
> {"f2B"=>4,
>  "f1A"=>3,
>  "f2C"=>5,
>  "f1B"=>4,
>  "f2D"=>7,
>  "f1C"=>5,
>  "f2E"=>8,
>  "f1D"=>7,
>  "f1F"=>8,
>  "f2A"=>3}
>
> I want the following pairs out
>
> {"B"=>4,
>  "A"=>3,
>  "C"=>5,
>  "B"=>4,
>  "D"=>7,
>  "C"=>5,
>  "E"=>8,
>  "D"=>7,
>  "F"=>8,
>  "A"=>3}
>
> but I don't want E and F in my final result of reduced pairs
>
> {"B"=>8,
>  "A"=>6,
>  "C"=>10,
>  "D"=>14,
> }
> --

You will need to do this in your reducer..

-- 
Harsh J