You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by mallik arjun <ma...@gmail.com> on 2013/09/05 15:43:16 UTC
what is the difference between mapper and identity mapper, reducer
and identity reducer?
hi all,
please tell me what is the difference between mapper and identtiy mapper ,
reducer and identity reducer.
thanks in advance.
Re: what is the difference between mapper and identity mapper,
reducer and identity reducer?
Posted by Shahab Yunus <sh...@gmail.com>.
Identity Mapper and Reducer just like the concept of Identity function in
mathematics i.e. do not transform the input and return it as it is in
output form. Identity Mapper takes the input key/value pair and spits it
out without any processing.
The case of identity reducer is a bit different. It does not mean that the
reduce step will not take place. It will take place and the related sorting
and shuffling will also be performed but there will be no aggregation. So
you can use identity reducer if you want to sort your data that is coming
from map but don't care for any grouping and also fine with multiple
reducer outputs (unlike using 1 reducer.)
Regards,
Shahab
On Thu, Sep 5, 2013 at 9:43 AM, mallik arjun <ma...@gmail.com> wrote:
> hi all,
>
> please tell me what is the difference between mapper and identtiy mapper
> , reducer and identity reducer.
>
> thanks in advance.
>
Re: what is the difference between mapper and identity mapper,
reducer and identity reducer?
Posted by Shahab Yunus <sh...@gmail.com>.
Identity Mapper and Reducer just like the concept of Identity function in
mathematics i.e. do not transform the input and return it as it is in
output form. Identity Mapper takes the input key/value pair and spits it
out without any processing.
The case of identity reducer is a bit different. It does not mean that the
reduce step will not take place. It will take place and the related sorting
and shuffling will also be performed but there will be no aggregation. So
you can use identity reducer if you want to sort your data that is coming
from map but don't care for any grouping and also fine with multiple
reducer outputs (unlike using 1 reducer.)
Regards,
Shahab
On Thu, Sep 5, 2013 at 9:43 AM, mallik arjun <ma...@gmail.com> wrote:
> hi all,
>
> please tell me what is the difference between mapper and identtiy mapper
> , reducer and identity reducer.
>
> thanks in advance.
>
Re: what is the difference between mapper and identity mapper,
reducer and identity reducer?
Posted by Shahab Yunus <sh...@gmail.com>.
Identity Mapper and Reducer just like the concept of Identity function in
mathematics i.e. do not transform the input and return it as it is in
output form. Identity Mapper takes the input key/value pair and spits it
out without any processing.
The case of identity reducer is a bit different. It does not mean that the
reduce step will not take place. It will take place and the related sorting
and shuffling will also be performed but there will be no aggregation. So
you can use identity reducer if you want to sort your data that is coming
from map but don't care for any grouping and also fine with multiple
reducer outputs (unlike using 1 reducer.)
Regards,
Shahab
On Thu, Sep 5, 2013 at 9:43 AM, mallik arjun <ma...@gmail.com> wrote:
> hi all,
>
> please tell me what is the difference between mapper and identtiy mapper
> , reducer and identity reducer.
>
> thanks in advance.
>
Re: what is the difference between mapper and identity mapper,
reducer and identity reducer?
Posted by Shahab Yunus <sh...@gmail.com>.
Identity Mapper and Reducer just like the concept of Identity function in
mathematics i.e. do not transform the input and return it as it is in
output form. Identity Mapper takes the input key/value pair and spits it
out without any processing.
The case of identity reducer is a bit different. It does not mean that the
reduce step will not take place. It will take place and the related sorting
and shuffling will also be performed but there will be no aggregation. So
you can use identity reducer if you want to sort your data that is coming
from map but don't care for any grouping and also fine with multiple
reducer outputs (unlike using 1 reducer.)
Regards,
Shahab
On Thu, Sep 5, 2013 at 9:43 AM, mallik arjun <ma...@gmail.com> wrote:
> hi all,
>
> please tell me what is the difference between mapper and identtiy mapper
> , reducer and identity reducer.
>
> thanks in advance.
>