You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Anbarasan Murthy <an...@hotmail.com> on 2012/08/17 21:00:59 UTC
Map Reduce Question
Hi,
I have a question in mapreduce api.
Would like to know the significance of the following items under jobconf class.ValueGroupingComparator
KeyComparator
Thanks,Anbu.
Re: Map Reduce Question
Posted by Mohit Anchlia <mo...@gmail.com>.
See Hadoop definitive guide and search for chapter on Hadoop features
On Fri, Aug 17, 2012 at 6:20 PM, Manoj Khangaonkar <kh...@gmail.com>wrote:
> One usage of these is in a secondary sort , which is used , when you
> want the output values from Map sorted (within a key).
>
> You implement a KeyComparator and tell mapreduce to use it to order
> the keys using a composite key.
>
> To ensure that during partioning & Grouping , all the records for a
> key go to the same reducer
> you need to define and set a Partitioner and ValueGroupingComparator.
>
> search for Secondary Sort for more on this topic
>
> regards
>
> MJ
>
> On Fri, Aug 17, 2012 at 12:00 PM, Anbarasan Murthy
> <an...@hotmail.com> wrote:
> > Hi,
> >
> > I have a question in mapreduce api.
> >
> > Would like to know the significance of the following items under jobconf
> > class.
> > ValueGroupingComparator
> > KeyComparator
> >
> >
> >
> > Thanks,
> > Anbu.
>
>
>
> --
> http://khangaonkar.blogspot.com/
>
Re: Map Reduce Question
Posted by Mohit Anchlia <mo...@gmail.com>.
See Hadoop definitive guide and search for chapter on Hadoop features
On Fri, Aug 17, 2012 at 6:20 PM, Manoj Khangaonkar <kh...@gmail.com>wrote:
> One usage of these is in a secondary sort , which is used , when you
> want the output values from Map sorted (within a key).
>
> You implement a KeyComparator and tell mapreduce to use it to order
> the keys using a composite key.
>
> To ensure that during partioning & Grouping , all the records for a
> key go to the same reducer
> you need to define and set a Partitioner and ValueGroupingComparator.
>
> search for Secondary Sort for more on this topic
>
> regards
>
> MJ
>
> On Fri, Aug 17, 2012 at 12:00 PM, Anbarasan Murthy
> <an...@hotmail.com> wrote:
> > Hi,
> >
> > I have a question in mapreduce api.
> >
> > Would like to know the significance of the following items under jobconf
> > class.
> > ValueGroupingComparator
> > KeyComparator
> >
> >
> >
> > Thanks,
> > Anbu.
>
>
>
> --
> http://khangaonkar.blogspot.com/
>
Re: Map Reduce Question
Posted by Mohit Anchlia <mo...@gmail.com>.
See Hadoop definitive guide and search for chapter on Hadoop features
On Fri, Aug 17, 2012 at 6:20 PM, Manoj Khangaonkar <kh...@gmail.com>wrote:
> One usage of these is in a secondary sort , which is used , when you
> want the output values from Map sorted (within a key).
>
> You implement a KeyComparator and tell mapreduce to use it to order
> the keys using a composite key.
>
> To ensure that during partioning & Grouping , all the records for a
> key go to the same reducer
> you need to define and set a Partitioner and ValueGroupingComparator.
>
> search for Secondary Sort for more on this topic
>
> regards
>
> MJ
>
> On Fri, Aug 17, 2012 at 12:00 PM, Anbarasan Murthy
> <an...@hotmail.com> wrote:
> > Hi,
> >
> > I have a question in mapreduce api.
> >
> > Would like to know the significance of the following items under jobconf
> > class.
> > ValueGroupingComparator
> > KeyComparator
> >
> >
> >
> > Thanks,
> > Anbu.
>
>
>
> --
> http://khangaonkar.blogspot.com/
>
Re: Map Reduce Question
Posted by Mohit Anchlia <mo...@gmail.com>.
See Hadoop definitive guide and search for chapter on Hadoop features
On Fri, Aug 17, 2012 at 6:20 PM, Manoj Khangaonkar <kh...@gmail.com>wrote:
> One usage of these is in a secondary sort , which is used , when you
> want the output values from Map sorted (within a key).
>
> You implement a KeyComparator and tell mapreduce to use it to order
> the keys using a composite key.
>
> To ensure that during partioning & Grouping , all the records for a
> key go to the same reducer
> you need to define and set a Partitioner and ValueGroupingComparator.
>
> search for Secondary Sort for more on this topic
>
> regards
>
> MJ
>
> On Fri, Aug 17, 2012 at 12:00 PM, Anbarasan Murthy
> <an...@hotmail.com> wrote:
> > Hi,
> >
> > I have a question in mapreduce api.
> >
> > Would like to know the significance of the following items under jobconf
> > class.
> > ValueGroupingComparator
> > KeyComparator
> >
> >
> >
> > Thanks,
> > Anbu.
>
>
>
> --
> http://khangaonkar.blogspot.com/
>
Re: Map Reduce Question
Posted by Manoj Khangaonkar <kh...@gmail.com>.
One usage of these is in a secondary sort , which is used , when you
want the output values from Map sorted (within a key).
You implement a KeyComparator and tell mapreduce to use it to order
the keys using a composite key.
To ensure that during partioning & Grouping , all the records for a
key go to the same reducer
you need to define and set a Partitioner and ValueGroupingComparator.
search for Secondary Sort for more on this topic
regards
MJ
On Fri, Aug 17, 2012 at 12:00 PM, Anbarasan Murthy
<an...@hotmail.com> wrote:
> Hi,
>
> I have a question in mapreduce api.
>
> Would like to know the significance of the following items under jobconf
> class.
> ValueGroupingComparator
> KeyComparator
>
>
>
> Thanks,
> Anbu.
--
http://khangaonkar.blogspot.com/
Re: Map Reduce Question
Posted by Manoj Khangaonkar <kh...@gmail.com>.
One usage of these is in a secondary sort , which is used , when you
want the output values from Map sorted (within a key).
You implement a KeyComparator and tell mapreduce to use it to order
the keys using a composite key.
To ensure that during partioning & Grouping , all the records for a
key go to the same reducer
you need to define and set a Partitioner and ValueGroupingComparator.
search for Secondary Sort for more on this topic
regards
MJ
On Fri, Aug 17, 2012 at 12:00 PM, Anbarasan Murthy
<an...@hotmail.com> wrote:
> Hi,
>
> I have a question in mapreduce api.
>
> Would like to know the significance of the following items under jobconf
> class.
> ValueGroupingComparator
> KeyComparator
>
>
>
> Thanks,
> Anbu.
--
http://khangaonkar.blogspot.com/
Re: Map Reduce Question
Posted by Manoj Khangaonkar <kh...@gmail.com>.
One usage of these is in a secondary sort , which is used , when you
want the output values from Map sorted (within a key).
You implement a KeyComparator and tell mapreduce to use it to order
the keys using a composite key.
To ensure that during partioning & Grouping , all the records for a
key go to the same reducer
you need to define and set a Partitioner and ValueGroupingComparator.
search for Secondary Sort for more on this topic
regards
MJ
On Fri, Aug 17, 2012 at 12:00 PM, Anbarasan Murthy
<an...@hotmail.com> wrote:
> Hi,
>
> I have a question in mapreduce api.
>
> Would like to know the significance of the following items under jobconf
> class.
> ValueGroupingComparator
> KeyComparator
>
>
>
> Thanks,
> Anbu.
--
http://khangaonkar.blogspot.com/
Re: Map Reduce Question
Posted by Manoj Khangaonkar <kh...@gmail.com>.
One usage of these is in a secondary sort , which is used , when you
want the output values from Map sorted (within a key).
You implement a KeyComparator and tell mapreduce to use it to order
the keys using a composite key.
To ensure that during partioning & Grouping , all the records for a
key go to the same reducer
you need to define and set a Partitioner and ValueGroupingComparator.
search for Secondary Sort for more on this topic
regards
MJ
On Fri, Aug 17, 2012 at 12:00 PM, Anbarasan Murthy
<an...@hotmail.com> wrote:
> Hi,
>
> I have a question in mapreduce api.
>
> Would like to know the significance of the following items under jobconf
> class.
> ValueGroupingComparator
> KeyComparator
>
>
>
> Thanks,
> Anbu.
--
http://khangaonkar.blogspot.com/