You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by unmesha sreeveni <un...@gmail.com> on 2014/01/08 08:41:53 UTC
what all can be done using MR
Can we do aggregation with in Hadoop MR
like find min,max,sum,avg of a column in a csv file.
--
*Thanks & Regards*
Unmesha Sreeveni U.B
Junior Developer
http://www.unmeshasreeveni.blogspot.in/
Re: what all can be done using MR
Posted by unmesha sreeveni <un...@gmail.com>.
For that do we have to write a custom class for value inorder to pass all
the columns as value.
ie in the example 2 values. Or jst do a concatenation and emit values.
On Sat, Jan 11, 2014 at 9:46 PM, Chris Mawata <ch...@gmail.com>wrote:
> Results will be sorted by key so make A the key and put the rest in the
> value
> Chris
> On Jan 11, 2014 10:11 AM, "unmesha sreeveni" <un...@gmail.com>
> wrote:
>
>> What about sorting .
>> Acutually it is done by MapReduce itself.
>> But if we are giving a csv file as input and trying to sort one/multiple
>> column...Whether the corresponting columns also get reflectted??
>>
>> eg: foo.csv
>> B,2,3
>> A,4,6
>>
>> When we apply sorting to first column:whether the resultent will be
>> A,4,6
>> B,2,3
>> A will be mapped to its correct values right?
>> If so what will be context.write() of mapper?
>>
>>
>> On Wed, Jan 8, 2014 at 8:18 PM, Chris Mawata <ch...@gmail.com>wrote:
>>
>>> Yes.
>>> Check out, for example,
>>> http://packtlib.packtpub.com/library/hadoop-mapreduce-cookbook/ch06lvl1sec66#
>>>
>>>
>>>
>>> On 1/8/2014 2:41 AM, unmesha sreeveni wrote:
>>>
>>> Can we do aggregation with in Hadoop MR
>>> like find min,max,sum,avg of a column in a csv file.
>>>
>>> --
>>> *Thanks & Regards*
>>>
>>> Unmesha Sreeveni U.B
>>> Junior Developer
>>>
>>> http://www.unmeshasreeveni.blogspot.in/
>>>
>>>
>>>
>>>
>>
>>
>> --
>> *Thanks & Regards*
>>
>> Unmesha Sreeveni U.B
>> Junior Developer
>>
>> http://www.unmeshasreeveni.blogspot.in/
>>
>>
>>
--
*Thanks & Regards*
Unmesha Sreeveni U.B
Junior Developer
http://www.unmeshasreeveni.blogspot.in/
Re: what all can be done using MR
Posted by unmesha sreeveni <un...@gmail.com>.
For that do we have to write a custom class for value inorder to pass all
the columns as value.
ie in the example 2 values. Or jst do a concatenation and emit values.
On Sat, Jan 11, 2014 at 9:46 PM, Chris Mawata <ch...@gmail.com>wrote:
> Results will be sorted by key so make A the key and put the rest in the
> value
> Chris
> On Jan 11, 2014 10:11 AM, "unmesha sreeveni" <un...@gmail.com>
> wrote:
>
>> What about sorting .
>> Acutually it is done by MapReduce itself.
>> But if we are giving a csv file as input and trying to sort one/multiple
>> column...Whether the corresponting columns also get reflectted??
>>
>> eg: foo.csv
>> B,2,3
>> A,4,6
>>
>> When we apply sorting to first column:whether the resultent will be
>> A,4,6
>> B,2,3
>> A will be mapped to its correct values right?
>> If so what will be context.write() of mapper?
>>
>>
>> On Wed, Jan 8, 2014 at 8:18 PM, Chris Mawata <ch...@gmail.com>wrote:
>>
>>> Yes.
>>> Check out, for example,
>>> http://packtlib.packtpub.com/library/hadoop-mapreduce-cookbook/ch06lvl1sec66#
>>>
>>>
>>>
>>> On 1/8/2014 2:41 AM, unmesha sreeveni wrote:
>>>
>>> Can we do aggregation with in Hadoop MR
>>> like find min,max,sum,avg of a column in a csv file.
>>>
>>> --
>>> *Thanks & Regards*
>>>
>>> Unmesha Sreeveni U.B
>>> Junior Developer
>>>
>>> http://www.unmeshasreeveni.blogspot.in/
>>>
>>>
>>>
>>>
>>
>>
>> --
>> *Thanks & Regards*
>>
>> Unmesha Sreeveni U.B
>> Junior Developer
>>
>> http://www.unmeshasreeveni.blogspot.in/
>>
>>
>>
--
*Thanks & Regards*
Unmesha Sreeveni U.B
Junior Developer
http://www.unmeshasreeveni.blogspot.in/
Re: what all can be done using MR
Posted by unmesha sreeveni <un...@gmail.com>.
For that do we have to write a custom class for value inorder to pass all
the columns as value.
ie in the example 2 values. Or jst do a concatenation and emit values.
On Sat, Jan 11, 2014 at 9:46 PM, Chris Mawata <ch...@gmail.com>wrote:
> Results will be sorted by key so make A the key and put the rest in the
> value
> Chris
> On Jan 11, 2014 10:11 AM, "unmesha sreeveni" <un...@gmail.com>
> wrote:
>
>> What about sorting .
>> Acutually it is done by MapReduce itself.
>> But if we are giving a csv file as input and trying to sort one/multiple
>> column...Whether the corresponting columns also get reflectted??
>>
>> eg: foo.csv
>> B,2,3
>> A,4,6
>>
>> When we apply sorting to first column:whether the resultent will be
>> A,4,6
>> B,2,3
>> A will be mapped to its correct values right?
>> If so what will be context.write() of mapper?
>>
>>
>> On Wed, Jan 8, 2014 at 8:18 PM, Chris Mawata <ch...@gmail.com>wrote:
>>
>>> Yes.
>>> Check out, for example,
>>> http://packtlib.packtpub.com/library/hadoop-mapreduce-cookbook/ch06lvl1sec66#
>>>
>>>
>>>
>>> On 1/8/2014 2:41 AM, unmesha sreeveni wrote:
>>>
>>> Can we do aggregation with in Hadoop MR
>>> like find min,max,sum,avg of a column in a csv file.
>>>
>>> --
>>> *Thanks & Regards*
>>>
>>> Unmesha Sreeveni U.B
>>> Junior Developer
>>>
>>> http://www.unmeshasreeveni.blogspot.in/
>>>
>>>
>>>
>>>
>>
>>
>> --
>> *Thanks & Regards*
>>
>> Unmesha Sreeveni U.B
>> Junior Developer
>>
>> http://www.unmeshasreeveni.blogspot.in/
>>
>>
>>
--
*Thanks & Regards*
Unmesha Sreeveni U.B
Junior Developer
http://www.unmeshasreeveni.blogspot.in/
Re: what all can be done using MR
Posted by unmesha sreeveni <un...@gmail.com>.
For that do we have to write a custom class for value inorder to pass all
the columns as value.
ie in the example 2 values. Or jst do a concatenation and emit values.
On Sat, Jan 11, 2014 at 9:46 PM, Chris Mawata <ch...@gmail.com>wrote:
> Results will be sorted by key so make A the key and put the rest in the
> value
> Chris
> On Jan 11, 2014 10:11 AM, "unmesha sreeveni" <un...@gmail.com>
> wrote:
>
>> What about sorting .
>> Acutually it is done by MapReduce itself.
>> But if we are giving a csv file as input and trying to sort one/multiple
>> column...Whether the corresponting columns also get reflectted??
>>
>> eg: foo.csv
>> B,2,3
>> A,4,6
>>
>> When we apply sorting to first column:whether the resultent will be
>> A,4,6
>> B,2,3
>> A will be mapped to its correct values right?
>> If so what will be context.write() of mapper?
>>
>>
>> On Wed, Jan 8, 2014 at 8:18 PM, Chris Mawata <ch...@gmail.com>wrote:
>>
>>> Yes.
>>> Check out, for example,
>>> http://packtlib.packtpub.com/library/hadoop-mapreduce-cookbook/ch06lvl1sec66#
>>>
>>>
>>>
>>> On 1/8/2014 2:41 AM, unmesha sreeveni wrote:
>>>
>>> Can we do aggregation with in Hadoop MR
>>> like find min,max,sum,avg of a column in a csv file.
>>>
>>> --
>>> *Thanks & Regards*
>>>
>>> Unmesha Sreeveni U.B
>>> Junior Developer
>>>
>>> http://www.unmeshasreeveni.blogspot.in/
>>>
>>>
>>>
>>>
>>
>>
>> --
>> *Thanks & Regards*
>>
>> Unmesha Sreeveni U.B
>> Junior Developer
>>
>> http://www.unmeshasreeveni.blogspot.in/
>>
>>
>>
--
*Thanks & Regards*
Unmesha Sreeveni U.B
Junior Developer
http://www.unmeshasreeveni.blogspot.in/
Re: what all can be done using MR
Posted by Chris Mawata <ch...@gmail.com>.
Results will be sorted by key so make A the key and put the rest in the
value
Chris
On Jan 11, 2014 10:11 AM, "unmesha sreeveni" <un...@gmail.com> wrote:
> What about sorting .
> Acutually it is done by MapReduce itself.
> But if we are giving a csv file as input and trying to sort one/multiple
> column...Whether the corresponting columns also get reflectted??
>
> eg: foo.csv
> B,2,3
> A,4,6
>
> When we apply sorting to first column:whether the resultent will be
> A,4,6
> B,2,3
> A will be mapped to its correct values right?
> If so what will be context.write() of mapper?
>
>
> On Wed, Jan 8, 2014 at 8:18 PM, Chris Mawata <ch...@gmail.com>wrote:
>
>> Yes.
>> Check out, for example,
>> http://packtlib.packtpub.com/library/hadoop-mapreduce-cookbook/ch06lvl1sec66#
>>
>>
>>
>> On 1/8/2014 2:41 AM, unmesha sreeveni wrote:
>>
>> Can we do aggregation with in Hadoop MR
>> like find min,max,sum,avg of a column in a csv file.
>>
>> --
>> *Thanks & Regards*
>>
>> Unmesha Sreeveni U.B
>> Junior Developer
>>
>> http://www.unmeshasreeveni.blogspot.in/
>>
>>
>>
>>
>
>
> --
> *Thanks & Regards*
>
> Unmesha Sreeveni U.B
> Junior Developer
>
> http://www.unmeshasreeveni.blogspot.in/
>
>
>
Re: what all can be done using MR
Posted by Chris Mawata <ch...@gmail.com>.
Results will be sorted by key so make A the key and put the rest in the
value
Chris
On Jan 11, 2014 10:11 AM, "unmesha sreeveni" <un...@gmail.com> wrote:
> What about sorting .
> Acutually it is done by MapReduce itself.
> But if we are giving a csv file as input and trying to sort one/multiple
> column...Whether the corresponting columns also get reflectted??
>
> eg: foo.csv
> B,2,3
> A,4,6
>
> When we apply sorting to first column:whether the resultent will be
> A,4,6
> B,2,3
> A will be mapped to its correct values right?
> If so what will be context.write() of mapper?
>
>
> On Wed, Jan 8, 2014 at 8:18 PM, Chris Mawata <ch...@gmail.com>wrote:
>
>> Yes.
>> Check out, for example,
>> http://packtlib.packtpub.com/library/hadoop-mapreduce-cookbook/ch06lvl1sec66#
>>
>>
>>
>> On 1/8/2014 2:41 AM, unmesha sreeveni wrote:
>>
>> Can we do aggregation with in Hadoop MR
>> like find min,max,sum,avg of a column in a csv file.
>>
>> --
>> *Thanks & Regards*
>>
>> Unmesha Sreeveni U.B
>> Junior Developer
>>
>> http://www.unmeshasreeveni.blogspot.in/
>>
>>
>>
>>
>
>
> --
> *Thanks & Regards*
>
> Unmesha Sreeveni U.B
> Junior Developer
>
> http://www.unmeshasreeveni.blogspot.in/
>
>
>
Re: what all can be done using MR
Posted by Chris Mawata <ch...@gmail.com>.
Results will be sorted by key so make A the key and put the rest in the
value
Chris
On Jan 11, 2014 10:11 AM, "unmesha sreeveni" <un...@gmail.com> wrote:
> What about sorting .
> Acutually it is done by MapReduce itself.
> But if we are giving a csv file as input and trying to sort one/multiple
> column...Whether the corresponting columns also get reflectted??
>
> eg: foo.csv
> B,2,3
> A,4,6
>
> When we apply sorting to first column:whether the resultent will be
> A,4,6
> B,2,3
> A will be mapped to its correct values right?
> If so what will be context.write() of mapper?
>
>
> On Wed, Jan 8, 2014 at 8:18 PM, Chris Mawata <ch...@gmail.com>wrote:
>
>> Yes.
>> Check out, for example,
>> http://packtlib.packtpub.com/library/hadoop-mapreduce-cookbook/ch06lvl1sec66#
>>
>>
>>
>> On 1/8/2014 2:41 AM, unmesha sreeveni wrote:
>>
>> Can we do aggregation with in Hadoop MR
>> like find min,max,sum,avg of a column in a csv file.
>>
>> --
>> *Thanks & Regards*
>>
>> Unmesha Sreeveni U.B
>> Junior Developer
>>
>> http://www.unmeshasreeveni.blogspot.in/
>>
>>
>>
>>
>
>
> --
> *Thanks & Regards*
>
> Unmesha Sreeveni U.B
> Junior Developer
>
> http://www.unmeshasreeveni.blogspot.in/
>
>
>
Re: what all can be done using MR
Posted by Chris Mawata <ch...@gmail.com>.
Results will be sorted by key so make A the key and put the rest in the
value
Chris
On Jan 11, 2014 10:11 AM, "unmesha sreeveni" <un...@gmail.com> wrote:
> What about sorting .
> Acutually it is done by MapReduce itself.
> But if we are giving a csv file as input and trying to sort one/multiple
> column...Whether the corresponting columns also get reflectted??
>
> eg: foo.csv
> B,2,3
> A,4,6
>
> When we apply sorting to first column:whether the resultent will be
> A,4,6
> B,2,3
> A will be mapped to its correct values right?
> If so what will be context.write() of mapper?
>
>
> On Wed, Jan 8, 2014 at 8:18 PM, Chris Mawata <ch...@gmail.com>wrote:
>
>> Yes.
>> Check out, for example,
>> http://packtlib.packtpub.com/library/hadoop-mapreduce-cookbook/ch06lvl1sec66#
>>
>>
>>
>> On 1/8/2014 2:41 AM, unmesha sreeveni wrote:
>>
>> Can we do aggregation with in Hadoop MR
>> like find min,max,sum,avg of a column in a csv file.
>>
>> --
>> *Thanks & Regards*
>>
>> Unmesha Sreeveni U.B
>> Junior Developer
>>
>> http://www.unmeshasreeveni.blogspot.in/
>>
>>
>>
>>
>
>
> --
> *Thanks & Regards*
>
> Unmesha Sreeveni U.B
> Junior Developer
>
> http://www.unmeshasreeveni.blogspot.in/
>
>
>
Re: what all can be done using MR
Posted by unmesha sreeveni <un...@gmail.com>.
What about sorting .
Acutually it is done by MapReduce itself.
But if we are giving a csv file as input and trying to sort one/multiple
column...Whether the corresponting columns also get reflectted??
eg: foo.csv
B,2,3
A,4,6
When we apply sorting to first column:whether the resultent will be
A,4,6
B,2,3
A will be mapped to its correct values right?
If so what will be context.write() of mapper?
On Wed, Jan 8, 2014 at 8:18 PM, Chris Mawata <ch...@gmail.com> wrote:
> Yes.
> Check out, for example,
> http://packtlib.packtpub.com/library/hadoop-mapreduce-cookbook/ch06lvl1sec66#
>
>
>
> On 1/8/2014 2:41 AM, unmesha sreeveni wrote:
>
> Can we do aggregation with in Hadoop MR
> like find min,max,sum,avg of a column in a csv file.
>
> --
> *Thanks & Regards*
>
> Unmesha Sreeveni U.B
> Junior Developer
>
> http://www.unmeshasreeveni.blogspot.in/
>
>
>
>
--
*Thanks & Regards*
Unmesha Sreeveni U.B
Junior Developer
http://www.unmeshasreeveni.blogspot.in/
Re: what all can be done using MR
Posted by unmesha sreeveni <un...@gmail.com>.
What about sorting .
Acutually it is done by MapReduce itself.
But if we are giving a csv file as input and trying to sort one/multiple
column...Whether the corresponting columns also get reflectted??
eg: foo.csv
B,2,3
A,4,6
When we apply sorting to first column:whether the resultent will be
A,4,6
B,2,3
A will be mapped to its correct values right?
If so what will be context.write() of mapper?
On Wed, Jan 8, 2014 at 8:18 PM, Chris Mawata <ch...@gmail.com> wrote:
> Yes.
> Check out, for example,
> http://packtlib.packtpub.com/library/hadoop-mapreduce-cookbook/ch06lvl1sec66#
>
>
>
> On 1/8/2014 2:41 AM, unmesha sreeveni wrote:
>
> Can we do aggregation with in Hadoop MR
> like find min,max,sum,avg of a column in a csv file.
>
> --
> *Thanks & Regards*
>
> Unmesha Sreeveni U.B
> Junior Developer
>
> http://www.unmeshasreeveni.blogspot.in/
>
>
>
>
--
*Thanks & Regards*
Unmesha Sreeveni U.B
Junior Developer
http://www.unmeshasreeveni.blogspot.in/
Re: what all can be done using MR
Posted by unmesha sreeveni <un...@gmail.com>.
What about sorting .
Acutually it is done by MapReduce itself.
But if we are giving a csv file as input and trying to sort one/multiple
column...Whether the corresponting columns also get reflectted??
eg: foo.csv
B,2,3
A,4,6
When we apply sorting to first column:whether the resultent will be
A,4,6
B,2,3
A will be mapped to its correct values right?
If so what will be context.write() of mapper?
On Wed, Jan 8, 2014 at 8:18 PM, Chris Mawata <ch...@gmail.com> wrote:
> Yes.
> Check out, for example,
> http://packtlib.packtpub.com/library/hadoop-mapreduce-cookbook/ch06lvl1sec66#
>
>
>
> On 1/8/2014 2:41 AM, unmesha sreeveni wrote:
>
> Can we do aggregation with in Hadoop MR
> like find min,max,sum,avg of a column in a csv file.
>
> --
> *Thanks & Regards*
>
> Unmesha Sreeveni U.B
> Junior Developer
>
> http://www.unmeshasreeveni.blogspot.in/
>
>
>
>
--
*Thanks & Regards*
Unmesha Sreeveni U.B
Junior Developer
http://www.unmeshasreeveni.blogspot.in/
Re: what all can be done using MR
Posted by unmesha sreeveni <un...@gmail.com>.
What about sorting .
Acutually it is done by MapReduce itself.
But if we are giving a csv file as input and trying to sort one/multiple
column...Whether the corresponting columns also get reflectted??
eg: foo.csv
B,2,3
A,4,6
When we apply sorting to first column:whether the resultent will be
A,4,6
B,2,3
A will be mapped to its correct values right?
If so what will be context.write() of mapper?
On Wed, Jan 8, 2014 at 8:18 PM, Chris Mawata <ch...@gmail.com> wrote:
> Yes.
> Check out, for example,
> http://packtlib.packtpub.com/library/hadoop-mapreduce-cookbook/ch06lvl1sec66#
>
>
>
> On 1/8/2014 2:41 AM, unmesha sreeveni wrote:
>
> Can we do aggregation with in Hadoop MR
> like find min,max,sum,avg of a column in a csv file.
>
> --
> *Thanks & Regards*
>
> Unmesha Sreeveni U.B
> Junior Developer
>
> http://www.unmeshasreeveni.blogspot.in/
>
>
>
>
--
*Thanks & Regards*
Unmesha Sreeveni U.B
Junior Developer
http://www.unmeshasreeveni.blogspot.in/
Re: what all can be done using MR
Posted by Chris Mawata <ch...@gmail.com>.
Yes.
Check out, for example,
http://packtlib.packtpub.com/library/hadoop-mapreduce-cookbook/ch06lvl1sec66#
On 1/8/2014 2:41 AM, unmesha sreeveni wrote:
> Can we do aggregation with in Hadoop MR
> like find min,max,sum,avg of a column in a csv file.
>
> --
> /Thanks & Regards/
> /
> /
> Unmesha Sreeveni U.B/
> /
> Junior Developer
>
> http://www.unmeshasreeveni.blogspot.in/
>
> /
> /
Re: what all can be done using MR
Posted by Chris Mawata <ch...@gmail.com>.
Yes.
Check out, for example,
http://packtlib.packtpub.com/library/hadoop-mapreduce-cookbook/ch06lvl1sec66#
On 1/8/2014 2:41 AM, unmesha sreeveni wrote:
> Can we do aggregation with in Hadoop MR
> like find min,max,sum,avg of a column in a csv file.
>
> --
> /Thanks & Regards/
> /
> /
> Unmesha Sreeveni U.B/
> /
> Junior Developer
>
> http://www.unmeshasreeveni.blogspot.in/
>
> /
> /
Re: what all can be done using MR
Posted by Chris Mawata <ch...@gmail.com>.
Yes.
Check out, for example,
http://packtlib.packtpub.com/library/hadoop-mapreduce-cookbook/ch06lvl1sec66#
On 1/8/2014 2:41 AM, unmesha sreeveni wrote:
> Can we do aggregation with in Hadoop MR
> like find min,max,sum,avg of a column in a csv file.
>
> --
> /Thanks & Regards/
> /
> /
> Unmesha Sreeveni U.B/
> /
> Junior Developer
>
> http://www.unmeshasreeveni.blogspot.in/
>
> /
> /
Re: what all can be done using MR
Posted by Chris Mawata <ch...@gmail.com>.
Yes.
Check out, for example,
http://packtlib.packtpub.com/library/hadoop-mapreduce-cookbook/ch06lvl1sec66#
On 1/8/2014 2:41 AM, unmesha sreeveni wrote:
> Can we do aggregation with in Hadoop MR
> like find min,max,sum,avg of a column in a csv file.
>
> --
> /Thanks & Regards/
> /
> /
> Unmesha Sreeveni U.B/
> /
> Junior Developer
>
> http://www.unmeshasreeveni.blogspot.in/
>
> /
> /