You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Thamizhannal Paramasivam <th...@gmail.com> on 2013/05/08 17:10:05 UTC
Set on reducer issue
Hi,
I require clarification on using IntWritable in Set type in Java.
Below is my sample code. When I execute reducer with E.x. key="abcd"
values=12,3,4, then categorySet have any one value from values.
My hadoop version is 0.20.2. Any help would be appreciated.
public void reduce(Text key, Iterable<IntWritable> values,
Context context) throws IOException, InterruptedException {
Set<IntWritable> categorySet = new TreeSet<IntWritable>();
for (IntWritable val : values) {
categorySet.add(val);
}
----------------
----------------
}
Thanks,
Thamizh
Re: Set on reducer issue
Posted by Thamizhannal Paramasivam <th...@gmail.com>.
Thanks a lot Paul. It sorted out my issue.
Because of this issue, I was enforced to use Set<Integer> instead of
Set<IntWritable>.
On Wed, May 8, 2013 at 8:59 PM, Paul Wilkinson
<pa...@gmail.com>wrote:
> See
> http://cornercases.wordpress.com/2011/08/18/hadoop-object-reuse-pitfall-all-my-reducer-values-are-the-same/for an explanation of what's going on.
>
> Paul
>
>
> On 8 May 2013 16:10, Thamizhannal Paramasivam <th...@gmail.com>wrote:
>
>> Hi,
>> I require clarification on using IntWritable in Set type in Java.
>> Below is my sample code. When I execute reducer with E.x. key="abcd"
>> values=12,3,4, then categorySet have any one value from values.
>>
>> My hadoop version is 0.20.2. Any help would be appreciated.
>>
>> public void reduce(Text key, Iterable<IntWritable> values,
>> Context context) throws IOException, InterruptedException
>> {
>>
>> Set<IntWritable> categorySet = new TreeSet<IntWritable>();
>> for (IntWritable val : values) {
>> categorySet.add(val);
>> }
>> ----------------
>> ----------------
>> }
>>
>> Thanks,
>> Thamizh
>>
>
>
Re: Set on reducer issue
Posted by Thamizhannal Paramasivam <th...@gmail.com>.
Thanks a lot Paul. It sorted out my issue.
Because of this issue, I was enforced to use Set<Integer> instead of
Set<IntWritable>.
On Wed, May 8, 2013 at 8:59 PM, Paul Wilkinson
<pa...@gmail.com>wrote:
> See
> http://cornercases.wordpress.com/2011/08/18/hadoop-object-reuse-pitfall-all-my-reducer-values-are-the-same/for an explanation of what's going on.
>
> Paul
>
>
> On 8 May 2013 16:10, Thamizhannal Paramasivam <th...@gmail.com>wrote:
>
>> Hi,
>> I require clarification on using IntWritable in Set type in Java.
>> Below is my sample code. When I execute reducer with E.x. key="abcd"
>> values=12,3,4, then categorySet have any one value from values.
>>
>> My hadoop version is 0.20.2. Any help would be appreciated.
>>
>> public void reduce(Text key, Iterable<IntWritable> values,
>> Context context) throws IOException, InterruptedException
>> {
>>
>> Set<IntWritable> categorySet = new TreeSet<IntWritable>();
>> for (IntWritable val : values) {
>> categorySet.add(val);
>> }
>> ----------------
>> ----------------
>> }
>>
>> Thanks,
>> Thamizh
>>
>
>
Re: Set on reducer issue
Posted by Thamizhannal Paramasivam <th...@gmail.com>.
Thanks a lot Paul. It sorted out my issue.
Because of this issue, I was enforced to use Set<Integer> instead of
Set<IntWritable>.
On Wed, May 8, 2013 at 8:59 PM, Paul Wilkinson
<pa...@gmail.com>wrote:
> See
> http://cornercases.wordpress.com/2011/08/18/hadoop-object-reuse-pitfall-all-my-reducer-values-are-the-same/for an explanation of what's going on.
>
> Paul
>
>
> On 8 May 2013 16:10, Thamizhannal Paramasivam <th...@gmail.com>wrote:
>
>> Hi,
>> I require clarification on using IntWritable in Set type in Java.
>> Below is my sample code. When I execute reducer with E.x. key="abcd"
>> values=12,3,4, then categorySet have any one value from values.
>>
>> My hadoop version is 0.20.2. Any help would be appreciated.
>>
>> public void reduce(Text key, Iterable<IntWritable> values,
>> Context context) throws IOException, InterruptedException
>> {
>>
>> Set<IntWritable> categorySet = new TreeSet<IntWritable>();
>> for (IntWritable val : values) {
>> categorySet.add(val);
>> }
>> ----------------
>> ----------------
>> }
>>
>> Thanks,
>> Thamizh
>>
>
>
Re: Set on reducer issue
Posted by Thamizhannal Paramasivam <th...@gmail.com>.
Thanks a lot Paul. It sorted out my issue.
Because of this issue, I was enforced to use Set<Integer> instead of
Set<IntWritable>.
On Wed, May 8, 2013 at 8:59 PM, Paul Wilkinson
<pa...@gmail.com>wrote:
> See
> http://cornercases.wordpress.com/2011/08/18/hadoop-object-reuse-pitfall-all-my-reducer-values-are-the-same/for an explanation of what's going on.
>
> Paul
>
>
> On 8 May 2013 16:10, Thamizhannal Paramasivam <th...@gmail.com>wrote:
>
>> Hi,
>> I require clarification on using IntWritable in Set type in Java.
>> Below is my sample code. When I execute reducer with E.x. key="abcd"
>> values=12,3,4, then categorySet have any one value from values.
>>
>> My hadoop version is 0.20.2. Any help would be appreciated.
>>
>> public void reduce(Text key, Iterable<IntWritable> values,
>> Context context) throws IOException, InterruptedException
>> {
>>
>> Set<IntWritable> categorySet = new TreeSet<IntWritable>();
>> for (IntWritable val : values) {
>> categorySet.add(val);
>> }
>> ----------------
>> ----------------
>> }
>>
>> Thanks,
>> Thamizh
>>
>
>
Re: Set on reducer issue
Posted by Paul Wilkinson <pa...@gmail.com>.
See
http://cornercases.wordpress.com/2011/08/18/hadoop-object-reuse-pitfall-all-my-reducer-values-are-the-same/for
an explanation of what's going on.
Paul
On 8 May 2013 16:10, Thamizhannal Paramasivam <th...@gmail.com>wrote:
> Hi,
> I require clarification on using IntWritable in Set type in Java.
> Below is my sample code. When I execute reducer with E.x. key="abcd"
> values=12,3,4, then categorySet have any one value from values.
>
> My hadoop version is 0.20.2. Any help would be appreciated.
>
> public void reduce(Text key, Iterable<IntWritable> values,
> Context context) throws IOException, InterruptedException {
>
> Set<IntWritable> categorySet = new TreeSet<IntWritable>();
> for (IntWritable val : values) {
> categorySet.add(val);
> }
> ----------------
> ----------------
> }
>
> Thanks,
> Thamizh
>
Re: Set on reducer issue
Posted by Paul Wilkinson <pa...@gmail.com>.
See
http://cornercases.wordpress.com/2011/08/18/hadoop-object-reuse-pitfall-all-my-reducer-values-are-the-same/for
an explanation of what's going on.
Paul
On 8 May 2013 16:10, Thamizhannal Paramasivam <th...@gmail.com>wrote:
> Hi,
> I require clarification on using IntWritable in Set type in Java.
> Below is my sample code. When I execute reducer with E.x. key="abcd"
> values=12,3,4, then categorySet have any one value from values.
>
> My hadoop version is 0.20.2. Any help would be appreciated.
>
> public void reduce(Text key, Iterable<IntWritable> values,
> Context context) throws IOException, InterruptedException {
>
> Set<IntWritable> categorySet = new TreeSet<IntWritable>();
> for (IntWritable val : values) {
> categorySet.add(val);
> }
> ----------------
> ----------------
> }
>
> Thanks,
> Thamizh
>
Re: Set on reducer issue
Posted by Paul Wilkinson <pa...@gmail.com>.
See
http://cornercases.wordpress.com/2011/08/18/hadoop-object-reuse-pitfall-all-my-reducer-values-are-the-same/for
an explanation of what's going on.
Paul
On 8 May 2013 16:10, Thamizhannal Paramasivam <th...@gmail.com>wrote:
> Hi,
> I require clarification on using IntWritable in Set type in Java.
> Below is my sample code. When I execute reducer with E.x. key="abcd"
> values=12,3,4, then categorySet have any one value from values.
>
> My hadoop version is 0.20.2. Any help would be appreciated.
>
> public void reduce(Text key, Iterable<IntWritable> values,
> Context context) throws IOException, InterruptedException {
>
> Set<IntWritable> categorySet = new TreeSet<IntWritable>();
> for (IntWritable val : values) {
> categorySet.add(val);
> }
> ----------------
> ----------------
> }
>
> Thanks,
> Thamizh
>
Re: Set on reducer issue
Posted by Paul Wilkinson <pa...@gmail.com>.
See
http://cornercases.wordpress.com/2011/08/18/hadoop-object-reuse-pitfall-all-my-reducer-values-are-the-same/for
an explanation of what's going on.
Paul
On 8 May 2013 16:10, Thamizhannal Paramasivam <th...@gmail.com>wrote:
> Hi,
> I require clarification on using IntWritable in Set type in Java.
> Below is my sample code. When I execute reducer with E.x. key="abcd"
> values=12,3,4, then categorySet have any one value from values.
>
> My hadoop version is 0.20.2. Any help would be appreciated.
>
> public void reduce(Text key, Iterable<IntWritable> values,
> Context context) throws IOException, InterruptedException {
>
> Set<IntWritable> categorySet = new TreeSet<IntWritable>();
> for (IntWritable val : values) {
> categorySet.add(val);
> }
> ----------------
> ----------------
> }
>
> Thanks,
> Thamizh
>