You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by bharath vissapragada <bh...@students.iiit.ac.in> on 2010/01/02 08:29:56 UTC

error while using ArrayWritable

Hi all ,

I am using ArrayWritable in my MR job .

Map outputs <Text ,ArrayWritable>

Reduce takes <Text,Iterable<ArrayWritable>>

The moment Iam trying to use the ArrayWritable in reduce using the iterator
..I get the following error :

10/01/02 18:23:41 WARN mapred.LocalJobRunner: job_local_0001
java.lang.RuntimeException: java.lang.NoSuchMethodException:
org.apache.hadoop.io.ArrayWritable.<init>()
    at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:81)
    at
org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:62)
    at
org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:40)
    at
org.apache.hadoop.mapred.Task$ValuesIterator.readNextValue(Task.java:940)
    at org.apache.hadoop.mapred.Task$ValuesIterator.next(Task.java:880)
    at
org.apache.hadoop.mapred.ReduceTask$ReduceValuesIterator.moveToNext(ReduceTask.java:237)
    at
org.apache.hadoop.mapred.ReduceTask$ReduceValuesIterator.next(ReduceTask.java:233)
    at HashJoin.MR_hash$redu.reduce(MR_hash.java:132)
    at mtr.MyTableReduce.reduce(MyTableReduce.java:1)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:436)
    at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:170)
Caused by: java.lang.NoSuchMethodException:
org.apache.hadoop.io.ArrayWritable.<init>()
    at java.lang.Class.getConstructor0(Class.java:2723)
    at java.lang.Class.getDeclaredConstructor(Class.java:2002)
    at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:75)
    ... 10 more
java.io.IOException: Job failed!
    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1232)
    at HashJoin.MR_hash.run(MR_hash.java:294)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at HashJoin.MR_hash.join(MR_hash.java:320)
    at HashJoin.MR_hash.main(MR_hash.java:338)


Can Anyone tell me what the error is ??

Re: error while using ArrayWritable

Posted by bharath v <bh...@gmail.com>.

Thanks J-D , worked perfectly fine !

On Sat, Jan 2, 2010 at 1:16 PM, Jean-Daniel Cryans <jd...@apache.org>wrote:

> This is explained in the javadoc:
>
>
> http://hadoop.apache.org/common/docs/r0.20.1/api/org/apache/hadoop/io/ArrayWritable.html
>
> J-D
>
> On Fri, Jan 1, 2010 at 11:29 PM, bharath vissapragada
> <bh...@students.iiit.ac.in> wrote:
> > Hi all ,
> >
> > I am using ArrayWritable in my MR job .
> >
> > Map outputs <Text ,ArrayWritable>
> >
> > Reduce takes <Text,Iterable<ArrayWritable>>
> >
> > The moment Iam trying to use the ArrayWritable in reduce using the
> iterator
> > ..I get the following error :
> >
> > 10/01/02 18:23:41 WARN mapred.LocalJobRunner: job_local_0001
> > java.lang.RuntimeException: java.lang.NoSuchMethodException:
> > org.apache.hadoop.io.ArrayWritable.<init>()
> >    at
> >
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:81)
> >    at
> >
> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:62)
> >    at
> >
> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:40)
> >    at
> > org.apache.hadoop.mapred.Task$ValuesIterator.readNextValue(Task.java:940)
> >    at org.apache.hadoop.mapred.Task$ValuesIterator.next(Task.java:880)
> >    at
> >
> org.apache.hadoop.mapred.ReduceTask$ReduceValuesIterator.moveToNext(ReduceTask.java:237)
> >    at
> >
> org.apache.hadoop.mapred.ReduceTask$ReduceValuesIterator.next(ReduceTask.java:233)
> >    at HashJoin.MR_hash$redu.reduce(MR_hash.java:132)
> >    at mtr.MyTableReduce.reduce(MyTableReduce.java:1)
> >    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:436)
> >    at
> > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:170)
> > Caused by: java.lang.NoSuchMethodException:
> > org.apache.hadoop.io.ArrayWritable.<init>()
> >    at java.lang.Class.getConstructor0(Class.java:2723)
> >    at java.lang.Class.getDeclaredConstructor(Class.java:2002)
> >    at
> >
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:75)
> >    ... 10 more
> > java.io.IOException: Job failed!
> >    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1232)
> >    at HashJoin.MR_hash.run(MR_hash.java:294)
> >    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> >    at HashJoin.MR_hash.join(MR_hash.java:320)
> >    at HashJoin.MR_hash.main(MR_hash.java:338)
> >
> >
> > Can Anyone tell me what the error is ??
> >
>

Re: error while using ArrayWritable

Posted by Jean-Daniel Cryans <jd...@apache.org>.

This is explained in the javadoc:

http://hadoop.apache.org/common/docs/r0.20.1/api/org/apache/hadoop/io/ArrayWritable.html

J-D

On Fri, Jan 1, 2010 at 11:29 PM, bharath vissapragada
<bh...@students.iiit.ac.in> wrote:
> Hi all ,
>
> I am using ArrayWritable in my MR job .
>
> Map outputs <Text ,ArrayWritable>
>
> Reduce takes <Text,Iterable<ArrayWritable>>
>
> The moment Iam trying to use the ArrayWritable in reduce using the iterator
> ..I get the following error :
>
> 10/01/02 18:23:41 WARN mapred.LocalJobRunner: job_local_0001
> java.lang.RuntimeException: java.lang.NoSuchMethodException:
> org.apache.hadoop.io.ArrayWritable.<init>()
>    at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:81)
>    at
> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:62)
>    at
> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:40)
>    at
> org.apache.hadoop.mapred.Task$ValuesIterator.readNextValue(Task.java:940)
>    at org.apache.hadoop.mapred.Task$ValuesIterator.next(Task.java:880)
>    at
> org.apache.hadoop.mapred.ReduceTask$ReduceValuesIterator.moveToNext(ReduceTask.java:237)
>    at
> org.apache.hadoop.mapred.ReduceTask$ReduceValuesIterator.next(ReduceTask.java:233)
>    at HashJoin.MR_hash$redu.reduce(MR_hash.java:132)
>    at mtr.MyTableReduce.reduce(MyTableReduce.java:1)
>    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:436)
>    at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:170)
> Caused by: java.lang.NoSuchMethodException:
> org.apache.hadoop.io.ArrayWritable.<init>()
>    at java.lang.Class.getConstructor0(Class.java:2723)
>    at java.lang.Class.getDeclaredConstructor(Class.java:2002)
>    at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:75)
>    ... 10 more
> java.io.IOException: Job failed!
>    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1232)
>    at HashJoin.MR_hash.run(MR_hash.java:294)
>    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>    at HashJoin.MR_hash.join(MR_hash.java:320)
>    at HashJoin.MR_hash.main(MR_hash.java:338)
>
>
> Can Anyone tell me what the error is ??
>