You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hama.apache.org by "Edward J. Yoon" <ed...@apache.org> on 2008/12/01 02:17:43 UTC
Re: java.lang.ArrayIndexOutOfBoundsException
Samuel, can you guess why it only appears on multi-node? :)
On Sun, Nov 30, 2008 at 12:20 PM, Edward J. Yoon <ed...@apache.org> wrote:
> I filed this issue - HAMA-116
>
> Seems like bug related with BlockingMapRed.
>
> On Sun, Nov 30, 2008 at 12:02 PM, Edward J. Yoon <ed...@apache.org> wrote:
>> In large scale, Hama may able to beat the scalapack or other math
>> libraries :) BTW, current block multiplication seems have a lot of
>> bug.
>>
>> ----
>> 08/11/30 11:52:15 INFO mapred.JobClient: map 95% reduce 0%
>> 08/11/30 11:52:17 INFO mapred.JobClient: map 96% reduce 0%
>> 08/11/30 11:52:19 INFO mapred.JobClient: map 98% reduce 0%
>> 08/11/30 11:52:20 INFO mapred.JobClient: map 100% reduce 0%
>> 08/11/30 11:52:39 INFO mapred.JobClient: map 100% reduce 1%
>> 08/11/30 11:52:46 INFO mapred.JobClient: map 100% reduce 0%
>> 08/11/30 11:52:46 INFO mapred.JobClient: Task Id : attempt_200811271544_0006_r_0
>> 00000_0, Status : FAILED
>> java.lang.ArrayIndexOutOfBoundsException: 10
>> at org.apache.hama.SubMatrix.set(SubMatrix.java:61)
>> at org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking
>> MapRed.java:146)
>> at org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking
>> MapRed.java:127)
>> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:318)
>> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207
>> )
>>
>> 08/11/30 11:52:48 INFO mapred.JobClient: Task Id : attempt_200811271544_0006_r_0
>> 00002_0, Status : FAILED
>> java.lang.ArrayIndexOutOfBoundsException: 10
>> at org.apache.hama.SubMatrix.set(SubMatrix.java:61)
>> at org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking
>> MapRed.java:146)
>> at org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking
>> MapRed.java:127)
>> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:318)
>> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207
>> )
>>
>> 08/11/30 11:52:50 INFO mapred.JobClient: Task Id : attempt_200811271544_0006_r_0
>> 00001_0, Status : FAILED
>> java.lang.ArrayIndexOutOfBoundsException: 10
>> at org.apache.hama.SubMatrix.set(SubMatrix.java:61)
>> at org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking
>> MapRed.java:146)
>> at org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking
>> MapRed.java:127)
>> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:318)
>> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207
>> )
>>
>> 08/11/30 11:54:02 INFO mapred.JobClient: map 100% reduce 1%
>> 08/11/30 11:54:22 INFO mapred.JobClient: map 100% reduce 0%
>> 08/11/30 11:54:22 INFO mapred.JobClient: Task Id : attempt_200811271544_0006_r_0
>> 00003_0, Status : FAILED
>> java.lang.ArrayIndexOutOfBoundsException: 10
>> at org.apache.hama.SubMatrix.set(SubMatrix.java:61)
>> at org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking
>> MapRed.java:146)
>> at org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking
>> MapRed.java:127)
>> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:318)
>> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207
>> )
>>
>> 08/11/30 11:54:42 INFO mapred.JobClient: map 100% reduce 1%
>> 08/11/30 11:54:51 INFO mapred.JobClient: map 100% reduce 0%
>> 08/11/30 11:54:51 INFO mapred.JobClient: Task Id : attempt_200811271544_0006_r_0
>> 00000_1, Status : FAILED
>> java.lang.ArrayIndexOutOfBoundsException: 10
>> at org.apache.hama.SubMatrix.set(SubMatrix.java:61)
>> at org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking
>> MapRed.java:146)
>> at org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking
>> MapRed.java:127)
>> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:318)
>> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207
>> )
>>
>> --
>> Best Regards, Edward J. Yoon @ NHN, corp.
>> edwardyoon@apache.org
>> http://blog.udanax.org
>>
>
>
>
> --
> Best Regards, Edward J. Yoon @ NHN, corp.
> edwardyoon@apache.org
> http://blog.udanax.org
>
--
Best Regards, Edward J. Yoon @ NHN, corp.
edwardyoon@apache.org
http://blog.udanax.org
Re: java.lang.ArrayIndexOutOfBoundsException
Posted by "Edward J. Yoon" <ed...@apache.org>.
Oh. thanks.
On Mon, Dec 1, 2008 at 3:12 PM, Samuel Guo <gu...@gmail.com> wrote:
> something is wrong when *Reducer* group the Vector by *BlockID*. The bug is
> introduced by me, so I will fix it now and attach a patch.
>
> On Mon, Dec 1, 2008 at 9:17 AM, Edward J. Yoon <ed...@apache.org>wrote:
>
>> Samuel, can you guess why it only appears on multi-node? :)
>>
>> On Sun, Nov 30, 2008 at 12:20 PM, Edward J. Yoon <ed...@apache.org>
>> wrote:
>> > I filed this issue - HAMA-116
>> >
>> > Seems like bug related with BlockingMapRed.
>> >
>> > On Sun, Nov 30, 2008 at 12:02 PM, Edward J. Yoon <ed...@apache.org>
>> wrote:
>> >> In large scale, Hama may able to beat the scalapack or other math
>> >> libraries :) BTW, current block multiplication seems have a lot of
>> >> bug.
>> >>
>> >> ----
>> >> 08/11/30 11:52:15 INFO mapred.JobClient: map 95% reduce 0%
>> >> 08/11/30 11:52:17 INFO mapred.JobClient: map 96% reduce 0%
>> >> 08/11/30 11:52:19 INFO mapred.JobClient: map 98% reduce 0%
>> >> 08/11/30 11:52:20 INFO mapred.JobClient: map 100% reduce 0%
>> >> 08/11/30 11:52:39 INFO mapred.JobClient: map 100% reduce 1%
>> >> 08/11/30 11:52:46 INFO mapred.JobClient: map 100% reduce 0%
>> >> 08/11/30 11:52:46 INFO mapred.JobClient: Task Id :
>> attempt_200811271544_0006_r_0
>> >> 00000_0, Status : FAILED
>> >> java.lang.ArrayIndexOutOfBoundsException: 10
>> >> at org.apache.hama.SubMatrix.set(SubMatrix.java:61)
>> >> at
>> org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking
>> >> MapRed.java:146)
>> >> at
>> org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking
>> >> MapRed.java:127)
>> >> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:318)
>> >> at
>> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207
>> >> )
>> >>
>> >> 08/11/30 11:52:48 INFO mapred.JobClient: Task Id :
>> attempt_200811271544_0006_r_0
>> >> 00002_0, Status : FAILED
>> >> java.lang.ArrayIndexOutOfBoundsException: 10
>> >> at org.apache.hama.SubMatrix.set(SubMatrix.java:61)
>> >> at
>> org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking
>> >> MapRed.java:146)
>> >> at
>> org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking
>> >> MapRed.java:127)
>> >> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:318)
>> >> at
>> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207
>> >> )
>> >>
>> >> 08/11/30 11:52:50 INFO mapred.JobClient: Task Id :
>> attempt_200811271544_0006_r_0
>> >> 00001_0, Status : FAILED
>> >> java.lang.ArrayIndexOutOfBoundsException: 10
>> >> at org.apache.hama.SubMatrix.set(SubMatrix.java:61)
>> >> at
>> org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking
>> >> MapRed.java:146)
>> >> at
>> org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking
>> >> MapRed.java:127)
>> >> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:318)
>> >> at
>> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207
>> >> )
>> >>
>> >> 08/11/30 11:54:02 INFO mapred.JobClient: map 100% reduce 1%
>> >> 08/11/30 11:54:22 INFO mapred.JobClient: map 100% reduce 0%
>> >> 08/11/30 11:54:22 INFO mapred.JobClient: Task Id :
>> attempt_200811271544_0006_r_0
>> >> 00003_0, Status : FAILED
>> >> java.lang.ArrayIndexOutOfBoundsException: 10
>> >> at org.apache.hama.SubMatrix.set(SubMatrix.java:61)
>> >> at
>> org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking
>> >> MapRed.java:146)
>> >> at
>> org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking
>> >> MapRed.java:127)
>> >> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:318)
>> >> at
>> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207
>> >> )
>> >>
>> >> 08/11/30 11:54:42 INFO mapred.JobClient: map 100% reduce 1%
>> >> 08/11/30 11:54:51 INFO mapred.JobClient: map 100% reduce 0%
>> >> 08/11/30 11:54:51 INFO mapred.JobClient: Task Id :
>> attempt_200811271544_0006_r_0
>> >> 00000_1, Status : FAILED
>> >> java.lang.ArrayIndexOutOfBoundsException: 10
>> >> at org.apache.hama.SubMatrix.set(SubMatrix.java:61)
>> >> at
>> org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking
>> >> MapRed.java:146)
>> >> at
>> org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking
>> >> MapRed.java:127)
>> >> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:318)
>> >> at
>> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207
>> >> )
>> >>
>> >> --
>> >> Best Regards, Edward J. Yoon @ NHN, corp.
>> >> edwardyoon@apache.org
>> >> http://blog.udanax.org
>> >>
>> >
>> >
>> >
>> > --
>> > Best Regards, Edward J. Yoon @ NHN, corp.
>> > edwardyoon@apache.org
>> > http://blog.udanax.org
>> >
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon @ NHN, corp.
>> edwardyoon@apache.org
>> http://blog.udanax.org
>>
>
--
Best Regards, Edward J. Yoon @ NHN, corp.
edwardyoon@apache.org
http://blog.udanax.org
Re: java.lang.ArrayIndexOutOfBoundsException
Posted by Samuel Guo <gu...@gmail.com>.
something is wrong when *Reducer* group the Vector by *BlockID*. The bug is
introduced by me, so I will fix it now and attach a patch.
On Mon, Dec 1, 2008 at 9:17 AM, Edward J. Yoon <ed...@apache.org>wrote:
> Samuel, can you guess why it only appears on multi-node? :)
>
> On Sun, Nov 30, 2008 at 12:20 PM, Edward J. Yoon <ed...@apache.org>
> wrote:
> > I filed this issue - HAMA-116
> >
> > Seems like bug related with BlockingMapRed.
> >
> > On Sun, Nov 30, 2008 at 12:02 PM, Edward J. Yoon <ed...@apache.org>
> wrote:
> >> In large scale, Hama may able to beat the scalapack or other math
> >> libraries :) BTW, current block multiplication seems have a lot of
> >> bug.
> >>
> >> ----
> >> 08/11/30 11:52:15 INFO mapred.JobClient: map 95% reduce 0%
> >> 08/11/30 11:52:17 INFO mapred.JobClient: map 96% reduce 0%
> >> 08/11/30 11:52:19 INFO mapred.JobClient: map 98% reduce 0%
> >> 08/11/30 11:52:20 INFO mapred.JobClient: map 100% reduce 0%
> >> 08/11/30 11:52:39 INFO mapred.JobClient: map 100% reduce 1%
> >> 08/11/30 11:52:46 INFO mapred.JobClient: map 100% reduce 0%
> >> 08/11/30 11:52:46 INFO mapred.JobClient: Task Id :
> attempt_200811271544_0006_r_0
> >> 00000_0, Status : FAILED
> >> java.lang.ArrayIndexOutOfBoundsException: 10
> >> at org.apache.hama.SubMatrix.set(SubMatrix.java:61)
> >> at
> org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking
> >> MapRed.java:146)
> >> at
> org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking
> >> MapRed.java:127)
> >> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:318)
> >> at
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207
> >> )
> >>
> >> 08/11/30 11:52:48 INFO mapred.JobClient: Task Id :
> attempt_200811271544_0006_r_0
> >> 00002_0, Status : FAILED
> >> java.lang.ArrayIndexOutOfBoundsException: 10
> >> at org.apache.hama.SubMatrix.set(SubMatrix.java:61)
> >> at
> org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking
> >> MapRed.java:146)
> >> at
> org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking
> >> MapRed.java:127)
> >> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:318)
> >> at
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207
> >> )
> >>
> >> 08/11/30 11:52:50 INFO mapred.JobClient: Task Id :
> attempt_200811271544_0006_r_0
> >> 00001_0, Status : FAILED
> >> java.lang.ArrayIndexOutOfBoundsException: 10
> >> at org.apache.hama.SubMatrix.set(SubMatrix.java:61)
> >> at
> org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking
> >> MapRed.java:146)
> >> at
> org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking
> >> MapRed.java:127)
> >> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:318)
> >> at
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207
> >> )
> >>
> >> 08/11/30 11:54:02 INFO mapred.JobClient: map 100% reduce 1%
> >> 08/11/30 11:54:22 INFO mapred.JobClient: map 100% reduce 0%
> >> 08/11/30 11:54:22 INFO mapred.JobClient: Task Id :
> attempt_200811271544_0006_r_0
> >> 00003_0, Status : FAILED
> >> java.lang.ArrayIndexOutOfBoundsException: 10
> >> at org.apache.hama.SubMatrix.set(SubMatrix.java:61)
> >> at
> org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking
> >> MapRed.java:146)
> >> at
> org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking
> >> MapRed.java:127)
> >> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:318)
> >> at
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207
> >> )
> >>
> >> 08/11/30 11:54:42 INFO mapred.JobClient: map 100% reduce 1%
> >> 08/11/30 11:54:51 INFO mapred.JobClient: map 100% reduce 0%
> >> 08/11/30 11:54:51 INFO mapred.JobClient: Task Id :
> attempt_200811271544_0006_r_0
> >> 00000_1, Status : FAILED
> >> java.lang.ArrayIndexOutOfBoundsException: 10
> >> at org.apache.hama.SubMatrix.set(SubMatrix.java:61)
> >> at
> org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking
> >> MapRed.java:146)
> >> at
> org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking
> >> MapRed.java:127)
> >> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:318)
> >> at
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207
> >> )
> >>
> >> --
> >> Best Regards, Edward J. Yoon @ NHN, corp.
> >> edwardyoon@apache.org
> >> http://blog.udanax.org
> >>
> >
> >
> >
> > --
> > Best Regards, Edward J. Yoon @ NHN, corp.
> > edwardyoon@apache.org
> > http://blog.udanax.org
> >
>
>
>
> --
> Best Regards, Edward J. Yoon @ NHN, corp.
> edwardyoon@apache.org
> http://blog.udanax.org
>