You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hama.apache.org by "Edward J. Yoon" <ed...@apache.org> on 2008/12/04 08:09:53 UTC
NullPointerException at AbstractVector.get(AbstractVector.java:44)
Can someone figure out this problem?
08/12/04 15:58:37 INFO hama.AbstractMatrix: Convert to 20 * 20 blocked matrix
08/12/04 15:58:37 WARN mapred.JobClient: Use GenericOptionsParser for
parsing the arguments. Applications should implement Tool for the
same.
08/12/04 15:58:37 WARN mapred.JobClient: Use genericOptions for the
option -libjars
08/12/04 15:58:38 WARN mapred.JobClient: No job jar file set. User
classes may not be found. See JobConf(Class) or
JobConf#setJar(String).
08/12/04 15:58:38 INFO mapred.JobClient: Running job: job_200812041310_0026
08/12/04 15:58:39 INFO mapred.JobClient: map 0% reduce 0%
08/12/04 15:59:29 INFO mapred.JobClient: Task Id :
attempt_200812041310_0026_m_000002_0, Status : FAILED
java.lang.NullPointerException: v(1) : java.lang.NullPointerException
at org.apache.hama.AbstractVector.get(AbstractVector.java:44)
at org.apache.hama.DenseVector.subVector(DenseVector.java:151)
at org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:112)
at org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:98)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
--
Best Regards, Edward J. Yoon @ NHN, corp.
edwardyoon@apache.org
http://blog.udanax.org
Re: NullPointerException at AbstractVector.get(AbstractVector.java:44)
Posted by Samuel Guo <gu...@gmail.com>.
Yes. I think it should be the number of BlockIDs. It is strange.
And the input records = 7556. I think it is the number of rows. Because we
block it 10 blocks, so we will got 10 times Map Input Records in Map Output/
Reduce Input stage. so Map output records = 75560 = 7556 * 10.
But why we have 7556 rows? I guess the size of the matrix in your test is
4000 or 1000.
On Mon, Dec 8, 2008 at 2:08 PM, Edward J. Yoon <ed...@apache.org>wrote:
> Somtimes, job done successfully. BTW, Reduce input groups=110. I guess
> this is a number of BlockIDs. Should it be a 10 * 10 = 100?
>
> ----
> 08/12/08 14:46:52 INFO hama.AbstractMatrix: Convert to 10 * 10 blocked
> matrix
> ...
> 08/12/08 14:52:27 INFO mapred.JobClient: Local bytes written=2662743172
> 08/12/08 14:52:27 INFO mapred.JobClient: Job Counters
> 08/12/08 14:52:27 INFO mapred.JobClient: Launched reduce tasks=2
> 08/12/08 14:52:27 INFO mapred.JobClient: Launched map tasks=13
> 08/12/08 14:52:27 INFO mapred.JobClient: Map-Reduce Framework
> 08/12/08 14:52:27 INFO mapred.JobClient: Reduce input groups=110
> 08/12/08 14:52:27 INFO mapred.JobClient: Combine output records=0
> 08/12/08 14:52:27 INFO mapred.JobClient: Map input records=7556
> 08/12/08 14:52:27 INFO mapred.JobClient: Reduce output records=0
> 08/12/08 14:52:27 INFO mapred.JobClient: Map output bytes=1020493900
> 08/12/08 14:52:27 INFO mapred.JobClient: Map input bytes=0
> 08/12/08 14:52:27 INFO mapred.JobClient: Combine input records=0
> 08/12/08 14:52:27 INFO mapred.JobClient: Map output records=75560
> 08/12/08 14:52:27 INFO mapred.JobClient: Reduce input records=75560
>
> On Mon, Dec 8, 2008 at 2:31 PM, Samuel Guo <gu...@gmail.com> wrote:
> > It is strange.
> > Let's review the code again to find what's wrong.
> >
> > On Mon, Dec 8, 2008 at 12:26 PM, Edward J. Yoon <edwardyoon@apache.org
> >wrote:
> >
> >> [d8g053:/root/hama-trunk]# bin/hama examples mult -m 10 -r 10 fA fB
> >> Size : 4,000
> >> block_num : 10
> >>
> >> [d8g053:/root/hama-trunk]# bin/hama examples random 4000 4000 fB
> >> ...
> >> [d8g053:/root/hama-trunk]# bin/hama examples mult -m 10 -r 10 fA fB
> >>
> >> 08/12/08 13:23:57 INFO hama.AbstractMatrix: Convert to 10 * 10 blocked
> >> matrix
> >> 08/12/08 13:23:57 WARN mapred.JobClient: Use GenericOptionsParser for
> >> parsing the arguments. Applications should implement Tool for the
> >> same.
> >> 08/12/08 13:23:57 WARN mapred.JobClient: Use genericOptions for the
> >> option -libjars
> >> 08/12/08 13:23:57 WARN mapred.JobClient: No job jar file set. User
> >> classes may not be found. See JobConf(Class) or
> >> JobConf#setJar(String).
> >> 08/12/08 13:23:57 INFO mapred.JobClient: Running job:
> job_200812051425_0027
> >> 08/12/08 13:23:58 INFO mapred.JobClient: map 0% reduce 0%
> >> 08/12/08 13:24:29 INFO mapred.JobClient: Task Id :
> >> attempt_200812051425_0027_m_000001_0, Status : FAILED
> >> java.lang.NullPointerException: v(1) : java.lang.NullPointerException
> >> at org.apache.hama.AbstractVector.get(AbstractVector.java:44)
> >> at org.apache.hama.DenseVector.subVector(DenseVector.java:151)
> >> at
> >>
> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:112)
> >> at
> >>
> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:98)
> >> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> >> at
> >> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
> >>
> >>
> >>
> >> On Thu, Dec 4, 2008 at 11:30 PM, Samuel Guo <gu...@gmail.com> wrote:
> >> > what is the matrix's size?
> >> > what is the block's size?
> >> > can the matrix's size be interge-divided by block's size?
> >> > if not, the last sub-vector's index range will be out of bound.
> >> >
> >> > so, the get() method in AbstractVector will get *Null* when calling
> >> > *this.entries.get(index)*. The code is as below.
> >> >
> >> > public double get(int index) throws NullPointerException {
> >> > return this.entries.get(index).getValue();
> >> > }
> >> >
> >> > It is a bug. A Jira may be filed to fix two files of the code.
> >> >
> >> > first:
> >> > Method *setBlockPosition* in DenseMatrix class;
> >> > second:
> >> > Class *BlockingMapper* in BlockingMapRed class;
> >> >
> >> > I am sorry. :-(
> >> > I have found the potential bug in DenseMatrix when I write the
> >> > BlockingMapRed class. But I forget to fix it and use the same way to
> >> > caculate the block ranges during mapping.
> >> >
> >> > On Thu, Dec 4, 2008 at 3:09 PM, Edward J. Yoon <edwardyoon@apache.org
> >> >wrote:
> >> >
> >> >> Can someone figure out this problem?
> >> >>
> >> >> 08/12/04 15:58:37 INFO hama.AbstractMatrix: Convert to 20 * 20
> blocked
> >> >> matrix
> >> >> 08/12/04 15:58:37 WARN mapred.JobClient: Use GenericOptionsParser for
> >> >> parsing the arguments. Applications should implement Tool for the
> >> >> same.
> >> >> 08/12/04 15:58:37 WARN mapred.JobClient: Use genericOptions for the
> >> >> option -libjars
> >> >> 08/12/04 15:58:38 WARN mapred.JobClient: No job jar file set. User
> >> >> classes may not be found. See JobConf(Class) or
> >> >> JobConf#setJar(String).
> >> >> 08/12/04 15:58:38 INFO mapred.JobClient: Running job:
> >> job_200812041310_0026
> >> >> 08/12/04 15:58:39 INFO mapred.JobClient: map 0% reduce 0%
> >> >> 08/12/04 15:59:29 INFO mapred.JobClient: Task Id :
> >> >> attempt_200812041310_0026_m_000002_0, Status : FAILED
> >> >> java.lang.NullPointerException: v(1) : java.lang.NullPointerException
> >> >> at org.apache.hama.AbstractVector.get(AbstractVector.java:44)
> >> >> at org.apache.hama.DenseVector.subVector(DenseVector.java:151)
> >> >> at
> >> >>
> >>
> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:112)
> >> >> at
> >> >>
> >>
> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:98)
> >> >> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> >> >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> >> >> at
> >> >>
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
> >> >> --
> >> >> Best Regards, Edward J. Yoon @ NHN, corp.
> >> >> edwardyoon@apache.org
> >> >> http://blog.udanax.org
> >> >>
> >> >
> >>
> >>
> >>
> >> --
> >> Best Regards, Edward J. Yoon @ NHN, corp.
> >> edwardyoon@apache.org
> >> http://blog.udanax.org
> >>
> >
>
>
>
> --
> Best Regards, Edward J. Yoon @ NHN, corp.
> edwardyoon@apache.org
> http://blog.udanax.org
>
Re: NullPointerException at AbstractVector.get(AbstractVector.java:44)
Posted by "Edward J. Yoon" <ed...@apache.org>.
This Issue addressed by HAMA-127.
And, "Task Id : attempt_200812081428_0020_m_000004_1, Status : FAILED"
seems a problem of Hbase/Hadoop.
On Mon, Dec 8, 2008 at 4:00 PM, Edward J. Yoon <ed...@apache.org> wrote:
> In random_mapred(), IntWritable(m) should be IntWritable(m-1) since
> start from 0.
>
> On Mon, Dec 8, 2008 at 3:08 PM, Edward J. Yoon <ed...@apache.org> wrote:
>> Somtimes, job done successfully. BTW, Reduce input groups=110. I guess
>> this is a number of BlockIDs. Should it be a 10 * 10 = 100?
>>
>> ----
>> 08/12/08 14:46:52 INFO hama.AbstractMatrix: Convert to 10 * 10 blocked matrix
>> ...
>> 08/12/08 14:52:27 INFO mapred.JobClient: Local bytes written=2662743172
>> 08/12/08 14:52:27 INFO mapred.JobClient: Job Counters
>> 08/12/08 14:52:27 INFO mapred.JobClient: Launched reduce tasks=2
>> 08/12/08 14:52:27 INFO mapred.JobClient: Launched map tasks=13
>> 08/12/08 14:52:27 INFO mapred.JobClient: Map-Reduce Framework
>> 08/12/08 14:52:27 INFO mapred.JobClient: Reduce input groups=110
>> 08/12/08 14:52:27 INFO mapred.JobClient: Combine output records=0
>> 08/12/08 14:52:27 INFO mapred.JobClient: Map input records=7556
>> 08/12/08 14:52:27 INFO mapred.JobClient: Reduce output records=0
>> 08/12/08 14:52:27 INFO mapred.JobClient: Map output bytes=1020493900
>> 08/12/08 14:52:27 INFO mapred.JobClient: Map input bytes=0
>> 08/12/08 14:52:27 INFO mapred.JobClient: Combine input records=0
>> 08/12/08 14:52:27 INFO mapred.JobClient: Map output records=75560
>> 08/12/08 14:52:27 INFO mapred.JobClient: Reduce input records=75560
>>
>> On Mon, Dec 8, 2008 at 2:31 PM, Samuel Guo <gu...@gmail.com> wrote:
>>> It is strange.
>>> Let's review the code again to find what's wrong.
>>>
>>> On Mon, Dec 8, 2008 at 12:26 PM, Edward J. Yoon <ed...@apache.org>wrote:
>>>
>>>> [d8g053:/root/hama-trunk]# bin/hama examples mult -m 10 -r 10 fA fB
>>>> Size : 4,000
>>>> block_num : 10
>>>>
>>>> [d8g053:/root/hama-trunk]# bin/hama examples random 4000 4000 fB
>>>> ...
>>>> [d8g053:/root/hama-trunk]# bin/hama examples mult -m 10 -r 10 fA fB
>>>>
>>>> 08/12/08 13:23:57 INFO hama.AbstractMatrix: Convert to 10 * 10 blocked
>>>> matrix
>>>> 08/12/08 13:23:57 WARN mapred.JobClient: Use GenericOptionsParser for
>>>> parsing the arguments. Applications should implement Tool for the
>>>> same.
>>>> 08/12/08 13:23:57 WARN mapred.JobClient: Use genericOptions for the
>>>> option -libjars
>>>> 08/12/08 13:23:57 WARN mapred.JobClient: No job jar file set. User
>>>> classes may not be found. See JobConf(Class) or
>>>> JobConf#setJar(String).
>>>> 08/12/08 13:23:57 INFO mapred.JobClient: Running job: job_200812051425_0027
>>>> 08/12/08 13:23:58 INFO mapred.JobClient: map 0% reduce 0%
>>>> 08/12/08 13:24:29 INFO mapred.JobClient: Task Id :
>>>> attempt_200812051425_0027_m_000001_0, Status : FAILED
>>>> java.lang.NullPointerException: v(1) : java.lang.NullPointerException
>>>> at org.apache.hama.AbstractVector.get(AbstractVector.java:44)
>>>> at org.apache.hama.DenseVector.subVector(DenseVector.java:151)
>>>> at
>>>> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:112)
>>>> at
>>>> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:98)
>>>> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
>>>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
>>>> at
>>>> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
>>>>
>>>>
>>>>
>>>> On Thu, Dec 4, 2008 at 11:30 PM, Samuel Guo <gu...@gmail.com> wrote:
>>>> > what is the matrix's size?
>>>> > what is the block's size?
>>>> > can the matrix's size be interge-divided by block's size?
>>>> > if not, the last sub-vector's index range will be out of bound.
>>>> >
>>>> > so, the get() method in AbstractVector will get *Null* when calling
>>>> > *this.entries.get(index)*. The code is as below.
>>>> >
>>>> > public double get(int index) throws NullPointerException {
>>>> > return this.entries.get(index).getValue();
>>>> > }
>>>> >
>>>> > It is a bug. A Jira may be filed to fix two files of the code.
>>>> >
>>>> > first:
>>>> > Method *setBlockPosition* in DenseMatrix class;
>>>> > second:
>>>> > Class *BlockingMapper* in BlockingMapRed class;
>>>> >
>>>> > I am sorry. :-(
>>>> > I have found the potential bug in DenseMatrix when I write the
>>>> > BlockingMapRed class. But I forget to fix it and use the same way to
>>>> > caculate the block ranges during mapping.
>>>> >
>>>> > On Thu, Dec 4, 2008 at 3:09 PM, Edward J. Yoon <edwardyoon@apache.org
>>>> >wrote:
>>>> >
>>>> >> Can someone figure out this problem?
>>>> >>
>>>> >> 08/12/04 15:58:37 INFO hama.AbstractMatrix: Convert to 20 * 20 blocked
>>>> >> matrix
>>>> >> 08/12/04 15:58:37 WARN mapred.JobClient: Use GenericOptionsParser for
>>>> >> parsing the arguments. Applications should implement Tool for the
>>>> >> same.
>>>> >> 08/12/04 15:58:37 WARN mapred.JobClient: Use genericOptions for the
>>>> >> option -libjars
>>>> >> 08/12/04 15:58:38 WARN mapred.JobClient: No job jar file set. User
>>>> >> classes may not be found. See JobConf(Class) or
>>>> >> JobConf#setJar(String).
>>>> >> 08/12/04 15:58:38 INFO mapred.JobClient: Running job:
>>>> job_200812041310_0026
>>>> >> 08/12/04 15:58:39 INFO mapred.JobClient: map 0% reduce 0%
>>>> >> 08/12/04 15:59:29 INFO mapred.JobClient: Task Id :
>>>> >> attempt_200812041310_0026_m_000002_0, Status : FAILED
>>>> >> java.lang.NullPointerException: v(1) : java.lang.NullPointerException
>>>> >> at org.apache.hama.AbstractVector.get(AbstractVector.java:44)
>>>> >> at org.apache.hama.DenseVector.subVector(DenseVector.java:151)
>>>> >> at
>>>> >>
>>>> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:112)
>>>> >> at
>>>> >>
>>>> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:98)
>>>> >> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
>>>> >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
>>>> >> at
>>>> >> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
>>>> >> --
>>>> >> Best Regards, Edward J. Yoon @ NHN, corp.
>>>> >> edwardyoon@apache.org
>>>> >> http://blog.udanax.org
>>>> >>
>>>> >
>>>>
>>>>
>>>>
>>>> --
>>>> Best Regards, Edward J. Yoon @ NHN, corp.
>>>> edwardyoon@apache.org
>>>> http://blog.udanax.org
>>>>
>>>
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon @ NHN, corp.
>> edwardyoon@apache.org
>> http://blog.udanax.org
>>
>
>
>
> --
> Best Regards, Edward J. Yoon @ NHN, corp.
> edwardyoon@apache.org
> http://blog.udanax.org
>
--
Best Regards, Edward J. Yoon @ NHN, corp.
edwardyoon@apache.org
http://blog.udanax.org
Re: NullPointerException at AbstractVector.get(AbstractVector.java:44)
Posted by "Edward J. Yoon" <ed...@apache.org>.
In random_mapred(), IntWritable(m) should be IntWritable(m-1) since
start from 0.
On Mon, Dec 8, 2008 at 3:08 PM, Edward J. Yoon <ed...@apache.org> wrote:
> Somtimes, job done successfully. BTW, Reduce input groups=110. I guess
> this is a number of BlockIDs. Should it be a 10 * 10 = 100?
>
> ----
> 08/12/08 14:46:52 INFO hama.AbstractMatrix: Convert to 10 * 10 blocked matrix
> ...
> 08/12/08 14:52:27 INFO mapred.JobClient: Local bytes written=2662743172
> 08/12/08 14:52:27 INFO mapred.JobClient: Job Counters
> 08/12/08 14:52:27 INFO mapred.JobClient: Launched reduce tasks=2
> 08/12/08 14:52:27 INFO mapred.JobClient: Launched map tasks=13
> 08/12/08 14:52:27 INFO mapred.JobClient: Map-Reduce Framework
> 08/12/08 14:52:27 INFO mapred.JobClient: Reduce input groups=110
> 08/12/08 14:52:27 INFO mapred.JobClient: Combine output records=0
> 08/12/08 14:52:27 INFO mapred.JobClient: Map input records=7556
> 08/12/08 14:52:27 INFO mapred.JobClient: Reduce output records=0
> 08/12/08 14:52:27 INFO mapred.JobClient: Map output bytes=1020493900
> 08/12/08 14:52:27 INFO mapred.JobClient: Map input bytes=0
> 08/12/08 14:52:27 INFO mapred.JobClient: Combine input records=0
> 08/12/08 14:52:27 INFO mapred.JobClient: Map output records=75560
> 08/12/08 14:52:27 INFO mapred.JobClient: Reduce input records=75560
>
> On Mon, Dec 8, 2008 at 2:31 PM, Samuel Guo <gu...@gmail.com> wrote:
>> It is strange.
>> Let's review the code again to find what's wrong.
>>
>> On Mon, Dec 8, 2008 at 12:26 PM, Edward J. Yoon <ed...@apache.org>wrote:
>>
>>> [d8g053:/root/hama-trunk]# bin/hama examples mult -m 10 -r 10 fA fB
>>> Size : 4,000
>>> block_num : 10
>>>
>>> [d8g053:/root/hama-trunk]# bin/hama examples random 4000 4000 fB
>>> ...
>>> [d8g053:/root/hama-trunk]# bin/hama examples mult -m 10 -r 10 fA fB
>>>
>>> 08/12/08 13:23:57 INFO hama.AbstractMatrix: Convert to 10 * 10 blocked
>>> matrix
>>> 08/12/08 13:23:57 WARN mapred.JobClient: Use GenericOptionsParser for
>>> parsing the arguments. Applications should implement Tool for the
>>> same.
>>> 08/12/08 13:23:57 WARN mapred.JobClient: Use genericOptions for the
>>> option -libjars
>>> 08/12/08 13:23:57 WARN mapred.JobClient: No job jar file set. User
>>> classes may not be found. See JobConf(Class) or
>>> JobConf#setJar(String).
>>> 08/12/08 13:23:57 INFO mapred.JobClient: Running job: job_200812051425_0027
>>> 08/12/08 13:23:58 INFO mapred.JobClient: map 0% reduce 0%
>>> 08/12/08 13:24:29 INFO mapred.JobClient: Task Id :
>>> attempt_200812051425_0027_m_000001_0, Status : FAILED
>>> java.lang.NullPointerException: v(1) : java.lang.NullPointerException
>>> at org.apache.hama.AbstractVector.get(AbstractVector.java:44)
>>> at org.apache.hama.DenseVector.subVector(DenseVector.java:151)
>>> at
>>> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:112)
>>> at
>>> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:98)
>>> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
>>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
>>> at
>>> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
>>>
>>>
>>>
>>> On Thu, Dec 4, 2008 at 11:30 PM, Samuel Guo <gu...@gmail.com> wrote:
>>> > what is the matrix's size?
>>> > what is the block's size?
>>> > can the matrix's size be interge-divided by block's size?
>>> > if not, the last sub-vector's index range will be out of bound.
>>> >
>>> > so, the get() method in AbstractVector will get *Null* when calling
>>> > *this.entries.get(index)*. The code is as below.
>>> >
>>> > public double get(int index) throws NullPointerException {
>>> > return this.entries.get(index).getValue();
>>> > }
>>> >
>>> > It is a bug. A Jira may be filed to fix two files of the code.
>>> >
>>> > first:
>>> > Method *setBlockPosition* in DenseMatrix class;
>>> > second:
>>> > Class *BlockingMapper* in BlockingMapRed class;
>>> >
>>> > I am sorry. :-(
>>> > I have found the potential bug in DenseMatrix when I write the
>>> > BlockingMapRed class. But I forget to fix it and use the same way to
>>> > caculate the block ranges during mapping.
>>> >
>>> > On Thu, Dec 4, 2008 at 3:09 PM, Edward J. Yoon <edwardyoon@apache.org
>>> >wrote:
>>> >
>>> >> Can someone figure out this problem?
>>> >>
>>> >> 08/12/04 15:58:37 INFO hama.AbstractMatrix: Convert to 20 * 20 blocked
>>> >> matrix
>>> >> 08/12/04 15:58:37 WARN mapred.JobClient: Use GenericOptionsParser for
>>> >> parsing the arguments. Applications should implement Tool for the
>>> >> same.
>>> >> 08/12/04 15:58:37 WARN mapred.JobClient: Use genericOptions for the
>>> >> option -libjars
>>> >> 08/12/04 15:58:38 WARN mapred.JobClient: No job jar file set. User
>>> >> classes may not be found. See JobConf(Class) or
>>> >> JobConf#setJar(String).
>>> >> 08/12/04 15:58:38 INFO mapred.JobClient: Running job:
>>> job_200812041310_0026
>>> >> 08/12/04 15:58:39 INFO mapred.JobClient: map 0% reduce 0%
>>> >> 08/12/04 15:59:29 INFO mapred.JobClient: Task Id :
>>> >> attempt_200812041310_0026_m_000002_0, Status : FAILED
>>> >> java.lang.NullPointerException: v(1) : java.lang.NullPointerException
>>> >> at org.apache.hama.AbstractVector.get(AbstractVector.java:44)
>>> >> at org.apache.hama.DenseVector.subVector(DenseVector.java:151)
>>> >> at
>>> >>
>>> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:112)
>>> >> at
>>> >>
>>> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:98)
>>> >> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
>>> >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
>>> >> at
>>> >> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
>>> >> --
>>> >> Best Regards, Edward J. Yoon @ NHN, corp.
>>> >> edwardyoon@apache.org
>>> >> http://blog.udanax.org
>>> >>
>>> >
>>>
>>>
>>>
>>> --
>>> Best Regards, Edward J. Yoon @ NHN, corp.
>>> edwardyoon@apache.org
>>> http://blog.udanax.org
>>>
>>
>
>
>
> --
> Best Regards, Edward J. Yoon @ NHN, corp.
> edwardyoon@apache.org
> http://blog.udanax.org
>
--
Best Regards, Edward J. Yoon @ NHN, corp.
edwardyoon@apache.org
http://blog.udanax.org
Re: NullPointerException at AbstractVector.get(AbstractVector.java:44)
Posted by "Edward J. Yoon" <ed...@apache.org>.
Somtimes, job done successfully. BTW, Reduce input groups=110. I guess
this is a number of BlockIDs. Should it be a 10 * 10 = 100?
----
08/12/08 14:46:52 INFO hama.AbstractMatrix: Convert to 10 * 10 blocked matrix
...
08/12/08 14:52:27 INFO mapred.JobClient: Local bytes written=2662743172
08/12/08 14:52:27 INFO mapred.JobClient: Job Counters
08/12/08 14:52:27 INFO mapred.JobClient: Launched reduce tasks=2
08/12/08 14:52:27 INFO mapred.JobClient: Launched map tasks=13
08/12/08 14:52:27 INFO mapred.JobClient: Map-Reduce Framework
08/12/08 14:52:27 INFO mapred.JobClient: Reduce input groups=110
08/12/08 14:52:27 INFO mapred.JobClient: Combine output records=0
08/12/08 14:52:27 INFO mapred.JobClient: Map input records=7556
08/12/08 14:52:27 INFO mapred.JobClient: Reduce output records=0
08/12/08 14:52:27 INFO mapred.JobClient: Map output bytes=1020493900
08/12/08 14:52:27 INFO mapred.JobClient: Map input bytes=0
08/12/08 14:52:27 INFO mapred.JobClient: Combine input records=0
08/12/08 14:52:27 INFO mapred.JobClient: Map output records=75560
08/12/08 14:52:27 INFO mapred.JobClient: Reduce input records=75560
On Mon, Dec 8, 2008 at 2:31 PM, Samuel Guo <gu...@gmail.com> wrote:
> It is strange.
> Let's review the code again to find what's wrong.
>
> On Mon, Dec 8, 2008 at 12:26 PM, Edward J. Yoon <ed...@apache.org>wrote:
>
>> [d8g053:/root/hama-trunk]# bin/hama examples mult -m 10 -r 10 fA fB
>> Size : 4,000
>> block_num : 10
>>
>> [d8g053:/root/hama-trunk]# bin/hama examples random 4000 4000 fB
>> ...
>> [d8g053:/root/hama-trunk]# bin/hama examples mult -m 10 -r 10 fA fB
>>
>> 08/12/08 13:23:57 INFO hama.AbstractMatrix: Convert to 10 * 10 blocked
>> matrix
>> 08/12/08 13:23:57 WARN mapred.JobClient: Use GenericOptionsParser for
>> parsing the arguments. Applications should implement Tool for the
>> same.
>> 08/12/08 13:23:57 WARN mapred.JobClient: Use genericOptions for the
>> option -libjars
>> 08/12/08 13:23:57 WARN mapred.JobClient: No job jar file set. User
>> classes may not be found. See JobConf(Class) or
>> JobConf#setJar(String).
>> 08/12/08 13:23:57 INFO mapred.JobClient: Running job: job_200812051425_0027
>> 08/12/08 13:23:58 INFO mapred.JobClient: map 0% reduce 0%
>> 08/12/08 13:24:29 INFO mapred.JobClient: Task Id :
>> attempt_200812051425_0027_m_000001_0, Status : FAILED
>> java.lang.NullPointerException: v(1) : java.lang.NullPointerException
>> at org.apache.hama.AbstractVector.get(AbstractVector.java:44)
>> at org.apache.hama.DenseVector.subVector(DenseVector.java:151)
>> at
>> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:112)
>> at
>> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:98)
>> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
>> at
>> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
>>
>>
>>
>> On Thu, Dec 4, 2008 at 11:30 PM, Samuel Guo <gu...@gmail.com> wrote:
>> > what is the matrix's size?
>> > what is the block's size?
>> > can the matrix's size be interge-divided by block's size?
>> > if not, the last sub-vector's index range will be out of bound.
>> >
>> > so, the get() method in AbstractVector will get *Null* when calling
>> > *this.entries.get(index)*. The code is as below.
>> >
>> > public double get(int index) throws NullPointerException {
>> > return this.entries.get(index).getValue();
>> > }
>> >
>> > It is a bug. A Jira may be filed to fix two files of the code.
>> >
>> > first:
>> > Method *setBlockPosition* in DenseMatrix class;
>> > second:
>> > Class *BlockingMapper* in BlockingMapRed class;
>> >
>> > I am sorry. :-(
>> > I have found the potential bug in DenseMatrix when I write the
>> > BlockingMapRed class. But I forget to fix it and use the same way to
>> > caculate the block ranges during mapping.
>> >
>> > On Thu, Dec 4, 2008 at 3:09 PM, Edward J. Yoon <edwardyoon@apache.org
>> >wrote:
>> >
>> >> Can someone figure out this problem?
>> >>
>> >> 08/12/04 15:58:37 INFO hama.AbstractMatrix: Convert to 20 * 20 blocked
>> >> matrix
>> >> 08/12/04 15:58:37 WARN mapred.JobClient: Use GenericOptionsParser for
>> >> parsing the arguments. Applications should implement Tool for the
>> >> same.
>> >> 08/12/04 15:58:37 WARN mapred.JobClient: Use genericOptions for the
>> >> option -libjars
>> >> 08/12/04 15:58:38 WARN mapred.JobClient: No job jar file set. User
>> >> classes may not be found. See JobConf(Class) or
>> >> JobConf#setJar(String).
>> >> 08/12/04 15:58:38 INFO mapred.JobClient: Running job:
>> job_200812041310_0026
>> >> 08/12/04 15:58:39 INFO mapred.JobClient: map 0% reduce 0%
>> >> 08/12/04 15:59:29 INFO mapred.JobClient: Task Id :
>> >> attempt_200812041310_0026_m_000002_0, Status : FAILED
>> >> java.lang.NullPointerException: v(1) : java.lang.NullPointerException
>> >> at org.apache.hama.AbstractVector.get(AbstractVector.java:44)
>> >> at org.apache.hama.DenseVector.subVector(DenseVector.java:151)
>> >> at
>> >>
>> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:112)
>> >> at
>> >>
>> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:98)
>> >> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
>> >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
>> >> at
>> >> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
>> >> --
>> >> Best Regards, Edward J. Yoon @ NHN, corp.
>> >> edwardyoon@apache.org
>> >> http://blog.udanax.org
>> >>
>> >
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon @ NHN, corp.
>> edwardyoon@apache.org
>> http://blog.udanax.org
>>
>
--
Best Regards, Edward J. Yoon @ NHN, corp.
edwardyoon@apache.org
http://blog.udanax.org
Re: NullPointerException at AbstractVector.get(AbstractVector.java:44)
Posted by Samuel Guo <gu...@gmail.com>.
It is strange.
Let's review the code again to find what's wrong.
On Mon, Dec 8, 2008 at 12:26 PM, Edward J. Yoon <ed...@apache.org>wrote:
> [d8g053:/root/hama-trunk]# bin/hama examples mult -m 10 -r 10 fA fB
> Size : 4,000
> block_num : 10
>
> [d8g053:/root/hama-trunk]# bin/hama examples random 4000 4000 fB
> ...
> [d8g053:/root/hama-trunk]# bin/hama examples mult -m 10 -r 10 fA fB
>
> 08/12/08 13:23:57 INFO hama.AbstractMatrix: Convert to 10 * 10 blocked
> matrix
> 08/12/08 13:23:57 WARN mapred.JobClient: Use GenericOptionsParser for
> parsing the arguments. Applications should implement Tool for the
> same.
> 08/12/08 13:23:57 WARN mapred.JobClient: Use genericOptions for the
> option -libjars
> 08/12/08 13:23:57 WARN mapred.JobClient: No job jar file set. User
> classes may not be found. See JobConf(Class) or
> JobConf#setJar(String).
> 08/12/08 13:23:57 INFO mapred.JobClient: Running job: job_200812051425_0027
> 08/12/08 13:23:58 INFO mapred.JobClient: map 0% reduce 0%
> 08/12/08 13:24:29 INFO mapred.JobClient: Task Id :
> attempt_200812051425_0027_m_000001_0, Status : FAILED
> java.lang.NullPointerException: v(1) : java.lang.NullPointerException
> at org.apache.hama.AbstractVector.get(AbstractVector.java:44)
> at org.apache.hama.DenseVector.subVector(DenseVector.java:151)
> at
> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:112)
> at
> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:98)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> at
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
>
>
>
> On Thu, Dec 4, 2008 at 11:30 PM, Samuel Guo <gu...@gmail.com> wrote:
> > what is the matrix's size?
> > what is the block's size?
> > can the matrix's size be interge-divided by block's size?
> > if not, the last sub-vector's index range will be out of bound.
> >
> > so, the get() method in AbstractVector will get *Null* when calling
> > *this.entries.get(index)*. The code is as below.
> >
> > public double get(int index) throws NullPointerException {
> > return this.entries.get(index).getValue();
> > }
> >
> > It is a bug. A Jira may be filed to fix two files of the code.
> >
> > first:
> > Method *setBlockPosition* in DenseMatrix class;
> > second:
> > Class *BlockingMapper* in BlockingMapRed class;
> >
> > I am sorry. :-(
> > I have found the potential bug in DenseMatrix when I write the
> > BlockingMapRed class. But I forget to fix it and use the same way to
> > caculate the block ranges during mapping.
> >
> > On Thu, Dec 4, 2008 at 3:09 PM, Edward J. Yoon <edwardyoon@apache.org
> >wrote:
> >
> >> Can someone figure out this problem?
> >>
> >> 08/12/04 15:58:37 INFO hama.AbstractMatrix: Convert to 20 * 20 blocked
> >> matrix
> >> 08/12/04 15:58:37 WARN mapred.JobClient: Use GenericOptionsParser for
> >> parsing the arguments. Applications should implement Tool for the
> >> same.
> >> 08/12/04 15:58:37 WARN mapred.JobClient: Use genericOptions for the
> >> option -libjars
> >> 08/12/04 15:58:38 WARN mapred.JobClient: No job jar file set. User
> >> classes may not be found. See JobConf(Class) or
> >> JobConf#setJar(String).
> >> 08/12/04 15:58:38 INFO mapred.JobClient: Running job:
> job_200812041310_0026
> >> 08/12/04 15:58:39 INFO mapred.JobClient: map 0% reduce 0%
> >> 08/12/04 15:59:29 INFO mapred.JobClient: Task Id :
> >> attempt_200812041310_0026_m_000002_0, Status : FAILED
> >> java.lang.NullPointerException: v(1) : java.lang.NullPointerException
> >> at org.apache.hama.AbstractVector.get(AbstractVector.java:44)
> >> at org.apache.hama.DenseVector.subVector(DenseVector.java:151)
> >> at
> >>
> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:112)
> >> at
> >>
> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:98)
> >> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> >> at
> >> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
> >> --
> >> Best Regards, Edward J. Yoon @ NHN, corp.
> >> edwardyoon@apache.org
> >> http://blog.udanax.org
> >>
> >
>
>
>
> --
> Best Regards, Edward J. Yoon @ NHN, corp.
> edwardyoon@apache.org
> http://blog.udanax.org
>
Re: NullPointerException at AbstractVector.get(AbstractVector.java:44)
Posted by "Edward J. Yoon" <ed...@apache.org>.
[d8g053:/root/hama-trunk]# bin/hama examples mult -m 10 -r 10 fA fB
Size : 4,000
block_num : 10
[d8g053:/root/hama-trunk]# bin/hama examples random 4000 4000 fB
...
[d8g053:/root/hama-trunk]# bin/hama examples mult -m 10 -r 10 fA fB
08/12/08 13:23:57 INFO hama.AbstractMatrix: Convert to 10 * 10 blocked matrix
08/12/08 13:23:57 WARN mapred.JobClient: Use GenericOptionsParser for
parsing the arguments. Applications should implement Tool for the
same.
08/12/08 13:23:57 WARN mapred.JobClient: Use genericOptions for the
option -libjars
08/12/08 13:23:57 WARN mapred.JobClient: No job jar file set. User
classes may not be found. See JobConf(Class) or
JobConf#setJar(String).
08/12/08 13:23:57 INFO mapred.JobClient: Running job: job_200812051425_0027
08/12/08 13:23:58 INFO mapred.JobClient: map 0% reduce 0%
08/12/08 13:24:29 INFO mapred.JobClient: Task Id :
attempt_200812051425_0027_m_000001_0, Status : FAILED
java.lang.NullPointerException: v(1) : java.lang.NullPointerException
at org.apache.hama.AbstractVector.get(AbstractVector.java:44)
at org.apache.hama.DenseVector.subVector(DenseVector.java:151)
at org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:112)
at org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:98)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
On Thu, Dec 4, 2008 at 11:30 PM, Samuel Guo <gu...@gmail.com> wrote:
> what is the matrix's size?
> what is the block's size?
> can the matrix's size be interge-divided by block's size?
> if not, the last sub-vector's index range will be out of bound.
>
> so, the get() method in AbstractVector will get *Null* when calling
> *this.entries.get(index)*. The code is as below.
>
> public double get(int index) throws NullPointerException {
> return this.entries.get(index).getValue();
> }
>
> It is a bug. A Jira may be filed to fix two files of the code.
>
> first:
> Method *setBlockPosition* in DenseMatrix class;
> second:
> Class *BlockingMapper* in BlockingMapRed class;
>
> I am sorry. :-(
> I have found the potential bug in DenseMatrix when I write the
> BlockingMapRed class. But I forget to fix it and use the same way to
> caculate the block ranges during mapping.
>
> On Thu, Dec 4, 2008 at 3:09 PM, Edward J. Yoon <ed...@apache.org>wrote:
>
>> Can someone figure out this problem?
>>
>> 08/12/04 15:58:37 INFO hama.AbstractMatrix: Convert to 20 * 20 blocked
>> matrix
>> 08/12/04 15:58:37 WARN mapred.JobClient: Use GenericOptionsParser for
>> parsing the arguments. Applications should implement Tool for the
>> same.
>> 08/12/04 15:58:37 WARN mapred.JobClient: Use genericOptions for the
>> option -libjars
>> 08/12/04 15:58:38 WARN mapred.JobClient: No job jar file set. User
>> classes may not be found. See JobConf(Class) or
>> JobConf#setJar(String).
>> 08/12/04 15:58:38 INFO mapred.JobClient: Running job: job_200812041310_0026
>> 08/12/04 15:58:39 INFO mapred.JobClient: map 0% reduce 0%
>> 08/12/04 15:59:29 INFO mapred.JobClient: Task Id :
>> attempt_200812041310_0026_m_000002_0, Status : FAILED
>> java.lang.NullPointerException: v(1) : java.lang.NullPointerException
>> at org.apache.hama.AbstractVector.get(AbstractVector.java:44)
>> at org.apache.hama.DenseVector.subVector(DenseVector.java:151)
>> at
>> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:112)
>> at
>> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:98)
>> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
>> at
>> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
>> --
>> Best Regards, Edward J. Yoon @ NHN, corp.
>> edwardyoon@apache.org
>> http://blog.udanax.org
>>
>
--
Best Regards, Edward J. Yoon @ NHN, corp.
edwardyoon@apache.org
http://blog.udanax.org
Re: NullPointerException at AbstractVector.get(AbstractVector.java:44)
Posted by Samuel Guo <gu...@gmail.com>.
what is the matrix's size?
what is the block's size?
can the matrix's size be interge-divided by block's size?
if not, the last sub-vector's index range will be out of bound.
so, the get() method in AbstractVector will get *Null* when calling
*this.entries.get(index)*. The code is as below.
public double get(int index) throws NullPointerException {
return this.entries.get(index).getValue();
}
It is a bug. A Jira may be filed to fix two files of the code.
first:
Method *setBlockPosition* in DenseMatrix class;
second:
Class *BlockingMapper* in BlockingMapRed class;
I am sorry. :-(
I have found the potential bug in DenseMatrix when I write the
BlockingMapRed class. But I forget to fix it and use the same way to
caculate the block ranges during mapping.
On Thu, Dec 4, 2008 at 3:09 PM, Edward J. Yoon <ed...@apache.org>wrote:
> Can someone figure out this problem?
>
> 08/12/04 15:58:37 INFO hama.AbstractMatrix: Convert to 20 * 20 blocked
> matrix
> 08/12/04 15:58:37 WARN mapred.JobClient: Use GenericOptionsParser for
> parsing the arguments. Applications should implement Tool for the
> same.
> 08/12/04 15:58:37 WARN mapred.JobClient: Use genericOptions for the
> option -libjars
> 08/12/04 15:58:38 WARN mapred.JobClient: No job jar file set. User
> classes may not be found. See JobConf(Class) or
> JobConf#setJar(String).
> 08/12/04 15:58:38 INFO mapred.JobClient: Running job: job_200812041310_0026
> 08/12/04 15:58:39 INFO mapred.JobClient: map 0% reduce 0%
> 08/12/04 15:59:29 INFO mapred.JobClient: Task Id :
> attempt_200812041310_0026_m_000002_0, Status : FAILED
> java.lang.NullPointerException: v(1) : java.lang.NullPointerException
> at org.apache.hama.AbstractVector.get(AbstractVector.java:44)
> at org.apache.hama.DenseVector.subVector(DenseVector.java:151)
> at
> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:112)
> at
> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:98)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> at
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
> --
> Best Regards, Edward J. Yoon @ NHN, corp.
> edwardyoon@apache.org
> http://blog.udanax.org
>