You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@hama.apache.org by Kyungho Jeon <kh...@dcslab.snu.ac.kr> on 2008/09/11 05:25:25 UTC

Error in Matrix Multiplication

Hello,

I changed the Matrix Addition example to do some matrix multiplications.

A NullPointException occurred when I tried to multiply two times like this.

 61     Matrix c = a.mult(b);
 62     Matrix d = c.mult(b);

(I changed line 61 from a.add(b) to a.mult(b) and added line 62.)

I ran it in a distributed operation (real cluster).

$ bin/hadoop jar hama-0.1.0-dev-examples.jar addition 3 3

08/09/11 12:07:58 INFO hama.AbstractMatrix: Initializing the matrix storage.
08/09/11 12:08:08 INFO hama.AbstractMatrix: Create the 3 * 3 random matrix :
randivtpz
08/09/11 12:08:09 INFO hama.AbstractMatrix: Initializing the matrix storage.
08/09/11 12:08:19 INFO hama.AbstractMatrix: Create the 3 * 3 random matrix :
randzzoyr

Matrix A
----------------------
0.12246664942306407
0.47492867854942666
0.753402452679103
0.11948311310699711
0.04760886057850422
0.04664804577467441
0.4865392863824721
0.7541368692517372
0.5844076068157283

Matrix B
----------------------
0.30800857170320417
0.250851984967709
0.5748486376994186
0.808254823959184
0.11114206173021035
0.021408513651528183
0.5489044167760238
0.23894781330682957
0.2000317464813144

08/09/11 12:08:19 INFO hama.AbstractMatrix: Initializing the matrix storage.
08/09/11 12:08:30 WARN mapred.JobClient: No job jar file set.  User classes
may not be found. See JobConf(Class) or JobConf#setJar(String).
08/09/11 12:08:30 INFO mapred.JobClient: Running job: job_200809102127_0018
08/09/11 12:08:31 INFO mapred.JobClient:  map 0% reduce 0%
08/09/11 12:08:37 INFO mapred.JobClient:  map 66% reduce 0%
08/09/11 12:08:38 INFO mapred.JobClient:  map 100% reduce 0%
08/09/11 12:08:45 INFO mapred.JobClient:  map 100% reduce 50%
08/09/11 12:08:47 INFO mapred.JobClient:  map 100% reduce 61%
08/09/11 12:09:36 INFO mapred.JobClient:  map 100% reduce 100%
08/09/11 12:09:37 INFO mapred.JobClient: Job complete: job_200809102127_0018
08/09/11 12:09:38 INFO mapred.JobClient: Counters: 13
08/09/11 12:09:38 INFO mapred.JobClient:   File Systems
08/09/11 12:09:38 INFO mapred.JobClient:     Local bytes read=1542
08/09/11 12:09:38 INFO mapred.JobClient:     Local bytes written=3540
08/09/11 12:09:38 INFO mapred.JobClient:   Job Counters
08/09/11 12:09:38 INFO mapred.JobClient:     Launched map tasks=3
08/09/11 12:09:38 INFO mapred.JobClient:     Launched reduce tasks=3
08/09/11 12:09:38 INFO mapred.JobClient:   Map-Reduce Framework
08/09/11 12:09:38 INFO mapred.JobClient:     Map input records=3
08/09/11 12:09:38 INFO mapred.JobClient:     Map output records=9
08/09/11 12:09:38 INFO mapred.JobClient:     Map input bytes=0
08/09/11 12:09:38 INFO mapred.JobClient:     Map output bytes=1224
08/09/11 12:09:38 INFO mapred.JobClient:     Combine input records=0
08/09/11 12:09:38 INFO mapred.JobClient:     Combine output records=0
08/09/11 12:09:38 INFO mapred.JobClient:     Reduce input groups=3
08/09/11 12:09:38 INFO mapred.JobClient:     Reduce input records=9
08/09/11 12:09:38 INFO mapred.JobClient:     Reduce output records=3
08/09/11 12:09:38 INFO hama.AbstractMatrix: Initializing the matrix storage.
08/09/11 12:09:48 WARN mapred.JobClient: No job jar file set.  User classes
may not be found. See JobConf(Class) or JobConf#setJar(String).
java.lang.NullPointerException
        at
org.apache.hama.mapred.MatrixInputFormatBase.getSplits(MatrixInputFormatBase.java:212)
        at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:712)
        at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:973)
        at org.apache.hama.DenseMatrix.mult(DenseMatrix.java:185)
        at
org.apache.hama.examples.MatrixAddition.main(MatrixAddition.java:62)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
        at
org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
        at
org.apache.hama.examples.ExampleDriver.main(ExampleDriver.java:30)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
        at org.apache.hadoop.mapred.JobShell.run(JobShell.java:194)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
        at org.apache.hadoop.mapred.JobShell.main(JobShell.java:220)

Re: Error in Matrix Multiplication

Posted by "Edward J. Yoon" <ed...@apache.org>.

OK, this issue also fixed.
Thanks.

/Edward

On Sun, Sep 14, 2008 at 4:22 PM, Edward J. Yoon <ed...@apache.org> wrote:
> Good report! We should write dimension attributes when job completed.
> I filed a jira on that as HAMA-55.
>
> -Ed
>
> On Wed, Sep 10, 2008 at 11:25 PM, Kyungho Jeon <kh...@dcslab.snu.ac.kr> wrote:
>> Hello,
>>
>> I changed the Matrix Addition example to do some matrix multiplications.
>>
>> A NullPointException occurred when I tried to multiply two times like this.
>>
>>  61     Matrix c = a.mult(b);
>>  62     Matrix d = c.mult(b);
>>
>> (I changed line 61 from a.add(b) to a.mult(b) and added line 62.)
>>
>> I ran it in a distributed operation (real cluster).
>>
>> $ bin/hadoop jar hama-0.1.0-dev-examples.jar addition 3 3
>>
>> 08/09/11 12:07:58 INFO hama.AbstractMatrix: Initializing the matrix storage.
>> 08/09/11 12:08:08 INFO hama.AbstractMatrix: Create the 3 * 3 random matrix :
>> randivtpz
>> 08/09/11 12:08:09 INFO hama.AbstractMatrix: Initializing the matrix storage.
>> 08/09/11 12:08:19 INFO hama.AbstractMatrix: Create the 3 * 3 random matrix :
>> randzzoyr
>>
>> Matrix A
>> ----------------------
>> 0.12246664942306407
>> 0.47492867854942666
>> 0.753402452679103
>> 0.11948311310699711
>> 0.04760886057850422
>> 0.04664804577467441
>> 0.4865392863824721
>> 0.7541368692517372
>> 0.5844076068157283
>>
>> Matrix B
>> ----------------------
>> 0.30800857170320417
>> 0.250851984967709
>> 0.5748486376994186
>> 0.808254823959184
>> 0.11114206173021035
>> 0.021408513651528183
>> 0.5489044167760238
>> 0.23894781330682957
>> 0.2000317464813144
>>
>> 08/09/11 12:08:19 INFO hama.AbstractMatrix: Initializing the matrix storage.
>> 08/09/11 12:08:30 WARN mapred.JobClient: No job jar file set.  User classes
>> may not be found. See JobConf(Class) or JobConf#setJar(String).
>> 08/09/11 12:08:30 INFO mapred.JobClient: Running job: job_200809102127_0018
>> 08/09/11 12:08:31 INFO mapred.JobClient:  map 0% reduce 0%
>> 08/09/11 12:08:37 INFO mapred.JobClient:  map 66% reduce 0%
>> 08/09/11 12:08:38 INFO mapred.JobClient:  map 100% reduce 0%
>> 08/09/11 12:08:45 INFO mapred.JobClient:  map 100% reduce 50%
>> 08/09/11 12:08:47 INFO mapred.JobClient:  map 100% reduce 61%
>> 08/09/11 12:09:36 INFO mapred.JobClient:  map 100% reduce 100%
>> 08/09/11 12:09:37 INFO mapred.JobClient: Job complete: job_200809102127_0018
>> 08/09/11 12:09:38 INFO mapred.JobClient: Counters: 13
>> 08/09/11 12:09:38 INFO mapred.JobClient:   File Systems
>> 08/09/11 12:09:38 INFO mapred.JobClient:     Local bytes read=1542
>> 08/09/11 12:09:38 INFO mapred.JobClient:     Local bytes written=3540
>> 08/09/11 12:09:38 INFO mapred.JobClient:   Job Counters
>> 08/09/11 12:09:38 INFO mapred.JobClient:     Launched map tasks=3
>> 08/09/11 12:09:38 INFO mapred.JobClient:     Launched reduce tasks=3
>> 08/09/11 12:09:38 INFO mapred.JobClient:   Map-Reduce Framework
>> 08/09/11 12:09:38 INFO mapred.JobClient:     Map input records=3
>> 08/09/11 12:09:38 INFO mapred.JobClient:     Map output records=9
>> 08/09/11 12:09:38 INFO mapred.JobClient:     Map input bytes=0
>> 08/09/11 12:09:38 INFO mapred.JobClient:     Map output bytes=1224
>> 08/09/11 12:09:38 INFO mapred.JobClient:     Combine input records=0
>> 08/09/11 12:09:38 INFO mapred.JobClient:     Combine output records=0
>> 08/09/11 12:09:38 INFO mapred.JobClient:     Reduce input groups=3
>> 08/09/11 12:09:38 INFO mapred.JobClient:     Reduce input records=9
>> 08/09/11 12:09:38 INFO mapred.JobClient:     Reduce output records=3
>> 08/09/11 12:09:38 INFO hama.AbstractMatrix: Initializing the matrix storage.
>> 08/09/11 12:09:48 WARN mapred.JobClient: No job jar file set.  User classes
>> may not be found. See JobConf(Class) or JobConf#setJar(String).
>> java.lang.NullPointerException
>>        at
>> org.apache.hama.mapred.MatrixInputFormatBase.getSplits(MatrixInputFormatBase.java:212)
>>        at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:712)
>>        at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:973)
>>        at org.apache.hama.DenseMatrix.mult(DenseMatrix.java:185)
>>        at
>> org.apache.hama.examples.MatrixAddition.main(MatrixAddition.java:62)
>>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>        at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>        at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>        at java.lang.reflect.Method.invoke(Method.java:597)
>>        at
>> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
>>        at
>> org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
>>        at
>> org.apache.hama.examples.ExampleDriver.main(ExampleDriver.java:30)
>>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>        at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>        at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>        at java.lang.reflect.Method.invoke(Method.java:597)
>>        at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
>>        at org.apache.hadoop.mapred.JobShell.run(JobShell.java:194)
>>        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>        at org.apache.hadoop.mapred.JobShell.main(JobShell.java:220)
>>
>
>
>
> --
> Best regards, Edward J. Yoon
> edwardyoon@apache.org
> http://blog.udanax.org
>



-- 
Best regards, Edward J. Yoon
edwardyoon@apache.org
http://blog.udanax.org

Re: Error in Matrix Multiplication

Posted by "Edward J. Yoon" <ed...@apache.org>.

Good report! We should write dimension attributes when job completed.
I filed a jira on that as HAMA-55.

-Ed

On Wed, Sep 10, 2008 at 11:25 PM, Kyungho Jeon <kh...@dcslab.snu.ac.kr> wrote:
> Hello,
>
> I changed the Matrix Addition example to do some matrix multiplications.
>
> A NullPointException occurred when I tried to multiply two times like this.
>
>  61     Matrix c = a.mult(b);
>  62     Matrix d = c.mult(b);
>
> (I changed line 61 from a.add(b) to a.mult(b) and added line 62.)
>
> I ran it in a distributed operation (real cluster).
>
> $ bin/hadoop jar hama-0.1.0-dev-examples.jar addition 3 3
>
> 08/09/11 12:07:58 INFO hama.AbstractMatrix: Initializing the matrix storage.
> 08/09/11 12:08:08 INFO hama.AbstractMatrix: Create the 3 * 3 random matrix :
> randivtpz
> 08/09/11 12:08:09 INFO hama.AbstractMatrix: Initializing the matrix storage.
> 08/09/11 12:08:19 INFO hama.AbstractMatrix: Create the 3 * 3 random matrix :
> randzzoyr
>
> Matrix A
> ----------------------
> 0.12246664942306407
> 0.47492867854942666
> 0.753402452679103
> 0.11948311310699711
> 0.04760886057850422
> 0.04664804577467441
> 0.4865392863824721
> 0.7541368692517372
> 0.5844076068157283
>
> Matrix B
> ----------------------
> 0.30800857170320417
> 0.250851984967709
> 0.5748486376994186
> 0.808254823959184
> 0.11114206173021035
> 0.021408513651528183
> 0.5489044167760238
> 0.23894781330682957
> 0.2000317464813144
>
> 08/09/11 12:08:19 INFO hama.AbstractMatrix: Initializing the matrix storage.
> 08/09/11 12:08:30 WARN mapred.JobClient: No job jar file set.  User classes
> may not be found. See JobConf(Class) or JobConf#setJar(String).
> 08/09/11 12:08:30 INFO mapred.JobClient: Running job: job_200809102127_0018
> 08/09/11 12:08:31 INFO mapred.JobClient:  map 0% reduce 0%
> 08/09/11 12:08:37 INFO mapred.JobClient:  map 66% reduce 0%
> 08/09/11 12:08:38 INFO mapred.JobClient:  map 100% reduce 0%
> 08/09/11 12:08:45 INFO mapred.JobClient:  map 100% reduce 50%
> 08/09/11 12:08:47 INFO mapred.JobClient:  map 100% reduce 61%
> 08/09/11 12:09:36 INFO mapred.JobClient:  map 100% reduce 100%
> 08/09/11 12:09:37 INFO mapred.JobClient: Job complete: job_200809102127_0018
> 08/09/11 12:09:38 INFO mapred.JobClient: Counters: 13
> 08/09/11 12:09:38 INFO mapred.JobClient:   File Systems
> 08/09/11 12:09:38 INFO mapred.JobClient:     Local bytes read=1542
> 08/09/11 12:09:38 INFO mapred.JobClient:     Local bytes written=3540
> 08/09/11 12:09:38 INFO mapred.JobClient:   Job Counters
> 08/09/11 12:09:38 INFO mapred.JobClient:     Launched map tasks=3
> 08/09/11 12:09:38 INFO mapred.JobClient:     Launched reduce tasks=3
> 08/09/11 12:09:38 INFO mapred.JobClient:   Map-Reduce Framework
> 08/09/11 12:09:38 INFO mapred.JobClient:     Map input records=3
> 08/09/11 12:09:38 INFO mapred.JobClient:     Map output records=9
> 08/09/11 12:09:38 INFO mapred.JobClient:     Map input bytes=0
> 08/09/11 12:09:38 INFO mapred.JobClient:     Map output bytes=1224
> 08/09/11 12:09:38 INFO mapred.JobClient:     Combine input records=0
> 08/09/11 12:09:38 INFO mapred.JobClient:     Combine output records=0
> 08/09/11 12:09:38 INFO mapred.JobClient:     Reduce input groups=3
> 08/09/11 12:09:38 INFO mapred.JobClient:     Reduce input records=9
> 08/09/11 12:09:38 INFO mapred.JobClient:     Reduce output records=3
> 08/09/11 12:09:38 INFO hama.AbstractMatrix: Initializing the matrix storage.
> 08/09/11 12:09:48 WARN mapred.JobClient: No job jar file set.  User classes
> may not be found. See JobConf(Class) or JobConf#setJar(String).
> java.lang.NullPointerException
>        at
> org.apache.hama.mapred.MatrixInputFormatBase.getSplits(MatrixInputFormatBase.java:212)
>        at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:712)
>        at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:973)
>        at org.apache.hama.DenseMatrix.mult(DenseMatrix.java:185)
>        at
> org.apache.hama.examples.MatrixAddition.main(MatrixAddition.java:62)
>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>        at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>        at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>        at java.lang.reflect.Method.invoke(Method.java:597)
>        at
> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
>        at
> org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
>        at
> org.apache.hama.examples.ExampleDriver.main(ExampleDriver.java:30)
>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>        at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>        at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>        at java.lang.reflect.Method.invoke(Method.java:597)
>        at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
>        at org.apache.hadoop.mapred.JobShell.run(JobShell.java:194)
>        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>        at org.apache.hadoop.mapred.JobShell.main(JobShell.java:220)
>



-- 
Best regards, Edward J. Yoon
edwardyoon@apache.org
http://blog.udanax.org