You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mahout.apache.org by Dmitriy Lyubimov <dl...@gmail.com> on 2011/08/10 02:22:52 UTC
Re: Errors in SSVD
I will get back on this after my vacation. At this point I haven't tested
the code with u0. Also one of the jobs must have one reducer (the
parallelism of this job is heavily vested on use of combiners and high
degree of aggregated output), this may have been broken by one of those
attempts to unify how mr gets configured in Mahout.
One of my mahout-frozen branches works fine with cdh3, although not with u0
yet, that's what it was originally written for. You can try to take one of
those from my virgin account if there's any sense of urgency, they have been
verified with cdh3b2 I think.
On Aug 5, 2011 1:55 PM, "Eshwaran Vijaya Kumar" <ev...@mozilla.com>
wrote:
> All,
> I am trying to test Stochastic SVD and am facing some errors where it
would be great if someone could clarifying what is going on. I am trying to
feed the solver a DistributedRowMatrix with the exact same parameters that
the test in LocalSSVDSolverSparseSequentialTest uses, i.e, Generate a 1000 X
100 DRM with SequentialSparseVectors and then ask for blockHeight 251, p
(oversampling) = 60, k (rank) = 40. I get the following error:
>
> Exception in thread "main" java.io.IOException: Unexpected overrun in
upper triangular matrix files
> at
org.apache.mahout.math.hadoop.stochasticsvd.SSVDSolver.loadUpperTriangularMatrix(SSVDSolver.java:471)
> at
org.apache.mahout.math.hadoop.stochasticsvd.SSVDSolver.run(SSVDSolver.java:268)
> at com.mozilla.SSVDCli.run(SSVDCli.java:89)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at com.mozilla.SSVDCli.main(SSVDCli.java:129)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
>
> Also, I am using CDH3 with Mahout recompiled to work with CDH3 jars.
>
> Thanks
> Esh
>
Re: Errors in SSVD
Posted by Dmitriy Lyubimov <dl...@gmail.com>.
Damn android autocorrection. I meant from one of my github ac ounts.
Sometimes it(android spellcheck) makes me feel stupid.
On Aug 9, 2011 5:22 PM, "Dmitriy Lyubimov" <dl...@gmail.com> wrote:
> I will get back on this after my vacation. At this point I haven't tested
> the code with u0. Also one of the jobs must have one reducer (the
> parallelism of this job is heavily vested on use of combiners and high
> degree of aggregated output), this may have been broken by one of those
> attempts to unify how mr gets configured in Mahout.
>
> One of my mahout-frozen branches works fine with cdh3, although not with
u0
> yet, that's what it was originally written for. You can try to take one of
> those from my virgin account if there's any sense of urgency, they have
been
> verified with cdh3b2 I think.
>
> On Aug 5, 2011 1:55 PM, "Eshwaran Vijaya Kumar" <ev...@mozilla.com>
> wrote:
>> All,
>> I am trying to test Stochastic SVD and am facing some errors where it
> would be great if someone could clarifying what is going on. I am trying
to
> feed the solver a DistributedRowMatrix with the exact same parameters that
> the test in LocalSSVDSolverSparseSequentialTest uses, i.e, Generate a 1000
X
> 100 DRM with SequentialSparseVectors and then ask for blockHeight 251, p
> (oversampling) = 60, k (rank) = 40. I get the following error:
>>
>> Exception in thread "main" java.io.IOException: Unexpected overrun in
> upper triangular matrix files
>> at
>
org.apache.mahout.math.hadoop.stochasticsvd.SSVDSolver.loadUpperTriangularMatrix(SSVDSolver.java:471)
>> at
>
org.apache.mahout.math.hadoop.stochasticsvd.SSVDSolver.run(SSVDSolver.java:268)
>> at com.mozilla.SSVDCli.run(SSVDCli.java:89)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>> at com.mozilla.SSVDCli.main(SSVDCli.java:129)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at
>
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
>>
>> Also, I am using CDH3 with Mahout recompiled to work with CDH3 jars.
>>
>> Thanks
>> Esh
>>