You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by Shannon Quinn <sq...@gatech.edu> on 2010/06/17 22:35:30 UTC
Multiple calls to DistributedRowMatrix.times()
Hi all,
I'm attempting to carry out the multiplication of two
DistributedRowMatrix objects. It's similar to a SVD multiplication in
that I have a matrix A and a matrix B, and the operation looks like BAB, or:
B.times(A.times(B))
When I run it, I get a FileAlreadyExistsException from Hadoop, since for
both A and B the result of outputTmpBasePath.getParent() is the same (A
is read from input, and B is calculated from A). Which would be a better
approach:
-Modify DistributedRowMatrix.times() so "productWith" is instead
"productWith" + (System.nanoTime() & 0xFF) .
-Change how the matrices are initialized and avoid the problem from
within the Driver.
I only ask because transpose() employs the first technique, and I don't
know how common (or uncommon) this particular situation is that might
warrant the first change I mentioned.
Thank you!
Regards,
Shannon
Re: Multiple calls to DistributedRowMatrix.times()
Posted by Shannon Quinn <sq...@gatech.edu>.
A is symmetric, B is diagonal, both have exactly the same dimensions.
I could do this operation linearly as someone else had suggested. Where
L = BAB, I could do L_ij = B_ii * A_ij * B_jj, and use a DenseVector for
B instead. I just felt this was more straightforward code-wise.
On 6/17/2010 5:28 PM, Ted Dunning wrote:
> And symmetric?
>
> On Thu, Jun 17, 2010 at 2:28 PM, Shannon Quinn<sq...@gatech.edu> wrote:
>
>
>> A and B are both square matrices
>>
>
Re: Multiple calls to DistributedRowMatrix.times()
Posted by Ted Dunning <te...@gmail.com>.
And symmetric?
On Thu, Jun 17, 2010 at 2:28 PM, Shannon Quinn <sq...@gatech.edu> wrote:
> A and B are both square matrices
Re: Multiple calls to DistributedRowMatrix.times()
Posted by Shannon Quinn <sq...@gatech.edu>.
Good question, actually - forgot to mention to mention that A and B are
both square matrices. Sorry about that.
On 6/17/2010 5:18 PM, Ted Dunning wrote:
> Off topic, but do you mean B' A B?
>
>
> On Thu, Jun 17, 2010 at 1:35 PM, Shannon Quinn<sq...@gatech.edu> wrote:
>
>
>> I'm attempting to carry out the multiplication of two DistributedRowMatrix
>> objects. It's similar to a SVD multiplication in that I have a matrix A and
>> a matrix B, and the operation looks like BAB
>>
>
Re: Multiple calls to DistributedRowMatrix.times()
Posted by Ted Dunning <te...@gmail.com>.
Off topic, but do you mean B' A B?
On Thu, Jun 17, 2010 at 1:35 PM, Shannon Quinn <sq...@gatech.edu> wrote:
> I'm attempting to carry out the multiplication of two DistributedRowMatrix
> objects. It's similar to a SVD multiplication in that I have a matrix A and
> a matrix B, and the operation looks like BAB
Re: Multiple calls to DistributedRowMatrix.times()
Posted by Jake Mannix <ja...@gmail.com>.
Hey Shannon,
The whole useage of temp directories in DistributedRowMatrix is...
"confused" at best. It needs to be better managed, really. But out of the
two approaches to this current problem,
-Modify DistributedRowMatrix.times() so "productWith" is instead
"productWith" + (System.nanoTime() & 0xFF) .
I think I like this approach, yes.
-jake
On Thu, Jun 17, 2010 at 1:35 PM, Shannon Quinn <sq...@gatech.edu> wrote:
> Hi all,
>
> I'm attempting to carry out the multiplication of two DistributedRowMatrix
> objects. It's similar to a SVD multiplication in that I have a matrix A and
> a matrix B, and the operation looks like BAB, or:
>
> B.times(A.times(B))
>
> When I run it, I get a FileAlreadyExistsException from Hadoop, since for
> both A and B the result of outputTmpBasePath.getParent() is the same (A is
> read from input, and B is calculated from A). Which would be a better
> approach:
>
> -Modify DistributedRowMatrix.times() so "productWith" is instead
> "productWith" + (System.nanoTime() & 0xFF) .
> -Change how the matrices are initialized and avoid the problem from within
> the Driver.
>
> I only ask because transpose() employs the first technique, and I don't
> know how common (or uncommon) this particular situation is that might
> warrant the first change I mentioned.
>
> Thank you!
>
> Regards,
> Shannon
>