You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "Dmitriy Lyubimov (JIRA)" <ji...@apache.org> on 2010/11/26 20:40:15 UTC
[jira] Issue Comment Edited: (MAHOUT-376) Implement Map-reduce
version of stochastic SVD
[ https://issues.apache.org/jira/browse/MAHOUT-376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12936080#action_12936080 ]
Dmitriy Lyubimov edited comment on MAHOUT-376 at 11/26/10 2:38 PM:
-------------------------------------------------------------------
git patch m1 : WIP but important milestone: prototype & MR implementation at the level of computing full Q and singlular values. as i mentioned, needs CDH3b2 (or b3).
Local MR test runs MR solver in local mode for a moderately low rank random matrix 80,000x100 (r=251, k+p=100). Test output i get on my laptop (first are singulars for 100x100 BBt matrix using commons-math Eigensolver; 2 -- output of SVs produced by Colt SVD of 80,000x100 same source matrix
--SSVD solver singular values:
svs: 4220.258342 4215.924299 4213.352353 4210.786495 4203.422385 4201.047189 4194.987920 4193.434856 4187.610381 4185.546818 4179.867986 4176.056232 4172.784145 4169.039073 4168.384457 4164.293827 4162.647531 4160.483398 4157.878385 4154.713189 4152.172788 4149.823917 4146.500139 4144.565227 4142.625983 4141.291209 4138.105799 4135.564939 4134.772833 4129.223450 4129.101594 4126.679080 4124.385614 4121.791730 4119.645948 4115.975993 4112.947092 4109.586452 4107.985419 4104.871381 4102.438854 4099.762117 4098.968505 4095.720204 4091.114871 4090.190141 (...omited) 3950.897035
--Colt SVD solver singular values:
svs: 4220.258342 4215.924299 4213.352353 4210.786495 4203.422385 4201.047189 4194.987920 4193.434856 4187.610381 4185.546818 4179.867986 4176.056232 4172.784145 4169.039073 4168.384457 4164.293827 4162.647531 4160.483398 4157.878385 4154.713189 4152.172788 4149.823917 4146.500139 4144.565227 4142.625983 4141.291209 4138.105799 4135.564939 4134.772833 4129.223450 4129.101594 4126.679080 4124.385614 4121.791730 4119.645948 4115.975993 4112.947092 4109.586452 4107.985419 4104.871381 4102.438854 4099.762117 4098.968505 4095.720204 4091.114871 4090.190141 (....omited) 3950.897035
I will be updating my notes with a couple of optimizations i applied in this code not yet mentioned.
-Dima
was (Author: dlyubimov2):
git patch m1 : WIP but important milestone: prototype & MR implementation at the level of computing full Q and singlular values. as i mentioned, needs CDH3b2 (or b3).
Local MR test runs MR solver in local mode for a moderately low rank random matrix 80,000x100 (r=251, k+p=100). Test output i get on my laptop (first are singulars for 100x100 BBt matrix using commons-math Eigensolver; 2 -- output of SVs produced by Colt SVD of 80,000x100 same source matrix
--SSVD solver singular values:
svs: 4220.258342 4215.924299 4213.352353 4210.786495 4203.422385 4201.047189 4194.987920 4193.434856 4187.610381 4185.546818 4179.867986 4176.056232 4172.784145 4169.039073 4168.384457 4164.293827 4162.647531 4160.483398 4157.878385 4154.713189 4152.172788 4149.823917 4146.500139 4144.565227 4142.625983 4141.291209 4138.105799 4135.564939 4134.772833 4129.223450 4129.101594 4126.679080 4124.385614 4121.791730 4119.645948 4115.975993 4112.947092 4109.586452 4107.985419 4104.871381 4102.438854 4099.762117 4098.968505 4095.720204 4091.114871 4090.190141 (...omited) 3950.897035
--Colt SVD solver singular values:
svs: 4220.258342 4215.924299 4213.352353 4210.786495 4203.422385 4201.047189 4194.987920 4193.434856 4187.610381 4185.546818 4179.867986 4176.056232 4172.784145 4169.039073 4168.384457 4164.293827 4162.647531 4160.483398 4157.878385 4154.713189 4152.172788 4149.823917 4146.500139 4144.565227 4142.625983 4141.291209 4138.105799 4135.564939 4134.772833 4129.223450 4129.101594 4126.679080 4124.385614 4121.791730 4119.645948 4115.975993 4112.947092 4109.586452 4107.985419 4104.871381 4102.438854 4099.762117 4098.968505 4095.720204 4091.114871 4090.190141 (....omited) 3950.897035
I will be updated my notes with a couple of optimizations i applied in this code not yet mentioned.
-Dima
> Implement Map-reduce version of stochastic SVD
> ----------------------------------------------
>
> Key: MAHOUT-376
> URL: https://issues.apache.org/jira/browse/MAHOUT-376
> Project: Mahout
> Issue Type: Improvement
> Components: Math
> Reporter: Ted Dunning
> Assignee: Ted Dunning
> Fix For: 0.5
>
> Attachments: MAHOUT-376.patch, Modified stochastic svd algorithm for mapreduce.pdf, QR decomposition for Map.pdf, QR decomposition for Map.pdf, QR decomposition for Map.pdf, sd-bib.bib, sd.pdf, sd.pdf, sd.pdf, sd.pdf, sd.tex, sd.tex, sd.tex, sd.tex, ssvd-m1.patch.gz, Stochastic SVD using eigensolver trick.pdf
>
>
> See attached pdf for outline of proposed method.
> All comments are welcome.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.