[jira] [Resolved] (SPARK-21049) why do we need computeGramianMatrix when computing SVD

["Sean Owen (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-21049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sean Owen resolved SPARK-21049.
-------------------------------
    Resolution: Invalid

Questions should go to the mailing list. 
Consider what "just" computing the SVD of the original matrix entails, when it's a huge distributed matrix. Assuming the matrix is huge but skinny, the Gramian is small and can be handled in-core.

> why do we need computeGramianMatrix when computing SVD
> ------------------------------------------------------
>
>                 Key: SPARK-21049
>                 URL: https://issues.apache.org/jira/browse/SPARK-21049
>             Project: Spark
>          Issue Type: Improvement
>          Components: ML, MLlib
>    Affects Versions: 2.1.1
>            Reporter: Vincent
>
> computeSVD will compute SVD for matrix A by computing AT*A first and svd on the Gramian matrix, we found that the gramian matrix computation is the hot spot of the overall SVD computation, but, per my understanding, we can simply do svd on the original matrix. The singular vector of the gramian matrix should be the same as the right singular vector of the original matrix A, while the singular value of the gramian matrix is double as that of the original matrix. why do we svd on the gramian matrix then?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org