[jira] [Commented] (SPARK-15882) Discuss distributed linear algebra in spark.ml package

["Yanbo Liang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-15882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15415345#comment-15415345 ] 

Yanbo Liang commented on SPARK-15882:
-------------------------------------

I think we should make this migration finished before the next release.
I’m more prefer to use Datasets underneath since spark.ml is the DataFrame-based API. But we should do some tests to verify the migration will not introduce performance degradation, since linear algebra is one of the low level operations. I can start to work on it if no others has started. Thanks. [~josephkb] 

> Discuss distributed linear algebra in spark.ml package
> ------------------------------------------------------
>
>                 Key: SPARK-15882
>                 URL: https://issues.apache.org/jira/browse/SPARK-15882
>             Project: Spark
>          Issue Type: Brainstorming
>          Components: ML
>            Reporter: Joseph K. Bradley
>
> This JIRA is for discussing how org.apache.spark.mllib.linalg.distributed.* should be migrated to org.apache.spark.ml.
> Initial questions:
> * Should we use Datasets or RDDs underneath?
> * If Datasets, are there missing features needed for the migration?
> * Do we want to redesign any aspects of the distributed matrices during this move?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org