You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Nick Pentreath (JIRA)" <ji...@apache.org> on 2016/07/01 11:26:11 UTC
[jira] [Commented] (SPARK-15575) Remove breeze from dependencies?
[ https://issues.apache.org/jira/browse/SPARK-15575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358820#comment-15358820 ]
Nick Pentreath commented on SPARK-15575:
----------------------------------------
So it seems from the various discussion on SPARK-15581, SPARK-13944, and due to the issue in SPARK-14438 that there is enough interest and immediacy around this that it may make sense to review linear algebra within Spark and JVM in general, to refresh the current state with respect to features, performance, API / Java compat, and license.
I've created a (very initial and WIP) [Google doc|https://docs.google.com/document/d/1UWhuP-YQXW6D-IZs2Dy9CrdxraUDMFG5DdX3eo6cuEs/edit?usp=sharing] to begin discussion and collating the details. Please feel free to discuss further here or on that doc.
I'm happy to move ahead with the evaluation and performance testing (also happy to split up the work if desired).
> Remove breeze from dependencies?
> --------------------------------
>
> Key: SPARK-15575
> URL: https://issues.apache.org/jira/browse/SPARK-15575
> Project: Spark
> Issue Type: Improvement
> Components: ML
> Reporter: Joseph K. Bradley
>
> This JIRA is for discussing whether we should remove Breeze from the dependencies of MLlib. The main issues with Breeze are Scala 2.12 support and performance issues.
> There are a few paths:
> # Keep dependency. This could be OK, especially if the Scala version issues are fixed within Breeze.
> # Remove dependency
> ## Implement our own linear algebra operators as needed
> ## Design a way to build Spark using custom linalg libraries of the user's choice. E.g., you could build MLlib using Breeze, or any other library supporting the required operations. This might require significant work. See [SPARK-6442] for related discussion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org