[jira] [Updated] (SPARK-12372) Document limitations of MLlib local linear algebra

["Joseph K. Bradley (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-12372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Joseph K. Bradley updated SPARK-12372:
--------------------------------------
    Summary: Document limitations of MLlib local linear algebra  (was: Document limitations of MLlib linear algebra)

> Document limitations of MLlib local linear algebra
> --------------------------------------------------
>
>                 Key: SPARK-12372
>                 URL: https://issues.apache.org/jira/browse/SPARK-12372
>             Project: Spark
>          Issue Type: Documentation
>          Components: Documentation, MLlib
>    Affects Versions: 1.5.2
>            Reporter: Christos Iraklis Tsatsoulis
>
> This JIRA is now for documenting limitations of MLlib's local linear algebra types.  Basically, we should make it clear in the user guide that they provide simple functionality but are not a full-fledged local linear library.  We should also recommend libraries for users to use in the meantime: probably Breeze for Scala (and Java?) and numpy/scipy for Python.
> *Original JIRA title*: Unary operator "-" fails for MLlib vectors
> *Original JIRA text, as an example of the need for better docs*:
> Consider the following snippet in pyspark 1.5.2:
> {code:none}
> >>> from pyspark.mllib.linalg import Vectors
> >>> x = Vectors.dense([0.0, 1.0, 0.0, 7.0, 0.0])
> >>> x
> DenseVector([0.0, 1.0, 0.0, 7.0, 0.0])
> >>> -x
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: func() takes exactly 2 arguments (1 given)
> >>> y = Vectors.dense([2.0, 0.0, 3.0, 4.0, 5.0])
> >>> y
> DenseVector([2.0, 0.0, 3.0, 4.0, 5.0])
> >>> x-y
> DenseVector([-2.0, 1.0, -3.0, 3.0, -5.0])
> >>> -y+x
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: func() takes exactly 2 arguments (1 given)
> >>> -1*x
> DenseVector([-0.0, -1.0, -0.0, -7.0, -0.0])
> {code}
> Clearly, the unary operator {{-}} (minus) for vectors fails, giving errors for expressions like {{-x}} and {{-y+x}}, despite the fact that {{x-y}} behaves as expected.
> The last operation, {{-1*x}}, although mathematically "correct", includes minus signs for the zero entries, which again is normally not expected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org