[jira] [Resolved] (SPARK-5419) Fix the logic in Vectors.sqdist

["Xiangrui Meng (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Xiangrui Meng resolved SPARK-5419.
----------------------------------
       Resolution: Fixed
    Fix Version/s: 1.3.0

Issue resolved by pull request 4217
[https://github.com/apache/spark/pull/4217]

> Fix the logic in Vectors.sqdist
> -------------------------------
>
>                 Key: SPARK-5419
>                 URL: https://issues.apache.org/jira/browse/SPARK-5419
>             Project: Spark
>          Issue Type: Improvement
>          Components: MLlib
>            Reporter: Xiangrui Meng
>            Assignee: Liang-Chi Hsieh
>             Fix For: 1.3.0
>
>
> The current implementation of sqdist tries to convert sparse vectors to dense if they are close to dense. This is not efficient because we need to allocate temp arrays. We should simply implement sqdist without allocating new memory.
> The current implementation also contains a bug on deciding whether to convert a sparse vector to dense.
> {code}
> v1.indices.length / v1.size < 0.5
> {code}
> which should get removed with the changes described above.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org