You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Xiangrui Meng (JIRA)" <ji...@apache.org> on 2015/01/27 03:20:34 UTC

[jira] [Created] (SPARK-5419) Fix the logic in Vectors.sqdist

Xiangrui Meng created SPARK-5419:
------------------------------------

             Summary: Fix the logic in Vectors.sqdist
                 Key: SPARK-5419
                 URL: https://issues.apache.org/jira/browse/SPARK-5419
             Project: Spark
          Issue Type: Improvement
          Components: MLlib
            Reporter: Xiangrui Meng
            Assignee: Liang-Chi Hsieh


The current implementation of sqdist tries to convert sparse vectors to dense if they are close to dense. This is not efficient because we need to allocate temp arrays. We should simply implement sqdist without allocating new memory.

The current implementation also contains a bug on deciding whether to convert a sparse vector to dense.

{code}
v1.indices.length / v1.size < 0.5
{code}

which should get removed with the changes described above.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org