You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Xiangrui Meng (JIRA)" <ji...@apache.org> on 2015/08/14 00:50:46 UTC
[jira] [Comment Edited] (SPARK-9750) SparseMatrix should override
equals
[ https://issues.apache.org/jira/browse/SPARK-9750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696101#comment-14696101 ]
Xiangrui Meng edited comment on SPARK-9750 at 8/13/15 10:50 PM:
----------------------------------------------------------------
This is really messy ... Using your terminology, this is what I found:
In 1.4 (which is the same as before this PR):
* Python
* DenseVector: Semantic eq but only with `DenseVector`. Default hash. -> *bug*
* SparseVector: Semantic eq but *wrong* (only with `SparseVector` and not handling explicit zeros). Default hash. -> *bug*
* DenseMatrix: Semantic eq but only with `DenseMatrix`. Default hash. -> *bug*
* SparseMatrix: Semantic eq but converting to dense. Default hash. -> *bug*
* Scala
* Vectors: Semantic eq. Hash uses first 16 entries.
* DenseMatrix: Semantic eq but *wrong* (only with `DenseMatrix` and not handling transpose). Guava Objects.hashCode (similar to default hash). -> *bug*
* SparseMatrix: Literal eq. Default hash.
was (Author: mengxr):
This is really messy ... Using your terminology, this is what I found:
In 1.4 (which is the same as before this PR):
* Python
* DenseVector: Semantic eq but only with `DenseVector`. Default hash.
* SparseVector: Semantic eq but *wrong* (only with `SparseVector` and not handling explicit zeros). Default hash.
* DenseMatrix: Semantic eq but only with `DenseMatrix`. Default hash.
* SparseMatrix: Semantic eq but converting to dense. Default hash.
* Scala
* Vectors: Semantic eq. Hash uses first 16 entries.
* DenseMatrix: Semantic eq but *wrong* (only with `DenseMatrix` and not handling transpose). Guava Objects.hashCode (similar to default hash).
* SparseMatrix: Literal eq. Default hash.
> SparseMatrix should override equals
> -----------------------------------
>
> Key: SPARK-9750
> URL: https://issues.apache.org/jira/browse/SPARK-9750
> Project: Spark
> Issue Type: Bug
> Components: MLlib
> Reporter: Feynman Liang
> Assignee: Feynman Liang
> Priority: Critical
> Fix For: 1.5.0
>
>
> [SparseMatrix|https://github.com/apache/spark/blob/9897cc5e3d6c70f7e45e887e2c6fc24dfa1adada/mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala#L479] should override equals to ensure that two instances of the same matrix are equal.
> This implementation should take into account the {{isTransposed}} flag and {{values}} may not be in the same order.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org