[jira] [Commented] (SPARK-6022) GraphX `diff` test incorrectly operating on values (not VertexId's)

["Takeshi Yamamuro (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-6022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344699#comment-14344699 ] 

Takeshi Yamamuro commented on SPARK-6022:
-----------------------------------------

Is the test correct?
According to the code below, 'diff' is assumed to have same indices in VertexPartitionBase.
That is,  SetA and SetB has the same set of VertexIDs (Im not sure that this behaviour totally correct though).  

https://github.com/apache/spark/blob/master/graphx/src/main/scala/org/apache/spark/graphx/impl/VertexPartitionBaseOps.scala#L93

> GraphX `diff` test incorrectly operating on values (not VertexId's)
> -------------------------------------------------------------------
>
>                 Key: SPARK-6022
>                 URL: https://issues.apache.org/jira/browse/SPARK-6022
>             Project: Spark
>          Issue Type: Bug
>          Components: GraphX
>            Reporter: Brennon York
>
> The current GraphX {{diff}} test operates on values rather than the VertexId's and, if {{diff}} were working properly (per [SPARK-4600|https://issues.apache.org/jira/browse/SPARK-4600]), it should fail this test. The code to test {{diff}} should look like the below as it correctly generates {{VertexRDD}}'s with different {{VertexId}}'s to {{diff}} against.
> {code}
> test("diff functionality with small concrete values") {
>     withSpark { sc =>
>       val setA: VertexRDD[Int] = VertexRDD(sc.parallelize(0L until 2L).map(id => (id, id.toInt)))
>       // setA := Set((0L, 0), (1L, 1))
>       val setB: VertexRDD[Int] = VertexRDD(sc.parallelize(1L until 3L).map(id => (id, id.toInt+2)))
>       // setB := Set((1L, 3), (2L, 4))
>       val diff = setA.diff(setB)
>       assert(diff.collect.toSet == Set((2L, 4)))
>     }
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org