You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sean Owen (JIRA)" <ji...@apache.org> on 2019/02/06 16:50:00 UTC
[jira] [Updated] (SPARK-26757) GraphX EdgeRDDImpl and VertexRDDImpl
`count` method cannot handle empty RDDs
[ https://issues.apache.org/jira/browse/SPARK-26757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen updated SPARK-26757:
------------------------------
Fix Version/s: (was: 2.3.4)
2.3.3
> GraphX EdgeRDDImpl and VertexRDDImpl `count` method cannot handle empty RDDs
> ----------------------------------------------------------------------------
>
> Key: SPARK-26757
> URL: https://issues.apache.org/jira/browse/SPARK-26757
> Project: Spark
> Issue Type: Bug
> Components: GraphX
> Affects Versions: 2.3.1, 2.3.2, 2.4.0
> Reporter: Huon Wilson
> Assignee: Huon Wilson
> Priority: Minor
> Fix For: 2.3.3, 2.4.1, 3.0.0
>
>
> The {{EdgeRDDImpl}} and {{VertexRDDImpl}} types provided by {{GraphX}} throw an {{java.lang.UnsupportedOperationException: empty collection}} exception if {{count}} is called on an empty instance, when they should return 0.
> {code:scala}
> import org.apache.spark.graphx.{Graph, Edge}
> val graph = Graph.fromEdges(sc.emptyRDD[Edge[Unit]], 0)
> graph.vertices.count
> graph.edges.count
> {code}
> Running that code in a spark-shell:
> {code:none}
> scala> import org.apache.spark.graphx.{Graph, Edge}
> import org.apache.spark.graphx.{Graph, Edge}
> scala> val graph = Graph.fromEdges(sc.emptyRDD[Edge[Unit]], 0)
> graph: org.apache.spark.graphx.Graph[Int,Unit] = org.apache.spark.graphx.impl.GraphImpl@6879e983
> scala> graph.vertices.count
> java.lang.UnsupportedOperationException: empty collection
> at org.apache.spark.rdd.RDD$$anonfun$reduce$1$$anonfun$apply$36.apply(RDD.scala:1031)
> at org.apache.spark.rdd.RDD$$anonfun$reduce$1$$anonfun$apply$36.apply(RDD.scala:1031)
> at scala.Option.getOrElse(Option.scala:121)
> at org.apache.spark.rdd.RDD$$anonfun$reduce$1.apply(RDD.scala:1031)
> at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
> at org.apache.spark.rdd.RDD.withScope(RDD.scala:363)
> at org.apache.spark.rdd.RDD.reduce(RDD.scala:1011)
> at org.apache.spark.graphx.impl.VertexRDDImpl.count(VertexRDDImpl.scala:90)
> ... 49 elided
> scala> graph.edges.count
> java.lang.UnsupportedOperationException: empty collection
> at org.apache.spark.rdd.RDD$$anonfun$reduce$1$$anonfun$apply$36.apply(RDD.scala:1031)
> at org.apache.spark.rdd.RDD$$anonfun$reduce$1$$anonfun$apply$36.apply(RDD.scala:1031)
> at scala.Option.getOrElse(Option.scala:121)
> at org.apache.spark.rdd.RDD$$anonfun$reduce$1.apply(RDD.scala:1031)
> at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
> at org.apache.spark.rdd.RDD.withScope(RDD.scala:363)
> at org.apache.spark.rdd.RDD.reduce(RDD.scala:1011)
> at org.apache.spark.graphx.impl.EdgeRDDImpl.count(EdgeRDDImpl.scala:90)
> ... 49 elided
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org