You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Huon Wilson (JIRA)" <ji...@apache.org> on 2019/01/29 03:33:00 UTC
[jira] [Created] (SPARK-26757) GraphX EdgeRDDImpl and VertexRDDImpl
`count` method cannot handle empty RDDs
Huon Wilson created SPARK-26757:
-----------------------------------
Summary: GraphX EdgeRDDImpl and VertexRDDImpl `count` method cannot handle empty RDDs
Key: SPARK-26757
URL: https://issues.apache.org/jira/browse/SPARK-26757
Project: Spark
Issue Type: Bug
Components: GraphX
Affects Versions: 2.4.0, 2.3.2, 2.3.1
Reporter: Huon Wilson
The {{EdgeRDDImpl}} and {{VertexRDDImpl}} types provided by {{GraphX}} throw an {{java.lang.UnsupportedOperationException: empty collection}} exception if {{count}} is called on an empty instance, when they should return 0.
{code:scala}
import org.apache.spark.graphx.{Graph, Edge}
val graph = Graph.fromEdges(sc.emptyRDD[Edge[Unit]], 0)
graph.vertices.count
graph.edges.count
{code}
Running that code in a spark-shell:
{code:none}
scala> import org.apache.spark.graphx.{Graph, Edge}
import org.apache.spark.graphx.{Graph, Edge}
scala> val graph = Graph.fromEdges(sc.emptyRDD[Edge[Unit]], 0)
graph: org.apache.spark.graphx.Graph[Int,Unit] = org.apache.spark.graphx.impl.GraphImpl@6879e983
scala> graph.vertices.count
java.lang.UnsupportedOperationException: empty collection
at org.apache.spark.rdd.RDD$$anonfun$reduce$1$$anonfun$apply$36.apply(RDD.scala:1031)
at org.apache.spark.rdd.RDD$$anonfun$reduce$1$$anonfun$apply$36.apply(RDD.scala:1031)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.rdd.RDD$$anonfun$reduce$1.apply(RDD.scala:1031)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:363)
at org.apache.spark.rdd.RDD.reduce(RDD.scala:1011)
at org.apache.spark.graphx.impl.VertexRDDImpl.count(VertexRDDImpl.scala:90)
... 49 elided
scala> graph.edges.count
java.lang.UnsupportedOperationException: empty collection
at org.apache.spark.rdd.RDD$$anonfun$reduce$1$$anonfun$apply$36.apply(RDD.scala:1031)
at org.apache.spark.rdd.RDD$$anonfun$reduce$1$$anonfun$apply$36.apply(RDD.scala:1031)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.rdd.RDD$$anonfun$reduce$1.apply(RDD.scala:1031)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:363)
at org.apache.spark.rdd.RDD.reduce(RDD.scala:1011)
at org.apache.spark.graphx.impl.EdgeRDDImpl.count(EdgeRDDImpl.scala:90)
... 49 elided
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org