GraphX :Running on a Cluster

[isaranto <sa...@aueb.gr>]

Hi,
I have been trying to run some algorithms i have implemented using GraphX
and Spark.
I have been running these algorithms locally by starting a local spark
instance through IntelliJ (in scala).
However when I try to run them on a cluster with 10 machines I get 

java.lang.ClassNotFoundException: org.centrality.spark.DegreeCentrality
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

when running 

spark-submit --master spark://sparkmaster:7077 --class
org.centrality.spark.DegreeCentrality target/spark-1.0-SNAPSHOT.jar

the jar is a fat jar which includes all the dependencies..

I also tried running each algorithm as a script which is being loaded in
spark shell ("spark-shell -i DegreeCentrality.scala", I also include the
System.exit(0) at the end.) . However when I ran the script it gets stuck
while the graph is being created "val graph = GraphLoader.edgeListFile(sc,
"hdfs://sparkmaster:7077/user/ilias/graphx/followers.txt"). I have been able
to run a simple WordCount (by reading from an hdfs file as well) example but
have still not been able to create a graph from a textfile.

Thanks a lot...
Ilias




--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/GraphX-Running-on-a-Cluster-tp27236.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org