[jira] [Created] (SPARK-15847) DecisionTreeRunner example stucks with "NoClassDefFoundError: org/apache/avro/generic/GenericRecord"

["Yesha Vora (JIRA)" <ji...@apache.org>]

Yesha Vora created SPARK-15847:
----------------------------------

             Summary: DecisionTreeRunner example stucks with "NoClassDefFoundError: org/apache/avro/generic/GenericRecord"
                 Key: SPARK-15847
                 URL: https://issues.apache.org/jira/browse/SPARK-15847
             Project: Spark
          Issue Type: Bug
          Components: Spark Core
            Reporter: Yesha Vora


In Spark-2, DecisionTreeRunner example hangs with "NoClassDefFoundError: org/apache/avro/generic/GenericRecord".

The same application passes in yarn-cluster mode. I'm hitting this issue with yarn-client mode only.

{code}
spark-submit  --class org.apache.spark.examples.mllib.DecisionTreeRunner --master yarn-client --jars hadoop-lzo-*.jar /xxx/lib/spark-examples_*jar  /tmp/sparkMLLInput/sample_libsvm_data.txt{code}

{code}
16/05/27 02:37:50 INFO SparkContext: Starting job: countByValue at DecisionTreeRunner.scala:185
Exception in thread "dag-scheduler-event-loop" java.lang.NoClassDefFoundError: org/apache/avro/generic/GenericRecord
at org.apache.spark.serializer.KryoSerializer.newKryo(KryoSerializer.scala:112)
at org.apache.spark.serializer.KryoSerializerInstance.borrowKryo(KryoSerializer.scala:274)
at org.apache.spark.serializer.KryoSerializerInstance.<init>(KryoSerializer.scala:259)
at org.apache.spark.serializer.KryoSerializer.newInstance(KryoSerializer.scala:175)
at org.apache.spark.serializer.KryoSerializer.supportsRelocationOfSerializedObjects$lzycompute(KryoSerializer.scala:182)
at org.apache.spark.serializer.KryoSerializer.supportsRelocationOfSerializedObjects(KryoSerializer.scala:178)
at org.apache.spark.shuffle.sort.SortShuffleManager$.canUseSerializedShuffle(SortShuffleManager.scala:187)
at org.apache.spark.shuffle.sort.SortShuffleManager.registerShuffle(SortShuffleManager.scala:99)
at org.apache.spark.ShuffleDependency.<init>(Dependency.scala:90)
at org.apache.spark.rdd.ShuffledRDD.getDependencies(ShuffledRDD.scala:91)
at org.apache.spark.rdd.RDD$$anonfun$dependencies$2.apply(RDD.scala:234)
at org.apache.spark.rdd.RDD$$anonfun$dependencies$2.apply(RDD.scala:232)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.rdd.RDD.dependencies(RDD.scala:232)
at org.apache.spark.scheduler.DAGScheduler.visit$1(DAGScheduler.scala:391)
at org.apache.spark.scheduler.DAGScheduler.getParentStages(DAGScheduler.scala:403)
at org.apache.spark.scheduler.DAGScheduler.getParentStagesAndId(DAGScheduler.scala:304)
at org.apache.spark.scheduler.DAGScheduler.newResultStage(DAGScheduler.scala:339)
at org.apache.spark.scheduler.DAGScheduler.handleJobSubmitted(DAGScheduler.scala:849)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1626)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1618)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1607)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
Caused by: java.lang.ClassNotFoundException: org.apache.avro.generic.GenericRecord
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
.. 23 more{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org