You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Krishnanand Khambadkone <kk...@yahoo.com.INVALID> on 2015/03/03 23:36:12 UTC

java.lang.NoClassDefFoundError: org/apache/spark/streaming/kafka/KafkaUtils

Hi,  When I submit my spark job, I see the following runtime exception in the log,
Exception in thread "Thread-1" java.lang.NoClassDefFoundError: org/apache/spark/streaming/kafka/KafkaUtils
	at SparkHdfs.run(SparkHdfs.java:56)
Caused by: java.lang.ClassNotFoundException: org.apache.spark.streaming.kafka.KafkaUtils
	at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:358)I am including the assembly and streaming jars during submit,
./spark-submit --jars/Users/hadoop/kafka-0.8.1.1-src/core/build/libs/kafka_2.8.0-0.8.1.1.jar,/Users/hadoop/spark-1.2.1/external/kafka/target/spark-streaming-kafka_2.10-1.2.1.jar--class KafkaMain --master spark://krishs-mbp:7077 --deploy-mode cluster--num-executors 100 --driver-memory 1g --executor-memory 1g --supervisefile:///Users/hadoop/dev/kafkahbase/kafka/src/sparkhbase.jar