You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Reza zade <kn...@gmail.com> on 2016/12/21 13:02:55 UTC
NoClassDefFoundError
Hello
I've extended the JavaSparkJob (job-server-0.6.2) and created an object
of SQLContext class. my maven project doesn't have any problem during
compile and packaging phase. but when I send .jar of project to sjs and run
it "NoClassDefFoundError" will be issued. the trace of exception is :
job-server[ERROR] Exception in thread "pool-20-thread-1"
java.lang.NoClassDefFoundError:
org/apache/spark/sql/SQLContext
job-server[ERROR] at sparkdesk.SparkSQLJob2.runJob(SparkSQLJob2.java:61)
job-server[ERROR] at sparkdesk.SparkSQLJob2.runJob(SparkSQLJob2.java:45)
job-server[ERROR] at spark.jobserver.JavaSparkJob.
runJob(JavaSparkJob.scala:17)
job-server[ERROR] at spark.jobserver.JavaSparkJob.
runJob(JavaSparkJob.scala:14)
job-server[ERROR] at spark.jobserver.JobManagerActor$$anonfun$
spark$jobserver$JobManagerActor$$getJobFuture$4.apply(JobManagerActor.scala:
301)
job-server[ERROR] at scala.concurrent.impl.Future$
PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
job-server[ERROR] at scala.concurrent.impl.Future$
PromiseCompletingRunnable.run(Future.scala:24)
job-server[ERROR] at java.util.concurrent.ThreadPoolExecutor.runWorker(
ThreadPoolExecutor.java:1145)
job-server[ERROR] at java.util.concurrent.ThreadPoolExecutor$Worker.run(
ThreadPoolExecutor.java:615)
job-server[ERROR] at java.lang.Thread.run(Thread.java:745)
job-server[ERROR] Caused by: java.lang.ClassNotFoundException:
org.apache.spark.sql.SQLContext
job-server[ERROR] at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
job-server[ERROR] at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
job-server[ERROR] at java.security.AccessController.doPrivileged(Native
Method)
job-server[ERROR] at java.net.URLClassLoader.findClass(URLClassLoader.java:
354)
job-server[ERROR] at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
job-server[ERROR] at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
job-server[ERROR] ... 10 more
what is the problem?
do you have any solution about this?
Re: NoClassDefFoundError
Posted by Vadim Semenov <va...@datadoghq.com>.
You better ask folks in the spark-jobserver gitter channel:
https://github.com/spark-jobserver/spark-jobserver
On Wed, Dec 21, 2016 at 8:02 AM, Reza zade <kn...@gmail.com> wrote:
> Hello
>
> I've extended the JavaSparkJob (job-server-0.6.2) and created an object
> of SQLContext class. my maven project doesn't have any problem during
> compile and packaging phase. but when I send .jar of project to sjs and run
> it "NoClassDefFoundError" will be issued. the trace of exception is :
>
>
> job-server[ERROR] Exception in thread "pool-20-thread-1"
> java.lang.NoClassDefFoundError: org/apache/spark/sql/SQLContext
> job-server[ERROR] at sparkdesk.SparkSQLJob2.runJob(SparkSQLJob2.java:61)
> job-server[ERROR] at sparkdesk.SparkSQLJob2.runJob(SparkSQLJob2.java:45)
> job-server[ERROR] at spark.jobserver.JavaSparkJob.r
> unJob(JavaSparkJob.scala:17)
> job-server[ERROR] at spark.jobserver.JavaSparkJob.r
> unJob(JavaSparkJob.scala:14)
> job-server[ERROR] at spark.jobserver.JobManagerActo
> r$$anonfun$spark$jobserver$JobManagerActor$$getJobFuture$4.
> apply(JobManagerActor.scala:301)
> job-server[ERROR] at scala.concurrent.impl.Future$P
> romiseCompletingRunnable.liftedTree1$1(Future.scala:24)
> job-server[ERROR] at scala.concurrent.impl.Future$P
> romiseCompletingRunnable.run(Future.scala:24)
> job-server[ERROR] at java.util.concurrent.ThreadPoo
> lExecutor.runWorker(ThreadPoolExecutor.java:1145)
> job-server[ERROR] at java.util.concurrent.ThreadPoo
> lExecutor$Worker.run(ThreadPoolExecutor.java:615)
> job-server[ERROR] at java.lang.Thread.run(Thread.java:745)
> job-server[ERROR] Caused by: java.lang.ClassNotFoundException:
> org.apache.spark.sql.SQLContext
> job-server[ERROR] at java.net.URLClassLoader$1.run(
> URLClassLoader.java:366)
> job-server[ERROR] at java.net.URLClassLoader$1.run(
> URLClassLoader.java:355)
> job-server[ERROR] at java.security.AccessController.doPrivileged(Native
> Method)
> job-server[ERROR] at java.net.URLClassLoader.findCl
> ass(URLClassLoader.java:354)
> job-server[ERROR] at java.lang.ClassLoader.loadClas
> s(ClassLoader.java:425)
> job-server[ERROR] at java.lang.ClassLoader.loadClas
> s(ClassLoader.java:358)
> job-server[ERROR] ... 10 more
>
>
> what is the problem?
> do you have any solution about this?
>