You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Yafeng Guo <da...@gmail.com> on 2015/02/01 05:01:06 UTC
Intellij IDEA 14 env setup; NoClassDefFoundError when run examples
Hi,
I'm setting up a dev environment with Intellij IDEA 14. I selected profile
scala-2.10, maven-3, hadoop 2.4, hive, hive 0.13.1. The compilation passed.
But when I try to run LogQuery in examples, I met below issue:
Connected to the target VM, address: '127.0.0.1:37182', transport: 'socket'
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/spark/SparkConf
at org.apache.spark.examples.LogQuery$.main(LogQuery.scala:46)
at org.apache.spark.examples.LogQuery.main(LogQuery.scala)
Caused by: java.lang.ClassNotFoundException: org.apache.spark.SparkConf
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 2 more
Disconnected from the target VM, address: '127.0.0.1:37182', transport:
'socket'
anyone met similar issue before? Thanks a lot
Regards,
Ya-Feng
Re: Intellij IDEA 14 env setup; NoClassDefFoundError when run examples
Posted by Yafeng Guo <da...@gmail.com>.
Finally it works.
@Sean, I'm trying to setup env in IDE so I can track into to Spark -- that
will help me understand Spark internal mechanism.
@Ted, thanks. I'm using Maven, not SBT, but thanks for the suggestion
anyway.
For others who might interested in:
I choose "bigtop-dist" profile so under "Spark-Assembly" maven will build a
fat jar file, which include all artifacts built. Then I added such fat jar
as a dependency of examples module, set it as "runtime".
Then I find I can debug and track code step by step.
This is maybe not a formal way but at least works for me.
Regards,
Ya-Feng
On Sun, Feb 1, 2015 at 6:44 PM, Sean Owen <so...@cloudera.com> wrote:
> How do you mean you run LogQuery? you would run these using the
> run-example script rather than in IntelliJ.
>
> On Sun, Feb 1, 2015 at 4:01 AM, Yafeng Guo <da...@gmail.com>
> wrote:
> > Hi,
> >
> > I'm setting up a dev environment with Intellij IDEA 14. I selected
> profile
> > scala-2.10, maven-3, hadoop 2.4, hive, hive 0.13.1. The compilation
> passed.
> > But when I try to run LogQuery in examples, I met below issue:
> >
> > Connected to the target VM, address: '127.0.0.1:37182', transport:
> 'socket'
> > Exception in thread "main" java.lang.NoClassDefFoundError:
> > org/apache/spark/SparkConf
> > at org.apache.spark.examples.LogQuery$.main(LogQuery.scala:46)
> > at org.apache.spark.examples.LogQuery.main(LogQuery.scala)
> > Caused by: java.lang.ClassNotFoundException: org.apache.spark.SparkConf
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > ... 2 more
> > Disconnected from the target VM, address: '127.0.0.1:37182', transport:
> > 'socket'
> >
> > anyone met similar issue before? Thanks a lot
> >
> > Regards,
> > Ya-Feng
>
Re: Intellij IDEA 14 env setup; NoClassDefFoundError when run examples
Posted by Sean Owen <so...@cloudera.com>.
How do you mean you run LogQuery? you would run these using the
run-example script rather than in IntelliJ.
On Sun, Feb 1, 2015 at 4:01 AM, Yafeng Guo <da...@gmail.com> wrote:
> Hi,
>
> I'm setting up a dev environment with Intellij IDEA 14. I selected profile
> scala-2.10, maven-3, hadoop 2.4, hive, hive 0.13.1. The compilation passed.
> But when I try to run LogQuery in examples, I met below issue:
>
> Connected to the target VM, address: '127.0.0.1:37182', transport: 'socket'
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/spark/SparkConf
> at org.apache.spark.examples.LogQuery$.main(LogQuery.scala:46)
> at org.apache.spark.examples.LogQuery.main(LogQuery.scala)
> Caused by: java.lang.ClassNotFoundException: org.apache.spark.SparkConf
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> ... 2 more
> Disconnected from the target VM, address: '127.0.0.1:37182', transport:
> 'socket'
>
> anyone met similar issue before? Thanks a lot
>
> Regards,
> Ya-Feng
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: Intellij IDEA 14 env setup; NoClassDefFoundError when run examples
Posted by Ted Yu <yu...@gmail.com>.
Have you read / followed this ?
https://cwiki.apache.org/confluence/display/SPARK
/Useful+Developer+Tools#UsefulDeveloperTools-BuildingSparkinIntelliJIDEA
Cheers
On Sat, Jan 31, 2015 at 8:01 PM, Yafeng Guo <da...@gmail.com>
wrote:
> Hi,
>
> I'm setting up a dev environment with Intellij IDEA 14. I selected profile
> scala-2.10, maven-3, hadoop 2.4, hive, hive 0.13.1. The compilation passed.
> But when I try to run LogQuery in examples, I met below issue:
>
> Connected to the target VM, address: '127.0.0.1:37182', transport:
> 'socket'
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/spark/SparkConf
> at org.apache.spark.examples.LogQuery$.main(LogQuery.scala:46)
> at org.apache.spark.examples.LogQuery.main(LogQuery.scala)
> Caused by: java.lang.ClassNotFoundException: org.apache.spark.SparkConf
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> ... 2 more
> Disconnected from the target VM, address: '127.0.0.1:37182', transport:
> 'socket'
>
> anyone met similar issue before? Thanks a lot
>
> Regards,
> Ya-Feng
>