You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Kay Ousterhout <ke...@eecs.berkeley.edu> on 2014/05/28 07:08:12 UTC
FYI -- javax.servlet dependency issue workaround
Hi all,
I had some trouble compiling an application (Shark) against Spark 1.0,
where Shark had a runtime exception (at the bottom of this message) because
it couldn't find the javax.servlet classes. SBT seemed to have trouble
downloading the servlet APIs that are dependencies of Jetty (used by the
Spark web UI), so I had to manually add them to the application's build
file:
libraryDependencies += "org.mortbay.jetty" % "servlet-api" % "3.0.20100224"
Not exactly sure why this happens but thought it might be useful in case
others run into the same problem.
-Kay
-------------
Exception in thread "main" java.lang.NoClassDefFoundError:
javax/servlet/FilterRegistration
at
org.eclipse.jetty.servlet.ServletContextHandler.<init>(ServletContextHandler.java:136)
at
org.eclipse.jetty.servlet.ServletContextHandler.<init>(ServletContextHandler.java:129)
at
org.eclipse.jetty.servlet.ServletContextHandler.<init>(ServletContextHandler.java:98)
at org.apache.spark.ui.JettyUtils$.createServletHandler(JettyUtils.scala:98)
at org.apache.spark.ui.JettyUtils$.createServletHandler(JettyUtils.scala:89)
at org.apache.spark.ui.WebUI.attachPage(WebUI.scala:64)
at org.apache.spark.ui.WebUI$anonfun$attachTab$1.apply(WebUI.scala:57)
at org.apache.spark.ui.WebUI$anonfun$attachTab$1.apply(WebUI.scala:57)
at
scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
at org.apache.spark.ui.WebUI.attachTab(WebUI.scala:57)
at org.apache.spark.ui.SparkUI.initialize(SparkUI.scala:66)
at org.apache.spark.ui.SparkUI.<init>(SparkUI.scala:60)
at org.apache.spark.ui.SparkUI.<init>(SparkUI.scala:42)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:222)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:85)
at shark.SharkContext.<init>(SharkContext.scala:42)
at shark.SharkContext.<init>(SharkContext.scala:61)
at shark.SharkEnv$.initWithSharkContext(SharkEnv.scala:78)
at shark.SharkEnv$.init(SharkEnv.scala:38)
at shark.SharkCliDriver.<init>(SharkCliDriver.scala:280)
at shark.SharkCliDriver$.main(SharkCliDriver.scala:162)
at shark.SharkCliDriver.main(SharkCliDriver.scala)
Caused by: java.lang.ClassNotFoundException:
javax.servlet.FilterRegistration
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 23 more
Re: FYI -- javax.servlet dependency issue workaround
Posted by Prashant Sharma <sc...@gmail.com>.
Also just for sake of completeness, sometimes the desired dependency might
just be an older version in that case even if you include it like above it
may get evicted (Sbt's default strategy for conflict manager is to choose
the latest version).
So to further ensure that it does include it. We can
libraryDependencies += "org.mortbay.jetty" % "servlet-api" % "3.0.20100224"
*force()*
force it.
Thanks
Prashant Sharma
On Wed, May 28, 2014 at 10:38 AM, Kay Ousterhout <ke...@eecs.berkeley.edu>wrote:
> Hi all,
>
> I had some trouble compiling an application (Shark) against Spark 1.0,
> where Shark had a runtime exception (at the bottom of this message) because
> it couldn't find the javax.servlet classes. SBT seemed to have trouble
> downloading the servlet APIs that are dependencies of Jetty (used by the
> Spark web UI), so I had to manually add them to the application's build
> file:
>
> libraryDependencies += "org.mortbay.jetty" % "servlet-api" % "3.0.20100224"
>
> Not exactly sure why this happens but thought it might be useful in case
> others run into the same problem.
>
> -Kay
>
> -------------
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> javax/servlet/FilterRegistration
>
> at
>
> org.eclipse.jetty.servlet.ServletContextHandler.<init>(ServletContextHandler.java:136)
>
> at
>
> org.eclipse.jetty.servlet.ServletContextHandler.<init>(ServletContextHandler.java:129)
>
> at
>
> org.eclipse.jetty.servlet.ServletContextHandler.<init>(ServletContextHandler.java:98)
>
> at
> org.apache.spark.ui.JettyUtils$.createServletHandler(JettyUtils.scala:98)
>
> at
> org.apache.spark.ui.JettyUtils$.createServletHandler(JettyUtils.scala:89)
>
> at org.apache.spark.ui.WebUI.attachPage(WebUI.scala:64)
>
> at org.apache.spark.ui.WebUI$anonfun$attachTab$1.apply(WebUI.scala:57)
>
> at org.apache.spark.ui.WebUI$anonfun$attachTab$1.apply(WebUI.scala:57)
>
> at
>
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
>
> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
>
> at org.apache.spark.ui.WebUI.attachTab(WebUI.scala:57)
>
> at org.apache.spark.ui.SparkUI.initialize(SparkUI.scala:66)
>
> at org.apache.spark.ui.SparkUI.<init>(SparkUI.scala:60)
>
> at org.apache.spark.ui.SparkUI.<init>(SparkUI.scala:42)
>
> at org.apache.spark.SparkContext.<init>(SparkContext.scala:222)
>
> at org.apache.spark.SparkContext.<init>(SparkContext.scala:85)
>
> at shark.SharkContext.<init>(SharkContext.scala:42)
>
> at shark.SharkContext.<init>(SharkContext.scala:61)
>
> at shark.SharkEnv$.initWithSharkContext(SharkEnv.scala:78)
>
> at shark.SharkEnv$.init(SharkEnv.scala:38)
>
> at shark.SharkCliDriver.<init>(SharkCliDriver.scala:280)
>
> at shark.SharkCliDriver$.main(SharkCliDriver.scala:162)
>
> at shark.SharkCliDriver.main(SharkCliDriver.scala)
>
> Caused by: java.lang.ClassNotFoundException:
> javax.servlet.FilterRegistration
>
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>
> at java.security.AccessController.doPrivileged(Native Method)
>
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>
> ... 23 more
>
Re: FYI -- javax.servlet dependency issue workaround
Posted by Sean Owen <so...@cloudera.com>.
This class was introduced in Servlet 3.0. We have in the dependency
tree some references to Servlet 2.5 and Servlet 3.0. The latter is a
superset of the former. So we standardized on depending on Servlet
3.0.
At least, that seems to have been successful in the Maven build, but
this is just evidence that the SBT build ends up including Servlet 2.5
dependencies.
You shouldn't have to work around it in this way. Let me see if I can
debug and propose the right exclusion for SBT.
(Related: is the SBT build going to continue to live separately from
Maven, or is it going to be auto-generated? that is -- worth fixing
this?)
On Wed, May 28, 2014 at 6:08 AM, Kay Ousterhout <ke...@eecs.berkeley.edu> wrote:
> Hi all,
>
> I had some trouble compiling an application (Shark) against Spark 1.0,
> where Shark had a runtime exception (at the bottom of this message) because
> it couldn't find the javax.servlet classes. SBT seemed to have trouble
> downloading the servlet APIs that are dependencies of Jetty (used by the
> Spark web UI), so I had to manually add them to the application's build
> file:
>
> libraryDependencies += "org.mortbay.jetty" % "servlet-api" % "3.0.20100224"
>
> Not exactly sure why this happens but thought it might be useful in case
> others run into the same problem.
>
> -Kay
>
> -------------
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> javax/servlet/FilterRegistration
>
> at
> org.eclipse.jetty.servlet.ServletContextHandler.<init>(ServletContextHandler.java:136)
>
> at
> org.eclipse.jetty.servlet.ServletContextHandler.<init>(ServletContextHandler.java:129)
>
> at
> org.eclipse.jetty.servlet.ServletContextHandler.<init>(ServletContextHandler.java:98)
>
> at org.apache.spark.ui.JettyUtils$.createServletHandler(JettyUtils.scala:98)
>
> at org.apache.spark.ui.JettyUtils$.createServletHandler(JettyUtils.scala:89)
>
> at org.apache.spark.ui.WebUI.attachPage(WebUI.scala:64)
>
> at org.apache.spark.ui.WebUI$anonfun$attachTab$1.apply(WebUI.scala:57)
>
> at org.apache.spark.ui.WebUI$anonfun$attachTab$1.apply(WebUI.scala:57)
>
> at
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
>
> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
>
> at org.apache.spark.ui.WebUI.attachTab(WebUI.scala:57)
>
> at org.apache.spark.ui.SparkUI.initialize(SparkUI.scala:66)
>
> at org.apache.spark.ui.SparkUI.<init>(SparkUI.scala:60)
>
> at org.apache.spark.ui.SparkUI.<init>(SparkUI.scala:42)
>
> at org.apache.spark.SparkContext.<init>(SparkContext.scala:222)
>
> at org.apache.spark.SparkContext.<init>(SparkContext.scala:85)
>
> at shark.SharkContext.<init>(SharkContext.scala:42)
>
> at shark.SharkContext.<init>(SharkContext.scala:61)
>
> at shark.SharkEnv$.initWithSharkContext(SharkEnv.scala:78)
>
> at shark.SharkEnv$.init(SharkEnv.scala:38)
>
> at shark.SharkCliDriver.<init>(SharkCliDriver.scala:280)
>
> at shark.SharkCliDriver$.main(SharkCliDriver.scala:162)
>
> at shark.SharkCliDriver.main(SharkCliDriver.scala)
>
> Caused by: java.lang.ClassNotFoundException:
> javax.servlet.FilterRegistration
>
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>
> at java.security.AccessController.doPrivileged(Native Method)
>
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>
> ... 23 more