You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Sean Owen <so...@cloudera.com> on 2015/05/20 14:03:18 UTC
userClassPathFirst and loader constraint violation
(Marcelo you might have some insight on this one)
Warning: this may just be because I'm doing something non-standard --
trying embed Spark in a Java app and feed it all the classpath it
needs manually. But this was surprising enough I wanted to ask.
I have an app that includes among other things SLF4J. I have set
spark.{driver,executor}.userClassPathFirst to true. If I run it and
let it start a Spark job, it quickly fails with:
2015-05-20 04:35:01,747 WARN TaskSetManager:71 Lost task 0.0 in stage
0.0 (TID 0, x.cloudera.com): java.lang.LinkageError: loader constraint
violation: loader (instance of
org/apache/spark/util/ChildFirstURLClassLoader) previously initiated
loading for a different type with name "org/slf4j/Logger"
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at org.apache.spark.util.ChildFirstURLClassLoader.liftedTree1$1(MutableURLClassLoader.scala:74)
at org.apache.spark.util.ChildFirstURLClassLoader.loadClass(MutableURLClassLoader.scala:73)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at org.apache.spark.streaming.kafka.KafkaRDD.compute(KafkaRDD.scala:89)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
...
I can see that this class was loaded from my app JAR:
[Loaded org.slf4j.Logger from file:/home/sowen/oryx-batch-2.0.0-SNAPSHOT.jar]
I'm assuming it's also loaded in some Spark classloader.
Tracing the code, I don't see that it ever gets to consulting any
other classloader; this happens during its own child-first attempt to
load the class.
This didn't happen in 1.2, FWIW, when the implementation was
different, but that's only to say it was different, not correct.
Anyone have thoughts on what this indicates? something to be expected
or surprising?
I think that disabling userClassPathFirst gets rid of this of course,
although that may cause other issues later.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: userClassPathFirst and loader constraint violation
Posted by Marcelo Vanzin <va...@cloudera.com>.
Hmm... this seems to be particular to logging (KafkaRDD.scala:89 in my tree
is a log statement). I'd expect KafkaRDD to be loaded from the system class
loader - or are you repackaging it in your app?
I'd have to investigate more to come with an accurate explanation here...
but it seems that the initialization of the logging system, which happens
after SparkSubmit runs and sets the context class loader to be an instance
of ChildFirstURLClassLoader, is causing things to blow up. I'll see if I
can spend some cycles coming up with a proper explanation (and hopefully a
fix or workaround).
For now, you could probably avoid this by not repackaging the logging
dependencies in your app.
On Wed, May 20, 2015 at 5:03 AM, Sean Owen <so...@cloudera.com> wrote:
> (Marcelo you might have some insight on this one)
>
> Warning: this may just be because I'm doing something non-standard --
> trying embed Spark in a Java app and feed it all the classpath it
> needs manually. But this was surprising enough I wanted to ask.
>
> I have an app that includes among other things SLF4J. I have set
> spark.{driver,executor}.userClassPathFirst to true. If I run it and
> let it start a Spark job, it quickly fails with:
>
> 2015-05-20 04:35:01,747 WARN TaskSetManager:71 Lost task 0.0 in stage
> 0.0 (TID 0, x.cloudera.com): java.lang.LinkageError: loader constraint
> violation: loader (instance of
> org/apache/spark/util/ChildFirstURLClassLoader) previously initiated
> loading for a different type with name "org/slf4j/Logger"
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at
> org.apache.spark.util.ChildFirstURLClassLoader.liftedTree1$1(MutableURLClassLoader.scala:74)
> at
> org.apache.spark.util.ChildFirstURLClassLoader.loadClass(MutableURLClassLoader.scala:73)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at org.apache.spark.streaming.kafka.KafkaRDD.compute(KafkaRDD.scala:89)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
> ...
>
> I can see that this class was loaded from my app JAR:
>
> [Loaded org.slf4j.Logger from
> file:/home/sowen/oryx-batch-2.0.0-SNAPSHOT.jar]
>
> I'm assuming it's also loaded in some Spark classloader.
> Tracing the code, I don't see that it ever gets to consulting any
> other classloader; this happens during its own child-first attempt to
> load the class.
>
> This didn't happen in 1.2, FWIW, when the implementation was
> different, but that's only to say it was different, not correct.
>
> Anyone have thoughts on what this indicates? something to be expected
> or surprising?
>
> I think that disabling userClassPathFirst gets rid of this of course,
> although that may cause other issues later.
>
--
Marcelo