You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Andrew Otto (JIRA)" <ji...@apache.org> on 2017/03/02 21:42:45 UTC
[jira] [Commented] (SPARK-1693) Dependent on multiple versions of
servlet-api jars lead to throw an SecurityException when Spark built for
hadoop 2.3.0 , 2.4.0
[ https://issues.apache.org/jira/browse/SPARK-1693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15893074#comment-15893074 ]
Andrew Otto commented on SPARK-1693:
------------------------------------
We just upgraded to CDH 5.10, which has Spark 1.6.0, Hadoop 2.6.0, Hive 1.1.0, and Oozie 4.1.0.
We are having trouble running Spark jobs that use HiveContext from Oozie. They run perfectly fine from the CLI with spark-submit, just not in Oozie. We aren't certain that HiveContext is related, but we can reproduce regularly with a job that uses HiveContext.
Anyway, I post this here, because the error we are getting is the same that started this issue:
{code}class "javax.servlet.FilterRegistration"'s signer information does not match signer information of other classes in the same package{code}
I've noticed that the Oozie sharelib includes javax.servlet-3.0.0.v201112011016.jar. I also see that spark-assembly.jar includes a javax.servlet.FilterRegistration class, although its hard for me to tell which version. The jetty pom.xml files in spark-assembly.jar seem to say {{javax.servlet.*;version="2.6.0"}}, but I'm a little green on how all these dependencies get resolved. I don't see any javax.servlet .jars in any of /usr/lib/hadoop* (where CDH installs hadoop jars).
Help! :) If this is not related to this issue, I'll open a new one.
> Dependent on multiple versions of servlet-api jars lead to throw an SecurityException when Spark built for hadoop 2.3.0 , 2.4.0
> --------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-1693
> URL: https://issues.apache.org/jira/browse/SPARK-1693
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Reporter: Guoqiang Li
> Assignee: Guoqiang Li
> Priority: Blocker
> Fix For: 1.0.0
>
> Attachments: log.txt
>
>
> {code}mvn test -Pyarn -Dhadoop.version=2.4.0 -Dyarn.version=2.4.0 > log.txt{code}
> The log:
> {code}
> UnpersistSuite:
> - unpersist RDD *** FAILED ***
> java.lang.SecurityException: class "javax.servlet.FilterRegistration"'s signer information does not match signer information of other classes in the same package
> at java.lang.ClassLoader.checkCerts(ClassLoader.java:952)
> at java.lang.ClassLoader.preDefineClass(ClassLoader.java:666)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:794)
> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org