You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@samza.apache.org by Chad Greenberg <ev...@hotmail.com> on 2017/02/14 05:44:11 UTC

Jar conflicts after upgrading from 0.10.0 to 0.11.0

I received the following error after upgrading from 0.10.0 to 0.11.0. I understand the underlying issue, but I am not finding the exact cause. No other jar contains ServletRegistration except for avax.servlet-3.0.0.v201112011016.jar which is a dependency of samza-core. Hadoop has a dependency on older servlet and jetty jars. Would love to upgrade everything, but many of the transitive dependencies are quite old.

I have seen this error in the mailing list (triggered by an integration test), but there was no resolve. Hoping someone has encountered and fixed this issue.

2017-02-13 21:29:33.397 [main] JobModelManager$ [INFO] Saving task-to-changelog partition mapping: Map(Partition 0 -> 0)
Exception in thread "main" java.lang.SecurityException: class "javax.servlet.ServletRegistration$Dynamic"'s signer information does not match signer information of other classes in the same package
    at java.lang.ClassLoader.checkCerts(ClassLoader.java:895)
    at java.lang.ClassLoader.preDefineClass(ClassLoader.java:665)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:758)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at org.apache.samza.coordinator.server.HttpServer$.$lessinit$greater$default$4(HttpServer.scala:63)
    at org.apache.samza.coordinator.JobModelManager$.getJobCoordinator(JobCoordinator.scala:128)
    at org.apache.samza.coordinator.JobModelManager$.apply(JobCoordinator.scala:109)
    at org.apache.samza.coordinator.JobModelManager$.apply(JobCoordinator.scala:115)
    at org.apache.samza.job.local.ThreadJobFactory.getJob(ThreadJobFactory.scala:40)
    at org.apache.samza.job.JobRunner.run(JobRunner.scala:125)
    at org.apache.samza.job.JobRunner$.main(JobRunner.scala:65)
    at org.apache.samza.job.JobRunner.main(JobRunner.scala)

Re: Jar conflicts after upgrading from 0.10.0 to 0.11.0

Posted by Chad Greenberg <ev...@hotmail.com>.

Thanks Liu. I already know of all that. The conflicts I see are with Samza and the version of Hadoop that Samza depends on. I would love to upgrade Hadoop, but it is Samza that is using the older version.

________________________________
From: Liu Bo <di...@gmail.com>
Sent: Tuesday, February 14, 2017 2:14:05 AM
To: dev@samza.apache.org
Subject: Re: Jar conflicts after upgrading from 0.10.0 to 0.11.0

Hi chad,

there should be conflicts on maven dependency, and it's very likely to be jetty related as you mentioned, we end up remove all other jetty related dependencies and only keep the ones in samza-core

you can check it using mvn dependency:tree
at the project that handles packaging your job's tar.gz and find out which project cause that conflict, then you can remove it using maven exclusion.

ref: https://maven.apache.org/plugins/maven-dependency-plugin/examples/resolving-conflicts-using-the-dependency-tree.html

IDE such as eclipse or idea has a nicer visualization of this dependency tree.  The jetty dependencies in samza-core is attached for reference

On 14 February 2017 at 13:44, Chad Greenberg <ev...@hotmail.com>> wrote:
I received the following error after upgrading from 0.10.0 to 0.11.0. I understand the underlying issue, but I am not finding the exact cause. No other jar contains ServletRegistration except for avax.servlet-3.0.0.v201112011016.jar which is a dependency of samza-core. Hadoop has a dependency on older servlet and jetty jars. Would love to upgrade everything, but many of the transitive dependencies are quite old.

I have seen this error in the mailing list (triggered by an integration test), but there was no resolve. Hoping someone has encountered and fixed this issue.

2017-02-13 21:29:33.397 [main] JobModelManager$ [INFO] Saving task-to-changelog partition mapping: Map(Partition 0 -> 0)
Exception in thread "main" java.lang.SecurityException: class "javax.servlet.ServletRegistration$Dynamic"'s signer information does not match signer information of other classes in the same package
    at java.lang.ClassLoader.checkCerts(ClassLoader.java:895)
    at java.lang.ClassLoader.preDefineClass(ClassLoader.java:665)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:758)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at org.apache.samza.coordinator.server.HttpServer$.$lessinit$greater$default$4(HttpServer.scala:63)
    at org.apache.samza.coordinator.JobModelManager$.getJobCoordinator(JobCoordinator.scala:128)
    at org.apache.samza.coordinator.JobModelManager$.apply(JobCoordinator.scala:109)
    at org.apache.samza.coordinator.JobModelManager$.apply(JobCoordinator.scala:115)
    at org.apache.samza.job.local.ThreadJobFactory.getJob(ThreadJobFactory.scala:40)
    at org.apache.samza.job.JobRunner.run(JobRunner.scala:125)
    at org.apache.samza.job.JobRunner$.main(JobRunner.scala:65)
    at org.apache.samza.job.JobRunner.main(JobRunner.scala)

--
All the best

Liu Bo

Re: Jar conflicts after upgrading from 0.10.0 to 0.11.0

Posted by Liu Bo <di...@gmail.com>.

Hi chad,

there should be conflicts on maven dependency, and it's very likely to be
jetty related as you mentioned, we end up remove all other jetty related
dependencies and only keep the ones in samza-core

you can check it using mvn dependency:tree
at the project that handles packaging your job's tar.gz and find out which
project cause that conflict, then you can remove it using maven exclusion.

ref:
https://maven.apache.org/plugins/maven-dependency-plugin/examples/resolving-conflicts-using-the-dependency-tree.html

IDE such as eclipse or idea has a nicer visualization of this dependency
tree.  The jetty dependencies in samza-core is attached for reference

On 14 February 2017 at 13:44, Chad Greenberg <ev...@hotmail.com>
wrote:

> I received the following error after upgrading from 0.10.0 to 0.11.0. I
> understand the underlying issue, but I am not finding the exact cause. No
> other jar contains ServletRegistration except for avax.servlet-3.0.0.v201112011016.jar
> which is a dependency of samza-core. Hadoop has a dependency on older
> servlet and jetty jars. Would love to upgrade everything, but many of the
> transitive dependencies are quite old.
>
> I have seen this error in the mailing list (triggered by an integration
> test), but there was no resolve. Hoping someone has encountered and fixed
> this issue.
>
> 2017-02-13 21:29:33.397 [main] JobModelManager$ [INFO] Saving
> task-to-changelog partition mapping: Map(Partition 0 -> 0)
> Exception in thread "main" java.lang.SecurityException: class
> "javax.servlet.ServletRegistration$Dynamic"'s signer information does not
> match signer information of other classes in the same package
>     at java.lang.ClassLoader.checkCerts(ClassLoader.java:895)
>     at java.lang.ClassLoader.preDefineClass(ClassLoader.java:665)
>     at java.lang.ClassLoader.defineClass(ClassLoader.java:758)
>     at java.security.SecureClassLoader.defineClass(
> SecureClassLoader.java:142)
>     at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
>     at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
>     at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
>     at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>     at org.apache.samza.coordinator.server.HttpServer$.$lessinit$
> greater$default$4(HttpServer.scala:63)
>     at org.apache.samza.coordinator.JobModelManager$.getJobCoordinator(
> JobCoordinator.scala:128)
>     at org.apache.samza.coordinator.JobModelManager$.apply(
> JobCoordinator.scala:109)
>     at org.apache.samza.coordinator.JobModelManager$.apply(
> JobCoordinator.scala:115)
>     at org.apache.samza.job.local.ThreadJobFactory.getJob(
> ThreadJobFactory.scala:40)
>     at org.apache.samza.job.JobRunner.run(JobRunner.scala:125)
>     at org.apache.samza.job.JobRunner$.main(JobRunner.scala:65)
>     at org.apache.samza.job.JobRunner.main(JobRunner.scala)
>
>


-- 
All the best

Liu Bo