You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ratis.apache.org by Henrik Hegardt <he...@gmail.com> on 2019/09/29 18:11:10 UTC
ClassNotFoundException in Ratis
Hi everyone,
I'm getting this error while running Ratis:
Caused by: java.io.IOException: java.lang.NoClassDefFoundError:
org/apache/hadoop/metrics2/lib/DefaultMetricsSystem
at org.apache.ratis.util.IOUtils.asIOException(IOUtils.java:54)
at org.apache.ratis.util.IOUtils.toIOException(IOUtils.java:61)
at org.apache.ratis.util.IOUtils.getFromFuture(IOUtils.java:70)
at
org.apache.ratis.server.impl.RaftServerProxy.getImpls(RaftServerProxy.java:284)
at
org.apache.ratis.server.impl.RaftServerProxy.start(RaftServerProxy.java:296)
at test.cluster.ClusterSevice.start(ClusterSevice.java:85)
... 65 more
Caused by: java.lang.NoClassDefFoundError:
org/apache/hadoop/metrics2/lib/DefaultMetricsSystem
at
org.apache.ratis.metrics.MetricsReporting.<clinit>(MetricsReporting.java:42)
at
org.apache.ratis.server.metrics.RatisMetrics.<clinit>(RatisMetrics.java:39)
at
org.apache.ratis.server.raftlog.segmented.SegmentedRaftLogWorker.<init>(SegmentedRaftLogWorker.java:176)
at
org.apache.ratis.server.raftlog.segmented.SegmentedRaftLog.<init>(SegmentedRaftLog.java:197)
at
org.apache.ratis.server.raftlog.segmented.SegmentedRaftLog.<init>(SegmentedRaftLog.java:183)
at
org.apache.ratis.server.impl.ServerState.initRaftLog(ServerState.java:188)
at
org.apache.ratis.server.impl.ServerState.<init>(ServerState.java:120)
at
org.apache.ratis.server.impl.RaftServerImpl.<init>(RaftServerImpl.java:110)
at
org.apache.ratis.server.impl.RaftServerProxy.lambda$newRaftServerImpl$2(RaftServerProxy.java:208)
at
java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem
at
java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583)
at
java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
... 13 more
AFAICT this is a class in the dependency
org.apache.hadoop:hadoop-common:3.1.1. I'm running with the dependencies
and Ratis 0.4.0
<dependency>
<artifactId>ratis-server</artifactId>
<groupId>org.apache.ratis</groupId>
<exclusions>
<exclusion>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
</exclusion>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<artifactId>ratis-grpc</artifactId>
<groupId>org.apache.ratis</groupId>
</dependency>
And which should be enough according to the documentation. While it's
possible to include this dependency, it pulls in a lot of other
dependencies. While org.apache.ratis.metrics.MetricsReporting class does
have a dependency on the
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem class, ratis-metrics
doesn't define a dependency to org.apache.hadoop:hadoop-common:3.1.1 in
its pom. Is this intended or am I missing something?
Thank you for all your efforts.
/Henrik
Re: ClassNotFoundException in Ratis
Posted by Tsz Wo Sze <sz...@gmail.com>.
> ... updating so ratis-metric is depending directly on the 3.1.2 ...
Sound great. Thanks for digging it out! Let's continue the
discussion in RATIS-699.
Tsz-Wo
On Wed, Oct 2, 2019 at 5:18 AM Henrik Hegardt
<he...@gmail.com> wrote:
>
> Hi Tsz-Wo,
>
> Thank you for your reply. I've been checking up on this a little bit
> more and I believe the CNFE is my fault because I was trying to fix
> another problem, which I'm going to try to explain:
>
> Ratis-Metric is depending on the dependency
> com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter: 0.1.2.
> This in turn depends on the org.apache.hadoop:hadoop-common:2.6.0. This
> in turn depends on org.apache.hadoop:hadoop-annotations:2.6.0 which have
> an dependency on a system scoped dependency named jdk.tools:1.6. I'm
> running java 11 and this dependency doesn't exist in my environment so I
> can't compile with ratis included in my project because it fails trying
> to resolve that dependency.
>
> However, the project ratis-hadoop depends on the the dependency
> org.apache.hadoop:hadoop-common:3.1.1 (which should be upgraded to 3.1.2
> because the sources are missing in 3.1.1) which doesn't have the
> jdk.tools:1.6 dependency. So if I depend on this I can build the project
> because the faulty 2.6.0 dependency is shadowed transitively by the
> 3.1.1 dependency and everything is compiling.
>
> The simple fix would be updating so ratis-metric is depending directly
> on the 3.1.2 dependency instead since it effectively will be the 3.1.1
> code you are going run with anyway when running with the ratis-hadoop
> dependency. I can open a bug for this.
>
> I also have a question; right now there's a dependency on
> io.dropwizard.metrics:metrics-core:3.2.5. It seems like there's an idea
> that the metric loader functionality should/would be more pluggable
> through the serviceloader API? Is this correct? I was merely thinking
> that the ratis-metric project should be less opinionated on what library
> one should use to measure things? I am willing to look into this if
> needed and see if I can do anything about it.
>
> I'm using dropwizard-metrics:4.0.5 which isn't compatible with the
> current ratis-metrics implementation, and unfortunately I cannot
> downgrade. I'd love to use ratis in my project, but this is something I
> need to fix to be able to use it.
>
> /Henrik
>
> Den 2019-10-01 kl. 00:05, skrev Tsz Wo Sze:
> > Hi Henrik,
> >
> > Thanks for reporting the problem.
> > org.apache.ratis.metrics.MetricsReporting imports
> > org.apache.hadoop.metrics2.lib.DefaultMetricsSystem, I guess it is
> > causing the ClassNotFoundException. It is a bug. Do you want to file
> > a JIRA?
> >
> > Tsz-Wo
> >
> > On Mon, Sep 30, 2019 at 2:23 AM Henrik Hegardt
> > <he...@gmail.com> wrote:
> >> Hi everyone,
> >>
> >> I'm getting this error while running Ratis:
> >>
> >> Caused by: java.io.IOException: java.lang.NoClassDefFoundError:
> >> org/apache/hadoop/metrics2/lib/DefaultMetricsSystem
> >> at org.apache.ratis.util.IOUtils.asIOException(IOUtils.java:54)
> >> at org.apache.ratis.util.IOUtils.toIOException(IOUtils.java:61)
> >> at org.apache.ratis.util.IOUtils.getFromFuture(IOUtils.java:70)
> >> at
> >> org.apache.ratis.server.impl.RaftServerProxy.getImpls(RaftServerProxy.java:284)
> >> at
> >> org.apache.ratis.server.impl.RaftServerProxy.start(RaftServerProxy.java:296)
> >> at test.cluster.ClusterSevice.start(ClusterSevice.java:85)
> >> ... 65 more
> >> Caused by: java.lang.NoClassDefFoundError:
> >> org/apache/hadoop/metrics2/lib/DefaultMetricsSystem
> >> at
> >> org.apache.ratis.metrics.MetricsReporting.<clinit>(MetricsReporting.java:42)
> >> at
> >> org.apache.ratis.server.metrics.RatisMetrics.<clinit>(RatisMetrics.java:39)
> >> at
> >> org.apache.ratis.server.raftlog.segmented.SegmentedRaftLogWorker.<init>(SegmentedRaftLogWorker.java:176)
> >> at
> >> org.apache.ratis.server.raftlog.segmented.SegmentedRaftLog.<init>(SegmentedRaftLog.java:197)
> >> at
> >> org.apache.ratis.server.raftlog.segmented.SegmentedRaftLog.<init>(SegmentedRaftLog.java:183)
> >> at
> >> org.apache.ratis.server.impl.ServerState.initRaftLog(ServerState.java:188)
> >> at
> >> org.apache.ratis.server.impl.ServerState.<init>(ServerState.java:120)
> >> at
> >> org.apache.ratis.server.impl.RaftServerImpl.<init>(RaftServerImpl.java:110)
> >> at
> >> org.apache.ratis.server.impl.RaftServerProxy.lambda$newRaftServerImpl$2(RaftServerProxy.java:208)
> >> at
> >> java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700)
> >> at
> >> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> >> at
> >> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> >> at java.base/java.lang.Thread.run(Thread.java:834)
> >> Caused by: java.lang.ClassNotFoundException:
> >> org.apache.hadoop.metrics2.lib.DefaultMetricsSystem
> >> at
> >> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583)
> >> at
> >> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
> >> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
> >> ... 13 more
> >>
> >> AFAICT this is a class in the dependency
> >> org.apache.hadoop:hadoop-common:3.1.1. I'm running with the dependencies
> >> and Ratis 0.4.0
> >>
> >> <dependency>
> >> <artifactId>ratis-server</artifactId>
> >> <groupId>org.apache.ratis</groupId>
> >>
> >> <exclusions>
> >> <exclusion>
> >> <groupId>log4j</groupId>
> >> <artifactId>log4j</artifactId>
> >> </exclusion>
> >> <exclusion>
> >> <groupId>org.slf4j</groupId>
> >> <artifactId>slf4j-log4j12</artifactId>
> >> </exclusion>
> >> </exclusions>
> >> </dependency>
> >> <dependency>
> >> <artifactId>ratis-grpc</artifactId>
> >> <groupId>org.apache.ratis</groupId>
> >> </dependency>
> >>
> >> And which should be enough according to the documentation. While it's
> >> possible to include this dependency, it pulls in a lot of other
> >> dependencies. While org.apache.ratis.metrics.MetricsReporting class does
> >> have a dependency on the
> >> org.apache.hadoop.metrics2.lib.DefaultMetricsSystem class, ratis-metrics
> >> doesn't define a dependency to org.apache.hadoop:hadoop-common:3.1.1 in
> >> its pom. Is this intended or am I missing something?
> >>
> >> Thank you for all your efforts.
> >>
> >> /Henrik
> >>
> >>
> >>
> >>
Re: ClassNotFoundException in Ratis
Posted by Henrik Hegardt <he...@gmail.com>.
Hi Tsz-Wo,
Thank you for your reply. I've been checking up on this a little bit
more and I believe the CNFE is my fault because I was trying to fix
another problem, which I'm going to try to explain:
Ratis-Metric is depending on the dependency
com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter: 0.1.2.
This in turn depends on the org.apache.hadoop:hadoop-common:2.6.0. This
in turn depends on org.apache.hadoop:hadoop-annotations:2.6.0 which have
an dependency on a system scoped dependency named jdk.tools:1.6. I'm
running java 11 and this dependency doesn't exist in my environment so I
can't compile with ratis included in my project because it fails trying
to resolve that dependency.
However, the project ratis-hadoop depends on the the dependency
org.apache.hadoop:hadoop-common:3.1.1 (which should be upgraded to 3.1.2
because the sources are missing in 3.1.1) which doesn't have the
jdk.tools:1.6 dependency. So if I depend on this I can build the project
because the faulty 2.6.0 dependency is shadowed transitively by the
3.1.1 dependency and everything is compiling.
The simple fix would be updating so ratis-metric is depending directly
on the 3.1.2 dependency instead since it effectively will be the 3.1.1
code you are going run with anyway when running with the ratis-hadoop
dependency. I can open a bug for this.
I also have a question; right now there's a dependency on
io.dropwizard.metrics:metrics-core:3.2.5. It seems like there's an idea
that the metric loader functionality should/would be more pluggable
through the serviceloader API? Is this correct? I was merely thinking
that the ratis-metric project should be less opinionated on what library
one should use to measure things? I am willing to look into this if
needed and see if I can do anything about it.
I'm using dropwizard-metrics:4.0.5 which isn't compatible with the
current ratis-metrics implementation, and unfortunately I cannot
downgrade. I'd love to use ratis in my project, but this is something I
need to fix to be able to use it.
/Henrik
Den 2019-10-01 kl. 00:05, skrev Tsz Wo Sze:
> Hi Henrik,
>
> Thanks for reporting the problem.
> org.apache.ratis.metrics.MetricsReporting imports
> org.apache.hadoop.metrics2.lib.DefaultMetricsSystem, I guess it is
> causing the ClassNotFoundException. It is a bug. Do you want to file
> a JIRA?
>
> Tsz-Wo
>
> On Mon, Sep 30, 2019 at 2:23 AM Henrik Hegardt
> <he...@gmail.com> wrote:
>> Hi everyone,
>>
>> I'm getting this error while running Ratis:
>>
>> Caused by: java.io.IOException: java.lang.NoClassDefFoundError:
>> org/apache/hadoop/metrics2/lib/DefaultMetricsSystem
>> at org.apache.ratis.util.IOUtils.asIOException(IOUtils.java:54)
>> at org.apache.ratis.util.IOUtils.toIOException(IOUtils.java:61)
>> at org.apache.ratis.util.IOUtils.getFromFuture(IOUtils.java:70)
>> at
>> org.apache.ratis.server.impl.RaftServerProxy.getImpls(RaftServerProxy.java:284)
>> at
>> org.apache.ratis.server.impl.RaftServerProxy.start(RaftServerProxy.java:296)
>> at test.cluster.ClusterSevice.start(ClusterSevice.java:85)
>> ... 65 more
>> Caused by: java.lang.NoClassDefFoundError:
>> org/apache/hadoop/metrics2/lib/DefaultMetricsSystem
>> at
>> org.apache.ratis.metrics.MetricsReporting.<clinit>(MetricsReporting.java:42)
>> at
>> org.apache.ratis.server.metrics.RatisMetrics.<clinit>(RatisMetrics.java:39)
>> at
>> org.apache.ratis.server.raftlog.segmented.SegmentedRaftLogWorker.<init>(SegmentedRaftLogWorker.java:176)
>> at
>> org.apache.ratis.server.raftlog.segmented.SegmentedRaftLog.<init>(SegmentedRaftLog.java:197)
>> at
>> org.apache.ratis.server.raftlog.segmented.SegmentedRaftLog.<init>(SegmentedRaftLog.java:183)
>> at
>> org.apache.ratis.server.impl.ServerState.initRaftLog(ServerState.java:188)
>> at
>> org.apache.ratis.server.impl.ServerState.<init>(ServerState.java:120)
>> at
>> org.apache.ratis.server.impl.RaftServerImpl.<init>(RaftServerImpl.java:110)
>> at
>> org.apache.ratis.server.impl.RaftServerProxy.lambda$newRaftServerImpl$2(RaftServerProxy.java:208)
>> at
>> java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700)
>> at
>> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>> at
>> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>> at java.base/java.lang.Thread.run(Thread.java:834)
>> Caused by: java.lang.ClassNotFoundException:
>> org.apache.hadoop.metrics2.lib.DefaultMetricsSystem
>> at
>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583)
>> at
>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
>> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
>> ... 13 more
>>
>> AFAICT this is a class in the dependency
>> org.apache.hadoop:hadoop-common:3.1.1. I'm running with the dependencies
>> and Ratis 0.4.0
>>
>> <dependency>
>> <artifactId>ratis-server</artifactId>
>> <groupId>org.apache.ratis</groupId>
>>
>> <exclusions>
>> <exclusion>
>> <groupId>log4j</groupId>
>> <artifactId>log4j</artifactId>
>> </exclusion>
>> <exclusion>
>> <groupId>org.slf4j</groupId>
>> <artifactId>slf4j-log4j12</artifactId>
>> </exclusion>
>> </exclusions>
>> </dependency>
>> <dependency>
>> <artifactId>ratis-grpc</artifactId>
>> <groupId>org.apache.ratis</groupId>
>> </dependency>
>>
>> And which should be enough according to the documentation. While it's
>> possible to include this dependency, it pulls in a lot of other
>> dependencies. While org.apache.ratis.metrics.MetricsReporting class does
>> have a dependency on the
>> org.apache.hadoop.metrics2.lib.DefaultMetricsSystem class, ratis-metrics
>> doesn't define a dependency to org.apache.hadoop:hadoop-common:3.1.1 in
>> its pom. Is this intended or am I missing something?
>>
>> Thank you for all your efforts.
>>
>> /Henrik
>>
>>
>>
>>
Re: ClassNotFoundException in Ratis
Posted by Tsz Wo Sze <sz...@gmail.com>.
Hi Henrik,
Thanks for reporting the problem.
org.apache.ratis.metrics.MetricsReporting imports
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem, I guess it is
causing the ClassNotFoundException. It is a bug. Do you want to file
a JIRA?
Tsz-Wo
On Mon, Sep 30, 2019 at 2:23 AM Henrik Hegardt
<he...@gmail.com> wrote:
>
> Hi everyone,
>
> I'm getting this error while running Ratis:
>
> Caused by: java.io.IOException: java.lang.NoClassDefFoundError:
> org/apache/hadoop/metrics2/lib/DefaultMetricsSystem
> at org.apache.ratis.util.IOUtils.asIOException(IOUtils.java:54)
> at org.apache.ratis.util.IOUtils.toIOException(IOUtils.java:61)
> at org.apache.ratis.util.IOUtils.getFromFuture(IOUtils.java:70)
> at
> org.apache.ratis.server.impl.RaftServerProxy.getImpls(RaftServerProxy.java:284)
> at
> org.apache.ratis.server.impl.RaftServerProxy.start(RaftServerProxy.java:296)
> at test.cluster.ClusterSevice.start(ClusterSevice.java:85)
> ... 65 more
> Caused by: java.lang.NoClassDefFoundError:
> org/apache/hadoop/metrics2/lib/DefaultMetricsSystem
> at
> org.apache.ratis.metrics.MetricsReporting.<clinit>(MetricsReporting.java:42)
> at
> org.apache.ratis.server.metrics.RatisMetrics.<clinit>(RatisMetrics.java:39)
> at
> org.apache.ratis.server.raftlog.segmented.SegmentedRaftLogWorker.<init>(SegmentedRaftLogWorker.java:176)
> at
> org.apache.ratis.server.raftlog.segmented.SegmentedRaftLog.<init>(SegmentedRaftLog.java:197)
> at
> org.apache.ratis.server.raftlog.segmented.SegmentedRaftLog.<init>(SegmentedRaftLog.java:183)
> at
> org.apache.ratis.server.impl.ServerState.initRaftLog(ServerState.java:188)
> at
> org.apache.ratis.server.impl.ServerState.<init>(ServerState.java:120)
> at
> org.apache.ratis.server.impl.RaftServerImpl.<init>(RaftServerImpl.java:110)
> at
> org.apache.ratis.server.impl.RaftServerProxy.lambda$newRaftServerImpl$2(RaftServerProxy.java:208)
> at
> java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:834)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.metrics2.lib.DefaultMetricsSystem
> at
> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583)
> at
> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
> ... 13 more
>
> AFAICT this is a class in the dependency
> org.apache.hadoop:hadoop-common:3.1.1. I'm running with the dependencies
> and Ratis 0.4.0
>
> <dependency>
> <artifactId>ratis-server</artifactId>
> <groupId>org.apache.ratis</groupId>
>
> <exclusions>
> <exclusion>
> <groupId>log4j</groupId>
> <artifactId>log4j</artifactId>
> </exclusion>
> <exclusion>
> <groupId>org.slf4j</groupId>
> <artifactId>slf4j-log4j12</artifactId>
> </exclusion>
> </exclusions>
> </dependency>
> <dependency>
> <artifactId>ratis-grpc</artifactId>
> <groupId>org.apache.ratis</groupId>
> </dependency>
>
> And which should be enough according to the documentation. While it's
> possible to include this dependency, it pulls in a lot of other
> dependencies. While org.apache.ratis.metrics.MetricsReporting class does
> have a dependency on the
> org.apache.hadoop.metrics2.lib.DefaultMetricsSystem class, ratis-metrics
> doesn't define a dependency to org.apache.hadoop:hadoop-common:3.1.1 in
> its pom. Is this intended or am I missing something?
>
> Thank you for all your efforts.
>
> /Henrik
>
>
>
>