You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Jayant Ameta <wi...@gmail.com> on 2020/01/08 11:20:30 UTC
Elasticsink sometimes gives NoClassDefFoundError
Hi,
I see the following error sometimes on my flink job, even though the class
is present in my uber jar.
java.lang.NoClassDefFoundError:
org/apache/flink/streaming/connectors/elasticsearch5/shaded/org/jboss/netty/channel/socket/nio/NioClientSocketPipelineSink$1
at
org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:111)
... 17 common frames omitted Wrapped by:
org.elasticsearch.ElasticsearchException: java.lang.NoClassDefFoundError:
org/apache/flink/streaming/connectors/elasticsearch5/shaded/org/jboss/netty/channel/socket/nio/NioClientSocketPipelineSink$1
at
org.elasticsearch.transport.netty3.Netty3Transport.exceptionCaught(Netty3Transport.java:325)
at
org.elasticsearch.transport.netty3.Netty3MessageChannelHandler.exceptionCaught(Netty3MessageChannelHandler.java:83)
at
org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112)
at
org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
... 23 frames truncated
Jayant
Re: Elasticsink sometimes gives NoClassDefFoundError
Posted by Chesnay Schepler <ch...@apache.org>.
Could you clarify under what circumstances you see this issue? You say
"sometimes"; is the job running normally and then failing due to this
error? Does it happen when submitting/canceling/restarting a job etc.
On 08/01/2020 12:20, Jayant Ameta wrote:
> Hi,
> I see the following error sometimesĀ on my flink job, even though the
> class is present in my uber jar.
>
> java.lang.NoClassDefFoundError:
> org/apache/flink/streaming/connectors/elasticsearch5/shaded/org/jboss/netty/channel/socket/nio/NioClientSocketPipelineSink$1
> at
> org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:111)
> ... 17 common frames omitted Wrapped by:
> org.elasticsearch.ElasticsearchException:
> java.lang.NoClassDefFoundError:
> org/apache/flink/streaming/connectors/elasticsearch5/shaded/org/jboss/netty/channel/socket/nio/NioClientSocketPipelineSink$1
> at
> org.elasticsearch.transport.netty3.Netty3Transport.exceptionCaught(Netty3Transport.java:325)
> at
> org.elasticsearch.transport.netty3.Netty3MessageChannelHandler.exceptionCaught(Netty3MessageChannelHandler.java:83)
> at
> org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112)
> at
> org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at
> org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> ... 23 frames truncated
>
>
> Jayant
Re: Elasticsink sometimes gives NoClassDefFoundError
Posted by Arvid Heise <ar...@ververica.com>.
I don't see a particular reason why you see this behavior. Chesnay's
explanation is the only plausible way that this behavior can happen.
I fear that without a specific log, we cannot help further.
On Fri, Jan 10, 2020 at 5:06 AM Jayant Ameta <wi...@gmail.com> wrote:
> Also, the ES version I'm using is 5.6.7
>
> Jayant
>
>
> On Thu, Jan 9, 2020 at 10:39 AM Jayant Ameta <wi...@gmail.com> wrote:
>
>> Hi,
>> The elastic connector is packaged in the uber jar that is submitted.
>> There is only 1 version of the connector:
>> flink-connector-elasticsearch5_2.11:1.7.1
>> I'm using Flink 1.7.1
>>
>> I couldn't figure out whether this error causes the job to fail, or
>> whether I see this error when the job is restarting after some other
>> failure.
>> But, the occurrence of this error and job restarts is correlated.
>>
>>
>> Jayant Ameta
>>
>>
>> On Wed, Jan 8, 2020 at 6:47 PM Arvid Heise <ar...@ververica.com> wrote:
>>
>>> Hi Jayant,
>>>
>>> if you only see it sometimes that indicates that you have it in two
>>> different versions of the connectors where class loader order is
>>> non-deterministic. Could you post the classpath?
>>>
>>> Btw, it's always good to add which Flink version you use.
>>>
>>> Best,
>>>
>>> Arvid
>>>
>>> On Wed, Jan 8, 2020 at 12:20 PM Jayant Ameta <wi...@gmail.com>
>>> wrote:
>>>
>>>> Hi,
>>>> I see the following error sometimes on my flink job, even though the
>>>> class is present in my uber jar.
>>>>
>>>> java.lang.NoClassDefFoundError:
>>>> org/apache/flink/streaming/connectors/elasticsearch5/shaded/org/jboss/netty/channel/socket/nio/NioClientSocketPipelineSink$1
>>>> at
>>>> org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:111)
>>>> ... 17 common frames omitted Wrapped by:
>>>> org.elasticsearch.ElasticsearchException: java.lang.NoClassDefFoundError:
>>>> org/apache/flink/streaming/connectors/elasticsearch5/shaded/org/jboss/netty/channel/socket/nio/NioClientSocketPipelineSink$1
>>>> at
>>>> org.elasticsearch.transport.netty3.Netty3Transport.exceptionCaught(Netty3Transport.java:325)
>>>> at
>>>> org.elasticsearch.transport.netty3.Netty3MessageChannelHandler.exceptionCaught(Netty3MessageChannelHandler.java:83)
>>>> at
>>>> org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112)
>>>> at
>>>> org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
>>>> at
>>>> org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
>>>> ... 23 frames truncated
>>>>
>>>>
>>>> Jayant
>>>>
>>>
Re: Elasticsink sometimes gives NoClassDefFoundError
Posted by Jayant Ameta <wi...@gmail.com>.
Also, the ES version I'm using is 5.6.7
Jayant
On Thu, Jan 9, 2020 at 10:39 AM Jayant Ameta <wi...@gmail.com> wrote:
> Hi,
> The elastic connector is packaged in the uber jar that is submitted. There
> is only 1 version of the connector:
> flink-connector-elasticsearch5_2.11:1.7.1
> I'm using Flink 1.7.1
>
> I couldn't figure out whether this error causes the job to fail, or
> whether I see this error when the job is restarting after some other
> failure.
> But, the occurrence of this error and job restarts is correlated.
>
>
> Jayant Ameta
>
>
> On Wed, Jan 8, 2020 at 6:47 PM Arvid Heise <ar...@ververica.com> wrote:
>
>> Hi Jayant,
>>
>> if you only see it sometimes that indicates that you have it in two
>> different versions of the connectors where class loader order is
>> non-deterministic. Could you post the classpath?
>>
>> Btw, it's always good to add which Flink version you use.
>>
>> Best,
>>
>> Arvid
>>
>> On Wed, Jan 8, 2020 at 12:20 PM Jayant Ameta <wi...@gmail.com>
>> wrote:
>>
>>> Hi,
>>> I see the following error sometimes on my flink job, even though the
>>> class is present in my uber jar.
>>>
>>> java.lang.NoClassDefFoundError:
>>> org/apache/flink/streaming/connectors/elasticsearch5/shaded/org/jboss/netty/channel/socket/nio/NioClientSocketPipelineSink$1
>>> at
>>> org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:111)
>>> ... 17 common frames omitted Wrapped by:
>>> org.elasticsearch.ElasticsearchException: java.lang.NoClassDefFoundError:
>>> org/apache/flink/streaming/connectors/elasticsearch5/shaded/org/jboss/netty/channel/socket/nio/NioClientSocketPipelineSink$1
>>> at
>>> org.elasticsearch.transport.netty3.Netty3Transport.exceptionCaught(Netty3Transport.java:325)
>>> at
>>> org.elasticsearch.transport.netty3.Netty3MessageChannelHandler.exceptionCaught(Netty3MessageChannelHandler.java:83)
>>> at
>>> org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112)
>>> at
>>> org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
>>> at
>>> org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
>>> ... 23 frames truncated
>>>
>>>
>>> Jayant
>>>
>>
Re: Elasticsink sometimes gives NoClassDefFoundError
Posted by Jayant Ameta <wi...@gmail.com>.
Hi,
The elastic connector is packaged in the uber jar that is submitted. There
is only 1 version of the connector:
flink-connector-elasticsearch5_2.11:1.7.1
I'm using Flink 1.7.1
I couldn't figure out whether this error causes the job to fail, or whether
I see this error when the job is restarting after some other failure.
But, the occurrence of this error and job restarts is correlated.
Jayant Ameta
On Wed, Jan 8, 2020 at 6:47 PM Arvid Heise <ar...@ververica.com> wrote:
> Hi Jayant,
>
> if you only see it sometimes that indicates that you have it in two
> different versions of the connectors where class loader order is
> non-deterministic. Could you post the classpath?
>
> Btw, it's always good to add which Flink version you use.
>
> Best,
>
> Arvid
>
> On Wed, Jan 8, 2020 at 12:20 PM Jayant Ameta <wi...@gmail.com> wrote:
>
>> Hi,
>> I see the following error sometimes on my flink job, even though the
>> class is present in my uber jar.
>>
>> java.lang.NoClassDefFoundError:
>> org/apache/flink/streaming/connectors/elasticsearch5/shaded/org/jboss/netty/channel/socket/nio/NioClientSocketPipelineSink$1
>> at
>> org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:111)
>> ... 17 common frames omitted Wrapped by:
>> org.elasticsearch.ElasticsearchException: java.lang.NoClassDefFoundError:
>> org/apache/flink/streaming/connectors/elasticsearch5/shaded/org/jboss/netty/channel/socket/nio/NioClientSocketPipelineSink$1
>> at
>> org.elasticsearch.transport.netty3.Netty3Transport.exceptionCaught(Netty3Transport.java:325)
>> at
>> org.elasticsearch.transport.netty3.Netty3MessageChannelHandler.exceptionCaught(Netty3MessageChannelHandler.java:83)
>> at
>> org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112)
>> at
>> org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
>> at
>> org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
>> ... 23 frames truncated
>>
>>
>> Jayant
>>
>
Re: Elasticsink sometimes gives NoClassDefFoundError
Posted by Arvid Heise <ar...@ververica.com>.
Hi Jayant,
if you only see it sometimes that indicates that you have it in two
different versions of the connectors where class loader order is
non-deterministic. Could you post the classpath?
Btw, it's always good to add which Flink version you use.
Best,
Arvid
On Wed, Jan 8, 2020 at 12:20 PM Jayant Ameta <wi...@gmail.com> wrote:
> Hi,
> I see the following error sometimes on my flink job, even though the class
> is present in my uber jar.
>
> java.lang.NoClassDefFoundError:
> org/apache/flink/streaming/connectors/elasticsearch5/shaded/org/jboss/netty/channel/socket/nio/NioClientSocketPipelineSink$1
> at
> org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:111)
> ... 17 common frames omitted Wrapped by:
> org.elasticsearch.ElasticsearchException: java.lang.NoClassDefFoundError:
> org/apache/flink/streaming/connectors/elasticsearch5/shaded/org/jboss/netty/channel/socket/nio/NioClientSocketPipelineSink$1
> at
> org.elasticsearch.transport.netty3.Netty3Transport.exceptionCaught(Netty3Transport.java:325)
> at
> org.elasticsearch.transport.netty3.Netty3MessageChannelHandler.exceptionCaught(Netty3MessageChannelHandler.java:83)
> at
> org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112)
> at
> org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at
> org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> ... 23 frames truncated
>
>
> Jayant
>