You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Yassine MARZOUGUI <y....@mindlytix.com> on 2016/10/19 16:41:36 UTC
NoClassDefFoundError on cluster with httpclient 4.5.2
Hi all,
I'm using httpclient with the following dependency:
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.5.2</version>
</dependency>
On local mode, the program works correctly, but when executed on the
cluster, I get the following exception:
java.lang.Exception: The user defined 'open(Configuration)' method in class
org.myorg.quickstart.Frequencies$2 caused an exception: Could not
initialize class org.apache.http.conn.ssl.SSLConnectionSocketFactory
at
org.apache.flink.runtime.operators.BatchTask.openUserCode(BatchTask.java:1337)
at
org.apache.flink.runtime.operators.chaining.ChainedFlatMapDriver.openTask(ChainedFlatMapDriver.java:47)
at
org.apache.flink.runtime.operators.BatchTask.openChainedTasks(BatchTask.java:1377)
at
org.apache.flink.runtime.operators.DataSourceTask.invoke(DataSourceTask.java:124)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:585)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.NoClassDefFoundError: Could not initialize class
org.apache.http.conn.ssl.SSLConnectionSocketFactory
at
org.apache.http.impl.client.HttpClientBuilder.build(HttpClientBuilder.java:966)
at org.myorg.quickstart.Frequencies$2.open(Frequencies.java:82)
at
org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:38)
at
org.apache.flink.runtime.operators.BatchTask.openUserCode(BatchTask.java:1335)
... 5 more
I'm using Flink 1.1.3. Any idea how to solve the problem? Thank you.
Best,
Yassine
Re: NoClassDefFoundError on cluster with httpclient 4.5.2
Posted by Till Rohrmann <tr...@apache.org>.
Great to hear that you solved your problem :-)
Cheers,
Till
On Fri, Oct 21, 2016 at 12:34 PM, Yassine MARZOUGUI <
y.marzougui@mindlytix.com> wrote:
> Hi Till,
>
> The httpclient jar is included in the job jar. Looking at a similar issue
> FLINK-4587 <https://issues.apache.org/jira/browse/FLINK-4587>, It turns
> out the problem is with maven shade plugin, since I'm building Flink from
> sources with maven 3.3.x.
> I was able to solve the problem by rebuilding "flink-dist" as suggested by
> Stephan in the comments.
>
> Best,
> Yassine
>
> 2016-10-20 11:05 GMT+02:00 Till Rohrmann <tr...@apache.org>:
>
>> Hi Yassine,
>>
>> can you check whether the httpclient jar is contained in your job jar
>> which you submit to the cluster?
>>
>> Cheers,
>> Till
>>
>> On Wed, Oct 19, 2016 at 6:41 PM, Yassine MARZOUGUI <
>> y.marzougui@mindlytix.com> wrote:
>>
>>> Hi all,
>>>
>>> I'm using httpclient with the following dependency:
>>>
>>> <dependency>
>>> <groupId>org.apache.httpcomponents</groupId>
>>> <artifactId>httpclient</artifactId>
>>> <version>4.5.2</version>
>>> </dependency>
>>>
>>> On local mode, the program works correctly, but when executed on the
>>> cluster, I get the following exception:
>>>
>>> java.lang.Exception: The user defined 'open(Configuration)' method in
>>> class org.myorg.quickstart.Frequencies$2 caused an exception: Could not
>>> initialize class org.apache.http.conn.ssl.SSLConnectionSocketFactory
>>> at org.apache.flink.runtime.operators.BatchTask.openUserCode(Ba
>>> tchTask.java:1337)
>>> at org.apache.flink.runtime.operators.chaining.ChainedFlatMapDr
>>> iver.openTask(ChainedFlatMapDriver.java:47)
>>> at org.apache.flink.runtime.operators.BatchTask.openChainedTask
>>> s(BatchTask.java:1377)
>>> at org.apache.flink.runtime.operators.DataSourceTask.invoke(Dat
>>> aSourceTask.java:124)
>>> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:585)
>>> at java.lang.Thread.run(Unknown Source)
>>> Caused by: java.lang.NoClassDefFoundError: Could not initialize class
>>> org.apache.http.conn.ssl.SSLConnectionSocketFactory
>>> at org.apache.http.impl.client.HttpClientBuilder.build(HttpClie
>>> ntBuilder.java:966)
>>> at org.myorg.quickstart.Frequencies$2.open(Frequencies.java:82)
>>> at org.apache.flink.api.common.functions.util.FunctionUtils.ope
>>> nFunction(FunctionUtils.java:38)
>>> at org.apache.flink.runtime.operators.BatchTask.openUserCode(Ba
>>> tchTask.java:1335)
>>> ... 5 more
>>>
>>> I'm using Flink 1.1.3. Any idea how to solve the problem? Thank you.
>>>
>>> Best,
>>> Yassine
>>>
>>
>>
>
Re: NoClassDefFoundError on cluster with httpclient 4.5.2
Posted by Yassine MARZOUGUI <y....@mindlytix.com>.
Hi Till,
The httpclient jar is included in the job jar. Looking at a similar issue
FLINK-4587 <https://issues.apache.org/jira/browse/FLINK-4587>, It turns out
the problem is with maven shade plugin, since I'm building Flink from
sources with maven 3.3.x.
I was able to solve the problem by rebuilding "flink-dist" as suggested by
Stephan in the comments.
Best,
Yassine
2016-10-20 11:05 GMT+02:00 Till Rohrmann <tr...@apache.org>:
> Hi Yassine,
>
> can you check whether the httpclient jar is contained in your job jar
> which you submit to the cluster?
>
> Cheers,
> Till
>
> On Wed, Oct 19, 2016 at 6:41 PM, Yassine MARZOUGUI <
> y.marzougui@mindlytix.com> wrote:
>
>> Hi all,
>>
>> I'm using httpclient with the following dependency:
>>
>> <dependency>
>> <groupId>org.apache.httpcomponents</groupId>
>> <artifactId>httpclient</artifactId>
>> <version>4.5.2</version>
>> </dependency>
>>
>> On local mode, the program works correctly, but when executed on the
>> cluster, I get the following exception:
>>
>> java.lang.Exception: The user defined 'open(Configuration)' method in
>> class org.myorg.quickstart.Frequencies$2 caused an exception: Could not
>> initialize class org.apache.http.conn.ssl.SSLConnectionSocketFactory
>> at org.apache.flink.runtime.operators.BatchTask.openUserCode(
>> BatchTask.java:1337)
>> at org.apache.flink.runtime.operators.chaining.ChainedFlatMapDr
>> iver.openTask(ChainedFlatMapDriver.java:47)
>> at org.apache.flink.runtime.operators.BatchTask.openChainedTask
>> s(BatchTask.java:1377)
>> at org.apache.flink.runtime.operators.DataSourceTask.invoke(
>> DataSourceTask.java:124)
>> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:585)
>> at java.lang.Thread.run(Unknown Source)
>> Caused by: java.lang.NoClassDefFoundError: Could not initialize class
>> org.apache.http.conn.ssl.SSLConnectionSocketFactory
>> at org.apache.http.impl.client.HttpClientBuilder.build(HttpClie
>> ntBuilder.java:966)
>> at org.myorg.quickstart.Frequencies$2.open(Frequencies.java:82)
>> at org.apache.flink.api.common.functions.util.FunctionUtils.ope
>> nFunction(FunctionUtils.java:38)
>> at org.apache.flink.runtime.operators.BatchTask.openUserCode(
>> BatchTask.java:1335)
>> ... 5 more
>>
>> I'm using Flink 1.1.3. Any idea how to solve the problem? Thank you.
>>
>> Best,
>> Yassine
>>
>
>
Re: NoClassDefFoundError on cluster with httpclient 4.5.2
Posted by Till Rohrmann <tr...@apache.org>.
Hi Yassine,
can you check whether the httpclient jar is contained in your job jar which
you submit to the cluster?
Cheers,
Till
On Wed, Oct 19, 2016 at 6:41 PM, Yassine MARZOUGUI <
y.marzougui@mindlytix.com> wrote:
> Hi all,
>
> I'm using httpclient with the following dependency:
>
> <dependency>
> <groupId>org.apache.httpcomponents</groupId>
> <artifactId>httpclient</artifactId>
> <version>4.5.2</version>
> </dependency>
>
> On local mode, the program works correctly, but when executed on the
> cluster, I get the following exception:
>
> java.lang.Exception: The user defined 'open(Configuration)' method in
> class org.myorg.quickstart.Frequencies$2 caused an exception: Could not
> initialize class org.apache.http.conn.ssl.SSLConnectionSocketFactory
> at org.apache.flink.runtime.operators.BatchTask.
> openUserCode(BatchTask.java:1337)
> at org.apache.flink.runtime.operators.chaining.
> ChainedFlatMapDriver.openTask(ChainedFlatMapDriver.java:47)
> at org.apache.flink.runtime.operators.BatchTask.
> openChainedTasks(BatchTask.java:1377)
> at org.apache.flink.runtime.operators.DataSourceTask.
> invoke(DataSourceTask.java:124)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:585)
> at java.lang.Thread.run(Unknown Source)
> Caused by: java.lang.NoClassDefFoundError: Could not initialize class
> org.apache.http.conn.ssl.SSLConnectionSocketFactory
> at org.apache.http.impl.client.HttpClientBuilder.build(
> HttpClientBuilder.java:966)
> at org.myorg.quickstart.Frequencies$2.open(Frequencies.java:82)
> at org.apache.flink.api.common.functions.util.FunctionUtils.
> openFunction(FunctionUtils.java:38)
> at org.apache.flink.runtime.operators.BatchTask.
> openUserCode(BatchTask.java:1335)
> ... 5 more
>
> I'm using Flink 1.1.3. Any idea how to solve the problem? Thank you.
>
> Best,
> Yassine
>