[jira] [Commented] (FLINK-3373) Using a newer library of Apache HttpClient than 4.2.6 will get class loading problems

["Stephan Ewen (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-3373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15138787#comment-15138787 ] 

Stephan Ewen commented on FLINK-3373:
-------------------------------------

Would it help if Flink simply updated the HTTP Client to the latest version (is it backwards compatible)?

If not, then we need to shade the dependency, but if yes, that would be a very lightweight fix.

> Using a newer library of Apache HttpClient than 4.2.6 will get class loading problems
> -------------------------------------------------------------------------------------
>
>                 Key: FLINK-3373
>                 URL: https://issues.apache.org/jira/browse/FLINK-3373
>             Project: Flink
>          Issue Type: Bug
>         Environment: Latest Flink snapshot 1.0
>            Reporter: Jakob Sultan Ericsson
>
> When I trying to use Apache HTTP client 4.5.1 in my flink job it will crash with NoClassDefFound.
> This has to do that it load some classes from provided httpclient 4.2.5/6 in core flink.
> {noformat}
> 17:05:56,193 INFO  org.apache.flink.runtime.taskmanager.Task                     - DuplicateFilter -> InstallKeyLookup (11/16) switched to FAILED with exception.
> java.lang.NoSuchFieldError: INSTANCE
>         at org.apache.http.conn.ssl.SSLConnectionSocketFactory.<clinit>(SSLConnectionSocketFactory.java:144)
>         at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.getDefaultRegistry(PoolingHttpClientConnectionManager.java:109)
>         at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.<init>(PoolingHttpClientConnectionManager.java:116)
>         ...<internal classes>
>         at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
>         at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:89)
>         at org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:305)
>         at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:227)
>         at org.apache.flink.runtime.taskmanager.Task.run(Task.java:566)
>         at java.lang.Thread.run(Thread.java:745)
> {noformat}
> SSLConnectionSocketFactory and finds an earlier version of the AllowAllHostnameVerifier that does have the INSTANCE variable (instance variable was probably added in 4.3).
> {noformat}
> jar tvf lib/flink-dist-1.0-SNAPSHOT.jar |grep AllowAllHostnameVerifier  
>    791 Thu Dec 17 09:55:46 CET 2015 org/apache/http/conn/ssl/AllowAllHostnameVerifier.class
> {noformat}
> Solutions would be:
> - Fix the classloader so that my custom job does not conflict with internal flink-core classes... pretty hard
> - Remove the dependency somehow.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)