[jira] [Commented] (FLINK-5297) Relocate Flink's Hadoop dependency and its transitive dependencies

["Robert Metzger (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-5297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15815050#comment-15815050 ] 

Robert Metzger commented on FLINK-5297:
---------------------------------------

I've created a branch that shades ALL hadoop dependencies (I've only validated this with Hadoop 2.3.0 and 2.7.3).
https://github.com/rmetzger/flink/tree/flink5297

The YARN tests are NOT working, because some web servers of YARN are not properly starting. The problem is that jetty is dynamically loading some servlet classes from a configuration file. The config file is not rewritten, that's why jetty can not find the classes.
I've manually tested the branch on a YARN cluster and it works.

> Relocate Flink's Hadoop dependency and its transitive dependencies
> ------------------------------------------------------------------
>
>                 Key: FLINK-5297
>                 URL: https://issues.apache.org/jira/browse/FLINK-5297
>             Project: Flink
>          Issue Type: Improvement
>          Components: Build System
>    Affects Versions: 1.2.0
>            Reporter: Till Rohrmann
>             Fix For: 1.2.0
>
>
> A user reported that they have a dependency conflict with one of Hadoop's dependencies. More concretely it is the {{aws-java-sdk-*}} dependency which is not backward compatible. The user is dependent on a newer {{aws-java-sdk}} version which cannot be used by Hadoop version 2.7.
> A solution for future dependency conflicts could be to relocate Hadoop's dependencies or even all of the Hadoop dependency.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)