You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2017/12/30 14:33:00 UTC

[jira] [Commented] (HADOOP-15139) [Umbrella] Improvements and fixes for Hadoop shaded client work

    [ https://issues.apache.org/jira/browse/HADOOP-15139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16306788#comment-16306788 ] 

Steve Loughran commented on HADOOP-15139:
-----------------------------------------

+1 for this. 

now that hadoop-aws has cut out s3n:// then it mostly depends on the aws-sdk JAR, which is preshaded: we shouldn't have to reshade the 50+MB JAR, and instead deal with the smaller things like slf4j. Jackson is no longer explicitly used here, as the aws SDK has its own copy

> [Umbrella] Improvements and fixes for Hadoop shaded client work 
> ----------------------------------------------------------------
>
>                 Key: HADOOP-15139
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15139
>             Project: Hadoop Common
>          Issue Type: Bug
>            Reporter: Junping Du
>            Assignee: Bharat Viswanadham
>            Priority: Critical
>
> In HADOOP-11656, we have made great progress in splitting out third-party dependencies from shaded hadoop client jar (hadoop-client-api), put runtime dependencies in hadoop-client-runtime, and have shaded version of hadoop-client-minicluster for test. However, there are still some left work for this feature to be fully completed:
> - We don't have a comprehensive documentation to guide downstream projects/users to use shaded JARs instead of previous JARs
> - We should consider to wrap up hadoop tools (distcp, aws, azure) to have shaded version
> - More issues could be identified when shaded jars are adopted in more test and production environment, like HADOOP-15137.
> Let's have this umbrella JIRA to track all efforts that left to improve hadoop shaded client effort.
> CC [~busbey], [~bharatviswa] and [~vinodkv].



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org