You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@hive.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2018/03/05 15:40:00 UTC

[jira] [Commented] (HIVE-18861) druid-hdfs-storage is pulling in hadoop-aws-2.7.x and aws SDK, creating classpath problems on hadoop 3.x

    [ https://issues.apache.org/jira/browse/HIVE-18861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16386225#comment-16386225 ] 

Steve Loughran commented on HIVE-18861:
---------------------------------------

Patch 001: pulls the hadoop JAR and the aws sdk version, which is imported in parallel.

> druid-hdfs-storage is pulling in hadoop-aws-2.7.x and aws SDK, creating classpath problems on hadoop 3.x
> --------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-18861
>                 URL: https://issues.apache.org/jira/browse/HIVE-18861
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Druid integration
>    Affects Versions: 3.0.0
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>            Priority: Major
>         Attachments: HIVE-18861-001.patch
>
>
> druid-hdfs-storage JAR is transitively pulling in hadoop-aws JAR 2.7.3, which creates classpath problems as a set of aws-sdk 1.10.77 JARs get on the CP, even with Hadoop 3 & its move to a full aws-sdk-bundle JAR.
> Two options
> # exclude the dependency
> # force it up to whatever ${hadoop.version} is, so make it consistent



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)