You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2018/06/21 17:52:00 UTC
[jira] [Updated] (HADOOP-15553) DOCO - not clear how to integrate
hadoop-aws and aws-sdk jars with S3A
[ https://issues.apache.org/jira/browse/HADOOP-15553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Loughran updated HADOOP-15553:
------------------------------------
Priority: Minor (was: Major)
Component/s: (was: tools)
fs/s3
documentation
> DOCO - not clear how to integrate hadoop-aws and aws-sdk jars with S3A
> ----------------------------------------------------------------------
>
> Key: HADOOP-15553
> URL: https://issues.apache.org/jira/browse/HADOOP-15553
> Project: Hadoop Common
> Issue Type: New Feature
> Components: documentation, fs/s3
> Affects Versions: 2.8.3
> Reporter: t oo
> Priority: Minor
>
> Documentation update, a compatibility matrix would be nice, especially since upgrading to new aws java sdk does that work :
>
> On a hadoop common 2.7.5 cluster, I take AWS Java SDK [1.7.5|https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk/1.7.5] and Hadoop-aws [2.7.5|https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-aws/2.7.5] and use them in my Spark app writing to S3a, this is fine.
> On another cluster (this time hadoop common 2.8.3 cluster) I take AWS Java SDK 1.10.6 and Hadoop-aws 2.8.3 and try to use them in my Spark app writing to S3a, but get errors about missing classes. Turns out this version of aws java sdk is tiny with basically nothing inside.
> Couldn't find any documentation helping users on this incompatibility.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org