[jira] [Commented] (HADOOP-11670) Regression: s3a auth setup broken

["Adam Budde (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-11670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14350966#comment-14350966 ] 

Adam Budde commented on HADOOP-11670:
-------------------------------------

Ok, I found the comment included in hadoop-tools/hadoop-aws/test/resources/core-site.xml about creating an auth-keys.xml file, so you can disregard my previous comment.

How would you feel about changing the parameter names in Constants.java to match the changes made in HADOOP-10714? While this would make it so this patch isn't a pure revert, the changed names in HADOOP-10714 are consistent with those used for s3 and s3n. Could save some frustration for users who are on a snapshot build and have already adopted this change.

> Regression: s3a auth setup broken 
> ----------------------------------
>
>                 Key: HADOOP-11670
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11670
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 2.7.0
>            Reporter: Adam Budde
>            Assignee: Adam Budde
>            Priority: Blocker
>             Fix For: 2.7.0
>
>         Attachments: HADOOP-11670-001.patch, HADOOP-11670-003.patch, HADOOP-11670.002.patch
>
>
> One big advantage provided by the s3a filesystem is the ability to use an IAM instance profile in order to authenticate when attempting to access an S3 bucket from an EC2 instance. This eliminates the need to deploy AWS account credentials to the instance or to provide them to Hadoop via the fs.s3a.awsAccessKeyId and fs.s3a.awsSecretAccessKey params.
> The patch submitted to resolve HADOOP-10714 breaks this behavior by using the S3Credentials class to read the value of these two params. The change in question is presented below:
> S3AFileSystem.java, lines 161-170:
> {code}
>     // Try to get our credentials or just connect anonymously
>     S3Credentials s3Credentials = new S3Credentials();
>     s3Credentials.initialize(name, conf);
>     AWSCredentialsProviderChain credentials = new AWSCredentialsProviderChain(
>         new BasicAWSCredentialsProvider(s3Credentials.getAccessKey(),
>                                         s3Credentials.getSecretAccessKey()),
>         new InstanceProfileCredentialsProvider(),
>         new AnonymousAWSCredentialsProvider()
>     );
> {code}
> As you can see, the getAccessKey() and getSecretAccessKey() methods from the S3Credentials class are now used to provide constructor arguments to BasicAWSCredentialsProvider. These methods will raise an exception if the fs.s3a.awsAccessKeyId or fs.s3a.awsSecretAccessKey params are missing, respectively. If a user is relying on an IAM instance profile to authenticate to an S3 bucket and therefore doesn't supply values for these params, they will receive an exception and won't be able to access the bucket.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)