You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-dev@hadoop.apache.org by "Kazuyuki Tanimura (JIRA)" <ji...@apache.org> on 2017/03/25 00:35:41 UTC

[jira] [Created] (HADOOP-14237) S3A Support Shared Instance Profile Credentials Across All Instances

Kazuyuki Tanimura created HADOOP-14237:
------------------------------------------

             Summary: S3A Support Shared Instance Profile Credentials Across All Instances
                 Key: HADOOP-14237
                 URL: https://issues.apache.org/jira/browse/HADOOP-14237
             Project: Hadoop Common
          Issue Type: Bug
          Components: fs/s3
    Affects Versions: 3.0.0-alpha2, 3.0.0-alpha1, 2.8.0, 2.8.1
         Environment: EC2, AWS
            Reporter: Kazuyuki Tanimura


When I run a large Hadoop cluster on EC2 instances with IAM Role, it fails getting the instance profile credentials, eventually all jobs on the cluster fail. Since a number of S3A clients (all mappers and reducers) try to get the credentials, the AWS credential endpoint starts responding 5xx and 4xx error codes.

SharedInstanceProfileCredentialsProvider.java is sort of trying to solve it, but it still does not share the credentials with other EC2 instances / processes.

This issue prevents users from creating Hadoop clusters on EC2



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org