You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2017/03/25 00:44:42 UTC
[jira] [Commented] (HADOOP-14237) S3A Support Shared Instance
Profile Credentials Across All Instances
[ https://issues.apache.org/jira/browse/HADOOP-14237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15941404#comment-15941404 ]
ASF GitHub Bot commented on HADOOP-14237:
-----------------------------------------
GitHub user kazuyukitanimura opened a pull request:
https://github.com/apache/hadoop/pull/207
HADOOP-14237. S3A Support Shared Instance Profile Credentials Across All Instances
Hi @steveloughran
Yet another patch that I made a few months back.
I explained the issue at https://issues.apache.org/jira/browse/HADOOP-14237
This pull request is aiming for more open discussions rather than a complete solution. It would be great if you could offer your thoughts.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/bloomreach/hadoop HADOOP-14237
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/hadoop/pull/207.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #207
----
commit e3cfeaeddf99aba69e64f502b3d1332dc09b10f4
Author: kazu <ka...@bloomreach.com>
Date: 2017-03-25T00:37:03Z
HADOOP-14237. S3A Support Shared Instance Profile Credentials Across All Instances
----
> S3A Support Shared Instance Profile Credentials Across All Instances
> --------------------------------------------------------------------
>
> Key: HADOOP-14237
> URL: https://issues.apache.org/jira/browse/HADOOP-14237
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
> Affects Versions: 2.8.0, 3.0.0-alpha1, 3.0.0-alpha2, 2.8.1
> Environment: EC2, AWS
> Reporter: Kazuyuki Tanimura
>
> When I run a large Hadoop cluster on EC2 instances with IAM Role, it fails getting the instance profile credentials, eventually all jobs on the cluster fail. Since a number of S3A clients (all mappers and reducers) try to get the credentials, the AWS credential endpoint starts responding 5xx and 4xx error codes.
> SharedInstanceProfileCredentialsProvider.java is sort of trying to solve it, but it still does not share the credentials with other EC2 instances / processes.
> This issue prevents users from creating Hadoop clusters on EC2
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org