[jira] [Commented] (AIRFLOW-1968) Re-Add Assume Role Support to S3 (And probably other AWS hooks)

["ASF subversion and git services (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-1968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16352359#comment-16352359 ] 

ASF subversion and git services commented on AIRFLOW-1968:
----------------------------------------------------------

Commit 82a65eeca9bc654cec4d0f356c3db70bc8ab6838 in incubator-airflow's branch refs/heads/master from [~baynham]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=82a65ee ]

[AIRFLOW-1968][AIRFLOW-1520] Add role_arn and aws_account_id/aws_iam_role support back to aws hook

In PR2532 (AIRFLOW-1520), the AWS credential code
was refactored into a general
 AWS hook.  When that change was made, the existing
assume role code was
 removed, leaving only ID/Secret credentials as an
option.  Our dags rely on
 role assumption to access external S3 buckets, so
this code re-adds role
 assumption via STS.

Additionally, in order to make this a bit easier,
I changed _get_credentials to
 return a functioning boto3 session which is used
by the public methods to
 initialize clients/resources/whatever.  This
seemed a better route than
 adding another returnval in an already long list.

Closes #2918 from CannibalVox/aws_hook_support_sts


> Re-Add Assume Role Support to S3 (And probably other AWS hooks)
> ---------------------------------------------------------------
>
>                 Key: AIRFLOW-1968
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1968
>             Project: Apache Airflow
>          Issue Type: New Feature
>          Components: aws, boto3
>    Affects Versions: 1.9.0
>            Reporter: Stephen Baynham
>            Assignee: Stephen Baynham
>            Priority: Minor
>
> In Airflow 1.8, you could add S3 connections and use S3 hooks with assumed roles.  When the AWS credential functionality was refactored for 1.9, this capability was removed and only key/secret credentials are now allowed.  This has broken a number of dags at Twitch- it's not unusual for us to connect to buckets from other teams or third parties with an assumed role to decouple this capability from local IAM concerns (having a lot of third parties change the permitted ARN can be labor intensive so we'd prefer not to ever have to do it).
> Please re-add this capability, ideally in the new AWS hook.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)