You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2019/11/25 19:03:00 UTC

[jira] [Commented] (AIRFLOW-3682) Default aws_conn_id is wrong for emr_base_sensor and emr_create_job_flow_operator

    [ https://issues.apache.org/jira/browse/AIRFLOW-3682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16981808#comment-16981808 ] 

ASF subversion and git services commented on AIRFLOW-3682:
----------------------------------------------------------

Commit 10052e52c2c51062db26f075b2902d14881a0d93 in airflow's branch refs/heads/master from laakejs
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=10052e5 ]

[AIRFLOW-3682] Use aws_default in EMR related operators (#4465)




> Default aws_conn_id is wrong for emr_base_sensor and emr_create_job_flow_operator
> ---------------------------------------------------------------------------------
>
>                 Key: AIRFLOW-3682
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-3682
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: aws
>    Affects Versions: 1.10.1
>            Reporter: Jonas Laake
>            Assignee: Jonas Laake
>            Priority: Minor
>             Fix For: 2.0.0
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> Change default aws_conn_id in emr_terminate_job_flow_operator, emr_create_job_flow_operator and emr_add_steps_operator for consistency with emr_base_sensor
> In the emr_base_sensor, the default aws_conn_id is set to 'aws_default'. It should be the same for the emr_add_steps_operator, emr_terminate_job_flow_operator and the emr_create_job_flow_operator, so they land inside the same boto3 session.
> For the current implementation it leads to errors where the EMR cluster is not found by the emr_step_sensor.
> Solution is to change aws_conn_id in both files to 'aws_default'.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)