[jira] [Created] (AIRFLOW-5328) EmrCreateJobFlowOperator success when EMR isn't ready

["Elad (Jira)" <ji...@apache.org>]

Elad created AIRFLOW-5328:
-----------------------------

             Summary: EmrCreateJobFlowOperator success when EMR isn't ready
                 Key: AIRFLOW-5328
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-5328
             Project: Apache Airflow
          Issue Type: Bug
          Components: aws
    Affects Versions: 1.10.4
            Reporter: Elad


EmrCreateJobFlowOperator returns almost instantly.

It shouldn't return until the Emr cluster id ready.
It doesn't make sense to submit tasks to EMR when the cluster isn't ready.

Currently, tasks are submitted and waiting until the cluster will finish its initialization. 

This can be checked with Waiter:

[https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/emr.html]

something like:
{code:java}
waiter = client.get_waiter('cluster_running').wait(ClusterId=response['JobFlowId'], WaiterConfig={'Delay': 32, 'MaxAttempts': 60}){code}



--
This message was sent by Atlassian Jira
(v8.3.2#803003)