You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by "Xiaodong DENG (JIRA)" <ji...@apache.org> on 2018/08/03 14:23:00 UTC

[jira] [Created] (AIRFLOW-2843) ExternalTaskSensor: Add option to cease waiting immediately if the external task doesn't exist

Xiaodong DENG created AIRFLOW-2843:
--------------------------------------

             Summary: ExternalTaskSensor: Add option to cease waiting immediately if the external task doesn't exist
                 Key: AIRFLOW-2843
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2843
             Project: Apache Airflow
          Issue Type: Improvement
          Components: operators
            Reporter: Xiaodong DENG
            Assignee: Xiaodong DENG


h2. Background

*ExternalTaskSensor* will keep waiting (given restrictions of retries, poke_interval, etc), even if the external task specified doesn't exist at all. In some cases, this waiting may still make sense as new DAG may backfill.

But it may be good to provide an option to cease waiting immediately if the external task specified doesn't exist.
h2. Proposal

Provide an argument "check_existence". Set to *True* to check if the external task exists, and immediately cease waiting if the external task does not exist.

The default value is set to *False* (no check or ceasing will happen) so it will not affect any existing DAGs or user expectation.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)