[GitHub] [airflow] eitanme opened a new issue #16204: ExternalTaskSensor does not fail when failed_states is set along with a execution_date_fn

[GitBox <gi...@apache.org>]

eitanme opened a new issue #16204:
URL: https://github.com/apache/airflow/issues/16204


   **Apache Airflow version**: 2.x including main
   
   **What happened**:
   
   I am using an `execution_date_fn` in an `ExternalTaskSensor` that also sets `allowed_states=['success']` and `failed_states=['failed']`. When one of the N upstream tasks fails, the sensor will hang forever in the `poke` method because there is a bug in checking for failed_states.
   
   **What you expected to happen**:
   
   I would expect the `ExternalTaskSensor` to fail.
   
   I think this is due to a bug in the `poke` method where it should check if `count_failed > 0` as opposed to checking `count_failed == len(dttm_filter)`. I've created a fix locally that works for my case and will submit a PR for it as reference.
   
   **How to reproduce it**:
   
   Create any `ExternalTaskSensor` that checks for `failed_states` and have one of the external DAGs tasks fail while others succeed.
   
   E.g.
   ```
   ExternalTaskSensor(
           task_id='check_external_dag',                                                     
           external_dag_id='external_dag',                                                               
           external_task_id=None,                              
           execution_date_fn=dependent_date_fn,
           allowed_states=['success'],
           failed_states=['failed'],                                                                     
           check_existence=True)
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #16204: ExternalTaskSensor does not fail when failed_states is set along with a execution_date_fn

[GitBox <gi...@apache.org>]

boring-cyborg[bot] commented on issue #16204:
URL: https://github.com/apache/airflow/issues/16204#issuecomment-852378041


   Thanks for opening your first issue here! Be sure to follow the issue template!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org