You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Qian Yu (Jira)" <ji...@apache.org> on 2020/02/29 06:43:00 UTC

[jira] [Commented] (AIRFLOW-1230) Upstream_failed tasks are not executed when the DAG is restarted after failure

    [ https://issues.apache.org/jira/browse/AIRFLOW-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17048223#comment-17048223 ] 

Qian Yu commented on AIRFLOW-1230:
----------------------------------

The problem is seems to be caused by what you are doing here:
{quote}4. Execute the DAG again:
 airflow backfill new_example_bash_operator -s 2017-05-02 -e 2017-05-02
{quote}
 

After fixing run_before_loop, you should not "backfill". Instead, just Clear it with Downstream selected and it'll re-run and downstream tasks such as runme_0, runme_1, etc will re-run too.

"backfill" is not for this purpose. It's for backpopulating missed historical dates.

 

Also, a lot has changed since this Jira was reported. We should close this and open a new one if there are new issues.

> Upstream_failed tasks are not executed when the DAG is restarted after failure
> ------------------------------------------------------------------------------
>
>                 Key: AIRFLOW-1230
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1230
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: DAG, DagRun
>    Affects Versions: 1.8.0, 1.8.1
>         Environment: CentOS release 6.8 (Final)
> Python 2.7.10
>            Reporter: Rostislaw Krassow
>            Priority: Major
>         Attachments: DAG_cleared.gif, DAG_failed.gif, DAG_partly_successful.gif, new_example_bash_operator.py
>
>
> The issue is reproducible with Airflow 1.8.0 and 1.8.1.
> Steps to reproduce:
> 1. Use the attached DAG [new_example_bash_operator|^new_example_bash_operator.py]. This is a modified standard example DAG. The task run_before_loop will fail because it contains an error.
> 2. Execute the DAG:
> airflow backfill new_example_bash_operator -s 2017-05-02 -e 2017-05-02
> The task run_before_loop fails as expected. The DAG fails. The screenshot of the UI is attached. !DAG_failed.gif!
> All dependend tasks  like runme_0, runme_1, runme_2 are going to state "upstream_failed".
> 3. Fix the BashOperator in the task run_before_loop (just put "echo 1" as bash_command).
> 4. Execute the DAG again:
> airflow backfill new_example_bash_operator -s 2017-05-02 -e 2017-05-02
> Expected behavior:
> Restart of the DAG leads to execution of all failed tasks including upstream_failed tasks.
> Observed behavior:
> 1. The failed task is not restarted.
> 2. All dependend tasks are not restarted.
> 3. In order to get the DAG reexecuted its state must be cleared manually:
> airflow clear -f -c new_example_bash_operator -s 2017-05-02 -e 2017-05-02  
> !DAG_cleared.gif!
> After the clearance the same DAG can be restarted:
> airflow backfill new_example_bash_operator -s 2017-05-02 -e 2017-05-02
> Then the task run_before_loop is executed. All other tasks still remain in state "upstream_failed". !DAG_partly_successful.gif!
> To get all tasks executed their state must be cleared explicitely.
> Conclusion:
> This is a blocker issue for productive usage. We run several dozens of DAGs with high number of tasks. In the production environment there are always failed tasks. In such cases the restart of the DAG must be simple possible.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)