You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Scato Eggen (Jira)" <ji...@apache.org> on 2019/12/13 09:48:00 UTC
[jira] [Updated] (AIRFLOW-6252) DAG.clear() does not clear certain
task instances
[ https://issues.apache.org/jira/browse/AIRFLOW-6252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Scato Eggen updated AIRFLOW-6252:
---------------------------------
Description:
Use case:
When running a task from a unit test, I need to clear the DAG as demonstrated in [https://blog.godatadriven.com/testing-and-debugging-apache-airflow].
Observed behavior:
At first, everything worked as expected, but at some point the task wouldn't start anymore. Upon inspection, it turned out that the task instance had status RUNNING but no job_id. airflow.models.taskinstance.clear_task_instances was keeping the task instance in state running and TaskInstance._check_and_change_state_before_execution started returning False.
A factor that could have contributed was that changed the task from having retries=1 to retries=0. Also, I was stepping through the code with the debugger.
Expected behavior:
DAG.clear() clears all task instances, even the ones that have weird states.
Workaround:
Delete ~/airflow/airflow.db and rerun airflow initdb.
was:
Use case:
When running a task from a unit test, I need to clear the DAG as demonstrated in [https://blog.godatadriven.com/testing-and-debugging-apache-airflow].
Observed behavior:
At first, everything worked as expected, but at some point the task would start anymore. Upon inspection, it turned out that the task instance had status RUNNING but no job_id. airflow.models.taskinstance.clear_task_instances was keeping the task instance in state running and TaskInstance._check_and_change_state_before_execution started returning False.
A factor that could have contributed was that changed the task from having retries=1 to retries=0. Also, I was stepping through the code with the debugger.
Expected behavior:
DAG.clear() clears all task instances, even the ones that have weird states.
Workaround:
Delete ~/airflow/airflow.db and rerun airflow initdb.
> DAG.clear() does not clear certain task instances
> -------------------------------------------------
>
> Key: AIRFLOW-6252
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6252
> Project: Apache Airflow
> Issue Type: Bug
> Components: DAG
> Affects Versions: 1.10.3
> Reporter: Scato Eggen
> Priority: Major
>
> Use case:
> When running a task from a unit test, I need to clear the DAG as demonstrated in [https://blog.godatadriven.com/testing-and-debugging-apache-airflow].
> Observed behavior:
> At first, everything worked as expected, but at some point the task wouldn't start anymore. Upon inspection, it turned out that the task instance had status RUNNING but no job_id. airflow.models.taskinstance.clear_task_instances was keeping the task instance in state running and TaskInstance._check_and_change_state_before_execution started returning False.
> A factor that could have contributed was that changed the task from having retries=1 to retries=0. Also, I was stepping through the code with the debugger.
> Expected behavior:
> DAG.clear() clears all task instances, even the ones that have weird states.
> Workaround:
> Delete ~/airflow/airflow.db and rerun airflow initdb.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)