You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/12/08 09:36:20 UTC
[GitHub] [airflow] dmgburg opened a new issue #12912: dagrun_timeout doesn't kill task instances on timeout
dmgburg opened a new issue #12912:
URL: https://github.com/apache/airflow/issues/12912
**Apache Airflow version**:
1.10.12
**What happened**:
I created dag with dagrun_timeout=2 minutes.
After 2 minutes dagrun is marked as failed and the next one is started, but task keeps going.
**What you expected to happen**:
Task is killed with dag run as it is done when you mark dagrun failed manually.
**How to reproduce it**:
```
dag = DAG(dag_id='platform.airflow-test',
# FIXME: TypeError: object of type 'NoneType' has no len()
description='',
schedule_interval="0 0 * * *",
start_date=datetime(2020, 7, 1),
max_active_runs=1,
catchup=True,
dagrun_timeout=timedelta(minutes=2))
run_this = BashOperator(
task_id='run_after_loop',
bash_command=' for((i=1;i<=600;i+=1)); do echo "Welcome $i times"; sleep 1; done',
dag=dag,
)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] RNHTTR edited a comment on issue #12912: dagrun_timeout doesn't kill task instances on timeout
Posted by GitBox <gi...@apache.org>.
RNHTTR edited a comment on issue #12912:
URL: https://github.com/apache/airflow/issues/12912#issuecomment-780903216
Would it make sense to get the DAG's task instances and set any tasks to `SKIPPED` state if they haven't finished when the DAG state is marked as `FAILED`?
https://github.com/apache/airflow/blob/master/airflow/jobs/scheduler_job.py#L1694-L1718
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] yogyang commented on issue #12912: dagrun_timeout doesn't kill task instances on timeout
Posted by GitBox <gi...@apache.org>.
yogyang commented on issue #12912:
URL: https://github.com/apache/airflow/issues/12912#issuecomment-755877996
hit this problem too in airflow 2.0
And it blocked the next Dag_run as
```
[2021-01-07 04:35:59,145] {scheduler_job.py:1671} INFO - DAG braze_custom_attrs_v7 already has 1 active runs, not queuing any tasks for run 2020-10-21 08:00:00+00:00
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #12912: dagrun_timeout doesn't kill task instances on timeout
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #12912:
URL: https://github.com/apache/airflow/issues/12912#issuecomment-740503667
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] RNHTTR commented on issue #12912: dagrun_timeout doesn't kill task instances on timeout
Posted by GitBox <gi...@apache.org>.
RNHTTR commented on issue #12912:
URL: https://github.com/apache/airflow/issues/12912#issuecomment-780903216
Would it make sense to get the DAG's task instances and set any tasks to `SHUTDOWN` state if they haven't finished when the DAG state is marked as `FAILED`?
https://github.com/apache/airflow/blob/master/airflow/jobs/scheduler_job.py#L1694-L1718
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil closed issue #12912: dagrun_timeout doesn't kill task instances on timeout
Posted by GitBox <gi...@apache.org>.
kaxil closed issue #12912:
URL: https://github.com/apache/airflow/issues/12912
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] yogyang edited a comment on issue #12912: dagrun_timeout doesn't kill task instances on timeout
Posted by GitBox <gi...@apache.org>.
yogyang edited a comment on issue #12912:
URL: https://github.com/apache/airflow/issues/12912#issuecomment-755877996
hit this problem too in airflow 2.0
And it blocked the next Dag_run as
```
[2021-01-07 04:35:59,145] {scheduler_job.py:1671} INFO - DAG braze_custom_attrs_v7 already has 1 active runs, not queuing any tasks for run 2020-10-21 08:00:00+00:00
```
I'm not sure if the block of the next Dag_run is caused by the same issue:
https://github.com/apache/airflow/issues/13407
@turbaszek could you help check this?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] RNHTTR edited a comment on issue #12912: dagrun_timeout doesn't kill task instances on timeout
Posted by GitBox <gi...@apache.org>.
RNHTTR edited a comment on issue #12912:
URL: https://github.com/apache/airflow/issues/12912#issuecomment-780903216
Would it make sense to get the DAG's task instances and set any tasks to `SKIPPED` state (or a new state e.g. `DAG_TIMED_OUT`) if they haven't finished when the DAG state is marked as `FAILED`?
https://github.com/apache/airflow/blob/master/airflow/jobs/scheduler_job.py#L1694-L1718
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org