You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/12/24 01:51:33 UTC
[GitHub] [airflow] yarongolan opened a new issue #13295: In triggered SubDag (schedule_interval=None), when clearing a successful Subdag, child tasks aren't run
yarongolan opened a new issue #13295:
URL: https://github.com/apache/airflow/issues/13295
**Apache Airflow version**:
Airflow 2.0
**Environment**:
Ubuntu 20.04 (WSL on Windows 10)
- **OS** (e.g. from /etc/os-release):
VERSION="20.04.1 LTS (Focal Fossa)"
- **Kernel** (e.g. `uname -a`):
Linux XXX 4.19.128-microsoft-standard #1 SMP Tue Jun 23 12:58:10 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
**What happened**:
After successfully running a SUBDAG, clearing it (including downstream+recursive) doesn't trigger the inner tasks. Instead, the subdag is marked successful and the inner tasks all stay cleared and aren't re-run.
**What you expected to happen**:
Expected Clear with DownStream + Recursive to re-run all subdag tasks.
<!-- What do you think went wrong? -->
**How to reproduce it**:
1. Using a slightly modified version of https://airflow.apache.org/docs/apache-airflow/stable/concepts.html#subdags:
`from airflow import DAG
from airflow.example_dags.subdags.subdag import subdag
from airflow.operators.dummy import DummyOperator
from airflow.operators.subdag import SubDagOperator
from airflow.utils.dates import days_ago
def subdag(parent_dag_name, child_dag_name, args):
"""
Generate a DAG to be used as a subdag.
:param str parent_dag_name: Id of the parent DAG
:param str child_dag_name: Id of the child DAG
:param dict args: Default arguments to provide to the subdag
:return: DAG to use as a subdag
:rtype: airflow.models.DAG
"""
dag_subdag = DAG(
dag_id=f'{parent_dag_name}.{child_dag_name}',
default_args=args,
start_date=days_ago(2),
schedule_interval=None,
)
for i in range(5):
DummyOperator(
task_id='{}-task-{}'.format(child_dag_name, i + 1),
default_args=args,
dag=dag_subdag,
)
return dag_subdag
DAG_NAME = 'example_subdag_operator'
args = {
'owner': 'airflow',
}
dag = DAG(
dag_id=DAG_NAME, default_args=args, start_date=days_ago(2), schedule_interval=None, tags=['example']
)
start = DummyOperator(
task_id='start',
dag=dag,
)
section_1 = SubDagOperator(
task_id='section-1',
subdag=subdag(DAG_NAME, 'section-1', args),
dag=dag,
)
some_other_task = DummyOperator(
task_id='some-other-task',
dag=dag,
)
section_2 = SubDagOperator(
task_id='section-2',
subdag=subdag(DAG_NAME, 'section-2', args),
dag=dag,
)
end = DummyOperator(
task_id='end',
dag=dag,
)
start >> section_1 >> some_other_task >> section_2 >> end`
2. Run the subdag fully.
3. Clear (with recursive/downstream) any of the SubDags.
4. The Subdag will be marked successful, but if you zoom into the subdag, you'll see all the child tasks were not run.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #13295: In triggered SubDag (schedule_interval=None), when clearing a successful Subdag, child tasks aren't run
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #13295:
URL: https://github.com/apache/airflow/issues/13295#issuecomment-750700792
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb closed issue #13295: In triggered SubDag (schedule_interval=None), when clearing a successful Subdag, child tasks aren't run
Posted by GitBox <gi...@apache.org>.
ashb closed issue #13295:
URL: https://github.com/apache/airflow/issues/13295
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] yarongolan commented on issue #13295: In triggered SubDag (schedule_interval=None), when clearing a successful Subdag, child tasks aren't run
Posted by GitBox <gi...@apache.org>.
yarongolan commented on issue #13295:
URL: https://github.com/apache/airflow/issues/13295#issuecomment-759139396
Just clarifying that this makes the SUBDAG functionality useless
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] yarongolan edited a comment on issue #13295: In triggered SubDag (schedule_interval=None), when clearing a successful Subdag, child tasks aren't run
Posted by GitBox <gi...@apache.org>.
yarongolan edited a comment on issue #13295:
URL: https://github.com/apache/airflow/issues/13295#issuecomment-759139396
Just clarifying that this breaks the SUBDAG functionality completely
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org