You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2020/01/10 01:44:00 UTC
[jira] [Commented] (AIRFLOW-2279) Clearing Tasks Across DAGs
[ https://issues.apache.org/jira/browse/AIRFLOW-2279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17012375#comment-17012375 ]
ASF GitHub Bot commented on AIRFLOW-2279:
-----------------------------------------
kaxil commented on pull request #6633: [AIRFLOW-2279] Clear tasks across DAGs if marked by ExternalTaskMarker
URL: https://github.com/apache/airflow/pull/6633
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
> Clearing Tasks Across DAGs
> --------------------------
>
> Key: AIRFLOW-2279
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2279
> Project: Apache Airflow
> Issue Type: Improvement
> Reporter: Achal Soni
> Assignee: Qian Yu
> Priority: Major
> Attachments: cross_dag_ui_screenshot.png
>
>
> At Stripe, we commonly have discrete dags that depend on each other by leveraging ExternalTaskSensors. We also find ourselves routinely wanting to not only clear tasks and their downstream tasks in a particular dag, but also their downstream tasks in their dependent dags (linked by ExternalTaskSensors).
> We currently have extended Airflow to handle this by modifying the webapp and cli tool to optionally clear dependent tasks across multiple dags (see attached screenshot).
> We want to open the floor for discussion with the larger Airflow community about the usage of ExternalTaskSensors and specifically how to handle clearing across dags. We are interested in learning more about the accepted practices in this regard, and are very open/willing to contribute in this area if there is interest!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)