[jira] [Assigned] (AIRFLOW-2680) Don't automatically percolate skipped stae

["Andrei-Alin Popescu (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AIRFLOW-2680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrei-Alin Popescu reassigned AIRFLOW-2680:
--------------------------------------------

    Assignee: Andrei-Alin Popescu

> Don't automatically percolate skipped stae
> ------------------------------------------
>
>                 Key: AIRFLOW-2680
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2680
>             Project: Apache Airflow
>          Issue Type: Improvement
>            Reporter: Andrei-Alin Popescu
>            Assignee: Andrei-Alin Popescu
>            Priority: Minor
>
> Dear Airflow Maintainers,
>  
> As part of our workflow, we have cases where all the upstream of a certain task A can be skipped. In this case, airflow seems to automatically mark A as skipped.
> However, this does not quite work for us, since there are changes external to the DAG which A needs to process, regardless of whether its upstream ran or not. Additionally, we require A to get into an "upstream_failed" state and not run if any its upstream tasks failed.
> I don't see a trigger rule to cover this, so what would be the best way to achieve this? I was thinking we could attach a DummyOperator as an upstream to A, which in a way marks the fact that A depends on some external data and needs to run anyway, but this can get really ugly for big DAGs.
> I was also thinking we could have a new trigger_rule, e.g. "no_failure", which would only trigger tasks if no upstream has failed. It differs from "all_success" in that it will also trigger if all upstream has been skipped, rather than percolating the skipped state on.
> I'd really appreciate your feedback on this, and I'd like to know if in fact there is already a good way of doing this with airflow that I don't know of.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)