You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/07/01 12:30:39 UTC
[GitHub] [airflow] hterik commented on issue #24778: "Reversing" the branching concept, to execute tasks based on final downstream dependencies.
hterik commented on issue #24778:
URL: https://github.com/apache/airflow/issues/24778#issuecomment-1172296081
> If there is a condition of which downstream task can be skipped then place `ShortCircuitOperator` to achieve that.
What i'm looking for is a condition of which _upstream_ tasks that can be skipped :)
The deciding task must still be run first of course, sortof like the "configure" stage of a build-system.
Excuse my paint-skills, imagine following (this is not the same dag as my code from OP):
Here if one decides that A is the desired outcome, then all of x+s1+s2+t1+t2+t3 must be run, but s3 can be skipped.
With the BranchOperator, one must run both of the tracks completely, because otherwise there is no way to get s2 into t3, if i'm not mistaken.
This is the most simple case, real world dag dependencies can be a lot more complex than that.
I don't have experience with the `ShortCircuitOperator`, will look into it a bit more to understand better.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org