You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/02/07 11:17:13 UTC
[GitHub] [airflow] yuqian90 commented on issue #7276: [AIRFLOW-5391] Add a
new dependency rule to evaluate branching result
yuqian90 commented on issue #7276: [AIRFLOW-5391] Add a new dependency rule to evaluate branching result
URL: https://github.com/apache/airflow/pull/7276#issuecomment-583347659
@ashb @kaxil @feluelle @tooptoop4 Hoping to get a review here.
I have updated the PR a bit. The logic is still the same, but the code that does the branching evaluation has been moved into a new dependency rule inside ``ti_deps/deps/branch_dep.py``.
When `BranchDep` is evaluated, it sets the state of a task to "SKIPPED" if it has a parent that decided to skip it.
For example, this comes from "example_short_circuit_operatorschedule":
For a DAG that looks like this, `condition_is_False` decided to skip `false_1`. Before this PR, the problem is if someone clears `false_1`, it'll execute. This is very counter-intuitive for users:
After this PR, when the scheduler evaluates the dependency rules for `false_1`, `BranchDep` will skip `false_1` because it knows that the parent task `condition_is_False` had already decided to skip `false_1`. In other words, this makes the "skipped" status "sticky".
So after this PR, the DAG will look like this after someone clears `false_1`. This is much more intuitive:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services