[GitHub] [airflow] ChrisEverling commented on a change in pull request #4182: [AIRFLOW-3336] Add new TriggerRule that will consider skipped ancestors as success

[GitBox <gi...@apache.org>]

ChrisEverling commented on a change in pull request #4182: [AIRFLOW-3336] Add new TriggerRule that will consider skipped ancestors as success 
URL: https://github.com/apache/airflow/pull/4182#discussion_r362862918
 
 

 ##########
 File path: airflow/ti_deps/deps/trigger_rule_dep.py
 ##########
 @@ -152,6 +152,11 @@ def _evaluate_trigger_rule(
             elif tr == TR.ONE_FAILED:
                 if upstream_done and not (failed or upstream_failed):
                     ti.set_state(State.SKIPPED, session)
+            elif tr == TR.NONE_FAILED:
+                if upstream_failed or failed:
+                    ti.set_state(State.UPSTREAM_FAILED, session)
+                elif skipped == upstream:
 
 Review comment:
   > Pls note that this can be a feature worth preserving, because if someone wants his task op2 to only run if at least some upstream tasks have succeeded (while others can be skipped), he can use "none_failed" to do that because that's exactly what this line does.
   > 
   > However, if he wants to have his task op2 succeed as long as none of the upstream tasks fail (either skipped for success), he can still use "none_failed" for it. The only workaround he needs to add is to introduce a DummyOperator that always succeeds directly upstream of op2.
   > 
   > So we can keep this feature, but we should make it clear in the doc that if all the upstream tasks are skipped, a task marked with "none_failed" will be skipped. Pls let me know. i don't mind creating a PR to update the doc.
   
   In my opinion, this isn't a reasonable workaround. I'm developing complex DAGs that consist of several tasks with only one upstream dependency-all of which could be potentially skipped. The `none_failed` trigger rule seemed to be the perfect rule for my use case, but anytime I skip a task, skips propagate down until the next task with multiple upstream deps. Wouldn't the reason you mentioned to preserve this feature be satisfied by the ONE_SUCCESS trigger rule? Either way, I believe that `none_failed` communicates "trigger if all upstream are success or skipped" as your docs suggest.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services