You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2020/10/06 18:11:00 UTC
[jira] [Commented] (AIRFLOW-3823) Exclude BranchPythonOperator
branch's downstream tasks from the tasks to skip
[ https://issues.apache.org/jira/browse/AIRFLOW-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17209015#comment-17209015 ]
ASF GitHub Bot commented on AIRFLOW-3823:
-----------------------------------------
rika commented on pull request #4666:
URL: https://github.com/apache/airflow/pull/4666#issuecomment-704454509
Hello, I started getting this behaviour when I updated my airflow from `1.10.11` to `1.10.12`.
I tested with the example in: https://issues.apache.org/jira/browse/AIRFLOW-3823
And it's skipping the t4 and t5 tasks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
> Exclude BranchPythonOperator branch's downstream tasks from the tasks to skip
> -----------------------------------------------------------------------------
>
> Key: AIRFLOW-3823
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3823
> Project: Apache Airflow
> Issue Type: Improvement
> Reporter: Bas Harenslak
> Assignee: Bas Harenslak
> Priority: Major
> Fix For: 1.10.3
>
> Attachments: Greenshot 2019-02-07 19.48.16.png, Greenshot 2019-02-07 20.48.54.png, image-2019-02-07-16-45-07-900.png, image-2019-02-07-16-50-15-163.png
>
>
> I encountered this edge case where one of the BranchPythonOperator's options was a downstream task of another option. This resulted in the task being skipped, even though it was eligible to be run as normal (depending on your definition of "normal" - either prefer to run all downstream tasks of the chosen task, or either prefer to skip any task that is not chosen). For example:
> !image-2019-02-07-16-45-07-900.png!
> t4 is an option of the branch task, but also a downstream task of t1. IMO it shouldn't be skipped.
> I see several options:
> # hack around by adding dummy task between branch - t4 and set TriggerRule.ONE_SUCCESS for t4 (and educate people to do this....)
> # throw exception and don't allow this
> # when creating the list of tasks to skip, exclude tasks that are downstream tasks of the selected task
> I opt for option #3 and will create PR for this, resulting in:
> !image-2019-02-07-16-50-15-163.png!
> Branch task completed successfully and selected t1, but since t4 is one of its downstream tasks, it wasn't skipped.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)