You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/05/06 11:05:07 UTC
[GitHub] [airflow] thejens opened a new issue #15697: Reset "upstream_failed" on upstream success
thejens opened a new issue #15697:
URL: https://github.com/apache/airflow/issues/15697
**Description**
When manually retrying a failed task, and it succeeds, the downstream tasks with an "upstream_failed" state should be reset, or possibly added to a new/different state as the upstream is now indeed successful. This also applies when manually setting the task to success.
**Use case / motivation**
When retrying a failed task so it succeeds, or manually setting it to success, I believe the default expected behaviour is for downstream tasks to be automatically "unblocked". I realize the current behaviour of retaining the "upstream_failed" state is in some cases desired, but I believe those situations are in minority.
The best example would be a task that has several branches of downstream tasks. When it fails, all those branches fail with "upstream_failed". Once the task is manually set to success (or manually re-triggered) it is sometimes painful to go through all those downstream branches and clear them one-by-one, especially for more complex workflows where the recursive clearing might be too aggressive.
My current workaround is to manually search for all upstream_failed tasks and clear them from the tasks overview interface, or to use recursive clear on the now successful "root" task, then quickly marking it back to success before it re-triggers.
**Are you willing to submit a PR?**
Yes! But I wouldn't know where to implement this.
**Related Issues**
n/a
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] thejens edited a comment on issue #15697: Reset "upstream_failed" on upstream success
Posted by GitBox <gi...@apache.org>.
thejens edited a comment on issue #15697:
URL: https://github.com/apache/airflow/issues/15697#issuecomment-833607711
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #15697: Reset "upstream_failed" on upstream success
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #15697:
URL: https://github.com/apache/airflow/issues/15697#issuecomment-833436495
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] thejens commented on issue #15697: Reset "upstream_failed" on upstream success
Posted by GitBox <gi...@apache.org>.
thejens commented on issue #15697:
URL: https://github.com/apache/airflow/issues/15697#issuecomment-898879060
Yes, that's indeed a solution, I'd go as far as saying that should happen automatically when upstream succeeds
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on issue #15697: Reset "upstream_failed" on upstream success
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #15697:
URL: https://github.com/apache/airflow/issues/15697#issuecomment-833438760
How are you manually retrying the task? (There are at least two ways of doing this.)
If you use clear, one of the options is to Include Downstream
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] edwardwang888 commented on issue #15697: Reset "upstream_failed" on upstream success
Posted by GitBox <gi...@apache.org>.
edwardwang888 commented on issue #15697:
URL: https://github.com/apache/airflow/issues/15697#issuecomment-873539266
Hi @thejens, I have a different issue but it is also related to the situation where full recursive clearing of downstream tasks is undesirable. In your case, would having the option to clear only downstream tasks with "upstream failed" solve your issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] thejens commented on issue #15697: Reset "upstream_failed" on upstream success
Posted by GitBox <gi...@apache.org>.
thejens commented on issue #15697:
URL: https://github.com/apache/airflow/issues/15697#issuecomment-833607711
I don't want to clear all downstreams though, and I guess that's the crux of the problem, some of those tasks have been allowed to execute even if the current task failed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org