You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Pad71 (via GitHub)" <gi...@apache.org> on 2025/07/16 06:09:39 UTC
[I] Add support for automatic clearing of failed child DAG tasks in TriggerDagRunOperator [airflow]
Pad71 opened a new issue, #53402:
URL: https://github.com/apache/airflow/issues/53402
### Description
We propose an enhancement to TriggerDagRunOperator that would allow automatic clearing of failed task instances in a triggered DAG run if that DAG run previously failed.###
### Use case/motivation
### Motivation
When using TriggerDagRunOperator in synchronous mode (i.e., wait_for_completion=True), if the triggered child DAG fails (due to a failed task), the parent task also fails. However, when the operator is cleared in the UI (e.g., to retry), the associated failed child DAG run remains unchanged.
As a result:
- The retried TriggerDagRunOperator reuses the same failed DAG run,
- The child DAG run is still in failed state → the operator fails again,
- The user must manually clear failed tasks or the entire child DAG run.
This creates unnecessary friction when using DAG chaining.
### Proposed solution:
Introduce a new optional parameter:
```
TriggerDagRunOperator(
...,
auto_clear_failed_tasks=True
)
```
When set:
- Upon execution (or possibly in the clear() method logic),
- If a matching child DAG run exists and is in failed state,
- All failed task instances (and optionally downstreams) in that DAG run are automatically cleared.
This enables retrying the parent task to also retry the corresponding child DAG.
### Benefits
- Smoother reprocessing of chained DAGs
- Reduces manual intervention
- Fits naturally with Airflow's existing retry model
### Related issues
_No response_
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add support for automatic clearing of failed child DAG tasks in TriggerDagRunOperator [airflow]
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on issue #53402:
URL: https://github.com/apache/airflow/issues/53402#issuecomment-3077245381
The clear action may change depends on use case. It needs to be controlled from the UI not from operator code.
What am I missing?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add support for automatic clearing of failed child DAG tasks in TriggerDagRunOperator [airflow]
Posted by "kirubankamaraj (via GitHub)" <gi...@apache.org>.
kirubankamaraj commented on issue #53402:
URL: https://github.com/apache/airflow/issues/53402#issuecomment-3199220998
Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add support for automatic clearing of failed child DAG tasks in TriggerDagRunOperator [airflow]
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on issue #53402:
URL: https://github.com/apache/airflow/issues/53402#issuecomment-3077098394
> However, when the operator is cleared in the UI (e.g., to retry), the associated failed child DAG run remains unchanged.
This is not something that should be controlled by parameter in operator.
You should clear with recursive.
`Recursive - All the tasks in the child dags and parent dags`
https://airflow.apache.org/docs/apache-airflow/3.0.3/core-concepts/dag-run.html#re-run-tasks
@bbovenzi @pierrejeambrun I see that this option is not available in Airflow 3 but it does show up in the docs?
I see in https://github.com/apache/airflow/pull/52130 it was mentioned as something we used to have as default for Airflow 2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add support for automatic clearing of failed child DAG tasks in TriggerDagRunOperator [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on issue #53402:
URL: https://github.com/apache/airflow/issues/53402#issuecomment-3197372846
Go for it @kirubankamaraj, I just assigned you :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add support for automatic clearing of failed child DAG tasks in TriggerDagRunOperator [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on issue #53402:
URL: https://github.com/apache/airflow/issues/53402#issuecomment-3201501773
@Pad71 most likely no. 2.11 is bridge release and will not hold any additional new feature. Only security patches and bugfixes. You'll have to upgrade to airflow 3 to benefit from latest new features.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add support for automatic clearing of failed child DAG tasks in TriggerDagRunOperator [airflow]
Posted by "Pad71 (via GitHub)" <gi...@apache.org>.
Pad71 commented on issue #53402:
URL: https://github.com/apache/airflow/issues/53402#issuecomment-3077708182
> The clear action may change depends on use case. It needs to be controlled from the UI not from operator code. What am I missing?
@eladkal
The main point of this request is that currently, clearing a TriggerDagRunOperator task does not allow you to automatically clear failed tasks within the corresponding child DAG run.
This means that if one or more tasks in the child DAG fail, you must first manually fix or clear those failed tasks, then wait for the entire child DAG run to complete successfully, and only afterwards can you mark the TriggerDagRunOperator task as successful.
This creates a significant operational burden for the Operations team — especially in cases where the child DAG run is long-running.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add support for automatic clearing of failed child DAG tasks in TriggerDagRunOperator [airflow]
Posted by "Pad71 (via GitHub)" <gi...@apache.org>.
Pad71 commented on issue #53402:
URL: https://github.com/apache/airflow/issues/53402#issuecomment-3078172621
> I think Elad is right, and the closest we had in AF2 was `recursive` option in clear. I also agree that this shouldn't be controlled by the operator but by the UI.
>
> Recursive controlled `include_subdags` and `include_parentdag`. (is_subdag, parent_dag, etc.) All that was removed with airflow 3 removal of subdags.
>
> Also I don't think this would handle 'external triggers', only subdags (here i'm not sure), so this request might be slightly different and still make sense for AF3. (we want to clear external dependencies basically)
hello @pierrejeambrun Again, if it is done by UI, it's ok.
Problem is, that we can't do it now and it is realy painful for operations team (in our company we have lot's of dags triggered by triggerdagrunoperator tasks).
Recursive option does not work for that use case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add support for automatic clearing of failed child DAG tasks in TriggerDagRunOperator [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on issue #53402:
URL: https://github.com/apache/airflow/issues/53402#issuecomment-3089839139
Yes I think that could be a good feature to add, removing the `needs-triage` tag.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add support for automatic clearing of failed child DAG tasks in TriggerDagRunOperator [airflow]
Posted by "eugeneo17 (via GitHub)" <gi...@apache.org>.
eugeneo17 commented on issue #53402:
URL: https://github.com/apache/airflow/issues/53402#issuecomment-6060020492
I've opened #71730 implementing this as an opt-in `auto_clear_failed_tasks` flag on `TriggerDagRunOperator`. I'd appreciate it if a maintainer could assign this issue to me so the work is tracked here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add support for automatic clearing of failed child DAG tasks in TriggerDagRunOperator [airflow]
Posted by "Pad71 (via GitHub)" <gi...@apache.org>.
Pad71 commented on issue #53402:
URL: https://github.com/apache/airflow/issues/53402#issuecomment-3104490488
> [@Pad71](https://github.com/Pad71) No worries, thanks for reporting that :)
>
> Also it's never to late to become one of the 3400+ contributors of apache airflow, it's the best way to be sure the feature you want gets implemented, and it's a great way to give back to the community for the free software you are using. (just saying 🙂)
@pierrejeambrun Yes, I'm considering it. I think I would actually enjoy it :) But first, I need to find out how the process actually works. I have no experience with this type of development. Do you have any recommendations on what I should study or look into?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add support for automatic clearing of failed child DAG tasks in TriggerDagRunOperator [airflow]
Posted by "kirubankamaraj (via GitHub)" <gi...@apache.org>.
kirubankamaraj commented on issue #53402:
URL: https://github.com/apache/airflow/issues/53402#issuecomment-3181978078
@Pad71 @pierrejeambrun I'm interested on this, shall I analyse and work on this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add support for automatic clearing of failed child DAG tasks in TriggerDagRunOperator [airflow]
Posted by "Pad71 (via GitHub)" <gi...@apache.org>.
Pad71 commented on issue #53402:
URL: https://github.com/apache/airflow/issues/53402#issuecomment-3201089669
@pierrejeambrun , @kirubankamaraj
Is it possible to make this change also in version 2.11.*?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add support for automatic clearing of failed child DAG tasks in TriggerDagRunOperator [airflow]
Posted by "Pad71 (via GitHub)" <gi...@apache.org>.
Pad71 commented on issue #53402:
URL: https://github.com/apache/airflow/issues/53402#issuecomment-3095533264
> [@Pad71](https://github.com/Pad71) Are you willing to work on that and open a PR for this ?
@pierrejeambrun I'm just a user, I'm not that familiar with Python to be able to do this :(
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add support for automatic clearing of failed child DAG tasks in TriggerDagRunOperator [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on issue #53402:
URL: https://github.com/apache/airflow/issues/53402#issuecomment-3078101759
I think Elad is right, and the closest we had in AF2 was `recursive` option in clear. I also agree that this shouldn't be controlled by the operator but by the UI.
Recursive controlled `include_subdags` and `include_parentdag`. (is_subdag, parent_dag, etc.) All that was removed with airflow 3 removal of subdags.
Also I don't think this would handle 'external triggers', only subdags (here i'm not sure), so this request might be slightly different and still make sense for AF3.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add support for automatic clearing of failed child DAG tasks in TriggerDagRunOperator [airflow]
Posted by "Pad71 (via GitHub)" <gi...@apache.org>.
Pad71 commented on issue #53402:
URL: https://github.com/apache/airflow/issues/53402#issuecomment-3182954101
> [@Pad71](https://github.com/Pad71) [@pierrejeambrun](https://github.com/pierrejeambrun) I'm interested on this, shall I analyse and work on this?
It would be great,, thanks :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add support for automatic clearing of failed child DAG tasks in TriggerDagRunOperator [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #53402:
URL: https://github.com/apache/airflow/issues/53402#issuecomment-3986446097
@kirubankamaraj We are unassigning you from this issue as part of our updated [assignment policy](https://github.com/apache/airflow/blob/main/contributing-docs/04_how_to_contribute.rst#contribute-code-changes).
This is not meant to discourage your contribution — quite the opposite! You are still very welcome to work on this issue and submit a PR for it. Simply comment that you are working on it and open a PR when ready.
We found that formal assignments were not working well, as they often prevented others from contributing when the assignee was not actively working on the issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add support for automatic clearing of failed child DAG tasks in TriggerDagRunOperator [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on issue #53402:
URL: https://github.com/apache/airflow/issues/53402#issuecomment-3089840100
@Pad71 Are you willing to work on that and open a PR for this ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add support for automatic clearing of failed child DAG tasks in TriggerDagRunOperator [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on issue #53402:
URL: https://github.com/apache/airflow/issues/53402#issuecomment-3108879901
You can take a look at the contributions located there:
https://github.com/apache/airflow/tree/main/contributing-docs
This covers everything (local env setup, PRs, internal processes etc...)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add support for automatic clearing of failed child DAG tasks in TriggerDagRunOperator [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on issue #53402:
URL: https://github.com/apache/airflow/issues/53402#issuecomment-3101782338
@Pad71 No worries, thanks for reporting that :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add support for automatic clearing of failed child DAG tasks in TriggerDagRunOperator [airflow]
Posted by "Pad71 (via GitHub)" <gi...@apache.org>.
Pad71 commented on issue #53402:
URL: https://github.com/apache/airflow/issues/53402#issuecomment-3078218909
I think, there are several use cases for TriggerDagRunOperator task Clear action:
### Use Case 1: Do not clear child DAG run – raise exception
**Description**: The triggered DAG run is not cleared.
**Behavior**: If the DAG run is still in a failed or running state, the TriggerDagRunOperator task will raise an exception during the next attempt.
**Current status**: ✅ This is the current behavior when using reset_dag_run = False.
**Purpose**: Maintains strict separation and manual control between parent and child DAGs.
### Use Case 2: Clear entire child DAG run
**Description**: When clearing the TriggerDagRunOperator, the entire child DAG run (triggered run) is also cleared.
**Behavior**: All tasks in the child DAG run are cleared (including successful ones).
The DAG run is reset and will be re-executed from scratch.
This may be time-consuming for large DAGs or long-running pipelines.
**Current status**: ✅ This is the default behavior when using reset_dag_run = True.
**Purpose**: Ensures full re-processing of the child DAG, regardless of current state.
### Use Case 3: Clear only failed tasks in child DAG run (with downstream)
**Description**: When the TriggerDagRunOperator task is cleared, the system attempts to clear only failed tasks and their downstreams within the triggered child DAG run — without resetting the entire run.
**Behavior** (if implemented):
The child DAG run is not deleted, but failed tasks and their downstreams are cleared and re-executed.
This can reduce reprocessing time and resource usage significantly.
**Current status**: ❌ Not supported by Airflow out-of-the-box.
**Workaround today:**
The user must manually clear failed tasks in the child DAG (via UI/API/CLI).
Wait until the child DAG finishes successfully.
Then manually mark the TriggerDagRunOperator as successful.
**Operations pain point:**
If the child DAG run takes hours, this introduces significant overhead for Ops teams, who must monitor and intervene manually.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add support for automatic clearing of failed child DAG tasks in TriggerDagRunOperator [airflow]
Posted by "Pad71 (via GitHub)" <gi...@apache.org>.
Pad71 commented on issue #53402:
URL: https://github.com/apache/airflow/issues/53402#issuecomment-3077184579
> > However, when the operator is cleared in the UI (e.g., to retry), the associated failed child DAG run remains unchanged.
>
> This is not something that should be controlled by parameter in operator. You should clear with recursive.
>
> `Recursive - All the tasks in the child dags and parent dags` https://airflow.apache.org/docs/apache-airflow/3.0.3/core-concepts/dag-run.html#re-run-tasks
>
> [@bbovenzi](https://github.com/bbovenzi) [@pierrejeambrun](https://github.com/pierrejeambrun) I see that this option is not available in Airflow 3 but it does show up in the docs? I see in [#52130](https://github.com/apache/airflow/pull/52130) it was mentioned as something we used to have as default for Airflow 2
@eladkal
I believe that sometimes a user may simply want to reuse an existing child DAG run by clearing it entirely or prevent to clear child dag run at all rather than always triggering a new one. Therefore, instead of a single boolean parameter to control this behavior, it might make more sense to introduce a clear_mode option — allowing users to define how the TriggerDagRunOperator should behave when it's cleared.
This could support modes like:
"none" (default) – as-is behaviour
"failed_tasks" – clear only failed tasks in the triggered DAG
This would give users more flexibility when designing DAG chaining or orchestration patterns.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org