You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "punx120 (via GitHub)" <gi...@apache.org> on 2025/10/03 17:50:06 UTC
[I] Trigger cleanup [airflow]
punx120 opened a new issue, #56372:
URL: https://github.com/apache/airflow/issues/56372
### Apache Airflow version
3.1.0
### If "Other Airflow 2 version" selected, which one?
_No response_
### What happened?
Hi,
This is related to #56366, I would like to do some clean up in `BaseTrigger.cleanup`, the doc says that the cleanup method is run in the triggerer main event loop, but I don't think that's always the case, e.g when I mark a task as failed, I believe it gets cancelled via `future.cancel` (triggerer_job_runner.py:976), which triggers a `asyncio.CancelledError`, when i log `ayncio.current_task()` in my `cleanup` method, it shows it still the coroutine of the trigger which has just been cancelled which means it's not run in the main triggerer event loop unless i'm missing something?
Thanks
Sylvain
### What you think should happen instead?
_No response_
### How to reproduce
Run a trigger, mark the task as failed
### Operating System
linux
### Versions of Apache Airflow Providers
_No response_
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Trigger cleanup [airflow]
Posted by "hussein-awala (via GitHub)" <gi...@apache.org>.
hussein-awala commented on issue #56372:
URL: https://github.com/apache/airflow/issues/56372#issuecomment-3368549284
I confirm there isn’t a bug here.
The async method `run_trigger` is executed within an asyncio task, and the `cleanup` method runs in the `finally` block of that same task. In fact, the method `cleanup` executes in the main event loop (not a new one). Whether it’s called from the same asyncio task as `run_trigger` or from a new task, it still runs in the main event loop.
This doc was introduced in [apache/airflow#30152](https://github.com/apache/airflow/pull/30152), which replaced the synchronous method with an asynchronous one. The goal of that doc was saying that the method runs asynchronously within the event loop.
To be honest, I’m not entirely sure how this might impact your cleanup logic.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Trigger cleanup [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #56372:
URL: https://github.com/apache/airflow/issues/56372#issuecomment-3550003239
This issue has been closed because it has not received response from the issue author.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Trigger cleanup [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #56372:
URL: https://github.com/apache/airflow/issues/56372#issuecomment-3424208039
This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Trigger cleanup [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #56372:
URL: https://github.com/apache/airflow/issues/56372#issuecomment-3519296325
This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Trigger cleanup [airflow]
Posted by "punx120 (via GitHub)" <gi...@apache.org>.
punx120 commented on issue #56372:
URL: https://github.com/apache/airflow/issues/56372#issuecomment-3428785944
@hussein-awala does my answer make sense? is it due to my lack of understanding of asyncio?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Trigger cleanup [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #56372: Trigger cleanup
URL: https://github.com/apache/airflow/issues/56372
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Trigger cleanup [airflow]
Posted by "punx120 (via GitHub)" <gi...@apache.org>.
punx120 commented on issue #56372:
URL: https://github.com/apache/airflow/issues/56372#issuecomment-3369397902
That's fair, one issue that i ran into is that my task is cancelled and i was using `await` within my `cleanup` method when running slow operation like accessing the db, but this doesn't work as the task is cancelled. I suppose it's ok to remove the await and let it run "synchronously" .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org