You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "haritamar (via GitHub)" <gi...@apache.org> on 2025/06/25 13:53:57 UTC
[I] Deferred trigger tasks seem to get stuck in Airflow 3.0.2 [airflow]
haritamar opened a new issue, #52247:
URL: https://github.com/apache/airflow/issues/52247
### Apache Airflow version
3.0.2
### If "Other Airflow 2 version" selected, which one?
_No response_
### What happened?
Hi, I'm currently in the process of migrating to Airflow 3.0.2.
As a part of our Airflow project, we are utilizing `TriggerDagRunOperator` with `deferrable=True` to trigger downstream DAGs, and we also use `wait_for_completion=True` since we measure SLAs using the duration of the main DAG.
We noticed that after some time the triggering tasks seem to get stuck at "deferred" even though the underlying DAG is completed:
<img width="558" alt="Image" src="https://github.com/user-attachments/assets/7c94c2f9-6218-4162-be72-a59e60ed28d6" />
I believe what happens behind the scenes is that the async loop in the triggerer gets stuck, there are logs that seem to indicate that:
<img width="1723" alt="Image" src="https://github.com/user-attachments/assets/99aa6595-e46b-4a17-a860-8f762257fa1f" />
### What you think should happen instead?
All deferred tasks should be completed after the underlying DAGs completed (with an acceptable small delay)
### How to reproduce
The example below causes the issue to reproduce for me.
The issue seems to be **statistical** - I need to trigger the DAG about 15 times in a row and it will eventually get stuck.
```python
from datetime import datetime
from airflow.decorators import dag, task
from airflow.providers.standard.operators.trigger_dagrun import TriggerDagRunOperator
@dag(
dag_id="test_defer_issue_a",
start_date=datetime(2025, 6, 25),
schedule=None,
)
def test_defer_issue_a():
TriggerDagRunOperator(
task_id="trigger_b_task",
trigger_dag_id="test_defer_issue_b",
deferrable=True,
wait_for_completion=True
)
test_defer_issue_a()
@dag(
dag_id="test_defer_issue_b",
start_date=datetime(2025, 6, 25),
schedule=None,
)
def test_defer_issue_b():
@task
def test_task():
print("test_task")
test_task()
test_defer_issue_b()
```
### Operating System
MacOS / Linux (both locally and in our dev envs)
### Versions of Apache Airflow Providers
_No response_
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
We use the official helm chart, version 1.16.0.
### Anything else?
The issue might be related to [this](https://github.com/apache/airflow/discussions/51875) one, but I wasn't sure if it's the same thing since the behavior is a bit different in my case.
Feel free to merge if you feel otherwise.
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Deferred trigger tasks seem to get stuck in Airflow 3.0.2 [airflow]
Posted by "haritamar (via GitHub)" <gi...@apache.org>.
haritamar closed issue #52247: Deferred trigger tasks seem to get stuck in Airflow 3.0.2
URL: https://github.com/apache/airflow/issues/52247
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Deferred trigger tasks seem to get stuck in Airflow 3.0.2 [airflow]
Posted by "gopidesupavan (via GitHub)" <gi...@apache.org>.
gopidesupavan commented on issue #52247:
URL: https://github.com/apache/airflow/issues/52247#issuecomment-3019071889
are you happy to close this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Deferred trigger tasks seem to get stuck in Airflow 3.0.2 [airflow]
Posted by "haritamar (via GitHub)" <gi...@apache.org>.
haritamar commented on issue #52247:
URL: https://github.com/apache/airflow/issues/52247#issuecomment-3016055961
@gopidesupavan will do, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Deferred trigger tasks seem to get stuck in Airflow 3.0.2 [airflow]
Posted by "gopidesupavan (via GitHub)" <gi...@apache.org>.
gopidesupavan commented on issue #52247:
URL: https://github.com/apache/airflow/issues/52247#issuecomment-3019070760
Thanks thats good to know. thanks for testing :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Deferred trigger tasks seem to get stuck in Airflow 3.0.2 [airflow]
Posted by "haritamar (via GitHub)" <gi...@apache.org>.
haritamar commented on issue #52247:
URL: https://github.com/apache/airflow/issues/52247#issuecomment-3019013844
@gopidesupavan I tested locally on the main branch and I think it looks good :)
Did so in a way that is a bit hacky (couldn't add it directly to our poetry project since the dependency "airflow-core==3.1.0" doesn't exist yet, but I just installed with "--no-dependencies" in the triggerer docker and restarted it)
Will verify again when an RC is available.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Deferred trigger tasks seem to get stuck in Airflow 3.0.2 [airflow]
Posted by "haritamar (via GitHub)" <gi...@apache.org>.
haritamar commented on issue #52247:
URL: https://github.com/apache/airflow/issues/52247#issuecomment-3020116592
Hey sure thing, I'll close.
Will re-open if it resurfaces.
Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Deferred trigger tasks seem to get stuck in Airflow 3.0.2 [airflow]
Posted by "tirkarthi (via GitHub)" <gi...@apache.org>.
tirkarthi commented on issue #52247:
URL: https://github.com/apache/airflow/issues/52247#issuecomment-3007124798
Is this observed consistently or when there are is high load of trigger dagruns being in deferred mode? What's the version of standard provider being used here? Below issues seem to be related to this issue
https://github.com/apache/airflow/issues/50185
https://github.com/apache/airflow/pull/51279
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Deferred trigger tasks seem to get stuck in Airflow 3.0.2 [airflow]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #52247:
URL: https://github.com/apache/airflow/issues/52247#issuecomment-3004869934
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org