You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "jorgeortizfuentes (via GitHub)" <gi...@apache.org> on 2023/10/16 13:42:30 UTC
[I] DAGs Stuck in Queued State with Google Composer and Airflow 2.5.3 [airflow]
jorgeortizfuentes opened a new issue, #34967:
URL: https://github.com/apache/airflow/issues/34967
### Apache Airflow version
Other Airflow 2 version (please specify below)
### What happened
Hi, I am running Airflow 2.5.3 on Google Composer 2.4.5, and I am experiencing issues with my DAGs.
All my DAGs are stuck in the queued state, including a test one that does nothing but print the current time.
Previously, I was running Airflow on a machine within a Kubernetes cluster, and the DAGs were functioning without any problems. Occasionally, I had issues and would resolve them by restarting the pods, which would then get everything back on track.
Now, on Google Composer, I've attempted to restart the Scheduler using the following command:
```
kubectl rollout restart deployment airflow-scheduler -n composer-2-4-5-airflow-2-5-3-6623aac0
```
However, this approach hasn't resolved the issue. The DAGs remain stuck in the queued state. I've also tried pausing and removing all DAGs, but to no avail.
### What you think should happen instead
_No response_
### How to reproduce
I only installed Airflow through Google Composer.
### Operating System
Google Cloud Composer
### Versions of Apache Airflow Providers
_No response_
### Deployment
Google Cloud Composer
### Deployment details
Resources
Workloads configuration
Scheduler
2 schedulers with 1 vCPU, 2 GB memory, 1 GB storage each
Triggerer
2 triggerers with 0.5 vCPU, 0.5 GB memory, 1 GB storage each
Web server
1 vCPU, 2 GB memory, 1 GB storage
Worker
Autoscaling between 1 and 8 workers, with 1 vCPU, 2 GB memory, 1 GB storage each
Core infrastructure
Environment size
Medium
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] DAGs Stuck in Queued State with Google Composer and Airflow 2.5.3 [airflow]
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on issue #34967:
URL: https://github.com/apache/airflow/issues/34967#issuecomment-1764700052
Queue state issues were mostly resolved with Airflow 2.6.0
I suggest to update Airflow version first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] DAGs Stuck in Queued State with Google Composer and Airflow 2.5.3 [airflow]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #34967:
URL: https://github.com/apache/airflow/issues/34967#issuecomment-1764512562
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] DAGs Stuck in Queued State with Google Composer and Airflow 2.5.3 [airflow]
Posted by "Taragolis (via GitHub)" <gi...@apache.org>.
Taragolis closed issue #34967: DAGs Stuck in Queued State with Google Composer and Airflow 2.5.3
URL: https://github.com/apache/airflow/issues/34967
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org