You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "xiankgx (via GitHub)" <gi...@apache.org> on 2023/06/21 01:41:48 UTC
[GitHub] [airflow] xiankgx opened a new issue, #32039: Failure to run all DAGs for Airflow in Kubernetes
xiankgx opened a new issue, #32039:
URL: https://github.com/apache/airflow/issues/32039
### Apache Airflow version
Other Airflow 2 version (please specify below)
### What happened
Hi, we are running Airflow on Kubernetes on AWS EKS.
It has been running for several years issues.
All of a sudden it seems none of the DAG runs work and trying to get the logs results in this.
Please help, there are much DAG runs scheduled daily and this error is catastrophic.
*** Falling back to local log
*** Trying to get logs (last 100 lines) from worker pod ***
*** Unable to fetch logs from worker pod ***
(400)
Reason: Bad Request
HTTP response headers: HTTPHeaderDict({'Audit-Id': '3bda0e6f-0ea8-4c2b-a950-20a21ed162f0', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Date': 'Wed, 21 Jun 2023 01:35:12 GMT', 'Content-Length': '136'})
HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"name must be provided","reason":"BadRequest","code":400}\n'
### What you think should happen instead
Not sure what happened.
### How to reproduce
Unsure
### Operating System
Kubernetes
### Versions of Apache Airflow Providers
Unsure
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
Kubernetes
### Anything else
No right now
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] arjunanan6 commented on issue #32039: Failure to run all DAGs for Airflow in Kubernetes
Posted by "arjunanan6 (via GitHub)" <gi...@apache.org>.
arjunanan6 commented on issue #32039:
URL: https://github.com/apache/airflow/issues/32039#issuecomment-1600106974
It looks like your ClusterRole might be broken. I'll take a look at our config and get back to you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] xiankgx commented on issue #32039: Failure to run all DAGs for Airflow in Kubernetes
Posted by "xiankgx (via GitHub)" <gi...@apache.org>.
xiankgx commented on issue #32039:
URL: https://github.com/apache/airflow/issues/32039#issuecomment-1600069481
Trying to restart the scheduler and it just seems to keep restarting. What can I do?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] hussein-awala commented on issue #32039: Failure to run all DAGs for Airflow in Kubernetes
Posted by "hussein-awala (via GitHub)" <gi...@apache.org>.
hussein-awala commented on issue #32039:
URL: https://github.com/apache/airflow/issues/32039#issuecomment-1601541708
Are you using the official helm chart or bitnami chart?
Which executor are you using? is it KubernetesExecutor?
Are you using remote logging? If not, do you have a shared persistent volume between the different services?
I convert it to discussion, and we can reopen it if needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] hussein-awala closed issue #32039: Failure to run all DAGs for Airflow in Kubernetes
Posted by "hussein-awala (via GitHub)" <gi...@apache.org>.
hussein-awala closed issue #32039: Failure to run all DAGs for Airflow in Kubernetes
URL: https://github.com/apache/airflow/issues/32039
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #32039: Failure to run all DAGs for Airflow in Kubernetes
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #32039:
URL: https://github.com/apache/airflow/issues/32039#issuecomment-1599944474
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org