You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/06/30 14:09:28 UTC
[GitHub] [airflow] zartstrom opened a new issue, #24755: Log server on celery worker does not work in IPv6-only setup
zartstrom opened a new issue, #24755:
URL: https://github.com/apache/airflow/issues/24755
### Apache Airflow version
2.1.2
### What happened
I deployed the Airflow helm chart in a Kubernetes cluster that only allows IPv6 traffic.
When I want to look at a task log in the UI there is this message:
```
*** Fetching from: http://airflow-v1-worker-0.airflow-v1-worker.airflow.svc.cluster.local:8793/log/my-dag/my-task/2022-06-28T00:00:00+00:00/1.log
*** Failed to fetch log file from worker. [Errno 111] Connection refused
```
So the webserver cannot fetch the logfile from the worker.
This happens in my opinion because the gunicorn application listens to `0.0.0.0` (IPv4), see [code](https://github.com/apache/airflow/blob/main/airflow/utils/serve_logs.py#L142), and the inter-pod communication in my cluster is IPv6.
### What you think should happen instead
The gunicorn webserver should (configurably) listen to IPv6 traffic.
### How to reproduce
_No response_
### Operating System
Debian GNU/Linux 10 (buster)
### Versions of Apache Airflow Providers
_No response_
### Deployment
Other 3rd-party Helm chart
### Deployment details
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #24755: Log server on celery worker does not work in IPv6-only setup
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #24755:
URL: https://github.com/apache/airflow/issues/24755#issuecomment-1172975282
Yeah. This is good observation. The `Dockerfile` defaults are meant to be useful for people who want to customize the image and use the "released" version of Airflow. What you should look at as a "developer" is breeze. Breeze has a `build-prod-image` command that uses local sources to build the image.
If you are courious - you might run it with `--dry-run` to understand which build-args should be passed to get the `source build` - but you can also look more at https://airflow.apache.org/docs/docker-stack/build-arg-ref.html to understand all the parameters passed. one of them is `AIRFLOW_SOURCES_FROM` set to `.` to use current sources, but there are few others.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] zartstrom commented on issue #24755: Log server on celery worker does not work in IPv6-only setup
Posted by GitBox <gi...@apache.org>.
zartstrom commented on issue #24755:
URL: https://github.com/apache/airflow/issues/24755#issuecomment-1172855771
Hi @potiuk, I'll try to craft a PR, thanks for the inspiration :)
I have one question, maybe you can help or point me to the right source:
What is the best way to create a airflow docker image with a change in the source code?
I read the contributor guide and set up the breeze environment on my machine, but did not find what I was looking for.
What I tried: I did a tiny code change (hardcoded the log server to some other number, `8888`), and then built an image with
```
DOCKER_BUILDKIT=1 docker build . -f Dockerfile --pull --tag my-image:0.0.1
```
and then I deployed that image with a helm chart to our Kubernetes Cluster.
Lookin a the worker log I still see log server running on `0.0.0.0:8379`. So either I build the image wrongly or I miss something else.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #24755: Log server on celery worker does not work in IPv6-only setup
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #24755:
URL: https://github.com/apache/airflow/issues/24755#issuecomment-1171613599
Would you like to fix it - you seem to know, where and how and you have nice environment to test the fix in. And you will not have to wait for someone to pick it up - this is one of the best ways you can contribute back to the software you get for free and join the community of more than 2100 contributors.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #24755: Log server on celery worker does not work in IPv6-only setup
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #24755: Log server on celery worker does not work in IPv6-only setup
URL: https://github.com/apache/airflow/issues/24755
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org