You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/10/12 07:47:33 UTC
[GitHub] [airflow] mielvds opened a new issue #11456: OSError: [Errno 38] Socket operation on non-socket
mielvds opened a new issue #11456:
URL: https://github.com/apache/airflow/issues/11456
I run into an problem with Python 3.8.5, which was originally reported by @BrainLogic is a closed issue:
Hi Airflow Team,
I tried to run airflow 1.10.10 with Python 3.8.3 (the same for 3.8.0) on macOs Mojave 10.14.6 as a demon process `airflow webserver -p 8080 -D`
this way does not work:
```
Traceback (most recent call last):
File "/.../airflow/.venv/bin/airflow", line 37, in <module>
args.func(args)
File "/.../airflow/.venv/lib/python3.8/site-packages/airflow/utils/cli.py", line 75, in wrapper
return f(*args, **kwargs)
File "/.../airflow/.venv/lib/python3.8/site-packages/airflow/bin/cli.py", line 966, in webserver
ctx = daemon.DaemonContext(
File "/.../airflow/.venv/lib/python3.8/site-packages/daemon/daemon.py", line 280, in __init__
detach_process = is_detach_process_context_required()
File "/.../airflow/.venv/lib/python3.8/site-packages/daemon/daemon.py", line 801, in is_detach_process_context_required
if is_process_started_by_init() or is_process_started_by_superserver():
File "/.../airflow/.venv/lib/python3.8/site-packages/daemon/daemon.py", line 778, in is_process_started_by_superserver
if is_socket(stdin_fd):
File "/.../airflow/.venv/lib/python3.8/site-packages/daemon/daemon.py", line 743, in is_socket
file_socket = socket.fromfd(fd, socket.AF_INET, socket.SOCK_RAW)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/socket.py", line 544, in fromfd
return socket(family, type, proto, nfd)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/socket.py", line 231, in __init__
_socket.socket.__init__(self, family, type, proto, fileno)
OSError: [Errno 38] Socket operation on non-socket
```
probably, it can be related to https://bugs.python.org/issue39685
_Originally posted by @BrainLogic in https://github.com/apache/airflow/issues/8674#issuecomment-635489054_
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #11456: OSError: [Errno 38] Socket operation on non-socket
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #11456:
URL: https://github.com/apache/airflow/issues/11456
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #11456: OSError: [Errno 38] Socket operation on non-socket
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #11456:
URL: https://github.com/apache/airflow/issues/11456#issuecomment-706942835
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #11456: OSError: [Errno 38] Socket operation on non-socket
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #11456:
URL: https://github.com/apache/airflow/issues/11456#issuecomment-756147218
PR to limit it in 2.0.1 here: https://github.com/apache/airflow/pull/13540
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on issue #11456: OSError: [Errno 38] Socket operation on non-socket
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #11456:
URL: https://github.com/apache/airflow/issues/11456#issuecomment-708479312
How frequently does this bug happen? From the stacktrace I'd say this might be a bug in the daemon module?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #11456: OSError: [Errno 38] Socket operation on non-socket
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #11456:
URL: https://github.com/apache/airflow/issues/11456#issuecomment-756142716
The problem is likely with the `python-daemon` version you have installed. The 'python-daemon` 2.2.4 solves this problem (it is Python 3.8 only) and it was released a year ago
Probably you have not installed airflow in recommended way and you have some really old version of this dependency (and probably others as well).
The recommended installation method is here: https://airflow.apache.org/docs/apache-airflow/stable/installation.html#installation
Can you please confirm that? I will add extra limitation to Airflow to require 2.2.4 or above for python 3.8, but in the meantime you can just upgrade python daemon and it should work (but it's best to install airflow according to our recommendations to avoid similar problems).
More info:
According to latest constraints we are using latest `python-daemon` released: 2.2.4: https://github.com/apache/airflow/blob/constraints-2-0/constraints-3.8.txt#L322
The error you are experiencing was fixed in 2.2.4 version:
https://pagure.io/python-daemon/blob/55ea71020486ea299ecbc03aec8e7f9dca2f5a85/f/ChangeLog
```
* Create the socket and catch “non-socket” errors.
Closes: Pagure #34. Thanks to Miro Hrončok for the bug report and
patch.
```
The issue (python 3.8) is described here: https://pagure.io/python-daemon/issue/34
And the fix is here: https://pagure.io/python-daemon/c/b708912
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dinsaw commented on issue #11456: OSError: [Errno 38] Socket operation on non-socket
Posted by GitBox <gi...@apache.org>.
dinsaw commented on issue #11456:
URL: https://github.com/apache/airflow/issues/11456#issuecomment-749993602
I have also faced this same issue on Python 3.8.5, Ubuntu 20.04.1 LTS.
As a workaround, I am starting both scheduler and web server on nohup instead of using `-D` option.
```
nohup airflow scheduler >> airflow-scheduler-nohup.log 2>&1 &
nohup airflow webserver -p 5044 >> airflow-webserver-nohup.log 2>&1 &
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org