You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/08/24 10:00:25 UTC
[GitHub] [airflow] UnightSun opened a new issue #17804: Webserver fail to start with virtualenv: No such file or directory: 'gunicorn': 'gunicorn'
UnightSun opened a new issue #17804:
URL: https://github.com/apache/airflow/issues/17804
<!--
Welcome to Apache Airflow!
Please complete the next sections or the issue will be closed.
-->
**Apache Airflow version**: 2.1.2
<!-- AIRFLOW VERSION IS MANDATORY -->
**OS**: CentOS Linux release 8.2.2004 (Core)
<!-- MANDATORY! You can get it via `cat /etc/oss-release` for example -->
**Apache Airflow Provider versions**:
apache-airflow-providers-celery==2.0.0
apache-airflow-providers-ftp==2.0.0
apache-airflow-providers-http==2.0.0
apache-airflow-providers-imap==2.0.0
apache-airflow-providers-mysql==2.1.0
apache-airflow-providers-redis==2.0.0
apache-airflow-providers-sftp==2.1.0
apache-airflow-providers-sqlite==2.0.0
apache-airflow-providers-ssh==2.1.0
<!-- You can use `pip freeze | grep apache-airflow-providers` (you can leave only relevant ones)-->
**Deployment**: Virtualenv 3.7.5
<!-- e.g. Virtualenv / VM / Docker-compose / K8S / Helm Chart / Managed Airflow Service -->
<!-- Please include your deployment tools and versions: docker-compose, k8s, helm, etc -->
**What happened**:
Got error when I try to start a webserver:
```bash
$ $VENV_PATH/bin/airflow webserver
...
=================================================================
Traceback (most recent call last):
File "/VENV_PATH/bin/airflow", line 8, in <module>
sys.exit(main())
File "/VENV_PATH/lib/python3.7/site-packages/airflow/__main__.py", line 40, in main
args.func(args)
File "/VENV_PATH/lib/python3.7/site-packages/airflow/cli/cli_parser.py", line 48, in command
return func(*args, **kwargs)
File "/VENV_PATH/lib/python3.7/site-packages/airflow/utils/cli.py", line 91, in wrapper
return f(*args, **kwargs)
File "/VENV_PATH/lib/python3.7/site-packages/airflow/cli/commands/webserver_command.py", line 483, in webserver
with subprocess.Popen(run_args, close_fds=True) as gunicorn_master_proc:
File "/home/user/.pyenv/versions/3.7.5/lib/python3.7/subprocess.py", line 800, in __init__
restore_signals, start_new_session)
File "/home/user/.pyenv/versions/3.7.5/lib/python3.7/subprocess.py", line 1551, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'gunicorn': 'gunicorn'
[user@xxxxx xxxxxxxxxx]$ /VENV_PATH/bin/python
Python 3.7.5 (default, Aug 20 2021, 17:16:24)
```
<!-- Please include exact error messages if you can -->
**What you expected to happen**:
Airflow start webserver success
<!-- What do you think went wrong? -->
**How to reproduce it**:
assume we already have `AIRFLOW_HOME`
1. create a venv: `virtualenv xxx`
2. `xxx/bin/pip install apache-airflow`
3. `xxx/bin/airflow webserver`
<!--
As minimally and precisely as possible. Keep in mind we do not have access to your cluster or dags.
If this is a UI bug, please provide a screenshot of the bug or a link to a youtube video of the bug in action
You can include images/screen-casts etc. by drag-dropping the image here.
-->
**Anything else we need to know**:
`cli/commands/webserver_command.py:394` use `gunicorn` as subprocess command, assume it's in PATH.
`airflow webserver` runs well after `export PATH=$PATH:$VENV_PATH/bin`.
<!--
How often does this problem occur? Once? Every time etc?
Any relevant logs to include? Put them here inside fenced
``` ``` blocks or inside a foldable details tag if it's long:
<details><summary>x.log</summary> lots of stuff </details>
-->
**Are you willing to submit a PR?**
I'm not good at python so maybe it's not a good idea for me to submit a PR.
<!---
This is absolutely not required, but we are happy to guide you in contribution process
especially if you already have a good understanding of how to implement the fix.
Airflow is a community-managed project and we love to bring new contributors in.
Find us in #airflow-how-to-pr on Slack!
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #17804: Webserver fail to start with virtualenv: No such file or directory: 'gunicorn': 'gunicorn'
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #17804:
URL: https://github.com/apache/airflow/issues/17804#issuecomment-904527765
I think this is a reasonable assumption to have `virtualenv/bin` on the path when you installed airflow (or anything there).
This happens when you source `activate` script as part of "activation" of the virtualenv and it is described in https://docs.python.org/3/library/venv.html :
> Once a virtual environment has been created, it can be “activated” using a script in the virtual environment’s binary directory. The invocation of the script is platform-specific (<venv> must be replaced by the path of the directory containing the virtual environment):
also there is an alternative method of "activation" of the virtualenv - which is precilsely about prepending the `bin` directory of the venv to your PATH::
> You don’t specifically need to activate an environment; activation just prepends the virtual environment’s binary directory to your path, so that “python” invokes the virtual environment’s Python interpreter and you can run installed scripts without having to use their full path. However, all scripts installed in a virtual environment should be runnable without activating it, and run with the virtual environment’s Python automatically.
I think what **might** not work if you run airlfow command after installing it in venv but without `activating` it. But I think expecting venv to be activated and gunicorn to be on the path is reasonable. Unless some people express different opinion (and propose a solution) I consider that as `won't fix`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on issue #17804: Webserver fail to start with virtualenv: No such file or directory: 'gunicorn': 'gunicorn'
Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #17804:
URL: https://github.com/apache/airflow/issues/17804#issuecomment-904529781
I already working on fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #17804: Webserver fail to start with virtualenv: No such file or directory: 'gunicorn': 'gunicorn'
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #17804:
URL: https://github.com/apache/airflow/issues/17804#issuecomment-904527765
I think this is a reasonable assumption to have `virtualenv/bin` on the path when you installed airflow (or anything else) there.
This happens when you source `activate` script as part of "activation" of the virtualenv and it is described in https://docs.python.org/3/library/venv.html :
> Once a virtual environment has been created, it can be “activated” using a script in the virtual environment’s binary directory. The invocation of the script is platform-specific (`<venv>` must be replaced by the path of the directory containing the virtual environment):
also there is an alternative method of "activation" of the virtualenv - which is precilsely about prepending the `bin` directory of the venv to your PATH::
> You don’t specifically need to activate an environment; activation just prepends the virtual environment’s binary directory to your path, so that “python” invokes the virtual environment’s Python interpreter and you can run installed scripts without having to use their full path. However, all scripts installed in a virtual environment should be runnable without activating it, and run with the virtual environment’s Python automatically.
I think what **might** not work if you run airlfow command after installing it in venv but without `activating` it. But I think expecting venv to be activated and gunicorn to be on the path is reasonable. Unless some people express different opinion (and propose a solution) I consider that as `won't fix`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #17804: Webserver fail to start with virtualenv: No such file or directory: 'gunicorn': 'gunicorn'
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #17804:
URL: https://github.com/apache/airflow/issues/17804#issuecomment-904527765
I think this is a reasonable assumption to have `virtualenv/bin` on the path when you installed airflow (or anything there).
This happens when you source `activate` script as part of "activation" of the virtualenv and it is described in https://docs.python.org/3/library/venv.html :
> Once a virtual environment has been created, it can be “activated” using a script in the virtual environment’s binary directory. The invocation of the script is platform-specific (<venv> must be replaced by the path of the directory containing the virtual environment):
also there is an alternative method of "activation" of the virtualenv - which is precilsely about prepending the `bin` directory of the venv to your PATH::
> You don’t specifically need to activate an environment; activation just prepends the virtual environment’s binary directory to your path, so that “python” invokes the virtual environment’s Python interpreter and you can run installed scripts without having to use their full path. However, all scripts installed in a virtual environment should be runnable without activating it, and run with the virtual environment’s Python automatically.
I think what **might** not work if you run airlfow command after installing it in venv but without `activating` it. But I think expecting venv to be activated and gunicorn to be on the path is reasonable. Unless some people will express different opnion (and propose a solution) I consider that as `won't fix`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #17804: Webserver fail to start with virtualenv: No such file or directory: 'gunicorn': 'gunicorn'
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #17804:
URL: https://github.com/apache/airflow/issues/17804#issuecomment-904527765
I think this is a reasonable assumption to have `virtualenv/bin` on the path when you installed airflow (or anything there).
This happens when you source `activate` script as part of "activation" of the virtualenv and it is described in https://docs.python.org/3/library/venv.html :
> Once a virtual environment has been created, it can be “activated” using a script in the virtual environment’s binary directory. The invocation of the script is platform-specific (`<venv>` must be replaced by the path of the directory containing the virtual environment):
also there is an alternative method of "activation" of the virtualenv - which is precilsely about prepending the `bin` directory of the venv to your PATH::
> You don’t specifically need to activate an environment; activation just prepends the virtual environment’s binary directory to your path, so that “python” invokes the virtual environment’s Python interpreter and you can run installed scripts without having to use their full path. However, all scripts installed in a virtual environment should be runnable without activating it, and run with the virtual environment’s Python automatically.
I think what **might** not work if you run airlfow command after installing it in venv but without `activating` it. But I think expecting venv to be activated and gunicorn to be on the path is reasonable. Unless some people express different opinion (and propose a solution) I consider that as `won't fix`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #17804: Webserver fail to start with virtualenv: No such file or directory: 'gunicorn': 'gunicorn'
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #17804:
URL: https://github.com/apache/airflow/issues/17804#issuecomment-904538653
Cool!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #17804: Webserver fail to start with virtualenv: No such file or directory: 'gunicorn': 'gunicorn'
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #17804:
URL: https://github.com/apache/airflow/issues/17804
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #17804: Webserver fail to start with virtualenv: No such file or directory: 'gunicorn': 'gunicorn'
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #17804:
URL: https://github.com/apache/airflow/issues/17804
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #17804: Webserver fail to start with virtualenv: No such file or directory: 'gunicorn': 'gunicorn'
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #17804:
URL: https://github.com/apache/airflow/issues/17804#issuecomment-904500873
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org