You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/05/10 20:34:20 UTC
[GitHub] [airflow] NickPancakes opened a new issue #15768: PythonVirtualenvOperator fails with error from pip execution: Can not perform a '--user' install.
NickPancakes opened a new issue #15768:
URL: https://github.com/apache/airflow/issues/15768
**Apache Airflow version**: 2.0.2
**Environment**:
- **Hardware configuration**: Macbook Pro 2017
- **OS**: macOS X Catalina 10.15.7
- **Kernel**: Darwin 19.6.0
- **Others**: Docker 20.10.6, docker-compose 1.29.1
**What happened**:
Running the demo `example_python_operator` dag fails on the `virtualenv_python` step. The call to pip via subprocess fails:
`subprocess.CalledProcessError: Command '['/tmp/venvt3_qnug6/bin/pip', 'install', 'colorama==0.4.0']' returned non-zero exit status 1.`
The error coming from pip is: `ERROR: Can not perform a '--user' install. User site-packages are not visible in this virtualenv.`
**What you expected to happen**:
The call to pip succeeds, and the colorama dependency is installed into the virtualenv without attempting to install to user packages. The `example_python_operator` dag execution succeeds.
**How to reproduce it**:
Setup airflow 2.0.2 in docker as detailed in the Quickstart guide: https://airflow.apache.org/docs/apache-airflow/stable/start/docker.html
Once running, enable and manually trigger the `example_python_operator` dag via the webUI.
The dag will fail at the `virtualenv_python` task.
**Anything else we need to know**:
Not a problem with the Airflow 2.0.1 docker-compose. Fairly certain this is due to the addition of the `PIP_USER` environment variable being set to `true` in this PR: https://github.com/apache/airflow/pull/14125
My proposed solution would be to prepend `PIP_USER=false` to the construction of the call to pip within `utils/python_virtualenv.py` here: https://github.com/apache/airflow/blob/25caeda58b50eae6ef425a52e794504bc63855d1/airflow/utils/python_virtualenv.py#L30
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #15768: PythonVirtualenvOperator fails with error from pip execution: Can not perform a '--user' install.
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #15768:
URL: https://github.com/apache/airflow/issues/15768#issuecomment-837975548
It's indeed problem of the image and PIP_USER but this can be very simply addressed. PIP_USER is only needed when you extend the image, but not when you run `airflow` commands. I will simply unset PIP_USER variable in the entrypoint.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #15768: PythonVirtualenvOperator fails with error from pip execution: Can not perform a '--user' install.
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #15768:
URL: https://github.com/apache/airflow/issues/15768#issuecomment-837296054
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #15768: PythonVirtualenvOperator fails with error from pip execution: Can not perform a '--user' install.
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #15768:
URL: https://github.com/apache/airflow/issues/15768
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #15768: PythonVirtualenvOperator fails with error from pip execution: Can not perform a '--user' install.
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #15768:
URL: https://github.com/apache/airflow/issues/15768#issuecomment-837995798
Fix in #15774 ( I tested that PIP_USER is unset there)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on issue #15768: PythonVirtualenvOperator fails with error from pip execution: Can not perform a '--user' install.
Posted by GitBox <gi...@apache.org>.
uranusjr commented on issue #15768:
URL: https://github.com/apache/airflow/issues/15768#issuecomment-837961151
> My proposed solution would be to prepend `PIP_USER=false` to the construction of the call to pip within `utils/python_virtualenv.py` here:
I don’t think that’s right, that command is used to invoke `virtualenv`, not `pip`. And prepending won’t work because that is a Bash-specific syntax and does not work with `subproces.Popen()` (unless `shell=True` but please don’t do that).
One possible fix would be to introduce an `environ` argument to `execute_in_subprocess()` so the `PIP_USER` environment variable can be removed when invoking pip.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org