You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Zixun Zhang (JIRA)" <ji...@apache.org> on 2019/05/09 16:28:00 UTC
[jira] [Commented] (AIRFLOW-4098) gcp_api dependency conflicts
[ https://issues.apache.org/jira/browse/AIRFLOW-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16836518#comment-16836518 ]
Zixun Zhang commented on AIRFLOW-4098:
--------------------------------------
Experience the same issue on 1.10.2
ERROR: google-cloud-spanner 1.8.0 has requirement google-cloud-core<0.30dev,>=0.29.0, but you'll have google-cloud-core 0.28.1 which is incompatible.
ERROR: google-cloud-bigquery 1.11.2 has requirement google-cloud-core<0.30dev,>=0.29.0, but you'll have google-cloud-core 0.28.1 which is incompatible.
> gcp_api dependency conflicts
> ----------------------------
>
> Key: AIRFLOW-4098
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4098
> Project: Apache Airflow
> Issue Type: Bug
> Components: gcp
> Affects Versions: 1.10.2
> Environment: Ubuntu 16.04.6 LTS
> Python 2.7 / Python 3.5 / Python 3.6
> Reporter: Manuel Rodríguez Guimeráns
> Priority: Major
>
> The {{gcp_api}} extra contains inconsistencies in its subdependencies. When installed with {{pip}} it completes with a warning, but {{pipenv}} and {{poetry}} are not so forgiving and it cannot be installed at all.
> Note that on the master branch as of 419e30 (where {{gcp_api}} is now called just {{gcp}}) there are still dependency errors as well.
> h3. pip with python 2.7
> {code:java}
> $ mkvirtualenv -p /usr/bin/python2.7 airflow_pip
> ...
> $ pip --version
> pip 19.0.3 from /home/manu/.local/share/virtualenvs/airflow_pip/local/lib/python2.7/site-packages/pip (python 2.7)
> $ SLUGIFY_USES_TEXT_UNIDECODE=yes pip install 'apache-airflow[gcp_api]==1.10.2'
> ...
> google-cloud-bigquery 1.10.0 has requirement google-cloud-core<0.30dev,>=0.29.0, but you'll have google-cloud-core 0.28.1 which is incompatible.
> google-cloud-spanner 1.8.0 has requirement google-cloud-core<0.30dev,>=0.29.0, but you'll have google-cloud-core 0.28.1 which is incompatible.
> ...
> {code}
> h3. pipenv with python 3.6
> {code:java}
> $ mkdir airflow_pipenv
> $ cd airflow_pipenv
> $ pipenv --version
> pipenv, version 2018.11.26
> $ pipenv --python 3.6
> Creating a virtualenv for this project...
> Pipfile: /tmp/airflow_pipenv/Pipfile
> Using /home/manu/.pyenv/versions/3.6.6/bin/python3.6 (3.6.6) to create virtualenv...
> ⠋ Creating virtual environment...Already using interpreter /home/manu/.pyenv/versions/3.6.6/bin/python3.6
> Using base prefix '/home/manu/.pyenv/versions/3.6.6'
> New python executable in /home/manu/.local/share/virtualenvs/airflow_pipenv-ePWbOumO/bin/python3.6
> Also creating executable in /home/manu/.local/share/virtualenvs/airflow_pipenv-ePWbOumO/bin/python
> Installing setuptools, pip, wheel...done.
> ✔ Successfully created virtual environment!
> Virtualenv location: /home/manu/.local/share/virtualenvs/airflow_pipenv-ePWbOumO
> $ airflow_pipenv pipenv install 'apache-airflow[gcp_api]==1.10.2'
> Installing apache-airflow[gcp_api]==1.10.2...
> Adding apache-airflow to Pipfile's [packages]...
> ✔ Installation Succeeded
> Pipfile.lock (e3de4b) out of date, updating to (ca72e7)...
> Locking [dev-packages] dependencies...
> Locking [packages] dependencies...
> ✘ Locking Failed!
> [pipenv.exceptions.ResolutionFailure]: File "/home/manu/.pyenv/versions/3.6.6/lib/python3.6/site-packages/pipenv/resolver.py", line 69, in resolve
> [pipenv.exceptions.ResolutionFailure]: req_dir=requirements_dir
> [pipenv.exceptions.ResolutionFailure]: File "/home/manu/.pyenv/versions/3.6.6/lib/python3.6/site-packages/pipenv/utils.py", line 726, in resolve_deps
> [pipenv.exceptions.ResolutionFailure]: req_dir=req_dir,
> [pipenv.exceptions.ResolutionFailure]: File "/home/manu/.pyenv/versions/3.6.6/lib/python3.6/site-packages/pipenv/utils.py", line 480, in actually_resolve_deps
> [pipenv.exceptions.ResolutionFailure]: resolved_tree = resolver.resolve()
> [pipenv.exceptions.ResolutionFailure]: File "/home/manu/.pyenv/versions/3.6.6/lib/python3.6/site-packages/pipenv/utils.py", line 395, in resolve
> [pipenv.exceptions.ResolutionFailure]: raise ResolutionFailure(message=str(e))
> [pipenv.exceptions.ResolutionFailure]: pipenv.exceptions.ResolutionFailure: ERROR: ERROR: Could not find a version that matches google-cloud-core<0.29dev,<0.30dev,>=0.28.0,>=0.29.0
> [pipenv.exceptions.ResolutionFailure]: Tried: 0.20.0, 0.20.0, 0.21.0, 0.21.0, 0.22.0, 0.22.0, 0.22.1, 0.22.1, 0.23.0, 0.23.0, 0.23.1, 0.23.1, 0.24.0, 0.24.0, 0.24.1, 0.24.1, 0.25.0, 0.25.0, 0.26.0, 0.26.0, 0.27.0, 0.27.0, 0.27.1, 0.27.1, 0.28.0, 0.28.0, 0.28.1, 0.28.1, 0.29.0, 0.29.0, 0.29.1, 0.29.1
> [pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
> First try clearing your dependency cache with $ pipenv lock --clear, then try the original command again.
> Alternatively, you can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
> Hint: try $ pipenv lock --pre if it is a pre-release dependency.
> ERROR: ERROR: Could not find a version that matches google-cloud-core<0.29dev,<0.30dev,>=0.28.0,>=0.29.0
> Tried: 0.20.0, 0.20.0, 0.21.0, 0.21.0, 0.22.0, 0.22.0, 0.22.1, 0.22.1, 0.23.0, 0.23.0, 0.23.1, 0.23.1, 0.24.0, 0.24.0, 0.24.1, 0.24.1, 0.25.0, 0.25.0, 0.26.0, 0.26.0, 0.27.0, 0.27.0, 0.27.1, 0.27.1, 0.28.0, 0.28.0, 0.28.1, 0.28.1, 0.29.0, 0.29.0, 0.29.1, 0.29.1
> There are incompatible versions in the resolved dependencies.
> [pipenv.exceptions.ResolutionFailure]: req_dir=requirements_dir
> [pipenv.exceptions.ResolutionFailure]: File "/home/manu/.pyenv/versions/3.6.6/lib/python3.6/site-packages/pipenv/utils.py", line 726, in resolve_deps
> [pipenv.exceptions.ResolutionFailure]: req_dir=req_dir,
> [pipenv.exceptions.ResolutionFailure]: File "/home/manu/.pyenv/versions/3.6.6/lib/python3.6/site-packages/pipenv/utils.py", line 480, in actually_resolve_deps
> [pipenv.exceptions.ResolutionFailure]: resolved_tree = resolver.resolve()
> [pipenv.exceptions.ResolutionFailure]: File "/home/manu/.pyenv/versions/3.6.6/lib/python3.6/site-packages/pipenv/utils.py", line 395, in resolve
> [pipenv.exceptions.ResolutionFailure]: raise ResolutionFailure(message=str(e))
> [pipenv.exceptions.ResolutionFailure]: pipenv.exceptions.ResolutionFailure: ERROR: ERROR: Could not find a version that matches google-cloud-core<0.29dev,<0.30dev,>=0.28.0,>=0.29.0
> [pipenv.exceptions.ResolutionFailure]: Tried: 0.20.0, 0.20.0, 0.21.0, 0.21.0, 0.22.0, 0.22.0, 0.22.1, 0.22.1, 0.23.0, 0.23.0, 0.23.1, 0.23.1, 0.24.0, 0.24.0, 0.24.1, 0.24.1, 0.25.0, 0.25.0, 0.26.0, 0.26.0, 0.27.0, 0.27.0, 0.27.1, 0.27.1, 0.28.0, 0.28.0, 0.28.1, 0.28.1, 0.29.0, 0.29.0, 0.29.1, 0.29.1
> [pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
> First try clearing your dependency cache with $ pipenv lock --clear, then try the original command again.
> Alternatively, you can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
> Hint: try $ pipenv lock --pre if it is a pre-release dependency.
> ERROR: ERROR: Could not find a version that matches google-cloud-core<0.29dev,<0.30dev,>=0.28.0,>=0.29.0
> Tried: 0.20.0, 0.20.0, 0.21.0, 0.21.0, 0.22.0, 0.22.0, 0.22.1, 0.22.1, 0.23.0, 0.23.0, 0.23.1, 0.23.1, 0.24.0, 0.24.0, 0.24.1, 0.24.1, 0.25.0, 0.25.0, 0.26.0, 0.26.0, 0.27.0, 0.27.0, 0.27.1, 0.27.1, 0.28.0, 0.28.0, 0.28.1, 0.28.1, 0.29.0, 0.29.0, 0.29.1, 0.29.1
> There are incompatible versions in the resolved dependencies.
> {code}
> h3. poetry with python 3.5
> {code:java}
> $ pyenv shell 3.5.5
> $ poetry --version
> Poetry 0.12.11
> $ poetry new airflow_poetry
> Created package airflow_poetry in airflow_poetry
> $ cd airflow_poetry
> $ poetry add apache-airflow==1.10.2 -E gcp_api
> Creating virtualenv airflow-poetry-py3.5 in /home/manu/.cache/pypoetry/virtualenvs
> Updating dependencies
> Resolving dependencies... (3.0s)
>
> [SolverProblemError]
> Because no versions of google-cloud-spanner match >1.7.1,<1.8.0 || >1.8.0
> and google-cloud-spanner (1.7.1) depends on google-cloud-core (>=0.29.0,<0.30dev), google-cloud-spanner (>=1.7.1,<1.8.0 || >1.8.0) requires google-cloud-core (>=0.29.0,<0.30dev).
> And because google-cloud-spanner (1.8.0) depends on google-cloud-core (>=0.29.0,<0.30dev), google-cloud-spanner (>=1.7.1) requires google-cloud-core (>=0.29.0,<0.30dev).
> And because apache-airflow (1.10.2) depends on google-cloud-bigtable (0.31.0) which depends on google-cloud-core (>=0.28.0,<0.29dev), google-cloud-spanner (>=1.7.1) is incompatible with apache-airflow (1.10.2
> ).
> So, because airflow-poetry depends on apache-airflow (=1.10.2) which depends on google-cloud-spanner (>=1.7.1), version solving failed.
>
> add [-D|--dev] [--git GIT] [--path PATH] [-E|--extras EXTRAS] [--optional] [--python PYTHON] [--platform PLATFORM] [--allow-prereleases] [--dry-run] [--] <name> (<name>)...
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)