[jira] [Commented] (AIRFLOW-2978) Airflow deps breaking on ancient Python packages

["Ash Berlin-Taylor (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596337#comment-16596337 ] 

Ash Berlin-Taylor commented on AIRFLOW-2978:
--------------------------------------------

What command/requirements line etc did you use to install? Do you have some special pip config? I notice you have {{tzlocal-2.0.0b1}} which is a pre-release and doesn't get installed by default - on a fresh virtualenv I don't get this problem.

Of the three packages you listed pendulum is the only one which is pinned to a specific version - the other two are already ranges

> Airflow deps breaking on ancient Python packages
> ------------------------------------------------
>
>                 Key: AIRFLOW-2978
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2978
>             Project: Apache Airflow
>          Issue Type: Bug
>            Reporter: Jon Davies
>            Priority: Major
>
> I have a Python package that depends on Airflow that forms part of our DAG build out process.
> This pipeline installs Airflow 1.10.0 and then the build fails with:
> {code:java}
> Installed /python-packages/pkg/pkg/.eggs/pytzdata-2018.5-py2.7.egg
> Traceback (most recent call last):
>   File "setup.py", line 39, in <module>
>     test_suite="tests"
>   File "/usr/local/lib/python2.7/site-packages/setuptools/__init__.py", line 131, in setup
>     return distutils.core.setup(**attrs)
>   File "/usr/local/lib/python2.7/distutils/core.py", line 151, in setup
>     dist.run_commands()
>   File "/usr/local/lib/python2.7/distutils/dist.py", line 953, in run_commands
>     self.run_command(cmd)
>   File "/usr/local/lib/python2.7/distutils/dist.py", line 972, in run_command
>     cmd_obj.run()
>   File "/usr/local/lib/python2.7/site-packages/setuptools/command/test.py", line 216, in run
>     installed_dists = self.install_dists(self.distribution)
>   File "/usr/local/lib/python2.7/site-packages/setuptools/command/test.py", line 207, in install_dists
>     ir_d = dist.fetch_build_eggs(dist.install_requires)
>   File "/usr/local/lib/python2.7/site-packages/setuptools/dist.py", line 514, in fetch_build_eggs
>     replace_conflicting=True,
>   File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 782, in resolve
>     raise VersionConflict(dist, req).with_context(dependent_req)
> pkg_resources.ContextualVersionConflict: (tzlocal 2.0.0b1 (/python-packages/pkg/pkg/.eggs/tzlocal-2.0.0b1-py2.7.egg), Requirement.parse('tzlocal<2.0.0.0,>=1.5.0.0'), set(['pendulum']))
> {code}
> It would seem that Airflow pins ancient versions here:
> - https://github.com/apache/incubator-airflow/blob/314232cc49ebe75cf12e58af9588123441976647/setup.py#L316
> For example:
> psutil is at 5.4.7
> pygments is at 2.2.0
> pendulum is at 2.0.3
> tzlocal is at 2.0.0b1
> All of these install fine with their latest versions together in a fresh 2.7.15 Docker container.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)