You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@airflow.apache.org by Jarek Potiuk <Ja...@polidea.com> on 2020/12/01 15:01:33 UTC
New PIP and Airflow 1.10.*
For those who want to install Airflow 1.10.* today and failing. Please
check your PIP version.
PIP 20.3 was released yesterday (https://pip.pypa.io/en/stable/news/) and
it contains a new resolver that might break installation for older versions
of Airflow.
The solution (for now) is to downgrade to 20.2.4 before installing Airflow:
pip install --upgrade pip==20.2.4
Our Dockerfiles (both for 2.0 and upcoming 1.10.14) will have pip pinned to
2.20.4 for now: https://github.com/apache/airflow/pull/12738
I have also managed to get non-conflicting dependencies a few days ago for
both 2.0 and 1.10 and while I had problems with the new resolver when I
tested it in the old PIP I will try to make it works.
Hopefully, both 2.0 and the upcoming 1.10.14 will work with the new pip.
Thank you for your understanding!
J.
--
Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer
M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>
Re: New PIP and Airflow 1.10.*
Posted by Jarek Potiuk <Ja...@polidea.com>.
FYI. Seems that there are quite many problems with many projects for the
new PIP.
If you want to avoid frustrations - you better downgrade to 20.2.4 now!.
If you want to follow the discussions, there are few interesting issues
(most of them affecting us as well):
- https://github.com/pypa/pip/issues/9215
- https://github.com/pypa/pip/issues/9209
- https://github.com/pypa/pip/issues/9206
- https://github.com/pypa/pip/issues/9203
- https://github.com/pypa/pip/issues/9197
- https://github.com/pypa/pip/issues/9200
- https://github.com/pypa/pip/issues/9217
J.
On Thu, Dec 3, 2020 at 12:25 PM Jarek Potiuk <Ja...@polidea.com>
wrote:
> Some progress on that one:
>
> * The PIP team acknowledged the issue and they work on a fix (but other
> issues have higher priority): https://github.com/pypa/pip/issues/9203
> * I have an idea (based on comments from oauthlib team) how we can fix it
> - stay tuned: https://github.com/oauthlib/oauthlib/issues/744
>
> J.
>
>
> On Wed, Dec 2, 2020 at 5:17 PM Jarek Potiuk <Ja...@polidea.com>
> wrote:
>
>> Unfortunately - while it seems that the constraint fixes I implemented
>> last week solve the problems that I observed previously, some of our
>> dependencies are still not ready to work with the new resolver:
>>
>> ERROR: Requested oauthlib[signedtoken]>=1.0.0 from
>> https://files.pythonhosted.org/packages/e5/54/4f96c51b171cf3a64a04b8c5167268803205bc5943b5cdf70bd770727b88/oauthlib-1.1.0-1.tar.gz#sha256=0f786c5573248a38efa86c48c59c0c93140ac836ab2a246aeefd8f9039e999ba
>> (from jira->apache-airflow==1.10.13) has different version in metadata:
>> '1.1.0'
>>
>> I am afraid for 1.10.14 we will have to limit PIP to the previous version
>> for now or use the legacy resolver.
>>
>> I've opened two issues for that - one in PIP and one in oauthlib (looks
>> like oauthlb's fault, but maybe there is a workaround or possibility to
>> turn such errors into warnings): https://github.com/pypa/pip/issues/9203
>> https://github.com/oauthlib/oauthlib/issues/744
>>
>> I will update the docs now and cherry-pick the PIP limitation in our
>> dockerfiles as I do not think we can wait for a fix. With 2.0 we might be
>> more lucky.
>>
>> J.
>>
>>
>>
>>
>> On Wed, Dec 2, 2020 at 3:17 PM Jarek Potiuk <Ja...@polidea.com>
>> wrote:
>>
>>> Ahh! good one :).
>>>
>>> On Wed, Dec 2, 2020 at 5:34 AM Philippe Gagnon <ph...@gmail.com>
>>> wrote:
>>>
>>>> There is also an option to force usage of the old dependency resolver:
>>>>
>>>> --use-deprecated=legacy-resolver
>>>>
>>>> But hopefully we find a more sustainable solution going forward!
>>>>
>>>> BR
>>>>
>>>> Philippe
>>>>
>>>> On Tue, Dec 1, 2020 at 10:01 AM Jarek Potiuk <Ja...@polidea.com>
>>>> wrote:
>>>>
>>>>> For those who want to install Airflow 1.10.* today and failing. Please
>>>>> check your PIP version.
>>>>>
>>>>> PIP 20.3 was released yesterday (https://pip.pypa.io/en/stable/news/)
>>>>> and it contains a new resolver that might break installation for older
>>>>> versions of Airflow.
>>>>>
>>>>> The solution (for now) is to downgrade to 20.2.4 before installing
>>>>> Airflow:
>>>>>
>>>>> pip install --upgrade pip==20.2.4
>>>>>
>>>>> Our Dockerfiles (both for 2.0 and upcoming 1.10.14) will have pip
>>>>> pinned to 2.20.4 for now: https://github.com/apache/airflow/pull/12738
>>>>>
>>>>>
>>>>> I have also managed to get non-conflicting dependencies a few days ago
>>>>> for both 2.0 and 1.10 and while I had problems with the new resolver when I
>>>>> tested it in the old PIP I will try to make it works.
>>>>>
>>>>> Hopefully, both 2.0 and the upcoming 1.10.14 will work with the new
>>>>> pip.
>>>>>
>>>>> Thank you for your understanding!
>>>>>
>>>>> J.
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Jarek Potiuk
>>>>> Polidea <https://www.polidea.com/> | Principal Software Engineer
>>>>>
>>>>> M: +48 660 796 129 <+48660796129>
>>>>> [image: Polidea] <https://www.polidea.com/>
>>>>>
>>>>>
>>>
>>> --
>>>
>>> Jarek Potiuk
>>> Polidea <https://www.polidea.com/> | Principal Software Engineer
>>>
>>> M: +48 660 796 129 <+48660796129>
>>> [image: Polidea] <https://www.polidea.com/>
>>>
>>>
>>
>> --
>>
>> Jarek Potiuk
>> Polidea <https://www.polidea.com/> | Principal Software Engineer
>>
>> M: +48 660 796 129 <+48660796129>
>> [image: Polidea] <https://www.polidea.com/>
>>
>>
>
> --
>
> Jarek Potiuk
> Polidea <https://www.polidea.com/> | Principal Software Engineer
>
> M: +48 660 796 129 <+48660796129>
> [image: Polidea] <https://www.polidea.com/>
>
>
--
Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer
M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>
Re: New PIP and Airflow 1.10.*
Posted by Jarek Potiuk <Ja...@polidea.com>.
FYI. Seems that there are quite many problems with many projects for the
new PIP.
If you want to avoid frustrations - you better downgrade to 20.2.4 now!.
If you want to follow the discussions, there are few interesting issues
(most of them affecting us as well):
- https://github.com/pypa/pip/issues/9215
- https://github.com/pypa/pip/issues/9209
- https://github.com/pypa/pip/issues/9206
- https://github.com/pypa/pip/issues/9203
- https://github.com/pypa/pip/issues/9197
- https://github.com/pypa/pip/issues/9200
- https://github.com/pypa/pip/issues/9217
J.
On Thu, Dec 3, 2020 at 12:25 PM Jarek Potiuk <Ja...@polidea.com>
wrote:
> Some progress on that one:
>
> * The PIP team acknowledged the issue and they work on a fix (but other
> issues have higher priority): https://github.com/pypa/pip/issues/9203
> * I have an idea (based on comments from oauthlib team) how we can fix it
> - stay tuned: https://github.com/oauthlib/oauthlib/issues/744
>
> J.
>
>
> On Wed, Dec 2, 2020 at 5:17 PM Jarek Potiuk <Ja...@polidea.com>
> wrote:
>
>> Unfortunately - while it seems that the constraint fixes I implemented
>> last week solve the problems that I observed previously, some of our
>> dependencies are still not ready to work with the new resolver:
>>
>> ERROR: Requested oauthlib[signedtoken]>=1.0.0 from
>> https://files.pythonhosted.org/packages/e5/54/4f96c51b171cf3a64a04b8c5167268803205bc5943b5cdf70bd770727b88/oauthlib-1.1.0-1.tar.gz#sha256=0f786c5573248a38efa86c48c59c0c93140ac836ab2a246aeefd8f9039e999ba
>> (from jira->apache-airflow==1.10.13) has different version in metadata:
>> '1.1.0'
>>
>> I am afraid for 1.10.14 we will have to limit PIP to the previous version
>> for now or use the legacy resolver.
>>
>> I've opened two issues for that - one in PIP and one in oauthlib (looks
>> like oauthlb's fault, but maybe there is a workaround or possibility to
>> turn such errors into warnings): https://github.com/pypa/pip/issues/9203
>> https://github.com/oauthlib/oauthlib/issues/744
>>
>> I will update the docs now and cherry-pick the PIP limitation in our
>> dockerfiles as I do not think we can wait for a fix. With 2.0 we might be
>> more lucky.
>>
>> J.
>>
>>
>>
>>
>> On Wed, Dec 2, 2020 at 3:17 PM Jarek Potiuk <Ja...@polidea.com>
>> wrote:
>>
>>> Ahh! good one :).
>>>
>>> On Wed, Dec 2, 2020 at 5:34 AM Philippe Gagnon <ph...@gmail.com>
>>> wrote:
>>>
>>>> There is also an option to force usage of the old dependency resolver:
>>>>
>>>> --use-deprecated=legacy-resolver
>>>>
>>>> But hopefully we find a more sustainable solution going forward!
>>>>
>>>> BR
>>>>
>>>> Philippe
>>>>
>>>> On Tue, Dec 1, 2020 at 10:01 AM Jarek Potiuk <Ja...@polidea.com>
>>>> wrote:
>>>>
>>>>> For those who want to install Airflow 1.10.* today and failing. Please
>>>>> check your PIP version.
>>>>>
>>>>> PIP 20.3 was released yesterday (https://pip.pypa.io/en/stable/news/)
>>>>> and it contains a new resolver that might break installation for older
>>>>> versions of Airflow.
>>>>>
>>>>> The solution (for now) is to downgrade to 20.2.4 before installing
>>>>> Airflow:
>>>>>
>>>>> pip install --upgrade pip==20.2.4
>>>>>
>>>>> Our Dockerfiles (both for 2.0 and upcoming 1.10.14) will have pip
>>>>> pinned to 2.20.4 for now: https://github.com/apache/airflow/pull/12738
>>>>>
>>>>>
>>>>> I have also managed to get non-conflicting dependencies a few days ago
>>>>> for both 2.0 and 1.10 and while I had problems with the new resolver when I
>>>>> tested it in the old PIP I will try to make it works.
>>>>>
>>>>> Hopefully, both 2.0 and the upcoming 1.10.14 will work with the new
>>>>> pip.
>>>>>
>>>>> Thank you for your understanding!
>>>>>
>>>>> J.
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Jarek Potiuk
>>>>> Polidea <https://www.polidea.com/> | Principal Software Engineer
>>>>>
>>>>> M: +48 660 796 129 <+48660796129>
>>>>> [image: Polidea] <https://www.polidea.com/>
>>>>>
>>>>>
>>>
>>> --
>>>
>>> Jarek Potiuk
>>> Polidea <https://www.polidea.com/> | Principal Software Engineer
>>>
>>> M: +48 660 796 129 <+48660796129>
>>> [image: Polidea] <https://www.polidea.com/>
>>>
>>>
>>
>> --
>>
>> Jarek Potiuk
>> Polidea <https://www.polidea.com/> | Principal Software Engineer
>>
>> M: +48 660 796 129 <+48660796129>
>> [image: Polidea] <https://www.polidea.com/>
>>
>>
>
> --
>
> Jarek Potiuk
> Polidea <https://www.polidea.com/> | Principal Software Engineer
>
> M: +48 660 796 129 <+48660796129>
> [image: Polidea] <https://www.polidea.com/>
>
>
--
Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer
M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>
Re: New PIP and Airflow 1.10.*
Posted by Jarek Potiuk <Ja...@polidea.com>.
Some progress on that one:
* The PIP team acknowledged the issue and they work on a fix (but other
issues have higher priority): https://github.com/pypa/pip/issues/9203
* I have an idea (based on comments from oauthlib team) how we can fix it -
stay tuned: https://github.com/oauthlib/oauthlib/issues/744
J.
On Wed, Dec 2, 2020 at 5:17 PM Jarek Potiuk <Ja...@polidea.com>
wrote:
> Unfortunately - while it seems that the constraint fixes I implemented
> last week solve the problems that I observed previously, some of our
> dependencies are still not ready to work with the new resolver:
>
> ERROR: Requested oauthlib[signedtoken]>=1.0.0 from
> https://files.pythonhosted.org/packages/e5/54/4f96c51b171cf3a64a04b8c5167268803205bc5943b5cdf70bd770727b88/oauthlib-1.1.0-1.tar.gz#sha256=0f786c5573248a38efa86c48c59c0c93140ac836ab2a246aeefd8f9039e999ba
> (from jira->apache-airflow==1.10.13) has different version in metadata:
> '1.1.0'
>
> I am afraid for 1.10.14 we will have to limit PIP to the previous version
> for now or use the legacy resolver.
>
> I've opened two issues for that - one in PIP and one in oauthlib (looks
> like oauthlb's fault, but maybe there is a workaround or possibility to
> turn such errors into warnings): https://github.com/pypa/pip/issues/9203
> https://github.com/oauthlib/oauthlib/issues/744
>
> I will update the docs now and cherry-pick the PIP limitation in our
> dockerfiles as I do not think we can wait for a fix. With 2.0 we might be
> more lucky.
>
> J.
>
>
>
>
> On Wed, Dec 2, 2020 at 3:17 PM Jarek Potiuk <Ja...@polidea.com>
> wrote:
>
>> Ahh! good one :).
>>
>> On Wed, Dec 2, 2020 at 5:34 AM Philippe Gagnon <ph...@gmail.com>
>> wrote:
>>
>>> There is also an option to force usage of the old dependency resolver:
>>>
>>> --use-deprecated=legacy-resolver
>>>
>>> But hopefully we find a more sustainable solution going forward!
>>>
>>> BR
>>>
>>> Philippe
>>>
>>> On Tue, Dec 1, 2020 at 10:01 AM Jarek Potiuk <Ja...@polidea.com>
>>> wrote:
>>>
>>>> For those who want to install Airflow 1.10.* today and failing. Please
>>>> check your PIP version.
>>>>
>>>> PIP 20.3 was released yesterday (https://pip.pypa.io/en/stable/news/)
>>>> and it contains a new resolver that might break installation for older
>>>> versions of Airflow.
>>>>
>>>> The solution (for now) is to downgrade to 20.2.4 before installing
>>>> Airflow:
>>>>
>>>> pip install --upgrade pip==20.2.4
>>>>
>>>> Our Dockerfiles (both for 2.0 and upcoming 1.10.14) will have pip
>>>> pinned to 2.20.4 for now: https://github.com/apache/airflow/pull/12738
>>>>
>>>> I have also managed to get non-conflicting dependencies a few days ago
>>>> for both 2.0 and 1.10 and while I had problems with the new resolver when I
>>>> tested it in the old PIP I will try to make it works.
>>>>
>>>> Hopefully, both 2.0 and the upcoming 1.10.14 will work with the new pip.
>>>>
>>>> Thank you for your understanding!
>>>>
>>>> J.
>>>>
>>>>
>>>> --
>>>>
>>>> Jarek Potiuk
>>>> Polidea <https://www.polidea.com/> | Principal Software Engineer
>>>>
>>>> M: +48 660 796 129 <+48660796129>
>>>> [image: Polidea] <https://www.polidea.com/>
>>>>
>>>>
>>
>> --
>>
>> Jarek Potiuk
>> Polidea <https://www.polidea.com/> | Principal Software Engineer
>>
>> M: +48 660 796 129 <+48660796129>
>> [image: Polidea] <https://www.polidea.com/>
>>
>>
>
> --
>
> Jarek Potiuk
> Polidea <https://www.polidea.com/> | Principal Software Engineer
>
> M: +48 660 796 129 <+48660796129>
> [image: Polidea] <https://www.polidea.com/>
>
>
--
Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer
M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>
Re: New PIP and Airflow 1.10.*
Posted by Jarek Potiuk <Ja...@polidea.com>.
Some progress on that one:
* The PIP team acknowledged the issue and they work on a fix (but other
issues have higher priority): https://github.com/pypa/pip/issues/9203
* I have an idea (based on comments from oauthlib team) how we can fix it -
stay tuned: https://github.com/oauthlib/oauthlib/issues/744
J.
On Wed, Dec 2, 2020 at 5:17 PM Jarek Potiuk <Ja...@polidea.com>
wrote:
> Unfortunately - while it seems that the constraint fixes I implemented
> last week solve the problems that I observed previously, some of our
> dependencies are still not ready to work with the new resolver:
>
> ERROR: Requested oauthlib[signedtoken]>=1.0.0 from
> https://files.pythonhosted.org/packages/e5/54/4f96c51b171cf3a64a04b8c5167268803205bc5943b5cdf70bd770727b88/oauthlib-1.1.0-1.tar.gz#sha256=0f786c5573248a38efa86c48c59c0c93140ac836ab2a246aeefd8f9039e999ba
> (from jira->apache-airflow==1.10.13) has different version in metadata:
> '1.1.0'
>
> I am afraid for 1.10.14 we will have to limit PIP to the previous version
> for now or use the legacy resolver.
>
> I've opened two issues for that - one in PIP and one in oauthlib (looks
> like oauthlb's fault, but maybe there is a workaround or possibility to
> turn such errors into warnings): https://github.com/pypa/pip/issues/9203
> https://github.com/oauthlib/oauthlib/issues/744
>
> I will update the docs now and cherry-pick the PIP limitation in our
> dockerfiles as I do not think we can wait for a fix. With 2.0 we might be
> more lucky.
>
> J.
>
>
>
>
> On Wed, Dec 2, 2020 at 3:17 PM Jarek Potiuk <Ja...@polidea.com>
> wrote:
>
>> Ahh! good one :).
>>
>> On Wed, Dec 2, 2020 at 5:34 AM Philippe Gagnon <ph...@gmail.com>
>> wrote:
>>
>>> There is also an option to force usage of the old dependency resolver:
>>>
>>> --use-deprecated=legacy-resolver
>>>
>>> But hopefully we find a more sustainable solution going forward!
>>>
>>> BR
>>>
>>> Philippe
>>>
>>> On Tue, Dec 1, 2020 at 10:01 AM Jarek Potiuk <Ja...@polidea.com>
>>> wrote:
>>>
>>>> For those who want to install Airflow 1.10.* today and failing. Please
>>>> check your PIP version.
>>>>
>>>> PIP 20.3 was released yesterday (https://pip.pypa.io/en/stable/news/)
>>>> and it contains a new resolver that might break installation for older
>>>> versions of Airflow.
>>>>
>>>> The solution (for now) is to downgrade to 20.2.4 before installing
>>>> Airflow:
>>>>
>>>> pip install --upgrade pip==20.2.4
>>>>
>>>> Our Dockerfiles (both for 2.0 and upcoming 1.10.14) will have pip
>>>> pinned to 2.20.4 for now: https://github.com/apache/airflow/pull/12738
>>>>
>>>> I have also managed to get non-conflicting dependencies a few days ago
>>>> for both 2.0 and 1.10 and while I had problems with the new resolver when I
>>>> tested it in the old PIP I will try to make it works.
>>>>
>>>> Hopefully, both 2.0 and the upcoming 1.10.14 will work with the new pip.
>>>>
>>>> Thank you for your understanding!
>>>>
>>>> J.
>>>>
>>>>
>>>> --
>>>>
>>>> Jarek Potiuk
>>>> Polidea <https://www.polidea.com/> | Principal Software Engineer
>>>>
>>>> M: +48 660 796 129 <+48660796129>
>>>> [image: Polidea] <https://www.polidea.com/>
>>>>
>>>>
>>
>> --
>>
>> Jarek Potiuk
>> Polidea <https://www.polidea.com/> | Principal Software Engineer
>>
>> M: +48 660 796 129 <+48660796129>
>> [image: Polidea] <https://www.polidea.com/>
>>
>>
>
> --
>
> Jarek Potiuk
> Polidea <https://www.polidea.com/> | Principal Software Engineer
>
> M: +48 660 796 129 <+48660796129>
> [image: Polidea] <https://www.polidea.com/>
>
>
--
Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer
M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>
Re: New PIP and Airflow 1.10.*
Posted by Jarek Potiuk <Ja...@polidea.com>.
Unfortunately - while it seems that the constraint fixes I implemented last
week solve the problems that I observed previously, some of our
dependencies are still not ready to work with the new resolver:
ERROR: Requested oauthlib[signedtoken]>=1.0.0 from
https://files.pythonhosted.org/packages/e5/54/4f96c51b171cf3a64a04b8c5167268803205bc5943b5cdf70bd770727b88/oauthlib-1.1.0-1.tar.gz#sha256=0f786c5573248a38efa86c48c59c0c93140ac836ab2a246aeefd8f9039e999ba
(from jira->apache-airflow==1.10.13) has different version in metadata:
'1.1.0'
I am afraid for 1.10.14 we will have to limit PIP to the previous version
for now or use the legacy resolver.
I've opened two issues for that - one in PIP and one in oauthlib (looks
like oauthlb's fault, but maybe there is a workaround or possibility to
turn such errors into warnings): https://github.com/pypa/pip/issues/9203
https://github.com/oauthlib/oauthlib/issues/744
I will update the docs now and cherry-pick the PIP limitation in our
dockerfiles as I do not think we can wait for a fix. With 2.0 we might be
more lucky.
J.
On Wed, Dec 2, 2020 at 3:17 PM Jarek Potiuk <Ja...@polidea.com>
wrote:
> Ahh! good one :).
>
> On Wed, Dec 2, 2020 at 5:34 AM Philippe Gagnon <ph...@gmail.com>
> wrote:
>
>> There is also an option to force usage of the old dependency resolver:
>>
>> --use-deprecated=legacy-resolver
>>
>> But hopefully we find a more sustainable solution going forward!
>>
>> BR
>>
>> Philippe
>>
>> On Tue, Dec 1, 2020 at 10:01 AM Jarek Potiuk <Ja...@polidea.com>
>> wrote:
>>
>>> For those who want to install Airflow 1.10.* today and failing. Please
>>> check your PIP version.
>>>
>>> PIP 20.3 was released yesterday (https://pip.pypa.io/en/stable/news/)
>>> and it contains a new resolver that might break installation for older
>>> versions of Airflow.
>>>
>>> The solution (for now) is to downgrade to 20.2.4 before installing
>>> Airflow:
>>>
>>> pip install --upgrade pip==20.2.4
>>>
>>> Our Dockerfiles (both for 2.0 and upcoming 1.10.14) will have pip pinned
>>> to 2.20.4 for now: https://github.com/apache/airflow/pull/12738
>>>
>>> I have also managed to get non-conflicting dependencies a few days ago
>>> for both 2.0 and 1.10 and while I had problems with the new resolver when I
>>> tested it in the old PIP I will try to make it works.
>>>
>>> Hopefully, both 2.0 and the upcoming 1.10.14 will work with the new pip.
>>>
>>> Thank you for your understanding!
>>>
>>> J.
>>>
>>>
>>> --
>>>
>>> Jarek Potiuk
>>> Polidea <https://www.polidea.com/> | Principal Software Engineer
>>>
>>> M: +48 660 796 129 <+48660796129>
>>> [image: Polidea] <https://www.polidea.com/>
>>>
>>>
>
> --
>
> Jarek Potiuk
> Polidea <https://www.polidea.com/> | Principal Software Engineer
>
> M: +48 660 796 129 <+48660796129>
> [image: Polidea] <https://www.polidea.com/>
>
>
--
Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer
M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>
Re: New PIP and Airflow 1.10.*
Posted by Jarek Potiuk <Ja...@polidea.com>.
Unfortunately - while it seems that the constraint fixes I implemented last
week solve the problems that I observed previously, some of our
dependencies are still not ready to work with the new resolver:
ERROR: Requested oauthlib[signedtoken]>=1.0.0 from
https://files.pythonhosted.org/packages/e5/54/4f96c51b171cf3a64a04b8c5167268803205bc5943b5cdf70bd770727b88/oauthlib-1.1.0-1.tar.gz#sha256=0f786c5573248a38efa86c48c59c0c93140ac836ab2a246aeefd8f9039e999ba
(from jira->apache-airflow==1.10.13) has different version in metadata:
'1.1.0'
I am afraid for 1.10.14 we will have to limit PIP to the previous version
for now or use the legacy resolver.
I've opened two issues for that - one in PIP and one in oauthlib (looks
like oauthlb's fault, but maybe there is a workaround or possibility to
turn such errors into warnings): https://github.com/pypa/pip/issues/9203
https://github.com/oauthlib/oauthlib/issues/744
I will update the docs now and cherry-pick the PIP limitation in our
dockerfiles as I do not think we can wait for a fix. With 2.0 we might be
more lucky.
J.
On Wed, Dec 2, 2020 at 3:17 PM Jarek Potiuk <Ja...@polidea.com>
wrote:
> Ahh! good one :).
>
> On Wed, Dec 2, 2020 at 5:34 AM Philippe Gagnon <ph...@gmail.com>
> wrote:
>
>> There is also an option to force usage of the old dependency resolver:
>>
>> --use-deprecated=legacy-resolver
>>
>> But hopefully we find a more sustainable solution going forward!
>>
>> BR
>>
>> Philippe
>>
>> On Tue, Dec 1, 2020 at 10:01 AM Jarek Potiuk <Ja...@polidea.com>
>> wrote:
>>
>>> For those who want to install Airflow 1.10.* today and failing. Please
>>> check your PIP version.
>>>
>>> PIP 20.3 was released yesterday (https://pip.pypa.io/en/stable/news/)
>>> and it contains a new resolver that might break installation for older
>>> versions of Airflow.
>>>
>>> The solution (for now) is to downgrade to 20.2.4 before installing
>>> Airflow:
>>>
>>> pip install --upgrade pip==20.2.4
>>>
>>> Our Dockerfiles (both for 2.0 and upcoming 1.10.14) will have pip pinned
>>> to 2.20.4 for now: https://github.com/apache/airflow/pull/12738
>>>
>>> I have also managed to get non-conflicting dependencies a few days ago
>>> for both 2.0 and 1.10 and while I had problems with the new resolver when I
>>> tested it in the old PIP I will try to make it works.
>>>
>>> Hopefully, both 2.0 and the upcoming 1.10.14 will work with the new pip.
>>>
>>> Thank you for your understanding!
>>>
>>> J.
>>>
>>>
>>> --
>>>
>>> Jarek Potiuk
>>> Polidea <https://www.polidea.com/> | Principal Software Engineer
>>>
>>> M: +48 660 796 129 <+48660796129>
>>> [image: Polidea] <https://www.polidea.com/>
>>>
>>>
>
> --
>
> Jarek Potiuk
> Polidea <https://www.polidea.com/> | Principal Software Engineer
>
> M: +48 660 796 129 <+48660796129>
> [image: Polidea] <https://www.polidea.com/>
>
>
--
Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer
M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>
Re: New PIP and Airflow 1.10.*
Posted by Jarek Potiuk <Ja...@polidea.com>.
Ahh! good one :).
On Wed, Dec 2, 2020 at 5:34 AM Philippe Gagnon <ph...@gmail.com>
wrote:
> There is also an option to force usage of the old dependency resolver:
>
> --use-deprecated=legacy-resolver
>
> But hopefully we find a more sustainable solution going forward!
>
> BR
>
> Philippe
>
> On Tue, Dec 1, 2020 at 10:01 AM Jarek Potiuk <Ja...@polidea.com>
> wrote:
>
>> For those who want to install Airflow 1.10.* today and failing. Please
>> check your PIP version.
>>
>> PIP 20.3 was released yesterday (https://pip.pypa.io/en/stable/news/)
>> and it contains a new resolver that might break installation for older
>> versions of Airflow.
>>
>> The solution (for now) is to downgrade to 20.2.4 before installing
>> Airflow:
>>
>> pip install --upgrade pip==20.2.4
>>
>> Our Dockerfiles (both for 2.0 and upcoming 1.10.14) will have pip pinned
>> to 2.20.4 for now: https://github.com/apache/airflow/pull/12738
>>
>> I have also managed to get non-conflicting dependencies a few days ago
>> for both 2.0 and 1.10 and while I had problems with the new resolver when I
>> tested it in the old PIP I will try to make it works.
>>
>> Hopefully, both 2.0 and the upcoming 1.10.14 will work with the new pip.
>>
>> Thank you for your understanding!
>>
>> J.
>>
>>
>> --
>>
>> Jarek Potiuk
>> Polidea <https://www.polidea.com/> | Principal Software Engineer
>>
>> M: +48 660 796 129 <+48660796129>
>> [image: Polidea] <https://www.polidea.com/>
>>
>>
--
Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer
M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>
Re: New PIP and Airflow 1.10.*
Posted by Jarek Potiuk <Ja...@polidea.com>.
Ahh! good one :).
On Wed, Dec 2, 2020 at 5:34 AM Philippe Gagnon <ph...@gmail.com>
wrote:
> There is also an option to force usage of the old dependency resolver:
>
> --use-deprecated=legacy-resolver
>
> But hopefully we find a more sustainable solution going forward!
>
> BR
>
> Philippe
>
> On Tue, Dec 1, 2020 at 10:01 AM Jarek Potiuk <Ja...@polidea.com>
> wrote:
>
>> For those who want to install Airflow 1.10.* today and failing. Please
>> check your PIP version.
>>
>> PIP 20.3 was released yesterday (https://pip.pypa.io/en/stable/news/)
>> and it contains a new resolver that might break installation for older
>> versions of Airflow.
>>
>> The solution (for now) is to downgrade to 20.2.4 before installing
>> Airflow:
>>
>> pip install --upgrade pip==20.2.4
>>
>> Our Dockerfiles (both for 2.0 and upcoming 1.10.14) will have pip pinned
>> to 2.20.4 for now: https://github.com/apache/airflow/pull/12738
>>
>> I have also managed to get non-conflicting dependencies a few days ago
>> for both 2.0 and 1.10 and while I had problems with the new resolver when I
>> tested it in the old PIP I will try to make it works.
>>
>> Hopefully, both 2.0 and the upcoming 1.10.14 will work with the new pip.
>>
>> Thank you for your understanding!
>>
>> J.
>>
>>
>> --
>>
>> Jarek Potiuk
>> Polidea <https://www.polidea.com/> | Principal Software Engineer
>>
>> M: +48 660 796 129 <+48660796129>
>> [image: Polidea] <https://www.polidea.com/>
>>
>>
--
Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer
M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>
Re: New PIP and Airflow 1.10.*
Posted by Philippe Gagnon <ph...@gmail.com>.
There is also an option to force usage of the old dependency resolver:
--use-deprecated=legacy-resolver
But hopefully we find a more sustainable solution going forward!
BR
Philippe
On Tue, Dec 1, 2020 at 10:01 AM Jarek Potiuk <Ja...@polidea.com>
wrote:
> For those who want to install Airflow 1.10.* today and failing. Please
> check your PIP version.
>
> PIP 20.3 was released yesterday (https://pip.pypa.io/en/stable/news/) and
> it contains a new resolver that might break installation for older versions
> of Airflow.
>
> The solution (for now) is to downgrade to 20.2.4 before installing Airflow:
>
> pip install --upgrade pip==20.2.4
>
> Our Dockerfiles (both for 2.0 and upcoming 1.10.14) will have pip pinned
> to 2.20.4 for now: https://github.com/apache/airflow/pull/12738
>
> I have also managed to get non-conflicting dependencies a few days ago for
> both 2.0 and 1.10 and while I had problems with the new resolver when I
> tested it in the old PIP I will try to make it works.
>
> Hopefully, both 2.0 and the upcoming 1.10.14 will work with the new pip.
>
> Thank you for your understanding!
>
> J.
>
>
> --
>
> Jarek Potiuk
> Polidea <https://www.polidea.com/> | Principal Software Engineer
>
> M: +48 660 796 129 <+48660796129>
> [image: Polidea] <https://www.polidea.com/>
>
>