You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Jarek Potiuk <ja...@potiuk.com> on 2021/09/08 12:28:33 UTC
Implemented workaround to installing airflow with recent setuptools upgrade
Hl everyone,
Tl;DR: if you (as a number of our users) experienced issues with installing
Airflow 2 over the last 2 days, the problem should be workarounded now for
all versions of Airflow 2 and it should install properly even with the
latest setuptools.
Context:
The recent setuptools release (58.0.2) has broken installation for ALL
Airflow 2 released versions (Details of the issue here:
https://github.com/apache/airflow/issues/18075 ). The actual problem is the
deprecation of 2_to_3 configuration in FlaskOpenID. We try to
communicate with FlaskOpenId maintainers, but it's likely they won't fix it
(the last release was in 2016)
This was rather unfortunate, because setuptools need to be installed before
airflow and even with our constraint mechanisms, we cannot force setuptools
to be downgraded.
Solution (Workaround really):
Luckily, (with the help of TP, our PIP expert :) ) we've figured out
and implemented the workaround, so as long as you use the "official" way of
installing airflow (with constraints), Airflow should continue to install
even with the latest setuptools.
Details of the workaround in
https://github.com/apache/airflow/issues/18075#issuecomment-915145181
We basically forked Flask-OpenID, applied removal of the deprecated option
and updated the constraints to include our patched version.
I applied the fix to constraints for all released Airflow 2 versions:
* 2.1.3:
https://github.com/apache/airflow/commit/64508c7c974f56ac05a919deca0b69ad1b9df19e
* 2.1.2:
https://github.com/apache/airflow/commit/641d6cad2b7221e58fdc117a0faa3d12652a10b2
* 2.1.1:
https://github.com/apache/airflow/commit/cf0834b1da66ad8424ecb6344c799a1e9a6abe2e
* 2.1.0:
https://github.com/apache/airflow/commit/1e52c69c608c417c9466dd512e4cd46a65dbad68
* 2.0.2:
https://github.com/apache/airflow/commit/82cc57b5f96f08f6d5976b3e6a3ea33b3b5459d0
* 2.0.1:
https://github.com/apache/airflow/commit/2984d415c1fd9cf535cacf26b57666acadf24706
* 2.0.0:
https://github.com/apache/airflow/commit/dd3e88fd6414ca9fda7108088a959e709b45037f
J.
Re: Implemented workaround to installing airflow with recent
setuptools upgrade
Posted by Jarek Potiuk <ja...@potiuk.com>.
Update: I became a maintainer of Flask-OpenID and we just released a
new-setuptools-compatible 1.3.0 version that fixes the problem entirely.
I updated the constraints for all versions and we are back to normal !
J.
On Wed, Sep 8, 2021 at 2:28 PM Jarek Potiuk <ja...@potiuk.com> wrote:
> Hl everyone,
>
> Tl;DR: if you (as a number of our users) experienced issues with
> installing Airflow 2 over the last 2 days, the problem should be
> workarounded now for all versions of Airflow 2 and it should install
> properly even with the latest setuptools.
>
> Context:
>
> The recent setuptools release (58.0.2) has broken installation for ALL
> Airflow 2 released versions (Details of the issue here:
> https://github.com/apache/airflow/issues/18075 ). The actual problem is
> the deprecation of 2_to_3 configuration in FlaskOpenID. We try to
> communicate with FlaskOpenId maintainers, but it's likely they won't fix it
> (the last release was in 2016)
>
> This was rather unfortunate, because setuptools need to be installed
> before airflow and even with our constraint mechanisms, we cannot force
> setuptools to be downgraded.
>
> Solution (Workaround really):
>
> Luckily, (with the help of TP, our PIP expert :) ) we've figured out
> and implemented the workaround, so as long as you use the "official" way of
> installing airflow (with constraints), Airflow should continue to install
> even with the latest setuptools.
>
> Details of the workaround in
> https://github.com/apache/airflow/issues/18075#issuecomment-915145181
>
> We basically forked Flask-OpenID, applied removal of the deprecated option
> and updated the constraints to include our patched version.
>
> I applied the fix to constraints for all released Airflow 2 versions:
>
> * 2.1.3:
> https://github.com/apache/airflow/commit/64508c7c974f56ac05a919deca0b69ad1b9df19e
> * 2.1.2:
> https://github.com/apache/airflow/commit/641d6cad2b7221e58fdc117a0faa3d12652a10b2
> * 2.1.1:
> https://github.com/apache/airflow/commit/cf0834b1da66ad8424ecb6344c799a1e9a6abe2e
> * 2.1.0:
> https://github.com/apache/airflow/commit/1e52c69c608c417c9466dd512e4cd46a65dbad68
> * 2.0.2:
> https://github.com/apache/airflow/commit/82cc57b5f96f08f6d5976b3e6a3ea33b3b5459d0
> * 2.0.1:
> https://github.com/apache/airflow/commit/2984d415c1fd9cf535cacf26b57666acadf24706
> * 2.0.0:
> https://github.com/apache/airflow/commit/dd3e88fd6414ca9fda7108088a959e709b45037f
>
>
> J.
>
>
Re: Implemented workaround to installing airflow with recent
setuptools upgrade
Posted by Jarek Potiuk <ja...@potiuk.com>.
Update: I became a maintainer of Flask-OpenID and we just released a
new-setuptools-compatible 1.3.0 version that fixes the problem entirely.
I updated the constraints for all versions and we are back to normal !
J.
On Wed, Sep 8, 2021 at 2:28 PM Jarek Potiuk <ja...@potiuk.com> wrote:
> Hl everyone,
>
> Tl;DR: if you (as a number of our users) experienced issues with
> installing Airflow 2 over the last 2 days, the problem should be
> workarounded now for all versions of Airflow 2 and it should install
> properly even with the latest setuptools.
>
> Context:
>
> The recent setuptools release (58.0.2) has broken installation for ALL
> Airflow 2 released versions (Details of the issue here:
> https://github.com/apache/airflow/issues/18075 ). The actual problem is
> the deprecation of 2_to_3 configuration in FlaskOpenID. We try to
> communicate with FlaskOpenId maintainers, but it's likely they won't fix it
> (the last release was in 2016)
>
> This was rather unfortunate, because setuptools need to be installed
> before airflow and even with our constraint mechanisms, we cannot force
> setuptools to be downgraded.
>
> Solution (Workaround really):
>
> Luckily, (with the help of TP, our PIP expert :) ) we've figured out
> and implemented the workaround, so as long as you use the "official" way of
> installing airflow (with constraints), Airflow should continue to install
> even with the latest setuptools.
>
> Details of the workaround in
> https://github.com/apache/airflow/issues/18075#issuecomment-915145181
>
> We basically forked Flask-OpenID, applied removal of the deprecated option
> and updated the constraints to include our patched version.
>
> I applied the fix to constraints for all released Airflow 2 versions:
>
> * 2.1.3:
> https://github.com/apache/airflow/commit/64508c7c974f56ac05a919deca0b69ad1b9df19e
> * 2.1.2:
> https://github.com/apache/airflow/commit/641d6cad2b7221e58fdc117a0faa3d12652a10b2
> * 2.1.1:
> https://github.com/apache/airflow/commit/cf0834b1da66ad8424ecb6344c799a1e9a6abe2e
> * 2.1.0:
> https://github.com/apache/airflow/commit/1e52c69c608c417c9466dd512e4cd46a65dbad68
> * 2.0.2:
> https://github.com/apache/airflow/commit/82cc57b5f96f08f6d5976b3e6a3ea33b3b5459d0
> * 2.0.1:
> https://github.com/apache/airflow/commit/2984d415c1fd9cf535cacf26b57666acadf24706
> * 2.0.0:
> https://github.com/apache/airflow/commit/dd3e88fd6414ca9fda7108088a959e709b45037f
>
>
> J.
>
>