You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@airflow.apache.org by Jarek Potiuk <ja...@potiuk.com> on 2021/12/21 13:41:53 UTC
[PROPOSAL] Removal of Python2 "virtualenv" support in Airlfow 2.3 ?
Following the discussion in
https://github.com/apache/airflow/pull/20238#issuecomment-998780796
Should we completely remove tests and python2 binary with Python 2
from the reference image?
We still keep Python 2 installed in our Reference Images and we have
tests in our CI that test that Python 2 works for PythonVirtualenv
https://github.com/apache/airflow/blob/ee87953652e1bea103aa1b338293306eb7ae041d/tests/operators/test_python.py#L820
We also advise our users
https://airflow.apache.org/docs/apache-airflow/stable/upgrading-from-1-10/index.html#step-1-switch-to-python-3
to use PythonVirtualenvOperator if they still want to run Python 2.
It's 2 years after the end of life for Python 2 in January.
My proposal is to finally put Python 2 to rest for our images and
tests. This does not mean that people won't be able to use it if they
REALLY want to. They will be able to - it's just our advise will be
that they move to Python 3 (or use custom Kubernetes POD Operator with
Python2 installed) and we remove it from both - test and reference
image so there is no guarantee it's going to work with Python
Virtualenv.
Is there anyone who thinks it's a bad idea ? If I do not see any
strong opposition, I will ask for a lazy consensus in a few days.
"Speak now or forever hold your peace" ;)
J.
[LAZY CONSENSUS] Removal of Python2 "virtualenv" support in Airlfow 2.3
Posted by Jarek Potiuk <ja...@potiuk.com>.
Right. I see there is a "strong" sentiment for the removal :).
So calling for the lazy consensus on:
"Let's Remove Python 2 from images, tests, documentation advising that
PythonVirtualenvOperator could be used for Python 2."
Details here: https://lists.apache.org/thread/rjyqw3cwsh4vgg6jycsbr1jr0slnych3
If I won't hear a strong -1 on that one - we will remove it after Xmas.
BTW. Merry Xmas everyone!
J.
Re: [PROPOSAL] Removal of Python2 "virtualenv" support in Airlfow 2.3 ?
Posted by Jarek Potiuk <ja...@potiuk.com>.
PR here: https://github.com/apache/airflow/pull/20680
On Wed, Dec 22, 2021 at 9:43 PM Kaxil Naik <ka...@gmail.com> wrote:
>
> +1
>
> On Wed, Dec 22, 2021 at 1:07 AM Leah Cole <co...@google.com.invalid> wrote:
>>
>> +1 for removal!
>>
>> On Tue, Dec 21, 2021 at 7:22 AM Tomasz Urbaszek <tu...@apache.org> wrote:
>>>
>>> +1
>>>
>>> On Tue, 21 Dec 2021 at 16:19, Elad Kalif <el...@apache.org> wrote:
>>>>
>>>> +1
>>>>
>>>> On Tue, Dec 21, 2021 at 4:02 PM Ash Berlin-Taylor <as...@apache.org> wrote:
>>>>>
>>>>> +1
>>>>>
>>>>> On 21 December 2021 13:41:53 GMT, Jarek Potiuk <ja...@potiuk.com> wrote:
>>>>>>
>>>>>> Following the discussion in
>>>>>> https://github.com/apache/airflow/pull/20238#issuecomment-998780796
>>>>>>
>>>>>> Should we completely remove tests and python2 binary with Python 2
>>>>>> from the reference image?
>>>>>>
>>>>>> We still keep Python 2 installed in our Reference Images and we have
>>>>>> tests in our CI that test that Python 2 works for PythonVirtualenv
>>>>>> https://github.com/apache/airflow/blob/ee87953652e1bea103aa1b338293306eb7ae041d/tests/operators/test_python.py#L820
>>>>>>
>>>>>> We also advise our users
>>>>>> https://airflow.apache.org/docs/apache-airflow/stable/upgrading-from-1-10/index.html#step-1-switch-to-python-3
>>>>>> to use PythonVirtualenvOperator if they still want to run Python 2.
>>>>>>
>>>>>> It's 2 years after the end of life for Python 2 in January.
>>>>>>
>>>>>> My proposal is to finally put Python 2 to rest for our images and
>>>>>> tests. This does not mean that people won't be able to use it if they
>>>>>> REALLY want to. They will be able to - it's just our advise will be
>>>>>> that they move to Python 3 (or use custom Kubernetes POD Operator with
>>>>>> Python2 installed) and we remove it from both - test and reference
>>>>>> image so there is no guarantee it's going to work with Python
>>>>>> Virtualenv.
>>>>>>
>>>>>> Is there anyone who thinks it's a bad idea ? If I do not see any
>>>>>> strong opposition, I will ask for a lazy consensus in a few days.
>>>>>>
>>>>>> "Speak now or forever hold your peace" ;)
>>>>>>
>>>>>> J.
>>
>>
>>
>> --
>>
>> Leah Cole (she/her) | Developer Programs Engineer, Data Analytics | coleleah@google.com | +1 (925) 257-2112
>> My working hours may not be your working hours. Please ping me anytime if you'd like a status update on anything we are working on together - my goal is to never be a blocker for you.
>>
Re: [PROPOSAL] Removal of Python2 "virtualenv" support in Airlfow 2.3 ?
Posted by Kaxil Naik <ka...@gmail.com>.
+1
On Wed, Dec 22, 2021 at 1:07 AM Leah Cole <co...@google.com.invalid>
wrote:
> +1 for removal!
>
> On Tue, Dec 21, 2021 at 7:22 AM Tomasz Urbaszek <tu...@apache.org>
> wrote:
>
>> +1
>>
>> On Tue, 21 Dec 2021 at 16:19, Elad Kalif <el...@apache.org> wrote:
>>
>>> +1
>>>
>>> On Tue, Dec 21, 2021 at 4:02 PM Ash Berlin-Taylor <as...@apache.org>
>>> wrote:
>>>
>>>> +1
>>>>
>>>> On 21 December 2021 13:41:53 GMT, Jarek Potiuk <ja...@potiuk.com>
>>>> wrote:
>>>>>
>>>>> Following the discussion in
>>>>> https://github.com/apache/airflow/pull/20238#issuecomment-998780796
>>>>>
>>>>> Should we completely remove tests and python2 binary with Python 2
>>>>> from the reference image?
>>>>>
>>>>> We still keep Python 2 installed in our Reference Images and we have
>>>>> tests in our CI that test that Python 2 works for PythonVirtualenv
>>>>> https://github.com/apache/airflow/blob/ee87953652e1bea103aa1b338293306eb7ae041d/tests/operators/test_python.py#L820
>>>>>
>>>>> We also advise our users
>>>>> https://airflow.apache.org/docs/apache-airflow/stable/upgrading-from-1-10/index.html#step-1-switch-to-python-3
>>>>> to use PythonVirtualenvOperator if they still want to run Python 2.
>>>>>
>>>>> It's 2 years after the end of life for Python 2 in January.
>>>>>
>>>>> My proposal is to finally put Python 2 to rest for our images and
>>>>> tests. This does not mean that people won't be able to use it if they
>>>>> REALLY want to. They will be able to - it's just our advise will be
>>>>> that they move to Python 3 (or use custom Kubernetes POD Operator with
>>>>> Python2 installed) and we remove it from both - test and reference
>>>>> image so there is no guarantee it's going to work with Python
>>>>> Virtualenv.
>>>>>
>>>>> Is there anyone who thinks it's a bad idea ? If I do not see any
>>>>> strong opposition, I will ask for a lazy consensus in a few days.
>>>>>
>>>>> "Speak now or forever hold your peace" ;)
>>>>>
>>>>> J.
>>>>>
>>>>>
>
> --
>
> Leah Cole (she/her) | Developer Programs Engineer, Data Analytics |
> coleleah@google.com | +1 (925) 257-2112
> *My working hours may not be your working hours. Please ping me anytime if
> you'd like a status update on anything we are working on together - my goal
> is to never be a blocker for you. *
>
>
Re: [PROPOSAL] Removal of Python2 "virtualenv" support in Airlfow 2.3 ?
Posted by Leah Cole <co...@google.com.INVALID>.
+1 for removal!
On Tue, Dec 21, 2021 at 7:22 AM Tomasz Urbaszek <tu...@apache.org>
wrote:
> +1
>
> On Tue, 21 Dec 2021 at 16:19, Elad Kalif <el...@apache.org> wrote:
>
>> +1
>>
>> On Tue, Dec 21, 2021 at 4:02 PM Ash Berlin-Taylor <as...@apache.org> wrote:
>>
>>> +1
>>>
>>> On 21 December 2021 13:41:53 GMT, Jarek Potiuk <ja...@potiuk.com> wrote:
>>>>
>>>> Following the discussion in
>>>> https://github.com/apache/airflow/pull/20238#issuecomment-998780796
>>>>
>>>> Should we completely remove tests and python2 binary with Python 2
>>>> from the reference image?
>>>>
>>>> We still keep Python 2 installed in our Reference Images and we have
>>>> tests in our CI that test that Python 2 works for PythonVirtualenv
>>>> https://github.com/apache/airflow/blob/ee87953652e1bea103aa1b338293306eb7ae041d/tests/operators/test_python.py#L820
>>>>
>>>> We also advise our users
>>>> https://airflow.apache.org/docs/apache-airflow/stable/upgrading-from-1-10/index.html#step-1-switch-to-python-3
>>>> to use PythonVirtualenvOperator if they still want to run Python 2.
>>>>
>>>> It's 2 years after the end of life for Python 2 in January.
>>>>
>>>> My proposal is to finally put Python 2 to rest for our images and
>>>> tests. This does not mean that people won't be able to use it if they
>>>> REALLY want to. They will be able to - it's just our advise will be
>>>> that they move to Python 3 (or use custom Kubernetes POD Operator with
>>>> Python2 installed) and we remove it from both - test and reference
>>>> image so there is no guarantee it's going to work with Python
>>>> Virtualenv.
>>>>
>>>> Is there anyone who thinks it's a bad idea ? If I do not see any
>>>> strong opposition, I will ask for a lazy consensus in a few days.
>>>>
>>>> "Speak now or forever hold your peace" ;)
>>>>
>>>> J.
>>>>
>>>>
--
Leah Cole (she/her) | Developer Programs Engineer, Data Analytics |
coleleah@google.com | +1 (925) 257-2112
*My working hours may not be your working hours. Please ping me anytime if
you'd like a status update on anything we are working on together - my goal
is to never be a blocker for you. *
Re: [PROPOSAL] Removal of Python2 "virtualenv" support in Airlfow 2.3 ?
Posted by Tomasz Urbaszek <tu...@apache.org>.
+1
On Tue, 21 Dec 2021 at 16:19, Elad Kalif <el...@apache.org> wrote:
> +1
>
> On Tue, Dec 21, 2021 at 4:02 PM Ash Berlin-Taylor <as...@apache.org> wrote:
>
>> +1
>>
>> On 21 December 2021 13:41:53 GMT, Jarek Potiuk <ja...@potiuk.com> wrote:
>>>
>>> Following the discussion in
>>> https://github.com/apache/airflow/pull/20238#issuecomment-998780796
>>>
>>> Should we completely remove tests and python2 binary with Python 2
>>> from the reference image?
>>>
>>> We still keep Python 2 installed in our Reference Images and we have
>>> tests in our CI that test that Python 2 works for PythonVirtualenv
>>> https://github.com/apache/airflow/blob/ee87953652e1bea103aa1b338293306eb7ae041d/tests/operators/test_python.py#L820
>>>
>>> We also advise our users
>>> https://airflow.apache.org/docs/apache-airflow/stable/upgrading-from-1-10/index.html#step-1-switch-to-python-3
>>> to use PythonVirtualenvOperator if they still want to run Python 2.
>>>
>>> It's 2 years after the end of life for Python 2 in January.
>>>
>>> My proposal is to finally put Python 2 to rest for our images and
>>> tests. This does not mean that people won't be able to use it if they
>>> REALLY want to. They will be able to - it's just our advise will be
>>> that they move to Python 3 (or use custom Kubernetes POD Operator with
>>> Python2 installed) and we remove it from both - test and reference
>>> image so there is no guarantee it's going to work with Python
>>> Virtualenv.
>>>
>>> Is there anyone who thinks it's a bad idea ? If I do not see any
>>> strong opposition, I will ask for a lazy consensus in a few days.
>>>
>>> "Speak now or forever hold your peace" ;)
>>>
>>> J.
>>>
>>>
Re: [PROPOSAL] Removal of Python2 "virtualenv" support in Airlfow 2.3 ?
Posted by Elad Kalif <el...@apache.org>.
+1
On Tue, Dec 21, 2021 at 4:02 PM Ash Berlin-Taylor <as...@apache.org> wrote:
> +1
>
> On 21 December 2021 13:41:53 GMT, Jarek Potiuk <ja...@potiuk.com> wrote:
>>
>> Following the discussion in
>> https://github.com/apache/airflow/pull/20238#issuecomment-998780796
>>
>> Should we completely remove tests and python2 binary with Python 2
>> from the reference image?
>>
>> We still keep Python 2 installed in our Reference Images and we have
>> tests in our CI that test that Python 2 works for PythonVirtualenv
>> https://github.com/apache/airflow/blob/ee87953652e1bea103aa1b338293306eb7ae041d/tests/operators/test_python.py#L820
>>
>> We also advise our users
>> https://airflow.apache.org/docs/apache-airflow/stable/upgrading-from-1-10/index.html#step-1-switch-to-python-3
>> to use PythonVirtualenvOperator if they still want to run Python 2.
>>
>> It's 2 years after the end of life for Python 2 in January.
>>
>> My proposal is to finally put Python 2 to rest for our images and
>> tests. This does not mean that people won't be able to use it if they
>> REALLY want to. They will be able to - it's just our advise will be
>> that they move to Python 3 (or use custom Kubernetes POD Operator with
>> Python2 installed) and we remove it from both - test and reference
>> image so there is no guarantee it's going to work with Python
>> Virtualenv.
>>
>> Is there anyone who thinks it's a bad idea ? If I do not see any
>> strong opposition, I will ask for a lazy consensus in a few days.
>>
>> "Speak now or forever hold your peace" ;)
>>
>> J.
>>
>>
Re: [PROPOSAL] Removal of Python2 "virtualenv" support in Airlfow 2.3 ?
Posted by Elad Kalif <el...@apache.org>.
+1
On Tue, Dec 21, 2021 at 4:02 PM Ash Berlin-Taylor <as...@apache.org> wrote:
> +1
>
> On 21 December 2021 13:41:53 GMT, Jarek Potiuk <ja...@potiuk.com> wrote:
>>
>> Following the discussion in
>> https://github.com/apache/airflow/pull/20238#issuecomment-998780796
>>
>> Should we completely remove tests and python2 binary with Python 2
>> from the reference image?
>>
>> We still keep Python 2 installed in our Reference Images and we have
>> tests in our CI that test that Python 2 works for PythonVirtualenv
>> https://github.com/apache/airflow/blob/ee87953652e1bea103aa1b338293306eb7ae041d/tests/operators/test_python.py#L820
>>
>> We also advise our users
>> https://airflow.apache.org/docs/apache-airflow/stable/upgrading-from-1-10/index.html#step-1-switch-to-python-3
>> to use PythonVirtualenvOperator if they still want to run Python 2.
>>
>> It's 2 years after the end of life for Python 2 in January.
>>
>> My proposal is to finally put Python 2 to rest for our images and
>> tests. This does not mean that people won't be able to use it if they
>> REALLY want to. They will be able to - it's just our advise will be
>> that they move to Python 3 (or use custom Kubernetes POD Operator with
>> Python2 installed) and we remove it from both - test and reference
>> image so there is no guarantee it's going to work with Python
>> Virtualenv.
>>
>> Is there anyone who thinks it's a bad idea ? If I do not see any
>> strong opposition, I will ask for a lazy consensus in a few days.
>>
>> "Speak now or forever hold your peace" ;)
>>
>> J.
>>
>>
Re: [PROPOSAL] Removal of Python2 "virtualenv" support in Airlfow 2.3 ?
Posted by Ash Berlin-Taylor <as...@apache.org>.
+1
On 21 December 2021 13:41:53 GMT, Jarek Potiuk <ja...@potiuk.com> wrote:
>Following the discussion in
>https://github.com/apache/airflow/pull/20238#issuecomment-998780796
>
>Should we completely remove tests and python2 binary with Python 2
>from the reference image?
>
>We still keep Python 2 installed in our Reference Images and we have
>tests in our CI that test that Python 2 works for PythonVirtualenv
>https://github.com/apache/airflow/blob/ee87953652e1bea103aa1b338293306eb7ae041d/tests/operators/test_python.py#L820
>
>We also advise our users
>https://airflow.apache.org/docs/apache-airflow/stable/upgrading-from-1-10/index.html#step-1-switch-to-python-3
>to use PythonVirtualenvOperator if they still want to run Python 2.
>
>It's 2 years after the end of life for Python 2 in January.
>
>My proposal is to finally put Python 2 to rest for our images and
>tests. This does not mean that people won't be able to use it if they
>REALLY want to. They will be able to - it's just our advise will be
>that they move to Python 3 (or use custom Kubernetes POD Operator with
>Python2 installed) and we remove it from both - test and reference
>image so there is no guarantee it's going to work with Python
>Virtualenv.
>
>Is there anyone who thinks it's a bad idea ? If I do not see any
>strong opposition, I will ask for a lazy consensus in a few days.
>
>"Speak now or forever hold your peace" ;)
>
>J.
Re: [PROPOSAL] Removal of Python2 "virtualenv" support in Airlfow 2.3 ?
Posted by Ash Berlin-Taylor <as...@apache.org>.
+1
On 21 December 2021 13:41:53 GMT, Jarek Potiuk <ja...@potiuk.com> wrote:
>Following the discussion in
>https://github.com/apache/airflow/pull/20238#issuecomment-998780796
>
>Should we completely remove tests and python2 binary with Python 2
>from the reference image?
>
>We still keep Python 2 installed in our Reference Images and we have
>tests in our CI that test that Python 2 works for PythonVirtualenv
>https://github.com/apache/airflow/blob/ee87953652e1bea103aa1b338293306eb7ae041d/tests/operators/test_python.py#L820
>
>We also advise our users
>https://airflow.apache.org/docs/apache-airflow/stable/upgrading-from-1-10/index.html#step-1-switch-to-python-3
>to use PythonVirtualenvOperator if they still want to run Python 2.
>
>It's 2 years after the end of life for Python 2 in January.
>
>My proposal is to finally put Python 2 to rest for our images and
>tests. This does not mean that people won't be able to use it if they
>REALLY want to. They will be able to - it's just our advise will be
>that they move to Python 3 (or use custom Kubernetes POD Operator with
>Python2 installed) and we remove it from both - test and reference
>image so there is no guarantee it's going to work with Python
>Virtualenv.
>
>Is there anyone who thinks it's a bad idea ? If I do not see any
>strong opposition, I will ask for a lazy consensus in a few days.
>
>"Speak now or forever hold your peace" ;)
>
>J.
Re: [PROPOSAL] Removal of Python2 "virtualenv" support in Airlfow 2.3 ?
Posted by Bas Harenslak <ba...@astronomer.io.INVALID>.
+1 for removing it.
Had a think whether removing the Python 2 binaries in Airflow 2.3 is right, or if we should first explicitly state the future removal. Since Python 2 has been EOL for a long time + we've advised Python 3 in the docs for a long time + users can still use Python 2 by installing it, I don’t see a problem removing it in Airflow 2.3.
Bas
> On 21 Dec 2021, at 14:41, Jarek Potiuk <ja...@potiuk.com> wrote:
>
> Following the discussion in
> https://github.com/apache/airflow/pull/20238#issuecomment-998780796
>
> Should we completely remove tests and python2 binary with Python 2
> from the reference image?
>
> We still keep Python 2 installed in our Reference Images and we have
> tests in our CI that test that Python 2 works for PythonVirtualenv
> https://github.com/apache/airflow/blob/ee87953652e1bea103aa1b338293306eb7ae041d/tests/operators/test_python.py#L820
>
> We also advise our users
> https://airflow.apache.org/docs/apache-airflow/stable/upgrading-from-1-10/index.html#step-1-switch-to-python-3
> to use PythonVirtualenvOperator if they still want to run Python 2.
>
> It's 2 years after the end of life for Python 2 in January.
>
> My proposal is to finally put Python 2 to rest for our images and
> tests. This does not mean that people won't be able to use it if they
> REALLY want to. They will be able to - it's just our advise will be
> that they move to Python 3 (or use custom Kubernetes POD Operator with
> Python2 installed) and we remove it from both - test and reference
> image so there is no guarantee it's going to work with Python
> Virtualenv.
>
> Is there anyone who thinks it's a bad idea ? If I do not see any
> strong opposition, I will ask for a lazy consensus in a few days.
>
> "Speak now or forever hold your peace" ;)
>
> J.