You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Dian Fu <di...@gmail.com> on 2019/10/08 14:34:26 UTC
[DISCUSS] Drop Python 2 support for 1.10
Hi everyone,
I would like to propose to drop Python 2 support(Currently Python 2.7, 3.5, 3.6, 3.7 are all supported in Flink) as it's coming to an end at Jan 1, 2020 [1]. A lot of projects [2][3][4] has already stated or are planning to drop Python 2 support.
The benefits of dropping Python 2 support are:
1. Maintaining Python 2/3 compatibility is a burden and it makes the code complicate as Python 2 and Python 3 is not compatible.
2. There are many features which are only available in Python 3.x such as Type Hints[5]. We can only make use of this kind of features after dropping the Python 2 support.
3. Flink-python depends on third-part projects, such as Apache Beam (may add more dependencies such as pandas, etc in the near future), it's not possible to upgrade them to the latest version once they drop the Python 2 support.
Here are the options we have:
1. Drop Python 2 support in 1.10:
As flink-python module is a new module added since 1.9.0 and so dropping Python 2 support at the early stage seems a good choice for us.
2. Deprecate Python 2 in 1.10 and drop its support in 1.11:
As 1.10 is planned to be released around the beginning of 2020. This is also aligned with the official Python 2 support.
Personally I prefer option 1 as flink-python is new module and there is no much history reasons to consider.
Looking forward to your feedback!
Regards,
Dian
[1] https://pythonclock.org/ <https://pythonclock.org/>
[2] https://python3statement.org/ <https://python3statement.org/>
[3] https://spark.apache.org/news/plan-for-dropping-python-2-support.html <https://spark.apache.org/news/plan-for-dropping-python-2-support.html>
[4] https://lists.apache.org/thread.html/eba6caa58ea79a7ecbc8560d1c680a366b44c531d96ce5c699d41535@%3Cdev.beam.apache.org%3E <https://lists.apache.org/thread.html/eba6caa58ea79a7ecbc8560d1c680a366b44c531d96ce5c699d41535@%3Cdev.beam.apache.org%3E>
[5] https://stackoverflow.com/questions/32557920/what-are-type-hints-in-python-3-5 <https://stackoverflow.com/questions/32557920/what-are-type-hints-in-python-3-5>
Re: [DISCUSS] Drop Python 2 support for 1.10
Posted by Dian Fu <di...@gmail.com>.
Thanks Jincheng for the remind and everyone for joining the discussion. I have started the vote thread in [1].
Thanks,
Dian
[1] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-Drop-Python-2-support-for-1-10-tt33962.html <http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-Drop-Python-2-support-for-1-10-tt33962.html>
> 在 2019年10月12日，下午5:44，jincheng sun <su...@gmail.com> 写道：
>
> Hi Dian,
>
> I think it's better to bring up the VOTE for this proposal. Then push this
> forward.:)
>
> Thanks,
> Jincheng
>
> Timo Walther <tw...@apache.org> 于2019年10月10日周四 下午8:07写道：
>
>> I also heard from other companies that upgrading to Python 3 is in
>> progress for data teams.
>>
>> +1 for simplifying the code base with option 1).
>>
>> Thanks,
>> Timo
>>
>> On 08.10.19 16:34, Dian Fu wrote:
>>> Hi everyone,
>>>
>>> I would like to propose to drop Python 2 support(Currently Python 2.7,
>> 3.5, 3.6, 3.7 are all supported in Flink) as it's coming to an end at Jan
>> 1, 2020 [1]. A lot of projects [2][3][4] has already stated or are planning
>> to drop Python 2 support.
>>>
>>> The benefits of dropping Python 2 support are:
>>> 1. Maintaining Python 2/3 compatibility is a burden and it makes the
>> code complicate as Python 2 and Python 3 is not compatible.
>>> 2. There are many features which are only available in Python 3.x such
>> as Type Hints[5]. We can only make use of this kind of features after
>> dropping the Python 2 support.
>>> 3. Flink-python depends on third-part projects, such as Apache Beam (may
>> add more dependencies such as pandas, etc in the near future), it's not
>> possible to upgrade them to the latest version once they drop the Python 2
>> support.
>>>
>>> Here are the options we have:
>>> 1. Drop Python 2 support in 1.10:
>>> As flink-python module is a new module added since 1.9.0 and so dropping
>> Python 2 support at the early stage seems a good choice for us.
>>> 2. Deprecate Python 2 in 1.10 and drop its support in 1.11:
>>> As 1.10 is planned to be released around the beginning of 2020. This is
>> also aligned with the official Python 2 support.
>>>
>>> Personally I prefer option 1 as flink-python is new module and there is
>> no much history reasons to consider.
>>>
>>> Looking forward to your feedback!
>>>
>>> Regards,
>>> Dian
>>>
>>> [1] https://pythonclock.org/ <https://pythonclock.org/>
>>> [2] https://python3statement.org/ <https://python3statement.org/>
>>> [3]
>> https://spark.apache.org/news/plan-for-dropping-python-2-support.html <
>> https://spark.apache.org/news/plan-for-dropping-python-2-support.html>
>>> [4]
>> https://lists.apache.org/thread.html/eba6caa58ea79a7ecbc8560d1c680a366b44c531d96ce5c699d41535@%3Cdev.beam.apache.org%3E
>> <
>> https://lists.apache.org/thread.html/eba6caa58ea79a7ecbc8560d1c680a366b44c531d96ce5c699d41535@%3Cdev.beam.apache.org%3E
>>>
>>> [5]
>> https://stackoverflow.com/questions/32557920/what-are-type-hints-in-python-3-5
>> <
>> https://stackoverflow.com/questions/32557920/what-are-type-hints-in-python-3-5
>>>
>>
>>
>>
Re: [DISCUSS] Drop Python 2 support for 1.10
Posted by jincheng sun <su...@gmail.com>.
Hi Dian,
I think it's better to bring up the VOTE for this proposal. Then push this
forward.:)
Thanks,
Jincheng
Timo Walther <tw...@apache.org> 于2019年10月10日周四 下午8:07写道：
> I also heard from other companies that upgrading to Python 3 is in
> progress for data teams.
>
> +1 for simplifying the code base with option 1).
>
> Thanks,
> Timo
>
> On 08.10.19 16:34, Dian Fu wrote:
> > Hi everyone,
> >
> > I would like to propose to drop Python 2 support(Currently Python 2.7,
> 3.5, 3.6, 3.7 are all supported in Flink) as it's coming to an end at Jan
> 1, 2020 [1]. A lot of projects [2][3][4] has already stated or are planning
> to drop Python 2 support.
> >
> > The benefits of dropping Python 2 support are:
> > 1. Maintaining Python 2/3 compatibility is a burden and it makes the
> code complicate as Python 2 and Python 3 is not compatible.
> > 2. There are many features which are only available in Python 3.x such
> as Type Hints[5]. We can only make use of this kind of features after
> dropping the Python 2 support.
> > 3. Flink-python depends on third-part projects, such as Apache Beam (may
> add more dependencies such as pandas, etc in the near future), it's not
> possible to upgrade them to the latest version once they drop the Python 2
> support.
> >
> > Here are the options we have:
> > 1. Drop Python 2 support in 1.10:
> > As flink-python module is a new module added since 1.9.0 and so dropping
> Python 2 support at the early stage seems a good choice for us.
> > 2. Deprecate Python 2 in 1.10 and drop its support in 1.11:
> > As 1.10 is planned to be released around the beginning of 2020. This is
> also aligned with the official Python 2 support.
> >
> > Personally I prefer option 1 as flink-python is new module and there is
> no much history reasons to consider.
> >
> > Looking forward to your feedback!
> >
> > Regards,
> > Dian
> >
> > [1] https://pythonclock.org/ <https://pythonclock.org/>
> > [2] https://python3statement.org/ <https://python3statement.org/>
> > [3]
> https://spark.apache.org/news/plan-for-dropping-python-2-support.html <
> https://spark.apache.org/news/plan-for-dropping-python-2-support.html>
> > [4]
> https://lists.apache.org/thread.html/eba6caa58ea79a7ecbc8560d1c680a366b44c531d96ce5c699d41535@%3Cdev.beam.apache.org%3E
> <
> https://lists.apache.org/thread.html/eba6caa58ea79a7ecbc8560d1c680a366b44c531d96ce5c699d41535@%3Cdev.beam.apache.org%3E
> >
> > [5]
> https://stackoverflow.com/questions/32557920/what-are-type-hints-in-python-3-5
> <
> https://stackoverflow.com/questions/32557920/what-are-type-hints-in-python-3-5
> >
>
>
>
Re: [DISCUSS] Drop Python 2 support for 1.10
Posted by Timo Walther <tw...@apache.org>.
I also heard from other companies that upgrading to Python 3 is in
progress for data teams.
+1 for simplifying the code base with option 1).
Thanks,
Timo
On 08.10.19 16:34, Dian Fu wrote:
> Hi everyone,
>
> I would like to propose to drop Python 2 support(Currently Python 2.7, 3.5, 3.6, 3.7 are all supported in Flink) as it's coming to an end at Jan 1, 2020 [1]. A lot of projects [2][3][4] has already stated or are planning to drop Python 2 support.
>
> The benefits of dropping Python 2 support are:
> 1. Maintaining Python 2/3 compatibility is a burden and it makes the code complicate as Python 2 and Python 3 is not compatible.
> 2. There are many features which are only available in Python 3.x such as Type Hints[5]. We can only make use of this kind of features after dropping the Python 2 support.
> 3. Flink-python depends on third-part projects, such as Apache Beam (may add more dependencies such as pandas, etc in the near future), it's not possible to upgrade them to the latest version once they drop the Python 2 support.
>
> Here are the options we have:
> 1. Drop Python 2 support in 1.10:
> As flink-python module is a new module added since 1.9.0 and so dropping Python 2 support at the early stage seems a good choice for us.
> 2. Deprecate Python 2 in 1.10 and drop its support in 1.11:
> As 1.10 is planned to be released around the beginning of 2020. This is also aligned with the official Python 2 support.
>
> Personally I prefer option 1 as flink-python is new module and there is no much history reasons to consider.
>
> Looking forward to your feedback!
>
> Regards,
> Dian
>
> [1] https://pythonclock.org/ <https://pythonclock.org/>
> [2] https://python3statement.org/ <https://python3statement.org/>
> [3] https://spark.apache.org/news/plan-for-dropping-python-2-support.html <https://spark.apache.org/news/plan-for-dropping-python-2-support.html>
> [4] https://lists.apache.org/thread.html/eba6caa58ea79a7ecbc8560d1c680a366b44c531d96ce5c699d41535@%3Cdev.beam.apache.org%3E <https://lists.apache.org/thread.html/eba6caa58ea79a7ecbc8560d1c680a366b44c531d96ce5c699d41535@%3Cdev.beam.apache.org%3E>
> [5] https://stackoverflow.com/questions/32557920/what-are-type-hints-in-python-3-5 <https://stackoverflow.com/questions/32557920/what-are-type-hints-in-python-3-5>
Re: [DISCUSS] Drop Python 2 support for 1.10
Posted by Dian Fu <di...@gmail.com>.
Thanks everyone for your reply.
So far all the reply tend to option 1 (dropping Python 2 support in 1.10) and will continue to hear if there are any other opinions.
@Jincheng @Hequn, you are right, things become more complicate if dropping Python 2 support is performed after Python UDF has been supported. Users will have to migrate their Python UDFs if they have used features which only are supported in Python 2.
Thanks @Yu for your suggestion. It makes much sense to me and will do that. Also CC @user and @user-zh ML in case any users are concerned about this.
Thanks,
Dian
> 在 2019年10月9日，下午1:14，Yu Li <ca...@gmail.com> 写道：
>
> Thanks for bringing this up Dian.
>
> Since python 2.7 support was added in 1.9.0 and would be EOL near the
> planned release time for 1.10, I could see a good reason to take option 1.
>
> Please remember to add an explicit release note and would be better to send
> a notification in user ML about the plan to drop it, just in case some
> 1.9.0 users are already using python 2.7 in their product env.
>
> Best Regards,
> Yu
>
>
> On Wed, 9 Oct 2019 at 11:13, Jeff Zhang <zj...@gmail.com> wrote:
>
>> +1
>>
>> Hequn Cheng <ch...@gmail.com> 于2019年10月9日周三 上午11:07写道：
>>
>>> Hi Dian,
>>>
>>> +1 to drop Python 2 directly.
>>>
>>> Just as @jincheng said, things would be more complicated if we are going
>> to
>>> support python UDFs.
>>> The python UDFs will introduce a lot of python dependencies which will
>> also
>>> drop the support of Python 2, such as beam, pandas, pyarrow, etc.
>>> Given this and Python 2 will reach EOL on Jan 1 2020. I think we can drop
>>> Python 2 in Flink as well.
>>>
>>> As for the two options, I think we can drop it directly in 1.10. The
>>> flink-python is introduced just from 1.9, I think it's safe to drop it
>> for
>>> now.
>>> And we can also benefit from it when we add support for python UDFs.
>>>
>>> Best, Hequn
>>>
>>>
>>> On Wed, Oct 9, 2019 at 8:40 AM jincheng sun <su...@gmail.com>
>>> wrote:
>>>
>>>> Hi Dian，
>>>>
>>>> Thanks for bringing this discussion!
>>>>
>>>> In Flink 1.9 we only add Python Table API mapping to Java Table
>>> API(without
>>>> Python UDFs), there no special requirements for Python version, so we
>> add
>>>> python 2,7 support. But for Flink 1.10, we add the Python UDFs support,
>>>> i.e., user will add more python code in Flink job and more requirements
>>> for
>>>> the features of the Python language.So I think It's better to follow
>> the
>>>> rhythm of Python official.
>>>>
>>>> Option 2 is the most conservative and correct approach, but for the
>>> current
>>>> situation, we cooperate with the Beam community and use Beam's
>>> portability
>>>> framework for UDFs support, so we prefer to adopt the Option 1.
>>>>
>>>> Best,
>>>> Jincheng
>>>>
>>>>
>>>>
>>>> Dian Fu <di...@gmail.com> 于2019年10月8日周二 下午10:34写道：
>>>>
>>>>> Hi everyone,
>>>>>
>>>>> I would like to propose to drop Python 2 support(Currently Python
>> 2.7,
>>>>> 3.5, 3.6, 3.7 are all supported in Flink) as it's coming to an end at
>>> Jan
>>>>> 1, 2020 [1]. A lot of projects [2][3][4] has already stated or are
>>>> planning
>>>>> to drop Python 2 support.
>>>>>
>>>>> The benefits of dropping Python 2 support are:
>>>>> 1. Maintaining Python 2/3 compatibility is a burden and it makes the
>>> code
>>>>> complicate as Python 2 and Python 3 is not compatible.
>>>>> 2. There are many features which are only available in Python 3.x
>> such
>>> as
>>>>> Type Hints[5]. We can only make use of this kind of features after
>>>> dropping
>>>>> the Python 2 support.
>>>>> 3. Flink-python depends on third-part projects, such as Apache Beam
>>> (may
>>>>> add more dependencies such as pandas, etc in the near future), it's
>> not
>>>>> possible to upgrade them to the latest version once they drop the
>>> Python
>>>> 2
>>>>> support.
>>>>>
>>>>> Here are the options we have:
>>>>> 1. Drop Python 2 support in 1.10:
>>>>> As flink-python module is a new module added since 1.9.0 and so
>>> dropping
>>>>> Python 2 support at the early stage seems a good choice for us.
>>>>> 2. Deprecate Python 2 in 1.10 and drop its support in 1.11:
>>>>> As 1.10 is planned to be released around the beginning of 2020. This
>> is
>>>>> also aligned with the official Python 2 support.
>>>>>
>>>>> Personally I prefer option 1 as flink-python is new module and there
>> is
>>>> no
>>>>> much history reasons to consider.
>>>>>
>>>>> Looking forward to your feedback!
>>>>>
>>>>> Regards,
>>>>> Dian
>>>>>
>>>>> [1] https://pythonclock.org/ <https://pythonclock.org/>
>>>>> [2] https://python3statement.org/ <https://python3statement.org/>
>>>>> [3]
>>>> https://spark.apache.org/news/plan-for-dropping-python-2-support.html
>>>>> <
>> https://spark.apache.org/news/plan-for-dropping-python-2-support.html
>>>>
>>>>> [4]
>>>>>
>>>>
>>>
>> https://lists.apache.org/thread.html/eba6caa58ea79a7ecbc8560d1c680a366b44c531d96ce5c699d41535@%3Cdev.beam.apache.org%3E
>>>>> <
>>>>>
>>>>
>>>
>> https://lists.apache.org/thread.html/eba6caa58ea79a7ecbc8560d1c680a366b44c531d96ce5c699d41535@%3Cdev.beam.apache.org%3E
>>>>>>
>>>>> [5]
>>>>>
>>>>
>>>
>> https://stackoverflow.com/questions/32557920/what-are-type-hints-in-python-3-5
>>>>> <
>>>>>
>>>>
>>>
>> https://stackoverflow.com/questions/32557920/what-are-type-hints-in-python-3-5
>>>>>>
>>>>
>>>
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
Re: [DISCUSS] Drop Python 2 support for 1.10
Posted by Dian Fu <di...@gmail.com>.
Thanks everyone for your reply.
So far all the reply tend to option 1 (dropping Python 2 support in 1.10) and will continue to hear if there are any other opinions.
@Jincheng @Hequn, you are right, things become more complicate if dropping Python 2 support is performed after Python UDF has been supported. Users will have to migrate their Python UDFs if they have used features which only are supported in Python 2.
Thanks @Yu for your suggestion. It makes much sense to me and will do that. Also CC @user and @user-zh ML in case any users are concerned about this.
Thanks,
Dian
> 在 2019年10月9日，下午1:14，Yu Li <ca...@gmail.com> 写道：
>
> Thanks for bringing this up Dian.
>
> Since python 2.7 support was added in 1.9.0 and would be EOL near the
> planned release time for 1.10, I could see a good reason to take option 1.
>
> Please remember to add an explicit release note and would be better to send
> a notification in user ML about the plan to drop it, just in case some
> 1.9.0 users are already using python 2.7 in their product env.
>
> Best Regards,
> Yu
>
>
> On Wed, 9 Oct 2019 at 11:13, Jeff Zhang <zj...@gmail.com> wrote:
>
>> +1
>>
>> Hequn Cheng <ch...@gmail.com> 于2019年10月9日周三 上午11:07写道：
>>
>>> Hi Dian,
>>>
>>> +1 to drop Python 2 directly.
>>>
>>> Just as @jincheng said, things would be more complicated if we are going
>> to
>>> support python UDFs.
>>> The python UDFs will introduce a lot of python dependencies which will
>> also
>>> drop the support of Python 2, such as beam, pandas, pyarrow, etc.
>>> Given this and Python 2 will reach EOL on Jan 1 2020. I think we can drop
>>> Python 2 in Flink as well.
>>>
>>> As for the two options, I think we can drop it directly in 1.10. The
>>> flink-python is introduced just from 1.9, I think it's safe to drop it
>> for
>>> now.
>>> And we can also benefit from it when we add support for python UDFs.
>>>
>>> Best, Hequn
>>>
>>>
>>> On Wed, Oct 9, 2019 at 8:40 AM jincheng sun <su...@gmail.com>
>>> wrote:
>>>
>>>> Hi Dian，
>>>>
>>>> Thanks for bringing this discussion!
>>>>
>>>> In Flink 1.9 we only add Python Table API mapping to Java Table
>>> API(without
>>>> Python UDFs), there no special requirements for Python version, so we
>> add
>>>> python 2,7 support. But for Flink 1.10, we add the Python UDFs support,
>>>> i.e., user will add more python code in Flink job and more requirements
>>> for
>>>> the features of the Python language.So I think It's better to follow
>> the
>>>> rhythm of Python official.
>>>>
>>>> Option 2 is the most conservative and correct approach, but for the
>>> current
>>>> situation, we cooperate with the Beam community and use Beam's
>>> portability
>>>> framework for UDFs support, so we prefer to adopt the Option 1.
>>>>
>>>> Best,
>>>> Jincheng
>>>>
>>>>
>>>>
>>>> Dian Fu <di...@gmail.com> 于2019年10月8日周二 下午10:34写道：
>>>>
>>>>> Hi everyone,
>>>>>
>>>>> I would like to propose to drop Python 2 support(Currently Python
>> 2.7,
>>>>> 3.5, 3.6, 3.7 are all supported in Flink) as it's coming to an end at
>>> Jan
>>>>> 1, 2020 [1]. A lot of projects [2][3][4] has already stated or are
>>>> planning
>>>>> to drop Python 2 support.
>>>>>
>>>>> The benefits of dropping Python 2 support are:
>>>>> 1. Maintaining Python 2/3 compatibility is a burden and it makes the
>>> code
>>>>> complicate as Python 2 and Python 3 is not compatible.
>>>>> 2. There are many features which are only available in Python 3.x
>> such
>>> as
>>>>> Type Hints[5]. We can only make use of this kind of features after
>>>> dropping
>>>>> the Python 2 support.
>>>>> 3. Flink-python depends on third-part projects, such as Apache Beam
>>> (may
>>>>> add more dependencies such as pandas, etc in the near future), it's
>> not
>>>>> possible to upgrade them to the latest version once they drop the
>>> Python
>>>> 2
>>>>> support.
>>>>>
>>>>> Here are the options we have:
>>>>> 1. Drop Python 2 support in 1.10:
>>>>> As flink-python module is a new module added since 1.9.0 and so
>>> dropping
>>>>> Python 2 support at the early stage seems a good choice for us.
>>>>> 2. Deprecate Python 2 in 1.10 and drop its support in 1.11:
>>>>> As 1.10 is planned to be released around the beginning of 2020. This
>> is
>>>>> also aligned with the official Python 2 support.
>>>>>
>>>>> Personally I prefer option 1 as flink-python is new module and there
>> is
>>>> no
>>>>> much history reasons to consider.
>>>>>
>>>>> Looking forward to your feedback!
>>>>>
>>>>> Regards,
>>>>> Dian
>>>>>
>>>>> [1] https://pythonclock.org/ <https://pythonclock.org/>
>>>>> [2] https://python3statement.org/ <https://python3statement.org/>
>>>>> [3]
>>>> https://spark.apache.org/news/plan-for-dropping-python-2-support.html
>>>>> <
>> https://spark.apache.org/news/plan-for-dropping-python-2-support.html
>>>>
>>>>> [4]
>>>>>
>>>>
>>>
>> https://lists.apache.org/thread.html/eba6caa58ea79a7ecbc8560d1c680a366b44c531d96ce5c699d41535@%3Cdev.beam.apache.org%3E
>>>>> <
>>>>>
>>>>
>>>
>> https://lists.apache.org/thread.html/eba6caa58ea79a7ecbc8560d1c680a366b44c531d96ce5c699d41535@%3Cdev.beam.apache.org%3E
>>>>>>
>>>>> [5]
>>>>>
>>>>
>>>
>> https://stackoverflow.com/questions/32557920/what-are-type-hints-in-python-3-5
>>>>> <
>>>>>
>>>>
>>>
>> https://stackoverflow.com/questions/32557920/what-are-type-hints-in-python-3-5
>>>>>>
>>>>
>>>
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
Re: [DISCUSS] Drop Python 2 support for 1.10
Posted by Dian Fu <di...@gmail.com>.
Thanks everyone for your reply.
So far all the reply tend to option 1 (dropping Python 2 support in 1.10) and will continue to hear if there are any other opinions.
@Jincheng @Hequn, you are right, things become more complicate if dropping Python 2 support is performed after Python UDF has been supported. Users will have to migrate their Python UDFs if they have used features which only are supported in Python 2.
Thanks @Yu for your suggestion. It makes much sense to me and will do that. Also CC @user and @user-zh ML in case any users are concerned about this.
Thanks,
Dian
> 在 2019年10月9日，下午1:14，Yu Li <ca...@gmail.com> 写道：
>
> Thanks for bringing this up Dian.
>
> Since python 2.7 support was added in 1.9.0 and would be EOL near the
> planned release time for 1.10, I could see a good reason to take option 1.
>
> Please remember to add an explicit release note and would be better to send
> a notification in user ML about the plan to drop it, just in case some
> 1.9.0 users are already using python 2.7 in their product env.
>
> Best Regards,
> Yu
>
>
> On Wed, 9 Oct 2019 at 11:13, Jeff Zhang <zj...@gmail.com> wrote:
>
>> +1
>>
>> Hequn Cheng <ch...@gmail.com> 于2019年10月9日周三 上午11:07写道：
>>
>>> Hi Dian,
>>>
>>> +1 to drop Python 2 directly.
>>>
>>> Just as @jincheng said, things would be more complicated if we are going
>> to
>>> support python UDFs.
>>> The python UDFs will introduce a lot of python dependencies which will
>> also
>>> drop the support of Python 2, such as beam, pandas, pyarrow, etc.
>>> Given this and Python 2 will reach EOL on Jan 1 2020. I think we can drop
>>> Python 2 in Flink as well.
>>>
>>> As for the two options, I think we can drop it directly in 1.10. The
>>> flink-python is introduced just from 1.9, I think it's safe to drop it
>> for
>>> now.
>>> And we can also benefit from it when we add support for python UDFs.
>>>
>>> Best, Hequn
>>>
>>>
>>> On Wed, Oct 9, 2019 at 8:40 AM jincheng sun <su...@gmail.com>
>>> wrote:
>>>
>>>> Hi Dian，
>>>>
>>>> Thanks for bringing this discussion!
>>>>
>>>> In Flink 1.9 we only add Python Table API mapping to Java Table
>>> API(without
>>>> Python UDFs), there no special requirements for Python version, so we
>> add
>>>> python 2,7 support. But for Flink 1.10, we add the Python UDFs support,
>>>> i.e., user will add more python code in Flink job and more requirements
>>> for
>>>> the features of the Python language.So I think It's better to follow
>> the
>>>> rhythm of Python official.
>>>>
>>>> Option 2 is the most conservative and correct approach, but for the
>>> current
>>>> situation, we cooperate with the Beam community and use Beam's
>>> portability
>>>> framework for UDFs support, so we prefer to adopt the Option 1.
>>>>
>>>> Best,
>>>> Jincheng
>>>>
>>>>
>>>>
>>>> Dian Fu <di...@gmail.com> 于2019年10月8日周二 下午10:34写道：
>>>>
>>>>> Hi everyone,
>>>>>
>>>>> I would like to propose to drop Python 2 support(Currently Python
>> 2.7,
>>>>> 3.5, 3.6, 3.7 are all supported in Flink) as it's coming to an end at
>>> Jan
>>>>> 1, 2020 [1]. A lot of projects [2][3][4] has already stated or are
>>>> planning
>>>>> to drop Python 2 support.
>>>>>
>>>>> The benefits of dropping Python 2 support are:
>>>>> 1. Maintaining Python 2/3 compatibility is a burden and it makes the
>>> code
>>>>> complicate as Python 2 and Python 3 is not compatible.
>>>>> 2. There are many features which are only available in Python 3.x
>> such
>>> as
>>>>> Type Hints[5]. We can only make use of this kind of features after
>>>> dropping
>>>>> the Python 2 support.
>>>>> 3. Flink-python depends on third-part projects, such as Apache Beam
>>> (may
>>>>> add more dependencies such as pandas, etc in the near future), it's
>> not
>>>>> possible to upgrade them to the latest version once they drop the
>>> Python
>>>> 2
>>>>> support.
>>>>>
>>>>> Here are the options we have:
>>>>> 1. Drop Python 2 support in 1.10:
>>>>> As flink-python module is a new module added since 1.9.0 and so
>>> dropping
>>>>> Python 2 support at the early stage seems a good choice for us.
>>>>> 2. Deprecate Python 2 in 1.10 and drop its support in 1.11:
>>>>> As 1.10 is planned to be released around the beginning of 2020. This
>> is
>>>>> also aligned with the official Python 2 support.
>>>>>
>>>>> Personally I prefer option 1 as flink-python is new module and there
>> is
>>>> no
>>>>> much history reasons to consider.
>>>>>
>>>>> Looking forward to your feedback!
>>>>>
>>>>> Regards,
>>>>> Dian
>>>>>
>>>>> [1] https://pythonclock.org/ <https://pythonclock.org/>
>>>>> [2] https://python3statement.org/ <https://python3statement.org/>
>>>>> [3]
>>>> https://spark.apache.org/news/plan-for-dropping-python-2-support.html
>>>>> <
>> https://spark.apache.org/news/plan-for-dropping-python-2-support.html
>>>>
>>>>> [4]
>>>>>
>>>>
>>>
>> https://lists.apache.org/thread.html/eba6caa58ea79a7ecbc8560d1c680a366b44c531d96ce5c699d41535@%3Cdev.beam.apache.org%3E
>>>>> <
>>>>>
>>>>
>>>
>> https://lists.apache.org/thread.html/eba6caa58ea79a7ecbc8560d1c680a366b44c531d96ce5c699d41535@%3Cdev.beam.apache.org%3E
>>>>>>
>>>>> [5]
>>>>>
>>>>
>>>
>> https://stackoverflow.com/questions/32557920/what-are-type-hints-in-python-3-5
>>>>> <
>>>>>
>>>>
>>>
>> https://stackoverflow.com/questions/32557920/what-are-type-hints-in-python-3-5
>>>>>>
>>>>
>>>
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
Re: [DISCUSS] Drop Python 2 support for 1.10
Posted by Yu Li <ca...@gmail.com>.
Thanks for bringing this up Dian.
Since python 2.7 support was added in 1.9.0 and would be EOL near the
planned release time for 1.10, I could see a good reason to take option 1.
Please remember to add an explicit release note and would be better to send
a notification in user ML about the plan to drop it, just in case some
1.9.0 users are already using python 2.7 in their product env.
Best Regards,
Yu
On Wed, 9 Oct 2019 at 11:13, Jeff Zhang <zj...@gmail.com> wrote:
> +1
>
> Hequn Cheng <ch...@gmail.com> 于2019年10月9日周三 上午11:07写道：
>
> > Hi Dian,
> >
> > +1 to drop Python 2 directly.
> >
> > Just as @jincheng said, things would be more complicated if we are going
> to
> > support python UDFs.
> > The python UDFs will introduce a lot of python dependencies which will
> also
> > drop the support of Python 2, such as beam, pandas, pyarrow, etc.
> > Given this and Python 2 will reach EOL on Jan 1 2020. I think we can drop
> > Python 2 in Flink as well.
> >
> > As for the two options, I think we can drop it directly in 1.10. The
> > flink-python is introduced just from 1.9, I think it's safe to drop it
> for
> > now.
> > And we can also benefit from it when we add support for python UDFs.
> >
> > Best, Hequn
> >
> >
> > On Wed, Oct 9, 2019 at 8:40 AM jincheng sun <su...@gmail.com>
> > wrote:
> >
> > > Hi Dian，
> > >
> > > Thanks for bringing this discussion!
> > >
> > > In Flink 1.9 we only add Python Table API mapping to Java Table
> > API(without
> > > Python UDFs), there no special requirements for Python version, so we
> add
> > > python 2,7 support. But for Flink 1.10, we add the Python UDFs support,
> > > i.e., user will add more python code in Flink job and more requirements
> > for
> > > the features of the Python language.So I think It's better to follow
> the
> > > rhythm of Python official.
> > >
> > > Option 2 is the most conservative and correct approach, but for the
> > current
> > > situation, we cooperate with the Beam community and use Beam's
> > portability
> > > framework for UDFs support, so we prefer to adopt the Option 1.
> > >
> > > Best,
> > > Jincheng
> > >
> > >
> > >
> > > Dian Fu <di...@gmail.com> 于2019年10月8日周二 下午10:34写道：
> > >
> > > > Hi everyone,
> > > >
> > > > I would like to propose to drop Python 2 support(Currently Python
> 2.7,
> > > > 3.5, 3.6, 3.7 are all supported in Flink) as it's coming to an end at
> > Jan
> > > > 1, 2020 [1]. A lot of projects [2][3][4] has already stated or are
> > > planning
> > > > to drop Python 2 support.
> > > >
> > > > The benefits of dropping Python 2 support are:
> > > > 1. Maintaining Python 2/3 compatibility is a burden and it makes the
> > code
> > > > complicate as Python 2 and Python 3 is not compatible.
> > > > 2. There are many features which are only available in Python 3.x
> such
> > as
> > > > Type Hints[5]. We can only make use of this kind of features after
> > > dropping
> > > > the Python 2 support.
> > > > 3. Flink-python depends on third-part projects, such as Apache Beam
> > (may
> > > > add more dependencies such as pandas, etc in the near future), it's
> not
> > > > possible to upgrade them to the latest version once they drop the
> > Python
> > > 2
> > > > support.
> > > >
> > > > Here are the options we have:
> > > > 1. Drop Python 2 support in 1.10:
> > > > As flink-python module is a new module added since 1.9.0 and so
> > dropping
> > > > Python 2 support at the early stage seems a good choice for us.
> > > > 2. Deprecate Python 2 in 1.10 and drop its support in 1.11:
> > > > As 1.10 is planned to be released around the beginning of 2020. This
> is
> > > > also aligned with the official Python 2 support.
> > > >
> > > > Personally I prefer option 1 as flink-python is new module and there
> is
> > > no
> > > > much history reasons to consider.
> > > >
> > > > Looking forward to your feedback!
> > > >
> > > > Regards,
> > > > Dian
> > > >
> > > > [1] https://pythonclock.org/ <https://pythonclock.org/>
> > > > [2] https://python3statement.org/ <https://python3statement.org/>
> > > > [3]
> > > https://spark.apache.org/news/plan-for-dropping-python-2-support.html
> > > > <
> https://spark.apache.org/news/plan-for-dropping-python-2-support.html
> > >
> > > > [4]
> > > >
> > >
> >
> https://lists.apache.org/thread.html/eba6caa58ea79a7ecbc8560d1c680a366b44c531d96ce5c699d41535@%3Cdev.beam.apache.org%3E
> > > > <
> > > >
> > >
> >
> https://lists.apache.org/thread.html/eba6caa58ea79a7ecbc8560d1c680a366b44c531d96ce5c699d41535@%3Cdev.beam.apache.org%3E
> > > > >
> > > > [5]
> > > >
> > >
> >
> https://stackoverflow.com/questions/32557920/what-are-type-hints-in-python-3-5
> > > > <
> > > >
> > >
> >
> https://stackoverflow.com/questions/32557920/what-are-type-hints-in-python-3-5
> > > > >
> > >
> >
>
>
> --
> Best Regards
>
> Jeff Zhang
>
Re: [DISCUSS] Drop Python 2 support for 1.10
Posted by Jeff Zhang <zj...@gmail.com>.
+1
Hequn Cheng <ch...@gmail.com> 于2019年10月9日周三 上午11:07写道：
> Hi Dian,
>
> +1 to drop Python 2 directly.
>
> Just as @jincheng said, things would be more complicated if we are going to
> support python UDFs.
> The python UDFs will introduce a lot of python dependencies which will also
> drop the support of Python 2, such as beam, pandas, pyarrow, etc.
> Given this and Python 2 will reach EOL on Jan 1 2020. I think we can drop
> Python 2 in Flink as well.
>
> As for the two options, I think we can drop it directly in 1.10. The
> flink-python is introduced just from 1.9, I think it's safe to drop it for
> now.
> And we can also benefit from it when we add support for python UDFs.
>
> Best, Hequn
>
>
> On Wed, Oct 9, 2019 at 8:40 AM jincheng sun <su...@gmail.com>
> wrote:
>
> > Hi Dian，
> >
> > Thanks for bringing this discussion!
> >
> > In Flink 1.9 we only add Python Table API mapping to Java Table
> API(without
> > Python UDFs), there no special requirements for Python version, so we add
> > python 2,7 support. But for Flink 1.10, we add the Python UDFs support,
> > i.e., user will add more python code in Flink job and more requirements
> for
> > the features of the Python language.So I think It's better to follow the
> > rhythm of Python official.
> >
> > Option 2 is the most conservative and correct approach, but for the
> current
> > situation, we cooperate with the Beam community and use Beam's
> portability
> > framework for UDFs support, so we prefer to adopt the Option 1.
> >
> > Best,
> > Jincheng
> >
> >
> >
> > Dian Fu <di...@gmail.com> 于2019年10月8日周二 下午10:34写道：
> >
> > > Hi everyone,
> > >
> > > I would like to propose to drop Python 2 support(Currently Python 2.7,
> > > 3.5, 3.6, 3.7 are all supported in Flink) as it's coming to an end at
> Jan
> > > 1, 2020 [1]. A lot of projects [2][3][4] has already stated or are
> > planning
> > > to drop Python 2 support.
> > >
> > > The benefits of dropping Python 2 support are:
> > > 1. Maintaining Python 2/3 compatibility is a burden and it makes the
> code
> > > complicate as Python 2 and Python 3 is not compatible.
> > > 2. There are many features which are only available in Python 3.x such
> as
> > > Type Hints[5]. We can only make use of this kind of features after
> > dropping
> > > the Python 2 support.
> > > 3. Flink-python depends on third-part projects, such as Apache Beam
> (may
> > > add more dependencies such as pandas, etc in the near future), it's not
> > > possible to upgrade them to the latest version once they drop the
> Python
> > 2
> > > support.
> > >
> > > Here are the options we have:
> > > 1. Drop Python 2 support in 1.10:
> > > As flink-python module is a new module added since 1.9.0 and so
> dropping
> > > Python 2 support at the early stage seems a good choice for us.
> > > 2. Deprecate Python 2 in 1.10 and drop its support in 1.11:
> > > As 1.10 is planned to be released around the beginning of 2020. This is
> > > also aligned with the official Python 2 support.
> > >
> > > Personally I prefer option 1 as flink-python is new module and there is
> > no
> > > much history reasons to consider.
> > >
> > > Looking forward to your feedback!
> > >
> > > Regards,
> > > Dian
> > >
> > > [1] https://pythonclock.org/ <https://pythonclock.org/>
> > > [2] https://python3statement.org/ <https://python3statement.org/>
> > > [3]
> > https://spark.apache.org/news/plan-for-dropping-python-2-support.html
> > > <https://spark.apache.org/news/plan-for-dropping-python-2-support.html
> >
> > > [4]
> > >
> >
> https://lists.apache.org/thread.html/eba6caa58ea79a7ecbc8560d1c680a366b44c531d96ce5c699d41535@%3Cdev.beam.apache.org%3E
> > > <
> > >
> >
> https://lists.apache.org/thread.html/eba6caa58ea79a7ecbc8560d1c680a366b44c531d96ce5c699d41535@%3Cdev.beam.apache.org%3E
> > > >
> > > [5]
> > >
> >
> https://stackoverflow.com/questions/32557920/what-are-type-hints-in-python-3-5
> > > <
> > >
> >
> https://stackoverflow.com/questions/32557920/what-are-type-hints-in-python-3-5
> > > >
> >
>
--
Best Regards
Jeff Zhang
Re: [DISCUSS] Drop Python 2 support for 1.10
Posted by Hequn Cheng <ch...@gmail.com>.
Hi Dian,
+1 to drop Python 2 directly.
Just as @jincheng said, things would be more complicated if we are going to
support python UDFs.
The python UDFs will introduce a lot of python dependencies which will also
drop the support of Python 2, such as beam, pandas, pyarrow, etc.
Given this and Python 2 will reach EOL on Jan 1 2020. I think we can drop
Python 2 in Flink as well.
As for the two options, I think we can drop it directly in 1.10. The
flink-python is introduced just from 1.9, I think it's safe to drop it for
now.
And we can also benefit from it when we add support for python UDFs.
Best, Hequn
On Wed, Oct 9, 2019 at 8:40 AM jincheng sun <su...@gmail.com>
wrote:
> Hi Dian，
>
> Thanks for bringing this discussion!
>
> In Flink 1.9 we only add Python Table API mapping to Java Table API(without
> Python UDFs), there no special requirements for Python version, so we add
> python 2,7 support. But for Flink 1.10, we add the Python UDFs support,
> i.e., user will add more python code in Flink job and more requirements for
> the features of the Python language.So I think It's better to follow the
> rhythm of Python official.
>
> Option 2 is the most conservative and correct approach, but for the current
> situation, we cooperate with the Beam community and use Beam's portability
> framework for UDFs support, so we prefer to adopt the Option 1.
>
> Best,
> Jincheng
>
>
>
> Dian Fu <di...@gmail.com> 于2019年10月8日周二 下午10:34写道：
>
> > Hi everyone,
> >
> > I would like to propose to drop Python 2 support(Currently Python 2.7,
> > 3.5, 3.6, 3.7 are all supported in Flink) as it's coming to an end at Jan
> > 1, 2020 [1]. A lot of projects [2][3][4] has already stated or are
> planning
> > to drop Python 2 support.
> >
> > The benefits of dropping Python 2 support are:
> > 1. Maintaining Python 2/3 compatibility is a burden and it makes the code
> > complicate as Python 2 and Python 3 is not compatible.
> > 2. There are many features which are only available in Python 3.x such as
> > Type Hints[5]. We can only make use of this kind of features after
> dropping
> > the Python 2 support.
> > 3. Flink-python depends on third-part projects, such as Apache Beam (may
> > add more dependencies such as pandas, etc in the near future), it's not
> > possible to upgrade them to the latest version once they drop the Python
> 2
> > support.
> >
> > Here are the options we have:
> > 1. Drop Python 2 support in 1.10:
> > As flink-python module is a new module added since 1.9.0 and so dropping
> > Python 2 support at the early stage seems a good choice for us.
> > 2. Deprecate Python 2 in 1.10 and drop its support in 1.11:
> > As 1.10 is planned to be released around the beginning of 2020. This is
> > also aligned with the official Python 2 support.
> >
> > Personally I prefer option 1 as flink-python is new module and there is
> no
> > much history reasons to consider.
> >
> > Looking forward to your feedback!
> >
> > Regards,
> > Dian
> >
> > [1] https://pythonclock.org/ <https://pythonclock.org/>
> > [2] https://python3statement.org/ <https://python3statement.org/>
> > [3]
> https://spark.apache.org/news/plan-for-dropping-python-2-support.html
> > <https://spark.apache.org/news/plan-for-dropping-python-2-support.html>
> > [4]
> >
> https://lists.apache.org/thread.html/eba6caa58ea79a7ecbc8560d1c680a366b44c531d96ce5c699d41535@%3Cdev.beam.apache.org%3E
> > <
> >
> https://lists.apache.org/thread.html/eba6caa58ea79a7ecbc8560d1c680a366b44c531d96ce5c699d41535@%3Cdev.beam.apache.org%3E
> > >
> > [5]
> >
> https://stackoverflow.com/questions/32557920/what-are-type-hints-in-python-3-5
> > <
> >
> https://stackoverflow.com/questions/32557920/what-are-type-hints-in-python-3-5
> > >
>
Re: [DISCUSS] Drop Python 2 support for 1.10
Posted by jincheng sun <su...@gmail.com>.
Hi Dian，
Thanks for bringing this discussion!
In Flink 1.9 we only add Python Table API mapping to Java Table API(without
Python UDFs), there no special requirements for Python version, so we add
python 2,7 support. But for Flink 1.10, we add the Python UDFs support,
i.e., user will add more python code in Flink job and more requirements for
the features of the Python language.So I think It's better to follow the
rhythm of Python official.
Option 2 is the most conservative and correct approach, but for the current
situation, we cooperate with the Beam community and use Beam's portability
framework for UDFs support, so we prefer to adopt the Option 1.
Best,
Jincheng
Dian Fu <di...@gmail.com> 于2019年10月8日周二 下午10:34写道：
> Hi everyone,
>
> I would like to propose to drop Python 2 support(Currently Python 2.7,
> 3.5, 3.6, 3.7 are all supported in Flink) as it's coming to an end at Jan
> 1, 2020 [1]. A lot of projects [2][3][4] has already stated or are planning
> to drop Python 2 support.
>
> The benefits of dropping Python 2 support are:
> 1. Maintaining Python 2/3 compatibility is a burden and it makes the code
> complicate as Python 2 and Python 3 is not compatible.
> 2. There are many features which are only available in Python 3.x such as
> Type Hints[5]. We can only make use of this kind of features after dropping
> the Python 2 support.
> 3. Flink-python depends on third-part projects, such as Apache Beam (may
> add more dependencies such as pandas, etc in the near future), it's not
> possible to upgrade them to the latest version once they drop the Python 2
> support.
>
> Here are the options we have:
> 1. Drop Python 2 support in 1.10:
> As flink-python module is a new module added since 1.9.0 and so dropping
> Python 2 support at the early stage seems a good choice for us.
> 2. Deprecate Python 2 in 1.10 and drop its support in 1.11:
> As 1.10 is planned to be released around the beginning of 2020. This is
> also aligned with the official Python 2 support.
>
> Personally I prefer option 1 as flink-python is new module and there is no
> much history reasons to consider.
>
> Looking forward to your feedback!
>
> Regards,
> Dian
>
> [1] https://pythonclock.org/ <https://pythonclock.org/>
> [2] https://python3statement.org/ <https://python3statement.org/>
> [3] https://spark.apache.org/news/plan-for-dropping-python-2-support.html
> <https://spark.apache.org/news/plan-for-dropping-python-2-support.html>
> [4]
> https://lists.apache.org/thread.html/eba6caa58ea79a7ecbc8560d1c680a366b44c531d96ce5c699d41535@%3Cdev.beam.apache.org%3E
> <
> https://lists.apache.org/thread.html/eba6caa58ea79a7ecbc8560d1c680a366b44c531d96ce5c699d41535@%3Cdev.beam.apache.org%3E
> >
> [5]
> https://stackoverflow.com/questions/32557920/what-are-type-hints-in-python-3-5
> <
> https://stackoverflow.com/questions/32557920/what-are-type-hints-in-python-3-5
> >