You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@beam.apache.org by Jarek Potiuk <ja...@potiuk.com> on 2022/01/23 22:33:40 UTC

Python SDK release good for Python 3.10/M1

Hello Apache Beam Friends,

I have attempted today (that was yet another attempt) to prepare an Apache
Airflow CI image for testing with Python 3.10.

Unlike previous attempts (where there were quite a few deps that lagged
behind)  - this one was **almost** successful.

I think the last (or at least one of the last ones that is a serious
blocker) limitation was Numpy < 1.21 which has no binary wheels for Python
3.10 (but also for MacOS M1). This one comes from the latest apache-beam
SDK 2.35.0.

I noticed, however, that the Beam Python SDK NumPy limit has already been
bumped to Numpy < 1.22  (for M1) in the main branch of your repo. This one
could solve the Python 3.10 problem (and we are also looking at M1 images
next):

https://github.com/apache/beam/commit/d845a0074d39a1604fde1879157f55e048a5d01b#diff-1275c48808de339ef6f282d844c83ec441b5cfa0debc373fdcb7dba497da4fc8


Any chance for a new release of Apache Beam Python SDK soon with numpy <
1.22 instead of <1.21?

In the past we did some exclusion for Beam when we wanted to release Python
3.9, but maybe - if we know that release is coming, we could simply wait :).

Any way we can help and speed this up ?

J.

Re: Python SDK release good for Python 3.10/M1

Posted by Jarek Potiuk <ja...@potiuk.com>.

Thanks Kamil! That will do! We shall wait :D.

On Mon, Jan 24, 2022 at 12:21 PM Kamil Bregula <ka...@snowflake.com>
wrote:

> The Beam 2.36.0 release is scheduled to be cut on 2021-12-29 (Wednesday)
> and released by 2022-02-02 according to the release calendar [1]. This
> release requires numpy>=1.14.3,<1.22.0' in Python SDK [2]. We should have
> the first RC this week.
>
> [1]
> https://calendar.google.com/calendar/u/0/embed?src=0p73sl034k80oob7seouanigd0@group.calendar.google.com&ctz=America/Los_Angeles
> [2]
> https://github.com/apache/beam/blob/release-2.36.0/sdks/python/setup.py#L145
>
>
> On Sun, Jan 23, 2022 at 11:34 PM Jarek Potiuk <ja...@potiuk.com> wrote:
>
>> Hello Apache Beam Friends,
>>
>> I have attempted today (that was yet another attempt) to prepare an
>> Apache Airflow CI image for testing with Python 3.10.
>>
>> Unlike previous attempts (where there were quite a few deps that lagged
>> behind)  - this one was **almost** successful.
>>
>> I think the last (or at least one of the last ones that is a serious
>> blocker) limitation was Numpy < 1.21 which has no binary wheels for Python
>> 3.10 (but also for MacOS M1). This one comes from the latest apache-beam
>> SDK 2.35.0.
>>
>> I noticed, however, that the Beam Python SDK NumPy limit has already been
>> bumped to Numpy < 1.22  (for M1) in the main branch of your repo. This one
>> could solve the Python 3.10 problem (and we are also looking at M1 images
>> next):
>>
>>
>> https://github.com/apache/beam/commit/d845a0074d39a1604fde1879157f55e048a5d01b#diff-1275c48808de339ef6f282d844c83ec441b5cfa0debc373fdcb7dba497da4fc8
>>
>>
>> Any chance for a new release of Apache Beam Python SDK soon with numpy <
>> 1.22 instead of <1.21?
>>
>> In the past we did some exclusion for Beam when we wanted to release
>> Python 3.9, but maybe - if we know that release is coming, we could simply
>> wait :).
>>
>> Any way we can help and speed this up ?
>>
>> J.
>>
>>

Re: Python SDK release good for Python 3.10/M1

Posted by Kamil Bregula <ka...@snowflake.com>.

The Beam 2.36.0 release is scheduled to be cut on 2021-12-29 (Wednesday)
and released by 2022-02-02 according to the release calendar [1]. This
release requires numpy>=1.14.3,<1.22.0' in Python SDK [2]. We should have
the first RC this week.

[1]
https://calendar.google.com/calendar/u/0/embed?src=0p73sl034k80oob7seouanigd0@group.calendar.google.com&ctz=America/Los_Angeles
[2]
https://github.com/apache/beam/blob/release-2.36.0/sdks/python/setup.py#L145


On Sun, Jan 23, 2022 at 11:34 PM Jarek Potiuk <ja...@potiuk.com> wrote:

> Hello Apache Beam Friends,
>
> I have attempted today (that was yet another attempt) to prepare an Apache
> Airflow CI image for testing with Python 3.10.
>
> Unlike previous attempts (where there were quite a few deps that lagged
> behind)  - this one was **almost** successful.
>
> I think the last (or at least one of the last ones that is a serious
> blocker) limitation was Numpy < 1.21 which has no binary wheels for Python
> 3.10 (but also for MacOS M1). This one comes from the latest apache-beam
> SDK 2.35.0.
>
> I noticed, however, that the Beam Python SDK NumPy limit has already been
> bumped to Numpy < 1.22  (for M1) in the main branch of your repo. This one
> could solve the Python 3.10 problem (and we are also looking at M1 images
> next):
>
>
> https://github.com/apache/beam/commit/d845a0074d39a1604fde1879157f55e048a5d01b#diff-1275c48808de339ef6f282d844c83ec441b5cfa0debc373fdcb7dba497da4fc8
>
>
> Any chance for a new release of Apache Beam Python SDK soon with numpy <
> 1.22 instead of <1.21?
>
> In the past we did some exclusion for Beam when we wanted to release
> Python 3.9, but maybe - if we know that release is coming, we could simply
> wait :).
>
> Any way we can help and speed this up ?
>
> J.
>
>