You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "liferoad (via GitHub)" <gi...@apache.org> on 2024/10/11 18:19:40 UTC
[PR] Update the python deps caused by numpy 2.0 [beam]
liferoad opened a new pull request, #32756:
URL: https://github.com/apache/beam/pull/32756
Addresses https://github.com/apache/beam/issues/32681
Following https://cwiki.apache.org/confluence/display/BEAM/Python+Tips#PythonTips-HowtoupdatedependenciesthatareinstalledinPythoncontainerimages and https://cwiki.apache.org/confluence/display/BEAM/Python+Tips#PythonTips-InstallingPythoninterpreters, updated the base image deps.
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://github.com/apache/beam/blob/master/CONTRIBUTING.md#make-the-reviewers-job-easier).
To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI or the [workflows README](https://github.com/apache/beam/blob/master/.github/workflows/README.md) to see a list of phrases to trigger workflows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update the python deps caused by numpy 2.0 [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on PR #32756:
URL: https://github.com/apache/beam/pull/32756#issuecomment-2432238242
We will pin protobuf==4.25.5 for now with Beam 2.61.0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update the python deps caused by numpy 2.0 [beam]
Posted by "tvalentyn (via GitHub)" <gi...@apache.org>.
tvalentyn commented on code in PR #32756:
URL: https://github.com/apache/beam/pull/32756#discussion_r1797364812
##########
sdks/python/container/py39/base_image_requirements.txt:
##########
@@ -118,7 +118,7 @@ pandas==2.1.4
parameterized==0.9.0
pluggy==1.5.0
proto-plus==1.24.0
-protobuf==4.25.5
+protobuf==5.28.2
Review Comment:
this is a notable change; numpy however didn't update.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update the python deps caused by numpy 2.0 [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32756:
URL: https://github.com/apache/beam/pull/32756#issuecomment-2441421992
Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment `assign to next reviewer`:
R: @tvalentyn for label python.
Available commands:
- `stop reviewer notifications` - opt out of the automated review tooling
- `remind me after tests pass` - tag the comment author after tests pass
- `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update the python deps caused by numpy 2.0 [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on code in PR #32756:
URL: https://github.com/apache/beam/pull/32756#discussion_r1797368838
##########
sdks/python/container/py39/base_image_requirements.txt:
##########
@@ -118,7 +118,7 @@ pandas==2.1.4
parameterized==0.9.0
pluggy==1.5.0
proto-plus==1.24.0
-protobuf==4.25.5
+protobuf==5.28.2
Review Comment:
@jrmccluskey I assume this is expected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update the python deps caused by numpy 2.0 [beam]
Posted by "tvalentyn (via GitHub)" <gi...@apache.org>.
tvalentyn commented on code in PR #32756:
URL: https://github.com/apache/beam/pull/32756#discussion_r1799819306
##########
sdks/python/container/py39/base_image_requirements.txt:
##########
@@ -118,7 +118,7 @@ pandas==2.1.4
parameterized==0.9.0
pluggy==1.5.0
proto-plus==1.24.0
-protobuf==4.25.5
+protobuf==5.28.2
Review Comment:
@jrmccluskey are protobuf 4.x and 5.x forwards/backwards compatible?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update the python deps caused by numpy 2.0 [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on PR #32756:
URL: https://github.com/apache/beam/pull/32756#issuecomment-2441541398
stop reviewer notifications
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update the python deps [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on PR #32756:
URL: https://github.com/apache/beam/pull/32756#issuecomment-2448365582
@damccorm what do you think? I am fine to do this after 2.61.0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update the python deps caused by numpy 2.0 [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32756:
URL: https://github.com/apache/beam/pull/32756#issuecomment-2408015859
Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`:
R: @damccorm for label python.
Available commands:
- `stop reviewer notifications` - opt out of the automated review tooling
- `remind me after tests pass` - tag the comment author after tests pass
- `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)
The PR bot will only process comments in the main thread (not review comments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update the python deps [beam]
Posted by "tvalentyn (via GitHub)" <gi...@apache.org>.
tvalentyn commented on PR #32756:
URL: https://github.com/apache/beam/pull/32756#issuecomment-2448337103
Given how close we are to the release cut, I would keep the dependency changes to a minimum or do this update after the release cut.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update the python deps caused by numpy 2.0 [beam]
Posted by "jrmccluskey (via GitHub)" <gi...@apache.org>.
jrmccluskey commented on code in PR #32756:
URL: https://github.com/apache/beam/pull/32756#discussion_r1797374963
##########
sdks/python/container/py39/base_image_requirements.txt:
##########
@@ -118,7 +118,7 @@ pandas==2.1.4
parameterized==0.9.0
pluggy==1.5.0
proto-plus==1.24.0
-protobuf==4.25.5
+protobuf==5.28.2
Review Comment:
Don't make this jump yet, some dependencies users provide themselves like tensorflow aren't supporting protobuf 5.x yet.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update the python deps caused by numpy 2.0 [beam]
Posted by "jrmccluskey (via GitHub)" <gi...@apache.org>.
jrmccluskey commented on code in PR #32756:
URL: https://github.com/apache/beam/pull/32756#discussion_r1801290356
##########
sdks/python/container/py39/base_image_requirements.txt:
##########
@@ -118,7 +118,7 @@ pandas==2.1.4
parameterized==0.9.0
pluggy==1.5.0
proto-plus==1.24.0
-protobuf==4.25.5
+protobuf==5.28.2
Review Comment:
As best I am aware the only incompatibility is the timestamp bound change, I can't find any documentation to the contrary
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update the python deps [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on PR #32756:
URL: https://github.com/apache/beam/pull/32756#issuecomment-2448373738
Its borderline, but I probably agree waiting is safer. I don't think we _need_ to do this for the current release
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update the python deps caused by numpy 2.0 [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on PR #32756:
URL: https://github.com/apache/beam/pull/32756#issuecomment-2432257783
cc @Abacn later I will polish this PR to update the deps. FYI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update the python deps [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on PR #32756:
URL: https://github.com/apache/beam/pull/32756#issuecomment-2479485636
Rebased. Based on the discussions, we should get this in ASAP after 2.61.0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update the python deps caused by numpy 2.0 [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on PR #32756:
URL: https://github.com/apache/beam/pull/32756#issuecomment-2407919072
cc @jrmccluskey
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update the python deps caused by numpy 2.0 [beam]
Posted by "tvalentyn (via GitHub)" <gi...@apache.org>.
tvalentyn commented on code in PR #32756:
URL: https://github.com/apache/beam/pull/32756#discussion_r1799820068
##########
sdks/python/container/py39/base_image_requirements.txt:
##########
@@ -118,7 +118,7 @@ pandas==2.1.4
parameterized==0.9.0
pluggy==1.5.0
proto-plus==1.24.0
-protobuf==4.25.5
+protobuf==5.28.2
Review Comment:
a smoke test would be to run a pipeline with protobuf 4.x at submission and 5.x installed in containers, and vice-versa
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update the python deps caused by numpy 2.0 [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32756:
URL: https://github.com/apache/beam/pull/32756#issuecomment-2441545862
Stopping reviewer notifications for this pull request: requested by reviewer. If you'd like to restart, comment `assign set of reviewers`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update the python deps caused by numpy 2.0 [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on code in PR #32756:
URL: https://github.com/apache/beam/pull/32756#discussion_r1797445692
##########
sdks/python/container/py39/base_image_requirements.txt:
##########
@@ -118,7 +118,7 @@ pandas==2.1.4
parameterized==0.9.0
pluggy==1.5.0
proto-plus==1.24.0
-protobuf==4.25.5
+protobuf==5.28.2
Review Comment:
ok. Do we plan to make this jump for Beam 2.61.0? or should we pin this to 4.xx?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update the python deps caused by numpy 2.0 [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32756:
URL: https://github.com/apache/beam/pull/32756#issuecomment-2431926626
Reminder, please take a look at this pr: @damccorm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update the python deps [beam]
Posted by "tvalentyn (via GitHub)" <gi...@apache.org>.
tvalentyn commented on PR #32756:
URL: https://github.com/apache/beam/pull/32756#issuecomment-2448339181
For example, if there is a regression in google-api-core==2.22.0, we have only two weeks to notice that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update the python deps caused by numpy 2.0 [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on code in PR #32756:
URL: https://github.com/apache/beam/pull/32756#discussion_r1821628034
##########
sdks/python/container/py39/base_image_requirements.txt:
##########
@@ -118,7 +118,7 @@ pandas==2.1.4
parameterized==0.9.0
pluggy==1.5.0
proto-plus==1.24.0
-protobuf==4.25.5
+protobuf==5.28.2
Review Comment:
pin protobuf to 4.25.5 for now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update the python deps [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on PR #32756:
URL: https://github.com/apache/beam/pull/32756#issuecomment-2479749288
https://github.com/apache/beam/actions/workflows/beam_PreCommit_Python_Coverage.yml, https://github.com/apache/beam/actions/workflows/beam_PreCommit_Python_ML.yml, are also read on the master. My PR is not related to these failed workflows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update the python deps [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad merged PR #32756:
URL: https://github.com/apache/beam/pull/32756
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org