You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "AnandInguva (via GitHub)" <gi...@apache.org> on 2023/02/22 14:06:57 UTC
[GitHub] [beam] AnandInguva opened a new issue, #25590: [Task]: Update the minor version of protobuf library prior to Beam release.
AnandInguva opened a new issue, #25590:
URL: https://github.com/apache/beam/issues/25590
### What needs to happen?
If a Beam dependency has a flexible upper bound, users will download the most recent compatible version of a dependency at sdk installation time. Overtime, the version used at job submission may become newer than the version installed in a released Beam container. Given that forwards-compatiblity of a protobuf library is not guaranteed, the pipeline may fail.
To mitigate, protobuf library should be specified in install_requires with tight version bounds limiting to a particular minor version. This can cause inconvenience to Beam user, if we depend on an old version of a library. Therefore, we should periodically update the version we use, at least once per release cycle.
### Issue Priority
Priority: 3 (nice-to-have improvement)
### Issue Components
- [X] Component: Python SDK
- [ ] Component: Java SDK
- [ ] Component: Go SDK
- [ ] Component: Typescript SDK
- [ ] Component: IO connector
- [ ] Component: Beam examples
- [ ] Component: Beam playground
- [ ] Component: Beam katas
- [ ] Component: Website
- [ ] Component: Spark Runner
- [ ] Component: Flink Runner
- [ ] Component: Samza Runner
- [ ] Component: Twister2 Runner
- [ ] Component: Hazelcast Jet Runner
- [ ] Component: Google Cloud Dataflow Runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Abacn commented on issue #25590: [Task]: Update the minor version of protobuf library in the upper bound prior to Beam release.
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #25590:
URL: https://github.com/apache/beam/issues/25590#issuecomment-1609967696
This is still up to date (4.23.3<4.24.0), moving to 2.50
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] riteshghorse commented on issue #25590: [Task]: Update the minor version of protobuf library in the upper bound prior to Beam release.
Posted by "riteshghorse (via GitHub)" <gi...@apache.org>.
riteshghorse commented on issue #25590:
URL: https://github.com/apache/beam/issues/25590#issuecomment-1662870036
Already on latest version 4.23.4. Moving to 2.51.0 Milestone
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on issue #25590: [Task]: Update the minor version of protobuf library in the upper bound prior to Beam release.
Posted by "tvalentyn (via GitHub)" <gi...@apache.org>.
tvalentyn commented on issue #25590:
URL: https://github.com/apache/beam/issues/25590#issuecomment-1448555928
cc: @tvalentyn
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Task]: Update the minor version of protobuf library in the upper bound prior to Beam release. [beam]
Posted by "lostluck (via GitHub)" <gi...@apache.org>.
lostluck commented on issue #25590:
URL: https://github.com/apache/beam/issues/25590#issuecomment-1896771317
There's one week until the 2.54.0 cut and this issue is tagged for that release, if possible/necessary please complete the necessary work before then, or move this to the [2.55.0 Release Milestone](https://github.com/apache/beam/milestone/19).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Task]: Update the minor version of protobuf library in the upper bound prior to Beam release. [beam]
Posted by "jrmccluskey (via GitHub)" <gi...@apache.org>.
jrmccluskey commented on issue #25590:
URL: https://github.com/apache/beam/issues/25590#issuecomment-1843329447
It looks like we're up to date with the upper bound at <4.26.0 and the latest release is 4.25.1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Task]: Update the minor version of protobuf library in the upper bound prior to Beam release. [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #25590:
URL: https://github.com/apache/beam/issues/25590#issuecomment-2047862705
We still need to wait for [google-api-core](https://pypi.org/project/google-api-core/#history) to release new version. The latest version of google-api-core has a dependency on protobuf<5 which would result in dependency conflict if we update the protobuf version for beam.
```
apache-beam[gcp,test] 2.55.0.dev0 depends on protobuf==5.26.0rc3
google-api-core 2.17.1 depends on protobuf!=3.20.0, !=3.20.1, !=4.21.0, !=4.21.1, !=4.21.2, !=4.21.3, !=4.21.4, !=4.21.5, <5.0.0.dev0 and >=3.19.5
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lostluck commented on issue #25590: [Task]: Update the minor version of protobuf library in the upper bound prior to Beam release.
Posted by "lostluck (via GitHub)" <gi...@apache.org>.
lostluck commented on issue #25590:
URL: https://github.com/apache/beam/issues/25590#issuecomment-1662852377
2.50 release manager here.
This issue is currently tagged for the 2.50.0 release, which cuts in a week on August 9th.
Please complete work and get it into the main branch in that time, or move this issue to the 2.51 Milestone: https://github.com/apache/beam/milestone/15
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on issue #25590: [Task]: Update the minor version of protobuf library prior to Beam release.
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #25590:
URL: https://github.com/apache/beam/issues/25590#issuecomment-1443822916
The next release is 2.47.0 (the current one is 2.46.0)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AnandInguva commented on issue #25590: [Task]: Update the minor version of protobuf library prior to Beam release.
Posted by "AnandInguva (via GitHub)" <gi...@apache.org>.
AnandInguva commented on issue #25590:
URL: https://github.com/apache/beam/issues/25590#issuecomment-1443821245
@damccorm can you add the milestone 2.48.0 to catch this for the next release?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on issue #25590: [Task]: Update the minor version of protobuf library in the upper bound prior to Beam release.
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #25590:
URL: https://github.com/apache/beam/issues/25590#issuecomment-1494951510
This is still up to date, moving to 2.48
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Task]: Update the minor version of protobuf library in the upper bound prior to Beam release. [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #25590:
URL: https://github.com/apache/beam/issues/25590#issuecomment-1783371210
We're up to date as of now looking at https://pypi.org/project/protobuf/ and https://github.com/apache/beam/blob/master/sdks/python/setup.py#L300
This might change if protobuf gets a fix in for #28246 - but I'll at least move this blocker forward and we can address that in the separate issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Task]: Update the minor version of protobuf library in the upper bound prior to Beam release. [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #25590:
URL: https://github.com/apache/beam/issues/25590#issuecomment-1981406944
4.25.3 is the latest 4.x release as of 2.55.0 cut day. and it is `<4.26.0`. Next release needs to aware that the upcoming release bumped the major version, 4.25 -> 5.26, indicating breaking change
note: protobuf currently encode its version in minor version number for all language impls; and the major version is reserved for language specific breaking change
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Task]: Update the minor version of protobuf library in the upper bound prior to Beam release. [beam]
Posted by "riteshghorse (via GitHub)" <gi...@apache.org>.
riteshghorse commented on issue #25590:
URL: https://github.com/apache/beam/issues/25590#issuecomment-1981553174
+1. I'm working with @tvalentyn on workaround for that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org