You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/09/09 04:15:03 UTC
[GitHub] [beam] tvalentyn opened a new issue, #23119: [Task]: Update the minor version of cloudpickle library prior to Beam release.
tvalentyn opened a new issue, #23119:
URL: https://github.com/apache/beam/issues/23119
### What needs to happen?
If a Beam dependency has a flexible upper bound, users will download the most recent compatible version of a dependency at sdk installation time. Overtime, the version used at job submission may become newer than the version installed in a released Beam container. Given that forwards-compatiblity of a pickle library is not guaranteed, the pipeline may fail to unpickle at runtime.
To mitigate, pickle libraries should be specified in `install_requires` with tight version bounds limiting to a particular minor version. This can cause inconvenience to Beam user, if we depend on an old version of a library. Therefore, we should periodically update the version we use, at least once per release cycle.
We can consider to close this issue when either condition is met:
- Beam vendors cloudpickle.
- Beam stages used version of cloudpickle at pipeline submission time, and uses it at runtime.
- Beam communicates to the users a list of vetted versions of each Beam dependency that users must install to use Beam in a supported configuration. Then, we can open up a version range.
Until then, don't close this issue, instead, move it to the next release milestone.
### Issue Priority
Priority: 3
### Issue Component
Component: sdk-py-core
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] riteshghorse commented on issue #23119: [Task]: Update the minor version of cloudpickle library prior to Beam release.
Posted by "riteshghorse (via GitHub)" <gi...@apache.org>.
riteshghorse commented on issue #23119:
URL: https://github.com/apache/beam/issues/23119#issuecomment-1662865263
No updates to cloudpickle version. Moving to 2.51.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on issue #23119: [Task]: Update the minor version of cloudpickle library prior to Beam release.
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #23119:
URL: https://github.com/apache/beam/issues/23119#issuecomment-1422728878
Moving to 2.47 since there is not a new release
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kennknowles commented on issue #23119: [Task]: Update the minor version of cloudpickle library prior to Beam release.
Posted by GitBox <gi...@apache.org>.
kennknowles commented on issue #23119:
URL: https://github.com/apache/beam/issues/23119#issuecomment-1334443058
Looks like we are on the latest
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Task]: Update the minor version of cloudpickle library prior to Beam release. [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #23119:
URL: https://github.com/apache/beam/issues/23119#issuecomment-1783368367
It looks like there's been a major version upgrade (and no minor version upgrade) - https://pypi.org/project/cloudpickle/#history
@tvalentyn I don't think this should be a release blocker and probably represents non-trivial work to investigate/upgrade.
I'm going to move the release blocker and we should think about a good way to fund this going forward
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lostluck commented on issue #23119: [Task]: Update the minor version of cloudpickle library prior to Beam release.
Posted by "lostluck (via GitHub)" <gi...@apache.org>.
lostluck commented on issue #23119:
URL: https://github.com/apache/beam/issues/23119#issuecomment-1662852627
2.50 release manager here.
This issue is currently tagged for the 2.50.0 release, which cuts in a week on August 9th.
Please complete work and get it into the main branch in that time, or move this issue to the 2.51 Milestone: https://github.com/apache/beam/milestone/15
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Task]: Update the minor version of cloudpickle library prior to Beam release. [beam]
Posted by "tvalentyn (via GitHub)" <gi...@apache.org>.
tvalentyn commented on issue #23119:
URL: https://github.com/apache/beam/issues/23119#issuecomment-1843458848
no objections.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Task]: Update the minor version of cloudpickle library prior to Beam release. [beam]
Posted by "jrmccluskey (via GitHub)" <gi...@apache.org>.
jrmccluskey commented on issue #23119:
URL: https://github.com/apache/beam/issues/23119#issuecomment-1843322396
Similar status as before for cloudpickle, major increment to 3.0.0 was before the 2.52.0 branch cut. @tvalentyn any objections to rolling this up to 2.54?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Task]: Update the minor version of cloudpickle library prior to Beam release. [beam]
Posted by "lostluck (via GitHub)" <gi...@apache.org>.
lostluck commented on issue #23119:
URL: https://github.com/apache/beam/issues/23119#issuecomment-1896778588
There's one week until the 2.54.0 cut and this issue is tagged for that release, if possible/necessary please complete the necessary work before then, or move this to the [2.55.0 Release Milestone](https://github.com/apache/beam/milestone/19).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] johnjcasey commented on issue #23119: [Task]: Update the minor version of cloudpickle library prior to Beam release.
Posted by "johnjcasey (via GitHub)" <gi...@apache.org>.
johnjcasey commented on issue #23119:
URL: https://github.com/apache/beam/issues/23119#issuecomment-1403750866
updated for 2.45: https://github.com/apache/beam/pull/25143
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Task]: Update the minor version of cloudpickle library prior to Beam release. [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #23119:
URL: https://github.com/apache/beam/issues/23119#issuecomment-2047870201
No update here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Task]: Update the minor version of cloudpickle library prior to Beam release. [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #23119:
URL: https://github.com/apache/beam/issues/23119#issuecomment-1981400099
2.2.1 is still the latest of 2.x as of 2.55.0 cut
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on issue #23119: [Task]: Update the minor version of cloudpickle library prior to Beam release.
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #23119:
URL: https://github.com/apache/beam/issues/23119#issuecomment-1494948904
Still on latest, so I'll move to 2.48
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] riteshghorse commented on issue #23119: [Task]: Update the minor version of cloudpickle library prior to Beam release.
Posted by "riteshghorse (via GitHub)" <gi...@apache.org>.
riteshghorse commented on issue #23119:
URL: https://github.com/apache/beam/issues/23119#issuecomment-1551584167
Still on latest, moving to 2.49.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Abacn commented on issue #23119: [Task]: Update the minor version of cloudpickle library prior to Beam release.
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #23119:
URL: https://github.com/apache/beam/issues/23119#issuecomment-1609968960
2.2.1 is still the latest, moving to 2.50.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org