You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Robert Burke <lo...@apache.org> on 2024/02/02 21:10:03 UTC
[VOTE] Release 2.54.0, release candidate #1
Hi everyone,
Please review and vote on the release candidate #1 for the version 2.54.0,
as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)
Reviewers are encouraged to test their own use cases with the release
candidate, and vote +1 if
no issues are found. Only PMC member votes will count towards the final
vote, but votes from all
community members is encouraged and helpful for finding regressions; you
can either test your own
use cases [13] or use cases from the validation sheet [10].
The complete staging area is available for your review, which includes:
* GitHub Release notes [1],
* the official Apache source release to be deployed to dist.apache.org [2],
which is signed with the key with fingerprint D20316F712213422 [3],
* all artifacts to be deployed to the Maven Central Repository [4],
* source code tag "v2.54.0-RC1" [5],
* website pull request listing the release [6], the blog post [6], and
publishing the API reference manual [7].
* Python artifacts are deployed along with the source release to the
dist.apache.org [2] and PyPI[8].
* Go artifacts and documentation are available at pkg.go.dev [9]
* Validation sheet with a tab for 2.54.0 release to help with validation
[10].
* Docker images published to Docker Hub [11].
* PR to run tests against release branch [12].
* Legacy Dataflow Java Worker image has been published, so the failing
tests are being re-run.
The vote will be open for at least 72 hours. It is adopted by majority
approval, with at least 3 PMC affirmative votes.
For guidelines on how to try the release in your projects, check out our RC
testing guide [13].
Thanks,
Robert Burke
Beam 2.54.0 Release Manager
[1] https://github.com/apache/beam/milestone/18?closed=1
[2] https://dist.apache.org/repos/dist/dev/beam/2.54.0/
[3] https://dist.apache.org/repos/dist/release/beam/KEYS
[4] https://repository.apache.org/content/repositories/orgapachebeam-1367/
[5] https://github.com/apache/beam/tree/v2.54.0-RC1
[6] https://github.com/apache/beam/pull/30201
[7] https://github.com/apache/beam-site/pull/658
[8] https://pypi.org/project/apache-beam/2.54.0rc1/
[9]
https://pkg.go.dev/github.com/apache/beam/sdks/v2@v2.54.0-RC1/go/pkg/beam
[10]
https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=28763708
[11] https://hub.docker.com/search?q=apache%2Fbeam&type=image
[12] https://github.com/apache/beam/pull/30104
[13]
https://github.com/apache/beam/blob/master/contributor-docs/rc-testing-guide.md
Re: [VOTE] Release 2.54.0, release candidate #1
Posted by Robert Burke <lo...@apache.org>.
The release branch nearly has all the suites passing!
I don't presently consider the failing ones blocking as they have been permared for at least 2 releases already (using just the number to refer to the tracking issues).
PostCommit XVR GoUsingJava Dataflow - Has never been successful, using #28339 to track.
PostCommit Java Sickbay - Has never been successful, filed #30236 to track.
PostCommit TransformService Direct - Last succeeded 4 months ago, filed #30238 to track.
PostCommit XVR Direct - Last succeeded 3 months ago, using #28972 to track.
PostCommit Java IO Performance Tests - Last succeed 5 months ago, using #28330 to track.
PostCommit Website Test - Can't interpret the results. It appears like it passed, but then there are failures I can't find. Will be tracked in the release blog PR. #30201
I'll be addressing the Website Test failures since I'd be nice to have a working website when we publish the release blog, and likely digging into the current issue with the GoUsingJava suite issues.
Absolute last call for cherry picks for RC2. I will be doing my final cleanup and then start the RC build in an hour or two.
Robert Burke
Beam 2.54.0 Release Manager
On 2024/02/05 19:42:32 Robert Burke wrote:
> I think that's serious enough to warrant another release candidate. However, please do continue validation so we can reduce iteration cycle time.
>
> Currently resolving [6] is the blocker for getting RC2 built, but if it's sorted out sooner, I'm putting a deadline of Noon PST on Feb 6th for any other small cherry picks. The following are ones that have been already requested, and will be cherry picked this afternoon.
>
> https://github.com/apache/beam/pull/30148 (resolves a caching error race condition on a non-volatile flag field in the non-portable Dataflow Java streaming harness. )
>
> https://github.com/apache/beam/pull/30156 (resolves an issue with long running python streaming jobs where an ID collision may occur).
>
> Please reply to this thread for other cherry picks. It's not a guarantee that the cherry pick will occur, but I can't evaluate it without knowing about it.
>
> Robert Burke
> Beam 2.54.0 Release Manager
>
>
>
>
>
> On 2024/02/05 03:31:36 Yi Hu via dev wrote:
> > Thanks for taking care of the release process! After validation two
> > breaking change was found
> > (1) Python Xlang Gcp Direct and Python Xlang Gcp Dataflow PostCommit tests
> > [1, 2]. It affects Python xlang BigQueryIO write (STORAGE_WRITE_API mode)
> > configuration. Filed [3] and pull request for cherry pick [4].
> > (2) Validation on Dataflow Template [5] found that OutputReceiver interface
> > requiring a new method (`outputWindowedValue`) to be implemented. Filed [6]
> > to determine if this is a release blocker or not, and I am working on a PR
> > for fix soon.
> >
> > That said I am -1 to this vote.
> >
> > [1]
> > https://github.com/apache/beam/actions/runs/7647377867/job/20838203146?pr=30104
> > [2]
> > https://github.com/apache/beam/actions/runs/7647377805/job/20985387650?pr=30104
> > [3] https://github.com/apache/beam/issues/30159
> > [4] https://github.com/apache/beam/pull/30189
> > [5]
> > https://github.com/GoogleCloudPlatform/DataflowTemplates/actions/runs/7762162776/job/21172087527
> > [6] https://github.com/apache/beam/issues/30203
> >
> > On Fri, Feb 2, 2024 at 6:01 PM XQ Hu via dev <de...@beam.apache.org> wrote:
> >
> > > +1 validated by running the simple RunInference ML pipeline:
> > > https://github.com/google/dataflow-ml-starter/actions/runs/7761835540/job/21171080332
> > >
> > > On Fri, Feb 2, 2024 at 4:10 PM Robert Burke <lo...@apache.org> wrote:
> > >
> > >> Hi everyone,
> > >> Please review and vote on the release candidate #1 for the version
> > >> 2.54.0, as follows:
> > >> [ ] +1, Approve the release
> > >> [ ] -1, Do not approve the release (please provide specific comments)
> > >>
> > >>
> > >> Reviewers are encouraged to test their own use cases with the release
> > >> candidate, and vote +1 if
> > >> no issues are found. Only PMC member votes will count towards the final
> > >> vote, but votes from all
> > >> community members is encouraged and helpful for finding regressions; you
> > >> can either test your own
> > >> use cases [13] or use cases from the validation sheet [10].
> > >>
> > >> The complete staging area is available for your review, which includes:
> > >> * GitHub Release notes [1],
> > >> * the official Apache source release to be deployed to dist.apache.org
> > >> [2], which is signed with the key with fingerprint D20316F712213422 [3],
> > >> * all artifacts to be deployed to the Maven Central Repository [4],
> > >> * source code tag "v2.54.0-RC1" [5],
> > >> * website pull request listing the release [6], the blog post [6], and
> > >> publishing the API reference manual [7].
> > >> * Python artifacts are deployed along with the source release to the
> > >> dist.apache.org [2] and PyPI[8].
> > >> * Go artifacts and documentation are available at pkg.go.dev [9]
> > >> * Validation sheet with a tab for 2.54.0 release to help with validation
> > >> [10].
> > >> * Docker images published to Docker Hub [11].
> > >> * PR to run tests against release branch [12].
> > >> * Legacy Dataflow Java Worker image has been published, so the failing
> > >> tests are being re-run.
> > >>
> > >> The vote will be open for at least 72 hours. It is adopted by majority
> > >> approval, with at least 3 PMC affirmative votes.
> > >>
> > >> For guidelines on how to try the release in your projects, check out our
> > >> RC testing guide [13].
> > >>
> > >> Thanks,
> > >> Robert Burke
> > >> Beam 2.54.0 Release Manager
> > >>
> > >> [1] https://github.com/apache/beam/milestone/18?closed=1
> > >> [2] https://dist.apache.org/repos/dist/dev/beam/2.54.0/
> > >> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> > >> [4]
> > >> https://repository.apache.org/content/repositories/orgapachebeam-1367/
> > >> [5] https://github.com/apache/beam/tree/v2.54.0-RC1
> > >> [6] https://github.com/apache/beam/pull/30201
> > >> [7] https://github.com/apache/beam-site/pull/658
> > >> [8] https://pypi.org/project/apache-beam/2.54.0rc1/
> > >> [9]
> > >> https://pkg.go.dev/github.com/apache/beam/sdks/v2@v2.54.0-RC1/go/pkg/beam
> > >> [10]
> > >> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=28763708
> > >> [11] https://hub.docker.com/search?q=apache%2Fbeam&type=image
> > >> [12] https://github.com/apache/beam/pull/30104
> > >> [13]
> > >> https://github.com/apache/beam/blob/master/contributor-docs/rc-testing-guide.md
> > >>
> > >
> >
>
Re: [VOTE] Release 2.54.0, release candidate #1
Posted by Robert Burke <lo...@apache.org>.
I think that's serious enough to warrant another release candidate. However, please do continue validation so we can reduce iteration cycle time.
Currently resolving [6] is the blocker for getting RC2 built, but if it's sorted out sooner, I'm putting a deadline of Noon PST on Feb 6th for any other small cherry picks. The following are ones that have been already requested, and will be cherry picked this afternoon.
https://github.com/apache/beam/pull/30148 (resolves a caching error race condition on a non-volatile flag field in the non-portable Dataflow Java streaming harness. )
https://github.com/apache/beam/pull/30156 (resolves an issue with long running python streaming jobs where an ID collision may occur).
Please reply to this thread for other cherry picks. It's not a guarantee that the cherry pick will occur, but I can't evaluate it without knowing about it.
Robert Burke
Beam 2.54.0 Release Manager
On 2024/02/05 03:31:36 Yi Hu via dev wrote:
> Thanks for taking care of the release process! After validation two
> breaking change was found
> (1) Python Xlang Gcp Direct and Python Xlang Gcp Dataflow PostCommit tests
> [1, 2]. It affects Python xlang BigQueryIO write (STORAGE_WRITE_API mode)
> configuration. Filed [3] and pull request for cherry pick [4].
> (2) Validation on Dataflow Template [5] found that OutputReceiver interface
> requiring a new method (`outputWindowedValue`) to be implemented. Filed [6]
> to determine if this is a release blocker or not, and I am working on a PR
> for fix soon.
>
> That said I am -1 to this vote.
>
> [1]
> https://github.com/apache/beam/actions/runs/7647377867/job/20838203146?pr=30104
> [2]
> https://github.com/apache/beam/actions/runs/7647377805/job/20985387650?pr=30104
> [3] https://github.com/apache/beam/issues/30159
> [4] https://github.com/apache/beam/pull/30189
> [5]
> https://github.com/GoogleCloudPlatform/DataflowTemplates/actions/runs/7762162776/job/21172087527
> [6] https://github.com/apache/beam/issues/30203
>
> On Fri, Feb 2, 2024 at 6:01 PM XQ Hu via dev <de...@beam.apache.org> wrote:
>
> > +1 validated by running the simple RunInference ML pipeline:
> > https://github.com/google/dataflow-ml-starter/actions/runs/7761835540/job/21171080332
> >
> > On Fri, Feb 2, 2024 at 4:10 PM Robert Burke <lo...@apache.org> wrote:
> >
> >> Hi everyone,
> >> Please review and vote on the release candidate #1 for the version
> >> 2.54.0, as follows:
> >> [ ] +1, Approve the release
> >> [ ] -1, Do not approve the release (please provide specific comments)
> >>
> >>
> >> Reviewers are encouraged to test their own use cases with the release
> >> candidate, and vote +1 if
> >> no issues are found. Only PMC member votes will count towards the final
> >> vote, but votes from all
> >> community members is encouraged and helpful for finding regressions; you
> >> can either test your own
> >> use cases [13] or use cases from the validation sheet [10].
> >>
> >> The complete staging area is available for your review, which includes:
> >> * GitHub Release notes [1],
> >> * the official Apache source release to be deployed to dist.apache.org
> >> [2], which is signed with the key with fingerprint D20316F712213422 [3],
> >> * all artifacts to be deployed to the Maven Central Repository [4],
> >> * source code tag "v2.54.0-RC1" [5],
> >> * website pull request listing the release [6], the blog post [6], and
> >> publishing the API reference manual [7].
> >> * Python artifacts are deployed along with the source release to the
> >> dist.apache.org [2] and PyPI[8].
> >> * Go artifacts and documentation are available at pkg.go.dev [9]
> >> * Validation sheet with a tab for 2.54.0 release to help with validation
> >> [10].
> >> * Docker images published to Docker Hub [11].
> >> * PR to run tests against release branch [12].
> >> * Legacy Dataflow Java Worker image has been published, so the failing
> >> tests are being re-run.
> >>
> >> The vote will be open for at least 72 hours. It is adopted by majority
> >> approval, with at least 3 PMC affirmative votes.
> >>
> >> For guidelines on how to try the release in your projects, check out our
> >> RC testing guide [13].
> >>
> >> Thanks,
> >> Robert Burke
> >> Beam 2.54.0 Release Manager
> >>
> >> [1] https://github.com/apache/beam/milestone/18?closed=1
> >> [2] https://dist.apache.org/repos/dist/dev/beam/2.54.0/
> >> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> >> [4]
> >> https://repository.apache.org/content/repositories/orgapachebeam-1367/
> >> [5] https://github.com/apache/beam/tree/v2.54.0-RC1
> >> [6] https://github.com/apache/beam/pull/30201
> >> [7] https://github.com/apache/beam-site/pull/658
> >> [8] https://pypi.org/project/apache-beam/2.54.0rc1/
> >> [9]
> >> https://pkg.go.dev/github.com/apache/beam/sdks/v2@v2.54.0-RC1/go/pkg/beam
> >> [10]
> >> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=28763708
> >> [11] https://hub.docker.com/search?q=apache%2Fbeam&type=image
> >> [12] https://github.com/apache/beam/pull/30104
> >> [13]
> >> https://github.com/apache/beam/blob/master/contributor-docs/rc-testing-guide.md
> >>
> >
>
Re: [VOTE] Release 2.54.0, release candidate #1
Posted by Yi Hu via dev <de...@beam.apache.org>.
Thanks for taking care of the release process! After validation two
breaking change was found
(1) Python Xlang Gcp Direct and Python Xlang Gcp Dataflow PostCommit tests
[1, 2]. It affects Python xlang BigQueryIO write (STORAGE_WRITE_API mode)
configuration. Filed [3] and pull request for cherry pick [4].
(2) Validation on Dataflow Template [5] found that OutputReceiver interface
requiring a new method (`outputWindowedValue`) to be implemented. Filed [6]
to determine if this is a release blocker or not, and I am working on a PR
for fix soon.
That said I am -1 to this vote.
[1]
https://github.com/apache/beam/actions/runs/7647377867/job/20838203146?pr=30104
[2]
https://github.com/apache/beam/actions/runs/7647377805/job/20985387650?pr=30104
[3] https://github.com/apache/beam/issues/30159
[4] https://github.com/apache/beam/pull/30189
[5]
https://github.com/GoogleCloudPlatform/DataflowTemplates/actions/runs/7762162776/job/21172087527
[6] https://github.com/apache/beam/issues/30203
On Fri, Feb 2, 2024 at 6:01 PM XQ Hu via dev <de...@beam.apache.org> wrote:
> +1 validated by running the simple RunInference ML pipeline:
> https://github.com/google/dataflow-ml-starter/actions/runs/7761835540/job/21171080332
>
> On Fri, Feb 2, 2024 at 4:10 PM Robert Burke <lo...@apache.org> wrote:
>
>> Hi everyone,
>> Please review and vote on the release candidate #1 for the version
>> 2.54.0, as follows:
>> [ ] +1, Approve the release
>> [ ] -1, Do not approve the release (please provide specific comments)
>>
>>
>> Reviewers are encouraged to test their own use cases with the release
>> candidate, and vote +1 if
>> no issues are found. Only PMC member votes will count towards the final
>> vote, but votes from all
>> community members is encouraged and helpful for finding regressions; you
>> can either test your own
>> use cases [13] or use cases from the validation sheet [10].
>>
>> The complete staging area is available for your review, which includes:
>> * GitHub Release notes [1],
>> * the official Apache source release to be deployed to dist.apache.org
>> [2], which is signed with the key with fingerprint D20316F712213422 [3],
>> * all artifacts to be deployed to the Maven Central Repository [4],
>> * source code tag "v2.54.0-RC1" [5],
>> * website pull request listing the release [6], the blog post [6], and
>> publishing the API reference manual [7].
>> * Python artifacts are deployed along with the source release to the
>> dist.apache.org [2] and PyPI[8].
>> * Go artifacts and documentation are available at pkg.go.dev [9]
>> * Validation sheet with a tab for 2.54.0 release to help with validation
>> [10].
>> * Docker images published to Docker Hub [11].
>> * PR to run tests against release branch [12].
>> * Legacy Dataflow Java Worker image has been published, so the failing
>> tests are being re-run.
>>
>> The vote will be open for at least 72 hours. It is adopted by majority
>> approval, with at least 3 PMC affirmative votes.
>>
>> For guidelines on how to try the release in your projects, check out our
>> RC testing guide [13].
>>
>> Thanks,
>> Robert Burke
>> Beam 2.54.0 Release Manager
>>
>> [1] https://github.com/apache/beam/milestone/18?closed=1
>> [2] https://dist.apache.org/repos/dist/dev/beam/2.54.0/
>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>> [4]
>> https://repository.apache.org/content/repositories/orgapachebeam-1367/
>> [5] https://github.com/apache/beam/tree/v2.54.0-RC1
>> [6] https://github.com/apache/beam/pull/30201
>> [7] https://github.com/apache/beam-site/pull/658
>> [8] https://pypi.org/project/apache-beam/2.54.0rc1/
>> [9]
>> https://pkg.go.dev/github.com/apache/beam/sdks/v2@v2.54.0-RC1/go/pkg/beam
>> [10]
>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=28763708
>> [11] https://hub.docker.com/search?q=apache%2Fbeam&type=image
>> [12] https://github.com/apache/beam/pull/30104
>> [13]
>> https://github.com/apache/beam/blob/master/contributor-docs/rc-testing-guide.md
>>
>
Re: [VOTE] Release 2.54.0, release candidate #1
Posted by XQ Hu via dev <de...@beam.apache.org>.
+1 validated by running the simple RunInference ML pipeline:
https://github.com/google/dataflow-ml-starter/actions/runs/7761835540/job/21171080332
On Fri, Feb 2, 2024 at 4:10 PM Robert Burke <lo...@apache.org> wrote:
> Hi everyone,
> Please review and vote on the release candidate #1 for the version 2.54.0,
> as follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
>
> Reviewers are encouraged to test their own use cases with the release
> candidate, and vote +1 if
> no issues are found. Only PMC member votes will count towards the final
> vote, but votes from all
> community members is encouraged and helpful for finding regressions; you
> can either test your own
> use cases [13] or use cases from the validation sheet [10].
>
> The complete staging area is available for your review, which includes:
> * GitHub Release notes [1],
> * the official Apache source release to be deployed to dist.apache.org
> [2], which is signed with the key with fingerprint D20316F712213422 [3],
> * all artifacts to be deployed to the Maven Central Repository [4],
> * source code tag "v2.54.0-RC1" [5],
> * website pull request listing the release [6], the blog post [6], and
> publishing the API reference manual [7].
> * Python artifacts are deployed along with the source release to the
> dist.apache.org [2] and PyPI[8].
> * Go artifacts and documentation are available at pkg.go.dev [9]
> * Validation sheet with a tab for 2.54.0 release to help with validation
> [10].
> * Docker images published to Docker Hub [11].
> * PR to run tests against release branch [12].
> * Legacy Dataflow Java Worker image has been published, so the failing
> tests are being re-run.
>
> The vote will be open for at least 72 hours. It is adopted by majority
> approval, with at least 3 PMC affirmative votes.
>
> For guidelines on how to try the release in your projects, check out our
> RC testing guide [13].
>
> Thanks,
> Robert Burke
> Beam 2.54.0 Release Manager
>
> [1] https://github.com/apache/beam/milestone/18?closed=1
> [2] https://dist.apache.org/repos/dist/dev/beam/2.54.0/
> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> [4] https://repository.apache.org/content/repositories/orgapachebeam-1367/
> [5] https://github.com/apache/beam/tree/v2.54.0-RC1
> [6] https://github.com/apache/beam/pull/30201
> [7] https://github.com/apache/beam-site/pull/658
> [8] https://pypi.org/project/apache-beam/2.54.0rc1/
> [9]
> https://pkg.go.dev/github.com/apache/beam/sdks/v2@v2.54.0-RC1/go/pkg/beam
> [10]
> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=28763708
> [11] https://hub.docker.com/search?q=apache%2Fbeam&type=image
> [12] https://github.com/apache/beam/pull/30104
> [13]
> https://github.com/apache/beam/blob/master/contributor-docs/rc-testing-guide.md
>