Re: [VOTE] Release 2.37.0, release candidate #2

[Niel Markwick <ni...@google.com>]

Sorry for the late reply.

The severity is that any errors while reading data (which get ignored and
not even reported) would mean that only parts of the data is read, but the
pipeline succeeds anyway.
You could imagine a data export, either as a backup or as a data transfer
to an analytics tool like BigQuery where in fact only a partial export
occurs, so the resulting dataset is invalid, but the user would never know.

Thanks for performing the CP.

-- 
<https://cloud.google.com>
* •  **Niel Markwick*
* •  *Cloud Solutions Architect <https://cloud.google.com/docs/tutorials>
* •  *Google Belgium
* •  *nielm@google.com
* •  *+32 2 894 6771

Google Belgium NV/SA, Steenweg op Etterbeek 180, 1040 Brussel, Belgie.
RPR: 0878.065.378

If you have received this communication by mistake, please don't forward it
to anyone else (it may contain confidential or privileged information),
please erase all copies of it, including all attachments, and please let
the sender know it went to the wrong person. Thanks


On Mon, 28 Feb 2022 at 19:02, Brian Hulette <bh...@google.com> wrote:

> Thanks everyone!
>
> Niel, I'm still trying to decide if BEAM-14005 is worth cherrypicking.
> Could you help clarify the severity? I'm curious if there's any possibility
> it will cause data loss.
> It looks like it *might* be possible, e.g. if a user runs a Beam
> pipeline to migrate some data from Spanner and uses Pipeline
> success/failure to decide whether or not to delete the original data. Even
> that is a bit of a gray area though, I'm not sure if we'd consider that a
> "data loss bug".
>
> Brian
>
> On Mon, Feb 28, 2022 at 8:28 AM Valentyn Tymofieiev <va...@google.com>
> wrote:
>
>> I have verified Dataflow containers and ran a few pipelines on Python
>> 3.9.
>>
>> I noticed that we didn't have Python 3.9 wheels. If we have another RC,
>> I'd like to include this change in the release branch:
>> https://github.com/apache/beam/pull/16964.
>>
>> If we don't build another RC, I will still vote +1 on this one since
>> Python 3.9 is a new functionality.
>>
>>
>> On Sat, Feb 26, 2022 at 11:50 AM Chamikara Jayalath <ch...@google.com>
>> wrote:
>>
>>> Agree. I think this is a critical bug so we should consider
>>> cherry-picking.
>>>
>>> - Cham
>>>
>>> On Sat, Feb 26, 2022 at 3:34 AM Niel Markwick <ni...@google.com> wrote:
>>>
>>>> > I have just found a regression from 2.34
>>>>
>>>> Meaning it was introduced in *v2.34...*
>>>>
>>>> On Sat, 26 Feb 2022, 12:31 Niel Markwick, <ni...@google.com> wrote:
>>>>
>>>>> I have just found a regression from 2.34
>>>>>
>>>>> https://issues.apache.org/jira/browse/BEAM-14005
>>>>>
>>>>> I will have a PR ready today if you want to include it in 2.37
>>>>> (depending on how far down the release process you are)
>>>>>
>>>>> The regression case is quite specific, but has caused a customer issue
>>>>> - if reading from Spanner fails, the pipeline will continue regardless with
>>>>> zero elements. (Normally it should abort with an exception).
>>>>>
>>>>> On Wed, 23 Feb 2022, 18:37 Brian Hulette, <bh...@google.com> wrote:
>>>>>
>>>>>> Hi everyone,
>>>>>> Please review and vote on release candidate #2 version 2.37.0, as
>>>>>> follows:
>>>>>> [ ] +1, Approve the release
>>>>>> [ ] -1, Do not approve the release (please provide specific comments)
>>>>>>
>>>>>> Reviewers are encouraged to test their own use cases with the release
>>>>>> candidate, and vote +1 if no issues are found.
>>>>>>
>>>>>> The complete staging area is available for your review, which
>>>>>> includes:
>>>>>> * JIRA release notes [1],
>>>>>> * the official Apache source release to be deployed to
>>>>>> dist.apache.org [2], which is signed with the key with fingerprint
>>>>>> 1D190C85 [3],
>>>>>> * all artifacts to be deployed to the Maven Central Repository [4],
>>>>>> * source code tag "v2.37.0-RC2" [5],
>>>>>> * website pull request listing the release [6], the blog post [6],
>>>>>> and publishing the API reference manual [7].
>>>>>> * Java artifacts were built with Gradle 7.3.2 and OpenJDK 8u232.
>>>>>> * Python artifacts are deployed along with the source release to the
>>>>>> dist.apache.org [2] and PyPI[8].
>>>>>> * Validation sheet with a tab for 2.37.0 release to help with
>>>>>> validation [9].
>>>>>> * Docker images published to Docker Hub [10].
>>>>>>
>>>>>> For guidelines on how to try the release in your projects, check out
>>>>>> our blog post at https://beam.apache.org/blog/validate-beam-release/.
>>>>>>
>>>>>> Thanks,
>>>>>> Brian
>>>>>>
>>>>>> [1]
>>>>>> https://jira.apache.org/jira/secure/ReleaseNote.jspa?projectId=projectId=12319527&version=12351168
>>>>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.37.0/
>>>>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>>>>> [4]
>>>>>> https://repository.apache.org/content/repositories/orgapachebeam-1253/
>>>>>> [5] https://github.com/apache/beam/tree/v2.37.0-RC2
>>>>>> [6] https://github.com/apache/beam/pull/16887
>>>>>> [7] https://github.com/apache/beam-site/pull/623
>>>>>> [8] https://pypi.org/project/apache-beam/2.37.0rc2/
>>>>>> [9]
>>>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1090588300
>>>>>> [10] https://hub.docker.com/search?q=apache%2Fbeam&type=image
>>>>>>
>>>>>