[VOTE] Release 2.33.0, release candidate 1

[Udi Meiri <ud...@apache.org>]

Hi everyone,
Please review and vote on the release candidate #1 for the version 2.33.0,
as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)


Reviewers are encouraged to test their own use cases with the release
candidate, and vote +1 if
no issues are found.

The complete staging area is available for your review, which includes:
* JIRA release notes [1],
* the official Apache source release to be deployed to dist.apache.org [2],
which is signed with the key with fingerprint 587B049C36DAAFE6 [3],
* all artifacts to be deployed to the Maven Central Repository [4],
* source code tag "v2.33.0-RC1" [5],
* website pull request listing the release [6], the blog post [6], and
publishing the API reference manual [7].
* Java artifacts were built with Maven 3.6.3 and OpenJDK 1.8.0_181.
* Python artifacts are deployed along with the source release to the
dist.apache.org [2] and pypy[8].
* Validation sheet with a tab for 2.33.0 release to help with validation
[9].
* Docker images published to Docker Hub [10].

The vote will be open for at least 72 hours. It is adopted by majority
approval, with at least 3 PMC affirmative votes.

For guidelines on how to try the release in your projects, check out our
blog post at https://beam.apache.org/blog/validate-beam-release/.

Thanks,
Release Manager

[1]
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
[2] https://dist.apache.org/repos/dist/dev/beam/2.33.0/
[3] https://dist.apache.org/repos/dist/release/beam/KEYS
[4] https://repository.apache.org/content/repositories/orgapachebeam-1234/
[5] https://github.com/apache/beam/tree/v2.33.0-RC1
[6] https://github.com/apache/beam/pull/15543
[7] https://github.com/apache/beam-site/pull/619
[8] https://pypi.org/project/apache-beam/2.33.0rc1/
[9]
https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
[10] https://hub.docker.com/search?q=apache%2Fbeam&type=image

Re: [VOTE] Release 2.33.0, release candidate 1

[Ahmet Altay <al...@google.com>]

Udi, how is the RC2 progress going? Did we cherry pick all the identified
issues in this thread? Are there any blockers we can help with?

Thank you for working on the release!

Ahmet

On Mon, Sep 27, 2021, 6:26 PM Zachary Houfek <zh...@google.com> wrote:

> Sure, I can do that, though it doesn't look like I can just revert the PR
> from Github. I'll need to manually remove the changes from that commit.
>
> Regards,
> Zach
>
> On Mon, Sep 27, 2021 at 9:02 PM Udi Meiri <eh...@google.com> wrote:
>
>> Zachary could you prepare a rollback PR for the 2.33.0 branch?
>>
>> On Mon, Sep 27, 2021 at 5:25 PM Zachary Houfek <zh...@google.com>
>> wrote:
>>
>>> > AfterProcessingTime and AfterCount should both be flagged...AfterEach
>>> is different, since if any of the sequenced triggers never finish, the
>>> whole thing will never finish
>>>
>>> Ah, looks like another necessary fix. AfterEach will be flagged if any
>>> should be flagged. That and AfterProcessingTime should be ready soon.
>>>
>>> Regards,
>>> Zach
>>>
>>> On Mon, Sep 27, 2021 at 7:36 PM Robert Bradshaw <ro...@google.com>
>>> wrote:
>>>
>>>> OK, given the fact that AfterWatermark(...) implicitly invokes
>>>> Repeatedly as well, and the sheer volume of concern/discussion on this
>>>> issue, I now change my vote to a -1 and think it's best we just cut a new
>>>> release with the fix in it.
>>>>
>>>>
>>>> On Mon, Sep 27, 2021 at 4:06 PM Kenneth Knowles <ke...@apache.org>
>>>> wrote:
>>>>
>>>>> Ah, thanks for the correction. AfterProcessingTime and AfterCount
>>>>> should both be flagged, as should AfterEvery, and AfterAny. AfterEach is
>>>>> different, since if any of the sequenced triggers never finish, the whole
>>>>> thing will never finish.
>>>>>
>>>>> Kenn
>>>>>
>>>>> On Mon, Sep 27, 2021 at 3:51 PM Zachary Houfek <zh...@google.com>
>>>>> wrote:
>>>>>
>>>>>> > This was discovered because Repeatedly(AfterProcessingTime(n))
>>>>>>
>>>>>> Wasn't it Repeatedly(AfterCount(n)) that caused the problem? At least
>>>>>> that's what caused the failing test that prompted the fix.
>>>>>>
>>>>>> With that said, is the concern that AfterProcessingTime *should* include
>>>>>> the CONDITION_NOT_GUARANTEED flag? It isn't right now, so I'm checking.
>>>>>>
>>>>>> Another thing I haven't seen mentioned: `Repeatedly` is or can be
>>>>>> used in other triggers like AfterWatermark (because of the late trigger),
>>>>>> AfterEach, AfterEvery, and AfterAny, so the bug will propagate to those
>>>>>> triggers as well. Two of these actually had to have their tests
>>>>>> updated in the fix
>>>>>> <https://github.com/apache/beam/commit/533a63299789ec8b4bbe629c95734be8dd4d9589#diff-9278a08a4f97eb63d783206bdea1a05377b9edd41646255110446512b8adad43>
>>>>>> .
>>>>>>
>>>>>> Regards,
>>>>>> Zach
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Sep 27, 2021 at 6:30 PM Robert Bradshaw <ro...@google.com>
>>>>>> wrote:
>>>>>>
>>>>>>> $ git checkout release-2.33.0
>>>>>>> Switched to branch 'release-2.33.0'
>>>>>>> Your branch is up to date with 'github/release-2.33.0'.
>>>>>>>
>>>>>>> $ python -c 'import apache_beam as beam;
>>>>>>>
>>>>>>> print(beam.transforms.trigger.Repeatedly(beam.transforms.trigger.AfterProcessingTime(5)).may_lose_data(None))'
>>>>>>> DataLossReason.NO_POTENTIAL_LOSS
>>>>>>>
>>>>>>> The MAY_FINISH and CONDITION_NOT_GUARANTEED flags are disjoint.
>>>>>>>
>>>>>>> On Mon, Sep 27, 2021 at 3:15 PM Kenneth Knowles <ke...@apache.org>
>>>>>>> wrote:
>>>>>>> >
>>>>>>> > This was discovered because Repeatedly(AfterProcessingTime(n))
>>>>>>> does require the workaround, because AfterProcessingTime(n) does terminate
>>>>>>> and that is propagated by the Repeatedly implementation.
>>>>>>> >
>>>>>>> > Kenn
>>>>>>> >
>>>>>>> > On Mon, Sep 27, 2021 at 2:49 PM Robert Bradshaw <
>>>>>>> robertwb@google.com> wrote:
>>>>>>> >>
>>>>>>> >> On Mon, Sep 27, 2021 at 2:42 PM Kenneth Knowles <ke...@apache.org>
>>>>>>> wrote:
>>>>>>> >> >
>>>>>>> >> > True. It is the use case of Repeatedly(ProcessingTime(n)) that
>>>>>>> I would guess to be the primary case of concern.
>>>>>>> >>
>>>>>>> >> This shouldn't require the workaround, right?
>>>>>>> >>
>>>>>>> >> If you're not comfortable with this, I'd say let's patch and
>>>>>>> re-cut.
>>>>>>> >>
>>>>>>> >>
>>>>>>> >> > On Mon, Sep 27, 2021 at 2:38 PM Robert Bradshaw <
>>>>>>> robertwb@google.com> wrote:
>>>>>>> >> >>
>>>>>>> >> >> As I read the code, it's only pipelines that use a Repeatedly
>>>>>>> trigger
>>>>>>> >> >> that wrap an already lossy trigger that are declared to be
>>>>>>> themselves
>>>>>>> >> >> lossy. If I'm mistaken, I'll certainly reconsider my vote (and
>>>>>>> thanks
>>>>>>> >> >> for bringing this up).
>>>>>>> >> >>
>>>>>>> >> >> On Mon, Sep 27, 2021 at 2:21 PM Kenneth Knowles <
>>>>>>> kenn@apache.org> wrote:
>>>>>>> >> >> >
>>>>>>> >> >> > My concern is that the error message is incorrect and every
>>>>>>> user of 2.33.0 may be educated wrong, or be worried about data loss in
>>>>>>> Beam, or fail to find the blog post or CHANGES, etc.
>>>>>>> >> >> >
>>>>>>> >> >> > Kenn
>>>>>>> >> >> >
>>>>>>> >> >> > On Mon, Sep 27, 2021 at 2:16 PM Udi Meiri <eh...@google.com>
>>>>>>> wrote:
>>>>>>> >> >> >>
>>>>>>> >> >> >> I don't know how rare it is, but there is a flag documented
>>>>>>> in CHANGES and blog post that reverts to the old behavior.
>>>>>>> >> >> >>
>>>>>>> >> >> >> On Mon, Sep 27, 2021 at 2:12 PM Kenneth Knowles <
>>>>>>> kenn@apache.org> wrote:
>>>>>>> >> >> >>>
>>>>>>> >> >> >>> I guess my vote is -0 since I don't have enough context on
>>>>>>> this issue. A number of people with more awareness of how severe this is
>>>>>>> have voted +1 so I will not try to block the release.
>>>>>>> >> >> >>>
>>>>>>> >> >> >>> Kenn
>>>>>>> >> >> >>>
>>>>>>> >> >> >>> On Mon, Sep 27, 2021 at 2:11 PM Kenneth Knowles <
>>>>>>> kenn@apache.org> wrote:
>>>>>>> >> >> >>>>
>>>>>>> >> >> >>>> I have to disagree with the other PMC members here, or at
>>>>>>> least dig in to the question: every pipeline that uses a Repeatedly trigger
>>>>>>> at the top level will be rejected. Is this so rare in Python that it is OK?
>>>>>>> >> >> >>>>
>>>>>>> >> >> >>>> Kenn
>>>>>>> >> >> >>>>
>>>>>>> >> >> >>>> On Mon, Sep 27, 2021 at 1:56 PM Robert Burke <
>>>>>>> rebo@google.com> wrote:
>>>>>>> >> >> >>>>>
>>>>>>> >> >> >>>>> On it. Thanks!
>>>>>>> >> >> >>>>>
>>>>>>> >> >> >>>>> On Mon, Sep 27, 2021 at 1:18 PM Udi Meiri <
>>>>>>> ehudm@google.com> wrote:
>>>>>>> >> >> >>>>>>
>>>>>>> >> >> >>>>>> Daniel/Robert, feel free to make changes to this PR:
>>>>>>> https://github.com/apache/beam/pull/15543
>>>>>>> >> >> >>>>>>
>>>>>>> >> >> >>>>>> On Mon, Sep 27, 2021 at 12:08 PM Daniel Oliveira <
>>>>>>> danoliveira@google.com> wrote:
>>>>>>> >> >> >>>>>>>
>>>>>>> >> >> >>>>>>> +1
>>>>>>> >> >> >>>>>>>
>>>>>>> >> >> >>>>>>> I hadn't realized the pipelines were still finishing
>>>>>>> successfully. I retried wordcount with that in mind and confirmed it
>>>>>>> finishes successfully, so this isn't a blocker.
>>>>>>> >> >> >>>>>>>
>>>>>>> >> >> >>>>>>> Although maybe we should add this to the "Known
>>>>>>> Issues" because I can easily see those messages being interpreted as a
>>>>>>> pipeline failure.
>>>>>>> >> >> >>>>>>>
>>>>>>> >> >> >>>>>>> On Sun, Sep 26, 2021 at 7:26 PM Robert Burke <
>>>>>>> robert@frantil.com> wrote:
>>>>>>> >> >> >>>>>>>>
>>>>>>> >> >> >>>>>>>> That's on the SDK side, and it just means the
>>>>>>> PCollection metrics are being returned, buy not handled by the SDK. At
>>>>>>> present the pipeline results only handle PTransform metrics.
>>>>>>> >> >> >>>>>>>>
>>>>>>> >> >> >>>>>>>> As such it's not a regression, as adding those is
>>>>>>> still under development.
>>>>>>> >> >> >>>>>>>>
>>>>>>> >> >> >>>>>>>> On Sun, Sep 26, 2021, 7:02 PM Daniel Oliveira <
>>>>>>> danoliveira@google.com> wrote:
>>>>>>> >> >> >>>>>>>>>
>>>>>>> >> >> >>>>>>>>> I tried validating wordcount with the Go SDK on
>>>>>>> Flink. The pipeline failed with a wall of errors like the following. I
>>>>>>> tried this on Flink 1.11, 1.12, and 1.13 job servers built from source at
>>>>>>> the RC1 commit, same errors on all of them.
>>>>>>> >> >> >>>>>>>>>
>>>>>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>>>>>> MonitoringInfo: urn:"beam:metric:element_count:v1"
>>>>>>> type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"
>>>>>>> labels:{key:"PCOLLECTION"  value:"n9"}
>>>>>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>>>>>> MonitoringInfo: urn:"beam:metric:element_count:v1"
>>>>>>> type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"
>>>>>>> labels:{key:"PCOLLECTION"  value:"n10"}
>>>>>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>>>>>> MonitoringInfo: urn:"beam:metric:element_count:v1"
>>>>>>> type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"
>>>>>>> labels:{key:"PCOLLECTION"  value:"n8"}
>>>>>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>>>>>> MonitoringInfo: urn:"beam:metric:element_count:v1"
>>>>>>> type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"
>>>>>>> labels:{key:"PCOLLECTION"  value:"n7"}
>>>>>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>>>>>> MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"
>>>>>>> type:"beam:metrics:distribution_int64:v1"
>>>>>>> payload:"\x8d%\xfa\xbf\x05\x04\xa8\x0c"  labels:{key:"PCOLLECTION"
>>>>>>> value:"n7"}
>>>>>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>>>>>> MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"
>>>>>>> type:"beam:metrics:distribution_int64:v1"  payload:"\xb9\x05\xcf6\x05\x11"
>>>>>>> labels:{key:"PCOLLECTION"  value:"n9"}
>>>>>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>>>>>> MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"
>>>>>>> type:"beam:metrics:distribution_int64:v1"  payload:"\xc5\x05\x8a1\x04\x12"
>>>>>>> labels:{key:"PCOLLECTION"  value:"n8"}
>>>>>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>>>>>> MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"
>>>>>>> type:"beam:metrics:distribution_int64:v1"
>>>>>>> payload:"\x8d%\xb3\xa7\x07\x14\""  labels:{key:"PCOLLECTION"  value:"n10"}
>>>>>>> >> >> >>>>>>>>>> {...}
>>>>>>> >> >> >>>>>>>>>
>>>>>>> >> >> >>>>>>>>>
>>>>>>> >> >> >>>>>>>>>  @Robert Burke I think you might know what's going
>>>>>>> on here. Is this solvable with a cherry-pick and a new RC?
>>>>>>> >> >> >>>>>>>>>
>>>>>>> >> >> >>>>>>>>> On Fri, Sep 24, 2021 at 4:25 PM Ahmet Altay <
>>>>>>> altay@google.com> wrote:
>>>>>>> >> >> >>>>>>>>>>
>>>>>>> >> >> >>>>>>>>>>
>>>>>>> >> >> >>>>>>>>>>
>>>>>>> >> >> >>>>>>>>>> On Fri, Sep 24, 2021 at 4:21 PM Udi Meiri <
>>>>>>> ehudm@google.com> wrote:
>>>>>>> >> >> >>>>>>>>>>>
>>>>>>> >> >> >>>>>>>>>>>
>>>>>>> >> >> >>>>>>>>>>>
>>>>>>> >> >> >>>>>>>>>>> On Fri, Sep 24, 2021 at 4:18 PM Ahmet Altay <
>>>>>>> altay@google.com> wrote:
>>>>>>> >> >> >>>>>>>>>>>>
>>>>>>> >> >> >>>>>>>>>>>> Thank you for the update related to
>>>>>>> allow_unsafe_triggers. My vote is still a +1.
>>>>>>> >> >> >>>>>>>>>>>>
>>>>>>> >> >> >>>>>>>>>>>> If we decide to move forward with this RC, could
>>>>>>> you please include this bug in the known issues list under the changes.md
>>>>>>> for this release?
>>>>>>> >> >> >>>>>>>>>>>
>>>>>>> >> >> >>>>>>>>>>> Yes, that's included in
>>>>>>> https://github.com/apache/beam/pull/15543/files.
>>>>>>> >> >> >>>>>>>>>>
>>>>>>> >> >> >>>>>>>>>>
>>>>>>> >> >> >>>>>>>>>> Thank you!
>>>>>>> >> >> >>>>>>>>>>
>>>>>>> >> >> >>>>>>>>>>>
>>>>>>> >> >> >>>>>>>>>>>
>>>>>>> >> >> >>>>>>>>>>>>
>>>>>>> >> >> >>>>>>>>>>>>
>>>>>>> >> >> >>>>>>>>>>>> On Fri, Sep 24, 2021 at 3:56 PM Chamikara
>>>>>>> Jayalath <ch...@google.com> wrote:
>>>>>>> >> >> >>>>>>>>>>>>>
>>>>>>> >> >> >>>>>>>>>>>>> +1 (binding)
>>>>>>> >> >> >>>>>>>>>>>>> Validated a few scenarios from the spreadsheet.
>>>>>>> >> >> >>>>>>>>>>>>>
>>>>>>> >> >> >>>>>>>>>>>>> Thanks,
>>>>>>> >> >> >>>>>>>>>>>>> Cham
>>>>>>> >> >> >>>>>>>>>>>>>
>>>>>>> >> >> >>>>>>>>>>>>> On Fri, Sep 24, 2021 at 3:07 PM Robert Bradshaw <
>>>>>>> robertwb@google.com> wrote:
>>>>>>> >> >> >>>>>>>>>>>>>>
>>>>>>> >> >> >>>>>>>>>>>>>> +1 (binding)
>>>>>>> >> >> >>>>>>>>>>>>>>
>>>>>>> >> >> >>>>>>>>>>>>>> All the artifacts and signatures look good.
>>>>>>> >> >> >>>>>>>>>>>>>>
>>>>>>> >> >> >>>>>>>>>>>>>> I don't think the unsafe trigger check is
>>>>>>> severe enough to block the release.
>>>>>>> >> >> >>>>>>>>>>>>>>
>>>>>>> >> >> >>>>>>>>>>>>>> On Fri, Sep 24, 2021 at 2:36 PM Udi Meiri <
>>>>>>> ehudm@google.com> wrote:
>>>>>>> >> >> >>>>>>>>>>>>>> >
>>>>>>> >> >> >>>>>>>>>>>>>> > Hi everyone, we found a bug during testing.
>>>>>>> It has to do with Python SDK's allow_unsafe_triggers check.
>>>>>>> >> >> >>>>>>>>>>>>>> > There is a preliminary fix that will go to
>>>>>>> master.
>>>>>>> >> >> >>>>>>>>>>>>>> >
>>>>>>> >> >> >>>>>>>>>>>>>> > For the 2.33.0 release, I'm leaning towards
>>>>>>> not making a new RC since there is a workaround: pass the flag
>>>>>>> --allow_unsafe_triggers.
>>>>>>> >> >> >>>>>>>>>>>>>> >
>>>>>>> >> >> >>>>>>>>>>>>>> > Please reevaluate your votes accordingly and
>>>>>>> recast if you've changed your vote.
>>>>>>> >> >> >>>>>>>>>>>>>> >
>>>>>>> >> >> >>>>>>>>>>>>>> > Thanks
>>>>>>> >> >> >>>>>>>>>>>>>> >
>>>>>>> >> >> >>>>>>>>>>>>>> > On Fri, Sep 24, 2021 at 12:48 PM Alexey
>>>>>>> Romanenko <ar...@gmail.com> wrote:
>>>>>>> >> >> >>>>>>>>>>>>>> >>
>>>>>>> >> >> >>>>>>>>>>>>>> >>
>>>>>>> >> >> >>>>>>>>>>>>>> >> On 24 Sep 2021, at 20:45, Udi Meiri <
>>>>>>> ehudm@google.com> wrote:
>>>>>>> >> >> >>>>>>>>>>>>>> >> Alexey is this something that we should put
>>>>>>> in the release notes, or some other change?
>>>>>>> >> >> >>>>>>>>>>>>>> >>
>>>>>>> >> >> >>>>>>>>>>>>>> >>
>>>>>>> >> >> >>>>>>>>>>>>>> >> Yes, I think it could be helpful to mention
>>>>>>> that Beam Jackson’s deps was bumped and it may require an update of
>>>>>>> Jackson’s runtime deps for Spark 2 users as well.
>>>>>>> >> >> >>>>>>>>>>>>>> >>
>>>>>>> >> >> >>>>>>>>>>>>>> >> —
>>>>>>> >> >> >>>>>>>>>>>>>> >> Alexey
>>>>>>> >> >> >>>>>>>>>>>>>> >>
>>>>>>> >> >> >>>>>>>>>>>>>> >>
>>>>>>> >> >> >>>>>>>>>>>>>> >>
>>>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>>>> >> >> >>>>>>>>>>>>>> >>> —
>>>>>>> >> >> >>>>>>>>>>>>>> >>> Alexey
>>>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>>>> >> >> >>>>>>>>>>>>>> >>> On 24 Sep 2021, at 16:17, Alexey Romanenko <
>>>>>>> aromanenko.dev@gmail.com> wrote:
>>>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>>>> >> >> >>>>>>>>>>>>>> >>> I checked with beam-samples [1] and noticed
>>>>>>> an issue to run some pipelines with Spark 2 runner (Spark 3 seems is ok).
>>>>>>> >> >> >>>>>>>>>>>>>> >>> It looks that it’s caused by new Jackson's
>>>>>>> version updated recently [2], even if it’s a minor update but it works fine
>>>>>>> with a previous one.
>>>>>>> >> >> >>>>>>>>>>>>>> >>> I’ll try to find a workaround and get back
>>>>>>> with a results of this.
>>>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>>>> >> >> >>>>>>>>>>>>>> >>> [1] https://github.com/Talend/beam-samples/
>>>>>>> >> >> >>>>>>>>>>>>>> >>> [2]
>>>>>>> https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c
>>>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>>>> >> >> >>>>>>>>>>>>>> >>> —
>>>>>>> >> >> >>>>>>>>>>>>>> >>> Alexey
>>>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>>>> >> >> >>>>>>>>>>>>>> >>> On 24 Sep 2021, at 11:17, Jan Lukavský <
>>>>>>> je.ik@seznam.cz> wrote:
>>>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>>>> >> >> >>>>>>>>>>>>>> >>> +1 (non-binding)
>>>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>>>> >> >> >>>>>>>>>>>>>> >>> Validated several use-cases using
>>>>>>> non-portable Flink with Java SDK.
>>>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>>>> >> >> >>>>>>>>>>>>>> >>> On 9/24/21 4:55 AM, Valentyn Tymofieiev
>>>>>>> wrote:
>>>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>>>> >> >> >>>>>>>>>>>>>> >>> +1. Ran several Python batch and streaming
>>>>>>> pipelines on Dataflow and checked that Dataflow containers have required
>>>>>>> dependencies of Beam.
>>>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>>>> >> >> >>>>>>>>>>>>>> >>> On Thu, Sep 23, 2021 at 7:03 PM Robert
>>>>>>> Burke <lo...@apache.org> wrote:
>>>>>>> >> >> >>>>>>>>>>>>>> >>>>
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> +1 (non-binding)
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> I validated the Go Quickstart (wordcount),
>>>>>>> and my ray tracer against the Go Direct runner, Dataflow, and Spark
>>>>>>> (ensuring the rc1 tagged container was used) and they executed successfully.
>>>>>>> >> >> >>>>>>>>>>>>>> >>>>
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> I needed to manually synthesize a
>>>>>>> pseudo-version to ensure I was using the tagged branch version
>>>>>>> (v2.0.0-20210914211513-b358127f9859) instead of simply using v2.33.0-RC1.
>>>>>>> >> >> >>>>>>>>>>>>>> >>>>
>>>>>>> >> >> >>>>>>>>>>>>>> >>>>  It either can't find the package with the
>>>>>>> right tagged version, or it can't find the version. It's not clear to me
>>>>>>> what the issue is, but it's not notionally a release blocker. I'll
>>>>>>> investigate further once we have a full release, as it's probably some
>>>>>>> unspecified behavior due to how we transitioned to Go Modules (which
>>>>>>> strongly recommended doing a major version bump for such transitions, which
>>>>>>> seems a bit excessive for Beam as a whole...).
>>>>>>> >> >> >>>>>>>>>>>>>> >>>>
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> On 2021/09/23 03:59:18, Ahmet Altay <
>>>>>>> altay@google.com> wrote:
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > +1 on the RC. I validated python quick
>>>>>>> start examples on direct runners.
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > Thank you Udi.
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> >
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > On Wed, Sep 22, 2021 at 2:20 PM Robert
>>>>>>> Burke <ro...@frantil.com> wrote:
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> >
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > > Just an FYI that intend to validate
>>>>>>> the Go SDK for this release but can't
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > > get to it until tomorrow (Thursday).
>>>>>>> I'm catching up from a week of
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > > vacation. Apologies for the
>>>>>>> inconvenience.
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > > On Tue, Sep 21, 2021, 10:59 AM Udi
>>>>>>> Meiri <eh...@google.com> wrote:
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >> I updated the affected and fixed
>>>>>>> version fields for
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>
>>>>>>> https://issues.apache.org/jira/browse/BEAM-12356.
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >> On Tue, Sep 21, 2021 at 10:48 AM
>>>>>>> Reuven Lax <re...@google.com> wrote:
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>> Unfortunate - I didn't realize that
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>
>>>>>>> https://github.com/apache/beam/pull/15480 didn't make the cut.
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>> This bug was a regression in Beam
>>>>>>> 2.32.0, and is blocking multiple users
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>> from updating.
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi
>>>>>>> Meiri <ud...@apache.org> wrote:
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> Hi everyone,
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> Please review and vote on the
>>>>>>> release candidate #1 for the version
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> 2.33.0, as follows:
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [ ] +1, Approve the release
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [ ] -1, Do not approve the release
>>>>>>> (please provide specific comments)
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> Reviewers are encouraged to test
>>>>>>> their own use cases with the release
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> candidate, and vote +1 if
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> no issues are found.
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> The complete staging area is
>>>>>>> available for your review, which includes:
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * JIRA release notes [1],
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * the official Apache source
>>>>>>> release to be deployed to dist.apache.org
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [2], which is signed with the key
>>>>>>> with fingerprint 587B049C36DAAFE6 [3],
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * all artifacts to be deployed to
>>>>>>> the Maven Central Repository [4],
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * source code tag "v2.33.0-RC1" [5],
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * website pull request listing the
>>>>>>> release [6], the blog post [6], and
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> publishing the API reference manual
>>>>>>> [7].
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * Java artifacts were built with
>>>>>>> Maven 3.6.3 and OpenJDK 1.8.0_181.
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * Python artifacts are deployed
>>>>>>> along with the source release to the
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> dist.apache.org [2] and pypy[8].
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * Validation sheet with a tab for
>>>>>>> 2.33.0 release to help with
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> validation [9].
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * Docker images published to Docker
>>>>>>> Hub [10].
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> The vote will be open for at least
>>>>>>> 72 hours. It is adopted by majority
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> approval, with at least 3 PMC
>>>>>>> affirmative votes.
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> For guidelines on how to try the
>>>>>>> release in your projects, check out
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> our blog post at
>>>>>>> https://beam.apache.org/blog/validate-beam-release/.
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> Thanks,
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> Release Manager
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [1]
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [2]
>>>>>>> https://dist.apache.org/repos/dist/dev/beam/2.33.0/
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [3]
>>>>>>> https://dist.apache.org/repos/dist/release/beam/KEYS
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [4]
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>>>> https://repository.apache.org/content/repositories/orgapachebeam-1234/
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [5]
>>>>>>> https://github.com/apache/beam/tree/v2.33.0-RC1
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [6]
>>>>>>> https://github.com/apache/beam/pull/15543
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [7]
>>>>>>> https://github.com/apache/beam-site/pull/619
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [8]
>>>>>>> https://pypi.org/project/apache-beam/2.33.0rc1/
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [9]
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [10]
>>>>>>> https://hub.docker.com/search?q=apache%2Fbeam&type=image
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>>>> >> >> >>>>>>>>>>>>>> >>>> >
>>>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>>>> >> >> >>>>>>>>>>>>>> >>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> Zachary Houfek
>>>>>>
>>>>>> Software Engineer
>>>>>>
>>>>>> DataPLS PLAT
>>>>>>
>>>>>> zhoufek@google.com
>>>>>>
>>>>>
>>>
>>> --
>>>
>>> Zachary Houfek
>>>
>>> Software Engineer
>>>
>>> DataPLS PLAT
>>>
>>> zhoufek@google.com
>>>
>>
>
> --
>
> Zachary Houfek
>
> Software Engineer
>
> DataPLS PLAT
>
> zhoufek@google.com
>

Re: [VOTE] Release 2.33.0, release candidate 1

[Zachary Houfek <zh...@google.com>]

Sure, I can do that, though it doesn't look like I can just revert the PR
from Github. I'll need to manually remove the changes from that commit.

Regards,
Zach

On Mon, Sep 27, 2021 at 9:02 PM Udi Meiri <eh...@google.com> wrote:

> Zachary could you prepare a rollback PR for the 2.33.0 branch?
>
> On Mon, Sep 27, 2021 at 5:25 PM Zachary Houfek <zh...@google.com> wrote:
>
>> > AfterProcessingTime and AfterCount should both be flagged...AfterEach
>> is different, since if any of the sequenced triggers never finish, the
>> whole thing will never finish
>>
>> Ah, looks like another necessary fix. AfterEach will be flagged if any
>> should be flagged. That and AfterProcessingTime should be ready soon.
>>
>> Regards,
>> Zach
>>
>> On Mon, Sep 27, 2021 at 7:36 PM Robert Bradshaw <ro...@google.com>
>> wrote:
>>
>>> OK, given the fact that AfterWatermark(...) implicitly invokes
>>> Repeatedly as well, and the sheer volume of concern/discussion on this
>>> issue, I now change my vote to a -1 and think it's best we just cut a new
>>> release with the fix in it.
>>>
>>>
>>> On Mon, Sep 27, 2021 at 4:06 PM Kenneth Knowles <ke...@apache.org> wrote:
>>>
>>>> Ah, thanks for the correction. AfterProcessingTime and AfterCount
>>>> should both be flagged, as should AfterEvery, and AfterAny. AfterEach is
>>>> different, since if any of the sequenced triggers never finish, the whole
>>>> thing will never finish.
>>>>
>>>> Kenn
>>>>
>>>> On Mon, Sep 27, 2021 at 3:51 PM Zachary Houfek <zh...@google.com>
>>>> wrote:
>>>>
>>>>> > This was discovered because Repeatedly(AfterProcessingTime(n))
>>>>>
>>>>> Wasn't it Repeatedly(AfterCount(n)) that caused the problem? At least
>>>>> that's what caused the failing test that prompted the fix.
>>>>>
>>>>> With that said, is the concern that AfterProcessingTime *should* include
>>>>> the CONDITION_NOT_GUARANTEED flag? It isn't right now, so I'm checking.
>>>>>
>>>>> Another thing I haven't seen mentioned: `Repeatedly` is or can be used
>>>>> in other triggers like AfterWatermark (because of the late trigger),
>>>>> AfterEach, AfterEvery, and AfterAny, so the bug will propagate to those
>>>>> triggers as well. Two of these actually had to have their tests
>>>>> updated in the fix
>>>>> <https://github.com/apache/beam/commit/533a63299789ec8b4bbe629c95734be8dd4d9589#diff-9278a08a4f97eb63d783206bdea1a05377b9edd41646255110446512b8adad43>
>>>>> .
>>>>>
>>>>> Regards,
>>>>> Zach
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Sep 27, 2021 at 6:30 PM Robert Bradshaw <ro...@google.com>
>>>>> wrote:
>>>>>
>>>>>> $ git checkout release-2.33.0
>>>>>> Switched to branch 'release-2.33.0'
>>>>>> Your branch is up to date with 'github/release-2.33.0'.
>>>>>>
>>>>>> $ python -c 'import apache_beam as beam;
>>>>>>
>>>>>> print(beam.transforms.trigger.Repeatedly(beam.transforms.trigger.AfterProcessingTime(5)).may_lose_data(None))'
>>>>>> DataLossReason.NO_POTENTIAL_LOSS
>>>>>>
>>>>>> The MAY_FINISH and CONDITION_NOT_GUARANTEED flags are disjoint.
>>>>>>
>>>>>> On Mon, Sep 27, 2021 at 3:15 PM Kenneth Knowles <ke...@apache.org>
>>>>>> wrote:
>>>>>> >
>>>>>> > This was discovered because Repeatedly(AfterProcessingTime(n)) does
>>>>>> require the workaround, because AfterProcessingTime(n) does terminate and
>>>>>> that is propagated by the Repeatedly implementation.
>>>>>> >
>>>>>> > Kenn
>>>>>> >
>>>>>> > On Mon, Sep 27, 2021 at 2:49 PM Robert Bradshaw <
>>>>>> robertwb@google.com> wrote:
>>>>>> >>
>>>>>> >> On Mon, Sep 27, 2021 at 2:42 PM Kenneth Knowles <ke...@apache.org>
>>>>>> wrote:
>>>>>> >> >
>>>>>> >> > True. It is the use case of Repeatedly(ProcessingTime(n)) that I
>>>>>> would guess to be the primary case of concern.
>>>>>> >>
>>>>>> >> This shouldn't require the workaround, right?
>>>>>> >>
>>>>>> >> If you're not comfortable with this, I'd say let's patch and
>>>>>> re-cut.
>>>>>> >>
>>>>>> >>
>>>>>> >> > On Mon, Sep 27, 2021 at 2:38 PM Robert Bradshaw <
>>>>>> robertwb@google.com> wrote:
>>>>>> >> >>
>>>>>> >> >> As I read the code, it's only pipelines that use a Repeatedly
>>>>>> trigger
>>>>>> >> >> that wrap an already lossy trigger that are declared to be
>>>>>> themselves
>>>>>> >> >> lossy. If I'm mistaken, I'll certainly reconsider my vote (and
>>>>>> thanks
>>>>>> >> >> for bringing this up).
>>>>>> >> >>
>>>>>> >> >> On Mon, Sep 27, 2021 at 2:21 PM Kenneth Knowles <
>>>>>> kenn@apache.org> wrote:
>>>>>> >> >> >
>>>>>> >> >> > My concern is that the error message is incorrect and every
>>>>>> user of 2.33.0 may be educated wrong, or be worried about data loss in
>>>>>> Beam, or fail to find the blog post or CHANGES, etc.
>>>>>> >> >> >
>>>>>> >> >> > Kenn
>>>>>> >> >> >
>>>>>> >> >> > On Mon, Sep 27, 2021 at 2:16 PM Udi Meiri <eh...@google.com>
>>>>>> wrote:
>>>>>> >> >> >>
>>>>>> >> >> >> I don't know how rare it is, but there is a flag documented
>>>>>> in CHANGES and blog post that reverts to the old behavior.
>>>>>> >> >> >>
>>>>>> >> >> >> On Mon, Sep 27, 2021 at 2:12 PM Kenneth Knowles <
>>>>>> kenn@apache.org> wrote:
>>>>>> >> >> >>>
>>>>>> >> >> >>> I guess my vote is -0 since I don't have enough context on
>>>>>> this issue. A number of people with more awareness of how severe this is
>>>>>> have voted +1 so I will not try to block the release.
>>>>>> >> >> >>>
>>>>>> >> >> >>> Kenn
>>>>>> >> >> >>>
>>>>>> >> >> >>> On Mon, Sep 27, 2021 at 2:11 PM Kenneth Knowles <
>>>>>> kenn@apache.org> wrote:
>>>>>> >> >> >>>>
>>>>>> >> >> >>>> I have to disagree with the other PMC members here, or at
>>>>>> least dig in to the question: every pipeline that uses a Repeatedly trigger
>>>>>> at the top level will be rejected. Is this so rare in Python that it is OK?
>>>>>> >> >> >>>>
>>>>>> >> >> >>>> Kenn
>>>>>> >> >> >>>>
>>>>>> >> >> >>>> On Mon, Sep 27, 2021 at 1:56 PM Robert Burke <
>>>>>> rebo@google.com> wrote:
>>>>>> >> >> >>>>>
>>>>>> >> >> >>>>> On it. Thanks!
>>>>>> >> >> >>>>>
>>>>>> >> >> >>>>> On Mon, Sep 27, 2021 at 1:18 PM Udi Meiri <
>>>>>> ehudm@google.com> wrote:
>>>>>> >> >> >>>>>>
>>>>>> >> >> >>>>>> Daniel/Robert, feel free to make changes to this PR:
>>>>>> https://github.com/apache/beam/pull/15543
>>>>>> >> >> >>>>>>
>>>>>> >> >> >>>>>> On Mon, Sep 27, 2021 at 12:08 PM Daniel Oliveira <
>>>>>> danoliveira@google.com> wrote:
>>>>>> >> >> >>>>>>>
>>>>>> >> >> >>>>>>> +1
>>>>>> >> >> >>>>>>>
>>>>>> >> >> >>>>>>> I hadn't realized the pipelines were still finishing
>>>>>> successfully. I retried wordcount with that in mind and confirmed it
>>>>>> finishes successfully, so this isn't a blocker.
>>>>>> >> >> >>>>>>>
>>>>>> >> >> >>>>>>> Although maybe we should add this to the "Known Issues"
>>>>>> because I can easily see those messages being interpreted as a pipeline
>>>>>> failure.
>>>>>> >> >> >>>>>>>
>>>>>> >> >> >>>>>>> On Sun, Sep 26, 2021 at 7:26 PM Robert Burke <
>>>>>> robert@frantil.com> wrote:
>>>>>> >> >> >>>>>>>>
>>>>>> >> >> >>>>>>>> That's on the SDK side, and it just means the
>>>>>> PCollection metrics are being returned, buy not handled by the SDK. At
>>>>>> present the pipeline results only handle PTransform metrics.
>>>>>> >> >> >>>>>>>>
>>>>>> >> >> >>>>>>>> As such it's not a regression, as adding those is
>>>>>> still under development.
>>>>>> >> >> >>>>>>>>
>>>>>> >> >> >>>>>>>> On Sun, Sep 26, 2021, 7:02 PM Daniel Oliveira <
>>>>>> danoliveira@google.com> wrote:
>>>>>> >> >> >>>>>>>>>
>>>>>> >> >> >>>>>>>>> I tried validating wordcount with the Go SDK on
>>>>>> Flink. The pipeline failed with a wall of errors like the following. I
>>>>>> tried this on Flink 1.11, 1.12, and 1.13 job servers built from source at
>>>>>> the RC1 commit, same errors on all of them.
>>>>>> >> >> >>>>>>>>>
>>>>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>>>>> MonitoringInfo: urn:"beam:metric:element_count:v1"
>>>>>> type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"
>>>>>> labels:{key:"PCOLLECTION"  value:"n9"}
>>>>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>>>>> MonitoringInfo: urn:"beam:metric:element_count:v1"
>>>>>> type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"
>>>>>> labels:{key:"PCOLLECTION"  value:"n10"}
>>>>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>>>>> MonitoringInfo: urn:"beam:metric:element_count:v1"
>>>>>> type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"
>>>>>> labels:{key:"PCOLLECTION"  value:"n8"}
>>>>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>>>>> MonitoringInfo: urn:"beam:metric:element_count:v1"
>>>>>> type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"
>>>>>> labels:{key:"PCOLLECTION"  value:"n7"}
>>>>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>>>>> MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"
>>>>>> type:"beam:metrics:distribution_int64:v1"
>>>>>> payload:"\x8d%\xfa\xbf\x05\x04\xa8\x0c"  labels:{key:"PCOLLECTION"
>>>>>> value:"n7"}
>>>>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>>>>> MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"
>>>>>> type:"beam:metrics:distribution_int64:v1"  payload:"\xb9\x05\xcf6\x05\x11"
>>>>>> labels:{key:"PCOLLECTION"  value:"n9"}
>>>>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>>>>> MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"
>>>>>> type:"beam:metrics:distribution_int64:v1"  payload:"\xc5\x05\x8a1\x04\x12"
>>>>>> labels:{key:"PCOLLECTION"  value:"n8"}
>>>>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>>>>> MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"
>>>>>> type:"beam:metrics:distribution_int64:v1"
>>>>>> payload:"\x8d%\xb3\xa7\x07\x14\""  labels:{key:"PCOLLECTION"  value:"n10"}
>>>>>> >> >> >>>>>>>>>> {...}
>>>>>> >> >> >>>>>>>>>
>>>>>> >> >> >>>>>>>>>
>>>>>> >> >> >>>>>>>>>  @Robert Burke I think you might know what's going on
>>>>>> here. Is this solvable with a cherry-pick and a new RC?
>>>>>> >> >> >>>>>>>>>
>>>>>> >> >> >>>>>>>>> On Fri, Sep 24, 2021 at 4:25 PM Ahmet Altay <
>>>>>> altay@google.com> wrote:
>>>>>> >> >> >>>>>>>>>>
>>>>>> >> >> >>>>>>>>>>
>>>>>> >> >> >>>>>>>>>>
>>>>>> >> >> >>>>>>>>>> On Fri, Sep 24, 2021 at 4:21 PM Udi Meiri <
>>>>>> ehudm@google.com> wrote:
>>>>>> >> >> >>>>>>>>>>>
>>>>>> >> >> >>>>>>>>>>>
>>>>>> >> >> >>>>>>>>>>>
>>>>>> >> >> >>>>>>>>>>> On Fri, Sep 24, 2021 at 4:18 PM Ahmet Altay <
>>>>>> altay@google.com> wrote:
>>>>>> >> >> >>>>>>>>>>>>
>>>>>> >> >> >>>>>>>>>>>> Thank you for the update related to
>>>>>> allow_unsafe_triggers. My vote is still a +1.
>>>>>> >> >> >>>>>>>>>>>>
>>>>>> >> >> >>>>>>>>>>>> If we decide to move forward with this RC, could
>>>>>> you please include this bug in the known issues list under the changes.md
>>>>>> for this release?
>>>>>> >> >> >>>>>>>>>>>
>>>>>> >> >> >>>>>>>>>>> Yes, that's included in
>>>>>> https://github.com/apache/beam/pull/15543/files.
>>>>>> >> >> >>>>>>>>>>
>>>>>> >> >> >>>>>>>>>>
>>>>>> >> >> >>>>>>>>>> Thank you!
>>>>>> >> >> >>>>>>>>>>
>>>>>> >> >> >>>>>>>>>>>
>>>>>> >> >> >>>>>>>>>>>
>>>>>> >> >> >>>>>>>>>>>>
>>>>>> >> >> >>>>>>>>>>>>
>>>>>> >> >> >>>>>>>>>>>> On Fri, Sep 24, 2021 at 3:56 PM Chamikara Jayalath
>>>>>> <ch...@google.com> wrote:
>>>>>> >> >> >>>>>>>>>>>>>
>>>>>> >> >> >>>>>>>>>>>>> +1 (binding)
>>>>>> >> >> >>>>>>>>>>>>> Validated a few scenarios from the spreadsheet.
>>>>>> >> >> >>>>>>>>>>>>>
>>>>>> >> >> >>>>>>>>>>>>> Thanks,
>>>>>> >> >> >>>>>>>>>>>>> Cham
>>>>>> >> >> >>>>>>>>>>>>>
>>>>>> >> >> >>>>>>>>>>>>> On Fri, Sep 24, 2021 at 3:07 PM Robert Bradshaw <
>>>>>> robertwb@google.com> wrote:
>>>>>> >> >> >>>>>>>>>>>>>>
>>>>>> >> >> >>>>>>>>>>>>>> +1 (binding)
>>>>>> >> >> >>>>>>>>>>>>>>
>>>>>> >> >> >>>>>>>>>>>>>> All the artifacts and signatures look good.
>>>>>> >> >> >>>>>>>>>>>>>>
>>>>>> >> >> >>>>>>>>>>>>>> I don't think the unsafe trigger check is severe
>>>>>> enough to block the release.
>>>>>> >> >> >>>>>>>>>>>>>>
>>>>>> >> >> >>>>>>>>>>>>>> On Fri, Sep 24, 2021 at 2:36 PM Udi Meiri <
>>>>>> ehudm@google.com> wrote:
>>>>>> >> >> >>>>>>>>>>>>>> >
>>>>>> >> >> >>>>>>>>>>>>>> > Hi everyone, we found a bug during testing. It
>>>>>> has to do with Python SDK's allow_unsafe_triggers check.
>>>>>> >> >> >>>>>>>>>>>>>> > There is a preliminary fix that will go to
>>>>>> master.
>>>>>> >> >> >>>>>>>>>>>>>> >
>>>>>> >> >> >>>>>>>>>>>>>> > For the 2.33.0 release, I'm leaning towards
>>>>>> not making a new RC since there is a workaround: pass the flag
>>>>>> --allow_unsafe_triggers.
>>>>>> >> >> >>>>>>>>>>>>>> >
>>>>>> >> >> >>>>>>>>>>>>>> > Please reevaluate your votes accordingly and
>>>>>> recast if you've changed your vote.
>>>>>> >> >> >>>>>>>>>>>>>> >
>>>>>> >> >> >>>>>>>>>>>>>> > Thanks
>>>>>> >> >> >>>>>>>>>>>>>> >
>>>>>> >> >> >>>>>>>>>>>>>> > On Fri, Sep 24, 2021 at 12:48 PM Alexey
>>>>>> Romanenko <ar...@gmail.com> wrote:
>>>>>> >> >> >>>>>>>>>>>>>> >>
>>>>>> >> >> >>>>>>>>>>>>>> >>
>>>>>> >> >> >>>>>>>>>>>>>> >> On 24 Sep 2021, at 20:45, Udi Meiri <
>>>>>> ehudm@google.com> wrote:
>>>>>> >> >> >>>>>>>>>>>>>> >> Alexey is this something that we should put
>>>>>> in the release notes, or some other change?
>>>>>> >> >> >>>>>>>>>>>>>> >>
>>>>>> >> >> >>>>>>>>>>>>>> >>
>>>>>> >> >> >>>>>>>>>>>>>> >> Yes, I think it could be helpful to mention
>>>>>> that Beam Jackson’s deps was bumped and it may require an update of
>>>>>> Jackson’s runtime deps for Spark 2 users as well.
>>>>>> >> >> >>>>>>>>>>>>>> >>
>>>>>> >> >> >>>>>>>>>>>>>> >> —
>>>>>> >> >> >>>>>>>>>>>>>> >> Alexey
>>>>>> >> >> >>>>>>>>>>>>>> >>
>>>>>> >> >> >>>>>>>>>>>>>> >>
>>>>>> >> >> >>>>>>>>>>>>>> >>
>>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>>> >> >> >>>>>>>>>>>>>> >>> —
>>>>>> >> >> >>>>>>>>>>>>>> >>> Alexey
>>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>>> >> >> >>>>>>>>>>>>>> >>> On 24 Sep 2021, at 16:17, Alexey Romanenko <
>>>>>> aromanenko.dev@gmail.com> wrote:
>>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>>> >> >> >>>>>>>>>>>>>> >>> I checked with beam-samples [1] and noticed
>>>>>> an issue to run some pipelines with Spark 2 runner (Spark 3 seems is ok).
>>>>>> >> >> >>>>>>>>>>>>>> >>> It looks that it’s caused by new Jackson's
>>>>>> version updated recently [2], even if it’s a minor update but it works fine
>>>>>> with a previous one.
>>>>>> >> >> >>>>>>>>>>>>>> >>> I’ll try to find a workaround and get back
>>>>>> with a results of this.
>>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>>> >> >> >>>>>>>>>>>>>> >>> [1] https://github.com/Talend/beam-samples/
>>>>>> >> >> >>>>>>>>>>>>>> >>> [2]
>>>>>> https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c
>>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>>> >> >> >>>>>>>>>>>>>> >>> —
>>>>>> >> >> >>>>>>>>>>>>>> >>> Alexey
>>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>>> >> >> >>>>>>>>>>>>>> >>> On 24 Sep 2021, at 11:17, Jan Lukavský <
>>>>>> je.ik@seznam.cz> wrote:
>>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>>> >> >> >>>>>>>>>>>>>> >>> +1 (non-binding)
>>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>>> >> >> >>>>>>>>>>>>>> >>> Validated several use-cases using
>>>>>> non-portable Flink with Java SDK.
>>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>>> >> >> >>>>>>>>>>>>>> >>> On 9/24/21 4:55 AM, Valentyn Tymofieiev
>>>>>> wrote:
>>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>>> >> >> >>>>>>>>>>>>>> >>> +1. Ran several Python batch and streaming
>>>>>> pipelines on Dataflow and checked that Dataflow containers have required
>>>>>> dependencies of Beam.
>>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>>> >> >> >>>>>>>>>>>>>> >>> On Thu, Sep 23, 2021 at 7:03 PM Robert Burke
>>>>>> <lo...@apache.org> wrote:
>>>>>> >> >> >>>>>>>>>>>>>> >>>>
>>>>>> >> >> >>>>>>>>>>>>>> >>>> +1 (non-binding)
>>>>>> >> >> >>>>>>>>>>>>>> >>>> I validated the Go Quickstart (wordcount),
>>>>>> and my ray tracer against the Go Direct runner, Dataflow, and Spark
>>>>>> (ensuring the rc1 tagged container was used) and they executed successfully.
>>>>>> >> >> >>>>>>>>>>>>>> >>>>
>>>>>> >> >> >>>>>>>>>>>>>> >>>> I needed to manually synthesize a
>>>>>> pseudo-version to ensure I was using the tagged branch version
>>>>>> (v2.0.0-20210914211513-b358127f9859) instead of simply using v2.33.0-RC1.
>>>>>> >> >> >>>>>>>>>>>>>> >>>>
>>>>>> >> >> >>>>>>>>>>>>>> >>>>  It either can't find the package with the
>>>>>> right tagged version, or it can't find the version. It's not clear to me
>>>>>> what the issue is, but it's not notionally a release blocker. I'll
>>>>>> investigate further once we have a full release, as it's probably some
>>>>>> unspecified behavior due to how we transitioned to Go Modules (which
>>>>>> strongly recommended doing a major version bump for such transitions, which
>>>>>> seems a bit excessive for Beam as a whole...).
>>>>>> >> >> >>>>>>>>>>>>>> >>>>
>>>>>> >> >> >>>>>>>>>>>>>> >>>> On 2021/09/23 03:59:18, Ahmet Altay <
>>>>>> altay@google.com> wrote:
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > +1 on the RC. I validated python quick
>>>>>> start examples on direct runners.
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > Thank you Udi.
>>>>>> >> >> >>>>>>>>>>>>>> >>>> >
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > On Wed, Sep 22, 2021 at 2:20 PM Robert
>>>>>> Burke <ro...@frantil.com> wrote:
>>>>>> >> >> >>>>>>>>>>>>>> >>>> >
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > > Just an FYI that intend to validate the
>>>>>> Go SDK for this release but can't
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > > get to it until tomorrow (Thursday).
>>>>>> I'm catching up from a week of
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > > vacation. Apologies for the
>>>>>> inconvenience.
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > > On Tue, Sep 21, 2021, 10:59 AM Udi
>>>>>> Meiri <eh...@google.com> wrote:
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >> I updated the affected and fixed
>>>>>> version fields for
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>
>>>>>> https://issues.apache.org/jira/browse/BEAM-12356.
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >> On Tue, Sep 21, 2021 at 10:48 AM
>>>>>> Reuven Lax <re...@google.com> wrote:
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>> Unfortunate - I didn't realize that
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>
>>>>>> https://github.com/apache/beam/pull/15480 didn't make the cut.
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>> This bug was a regression in Beam
>>>>>> 2.32.0, and is blocking multiple users
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>> from updating.
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi
>>>>>> Meiri <ud...@apache.org> wrote:
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> Hi everyone,
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> Please review and vote on the
>>>>>> release candidate #1 for the version
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> 2.33.0, as follows:
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [ ] +1, Approve the release
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [ ] -1, Do not approve the release
>>>>>> (please provide specific comments)
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> Reviewers are encouraged to test
>>>>>> their own use cases with the release
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> candidate, and vote +1 if
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> no issues are found.
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> The complete staging area is
>>>>>> available for your review, which includes:
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * JIRA release notes [1],
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * the official Apache source release
>>>>>> to be deployed to dist.apache.org
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [2], which is signed with the key
>>>>>> with fingerprint 587B049C36DAAFE6 [3],
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * all artifacts to be deployed to
>>>>>> the Maven Central Repository [4],
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * source code tag "v2.33.0-RC1" [5],
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * website pull request listing the
>>>>>> release [6], the blog post [6], and
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> publishing the API reference manual
>>>>>> [7].
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * Java artifacts were built with
>>>>>> Maven 3.6.3 and OpenJDK 1.8.0_181.
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * Python artifacts are deployed
>>>>>> along with the source release to the
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> dist.apache.org [2] and pypy[8].
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * Validation sheet with a tab for
>>>>>> 2.33.0 release to help with
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> validation [9].
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * Docker images published to Docker
>>>>>> Hub [10].
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> The vote will be open for at least
>>>>>> 72 hours. It is adopted by majority
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> approval, with at least 3 PMC
>>>>>> affirmative votes.
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> For guidelines on how to try the
>>>>>> release in your projects, check out
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> our blog post at
>>>>>> https://beam.apache.org/blog/validate-beam-release/.
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> Thanks,
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> Release Manager
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [1]
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [2]
>>>>>> https://dist.apache.org/repos/dist/dev/beam/2.33.0/
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [3]
>>>>>> https://dist.apache.org/repos/dist/release/beam/KEYS
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [4]
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>>> https://repository.apache.org/content/repositories/orgapachebeam-1234/
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [5]
>>>>>> https://github.com/apache/beam/tree/v2.33.0-RC1
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [6]
>>>>>> https://github.com/apache/beam/pull/15543
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [7]
>>>>>> https://github.com/apache/beam-site/pull/619
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [8]
>>>>>> https://pypi.org/project/apache-beam/2.33.0rc1/
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [9]
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [10]
>>>>>> https://hub.docker.com/search?q=apache%2Fbeam&type=image
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>>> >> >> >>>>>>>>>>>>>> >>>> >
>>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>>> >> >> >>>>>>>>>>>>>> >>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Zachary Houfek
>>>>>
>>>>> Software Engineer
>>>>>
>>>>> DataPLS PLAT
>>>>>
>>>>> zhoufek@google.com
>>>>>
>>>>
>>
>> --
>>
>> Zachary Houfek
>>
>> Software Engineer
>>
>> DataPLS PLAT
>>
>> zhoufek@google.com
>>
>

-- 

Zachary Houfek

Software Engineer

DataPLS PLAT

zhoufek@google.com

Re: [VOTE] Release 2.33.0, release candidate 1

[Udi Meiri <eh...@google.com>]

Zachary could you prepare a rollback PR for the 2.33.0 branch?

On Mon, Sep 27, 2021 at 5:25 PM Zachary Houfek <zh...@google.com> wrote:

> > AfterProcessingTime and AfterCount should both be flagged...AfterEach is
> different, since if any of the sequenced triggers never finish, the whole
> thing will never finish
>
> Ah, looks like another necessary fix. AfterEach will be flagged if any
> should be flagged. That and AfterProcessingTime should be ready soon.
>
> Regards,
> Zach
>
> On Mon, Sep 27, 2021 at 7:36 PM Robert Bradshaw <ro...@google.com>
> wrote:
>
>> OK, given the fact that AfterWatermark(...) implicitly invokes Repeatedly
>> as well, and the sheer volume of concern/discussion on this issue, I now
>> change my vote to a -1 and think it's best we just cut a new release
>> with the fix in it.
>>
>>
>> On Mon, Sep 27, 2021 at 4:06 PM Kenneth Knowles <ke...@apache.org> wrote:
>>
>>> Ah, thanks for the correction. AfterProcessingTime and AfterCount should
>>> both be flagged, as should AfterEvery, and AfterAny. AfterEach is
>>> different, since if any of the sequenced triggers never finish, the whole
>>> thing will never finish.
>>>
>>> Kenn
>>>
>>> On Mon, Sep 27, 2021 at 3:51 PM Zachary Houfek <zh...@google.com>
>>> wrote:
>>>
>>>> > This was discovered because Repeatedly(AfterProcessingTime(n))
>>>>
>>>> Wasn't it Repeatedly(AfterCount(n)) that caused the problem? At least
>>>> that's what caused the failing test that prompted the fix.
>>>>
>>>> With that said, is the concern that AfterProcessingTime *should* include
>>>> the CONDITION_NOT_GUARANTEED flag? It isn't right now, so I'm checking.
>>>>
>>>> Another thing I haven't seen mentioned: `Repeatedly` is or can be used
>>>> in other triggers like AfterWatermark (because of the late trigger),
>>>> AfterEach, AfterEvery, and AfterAny, so the bug will propagate to those
>>>> triggers as well. Two of these actually had to have their tests
>>>> updated in the fix
>>>> <https://github.com/apache/beam/commit/533a63299789ec8b4bbe629c95734be8dd4d9589#diff-9278a08a4f97eb63d783206bdea1a05377b9edd41646255110446512b8adad43>
>>>> .
>>>>
>>>> Regards,
>>>> Zach
>>>>
>>>>
>>>>
>>>> On Mon, Sep 27, 2021 at 6:30 PM Robert Bradshaw <ro...@google.com>
>>>> wrote:
>>>>
>>>>> $ git checkout release-2.33.0
>>>>> Switched to branch 'release-2.33.0'
>>>>> Your branch is up to date with 'github/release-2.33.0'.
>>>>>
>>>>> $ python -c 'import apache_beam as beam;
>>>>>
>>>>> print(beam.transforms.trigger.Repeatedly(beam.transforms.trigger.AfterProcessingTime(5)).may_lose_data(None))'
>>>>> DataLossReason.NO_POTENTIAL_LOSS
>>>>>
>>>>> The MAY_FINISH and CONDITION_NOT_GUARANTEED flags are disjoint.
>>>>>
>>>>> On Mon, Sep 27, 2021 at 3:15 PM Kenneth Knowles <ke...@apache.org>
>>>>> wrote:
>>>>> >
>>>>> > This was discovered because Repeatedly(AfterProcessingTime(n)) does
>>>>> require the workaround, because AfterProcessingTime(n) does terminate and
>>>>> that is propagated by the Repeatedly implementation.
>>>>> >
>>>>> > Kenn
>>>>> >
>>>>> > On Mon, Sep 27, 2021 at 2:49 PM Robert Bradshaw <ro...@google.com>
>>>>> wrote:
>>>>> >>
>>>>> >> On Mon, Sep 27, 2021 at 2:42 PM Kenneth Knowles <ke...@apache.org>
>>>>> wrote:
>>>>> >> >
>>>>> >> > True. It is the use case of Repeatedly(ProcessingTime(n)) that I
>>>>> would guess to be the primary case of concern.
>>>>> >>
>>>>> >> This shouldn't require the workaround, right?
>>>>> >>
>>>>> >> If you're not comfortable with this, I'd say let's patch and re-cut.
>>>>> >>
>>>>> >>
>>>>> >> > On Mon, Sep 27, 2021 at 2:38 PM Robert Bradshaw <
>>>>> robertwb@google.com> wrote:
>>>>> >> >>
>>>>> >> >> As I read the code, it's only pipelines that use a Repeatedly
>>>>> trigger
>>>>> >> >> that wrap an already lossy trigger that are declared to be
>>>>> themselves
>>>>> >> >> lossy. If I'm mistaken, I'll certainly reconsider my vote (and
>>>>> thanks
>>>>> >> >> for bringing this up).
>>>>> >> >>
>>>>> >> >> On Mon, Sep 27, 2021 at 2:21 PM Kenneth Knowles <ke...@apache.org>
>>>>> wrote:
>>>>> >> >> >
>>>>> >> >> > My concern is that the error message is incorrect and every
>>>>> user of 2.33.0 may be educated wrong, or be worried about data loss in
>>>>> Beam, or fail to find the blog post or CHANGES, etc.
>>>>> >> >> >
>>>>> >> >> > Kenn
>>>>> >> >> >
>>>>> >> >> > On Mon, Sep 27, 2021 at 2:16 PM Udi Meiri <eh...@google.com>
>>>>> wrote:
>>>>> >> >> >>
>>>>> >> >> >> I don't know how rare it is, but there is a flag documented
>>>>> in CHANGES and blog post that reverts to the old behavior.
>>>>> >> >> >>
>>>>> >> >> >> On Mon, Sep 27, 2021 at 2:12 PM Kenneth Knowles <
>>>>> kenn@apache.org> wrote:
>>>>> >> >> >>>
>>>>> >> >> >>> I guess my vote is -0 since I don't have enough context on
>>>>> this issue. A number of people with more awareness of how severe this is
>>>>> have voted +1 so I will not try to block the release.
>>>>> >> >> >>>
>>>>> >> >> >>> Kenn
>>>>> >> >> >>>
>>>>> >> >> >>> On Mon, Sep 27, 2021 at 2:11 PM Kenneth Knowles <
>>>>> kenn@apache.org> wrote:
>>>>> >> >> >>>>
>>>>> >> >> >>>> I have to disagree with the other PMC members here, or at
>>>>> least dig in to the question: every pipeline that uses a Repeatedly trigger
>>>>> at the top level will be rejected. Is this so rare in Python that it is OK?
>>>>> >> >> >>>>
>>>>> >> >> >>>> Kenn
>>>>> >> >> >>>>
>>>>> >> >> >>>> On Mon, Sep 27, 2021 at 1:56 PM Robert Burke <
>>>>> rebo@google.com> wrote:
>>>>> >> >> >>>>>
>>>>> >> >> >>>>> On it. Thanks!
>>>>> >> >> >>>>>
>>>>> >> >> >>>>> On Mon, Sep 27, 2021 at 1:18 PM Udi Meiri <
>>>>> ehudm@google.com> wrote:
>>>>> >> >> >>>>>>
>>>>> >> >> >>>>>> Daniel/Robert, feel free to make changes to this PR:
>>>>> https://github.com/apache/beam/pull/15543
>>>>> >> >> >>>>>>
>>>>> >> >> >>>>>> On Mon, Sep 27, 2021 at 12:08 PM Daniel Oliveira <
>>>>> danoliveira@google.com> wrote:
>>>>> >> >> >>>>>>>
>>>>> >> >> >>>>>>> +1
>>>>> >> >> >>>>>>>
>>>>> >> >> >>>>>>> I hadn't realized the pipelines were still finishing
>>>>> successfully. I retried wordcount with that in mind and confirmed it
>>>>> finishes successfully, so this isn't a blocker.
>>>>> >> >> >>>>>>>
>>>>> >> >> >>>>>>> Although maybe we should add this to the "Known Issues"
>>>>> because I can easily see those messages being interpreted as a pipeline
>>>>> failure.
>>>>> >> >> >>>>>>>
>>>>> >> >> >>>>>>> On Sun, Sep 26, 2021 at 7:26 PM Robert Burke <
>>>>> robert@frantil.com> wrote:
>>>>> >> >> >>>>>>>>
>>>>> >> >> >>>>>>>> That's on the SDK side, and it just means the
>>>>> PCollection metrics are being returned, buy not handled by the SDK. At
>>>>> present the pipeline results only handle PTransform metrics.
>>>>> >> >> >>>>>>>>
>>>>> >> >> >>>>>>>> As such it's not a regression, as adding those is still
>>>>> under development.
>>>>> >> >> >>>>>>>>
>>>>> >> >> >>>>>>>> On Sun, Sep 26, 2021, 7:02 PM Daniel Oliveira <
>>>>> danoliveira@google.com> wrote:
>>>>> >> >> >>>>>>>>>
>>>>> >> >> >>>>>>>>> I tried validating wordcount with the Go SDK on Flink.
>>>>> The pipeline failed with a wall of errors like the following. I tried this
>>>>> on Flink 1.11, 1.12, and 1.13 job servers built from source at the RC1
>>>>> commit, same errors on all of them.
>>>>> >> >> >>>>>>>>>
>>>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>>>> MonitoringInfo: urn:"beam:metric:element_count:v1"
>>>>> type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"
>>>>> labels:{key:"PCOLLECTION"  value:"n9"}
>>>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>>>> MonitoringInfo: urn:"beam:metric:element_count:v1"
>>>>> type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"
>>>>> labels:{key:"PCOLLECTION"  value:"n10"}
>>>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>>>> MonitoringInfo: urn:"beam:metric:element_count:v1"
>>>>> type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"
>>>>> labels:{key:"PCOLLECTION"  value:"n8"}
>>>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>>>> MonitoringInfo: urn:"beam:metric:element_count:v1"
>>>>> type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"
>>>>> labels:{key:"PCOLLECTION"  value:"n7"}
>>>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>>>> MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"
>>>>> type:"beam:metrics:distribution_int64:v1"
>>>>> payload:"\x8d%\xfa\xbf\x05\x04\xa8\x0c"  labels:{key:"PCOLLECTION"
>>>>> value:"n7"}
>>>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>>>> MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"
>>>>> type:"beam:metrics:distribution_int64:v1"  payload:"\xb9\x05\xcf6\x05\x11"
>>>>> labels:{key:"PCOLLECTION"  value:"n9"}
>>>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>>>> MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"
>>>>> type:"beam:metrics:distribution_int64:v1"  payload:"\xc5\x05\x8a1\x04\x12"
>>>>> labels:{key:"PCOLLECTION"  value:"n8"}
>>>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>>>> MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"
>>>>> type:"beam:metrics:distribution_int64:v1"
>>>>> payload:"\x8d%\xb3\xa7\x07\x14\""  labels:{key:"PCOLLECTION"  value:"n10"}
>>>>> >> >> >>>>>>>>>> {...}
>>>>> >> >> >>>>>>>>>
>>>>> >> >> >>>>>>>>>
>>>>> >> >> >>>>>>>>>  @Robert Burke I think you might know what's going on
>>>>> here. Is this solvable with a cherry-pick and a new RC?
>>>>> >> >> >>>>>>>>>
>>>>> >> >> >>>>>>>>> On Fri, Sep 24, 2021 at 4:25 PM Ahmet Altay <
>>>>> altay@google.com> wrote:
>>>>> >> >> >>>>>>>>>>
>>>>> >> >> >>>>>>>>>>
>>>>> >> >> >>>>>>>>>>
>>>>> >> >> >>>>>>>>>> On Fri, Sep 24, 2021 at 4:21 PM Udi Meiri <
>>>>> ehudm@google.com> wrote:
>>>>> >> >> >>>>>>>>>>>
>>>>> >> >> >>>>>>>>>>>
>>>>> >> >> >>>>>>>>>>>
>>>>> >> >> >>>>>>>>>>> On Fri, Sep 24, 2021 at 4:18 PM Ahmet Altay <
>>>>> altay@google.com> wrote:
>>>>> >> >> >>>>>>>>>>>>
>>>>> >> >> >>>>>>>>>>>> Thank you for the update related to
>>>>> allow_unsafe_triggers. My vote is still a +1.
>>>>> >> >> >>>>>>>>>>>>
>>>>> >> >> >>>>>>>>>>>> If we decide to move forward with this RC, could
>>>>> you please include this bug in the known issues list under the changes.md
>>>>> for this release?
>>>>> >> >> >>>>>>>>>>>
>>>>> >> >> >>>>>>>>>>> Yes, that's included in
>>>>> https://github.com/apache/beam/pull/15543/files.
>>>>> >> >> >>>>>>>>>>
>>>>> >> >> >>>>>>>>>>
>>>>> >> >> >>>>>>>>>> Thank you!
>>>>> >> >> >>>>>>>>>>
>>>>> >> >> >>>>>>>>>>>
>>>>> >> >> >>>>>>>>>>>
>>>>> >> >> >>>>>>>>>>>>
>>>>> >> >> >>>>>>>>>>>>
>>>>> >> >> >>>>>>>>>>>> On Fri, Sep 24, 2021 at 3:56 PM Chamikara Jayalath <
>>>>> chamikara@google.com> wrote:
>>>>> >> >> >>>>>>>>>>>>>
>>>>> >> >> >>>>>>>>>>>>> +1 (binding)
>>>>> >> >> >>>>>>>>>>>>> Validated a few scenarios from the spreadsheet.
>>>>> >> >> >>>>>>>>>>>>>
>>>>> >> >> >>>>>>>>>>>>> Thanks,
>>>>> >> >> >>>>>>>>>>>>> Cham
>>>>> >> >> >>>>>>>>>>>>>
>>>>> >> >> >>>>>>>>>>>>> On Fri, Sep 24, 2021 at 3:07 PM Robert Bradshaw <
>>>>> robertwb@google.com> wrote:
>>>>> >> >> >>>>>>>>>>>>>>
>>>>> >> >> >>>>>>>>>>>>>> +1 (binding)
>>>>> >> >> >>>>>>>>>>>>>>
>>>>> >> >> >>>>>>>>>>>>>> All the artifacts and signatures look good.
>>>>> >> >> >>>>>>>>>>>>>>
>>>>> >> >> >>>>>>>>>>>>>> I don't think the unsafe trigger check is severe
>>>>> enough to block the release.
>>>>> >> >> >>>>>>>>>>>>>>
>>>>> >> >> >>>>>>>>>>>>>> On Fri, Sep 24, 2021 at 2:36 PM Udi Meiri <
>>>>> ehudm@google.com> wrote:
>>>>> >> >> >>>>>>>>>>>>>> >
>>>>> >> >> >>>>>>>>>>>>>> > Hi everyone, we found a bug during testing. It
>>>>> has to do with Python SDK's allow_unsafe_triggers check.
>>>>> >> >> >>>>>>>>>>>>>> > There is a preliminary fix that will go to
>>>>> master.
>>>>> >> >> >>>>>>>>>>>>>> >
>>>>> >> >> >>>>>>>>>>>>>> > For the 2.33.0 release, I'm leaning towards not
>>>>> making a new RC since there is a workaround: pass the flag
>>>>> --allow_unsafe_triggers.
>>>>> >> >> >>>>>>>>>>>>>> >
>>>>> >> >> >>>>>>>>>>>>>> > Please reevaluate your votes accordingly and
>>>>> recast if you've changed your vote.
>>>>> >> >> >>>>>>>>>>>>>> >
>>>>> >> >> >>>>>>>>>>>>>> > Thanks
>>>>> >> >> >>>>>>>>>>>>>> >
>>>>> >> >> >>>>>>>>>>>>>> > On Fri, Sep 24, 2021 at 12:48 PM Alexey
>>>>> Romanenko <ar...@gmail.com> wrote:
>>>>> >> >> >>>>>>>>>>>>>> >>
>>>>> >> >> >>>>>>>>>>>>>> >>
>>>>> >> >> >>>>>>>>>>>>>> >> On 24 Sep 2021, at 20:45, Udi Meiri <
>>>>> ehudm@google.com> wrote:
>>>>> >> >> >>>>>>>>>>>>>> >> Alexey is this something that we should put in
>>>>> the release notes, or some other change?
>>>>> >> >> >>>>>>>>>>>>>> >>
>>>>> >> >> >>>>>>>>>>>>>> >>
>>>>> >> >> >>>>>>>>>>>>>> >> Yes, I think it could be helpful to mention
>>>>> that Beam Jackson’s deps was bumped and it may require an update of
>>>>> Jackson’s runtime deps for Spark 2 users as well.
>>>>> >> >> >>>>>>>>>>>>>> >>
>>>>> >> >> >>>>>>>>>>>>>> >> —
>>>>> >> >> >>>>>>>>>>>>>> >> Alexey
>>>>> >> >> >>>>>>>>>>>>>> >>
>>>>> >> >> >>>>>>>>>>>>>> >>
>>>>> >> >> >>>>>>>>>>>>>> >>
>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>> >> >> >>>>>>>>>>>>>> >>> —
>>>>> >> >> >>>>>>>>>>>>>> >>> Alexey
>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>> >> >> >>>>>>>>>>>>>> >>> On 24 Sep 2021, at 16:17, Alexey Romanenko <
>>>>> aromanenko.dev@gmail.com> wrote:
>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>> >> >> >>>>>>>>>>>>>> >>> I checked with beam-samples [1] and noticed
>>>>> an issue to run some pipelines with Spark 2 runner (Spark 3 seems is ok).
>>>>> >> >> >>>>>>>>>>>>>> >>> It looks that it’s caused by new Jackson's
>>>>> version updated recently [2], even if it’s a minor update but it works fine
>>>>> with a previous one.
>>>>> >> >> >>>>>>>>>>>>>> >>> I’ll try to find a workaround and get back
>>>>> with a results of this.
>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>> >> >> >>>>>>>>>>>>>> >>> [1] https://github.com/Talend/beam-samples/
>>>>> >> >> >>>>>>>>>>>>>> >>> [2]
>>>>> https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c
>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>> >> >> >>>>>>>>>>>>>> >>> —
>>>>> >> >> >>>>>>>>>>>>>> >>> Alexey
>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>> >> >> >>>>>>>>>>>>>> >>> On 24 Sep 2021, at 11:17, Jan Lukavský <
>>>>> je.ik@seznam.cz> wrote:
>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>> >> >> >>>>>>>>>>>>>> >>> +1 (non-binding)
>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>> >> >> >>>>>>>>>>>>>> >>> Validated several use-cases using
>>>>> non-portable Flink with Java SDK.
>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>> >> >> >>>>>>>>>>>>>> >>> On 9/24/21 4:55 AM, Valentyn Tymofieiev wrote:
>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>> >> >> >>>>>>>>>>>>>> >>> +1. Ran several Python batch and streaming
>>>>> pipelines on Dataflow and checked that Dataflow containers have required
>>>>> dependencies of Beam.
>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>> >> >> >>>>>>>>>>>>>> >>> On Thu, Sep 23, 2021 at 7:03 PM Robert Burke <
>>>>> lostluck@apache.org> wrote:
>>>>> >> >> >>>>>>>>>>>>>> >>>>
>>>>> >> >> >>>>>>>>>>>>>> >>>> +1 (non-binding)
>>>>> >> >> >>>>>>>>>>>>>> >>>> I validated the Go Quickstart (wordcount),
>>>>> and my ray tracer against the Go Direct runner, Dataflow, and Spark
>>>>> (ensuring the rc1 tagged container was used) and they executed successfully.
>>>>> >> >> >>>>>>>>>>>>>> >>>>
>>>>> >> >> >>>>>>>>>>>>>> >>>> I needed to manually synthesize a
>>>>> pseudo-version to ensure I was using the tagged branch version
>>>>> (v2.0.0-20210914211513-b358127f9859) instead of simply using v2.33.0-RC1.
>>>>> >> >> >>>>>>>>>>>>>> >>>>
>>>>> >> >> >>>>>>>>>>>>>> >>>>  It either can't find the package with the
>>>>> right tagged version, or it can't find the version. It's not clear to me
>>>>> what the issue is, but it's not notionally a release blocker. I'll
>>>>> investigate further once we have a full release, as it's probably some
>>>>> unspecified behavior due to how we transitioned to Go Modules (which
>>>>> strongly recommended doing a major version bump for such transitions, which
>>>>> seems a bit excessive for Beam as a whole...).
>>>>> >> >> >>>>>>>>>>>>>> >>>>
>>>>> >> >> >>>>>>>>>>>>>> >>>> On 2021/09/23 03:59:18, Ahmet Altay <
>>>>> altay@google.com> wrote:
>>>>> >> >> >>>>>>>>>>>>>> >>>> > +1 on the RC. I validated python quick
>>>>> start examples on direct runners.
>>>>> >> >> >>>>>>>>>>>>>> >>>> > Thank you Udi.
>>>>> >> >> >>>>>>>>>>>>>> >>>> >
>>>>> >> >> >>>>>>>>>>>>>> >>>> > On Wed, Sep 22, 2021 at 2:20 PM Robert
>>>>> Burke <ro...@frantil.com> wrote:
>>>>> >> >> >>>>>>>>>>>>>> >>>> >
>>>>> >> >> >>>>>>>>>>>>>> >>>> > > Just an FYI that intend to validate the
>>>>> Go SDK for this release but can't
>>>>> >> >> >>>>>>>>>>>>>> >>>> > > get to it until tomorrow (Thursday). I'm
>>>>> catching up from a week of
>>>>> >> >> >>>>>>>>>>>>>> >>>> > > vacation. Apologies for the
>>>>> inconvenience.
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >
>>>>> >> >> >>>>>>>>>>>>>> >>>> > > On Tue, Sep 21, 2021, 10:59 AM Udi Meiri
>>>>> <eh...@google.com> wrote:
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >> I updated the affected and fixed
>>>>> version fields for
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>
>>>>> https://issues.apache.org/jira/browse/BEAM-12356.
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >> On Tue, Sep 21, 2021 at 10:48 AM Reuven
>>>>> Lax <re...@google.com> wrote:
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>> Unfortunate - I didn't realize that
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>
>>>>> https://github.com/apache/beam/pull/15480 didn't make the cut.
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>> This bug was a regression in Beam
>>>>> 2.32.0, and is blocking multiple users
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>> from updating.
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi
>>>>> Meiri <ud...@apache.org> wrote:
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> Hi everyone,
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> Please review and vote on the release
>>>>> candidate #1 for the version
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> 2.33.0, as follows:
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [ ] +1, Approve the release
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [ ] -1, Do not approve the release
>>>>> (please provide specific comments)
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> Reviewers are encouraged to test
>>>>> their own use cases with the release
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> candidate, and vote +1 if
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> no issues are found.
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> The complete staging area is
>>>>> available for your review, which includes:
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * JIRA release notes [1],
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * the official Apache source release
>>>>> to be deployed to dist.apache.org
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [2], which is signed with the key
>>>>> with fingerprint 587B049C36DAAFE6 [3],
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * all artifacts to be deployed to the
>>>>> Maven Central Repository [4],
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * source code tag "v2.33.0-RC1" [5],
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * website pull request listing the
>>>>> release [6], the blog post [6], and
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> publishing the API reference manual
>>>>> [7].
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * Java artifacts were built with
>>>>> Maven 3.6.3 and OpenJDK 1.8.0_181.
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * Python artifacts are deployed along
>>>>> with the source release to the
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> dist.apache.org [2] and pypy[8].
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * Validation sheet with a tab for
>>>>> 2.33.0 release to help with
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> validation [9].
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * Docker images published to Docker
>>>>> Hub [10].
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> The vote will be open for at least 72
>>>>> hours. It is adopted by majority
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> approval, with at least 3 PMC
>>>>> affirmative votes.
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> For guidelines on how to try the
>>>>> release in your projects, check out
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> our blog post at
>>>>> https://beam.apache.org/blog/validate-beam-release/.
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> Thanks,
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> Release Manager
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [1]
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [2]
>>>>> https://dist.apache.org/repos/dist/dev/beam/2.33.0/
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [3]
>>>>> https://dist.apache.org/repos/dist/release/beam/KEYS
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [4]
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>> https://repository.apache.org/content/repositories/orgapachebeam-1234/
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [5]
>>>>> https://github.com/apache/beam/tree/v2.33.0-RC1
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [6]
>>>>> https://github.com/apache/beam/pull/15543
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [7]
>>>>> https://github.com/apache/beam-site/pull/619
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [8]
>>>>> https://pypi.org/project/apache-beam/2.33.0rc1/
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [9]
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [10]
>>>>> https://hub.docker.com/search?q=apache%2Fbeam&type=image
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>>> >> >> >>>>>>>>>>>>>> >>>> >
>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>> >> >> >>>>>>>>>>>>>> >>>
>>>>> >> >> >>>>>>>>>>>>>> >>
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Zachary Houfek
>>>>
>>>> Software Engineer
>>>>
>>>> DataPLS PLAT
>>>>
>>>> zhoufek@google.com
>>>>
>>>
>
> --
>
> Zachary Houfek
>
> Software Engineer
>
> DataPLS PLAT
>
> zhoufek@google.com
>

Re: [VOTE] Release 2.33.0, release candidate 1

[Zachary Houfek <zh...@google.com>]

> AfterProcessingTime and AfterCount should both be flagged...AfterEach is
different, since if any of the sequenced triggers never finish, the whole
thing will never finish

Ah, looks like another necessary fix. AfterEach will be flagged if any
should be flagged. That and AfterProcessingTime should be ready soon.

Regards,
Zach

On Mon, Sep 27, 2021 at 7:36 PM Robert Bradshaw <ro...@google.com> wrote:

> OK, given the fact that AfterWatermark(...) implicitly invokes Repeatedly
> as well, and the sheer volume of concern/discussion on this issue, I now
> change my vote to a -1 and think it's best we just cut a new release
> with the fix in it.
>
>
> On Mon, Sep 27, 2021 at 4:06 PM Kenneth Knowles <ke...@apache.org> wrote:
>
>> Ah, thanks for the correction. AfterProcessingTime and AfterCount should
>> both be flagged, as should AfterEvery, and AfterAny. AfterEach is
>> different, since if any of the sequenced triggers never finish, the whole
>> thing will never finish.
>>
>> Kenn
>>
>> On Mon, Sep 27, 2021 at 3:51 PM Zachary Houfek <zh...@google.com>
>> wrote:
>>
>>> > This was discovered because Repeatedly(AfterProcessingTime(n))
>>>
>>> Wasn't it Repeatedly(AfterCount(n)) that caused the problem? At least
>>> that's what caused the failing test that prompted the fix.
>>>
>>> With that said, is the concern that AfterProcessingTime *should* include
>>> the CONDITION_NOT_GUARANTEED flag? It isn't right now, so I'm checking.
>>>
>>> Another thing I haven't seen mentioned: `Repeatedly` is or can be used
>>> in other triggers like AfterWatermark (because of the late trigger),
>>> AfterEach, AfterEvery, and AfterAny, so the bug will propagate to those
>>> triggers as well. Two of these actually had to have their tests updated
>>> in the fix
>>> <https://github.com/apache/beam/commit/533a63299789ec8b4bbe629c95734be8dd4d9589#diff-9278a08a4f97eb63d783206bdea1a05377b9edd41646255110446512b8adad43>
>>> .
>>>
>>> Regards,
>>> Zach
>>>
>>>
>>>
>>> On Mon, Sep 27, 2021 at 6:30 PM Robert Bradshaw <ro...@google.com>
>>> wrote:
>>>
>>>> $ git checkout release-2.33.0
>>>> Switched to branch 'release-2.33.0'
>>>> Your branch is up to date with 'github/release-2.33.0'.
>>>>
>>>> $ python -c 'import apache_beam as beam;
>>>>
>>>> print(beam.transforms.trigger.Repeatedly(beam.transforms.trigger.AfterProcessingTime(5)).may_lose_data(None))'
>>>> DataLossReason.NO_POTENTIAL_LOSS
>>>>
>>>> The MAY_FINISH and CONDITION_NOT_GUARANTEED flags are disjoint.
>>>>
>>>> On Mon, Sep 27, 2021 at 3:15 PM Kenneth Knowles <ke...@apache.org>
>>>> wrote:
>>>> >
>>>> > This was discovered because Repeatedly(AfterProcessingTime(n)) does
>>>> require the workaround, because AfterProcessingTime(n) does terminate and
>>>> that is propagated by the Repeatedly implementation.
>>>> >
>>>> > Kenn
>>>> >
>>>> > On Mon, Sep 27, 2021 at 2:49 PM Robert Bradshaw <ro...@google.com>
>>>> wrote:
>>>> >>
>>>> >> On Mon, Sep 27, 2021 at 2:42 PM Kenneth Knowles <ke...@apache.org>
>>>> wrote:
>>>> >> >
>>>> >> > True. It is the use case of Repeatedly(ProcessingTime(n)) that I
>>>> would guess to be the primary case of concern.
>>>> >>
>>>> >> This shouldn't require the workaround, right?
>>>> >>
>>>> >> If you're not comfortable with this, I'd say let's patch and re-cut.
>>>> >>
>>>> >>
>>>> >> > On Mon, Sep 27, 2021 at 2:38 PM Robert Bradshaw <
>>>> robertwb@google.com> wrote:
>>>> >> >>
>>>> >> >> As I read the code, it's only pipelines that use a Repeatedly
>>>> trigger
>>>> >> >> that wrap an already lossy trigger that are declared to be
>>>> themselves
>>>> >> >> lossy. If I'm mistaken, I'll certainly reconsider my vote (and
>>>> thanks
>>>> >> >> for bringing this up).
>>>> >> >>
>>>> >> >> On Mon, Sep 27, 2021 at 2:21 PM Kenneth Knowles <ke...@apache.org>
>>>> wrote:
>>>> >> >> >
>>>> >> >> > My concern is that the error message is incorrect and every
>>>> user of 2.33.0 may be educated wrong, or be worried about data loss in
>>>> Beam, or fail to find the blog post or CHANGES, etc.
>>>> >> >> >
>>>> >> >> > Kenn
>>>> >> >> >
>>>> >> >> > On Mon, Sep 27, 2021 at 2:16 PM Udi Meiri <eh...@google.com>
>>>> wrote:
>>>> >> >> >>
>>>> >> >> >> I don't know how rare it is, but there is a flag documented in
>>>> CHANGES and blog post that reverts to the old behavior.
>>>> >> >> >>
>>>> >> >> >> On Mon, Sep 27, 2021 at 2:12 PM Kenneth Knowles <
>>>> kenn@apache.org> wrote:
>>>> >> >> >>>
>>>> >> >> >>> I guess my vote is -0 since I don't have enough context on
>>>> this issue. A number of people with more awareness of how severe this is
>>>> have voted +1 so I will not try to block the release.
>>>> >> >> >>>
>>>> >> >> >>> Kenn
>>>> >> >> >>>
>>>> >> >> >>> On Mon, Sep 27, 2021 at 2:11 PM Kenneth Knowles <
>>>> kenn@apache.org> wrote:
>>>> >> >> >>>>
>>>> >> >> >>>> I have to disagree with the other PMC members here, or at
>>>> least dig in to the question: every pipeline that uses a Repeatedly trigger
>>>> at the top level will be rejected. Is this so rare in Python that it is OK?
>>>> >> >> >>>>
>>>> >> >> >>>> Kenn
>>>> >> >> >>>>
>>>> >> >> >>>> On Mon, Sep 27, 2021 at 1:56 PM Robert Burke <
>>>> rebo@google.com> wrote:
>>>> >> >> >>>>>
>>>> >> >> >>>>> On it. Thanks!
>>>> >> >> >>>>>
>>>> >> >> >>>>> On Mon, Sep 27, 2021 at 1:18 PM Udi Meiri <eh...@google.com>
>>>> wrote:
>>>> >> >> >>>>>>
>>>> >> >> >>>>>> Daniel/Robert, feel free to make changes to this PR:
>>>> https://github.com/apache/beam/pull/15543
>>>> >> >> >>>>>>
>>>> >> >> >>>>>> On Mon, Sep 27, 2021 at 12:08 PM Daniel Oliveira <
>>>> danoliveira@google.com> wrote:
>>>> >> >> >>>>>>>
>>>> >> >> >>>>>>> +1
>>>> >> >> >>>>>>>
>>>> >> >> >>>>>>> I hadn't realized the pipelines were still finishing
>>>> successfully. I retried wordcount with that in mind and confirmed it
>>>> finishes successfully, so this isn't a blocker.
>>>> >> >> >>>>>>>
>>>> >> >> >>>>>>> Although maybe we should add this to the "Known Issues"
>>>> because I can easily see those messages being interpreted as a pipeline
>>>> failure.
>>>> >> >> >>>>>>>
>>>> >> >> >>>>>>> On Sun, Sep 26, 2021 at 7:26 PM Robert Burke <
>>>> robert@frantil.com> wrote:
>>>> >> >> >>>>>>>>
>>>> >> >> >>>>>>>> That's on the SDK side, and it just means the
>>>> PCollection metrics are being returned, buy not handled by the SDK. At
>>>> present the pipeline results only handle PTransform metrics.
>>>> >> >> >>>>>>>>
>>>> >> >> >>>>>>>> As such it's not a regression, as adding those is still
>>>> under development.
>>>> >> >> >>>>>>>>
>>>> >> >> >>>>>>>> On Sun, Sep 26, 2021, 7:02 PM Daniel Oliveira <
>>>> danoliveira@google.com> wrote:
>>>> >> >> >>>>>>>>>
>>>> >> >> >>>>>>>>> I tried validating wordcount with the Go SDK on Flink.
>>>> The pipeline failed with a wall of errors like the following. I tried this
>>>> on Flink 1.11, 1.12, and 1.13 job servers built from source at the RC1
>>>> commit, same errors on all of them.
>>>> >> >> >>>>>>>>>
>>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>>> MonitoringInfo: urn:"beam:metric:element_count:v1"
>>>> type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"
>>>> labels:{key:"PCOLLECTION"  value:"n9"}
>>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>>> MonitoringInfo: urn:"beam:metric:element_count:v1"
>>>> type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"
>>>> labels:{key:"PCOLLECTION"  value:"n10"}
>>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>>> MonitoringInfo: urn:"beam:metric:element_count:v1"
>>>> type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"
>>>> labels:{key:"PCOLLECTION"  value:"n8"}
>>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>>> MonitoringInfo: urn:"beam:metric:element_count:v1"
>>>> type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"
>>>> labels:{key:"PCOLLECTION"  value:"n7"}
>>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>>> MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"
>>>> type:"beam:metrics:distribution_int64:v1"
>>>> payload:"\x8d%\xfa\xbf\x05\x04\xa8\x0c"  labels:{key:"PCOLLECTION"
>>>> value:"n7"}
>>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>>> MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"
>>>> type:"beam:metrics:distribution_int64:v1"  payload:"\xb9\x05\xcf6\x05\x11"
>>>> labels:{key:"PCOLLECTION"  value:"n9"}
>>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>>> MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"
>>>> type:"beam:metrics:distribution_int64:v1"  payload:"\xc5\x05\x8a1\x04\x12"
>>>> labels:{key:"PCOLLECTION"  value:"n8"}
>>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>>> MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"
>>>> type:"beam:metrics:distribution_int64:v1"
>>>> payload:"\x8d%\xb3\xa7\x07\x14\""  labels:{key:"PCOLLECTION"  value:"n10"}
>>>> >> >> >>>>>>>>>> {...}
>>>> >> >> >>>>>>>>>
>>>> >> >> >>>>>>>>>
>>>> >> >> >>>>>>>>>  @Robert Burke I think you might know what's going on
>>>> here. Is this solvable with a cherry-pick and a new RC?
>>>> >> >> >>>>>>>>>
>>>> >> >> >>>>>>>>> On Fri, Sep 24, 2021 at 4:25 PM Ahmet Altay <
>>>> altay@google.com> wrote:
>>>> >> >> >>>>>>>>>>
>>>> >> >> >>>>>>>>>>
>>>> >> >> >>>>>>>>>>
>>>> >> >> >>>>>>>>>> On Fri, Sep 24, 2021 at 4:21 PM Udi Meiri <
>>>> ehudm@google.com> wrote:
>>>> >> >> >>>>>>>>>>>
>>>> >> >> >>>>>>>>>>>
>>>> >> >> >>>>>>>>>>>
>>>> >> >> >>>>>>>>>>> On Fri, Sep 24, 2021 at 4:18 PM Ahmet Altay <
>>>> altay@google.com> wrote:
>>>> >> >> >>>>>>>>>>>>
>>>> >> >> >>>>>>>>>>>> Thank you for the update related to
>>>> allow_unsafe_triggers. My vote is still a +1.
>>>> >> >> >>>>>>>>>>>>
>>>> >> >> >>>>>>>>>>>> If we decide to move forward with this RC, could you
>>>> please include this bug in the known issues list under the changes.md for
>>>> this release?
>>>> >> >> >>>>>>>>>>>
>>>> >> >> >>>>>>>>>>> Yes, that's included in
>>>> https://github.com/apache/beam/pull/15543/files.
>>>> >> >> >>>>>>>>>>
>>>> >> >> >>>>>>>>>>
>>>> >> >> >>>>>>>>>> Thank you!
>>>> >> >> >>>>>>>>>>
>>>> >> >> >>>>>>>>>>>
>>>> >> >> >>>>>>>>>>>
>>>> >> >> >>>>>>>>>>>>
>>>> >> >> >>>>>>>>>>>>
>>>> >> >> >>>>>>>>>>>> On Fri, Sep 24, 2021 at 3:56 PM Chamikara Jayalath <
>>>> chamikara@google.com> wrote:
>>>> >> >> >>>>>>>>>>>>>
>>>> >> >> >>>>>>>>>>>>> +1 (binding)
>>>> >> >> >>>>>>>>>>>>> Validated a few scenarios from the spreadsheet.
>>>> >> >> >>>>>>>>>>>>>
>>>> >> >> >>>>>>>>>>>>> Thanks,
>>>> >> >> >>>>>>>>>>>>> Cham
>>>> >> >> >>>>>>>>>>>>>
>>>> >> >> >>>>>>>>>>>>> On Fri, Sep 24, 2021 at 3:07 PM Robert Bradshaw <
>>>> robertwb@google.com> wrote:
>>>> >> >> >>>>>>>>>>>>>>
>>>> >> >> >>>>>>>>>>>>>> +1 (binding)
>>>> >> >> >>>>>>>>>>>>>>
>>>> >> >> >>>>>>>>>>>>>> All the artifacts and signatures look good.
>>>> >> >> >>>>>>>>>>>>>>
>>>> >> >> >>>>>>>>>>>>>> I don't think the unsafe trigger check is severe
>>>> enough to block the release.
>>>> >> >> >>>>>>>>>>>>>>
>>>> >> >> >>>>>>>>>>>>>> On Fri, Sep 24, 2021 at 2:36 PM Udi Meiri <
>>>> ehudm@google.com> wrote:
>>>> >> >> >>>>>>>>>>>>>> >
>>>> >> >> >>>>>>>>>>>>>> > Hi everyone, we found a bug during testing. It
>>>> has to do with Python SDK's allow_unsafe_triggers check.
>>>> >> >> >>>>>>>>>>>>>> > There is a preliminary fix that will go to
>>>> master.
>>>> >> >> >>>>>>>>>>>>>> >
>>>> >> >> >>>>>>>>>>>>>> > For the 2.33.0 release, I'm leaning towards not
>>>> making a new RC since there is a workaround: pass the flag
>>>> --allow_unsafe_triggers.
>>>> >> >> >>>>>>>>>>>>>> >
>>>> >> >> >>>>>>>>>>>>>> > Please reevaluate your votes accordingly and
>>>> recast if you've changed your vote.
>>>> >> >> >>>>>>>>>>>>>> >
>>>> >> >> >>>>>>>>>>>>>> > Thanks
>>>> >> >> >>>>>>>>>>>>>> >
>>>> >> >> >>>>>>>>>>>>>> > On Fri, Sep 24, 2021 at 12:48 PM Alexey
>>>> Romanenko <ar...@gmail.com> wrote:
>>>> >> >> >>>>>>>>>>>>>> >>
>>>> >> >> >>>>>>>>>>>>>> >>
>>>> >> >> >>>>>>>>>>>>>> >> On 24 Sep 2021, at 20:45, Udi Meiri <
>>>> ehudm@google.com> wrote:
>>>> >> >> >>>>>>>>>>>>>> >> Alexey is this something that we should put in
>>>> the release notes, or some other change?
>>>> >> >> >>>>>>>>>>>>>> >>
>>>> >> >> >>>>>>>>>>>>>> >>
>>>> >> >> >>>>>>>>>>>>>> >> Yes, I think it could be helpful to mention
>>>> that Beam Jackson’s deps was bumped and it may require an update of
>>>> Jackson’s runtime deps for Spark 2 users as well.
>>>> >> >> >>>>>>>>>>>>>> >>
>>>> >> >> >>>>>>>>>>>>>> >> —
>>>> >> >> >>>>>>>>>>>>>> >> Alexey
>>>> >> >> >>>>>>>>>>>>>> >>
>>>> >> >> >>>>>>>>>>>>>> >>
>>>> >> >> >>>>>>>>>>>>>> >>
>>>> >> >> >>>>>>>>>>>>>> >>>
>>>> >> >> >>>>>>>>>>>>>> >>>
>>>> >> >> >>>>>>>>>>>>>> >>> —
>>>> >> >> >>>>>>>>>>>>>> >>> Alexey
>>>> >> >> >>>>>>>>>>>>>> >>>
>>>> >> >> >>>>>>>>>>>>>> >>> On 24 Sep 2021, at 16:17, Alexey Romanenko <
>>>> aromanenko.dev@gmail.com> wrote:
>>>> >> >> >>>>>>>>>>>>>> >>>
>>>> >> >> >>>>>>>>>>>>>> >>> I checked with beam-samples [1] and noticed an
>>>> issue to run some pipelines with Spark 2 runner (Spark 3 seems is ok).
>>>> >> >> >>>>>>>>>>>>>> >>> It looks that it’s caused by new Jackson's
>>>> version updated recently [2], even if it’s a minor update but it works fine
>>>> with a previous one.
>>>> >> >> >>>>>>>>>>>>>> >>> I’ll try to find a workaround and get back
>>>> with a results of this.
>>>> >> >> >>>>>>>>>>>>>> >>>
>>>> >> >> >>>>>>>>>>>>>> >>> [1] https://github.com/Talend/beam-samples/
>>>> >> >> >>>>>>>>>>>>>> >>> [2]
>>>> https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c
>>>> >> >> >>>>>>>>>>>>>> >>>
>>>> >> >> >>>>>>>>>>>>>> >>> —
>>>> >> >> >>>>>>>>>>>>>> >>> Alexey
>>>> >> >> >>>>>>>>>>>>>> >>>
>>>> >> >> >>>>>>>>>>>>>> >>> On 24 Sep 2021, at 11:17, Jan Lukavský <
>>>> je.ik@seznam.cz> wrote:
>>>> >> >> >>>>>>>>>>>>>> >>>
>>>> >> >> >>>>>>>>>>>>>> >>> +1 (non-binding)
>>>> >> >> >>>>>>>>>>>>>> >>>
>>>> >> >> >>>>>>>>>>>>>> >>> Validated several use-cases using non-portable
>>>> Flink with Java SDK.
>>>> >> >> >>>>>>>>>>>>>> >>>
>>>> >> >> >>>>>>>>>>>>>> >>> On 9/24/21 4:55 AM, Valentyn Tymofieiev wrote:
>>>> >> >> >>>>>>>>>>>>>> >>>
>>>> >> >> >>>>>>>>>>>>>> >>> +1. Ran several Python batch and streaming
>>>> pipelines on Dataflow and checked that Dataflow containers have required
>>>> dependencies of Beam.
>>>> >> >> >>>>>>>>>>>>>> >>>
>>>> >> >> >>>>>>>>>>>>>> >>> On Thu, Sep 23, 2021 at 7:03 PM Robert Burke <
>>>> lostluck@apache.org> wrote:
>>>> >> >> >>>>>>>>>>>>>> >>>>
>>>> >> >> >>>>>>>>>>>>>> >>>> +1 (non-binding)
>>>> >> >> >>>>>>>>>>>>>> >>>> I validated the Go Quickstart (wordcount),
>>>> and my ray tracer against the Go Direct runner, Dataflow, and Spark
>>>> (ensuring the rc1 tagged container was used) and they executed successfully.
>>>> >> >> >>>>>>>>>>>>>> >>>>
>>>> >> >> >>>>>>>>>>>>>> >>>> I needed to manually synthesize a
>>>> pseudo-version to ensure I was using the tagged branch version
>>>> (v2.0.0-20210914211513-b358127f9859) instead of simply using v2.33.0-RC1.
>>>> >> >> >>>>>>>>>>>>>> >>>>
>>>> >> >> >>>>>>>>>>>>>> >>>>  It either can't find the package with the
>>>> right tagged version, or it can't find the version. It's not clear to me
>>>> what the issue is, but it's not notionally a release blocker. I'll
>>>> investigate further once we have a full release, as it's probably some
>>>> unspecified behavior due to how we transitioned to Go Modules (which
>>>> strongly recommended doing a major version bump for such transitions, which
>>>> seems a bit excessive for Beam as a whole...).
>>>> >> >> >>>>>>>>>>>>>> >>>>
>>>> >> >> >>>>>>>>>>>>>> >>>> On 2021/09/23 03:59:18, Ahmet Altay <
>>>> altay@google.com> wrote:
>>>> >> >> >>>>>>>>>>>>>> >>>> > +1 on the RC. I validated python quick
>>>> start examples on direct runners.
>>>> >> >> >>>>>>>>>>>>>> >>>> > Thank you Udi.
>>>> >> >> >>>>>>>>>>>>>> >>>> >
>>>> >> >> >>>>>>>>>>>>>> >>>> > On Wed, Sep 22, 2021 at 2:20 PM Robert
>>>> Burke <ro...@frantil.com> wrote:
>>>> >> >> >>>>>>>>>>>>>> >>>> >
>>>> >> >> >>>>>>>>>>>>>> >>>> > > Just an FYI that intend to validate the
>>>> Go SDK for this release but can't
>>>> >> >> >>>>>>>>>>>>>> >>>> > > get to it until tomorrow (Thursday). I'm
>>>> catching up from a week of
>>>> >> >> >>>>>>>>>>>>>> >>>> > > vacation. Apologies for the inconvenience.
>>>> >> >> >>>>>>>>>>>>>> >>>> > >
>>>> >> >> >>>>>>>>>>>>>> >>>> > > On Tue, Sep 21, 2021, 10:59 AM Udi Meiri <
>>>> ehudm@google.com> wrote:
>>>> >> >> >>>>>>>>>>>>>> >>>> > >
>>>> >> >> >>>>>>>>>>>>>> >>>> > >> I updated the affected and fixed version
>>>> fields for
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>
>>>> https://issues.apache.org/jira/browse/BEAM-12356.
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>
>>>> >> >> >>>>>>>>>>>>>> >>>> > >> On Tue, Sep 21, 2021 at 10:48 AM Reuven
>>>> Lax <re...@google.com> wrote:
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>> Unfortunate - I didn't realize that
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>
>>>> https://github.com/apache/beam/pull/15480 didn't make the cut.
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>> This bug was a regression in Beam
>>>> 2.32.0, and is blocking multiple users
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>> from updating.
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi
>>>> Meiri <ud...@apache.org> wrote:
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> Hi everyone,
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> Please review and vote on the release
>>>> candidate #1 for the version
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> 2.33.0, as follows:
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [ ] +1, Approve the release
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [ ] -1, Do not approve the release
>>>> (please provide specific comments)
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> Reviewers are encouraged to test their
>>>> own use cases with the release
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> candidate, and vote +1 if
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> no issues are found.
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> The complete staging area is available
>>>> for your review, which includes:
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * JIRA release notes [1],
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * the official Apache source release
>>>> to be deployed to dist.apache.org
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [2], which is signed with the key with
>>>> fingerprint 587B049C36DAAFE6 [3],
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * all artifacts to be deployed to the
>>>> Maven Central Repository [4],
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * source code tag "v2.33.0-RC1" [5],
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * website pull request listing the
>>>> release [6], the blog post [6], and
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> publishing the API reference manual
>>>> [7].
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * Java artifacts were built with Maven
>>>> 3.6.3 and OpenJDK 1.8.0_181.
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * Python artifacts are deployed along
>>>> with the source release to the
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> dist.apache.org [2] and pypy[8].
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * Validation sheet with a tab for
>>>> 2.33.0 release to help with
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> validation [9].
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * Docker images published to Docker
>>>> Hub [10].
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> The vote will be open for at least 72
>>>> hours. It is adopted by majority
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> approval, with at least 3 PMC
>>>> affirmative votes.
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> For guidelines on how to try the
>>>> release in your projects, check out
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> our blog post at
>>>> https://beam.apache.org/blog/validate-beam-release/.
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> Thanks,
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> Release Manager
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [1]
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [2]
>>>> https://dist.apache.org/repos/dist/dev/beam/2.33.0/
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [3]
>>>> https://dist.apache.org/repos/dist/release/beam/KEYS
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [4]
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>> https://repository.apache.org/content/repositories/orgapachebeam-1234/
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [5]
>>>> https://github.com/apache/beam/tree/v2.33.0-RC1
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [6]
>>>> https://github.com/apache/beam/pull/15543
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [7]
>>>> https://github.com/apache/beam-site/pull/619
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [8]
>>>> https://pypi.org/project/apache-beam/2.33.0rc1/
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [9]
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [10]
>>>> https://hub.docker.com/search?q=apache%2Fbeam&type=image
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>>> >> >> >>>>>>>>>>>>>> >>>> >
>>>> >> >> >>>>>>>>>>>>>> >>>
>>>> >> >> >>>>>>>>>>>>>> >>>
>>>> >> >> >>>>>>>>>>>>>> >>>
>>>> >> >> >>>>>>>>>>>>>> >>
>>>>
>>>
>>>
>>> --
>>>
>>> Zachary Houfek
>>>
>>> Software Engineer
>>>
>>> DataPLS PLAT
>>>
>>> zhoufek@google.com
>>>
>>

-- 

Zachary Houfek

Software Engineer

DataPLS PLAT

zhoufek@google.com

Re: [VOTE] Release 2.33.0, release candidate 1

[Robert Bradshaw <ro...@google.com>]

OK, given the fact that AfterWatermark(...) implicitly invokes Repeatedly
as well, and the sheer volume of concern/discussion on this issue, I now
change my vote to a -1 and think it's best we just cut a new release
with the fix in it.


On Mon, Sep 27, 2021 at 4:06 PM Kenneth Knowles <ke...@apache.org> wrote:

> Ah, thanks for the correction. AfterProcessingTime and AfterCount should
> both be flagged, as should AfterEvery, and AfterAny. AfterEach is
> different, since if any of the sequenced triggers never finish, the whole
> thing will never finish.
>
> Kenn
>
> On Mon, Sep 27, 2021 at 3:51 PM Zachary Houfek <zh...@google.com> wrote:
>
>> > This was discovered because Repeatedly(AfterProcessingTime(n))
>>
>> Wasn't it Repeatedly(AfterCount(n)) that caused the problem? At least
>> that's what caused the failing test that prompted the fix.
>>
>> With that said, is the concern that AfterProcessingTime *should* include
>> the CONDITION_NOT_GUARANTEED flag? It isn't right now, so I'm checking.
>>
>> Another thing I haven't seen mentioned: `Repeatedly` is or can be used in
>> other triggers like AfterWatermark (because of the late trigger),
>> AfterEach, AfterEvery, and AfterAny, so the bug will propagate to those
>> triggers as well. Two of these actually had to have their tests updated
>> in the fix
>> <https://github.com/apache/beam/commit/533a63299789ec8b4bbe629c95734be8dd4d9589#diff-9278a08a4f97eb63d783206bdea1a05377b9edd41646255110446512b8adad43>
>> .
>>
>> Regards,
>> Zach
>>
>>
>>
>> On Mon, Sep 27, 2021 at 6:30 PM Robert Bradshaw <ro...@google.com>
>> wrote:
>>
>>> $ git checkout release-2.33.0
>>> Switched to branch 'release-2.33.0'
>>> Your branch is up to date with 'github/release-2.33.0'.
>>>
>>> $ python -c 'import apache_beam as beam;
>>>
>>> print(beam.transforms.trigger.Repeatedly(beam.transforms.trigger.AfterProcessingTime(5)).may_lose_data(None))'
>>> DataLossReason.NO_POTENTIAL_LOSS
>>>
>>> The MAY_FINISH and CONDITION_NOT_GUARANTEED flags are disjoint.
>>>
>>> On Mon, Sep 27, 2021 at 3:15 PM Kenneth Knowles <ke...@apache.org> wrote:
>>> >
>>> > This was discovered because Repeatedly(AfterProcessingTime(n)) does
>>> require the workaround, because AfterProcessingTime(n) does terminate and
>>> that is propagated by the Repeatedly implementation.
>>> >
>>> > Kenn
>>> >
>>> > On Mon, Sep 27, 2021 at 2:49 PM Robert Bradshaw <ro...@google.com>
>>> wrote:
>>> >>
>>> >> On Mon, Sep 27, 2021 at 2:42 PM Kenneth Knowles <ke...@apache.org>
>>> wrote:
>>> >> >
>>> >> > True. It is the use case of Repeatedly(ProcessingTime(n)) that I
>>> would guess to be the primary case of concern.
>>> >>
>>> >> This shouldn't require the workaround, right?
>>> >>
>>> >> If you're not comfortable with this, I'd say let's patch and re-cut.
>>> >>
>>> >>
>>> >> > On Mon, Sep 27, 2021 at 2:38 PM Robert Bradshaw <
>>> robertwb@google.com> wrote:
>>> >> >>
>>> >> >> As I read the code, it's only pipelines that use a Repeatedly
>>> trigger
>>> >> >> that wrap an already lossy trigger that are declared to be
>>> themselves
>>> >> >> lossy. If I'm mistaken, I'll certainly reconsider my vote (and
>>> thanks
>>> >> >> for bringing this up).
>>> >> >>
>>> >> >> On Mon, Sep 27, 2021 at 2:21 PM Kenneth Knowles <ke...@apache.org>
>>> wrote:
>>> >> >> >
>>> >> >> > My concern is that the error message is incorrect and every user
>>> of 2.33.0 may be educated wrong, or be worried about data loss in Beam, or
>>> fail to find the blog post or CHANGES, etc.
>>> >> >> >
>>> >> >> > Kenn
>>> >> >> >
>>> >> >> > On Mon, Sep 27, 2021 at 2:16 PM Udi Meiri <eh...@google.com>
>>> wrote:
>>> >> >> >>
>>> >> >> >> I don't know how rare it is, but there is a flag documented in
>>> CHANGES and blog post that reverts to the old behavior.
>>> >> >> >>
>>> >> >> >> On Mon, Sep 27, 2021 at 2:12 PM Kenneth Knowles <
>>> kenn@apache.org> wrote:
>>> >> >> >>>
>>> >> >> >>> I guess my vote is -0 since I don't have enough context on
>>> this issue. A number of people with more awareness of how severe this is
>>> have voted +1 so I will not try to block the release.
>>> >> >> >>>
>>> >> >> >>> Kenn
>>> >> >> >>>
>>> >> >> >>> On Mon, Sep 27, 2021 at 2:11 PM Kenneth Knowles <
>>> kenn@apache.org> wrote:
>>> >> >> >>>>
>>> >> >> >>>> I have to disagree with the other PMC members here, or at
>>> least dig in to the question: every pipeline that uses a Repeatedly trigger
>>> at the top level will be rejected. Is this so rare in Python that it is OK?
>>> >> >> >>>>
>>> >> >> >>>> Kenn
>>> >> >> >>>>
>>> >> >> >>>> On Mon, Sep 27, 2021 at 1:56 PM Robert Burke <re...@google.com>
>>> wrote:
>>> >> >> >>>>>
>>> >> >> >>>>> On it. Thanks!
>>> >> >> >>>>>
>>> >> >> >>>>> On Mon, Sep 27, 2021 at 1:18 PM Udi Meiri <eh...@google.com>
>>> wrote:
>>> >> >> >>>>>>
>>> >> >> >>>>>> Daniel/Robert, feel free to make changes to this PR:
>>> https://github.com/apache/beam/pull/15543
>>> >> >> >>>>>>
>>> >> >> >>>>>> On Mon, Sep 27, 2021 at 12:08 PM Daniel Oliveira <
>>> danoliveira@google.com> wrote:
>>> >> >> >>>>>>>
>>> >> >> >>>>>>> +1
>>> >> >> >>>>>>>
>>> >> >> >>>>>>> I hadn't realized the pipelines were still finishing
>>> successfully. I retried wordcount with that in mind and confirmed it
>>> finishes successfully, so this isn't a blocker.
>>> >> >> >>>>>>>
>>> >> >> >>>>>>> Although maybe we should add this to the "Known Issues"
>>> because I can easily see those messages being interpreted as a pipeline
>>> failure.
>>> >> >> >>>>>>>
>>> >> >> >>>>>>> On Sun, Sep 26, 2021 at 7:26 PM Robert Burke <
>>> robert@frantil.com> wrote:
>>> >> >> >>>>>>>>
>>> >> >> >>>>>>>> That's on the SDK side, and it just means the PCollection
>>> metrics are being returned, buy not handled by the SDK. At present the
>>> pipeline results only handle PTransform metrics.
>>> >> >> >>>>>>>>
>>> >> >> >>>>>>>> As such it's not a regression, as adding those is still
>>> under development.
>>> >> >> >>>>>>>>
>>> >> >> >>>>>>>> On Sun, Sep 26, 2021, 7:02 PM Daniel Oliveira <
>>> danoliveira@google.com> wrote:
>>> >> >> >>>>>>>>>
>>> >> >> >>>>>>>>> I tried validating wordcount with the Go SDK on Flink.
>>> The pipeline failed with a wall of errors like the following. I tried this
>>> on Flink 1.11, 1.12, and 1.13 job servers built from source at the RC1
>>> commit, same errors on all of them.
>>> >> >> >>>>>>>>>
>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>> MonitoringInfo: urn:"beam:metric:element_count:v1"
>>> type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"
>>> labels:{key:"PCOLLECTION"  value:"n9"}
>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>> MonitoringInfo: urn:"beam:metric:element_count:v1"
>>> type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"
>>> labels:{key:"PCOLLECTION"  value:"n10"}
>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>> MonitoringInfo: urn:"beam:metric:element_count:v1"
>>> type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"
>>> labels:{key:"PCOLLECTION"  value:"n8"}
>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>> MonitoringInfo: urn:"beam:metric:element_count:v1"
>>> type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"
>>> labels:{key:"PCOLLECTION"  value:"n7"}
>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>> MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"
>>> type:"beam:metrics:distribution_int64:v1"
>>> payload:"\x8d%\xfa\xbf\x05\x04\xa8\x0c"  labels:{key:"PCOLLECTION"
>>> value:"n7"}
>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>> MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"
>>> type:"beam:metrics:distribution_int64:v1"  payload:"\xb9\x05\xcf6\x05\x11"
>>> labels:{key:"PCOLLECTION"  value:"n9"}
>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>> MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"
>>> type:"beam:metrics:distribution_int64:v1"  payload:"\xc5\x05\x8a1\x04\x12"
>>> labels:{key:"PCOLLECTION"  value:"n8"}
>>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>>> MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"
>>> type:"beam:metrics:distribution_int64:v1"
>>> payload:"\x8d%\xb3\xa7\x07\x14\""  labels:{key:"PCOLLECTION"  value:"n10"}
>>> >> >> >>>>>>>>>> {...}
>>> >> >> >>>>>>>>>
>>> >> >> >>>>>>>>>
>>> >> >> >>>>>>>>>  @Robert Burke I think you might know what's going on
>>> here. Is this solvable with a cherry-pick and a new RC?
>>> >> >> >>>>>>>>>
>>> >> >> >>>>>>>>> On Fri, Sep 24, 2021 at 4:25 PM Ahmet Altay <
>>> altay@google.com> wrote:
>>> >> >> >>>>>>>>>>
>>> >> >> >>>>>>>>>>
>>> >> >> >>>>>>>>>>
>>> >> >> >>>>>>>>>> On Fri, Sep 24, 2021 at 4:21 PM Udi Meiri <
>>> ehudm@google.com> wrote:
>>> >> >> >>>>>>>>>>>
>>> >> >> >>>>>>>>>>>
>>> >> >> >>>>>>>>>>>
>>> >> >> >>>>>>>>>>> On Fri, Sep 24, 2021 at 4:18 PM Ahmet Altay <
>>> altay@google.com> wrote:
>>> >> >> >>>>>>>>>>>>
>>> >> >> >>>>>>>>>>>> Thank you for the update related to
>>> allow_unsafe_triggers. My vote is still a +1.
>>> >> >> >>>>>>>>>>>>
>>> >> >> >>>>>>>>>>>> If we decide to move forward with this RC, could you
>>> please include this bug in the known issues list under the changes.md for
>>> this release?
>>> >> >> >>>>>>>>>>>
>>> >> >> >>>>>>>>>>> Yes, that's included in
>>> https://github.com/apache/beam/pull/15543/files.
>>> >> >> >>>>>>>>>>
>>> >> >> >>>>>>>>>>
>>> >> >> >>>>>>>>>> Thank you!
>>> >> >> >>>>>>>>>>
>>> >> >> >>>>>>>>>>>
>>> >> >> >>>>>>>>>>>
>>> >> >> >>>>>>>>>>>>
>>> >> >> >>>>>>>>>>>>
>>> >> >> >>>>>>>>>>>> On Fri, Sep 24, 2021 at 3:56 PM Chamikara Jayalath <
>>> chamikara@google.com> wrote:
>>> >> >> >>>>>>>>>>>>>
>>> >> >> >>>>>>>>>>>>> +1 (binding)
>>> >> >> >>>>>>>>>>>>> Validated a few scenarios from the spreadsheet.
>>> >> >> >>>>>>>>>>>>>
>>> >> >> >>>>>>>>>>>>> Thanks,
>>> >> >> >>>>>>>>>>>>> Cham
>>> >> >> >>>>>>>>>>>>>
>>> >> >> >>>>>>>>>>>>> On Fri, Sep 24, 2021 at 3:07 PM Robert Bradshaw <
>>> robertwb@google.com> wrote:
>>> >> >> >>>>>>>>>>>>>>
>>> >> >> >>>>>>>>>>>>>> +1 (binding)
>>> >> >> >>>>>>>>>>>>>>
>>> >> >> >>>>>>>>>>>>>> All the artifacts and signatures look good.
>>> >> >> >>>>>>>>>>>>>>
>>> >> >> >>>>>>>>>>>>>> I don't think the unsafe trigger check is severe
>>> enough to block the release.
>>> >> >> >>>>>>>>>>>>>>
>>> >> >> >>>>>>>>>>>>>> On Fri, Sep 24, 2021 at 2:36 PM Udi Meiri <
>>> ehudm@google.com> wrote:
>>> >> >> >>>>>>>>>>>>>> >
>>> >> >> >>>>>>>>>>>>>> > Hi everyone, we found a bug during testing. It
>>> has to do with Python SDK's allow_unsafe_triggers check.
>>> >> >> >>>>>>>>>>>>>> > There is a preliminary fix that will go to master.
>>> >> >> >>>>>>>>>>>>>> >
>>> >> >> >>>>>>>>>>>>>> > For the 2.33.0 release, I'm leaning towards not
>>> making a new RC since there is a workaround: pass the flag
>>> --allow_unsafe_triggers.
>>> >> >> >>>>>>>>>>>>>> >
>>> >> >> >>>>>>>>>>>>>> > Please reevaluate your votes accordingly and
>>> recast if you've changed your vote.
>>> >> >> >>>>>>>>>>>>>> >
>>> >> >> >>>>>>>>>>>>>> > Thanks
>>> >> >> >>>>>>>>>>>>>> >
>>> >> >> >>>>>>>>>>>>>> > On Fri, Sep 24, 2021 at 12:48 PM Alexey Romanenko
>>> <ar...@gmail.com> wrote:
>>> >> >> >>>>>>>>>>>>>> >>
>>> >> >> >>>>>>>>>>>>>> >>
>>> >> >> >>>>>>>>>>>>>> >> On 24 Sep 2021, at 20:45, Udi Meiri <
>>> ehudm@google.com> wrote:
>>> >> >> >>>>>>>>>>>>>> >> Alexey is this something that we should put in
>>> the release notes, or some other change?
>>> >> >> >>>>>>>>>>>>>> >>
>>> >> >> >>>>>>>>>>>>>> >>
>>> >> >> >>>>>>>>>>>>>> >> Yes, I think it could be helpful to mention that
>>> Beam Jackson’s deps was bumped and it may require an update of Jackson’s
>>> runtime deps for Spark 2 users as well.
>>> >> >> >>>>>>>>>>>>>> >>
>>> >> >> >>>>>>>>>>>>>> >> —
>>> >> >> >>>>>>>>>>>>>> >> Alexey
>>> >> >> >>>>>>>>>>>>>> >>
>>> >> >> >>>>>>>>>>>>>> >>
>>> >> >> >>>>>>>>>>>>>> >>
>>> >> >> >>>>>>>>>>>>>> >>>
>>> >> >> >>>>>>>>>>>>>> >>>
>>> >> >> >>>>>>>>>>>>>> >>> —
>>> >> >> >>>>>>>>>>>>>> >>> Alexey
>>> >> >> >>>>>>>>>>>>>> >>>
>>> >> >> >>>>>>>>>>>>>> >>> On 24 Sep 2021, at 16:17, Alexey Romanenko <
>>> aromanenko.dev@gmail.com> wrote:
>>> >> >> >>>>>>>>>>>>>> >>>
>>> >> >> >>>>>>>>>>>>>> >>> I checked with beam-samples [1] and noticed an
>>> issue to run some pipelines with Spark 2 runner (Spark 3 seems is ok).
>>> >> >> >>>>>>>>>>>>>> >>> It looks that it’s caused by new Jackson's
>>> version updated recently [2], even if it’s a minor update but it works fine
>>> with a previous one.
>>> >> >> >>>>>>>>>>>>>> >>> I’ll try to find a workaround and get back with
>>> a results of this.
>>> >> >> >>>>>>>>>>>>>> >>>
>>> >> >> >>>>>>>>>>>>>> >>> [1] https://github.com/Talend/beam-samples/
>>> >> >> >>>>>>>>>>>>>> >>> [2]
>>> https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c
>>> >> >> >>>>>>>>>>>>>> >>>
>>> >> >> >>>>>>>>>>>>>> >>> —
>>> >> >> >>>>>>>>>>>>>> >>> Alexey
>>> >> >> >>>>>>>>>>>>>> >>>
>>> >> >> >>>>>>>>>>>>>> >>> On 24 Sep 2021, at 11:17, Jan Lukavský <
>>> je.ik@seznam.cz> wrote:
>>> >> >> >>>>>>>>>>>>>> >>>
>>> >> >> >>>>>>>>>>>>>> >>> +1 (non-binding)
>>> >> >> >>>>>>>>>>>>>> >>>
>>> >> >> >>>>>>>>>>>>>> >>> Validated several use-cases using non-portable
>>> Flink with Java SDK.
>>> >> >> >>>>>>>>>>>>>> >>>
>>> >> >> >>>>>>>>>>>>>> >>> On 9/24/21 4:55 AM, Valentyn Tymofieiev wrote:
>>> >> >> >>>>>>>>>>>>>> >>>
>>> >> >> >>>>>>>>>>>>>> >>> +1. Ran several Python batch and streaming
>>> pipelines on Dataflow and checked that Dataflow containers have required
>>> dependencies of Beam.
>>> >> >> >>>>>>>>>>>>>> >>>
>>> >> >> >>>>>>>>>>>>>> >>> On Thu, Sep 23, 2021 at 7:03 PM Robert Burke <
>>> lostluck@apache.org> wrote:
>>> >> >> >>>>>>>>>>>>>> >>>>
>>> >> >> >>>>>>>>>>>>>> >>>> +1 (non-binding)
>>> >> >> >>>>>>>>>>>>>> >>>> I validated the Go Quickstart (wordcount), and
>>> my ray tracer against the Go Direct runner, Dataflow, and Spark (ensuring
>>> the rc1 tagged container was used) and they executed successfully.
>>> >> >> >>>>>>>>>>>>>> >>>>
>>> >> >> >>>>>>>>>>>>>> >>>> I needed to manually synthesize a
>>> pseudo-version to ensure I was using the tagged branch version
>>> (v2.0.0-20210914211513-b358127f9859) instead of simply using v2.33.0-RC1.
>>> >> >> >>>>>>>>>>>>>> >>>>
>>> >> >> >>>>>>>>>>>>>> >>>>  It either can't find the package with the
>>> right tagged version, or it can't find the version. It's not clear to me
>>> what the issue is, but it's not notionally a release blocker. I'll
>>> investigate further once we have a full release, as it's probably some
>>> unspecified behavior due to how we transitioned to Go Modules (which
>>> strongly recommended doing a major version bump for such transitions, which
>>> seems a bit excessive for Beam as a whole...).
>>> >> >> >>>>>>>>>>>>>> >>>>
>>> >> >> >>>>>>>>>>>>>> >>>> On 2021/09/23 03:59:18, Ahmet Altay <
>>> altay@google.com> wrote:
>>> >> >> >>>>>>>>>>>>>> >>>> > +1 on the RC. I validated python quick start
>>> examples on direct runners.
>>> >> >> >>>>>>>>>>>>>> >>>> > Thank you Udi.
>>> >> >> >>>>>>>>>>>>>> >>>> >
>>> >> >> >>>>>>>>>>>>>> >>>> > On Wed, Sep 22, 2021 at 2:20 PM Robert Burke
>>> <ro...@frantil.com> wrote:
>>> >> >> >>>>>>>>>>>>>> >>>> >
>>> >> >> >>>>>>>>>>>>>> >>>> > > Just an FYI that intend to validate the Go
>>> SDK for this release but can't
>>> >> >> >>>>>>>>>>>>>> >>>> > > get to it until tomorrow (Thursday). I'm
>>> catching up from a week of
>>> >> >> >>>>>>>>>>>>>> >>>> > > vacation. Apologies for the inconvenience.
>>> >> >> >>>>>>>>>>>>>> >>>> > >
>>> >> >> >>>>>>>>>>>>>> >>>> > > On Tue, Sep 21, 2021, 10:59 AM Udi Meiri <
>>> ehudm@google.com> wrote:
>>> >> >> >>>>>>>>>>>>>> >>>> > >
>>> >> >> >>>>>>>>>>>>>> >>>> > >> I updated the affected and fixed version
>>> fields for
>>> >> >> >>>>>>>>>>>>>> >>>> > >>
>>> https://issues.apache.org/jira/browse/BEAM-12356.
>>> >> >> >>>>>>>>>>>>>> >>>> > >>
>>> >> >> >>>>>>>>>>>>>> >>>> > >>
>>> >> >> >>>>>>>>>>>>>> >>>> > >> On Tue, Sep 21, 2021 at 10:48 AM Reuven
>>> Lax <re...@google.com> wrote:
>>> >> >> >>>>>>>>>>>>>> >>>> > >>
>>> >> >> >>>>>>>>>>>>>> >>>> > >>> Unfortunate - I didn't realize that
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>
>>> https://github.com/apache/beam/pull/15480 didn't make the cut.
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>
>>> >> >> >>>>>>>>>>>>>> >>>> > >>> This bug was a regression in Beam
>>> 2.32.0, and is blocking multiple users
>>> >> >> >>>>>>>>>>>>>> >>>> > >>> from updating.
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>
>>> >> >> >>>>>>>>>>>>>> >>>> > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi
>>> Meiri <ud...@apache.org> wrote:
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> Hi everyone,
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> Please review and vote on the release
>>> candidate #1 for the version
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> 2.33.0, as follows:
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [ ] +1, Approve the release
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [ ] -1, Do not approve the release
>>> (please provide specific comments)
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> Reviewers are encouraged to test their
>>> own use cases with the release
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> candidate, and vote +1 if
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> no issues are found.
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> The complete staging area is available
>>> for your review, which includes:
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * JIRA release notes [1],
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * the official Apache source release to
>>> be deployed to dist.apache.org
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [2], which is signed with the key with
>>> fingerprint 587B049C36DAAFE6 [3],
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * all artifacts to be deployed to the
>>> Maven Central Repository [4],
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * source code tag "v2.33.0-RC1" [5],
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * website pull request listing the
>>> release [6], the blog post [6], and
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> publishing the API reference manual [7].
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * Java artifacts were built with Maven
>>> 3.6.3 and OpenJDK 1.8.0_181.
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * Python artifacts are deployed along
>>> with the source release to the
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> dist.apache.org [2] and pypy[8].
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * Validation sheet with a tab for
>>> 2.33.0 release to help with
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> validation [9].
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * Docker images published to Docker Hub
>>> [10].
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> The vote will be open for at least 72
>>> hours. It is adopted by majority
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> approval, with at least 3 PMC
>>> affirmative votes.
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> For guidelines on how to try the
>>> release in your projects, check out
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> our blog post at
>>> https://beam.apache.org/blog/validate-beam-release/.
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> Thanks,
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> Release Manager
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [1]
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [2]
>>> https://dist.apache.org/repos/dist/dev/beam/2.33.0/
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [3]
>>> https://dist.apache.org/repos/dist/release/beam/KEYS
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [4]
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>> https://repository.apache.org/content/repositories/orgapachebeam-1234/
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [5]
>>> https://github.com/apache/beam/tree/v2.33.0-RC1
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [6]
>>> https://github.com/apache/beam/pull/15543
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [7]
>>> https://github.com/apache/beam-site/pull/619
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [8]
>>> https://pypi.org/project/apache-beam/2.33.0rc1/
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [9]
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [10]
>>> https://hub.docker.com/search?q=apache%2Fbeam&type=image
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>>> >> >> >>>>>>>>>>>>>> >>>> >
>>> >> >> >>>>>>>>>>>>>> >>>
>>> >> >> >>>>>>>>>>>>>> >>>
>>> >> >> >>>>>>>>>>>>>> >>>
>>> >> >> >>>>>>>>>>>>>> >>
>>>
>>
>>
>> --
>>
>> Zachary Houfek
>>
>> Software Engineer
>>
>> DataPLS PLAT
>>
>> zhoufek@google.com
>>
>

Re: [VOTE] Release 2.33.0, release candidate 1

[Kenneth Knowles <ke...@apache.org>]

Ah, thanks for the correction. AfterProcessingTime and AfterCount should
both be flagged, as should AfterEvery, and AfterAny. AfterEach is
different, since if any of the sequenced triggers never finish, the whole
thing will never finish.

Kenn

On Mon, Sep 27, 2021 at 3:51 PM Zachary Houfek <zh...@google.com> wrote:

> > This was discovered because Repeatedly(AfterProcessingTime(n))
>
> Wasn't it Repeatedly(AfterCount(n)) that caused the problem? At least
> that's what caused the failing test that prompted the fix.
>
> With that said, is the concern that AfterProcessingTime *should* include
> the CONDITION_NOT_GUARANTEED flag? It isn't right now, so I'm checking.
>
> Another thing I haven't seen mentioned: `Repeatedly` is or can be used in
> other triggers like AfterWatermark (because of the late trigger),
> AfterEach, AfterEvery, and AfterAny, so the bug will propagate to those
> triggers as well. Two of these actually had to have their tests updated
> in the fix
> <https://github.com/apache/beam/commit/533a63299789ec8b4bbe629c95734be8dd4d9589#diff-9278a08a4f97eb63d783206bdea1a05377b9edd41646255110446512b8adad43>
> .
>
> Regards,
> Zach
>
>
>
> On Mon, Sep 27, 2021 at 6:30 PM Robert Bradshaw <ro...@google.com>
> wrote:
>
>> $ git checkout release-2.33.0
>> Switched to branch 'release-2.33.0'
>> Your branch is up to date with 'github/release-2.33.0'.
>>
>> $ python -c 'import apache_beam as beam;
>>
>> print(beam.transforms.trigger.Repeatedly(beam.transforms.trigger.AfterProcessingTime(5)).may_lose_data(None))'
>> DataLossReason.NO_POTENTIAL_LOSS
>>
>> The MAY_FINISH and CONDITION_NOT_GUARANTEED flags are disjoint.
>>
>> On Mon, Sep 27, 2021 at 3:15 PM Kenneth Knowles <ke...@apache.org> wrote:
>> >
>> > This was discovered because Repeatedly(AfterProcessingTime(n)) does
>> require the workaround, because AfterProcessingTime(n) does terminate and
>> that is propagated by the Repeatedly implementation.
>> >
>> > Kenn
>> >
>> > On Mon, Sep 27, 2021 at 2:49 PM Robert Bradshaw <ro...@google.com>
>> wrote:
>> >>
>> >> On Mon, Sep 27, 2021 at 2:42 PM Kenneth Knowles <ke...@apache.org>
>> wrote:
>> >> >
>> >> > True. It is the use case of Repeatedly(ProcessingTime(n)) that I
>> would guess to be the primary case of concern.
>> >>
>> >> This shouldn't require the workaround, right?
>> >>
>> >> If you're not comfortable with this, I'd say let's patch and re-cut.
>> >>
>> >>
>> >> > On Mon, Sep 27, 2021 at 2:38 PM Robert Bradshaw <ro...@google.com>
>> wrote:
>> >> >>
>> >> >> As I read the code, it's only pipelines that use a Repeatedly
>> trigger
>> >> >> that wrap an already lossy trigger that are declared to be
>> themselves
>> >> >> lossy. If I'm mistaken, I'll certainly reconsider my vote (and
>> thanks
>> >> >> for bringing this up).
>> >> >>
>> >> >> On Mon, Sep 27, 2021 at 2:21 PM Kenneth Knowles <ke...@apache.org>
>> wrote:
>> >> >> >
>> >> >> > My concern is that the error message is incorrect and every user
>> of 2.33.0 may be educated wrong, or be worried about data loss in Beam, or
>> fail to find the blog post or CHANGES, etc.
>> >> >> >
>> >> >> > Kenn
>> >> >> >
>> >> >> > On Mon, Sep 27, 2021 at 2:16 PM Udi Meiri <eh...@google.com>
>> wrote:
>> >> >> >>
>> >> >> >> I don't know how rare it is, but there is a flag documented in
>> CHANGES and blog post that reverts to the old behavior.
>> >> >> >>
>> >> >> >> On Mon, Sep 27, 2021 at 2:12 PM Kenneth Knowles <ke...@apache.org>
>> wrote:
>> >> >> >>>
>> >> >> >>> I guess my vote is -0 since I don't have enough context on this
>> issue. A number of people with more awareness of how severe this is have
>> voted +1 so I will not try to block the release.
>> >> >> >>>
>> >> >> >>> Kenn
>> >> >> >>>
>> >> >> >>> On Mon, Sep 27, 2021 at 2:11 PM Kenneth Knowles <
>> kenn@apache.org> wrote:
>> >> >> >>>>
>> >> >> >>>> I have to disagree with the other PMC members here, or at
>> least dig in to the question: every pipeline that uses a Repeatedly trigger
>> at the top level will be rejected. Is this so rare in Python that it is OK?
>> >> >> >>>>
>> >> >> >>>> Kenn
>> >> >> >>>>
>> >> >> >>>> On Mon, Sep 27, 2021 at 1:56 PM Robert Burke <re...@google.com>
>> wrote:
>> >> >> >>>>>
>> >> >> >>>>> On it. Thanks!
>> >> >> >>>>>
>> >> >> >>>>> On Mon, Sep 27, 2021 at 1:18 PM Udi Meiri <eh...@google.com>
>> wrote:
>> >> >> >>>>>>
>> >> >> >>>>>> Daniel/Robert, feel free to make changes to this PR:
>> https://github.com/apache/beam/pull/15543
>> >> >> >>>>>>
>> >> >> >>>>>> On Mon, Sep 27, 2021 at 12:08 PM Daniel Oliveira <
>> danoliveira@google.com> wrote:
>> >> >> >>>>>>>
>> >> >> >>>>>>> +1
>> >> >> >>>>>>>
>> >> >> >>>>>>> I hadn't realized the pipelines were still finishing
>> successfully. I retried wordcount with that in mind and confirmed it
>> finishes successfully, so this isn't a blocker.
>> >> >> >>>>>>>
>> >> >> >>>>>>> Although maybe we should add this to the "Known Issues"
>> because I can easily see those messages being interpreted as a pipeline
>> failure.
>> >> >> >>>>>>>
>> >> >> >>>>>>> On Sun, Sep 26, 2021 at 7:26 PM Robert Burke <
>> robert@frantil.com> wrote:
>> >> >> >>>>>>>>
>> >> >> >>>>>>>> That's on the SDK side, and it just means the PCollection
>> metrics are being returned, buy not handled by the SDK. At present the
>> pipeline results only handle PTransform metrics.
>> >> >> >>>>>>>>
>> >> >> >>>>>>>> As such it's not a regression, as adding those is still
>> under development.
>> >> >> >>>>>>>>
>> >> >> >>>>>>>> On Sun, Sep 26, 2021, 7:02 PM Daniel Oliveira <
>> danoliveira@google.com> wrote:
>> >> >> >>>>>>>>>
>> >> >> >>>>>>>>> I tried validating wordcount with the Go SDK on Flink.
>> The pipeline failed with a wall of errors like the following. I tried this
>> on Flink 1.11, 1.12, and 1.13 job servers built from source at the RC1
>> commit, same errors on all of them.
>> >> >> >>>>>>>>>
>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>> MonitoringInfo: urn:"beam:metric:element_count:v1"
>> type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"
>> labels:{key:"PCOLLECTION"  value:"n9"}
>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>> MonitoringInfo: urn:"beam:metric:element_count:v1"
>> type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"
>> labels:{key:"PCOLLECTION"  value:"n10"}
>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>> MonitoringInfo: urn:"beam:metric:element_count:v1"
>> type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"
>> labels:{key:"PCOLLECTION"  value:"n8"}
>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>> MonitoringInfo: urn:"beam:metric:element_count:v1"
>> type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"
>> labels:{key:"PCOLLECTION"  value:"n7"}
>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>> MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"
>> type:"beam:metrics:distribution_int64:v1"
>> payload:"\x8d%\xfa\xbf\x05\x04\xa8\x0c"  labels:{key:"PCOLLECTION"
>> value:"n7"}
>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>> MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"
>> type:"beam:metrics:distribution_int64:v1"  payload:"\xb9\x05\xcf6\x05\x11"
>> labels:{key:"PCOLLECTION"  value:"n9"}
>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>> MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"
>> type:"beam:metrics:distribution_int64:v1"  payload:"\xc5\x05\x8a1\x04\x12"
>> labels:{key:"PCOLLECTION"  value:"n8"}
>> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
>> MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"
>> type:"beam:metrics:distribution_int64:v1"
>> payload:"\x8d%\xb3\xa7\x07\x14\""  labels:{key:"PCOLLECTION"  value:"n10"}
>> >> >> >>>>>>>>>> {...}
>> >> >> >>>>>>>>>
>> >> >> >>>>>>>>>
>> >> >> >>>>>>>>>  @Robert Burke I think you might know what's going on
>> here. Is this solvable with a cherry-pick and a new RC?
>> >> >> >>>>>>>>>
>> >> >> >>>>>>>>> On Fri, Sep 24, 2021 at 4:25 PM Ahmet Altay <
>> altay@google.com> wrote:
>> >> >> >>>>>>>>>>
>> >> >> >>>>>>>>>>
>> >> >> >>>>>>>>>>
>> >> >> >>>>>>>>>> On Fri, Sep 24, 2021 at 4:21 PM Udi Meiri <
>> ehudm@google.com> wrote:
>> >> >> >>>>>>>>>>>
>> >> >> >>>>>>>>>>>
>> >> >> >>>>>>>>>>>
>> >> >> >>>>>>>>>>> On Fri, Sep 24, 2021 at 4:18 PM Ahmet Altay <
>> altay@google.com> wrote:
>> >> >> >>>>>>>>>>>>
>> >> >> >>>>>>>>>>>> Thank you for the update related to
>> allow_unsafe_triggers. My vote is still a +1.
>> >> >> >>>>>>>>>>>>
>> >> >> >>>>>>>>>>>> If we decide to move forward with this RC, could you
>> please include this bug in the known issues list under the changes.md for
>> this release?
>> >> >> >>>>>>>>>>>
>> >> >> >>>>>>>>>>> Yes, that's included in
>> https://github.com/apache/beam/pull/15543/files.
>> >> >> >>>>>>>>>>
>> >> >> >>>>>>>>>>
>> >> >> >>>>>>>>>> Thank you!
>> >> >> >>>>>>>>>>
>> >> >> >>>>>>>>>>>
>> >> >> >>>>>>>>>>>
>> >> >> >>>>>>>>>>>>
>> >> >> >>>>>>>>>>>>
>> >> >> >>>>>>>>>>>> On Fri, Sep 24, 2021 at 3:56 PM Chamikara Jayalath <
>> chamikara@google.com> wrote:
>> >> >> >>>>>>>>>>>>>
>> >> >> >>>>>>>>>>>>> +1 (binding)
>> >> >> >>>>>>>>>>>>> Validated a few scenarios from the spreadsheet.
>> >> >> >>>>>>>>>>>>>
>> >> >> >>>>>>>>>>>>> Thanks,
>> >> >> >>>>>>>>>>>>> Cham
>> >> >> >>>>>>>>>>>>>
>> >> >> >>>>>>>>>>>>> On Fri, Sep 24, 2021 at 3:07 PM Robert Bradshaw <
>> robertwb@google.com> wrote:
>> >> >> >>>>>>>>>>>>>>
>> >> >> >>>>>>>>>>>>>> +1 (binding)
>> >> >> >>>>>>>>>>>>>>
>> >> >> >>>>>>>>>>>>>> All the artifacts and signatures look good.
>> >> >> >>>>>>>>>>>>>>
>> >> >> >>>>>>>>>>>>>> I don't think the unsafe trigger check is severe
>> enough to block the release.
>> >> >> >>>>>>>>>>>>>>
>> >> >> >>>>>>>>>>>>>> On Fri, Sep 24, 2021 at 2:36 PM Udi Meiri <
>> ehudm@google.com> wrote:
>> >> >> >>>>>>>>>>>>>> >
>> >> >> >>>>>>>>>>>>>> > Hi everyone, we found a bug during testing. It has
>> to do with Python SDK's allow_unsafe_triggers check.
>> >> >> >>>>>>>>>>>>>> > There is a preliminary fix that will go to master.
>> >> >> >>>>>>>>>>>>>> >
>> >> >> >>>>>>>>>>>>>> > For the 2.33.0 release, I'm leaning towards not
>> making a new RC since there is a workaround: pass the flag
>> --allow_unsafe_triggers.
>> >> >> >>>>>>>>>>>>>> >
>> >> >> >>>>>>>>>>>>>> > Please reevaluate your votes accordingly and
>> recast if you've changed your vote.
>> >> >> >>>>>>>>>>>>>> >
>> >> >> >>>>>>>>>>>>>> > Thanks
>> >> >> >>>>>>>>>>>>>> >
>> >> >> >>>>>>>>>>>>>> > On Fri, Sep 24, 2021 at 12:48 PM Alexey Romanenko <
>> aromanenko.dev@gmail.com> wrote:
>> >> >> >>>>>>>>>>>>>> >>
>> >> >> >>>>>>>>>>>>>> >>
>> >> >> >>>>>>>>>>>>>> >> On 24 Sep 2021, at 20:45, Udi Meiri <
>> ehudm@google.com> wrote:
>> >> >> >>>>>>>>>>>>>> >> Alexey is this something that we should put in
>> the release notes, or some other change?
>> >> >> >>>>>>>>>>>>>> >>
>> >> >> >>>>>>>>>>>>>> >>
>> >> >> >>>>>>>>>>>>>> >> Yes, I think it could be helpful to mention that
>> Beam Jackson’s deps was bumped and it may require an update of Jackson’s
>> runtime deps for Spark 2 users as well.
>> >> >> >>>>>>>>>>>>>> >>
>> >> >> >>>>>>>>>>>>>> >> —
>> >> >> >>>>>>>>>>>>>> >> Alexey
>> >> >> >>>>>>>>>>>>>> >>
>> >> >> >>>>>>>>>>>>>> >>
>> >> >> >>>>>>>>>>>>>> >>
>> >> >> >>>>>>>>>>>>>> >>>
>> >> >> >>>>>>>>>>>>>> >>>
>> >> >> >>>>>>>>>>>>>> >>> —
>> >> >> >>>>>>>>>>>>>> >>> Alexey
>> >> >> >>>>>>>>>>>>>> >>>
>> >> >> >>>>>>>>>>>>>> >>> On 24 Sep 2021, at 16:17, Alexey Romanenko <
>> aromanenko.dev@gmail.com> wrote:
>> >> >> >>>>>>>>>>>>>> >>>
>> >> >> >>>>>>>>>>>>>> >>> I checked with beam-samples [1] and noticed an
>> issue to run some pipelines with Spark 2 runner (Spark 3 seems is ok).
>> >> >> >>>>>>>>>>>>>> >>> It looks that it’s caused by new Jackson's
>> version updated recently [2], even if it’s a minor update but it works fine
>> with a previous one.
>> >> >> >>>>>>>>>>>>>> >>> I’ll try to find a workaround and get back with
>> a results of this.
>> >> >> >>>>>>>>>>>>>> >>>
>> >> >> >>>>>>>>>>>>>> >>> [1] https://github.com/Talend/beam-samples/
>> >> >> >>>>>>>>>>>>>> >>> [2]
>> https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c
>> >> >> >>>>>>>>>>>>>> >>>
>> >> >> >>>>>>>>>>>>>> >>> —
>> >> >> >>>>>>>>>>>>>> >>> Alexey
>> >> >> >>>>>>>>>>>>>> >>>
>> >> >> >>>>>>>>>>>>>> >>> On 24 Sep 2021, at 11:17, Jan Lukavský <
>> je.ik@seznam.cz> wrote:
>> >> >> >>>>>>>>>>>>>> >>>
>> >> >> >>>>>>>>>>>>>> >>> +1 (non-binding)
>> >> >> >>>>>>>>>>>>>> >>>
>> >> >> >>>>>>>>>>>>>> >>> Validated several use-cases using non-portable
>> Flink with Java SDK.
>> >> >> >>>>>>>>>>>>>> >>>
>> >> >> >>>>>>>>>>>>>> >>> On 9/24/21 4:55 AM, Valentyn Tymofieiev wrote:
>> >> >> >>>>>>>>>>>>>> >>>
>> >> >> >>>>>>>>>>>>>> >>> +1. Ran several Python batch and streaming
>> pipelines on Dataflow and checked that Dataflow containers have required
>> dependencies of Beam.
>> >> >> >>>>>>>>>>>>>> >>>
>> >> >> >>>>>>>>>>>>>> >>> On Thu, Sep 23, 2021 at 7:03 PM Robert Burke <
>> lostluck@apache.org> wrote:
>> >> >> >>>>>>>>>>>>>> >>>>
>> >> >> >>>>>>>>>>>>>> >>>> +1 (non-binding)
>> >> >> >>>>>>>>>>>>>> >>>> I validated the Go Quickstart (wordcount), and
>> my ray tracer against the Go Direct runner, Dataflow, and Spark (ensuring
>> the rc1 tagged container was used) and they executed successfully.
>> >> >> >>>>>>>>>>>>>> >>>>
>> >> >> >>>>>>>>>>>>>> >>>> I needed to manually synthesize a
>> pseudo-version to ensure I was using the tagged branch version
>> (v2.0.0-20210914211513-b358127f9859) instead of simply using v2.33.0-RC1.
>> >> >> >>>>>>>>>>>>>> >>>>
>> >> >> >>>>>>>>>>>>>> >>>>  It either can't find the package with the
>> right tagged version, or it can't find the version. It's not clear to me
>> what the issue is, but it's not notionally a release blocker. I'll
>> investigate further once we have a full release, as it's probably some
>> unspecified behavior due to how we transitioned to Go Modules (which
>> strongly recommended doing a major version bump for such transitions, which
>> seems a bit excessive for Beam as a whole...).
>> >> >> >>>>>>>>>>>>>> >>>>
>> >> >> >>>>>>>>>>>>>> >>>> On 2021/09/23 03:59:18, Ahmet Altay <
>> altay@google.com> wrote:
>> >> >> >>>>>>>>>>>>>> >>>> > +1 on the RC. I validated python quick start
>> examples on direct runners.
>> >> >> >>>>>>>>>>>>>> >>>> > Thank you Udi.
>> >> >> >>>>>>>>>>>>>> >>>> >
>> >> >> >>>>>>>>>>>>>> >>>> > On Wed, Sep 22, 2021 at 2:20 PM Robert Burke <
>> robert@frantil.com> wrote:
>> >> >> >>>>>>>>>>>>>> >>>> >
>> >> >> >>>>>>>>>>>>>> >>>> > > Just an FYI that intend to validate the Go
>> SDK for this release but can't
>> >> >> >>>>>>>>>>>>>> >>>> > > get to it until tomorrow (Thursday). I'm
>> catching up from a week of
>> >> >> >>>>>>>>>>>>>> >>>> > > vacation. Apologies for the inconvenience.
>> >> >> >>>>>>>>>>>>>> >>>> > >
>> >> >> >>>>>>>>>>>>>> >>>> > > On Tue, Sep 21, 2021, 10:59 AM Udi Meiri <
>> ehudm@google.com> wrote:
>> >> >> >>>>>>>>>>>>>> >>>> > >
>> >> >> >>>>>>>>>>>>>> >>>> > >> I updated the affected and fixed version
>> fields for
>> >> >> >>>>>>>>>>>>>> >>>> > >>
>> https://issues.apache.org/jira/browse/BEAM-12356.
>> >> >> >>>>>>>>>>>>>> >>>> > >>
>> >> >> >>>>>>>>>>>>>> >>>> > >>
>> >> >> >>>>>>>>>>>>>> >>>> > >> On Tue, Sep 21, 2021 at 10:48 AM Reuven
>> Lax <re...@google.com> wrote:
>> >> >> >>>>>>>>>>>>>> >>>> > >>
>> >> >> >>>>>>>>>>>>>> >>>> > >>> Unfortunate - I didn't realize that
>> >> >> >>>>>>>>>>>>>> >>>> > >>> https://github.com/apache/beam/pull/15480
>> didn't make the cut.
>> >> >> >>>>>>>>>>>>>> >>>> > >>>
>> >> >> >>>>>>>>>>>>>> >>>> > >>> This bug was a regression in Beam 2.32.0,
>> and is blocking multiple users
>> >> >> >>>>>>>>>>>>>> >>>> > >>> from updating.
>> >> >> >>>>>>>>>>>>>> >>>> > >>>
>> >> >> >>>>>>>>>>>>>> >>>> > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi
>> Meiri <ud...@apache.org> wrote:
>> >> >> >>>>>>>>>>>>>> >>>> > >>>
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> Hi everyone,
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> Please review and vote on the release
>> candidate #1 for the version
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> 2.33.0, as follows:
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [ ] +1, Approve the release
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [ ] -1, Do not approve the release
>> (please provide specific comments)
>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> Reviewers are encouraged to test their
>> own use cases with the release
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> candidate, and vote +1 if
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> no issues are found.
>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> The complete staging area is available
>> for your review, which includes:
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * JIRA release notes [1],
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * the official Apache source release to
>> be deployed to dist.apache.org
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [2], which is signed with the key with
>> fingerprint 587B049C36DAAFE6 [3],
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * all artifacts to be deployed to the
>> Maven Central Repository [4],
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * source code tag "v2.33.0-RC1" [5],
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * website pull request listing the
>> release [6], the blog post [6], and
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> publishing the API reference manual [7].
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * Java artifacts were built with Maven
>> 3.6.3 and OpenJDK 1.8.0_181.
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * Python artifacts are deployed along
>> with the source release to the
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> dist.apache.org [2] and pypy[8].
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * Validation sheet with a tab for 2.33.0
>> release to help with
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> validation [9].
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> * Docker images published to Docker Hub
>> [10].
>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> The vote will be open for at least 72
>> hours. It is adopted by majority
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> approval, with at least 3 PMC
>> affirmative votes.
>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> For guidelines on how to try the release
>> in your projects, check out
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> our blog post at
>> https://beam.apache.org/blog/validate-beam-release/.
>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> Thanks,
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> Release Manager
>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [1]
>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [2]
>> https://dist.apache.org/repos/dist/dev/beam/2.33.0/
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [3]
>> https://dist.apache.org/repos/dist/release/beam/KEYS
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [4]
>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>> https://repository.apache.org/content/repositories/orgapachebeam-1234/
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [5]
>> https://github.com/apache/beam/tree/v2.33.0-RC1
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [6]
>> https://github.com/apache/beam/pull/15543
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [7]
>> https://github.com/apache/beam-site/pull/619
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [8]
>> https://pypi.org/project/apache-beam/2.33.0rc1/
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [9]
>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
>> >> >> >>>>>>>>>>>>>> >>>> > >>>> [10]
>> https://hub.docker.com/search?q=apache%2Fbeam&type=image
>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>> >> >> >>>>>>>>>>>>>> >>>> > >>>>
>> >> >> >>>>>>>>>>>>>> >>>> >
>> >> >> >>>>>>>>>>>>>> >>>
>> >> >> >>>>>>>>>>>>>> >>>
>> >> >> >>>>>>>>>>>>>> >>>
>> >> >> >>>>>>>>>>>>>> >>
>>
>
>
> --
>
> Zachary Houfek
>
> Software Engineer
>
> DataPLS PLAT
>
> zhoufek@google.com
>

Re: [VOTE] Release 2.33.0, release candidate 1

[Zachary Houfek <zh...@google.com>]

> This was discovered because Repeatedly(AfterProcessingTime(n))

Wasn't it Repeatedly(AfterCount(n)) that caused the problem? At least
that's what caused the failing test that prompted the fix.

With that said, is the concern that AfterProcessingTime *should* include
the CONDITION_NOT_GUARANTEED flag? It isn't right now, so I'm checking.

Another thing I haven't seen mentioned: `Repeatedly` is or can be used in
other triggers like AfterWatermark (because of the late trigger),
AfterEach, AfterEvery, and AfterAny, so the bug will propagate to those
triggers as well. Two of these actually had to have their tests updated in
the fix
<https://github.com/apache/beam/commit/533a63299789ec8b4bbe629c95734be8dd4d9589#diff-9278a08a4f97eb63d783206bdea1a05377b9edd41646255110446512b8adad43>
.

Regards,
Zach



On Mon, Sep 27, 2021 at 6:30 PM Robert Bradshaw <ro...@google.com> wrote:

> $ git checkout release-2.33.0
> Switched to branch 'release-2.33.0'
> Your branch is up to date with 'github/release-2.33.0'.
>
> $ python -c 'import apache_beam as beam;
>
> print(beam.transforms.trigger.Repeatedly(beam.transforms.trigger.AfterProcessingTime(5)).may_lose_data(None))'
> DataLossReason.NO_POTENTIAL_LOSS
>
> The MAY_FINISH and CONDITION_NOT_GUARANTEED flags are disjoint.
>
> On Mon, Sep 27, 2021 at 3:15 PM Kenneth Knowles <ke...@apache.org> wrote:
> >
> > This was discovered because Repeatedly(AfterProcessingTime(n)) does
> require the workaround, because AfterProcessingTime(n) does terminate and
> that is propagated by the Repeatedly implementation.
> >
> > Kenn
> >
> > On Mon, Sep 27, 2021 at 2:49 PM Robert Bradshaw <ro...@google.com>
> wrote:
> >>
> >> On Mon, Sep 27, 2021 at 2:42 PM Kenneth Knowles <ke...@apache.org>
> wrote:
> >> >
> >> > True. It is the use case of Repeatedly(ProcessingTime(n)) that I
> would guess to be the primary case of concern.
> >>
> >> This shouldn't require the workaround, right?
> >>
> >> If you're not comfortable with this, I'd say let's patch and re-cut.
> >>
> >>
> >> > On Mon, Sep 27, 2021 at 2:38 PM Robert Bradshaw <ro...@google.com>
> wrote:
> >> >>
> >> >> As I read the code, it's only pipelines that use a Repeatedly trigger
> >> >> that wrap an already lossy trigger that are declared to be themselves
> >> >> lossy. If I'm mistaken, I'll certainly reconsider my vote (and thanks
> >> >> for bringing this up).
> >> >>
> >> >> On Mon, Sep 27, 2021 at 2:21 PM Kenneth Knowles <ke...@apache.org>
> wrote:
> >> >> >
> >> >> > My concern is that the error message is incorrect and every user
> of 2.33.0 may be educated wrong, or be worried about data loss in Beam, or
> fail to find the blog post or CHANGES, etc.
> >> >> >
> >> >> > Kenn
> >> >> >
> >> >> > On Mon, Sep 27, 2021 at 2:16 PM Udi Meiri <eh...@google.com>
> wrote:
> >> >> >>
> >> >> >> I don't know how rare it is, but there is a flag documented in
> CHANGES and blog post that reverts to the old behavior.
> >> >> >>
> >> >> >> On Mon, Sep 27, 2021 at 2:12 PM Kenneth Knowles <ke...@apache.org>
> wrote:
> >> >> >>>
> >> >> >>> I guess my vote is -0 since I don't have enough context on this
> issue. A number of people with more awareness of how severe this is have
> voted +1 so I will not try to block the release.
> >> >> >>>
> >> >> >>> Kenn
> >> >> >>>
> >> >> >>> On Mon, Sep 27, 2021 at 2:11 PM Kenneth Knowles <ke...@apache.org>
> wrote:
> >> >> >>>>
> >> >> >>>> I have to disagree with the other PMC members here, or at least
> dig in to the question: every pipeline that uses a Repeatedly trigger at
> the top level will be rejected. Is this so rare in Python that it is OK?
> >> >> >>>>
> >> >> >>>> Kenn
> >> >> >>>>
> >> >> >>>> On Mon, Sep 27, 2021 at 1:56 PM Robert Burke <re...@google.com>
> wrote:
> >> >> >>>>>
> >> >> >>>>> On it. Thanks!
> >> >> >>>>>
> >> >> >>>>> On Mon, Sep 27, 2021 at 1:18 PM Udi Meiri <eh...@google.com>
> wrote:
> >> >> >>>>>>
> >> >> >>>>>> Daniel/Robert, feel free to make changes to this PR:
> https://github.com/apache/beam/pull/15543
> >> >> >>>>>>
> >> >> >>>>>> On Mon, Sep 27, 2021 at 12:08 PM Daniel Oliveira <
> danoliveira@google.com> wrote:
> >> >> >>>>>>>
> >> >> >>>>>>> +1
> >> >> >>>>>>>
> >> >> >>>>>>> I hadn't realized the pipelines were still finishing
> successfully. I retried wordcount with that in mind and confirmed it
> finishes successfully, so this isn't a blocker.
> >> >> >>>>>>>
> >> >> >>>>>>> Although maybe we should add this to the "Known Issues"
> because I can easily see those messages being interpreted as a pipeline
> failure.
> >> >> >>>>>>>
> >> >> >>>>>>> On Sun, Sep 26, 2021 at 7:26 PM Robert Burke <
> robert@frantil.com> wrote:
> >> >> >>>>>>>>
> >> >> >>>>>>>> That's on the SDK side, and it just means the PCollection
> metrics are being returned, buy not handled by the SDK. At present the
> pipeline results only handle PTransform metrics.
> >> >> >>>>>>>>
> >> >> >>>>>>>> As such it's not a regression, as adding those is still
> under development.
> >> >> >>>>>>>>
> >> >> >>>>>>>> On Sun, Sep 26, 2021, 7:02 PM Daniel Oliveira <
> danoliveira@google.com> wrote:
> >> >> >>>>>>>>>
> >> >> >>>>>>>>> I tried validating wordcount with the Go SDK on Flink. The
> pipeline failed with a wall of errors like the following. I tried this on
> Flink 1.11, 1.12, and 1.13 job servers built from source at the RC1 commit,
> same errors on all of them.
> >> >> >>>>>>>>>
> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
> MonitoringInfo: urn:"beam:metric:element_count:v1"
> type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"
> labels:{key:"PCOLLECTION"  value:"n9"}
> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
> MonitoringInfo: urn:"beam:metric:element_count:v1"
> type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"
> labels:{key:"PCOLLECTION"  value:"n10"}
> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
> MonitoringInfo: urn:"beam:metric:element_count:v1"
> type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"
> labels:{key:"PCOLLECTION"  value:"n8"}
> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
> MonitoringInfo: urn:"beam:metric:element_count:v1"
> type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"
> labels:{key:"PCOLLECTION"  value:"n7"}
> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
> MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"
> type:"beam:metrics:distribution_int64:v1"
> payload:"\x8d%\xfa\xbf\x05\x04\xa8\x0c"  labels:{key:"PCOLLECTION"
> value:"n7"}
> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
> MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"
> type:"beam:metrics:distribution_int64:v1"  payload:"\xb9\x05\xcf6\x05\x11"
> labels:{key:"PCOLLECTION"  value:"n9"}
> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
> MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"
> type:"beam:metrics:distribution_int64:v1"  payload:"\xc5\x05\x8a1\x04\x12"
> labels:{key:"PCOLLECTION"  value:"n8"}
> >> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
> MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"
> type:"beam:metrics:distribution_int64:v1"
> payload:"\x8d%\xb3\xa7\x07\x14\""  labels:{key:"PCOLLECTION"  value:"n10"}
> >> >> >>>>>>>>>> {...}
> >> >> >>>>>>>>>
> >> >> >>>>>>>>>
> >> >> >>>>>>>>>  @Robert Burke I think you might know what's going on
> here. Is this solvable with a cherry-pick and a new RC?
> >> >> >>>>>>>>>
> >> >> >>>>>>>>> On Fri, Sep 24, 2021 at 4:25 PM Ahmet Altay <
> altay@google.com> wrote:
> >> >> >>>>>>>>>>
> >> >> >>>>>>>>>>
> >> >> >>>>>>>>>>
> >> >> >>>>>>>>>> On Fri, Sep 24, 2021 at 4:21 PM Udi Meiri <
> ehudm@google.com> wrote:
> >> >> >>>>>>>>>>>
> >> >> >>>>>>>>>>>
> >> >> >>>>>>>>>>>
> >> >> >>>>>>>>>>> On Fri, Sep 24, 2021 at 4:18 PM Ahmet Altay <
> altay@google.com> wrote:
> >> >> >>>>>>>>>>>>
> >> >> >>>>>>>>>>>> Thank you for the update related to
> allow_unsafe_triggers. My vote is still a +1.
> >> >> >>>>>>>>>>>>
> >> >> >>>>>>>>>>>> If we decide to move forward with this RC, could you
> please include this bug in the known issues list under the changes.md for
> this release?
> >> >> >>>>>>>>>>>
> >> >> >>>>>>>>>>> Yes, that's included in
> https://github.com/apache/beam/pull/15543/files.
> >> >> >>>>>>>>>>
> >> >> >>>>>>>>>>
> >> >> >>>>>>>>>> Thank you!
> >> >> >>>>>>>>>>
> >> >> >>>>>>>>>>>
> >> >> >>>>>>>>>>>
> >> >> >>>>>>>>>>>>
> >> >> >>>>>>>>>>>>
> >> >> >>>>>>>>>>>> On Fri, Sep 24, 2021 at 3:56 PM Chamikara Jayalath <
> chamikara@google.com> wrote:
> >> >> >>>>>>>>>>>>>
> >> >> >>>>>>>>>>>>> +1 (binding)
> >> >> >>>>>>>>>>>>> Validated a few scenarios from the spreadsheet.
> >> >> >>>>>>>>>>>>>
> >> >> >>>>>>>>>>>>> Thanks,
> >> >> >>>>>>>>>>>>> Cham
> >> >> >>>>>>>>>>>>>
> >> >> >>>>>>>>>>>>> On Fri, Sep 24, 2021 at 3:07 PM Robert Bradshaw <
> robertwb@google.com> wrote:
> >> >> >>>>>>>>>>>>>>
> >> >> >>>>>>>>>>>>>> +1 (binding)
> >> >> >>>>>>>>>>>>>>
> >> >> >>>>>>>>>>>>>> All the artifacts and signatures look good.
> >> >> >>>>>>>>>>>>>>
> >> >> >>>>>>>>>>>>>> I don't think the unsafe trigger check is severe
> enough to block the release.
> >> >> >>>>>>>>>>>>>>
> >> >> >>>>>>>>>>>>>> On Fri, Sep 24, 2021 at 2:36 PM Udi Meiri <
> ehudm@google.com> wrote:
> >> >> >>>>>>>>>>>>>> >
> >> >> >>>>>>>>>>>>>> > Hi everyone, we found a bug during testing. It has
> to do with Python SDK's allow_unsafe_triggers check.
> >> >> >>>>>>>>>>>>>> > There is a preliminary fix that will go to master.
> >> >> >>>>>>>>>>>>>> >
> >> >> >>>>>>>>>>>>>> > For the 2.33.0 release, I'm leaning towards not
> making a new RC since there is a workaround: pass the flag
> --allow_unsafe_triggers.
> >> >> >>>>>>>>>>>>>> >
> >> >> >>>>>>>>>>>>>> > Please reevaluate your votes accordingly and recast
> if you've changed your vote.
> >> >> >>>>>>>>>>>>>> >
> >> >> >>>>>>>>>>>>>> > Thanks
> >> >> >>>>>>>>>>>>>> >
> >> >> >>>>>>>>>>>>>> > On Fri, Sep 24, 2021 at 12:48 PM Alexey Romanenko <
> aromanenko.dev@gmail.com> wrote:
> >> >> >>>>>>>>>>>>>> >>
> >> >> >>>>>>>>>>>>>> >>
> >> >> >>>>>>>>>>>>>> >> On 24 Sep 2021, at 20:45, Udi Meiri <
> ehudm@google.com> wrote:
> >> >> >>>>>>>>>>>>>> >> Alexey is this something that we should put in the
> release notes, or some other change?
> >> >> >>>>>>>>>>>>>> >>
> >> >> >>>>>>>>>>>>>> >>
> >> >> >>>>>>>>>>>>>> >> Yes, I think it could be helpful to mention that
> Beam Jackson’s deps was bumped and it may require an update of Jackson’s
> runtime deps for Spark 2 users as well.
> >> >> >>>>>>>>>>>>>> >>
> >> >> >>>>>>>>>>>>>> >> —
> >> >> >>>>>>>>>>>>>> >> Alexey
> >> >> >>>>>>>>>>>>>> >>
> >> >> >>>>>>>>>>>>>> >>
> >> >> >>>>>>>>>>>>>> >>
> >> >> >>>>>>>>>>>>>> >>>
> >> >> >>>>>>>>>>>>>> >>>
> >> >> >>>>>>>>>>>>>> >>> —
> >> >> >>>>>>>>>>>>>> >>> Alexey
> >> >> >>>>>>>>>>>>>> >>>
> >> >> >>>>>>>>>>>>>> >>> On 24 Sep 2021, at 16:17, Alexey Romanenko <
> aromanenko.dev@gmail.com> wrote:
> >> >> >>>>>>>>>>>>>> >>>
> >> >> >>>>>>>>>>>>>> >>> I checked with beam-samples [1] and noticed an
> issue to run some pipelines with Spark 2 runner (Spark 3 seems is ok).
> >> >> >>>>>>>>>>>>>> >>> It looks that it’s caused by new Jackson's
> version updated recently [2], even if it’s a minor update but it works fine
> with a previous one.
> >> >> >>>>>>>>>>>>>> >>> I’ll try to find a workaround and get back with a
> results of this.
> >> >> >>>>>>>>>>>>>> >>>
> >> >> >>>>>>>>>>>>>> >>> [1] https://github.com/Talend/beam-samples/
> >> >> >>>>>>>>>>>>>> >>> [2]
> https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c
> >> >> >>>>>>>>>>>>>> >>>
> >> >> >>>>>>>>>>>>>> >>> —
> >> >> >>>>>>>>>>>>>> >>> Alexey
> >> >> >>>>>>>>>>>>>> >>>
> >> >> >>>>>>>>>>>>>> >>> On 24 Sep 2021, at 11:17, Jan Lukavský <
> je.ik@seznam.cz> wrote:
> >> >> >>>>>>>>>>>>>> >>>
> >> >> >>>>>>>>>>>>>> >>> +1 (non-binding)
> >> >> >>>>>>>>>>>>>> >>>
> >> >> >>>>>>>>>>>>>> >>> Validated several use-cases using non-portable
> Flink with Java SDK.
> >> >> >>>>>>>>>>>>>> >>>
> >> >> >>>>>>>>>>>>>> >>> On 9/24/21 4:55 AM, Valentyn Tymofieiev wrote:
> >> >> >>>>>>>>>>>>>> >>>
> >> >> >>>>>>>>>>>>>> >>> +1. Ran several Python batch and streaming
> pipelines on Dataflow and checked that Dataflow containers have required
> dependencies of Beam.
> >> >> >>>>>>>>>>>>>> >>>
> >> >> >>>>>>>>>>>>>> >>> On Thu, Sep 23, 2021 at 7:03 PM Robert Burke <
> lostluck@apache.org> wrote:
> >> >> >>>>>>>>>>>>>> >>>>
> >> >> >>>>>>>>>>>>>> >>>> +1 (non-binding)
> >> >> >>>>>>>>>>>>>> >>>> I validated the Go Quickstart (wordcount), and
> my ray tracer against the Go Direct runner, Dataflow, and Spark (ensuring
> the rc1 tagged container was used) and they executed successfully.
> >> >> >>>>>>>>>>>>>> >>>>
> >> >> >>>>>>>>>>>>>> >>>> I needed to manually synthesize a pseudo-version
> to ensure I was using the tagged branch version
> (v2.0.0-20210914211513-b358127f9859) instead of simply using v2.33.0-RC1.
> >> >> >>>>>>>>>>>>>> >>>>
> >> >> >>>>>>>>>>>>>> >>>>  It either can't find the package with the right
> tagged version, or it can't find the version. It's not clear to me what the
> issue is, but it's not notionally a release blocker. I'll investigate
> further once we have a full release, as it's probably some unspecified
> behavior due to how we transitioned to Go Modules (which strongly
> recommended doing a major version bump for such transitions, which seems a
> bit excessive for Beam as a whole...).
> >> >> >>>>>>>>>>>>>> >>>>
> >> >> >>>>>>>>>>>>>> >>>> On 2021/09/23 03:59:18, Ahmet Altay <
> altay@google.com> wrote:
> >> >> >>>>>>>>>>>>>> >>>> > +1 on the RC. I validated python quick start
> examples on direct runners.
> >> >> >>>>>>>>>>>>>> >>>> > Thank you Udi.
> >> >> >>>>>>>>>>>>>> >>>> >
> >> >> >>>>>>>>>>>>>> >>>> > On Wed, Sep 22, 2021 at 2:20 PM Robert Burke <
> robert@frantil.com> wrote:
> >> >> >>>>>>>>>>>>>> >>>> >
> >> >> >>>>>>>>>>>>>> >>>> > > Just an FYI that intend to validate the Go
> SDK for this release but can't
> >> >> >>>>>>>>>>>>>> >>>> > > get to it until tomorrow (Thursday). I'm
> catching up from a week of
> >> >> >>>>>>>>>>>>>> >>>> > > vacation. Apologies for the inconvenience.
> >> >> >>>>>>>>>>>>>> >>>> > >
> >> >> >>>>>>>>>>>>>> >>>> > > On Tue, Sep 21, 2021, 10:59 AM Udi Meiri <
> ehudm@google.com> wrote:
> >> >> >>>>>>>>>>>>>> >>>> > >
> >> >> >>>>>>>>>>>>>> >>>> > >> I updated the affected and fixed version
> fields for
> >> >> >>>>>>>>>>>>>> >>>> > >>
> https://issues.apache.org/jira/browse/BEAM-12356.
> >> >> >>>>>>>>>>>>>> >>>> > >>
> >> >> >>>>>>>>>>>>>> >>>> > >>
> >> >> >>>>>>>>>>>>>> >>>> > >> On Tue, Sep 21, 2021 at 10:48 AM Reuven Lax
> <re...@google.com> wrote:
> >> >> >>>>>>>>>>>>>> >>>> > >>
> >> >> >>>>>>>>>>>>>> >>>> > >>> Unfortunate - I didn't realize that
> >> >> >>>>>>>>>>>>>> >>>> > >>> https://github.com/apache/beam/pull/15480
> didn't make the cut.
> >> >> >>>>>>>>>>>>>> >>>> > >>>
> >> >> >>>>>>>>>>>>>> >>>> > >>> This bug was a regression in Beam 2.32.0,
> and is blocking multiple users
> >> >> >>>>>>>>>>>>>> >>>> > >>> from updating.
> >> >> >>>>>>>>>>>>>> >>>> > >>>
> >> >> >>>>>>>>>>>>>> >>>> > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi Meiri
> <ud...@apache.org> wrote:
> >> >> >>>>>>>>>>>>>> >>>> > >>>
> >> >> >>>>>>>>>>>>>> >>>> > >>>> Hi everyone,
> >> >> >>>>>>>>>>>>>> >>>> > >>>> Please review and vote on the release
> candidate #1 for the version
> >> >> >>>>>>>>>>>>>> >>>> > >>>> 2.33.0, as follows:
> >> >> >>>>>>>>>>>>>> >>>> > >>>> [ ] +1, Approve the release
> >> >> >>>>>>>>>>>>>> >>>> > >>>> [ ] -1, Do not approve the release
> (please provide specific comments)
> >> >> >>>>>>>>>>>>>> >>>> > >>>>
> >> >> >>>>>>>>>>>>>> >>>> > >>>>
> >> >> >>>>>>>>>>>>>> >>>> > >>>> Reviewers are encouraged to test their
> own use cases with the release
> >> >> >>>>>>>>>>>>>> >>>> > >>>> candidate, and vote +1 if
> >> >> >>>>>>>>>>>>>> >>>> > >>>> no issues are found.
> >> >> >>>>>>>>>>>>>> >>>> > >>>>
> >> >> >>>>>>>>>>>>>> >>>> > >>>> The complete staging area is available
> for your review, which includes:
> >> >> >>>>>>>>>>>>>> >>>> > >>>> * JIRA release notes [1],
> >> >> >>>>>>>>>>>>>> >>>> > >>>> * the official Apache source release to
> be deployed to dist.apache.org
> >> >> >>>>>>>>>>>>>> >>>> > >>>> [2], which is signed with the key with
> fingerprint 587B049C36DAAFE6 [3],
> >> >> >>>>>>>>>>>>>> >>>> > >>>> * all artifacts to be deployed to the
> Maven Central Repository [4],
> >> >> >>>>>>>>>>>>>> >>>> > >>>> * source code tag "v2.33.0-RC1" [5],
> >> >> >>>>>>>>>>>>>> >>>> > >>>> * website pull request listing the
> release [6], the blog post [6], and
> >> >> >>>>>>>>>>>>>> >>>> > >>>> publishing the API reference manual [7].
> >> >> >>>>>>>>>>>>>> >>>> > >>>> * Java artifacts were built with Maven
> 3.6.3 and OpenJDK 1.8.0_181.
> >> >> >>>>>>>>>>>>>> >>>> > >>>> * Python artifacts are deployed along
> with the source release to the
> >> >> >>>>>>>>>>>>>> >>>> > >>>> dist.apache.org [2] and pypy[8].
> >> >> >>>>>>>>>>>>>> >>>> > >>>> * Validation sheet with a tab for 2.33.0
> release to help with
> >> >> >>>>>>>>>>>>>> >>>> > >>>> validation [9].
> >> >> >>>>>>>>>>>>>> >>>> > >>>> * Docker images published to Docker Hub
> [10].
> >> >> >>>>>>>>>>>>>> >>>> > >>>>
> >> >> >>>>>>>>>>>>>> >>>> > >>>> The vote will be open for at least 72
> hours. It is adopted by majority
> >> >> >>>>>>>>>>>>>> >>>> > >>>> approval, with at least 3 PMC affirmative
> votes.
> >> >> >>>>>>>>>>>>>> >>>> > >>>>
> >> >> >>>>>>>>>>>>>> >>>> > >>>> For guidelines on how to try the release
> in your projects, check out
> >> >> >>>>>>>>>>>>>> >>>> > >>>> our blog post at
> https://beam.apache.org/blog/validate-beam-release/.
> >> >> >>>>>>>>>>>>>> >>>> > >>>>
> >> >> >>>>>>>>>>>>>> >>>> > >>>> Thanks,
> >> >> >>>>>>>>>>>>>> >>>> > >>>> Release Manager
> >> >> >>>>>>>>>>>>>> >>>> > >>>>
> >> >> >>>>>>>>>>>>>> >>>> > >>>> [1]
> >> >> >>>>>>>>>>>>>> >>>> > >>>>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
> >> >> >>>>>>>>>>>>>> >>>> > >>>> [2]
> https://dist.apache.org/repos/dist/dev/beam/2.33.0/
> >> >> >>>>>>>>>>>>>> >>>> > >>>> [3]
> https://dist.apache.org/repos/dist/release/beam/KEYS
> >> >> >>>>>>>>>>>>>> >>>> > >>>> [4]
> >> >> >>>>>>>>>>>>>> >>>> > >>>>
> https://repository.apache.org/content/repositories/orgapachebeam-1234/
> >> >> >>>>>>>>>>>>>> >>>> > >>>> [5]
> https://github.com/apache/beam/tree/v2.33.0-RC1
> >> >> >>>>>>>>>>>>>> >>>> > >>>> [6]
> https://github.com/apache/beam/pull/15543
> >> >> >>>>>>>>>>>>>> >>>> > >>>> [7]
> https://github.com/apache/beam-site/pull/619
> >> >> >>>>>>>>>>>>>> >>>> > >>>> [8]
> https://pypi.org/project/apache-beam/2.33.0rc1/
> >> >> >>>>>>>>>>>>>> >>>> > >>>> [9]
> >> >> >>>>>>>>>>>>>> >>>> > >>>>
> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
> >> >> >>>>>>>>>>>>>> >>>> > >>>> [10]
> https://hub.docker.com/search?q=apache%2Fbeam&type=image
> >> >> >>>>>>>>>>>>>> >>>> > >>>>
> >> >> >>>>>>>>>>>>>> >>>> > >>>>
> >> >> >>>>>>>>>>>>>> >>>> >
> >> >> >>>>>>>>>>>>>> >>>
> >> >> >>>>>>>>>>>>>> >>>
> >> >> >>>>>>>>>>>>>> >>>
> >> >> >>>>>>>>>>>>>> >>
>


-- 

Zachary Houfek

Software Engineer

DataPLS PLAT

zhoufek@google.com

Re: [VOTE] Release 2.33.0, release candidate 1

[Robert Bradshaw <ro...@google.com>]

$ git checkout release-2.33.0
Switched to branch 'release-2.33.0'
Your branch is up to date with 'github/release-2.33.0'.

$ python -c 'import apache_beam as beam;
print(beam.transforms.trigger.Repeatedly(beam.transforms.trigger.AfterProcessingTime(5)).may_lose_data(None))'
DataLossReason.NO_POTENTIAL_LOSS

The MAY_FINISH and CONDITION_NOT_GUARANTEED flags are disjoint.

On Mon, Sep 27, 2021 at 3:15 PM Kenneth Knowles <ke...@apache.org> wrote:
>
> This was discovered because Repeatedly(AfterProcessingTime(n)) does require the workaround, because AfterProcessingTime(n) does terminate and that is propagated by the Repeatedly implementation.
>
> Kenn
>
> On Mon, Sep 27, 2021 at 2:49 PM Robert Bradshaw <ro...@google.com> wrote:
>>
>> On Mon, Sep 27, 2021 at 2:42 PM Kenneth Knowles <ke...@apache.org> wrote:
>> >
>> > True. It is the use case of Repeatedly(ProcessingTime(n)) that I would guess to be the primary case of concern.
>>
>> This shouldn't require the workaround, right?
>>
>> If you're not comfortable with this, I'd say let's patch and re-cut.
>>
>>
>> > On Mon, Sep 27, 2021 at 2:38 PM Robert Bradshaw <ro...@google.com> wrote:
>> >>
>> >> As I read the code, it's only pipelines that use a Repeatedly trigger
>> >> that wrap an already lossy trigger that are declared to be themselves
>> >> lossy. If I'm mistaken, I'll certainly reconsider my vote (and thanks
>> >> for bringing this up).
>> >>
>> >> On Mon, Sep 27, 2021 at 2:21 PM Kenneth Knowles <ke...@apache.org> wrote:
>> >> >
>> >> > My concern is that the error message is incorrect and every user of 2.33.0 may be educated wrong, or be worried about data loss in Beam, or fail to find the blog post or CHANGES, etc.
>> >> >
>> >> > Kenn
>> >> >
>> >> > On Mon, Sep 27, 2021 at 2:16 PM Udi Meiri <eh...@google.com> wrote:
>> >> >>
>> >> >> I don't know how rare it is, but there is a flag documented in CHANGES and blog post that reverts to the old behavior.
>> >> >>
>> >> >> On Mon, Sep 27, 2021 at 2:12 PM Kenneth Knowles <ke...@apache.org> wrote:
>> >> >>>
>> >> >>> I guess my vote is -0 since I don't have enough context on this issue. A number of people with more awareness of how severe this is have voted +1 so I will not try to block the release.
>> >> >>>
>> >> >>> Kenn
>> >> >>>
>> >> >>> On Mon, Sep 27, 2021 at 2:11 PM Kenneth Knowles <ke...@apache.org> wrote:
>> >> >>>>
>> >> >>>> I have to disagree with the other PMC members here, or at least dig in to the question: every pipeline that uses a Repeatedly trigger at the top level will be rejected. Is this so rare in Python that it is OK?
>> >> >>>>
>> >> >>>> Kenn
>> >> >>>>
>> >> >>>> On Mon, Sep 27, 2021 at 1:56 PM Robert Burke <re...@google.com> wrote:
>> >> >>>>>
>> >> >>>>> On it. Thanks!
>> >> >>>>>
>> >> >>>>> On Mon, Sep 27, 2021 at 1:18 PM Udi Meiri <eh...@google.com> wrote:
>> >> >>>>>>
>> >> >>>>>> Daniel/Robert, feel free to make changes to this PR: https://github.com/apache/beam/pull/15543
>> >> >>>>>>
>> >> >>>>>> On Mon, Sep 27, 2021 at 12:08 PM Daniel Oliveira <da...@google.com> wrote:
>> >> >>>>>>>
>> >> >>>>>>> +1
>> >> >>>>>>>
>> >> >>>>>>> I hadn't realized the pipelines were still finishing successfully. I retried wordcount with that in mind and confirmed it finishes successfully, so this isn't a blocker.
>> >> >>>>>>>
>> >> >>>>>>> Although maybe we should add this to the "Known Issues" because I can easily see those messages being interpreted as a pipeline failure.
>> >> >>>>>>>
>> >> >>>>>>> On Sun, Sep 26, 2021 at 7:26 PM Robert Burke <ro...@frantil.com> wrote:
>> >> >>>>>>>>
>> >> >>>>>>>> That's on the SDK side, and it just means the PCollection metrics are being returned, buy not handled by the SDK. At present the pipeline results only handle PTransform metrics.
>> >> >>>>>>>>
>> >> >>>>>>>> As such it's not a regression, as adding those is still under development.
>> >> >>>>>>>>
>> >> >>>>>>>> On Sun, Sep 26, 2021, 7:02 PM Daniel Oliveira <da...@google.com> wrote:
>> >> >>>>>>>>>
>> >> >>>>>>>>> I tried validating wordcount with the Go SDK on Flink. The pipeline failed with a wall of errors like the following. I tried this on Flink 1.11, 1.12, and 1.13 job servers built from source at the RC1 commit, same errors on all of them.
>> >> >>>>>>>>>
>> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n9"}
>> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n10"}
>> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n8"}
>> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n7"}
>> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"  type:"beam:metrics:distribution_int64:v1"  payload:"\x8d%\xfa\xbf\x05\x04\xa8\x0c"  labels:{key:"PCOLLECTION"  value:"n7"}
>> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"  type:"beam:metrics:distribution_int64:v1"  payload:"\xb9\x05\xcf6\x05\x11"  labels:{key:"PCOLLECTION"  value:"n9"}
>> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"  type:"beam:metrics:distribution_int64:v1"  payload:"\xc5\x05\x8a1\x04\x12"  labels:{key:"PCOLLECTION"  value:"n8"}
>> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"  type:"beam:metrics:distribution_int64:v1"  payload:"\x8d%\xb3\xa7\x07\x14\""  labels:{key:"PCOLLECTION"  value:"n10"}
>> >> >>>>>>>>>> {...}
>> >> >>>>>>>>>
>> >> >>>>>>>>>
>> >> >>>>>>>>>  @Robert Burke I think you might know what's going on here. Is this solvable with a cherry-pick and a new RC?
>> >> >>>>>>>>>
>> >> >>>>>>>>> On Fri, Sep 24, 2021 at 4:25 PM Ahmet Altay <al...@google.com> wrote:
>> >> >>>>>>>>>>
>> >> >>>>>>>>>>
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> On Fri, Sep 24, 2021 at 4:21 PM Udi Meiri <eh...@google.com> wrote:
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> On Fri, Sep 24, 2021 at 4:18 PM Ahmet Altay <al...@google.com> wrote:
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> Thank you for the update related to allow_unsafe_triggers. My vote is still a +1.
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> If we decide to move forward with this RC, could you please include this bug in the known issues list under the changes.md for this release?
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> Yes, that's included in https://github.com/apache/beam/pull/15543/files.
>> >> >>>>>>>>>>
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> Thank you!
>> >> >>>>>>>>>>
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> On Fri, Sep 24, 2021 at 3:56 PM Chamikara Jayalath <ch...@google.com> wrote:
>> >> >>>>>>>>>>>>>
>> >> >>>>>>>>>>>>> +1 (binding)
>> >> >>>>>>>>>>>>> Validated a few scenarios from the spreadsheet.
>> >> >>>>>>>>>>>>>
>> >> >>>>>>>>>>>>> Thanks,
>> >> >>>>>>>>>>>>> Cham
>> >> >>>>>>>>>>>>>
>> >> >>>>>>>>>>>>> On Fri, Sep 24, 2021 at 3:07 PM Robert Bradshaw <ro...@google.com> wrote:
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> +1 (binding)
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> All the artifacts and signatures look good.
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> I don't think the unsafe trigger check is severe enough to block the release.
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> On Fri, Sep 24, 2021 at 2:36 PM Udi Meiri <eh...@google.com> wrote:
>> >> >>>>>>>>>>>>>> >
>> >> >>>>>>>>>>>>>> > Hi everyone, we found a bug during testing. It has to do with Python SDK's allow_unsafe_triggers check.
>> >> >>>>>>>>>>>>>> > There is a preliminary fix that will go to master.
>> >> >>>>>>>>>>>>>> >
>> >> >>>>>>>>>>>>>> > For the 2.33.0 release, I'm leaning towards not making a new RC since there is a workaround: pass the flag --allow_unsafe_triggers.
>> >> >>>>>>>>>>>>>> >
>> >> >>>>>>>>>>>>>> > Please reevaluate your votes accordingly and recast if you've changed your vote.
>> >> >>>>>>>>>>>>>> >
>> >> >>>>>>>>>>>>>> > Thanks
>> >> >>>>>>>>>>>>>> >
>> >> >>>>>>>>>>>>>> > On Fri, Sep 24, 2021 at 12:48 PM Alexey Romanenko <ar...@gmail.com> wrote:
>> >> >>>>>>>>>>>>>> >>
>> >> >>>>>>>>>>>>>> >>
>> >> >>>>>>>>>>>>>> >> On 24 Sep 2021, at 20:45, Udi Meiri <eh...@google.com> wrote:
>> >> >>>>>>>>>>>>>> >> Alexey is this something that we should put in the release notes, or some other change?
>> >> >>>>>>>>>>>>>> >>
>> >> >>>>>>>>>>>>>> >>
>> >> >>>>>>>>>>>>>> >> Yes, I think it could be helpful to mention that Beam Jackson’s deps was bumped and it may require an update of Jackson’s runtime deps for Spark 2 users as well.
>> >> >>>>>>>>>>>>>> >>
>> >> >>>>>>>>>>>>>> >> —
>> >> >>>>>>>>>>>>>> >> Alexey
>> >> >>>>>>>>>>>>>> >>
>> >> >>>>>>>>>>>>>> >>
>> >> >>>>>>>>>>>>>> >>
>> >> >>>>>>>>>>>>>> >>>
>> >> >>>>>>>>>>>>>> >>>
>> >> >>>>>>>>>>>>>> >>> —
>> >> >>>>>>>>>>>>>> >>> Alexey
>> >> >>>>>>>>>>>>>> >>>
>> >> >>>>>>>>>>>>>> >>> On 24 Sep 2021, at 16:17, Alexey Romanenko <ar...@gmail.com> wrote:
>> >> >>>>>>>>>>>>>> >>>
>> >> >>>>>>>>>>>>>> >>> I checked with beam-samples [1] and noticed an issue to run some pipelines with Spark 2 runner (Spark 3 seems is ok).
>> >> >>>>>>>>>>>>>> >>> It looks that it’s caused by new Jackson's version updated recently [2], even if it’s a minor update but it works fine with a previous one.
>> >> >>>>>>>>>>>>>> >>> I’ll try to find a workaround and get back with a results of this.
>> >> >>>>>>>>>>>>>> >>>
>> >> >>>>>>>>>>>>>> >>> [1] https://github.com/Talend/beam-samples/
>> >> >>>>>>>>>>>>>> >>> [2] https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c
>> >> >>>>>>>>>>>>>> >>>
>> >> >>>>>>>>>>>>>> >>> —
>> >> >>>>>>>>>>>>>> >>> Alexey
>> >> >>>>>>>>>>>>>> >>>
>> >> >>>>>>>>>>>>>> >>> On 24 Sep 2021, at 11:17, Jan Lukavský <je...@seznam.cz> wrote:
>> >> >>>>>>>>>>>>>> >>>
>> >> >>>>>>>>>>>>>> >>> +1 (non-binding)
>> >> >>>>>>>>>>>>>> >>>
>> >> >>>>>>>>>>>>>> >>> Validated several use-cases using non-portable Flink with Java SDK.
>> >> >>>>>>>>>>>>>> >>>
>> >> >>>>>>>>>>>>>> >>> On 9/24/21 4:55 AM, Valentyn Tymofieiev wrote:
>> >> >>>>>>>>>>>>>> >>>
>> >> >>>>>>>>>>>>>> >>> +1. Ran several Python batch and streaming pipelines on Dataflow and checked that Dataflow containers have required dependencies of Beam.
>> >> >>>>>>>>>>>>>> >>>
>> >> >>>>>>>>>>>>>> >>> On Thu, Sep 23, 2021 at 7:03 PM Robert Burke <lo...@apache.org> wrote:
>> >> >>>>>>>>>>>>>> >>>>
>> >> >>>>>>>>>>>>>> >>>> +1 (non-binding)
>> >> >>>>>>>>>>>>>> >>>> I validated the Go Quickstart (wordcount), and my ray tracer against the Go Direct runner, Dataflow, and Spark (ensuring the rc1 tagged container was used) and they executed successfully.
>> >> >>>>>>>>>>>>>> >>>>
>> >> >>>>>>>>>>>>>> >>>> I needed to manually synthesize a pseudo-version to ensure I was using the tagged branch version (v2.0.0-20210914211513-b358127f9859) instead of simply using v2.33.0-RC1.
>> >> >>>>>>>>>>>>>> >>>>
>> >> >>>>>>>>>>>>>> >>>>  It either can't find the package with the right tagged version, or it can't find the version. It's not clear to me what the issue is, but it's not notionally a release blocker. I'll investigate further once we have a full release, as it's probably some unspecified behavior due to how we transitioned to Go Modules (which strongly recommended doing a major version bump for such transitions, which seems a bit excessive for Beam as a whole...).
>> >> >>>>>>>>>>>>>> >>>>
>> >> >>>>>>>>>>>>>> >>>> On 2021/09/23 03:59:18, Ahmet Altay <al...@google.com> wrote:
>> >> >>>>>>>>>>>>>> >>>> > +1 on the RC. I validated python quick start examples on direct runners.
>> >> >>>>>>>>>>>>>> >>>> > Thank you Udi.
>> >> >>>>>>>>>>>>>> >>>> >
>> >> >>>>>>>>>>>>>> >>>> > On Wed, Sep 22, 2021 at 2:20 PM Robert Burke <ro...@frantil.com> wrote:
>> >> >>>>>>>>>>>>>> >>>> >
>> >> >>>>>>>>>>>>>> >>>> > > Just an FYI that intend to validate the Go SDK for this release but can't
>> >> >>>>>>>>>>>>>> >>>> > > get to it until tomorrow (Thursday). I'm catching up from a week of
>> >> >>>>>>>>>>>>>> >>>> > > vacation. Apologies for the inconvenience.
>> >> >>>>>>>>>>>>>> >>>> > >
>> >> >>>>>>>>>>>>>> >>>> > > On Tue, Sep 21, 2021, 10:59 AM Udi Meiri <eh...@google.com> wrote:
>> >> >>>>>>>>>>>>>> >>>> > >
>> >> >>>>>>>>>>>>>> >>>> > >> I updated the affected and fixed version fields for
>> >> >>>>>>>>>>>>>> >>>> > >> https://issues.apache.org/jira/browse/BEAM-12356.
>> >> >>>>>>>>>>>>>> >>>> > >>
>> >> >>>>>>>>>>>>>> >>>> > >>
>> >> >>>>>>>>>>>>>> >>>> > >> On Tue, Sep 21, 2021 at 10:48 AM Reuven Lax <re...@google.com> wrote:
>> >> >>>>>>>>>>>>>> >>>> > >>
>> >> >>>>>>>>>>>>>> >>>> > >>> Unfortunate - I didn't realize that
>> >> >>>>>>>>>>>>>> >>>> > >>> https://github.com/apache/beam/pull/15480 didn't make the cut.
>> >> >>>>>>>>>>>>>> >>>> > >>>
>> >> >>>>>>>>>>>>>> >>>> > >>> This bug was a regression in Beam 2.32.0, and is blocking multiple users
>> >> >>>>>>>>>>>>>> >>>> > >>> from updating.
>> >> >>>>>>>>>>>>>> >>>> > >>>
>> >> >>>>>>>>>>>>>> >>>> > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi Meiri <ud...@apache.org> wrote:
>> >> >>>>>>>>>>>>>> >>>> > >>>
>> >> >>>>>>>>>>>>>> >>>> > >>>> Hi everyone,
>> >> >>>>>>>>>>>>>> >>>> > >>>> Please review and vote on the release candidate #1 for the version
>> >> >>>>>>>>>>>>>> >>>> > >>>> 2.33.0, as follows:
>> >> >>>>>>>>>>>>>> >>>> > >>>> [ ] +1, Approve the release
>> >> >>>>>>>>>>>>>> >>>> > >>>> [ ] -1, Do not approve the release (please provide specific comments)
>> >> >>>>>>>>>>>>>> >>>> > >>>>
>> >> >>>>>>>>>>>>>> >>>> > >>>>
>> >> >>>>>>>>>>>>>> >>>> > >>>> Reviewers are encouraged to test their own use cases with the release
>> >> >>>>>>>>>>>>>> >>>> > >>>> candidate, and vote +1 if
>> >> >>>>>>>>>>>>>> >>>> > >>>> no issues are found.
>> >> >>>>>>>>>>>>>> >>>> > >>>>
>> >> >>>>>>>>>>>>>> >>>> > >>>> The complete staging area is available for your review, which includes:
>> >> >>>>>>>>>>>>>> >>>> > >>>> * JIRA release notes [1],
>> >> >>>>>>>>>>>>>> >>>> > >>>> * the official Apache source release to be deployed to dist.apache.org
>> >> >>>>>>>>>>>>>> >>>> > >>>> [2], which is signed with the key with fingerprint 587B049C36DAAFE6 [3],
>> >> >>>>>>>>>>>>>> >>>> > >>>> * all artifacts to be deployed to the Maven Central Repository [4],
>> >> >>>>>>>>>>>>>> >>>> > >>>> * source code tag "v2.33.0-RC1" [5],
>> >> >>>>>>>>>>>>>> >>>> > >>>> * website pull request listing the release [6], the blog post [6], and
>> >> >>>>>>>>>>>>>> >>>> > >>>> publishing the API reference manual [7].
>> >> >>>>>>>>>>>>>> >>>> > >>>> * Java artifacts were built with Maven 3.6.3 and OpenJDK 1.8.0_181.
>> >> >>>>>>>>>>>>>> >>>> > >>>> * Python artifacts are deployed along with the source release to the
>> >> >>>>>>>>>>>>>> >>>> > >>>> dist.apache.org [2] and pypy[8].
>> >> >>>>>>>>>>>>>> >>>> > >>>> * Validation sheet with a tab for 2.33.0 release to help with
>> >> >>>>>>>>>>>>>> >>>> > >>>> validation [9].
>> >> >>>>>>>>>>>>>> >>>> > >>>> * Docker images published to Docker Hub [10].
>> >> >>>>>>>>>>>>>> >>>> > >>>>
>> >> >>>>>>>>>>>>>> >>>> > >>>> The vote will be open for at least 72 hours. It is adopted by majority
>> >> >>>>>>>>>>>>>> >>>> > >>>> approval, with at least 3 PMC affirmative votes.
>> >> >>>>>>>>>>>>>> >>>> > >>>>
>> >> >>>>>>>>>>>>>> >>>> > >>>> For guidelines on how to try the release in your projects, check out
>> >> >>>>>>>>>>>>>> >>>> > >>>> our blog post at https://beam.apache.org/blog/validate-beam-release/.
>> >> >>>>>>>>>>>>>> >>>> > >>>>
>> >> >>>>>>>>>>>>>> >>>> > >>>> Thanks,
>> >> >>>>>>>>>>>>>> >>>> > >>>> Release Manager
>> >> >>>>>>>>>>>>>> >>>> > >>>>
>> >> >>>>>>>>>>>>>> >>>> > >>>> [1]
>> >> >>>>>>>>>>>>>> >>>> > >>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
>> >> >>>>>>>>>>>>>> >>>> > >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.33.0/
>> >> >>>>>>>>>>>>>> >>>> > >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>> >> >>>>>>>>>>>>>> >>>> > >>>> [4]
>> >> >>>>>>>>>>>>>> >>>> > >>>> https://repository.apache.org/content/repositories/orgapachebeam-1234/
>> >> >>>>>>>>>>>>>> >>>> > >>>> [5] https://github.com/apache/beam/tree/v2.33.0-RC1
>> >> >>>>>>>>>>>>>> >>>> > >>>> [6] https://github.com/apache/beam/pull/15543
>> >> >>>>>>>>>>>>>> >>>> > >>>> [7] https://github.com/apache/beam-site/pull/619
>> >> >>>>>>>>>>>>>> >>>> > >>>> [8] https://pypi.org/project/apache-beam/2.33.0rc1/
>> >> >>>>>>>>>>>>>> >>>> > >>>> [9]
>> >> >>>>>>>>>>>>>> >>>> > >>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
>> >> >>>>>>>>>>>>>> >>>> > >>>> [10] https://hub.docker.com/search?q=apache%2Fbeam&type=image
>> >> >>>>>>>>>>>>>> >>>> > >>>>
>> >> >>>>>>>>>>>>>> >>>> > >>>>
>> >> >>>>>>>>>>>>>> >>>> >
>> >> >>>>>>>>>>>>>> >>>
>> >> >>>>>>>>>>>>>> >>>
>> >> >>>>>>>>>>>>>> >>>
>> >> >>>>>>>>>>>>>> >>

Re: [VOTE] Release 2.33.0, release candidate 1

[Kenneth Knowles <ke...@apache.org>]

This was discovered because Repeatedly(AfterProcessingTime(n)) does require
the workaround, because AfterProcessingTime(n) does terminate and that is
propagated by the Repeatedly implementation.

Kenn

On Mon, Sep 27, 2021 at 2:49 PM Robert Bradshaw <ro...@google.com> wrote:

> On Mon, Sep 27, 2021 at 2:42 PM Kenneth Knowles <ke...@apache.org> wrote:
> >
> > True. It is the use case of Repeatedly(ProcessingTime(n)) that I would
> guess to be the primary case of concern.
>
> This shouldn't require the workaround, right?
>
> If you're not comfortable with this, I'd say let's patch and re-cut.
>
>
> > On Mon, Sep 27, 2021 at 2:38 PM Robert Bradshaw <ro...@google.com>
> wrote:
> >>
> >> As I read the code, it's only pipelines that use a Repeatedly trigger
> >> that wrap an already lossy trigger that are declared to be themselves
> >> lossy. If I'm mistaken, I'll certainly reconsider my vote (and thanks
> >> for bringing this up).
> >>
> >> On Mon, Sep 27, 2021 at 2:21 PM Kenneth Knowles <ke...@apache.org>
> wrote:
> >> >
> >> > My concern is that the error message is incorrect and every user of
> 2.33.0 may be educated wrong, or be worried about data loss in Beam, or
> fail to find the blog post or CHANGES, etc.
> >> >
> >> > Kenn
> >> >
> >> > On Mon, Sep 27, 2021 at 2:16 PM Udi Meiri <eh...@google.com> wrote:
> >> >>
> >> >> I don't know how rare it is, but there is a flag documented in
> CHANGES and blog post that reverts to the old behavior.
> >> >>
> >> >> On Mon, Sep 27, 2021 at 2:12 PM Kenneth Knowles <ke...@apache.org>
> wrote:
> >> >>>
> >> >>> I guess my vote is -0 since I don't have enough context on this
> issue. A number of people with more awareness of how severe this is have
> voted +1 so I will not try to block the release.
> >> >>>
> >> >>> Kenn
> >> >>>
> >> >>> On Mon, Sep 27, 2021 at 2:11 PM Kenneth Knowles <ke...@apache.org>
> wrote:
> >> >>>>
> >> >>>> I have to disagree with the other PMC members here, or at least
> dig in to the question: every pipeline that uses a Repeatedly trigger at
> the top level will be rejected. Is this so rare in Python that it is OK?
> >> >>>>
> >> >>>> Kenn
> >> >>>>
> >> >>>> On Mon, Sep 27, 2021 at 1:56 PM Robert Burke <re...@google.com>
> wrote:
> >> >>>>>
> >> >>>>> On it. Thanks!
> >> >>>>>
> >> >>>>> On Mon, Sep 27, 2021 at 1:18 PM Udi Meiri <eh...@google.com>
> wrote:
> >> >>>>>>
> >> >>>>>> Daniel/Robert, feel free to make changes to this PR:
> https://github.com/apache/beam/pull/15543
> >> >>>>>>
> >> >>>>>> On Mon, Sep 27, 2021 at 12:08 PM Daniel Oliveira <
> danoliveira@google.com> wrote:
> >> >>>>>>>
> >> >>>>>>> +1
> >> >>>>>>>
> >> >>>>>>> I hadn't realized the pipelines were still finishing
> successfully. I retried wordcount with that in mind and confirmed it
> finishes successfully, so this isn't a blocker.
> >> >>>>>>>
> >> >>>>>>> Although maybe we should add this to the "Known Issues" because
> I can easily see those messages being interpreted as a pipeline failure.
> >> >>>>>>>
> >> >>>>>>> On Sun, Sep 26, 2021 at 7:26 PM Robert Burke <
> robert@frantil.com> wrote:
> >> >>>>>>>>
> >> >>>>>>>> That's on the SDK side, and it just means the PCollection
> metrics are being returned, buy not handled by the SDK. At present the
> pipeline results only handle PTransform metrics.
> >> >>>>>>>>
> >> >>>>>>>> As such it's not a regression, as adding those is still under
> development.
> >> >>>>>>>>
> >> >>>>>>>> On Sun, Sep 26, 2021, 7:02 PM Daniel Oliveira <
> danoliveira@google.com> wrote:
> >> >>>>>>>>>
> >> >>>>>>>>> I tried validating wordcount with the Go SDK on Flink. The
> pipeline failed with a wall of errors like the following. I tried this on
> Flink 1.11, 1.12, and 1.13 job servers built from source at the RC1 commit,
> same errors on all of them.
> >> >>>>>>>>>
> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
> MonitoringInfo: urn:"beam:metric:element_count:v1"
> type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"
> labels:{key:"PCOLLECTION"  value:"n9"}
> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
> MonitoringInfo: urn:"beam:metric:element_count:v1"
> type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"
> labels:{key:"PCOLLECTION"  value:"n10"}
> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
> MonitoringInfo: urn:"beam:metric:element_count:v1"
> type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"
> labels:{key:"PCOLLECTION"  value:"n8"}
> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
> MonitoringInfo: urn:"beam:metric:element_count:v1"
> type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"
> labels:{key:"PCOLLECTION"  value:"n7"}
> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
> MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"
> type:"beam:metrics:distribution_int64:v1"
> payload:"\x8d%\xfa\xbf\x05\x04\xa8\x0c"  labels:{key:"PCOLLECTION"
> value:"n7"}
> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
> MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"
> type:"beam:metrics:distribution_int64:v1"  payload:"\xb9\x05\xcf6\x05\x11"
> labels:{key:"PCOLLECTION"  value:"n9"}
> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
> MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"
> type:"beam:metrics:distribution_int64:v1"  payload:"\xc5\x05\x8a1\x04\x12"
> labels:{key:"PCOLLECTION"  value:"n8"}
> >> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from
> MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"
> type:"beam:metrics:distribution_int64:v1"
> payload:"\x8d%\xb3\xa7\x07\x14\""  labels:{key:"PCOLLECTION"  value:"n10"}
> >> >>>>>>>>>> {...}
> >> >>>>>>>>>
> >> >>>>>>>>>
> >> >>>>>>>>>  @Robert Burke I think you might know what's going on here.
> Is this solvable with a cherry-pick and a new RC?
> >> >>>>>>>>>
> >> >>>>>>>>> On Fri, Sep 24, 2021 at 4:25 PM Ahmet Altay <al...@google.com>
> wrote:
> >> >>>>>>>>>>
> >> >>>>>>>>>>
> >> >>>>>>>>>>
> >> >>>>>>>>>> On Fri, Sep 24, 2021 at 4:21 PM Udi Meiri <eh...@google.com>
> wrote:
> >> >>>>>>>>>>>
> >> >>>>>>>>>>>
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> On Fri, Sep 24, 2021 at 4:18 PM Ahmet Altay <
> altay@google.com> wrote:
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> Thank you for the update related to allow_unsafe_triggers.
> My vote is still a +1.
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> If we decide to move forward with this RC, could you
> please include this bug in the known issues list under the changes.md for
> this release?
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> Yes, that's included in
> https://github.com/apache/beam/pull/15543/files.
> >> >>>>>>>>>>
> >> >>>>>>>>>>
> >> >>>>>>>>>> Thank you!
> >> >>>>>>>>>>
> >> >>>>>>>>>>>
> >> >>>>>>>>>>>
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> On Fri, Sep 24, 2021 at 3:56 PM Chamikara Jayalath <
> chamikara@google.com> wrote:
> >> >>>>>>>>>>>>>
> >> >>>>>>>>>>>>> +1 (binding)
> >> >>>>>>>>>>>>> Validated a few scenarios from the spreadsheet.
> >> >>>>>>>>>>>>>
> >> >>>>>>>>>>>>> Thanks,
> >> >>>>>>>>>>>>> Cham
> >> >>>>>>>>>>>>>
> >> >>>>>>>>>>>>> On Fri, Sep 24, 2021 at 3:07 PM Robert Bradshaw <
> robertwb@google.com> wrote:
> >> >>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>> +1 (binding)
> >> >>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>> All the artifacts and signatures look good.
> >> >>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>> I don't think the unsafe trigger check is severe enough
> to block the release.
> >> >>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>> On Fri, Sep 24, 2021 at 2:36 PM Udi Meiri <
> ehudm@google.com> wrote:
> >> >>>>>>>>>>>>>> >
> >> >>>>>>>>>>>>>> > Hi everyone, we found a bug during testing. It has to
> do with Python SDK's allow_unsafe_triggers check.
> >> >>>>>>>>>>>>>> > There is a preliminary fix that will go to master.
> >> >>>>>>>>>>>>>> >
> >> >>>>>>>>>>>>>> > For the 2.33.0 release, I'm leaning towards not making
> a new RC since there is a workaround: pass the flag --allow_unsafe_triggers.
> >> >>>>>>>>>>>>>> >
> >> >>>>>>>>>>>>>> > Please reevaluate your votes accordingly and recast if
> you've changed your vote.
> >> >>>>>>>>>>>>>> >
> >> >>>>>>>>>>>>>> > Thanks
> >> >>>>>>>>>>>>>> >
> >> >>>>>>>>>>>>>> > On Fri, Sep 24, 2021 at 12:48 PM Alexey Romanenko <
> aromanenko.dev@gmail.com> wrote:
> >> >>>>>>>>>>>>>> >>
> >> >>>>>>>>>>>>>> >>
> >> >>>>>>>>>>>>>> >> On 24 Sep 2021, at 20:45, Udi Meiri <eh...@google.com>
> wrote:
> >> >>>>>>>>>>>>>> >> Alexey is this something that we should put in the
> release notes, or some other change?
> >> >>>>>>>>>>>>>> >>
> >> >>>>>>>>>>>>>> >>
> >> >>>>>>>>>>>>>> >> Yes, I think it could be helpful to mention that Beam
> Jackson’s deps was bumped and it may require an update of Jackson’s runtime
> deps for Spark 2 users as well.
> >> >>>>>>>>>>>>>> >>
> >> >>>>>>>>>>>>>> >> —
> >> >>>>>>>>>>>>>> >> Alexey
> >> >>>>>>>>>>>>>> >>
> >> >>>>>>>>>>>>>> >>
> >> >>>>>>>>>>>>>> >>
> >> >>>>>>>>>>>>>> >>>
> >> >>>>>>>>>>>>>> >>>
> >> >>>>>>>>>>>>>> >>> —
> >> >>>>>>>>>>>>>> >>> Alexey
> >> >>>>>>>>>>>>>> >>>
> >> >>>>>>>>>>>>>> >>> On 24 Sep 2021, at 16:17, Alexey Romanenko <
> aromanenko.dev@gmail.com> wrote:
> >> >>>>>>>>>>>>>> >>>
> >> >>>>>>>>>>>>>> >>> I checked with beam-samples [1] and noticed an issue
> to run some pipelines with Spark 2 runner (Spark 3 seems is ok).
> >> >>>>>>>>>>>>>> >>> It looks that it’s caused by new Jackson's version
> updated recently [2], even if it’s a minor update but it works fine with a
> previous one.
> >> >>>>>>>>>>>>>> >>> I’ll try to find a workaround and get back with a
> results of this.
> >> >>>>>>>>>>>>>> >>>
> >> >>>>>>>>>>>>>> >>> [1] https://github.com/Talend/beam-samples/
> >> >>>>>>>>>>>>>> >>> [2]
> https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c
> >> >>>>>>>>>>>>>> >>>
> >> >>>>>>>>>>>>>> >>> —
> >> >>>>>>>>>>>>>> >>> Alexey
> >> >>>>>>>>>>>>>> >>>
> >> >>>>>>>>>>>>>> >>> On 24 Sep 2021, at 11:17, Jan Lukavský <
> je.ik@seznam.cz> wrote:
> >> >>>>>>>>>>>>>> >>>
> >> >>>>>>>>>>>>>> >>> +1 (non-binding)
> >> >>>>>>>>>>>>>> >>>
> >> >>>>>>>>>>>>>> >>> Validated several use-cases using non-portable Flink
> with Java SDK.
> >> >>>>>>>>>>>>>> >>>
> >> >>>>>>>>>>>>>> >>> On 9/24/21 4:55 AM, Valentyn Tymofieiev wrote:
> >> >>>>>>>>>>>>>> >>>
> >> >>>>>>>>>>>>>> >>> +1. Ran several Python batch and streaming pipelines
> on Dataflow and checked that Dataflow containers have required dependencies
> of Beam.
> >> >>>>>>>>>>>>>> >>>
> >> >>>>>>>>>>>>>> >>> On Thu, Sep 23, 2021 at 7:03 PM Robert Burke <
> lostluck@apache.org> wrote:
> >> >>>>>>>>>>>>>> >>>>
> >> >>>>>>>>>>>>>> >>>> +1 (non-binding)
> >> >>>>>>>>>>>>>> >>>> I validated the Go Quickstart (wordcount), and my
> ray tracer against the Go Direct runner, Dataflow, and Spark (ensuring the
> rc1 tagged container was used) and they executed successfully.
> >> >>>>>>>>>>>>>> >>>>
> >> >>>>>>>>>>>>>> >>>> I needed to manually synthesize a pseudo-version to
> ensure I was using the tagged branch version
> (v2.0.0-20210914211513-b358127f9859) instead of simply using v2.33.0-RC1.
> >> >>>>>>>>>>>>>> >>>>
> >> >>>>>>>>>>>>>> >>>>  It either can't find the package with the right
> tagged version, or it can't find the version. It's not clear to me what the
> issue is, but it's not notionally a release blocker. I'll investigate
> further once we have a full release, as it's probably some unspecified
> behavior due to how we transitioned to Go Modules (which strongly
> recommended doing a major version bump for such transitions, which seems a
> bit excessive for Beam as a whole...).
> >> >>>>>>>>>>>>>> >>>>
> >> >>>>>>>>>>>>>> >>>> On 2021/09/23 03:59:18, Ahmet Altay <
> altay@google.com> wrote:
> >> >>>>>>>>>>>>>> >>>> > +1 on the RC. I validated python quick start
> examples on direct runners.
> >> >>>>>>>>>>>>>> >>>> > Thank you Udi.
> >> >>>>>>>>>>>>>> >>>> >
> >> >>>>>>>>>>>>>> >>>> > On Wed, Sep 22, 2021 at 2:20 PM Robert Burke <
> robert@frantil.com> wrote:
> >> >>>>>>>>>>>>>> >>>> >
> >> >>>>>>>>>>>>>> >>>> > > Just an FYI that intend to validate the Go SDK
> for this release but can't
> >> >>>>>>>>>>>>>> >>>> > > get to it until tomorrow (Thursday). I'm
> catching up from a week of
> >> >>>>>>>>>>>>>> >>>> > > vacation. Apologies for the inconvenience.
> >> >>>>>>>>>>>>>> >>>> > >
> >> >>>>>>>>>>>>>> >>>> > > On Tue, Sep 21, 2021, 10:59 AM Udi Meiri <
> ehudm@google.com> wrote:
> >> >>>>>>>>>>>>>> >>>> > >
> >> >>>>>>>>>>>>>> >>>> > >> I updated the affected and fixed version
> fields for
> >> >>>>>>>>>>>>>> >>>> > >>
> https://issues.apache.org/jira/browse/BEAM-12356.
> >> >>>>>>>>>>>>>> >>>> > >>
> >> >>>>>>>>>>>>>> >>>> > >>
> >> >>>>>>>>>>>>>> >>>> > >> On Tue, Sep 21, 2021 at 10:48 AM Reuven Lax <
> relax@google.com> wrote:
> >> >>>>>>>>>>>>>> >>>> > >>
> >> >>>>>>>>>>>>>> >>>> > >>> Unfortunate - I didn't realize that
> >> >>>>>>>>>>>>>> >>>> > >>> https://github.com/apache/beam/pull/15480
> didn't make the cut.
> >> >>>>>>>>>>>>>> >>>> > >>>
> >> >>>>>>>>>>>>>> >>>> > >>> This bug was a regression in Beam 2.32.0, and
> is blocking multiple users
> >> >>>>>>>>>>>>>> >>>> > >>> from updating.
> >> >>>>>>>>>>>>>> >>>> > >>>
> >> >>>>>>>>>>>>>> >>>> > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi Meiri <
> udim@apache.org> wrote:
> >> >>>>>>>>>>>>>> >>>> > >>>
> >> >>>>>>>>>>>>>> >>>> > >>>> Hi everyone,
> >> >>>>>>>>>>>>>> >>>> > >>>> Please review and vote on the release
> candidate #1 for the version
> >> >>>>>>>>>>>>>> >>>> > >>>> 2.33.0, as follows:
> >> >>>>>>>>>>>>>> >>>> > >>>> [ ] +1, Approve the release
> >> >>>>>>>>>>>>>> >>>> > >>>> [ ] -1, Do not approve the release (please
> provide specific comments)
> >> >>>>>>>>>>>>>> >>>> > >>>>
> >> >>>>>>>>>>>>>> >>>> > >>>>
> >> >>>>>>>>>>>>>> >>>> > >>>> Reviewers are encouraged to test their own
> use cases with the release
> >> >>>>>>>>>>>>>> >>>> > >>>> candidate, and vote +1 if
> >> >>>>>>>>>>>>>> >>>> > >>>> no issues are found.
> >> >>>>>>>>>>>>>> >>>> > >>>>
> >> >>>>>>>>>>>>>> >>>> > >>>> The complete staging area is available for
> your review, which includes:
> >> >>>>>>>>>>>>>> >>>> > >>>> * JIRA release notes [1],
> >> >>>>>>>>>>>>>> >>>> > >>>> * the official Apache source release to be
> deployed to dist.apache.org
> >> >>>>>>>>>>>>>> >>>> > >>>> [2], which is signed with the key with
> fingerprint 587B049C36DAAFE6 [3],
> >> >>>>>>>>>>>>>> >>>> > >>>> * all artifacts to be deployed to the Maven
> Central Repository [4],
> >> >>>>>>>>>>>>>> >>>> > >>>> * source code tag "v2.33.0-RC1" [5],
> >> >>>>>>>>>>>>>> >>>> > >>>> * website pull request listing the release
> [6], the blog post [6], and
> >> >>>>>>>>>>>>>> >>>> > >>>> publishing the API reference manual [7].
> >> >>>>>>>>>>>>>> >>>> > >>>> * Java artifacts were built with Maven 3.6.3
> and OpenJDK 1.8.0_181.
> >> >>>>>>>>>>>>>> >>>> > >>>> * Python artifacts are deployed along with
> the source release to the
> >> >>>>>>>>>>>>>> >>>> > >>>> dist.apache.org [2] and pypy[8].
> >> >>>>>>>>>>>>>> >>>> > >>>> * Validation sheet with a tab for 2.33.0
> release to help with
> >> >>>>>>>>>>>>>> >>>> > >>>> validation [9].
> >> >>>>>>>>>>>>>> >>>> > >>>> * Docker images published to Docker Hub [10].
> >> >>>>>>>>>>>>>> >>>> > >>>>
> >> >>>>>>>>>>>>>> >>>> > >>>> The vote will be open for at least 72 hours.
> It is adopted by majority
> >> >>>>>>>>>>>>>> >>>> > >>>> approval, with at least 3 PMC affirmative
> votes.
> >> >>>>>>>>>>>>>> >>>> > >>>>
> >> >>>>>>>>>>>>>> >>>> > >>>> For guidelines on how to try the release in
> your projects, check out
> >> >>>>>>>>>>>>>> >>>> > >>>> our blog post at
> https://beam.apache.org/blog/validate-beam-release/.
> >> >>>>>>>>>>>>>> >>>> > >>>>
> >> >>>>>>>>>>>>>> >>>> > >>>> Thanks,
> >> >>>>>>>>>>>>>> >>>> > >>>> Release Manager
> >> >>>>>>>>>>>>>> >>>> > >>>>
> >> >>>>>>>>>>>>>> >>>> > >>>> [1]
> >> >>>>>>>>>>>>>> >>>> > >>>>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
> >> >>>>>>>>>>>>>> >>>> > >>>> [2]
> https://dist.apache.org/repos/dist/dev/beam/2.33.0/
> >> >>>>>>>>>>>>>> >>>> > >>>> [3]
> https://dist.apache.org/repos/dist/release/beam/KEYS
> >> >>>>>>>>>>>>>> >>>> > >>>> [4]
> >> >>>>>>>>>>>>>> >>>> > >>>>
> https://repository.apache.org/content/repositories/orgapachebeam-1234/
> >> >>>>>>>>>>>>>> >>>> > >>>> [5]
> https://github.com/apache/beam/tree/v2.33.0-RC1
> >> >>>>>>>>>>>>>> >>>> > >>>> [6]
> https://github.com/apache/beam/pull/15543
> >> >>>>>>>>>>>>>> >>>> > >>>> [7]
> https://github.com/apache/beam-site/pull/619
> >> >>>>>>>>>>>>>> >>>> > >>>> [8]
> https://pypi.org/project/apache-beam/2.33.0rc1/
> >> >>>>>>>>>>>>>> >>>> > >>>> [9]
> >> >>>>>>>>>>>>>> >>>> > >>>>
> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
> >> >>>>>>>>>>>>>> >>>> > >>>> [10]
> https://hub.docker.com/search?q=apache%2Fbeam&type=image
> >> >>>>>>>>>>>>>> >>>> > >>>>
> >> >>>>>>>>>>>>>> >>>> > >>>>
> >> >>>>>>>>>>>>>> >>>> >
> >> >>>>>>>>>>>>>> >>>
> >> >>>>>>>>>>>>>> >>>
> >> >>>>>>>>>>>>>> >>>
> >> >>>>>>>>>>>>>> >>
>

Re: [VOTE] Release 2.33.0, release candidate 1

[Robert Bradshaw <ro...@google.com>]

On Mon, Sep 27, 2021 at 2:42 PM Kenneth Knowles <ke...@apache.org> wrote:
>
> True. It is the use case of Repeatedly(ProcessingTime(n)) that I would guess to be the primary case of concern.

This shouldn't require the workaround, right?

If you're not comfortable with this, I'd say let's patch and re-cut.


> On Mon, Sep 27, 2021 at 2:38 PM Robert Bradshaw <ro...@google.com> wrote:
>>
>> As I read the code, it's only pipelines that use a Repeatedly trigger
>> that wrap an already lossy trigger that are declared to be themselves
>> lossy. If I'm mistaken, I'll certainly reconsider my vote (and thanks
>> for bringing this up).
>>
>> On Mon, Sep 27, 2021 at 2:21 PM Kenneth Knowles <ke...@apache.org> wrote:
>> >
>> > My concern is that the error message is incorrect and every user of 2.33.0 may be educated wrong, or be worried about data loss in Beam, or fail to find the blog post or CHANGES, etc.
>> >
>> > Kenn
>> >
>> > On Mon, Sep 27, 2021 at 2:16 PM Udi Meiri <eh...@google.com> wrote:
>> >>
>> >> I don't know how rare it is, but there is a flag documented in CHANGES and blog post that reverts to the old behavior.
>> >>
>> >> On Mon, Sep 27, 2021 at 2:12 PM Kenneth Knowles <ke...@apache.org> wrote:
>> >>>
>> >>> I guess my vote is -0 since I don't have enough context on this issue. A number of people with more awareness of how severe this is have voted +1 so I will not try to block the release.
>> >>>
>> >>> Kenn
>> >>>
>> >>> On Mon, Sep 27, 2021 at 2:11 PM Kenneth Knowles <ke...@apache.org> wrote:
>> >>>>
>> >>>> I have to disagree with the other PMC members here, or at least dig in to the question: every pipeline that uses a Repeatedly trigger at the top level will be rejected. Is this so rare in Python that it is OK?
>> >>>>
>> >>>> Kenn
>> >>>>
>> >>>> On Mon, Sep 27, 2021 at 1:56 PM Robert Burke <re...@google.com> wrote:
>> >>>>>
>> >>>>> On it. Thanks!
>> >>>>>
>> >>>>> On Mon, Sep 27, 2021 at 1:18 PM Udi Meiri <eh...@google.com> wrote:
>> >>>>>>
>> >>>>>> Daniel/Robert, feel free to make changes to this PR: https://github.com/apache/beam/pull/15543
>> >>>>>>
>> >>>>>> On Mon, Sep 27, 2021 at 12:08 PM Daniel Oliveira <da...@google.com> wrote:
>> >>>>>>>
>> >>>>>>> +1
>> >>>>>>>
>> >>>>>>> I hadn't realized the pipelines were still finishing successfully. I retried wordcount with that in mind and confirmed it finishes successfully, so this isn't a blocker.
>> >>>>>>>
>> >>>>>>> Although maybe we should add this to the "Known Issues" because I can easily see those messages being interpreted as a pipeline failure.
>> >>>>>>>
>> >>>>>>> On Sun, Sep 26, 2021 at 7:26 PM Robert Burke <ro...@frantil.com> wrote:
>> >>>>>>>>
>> >>>>>>>> That's on the SDK side, and it just means the PCollection metrics are being returned, buy not handled by the SDK. At present the pipeline results only handle PTransform metrics.
>> >>>>>>>>
>> >>>>>>>> As such it's not a regression, as adding those is still under development.
>> >>>>>>>>
>> >>>>>>>> On Sun, Sep 26, 2021, 7:02 PM Daniel Oliveira <da...@google.com> wrote:
>> >>>>>>>>>
>> >>>>>>>>> I tried validating wordcount with the Go SDK on Flink. The pipeline failed with a wall of errors like the following. I tried this on Flink 1.11, 1.12, and 1.13 job servers built from source at the RC1 commit, same errors on all of them.
>> >>>>>>>>>
>> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n9"}
>> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n10"}
>> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n8"}
>> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n7"}
>> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"  type:"beam:metrics:distribution_int64:v1"  payload:"\x8d%\xfa\xbf\x05\x04\xa8\x0c"  labels:{key:"PCOLLECTION"  value:"n7"}
>> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"  type:"beam:metrics:distribution_int64:v1"  payload:"\xb9\x05\xcf6\x05\x11"  labels:{key:"PCOLLECTION"  value:"n9"}
>> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"  type:"beam:metrics:distribution_int64:v1"  payload:"\xc5\x05\x8a1\x04\x12"  labels:{key:"PCOLLECTION"  value:"n8"}
>> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"  type:"beam:metrics:distribution_int64:v1"  payload:"\x8d%\xb3\xa7\x07\x14\""  labels:{key:"PCOLLECTION"  value:"n10"}
>> >>>>>>>>>> {...}
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>  @Robert Burke I think you might know what's going on here. Is this solvable with a cherry-pick and a new RC?
>> >>>>>>>>>
>> >>>>>>>>> On Fri, Sep 24, 2021 at 4:25 PM Ahmet Altay <al...@google.com> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> On Fri, Sep 24, 2021 at 4:21 PM Udi Meiri <eh...@google.com> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Fri, Sep 24, 2021 at 4:18 PM Ahmet Altay <al...@google.com> wrote:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Thank you for the update related to allow_unsafe_triggers. My vote is still a +1.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> If we decide to move forward with this RC, could you please include this bug in the known issues list under the changes.md for this release?
>> >>>>>>>>>>>
>> >>>>>>>>>>> Yes, that's included in https://github.com/apache/beam/pull/15543/files.
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> Thank you!
>> >>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On Fri, Sep 24, 2021 at 3:56 PM Chamikara Jayalath <ch...@google.com> wrote:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> +1 (binding)
>> >>>>>>>>>>>>> Validated a few scenarios from the spreadsheet.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>> Cham
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> On Fri, Sep 24, 2021 at 3:07 PM Robert Bradshaw <ro...@google.com> wrote:
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> +1 (binding)
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> All the artifacts and signatures look good.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> I don't think the unsafe trigger check is severe enough to block the release.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> On Fri, Sep 24, 2021 at 2:36 PM Udi Meiri <eh...@google.com> wrote:
>> >>>>>>>>>>>>>> >
>> >>>>>>>>>>>>>> > Hi everyone, we found a bug during testing. It has to do with Python SDK's allow_unsafe_triggers check.
>> >>>>>>>>>>>>>> > There is a preliminary fix that will go to master.
>> >>>>>>>>>>>>>> >
>> >>>>>>>>>>>>>> > For the 2.33.0 release, I'm leaning towards not making a new RC since there is a workaround: pass the flag --allow_unsafe_triggers.
>> >>>>>>>>>>>>>> >
>> >>>>>>>>>>>>>> > Please reevaluate your votes accordingly and recast if you've changed your vote.
>> >>>>>>>>>>>>>> >
>> >>>>>>>>>>>>>> > Thanks
>> >>>>>>>>>>>>>> >
>> >>>>>>>>>>>>>> > On Fri, Sep 24, 2021 at 12:48 PM Alexey Romanenko <ar...@gmail.com> wrote:
>> >>>>>>>>>>>>>> >>
>> >>>>>>>>>>>>>> >>
>> >>>>>>>>>>>>>> >> On 24 Sep 2021, at 20:45, Udi Meiri <eh...@google.com> wrote:
>> >>>>>>>>>>>>>> >> Alexey is this something that we should put in the release notes, or some other change?
>> >>>>>>>>>>>>>> >>
>> >>>>>>>>>>>>>> >>
>> >>>>>>>>>>>>>> >> Yes, I think it could be helpful to mention that Beam Jackson’s deps was bumped and it may require an update of Jackson’s runtime deps for Spark 2 users as well.
>> >>>>>>>>>>>>>> >>
>> >>>>>>>>>>>>>> >> —
>> >>>>>>>>>>>>>> >> Alexey
>> >>>>>>>>>>>>>> >>
>> >>>>>>>>>>>>>> >>
>> >>>>>>>>>>>>>> >>
>> >>>>>>>>>>>>>> >>>
>> >>>>>>>>>>>>>> >>>
>> >>>>>>>>>>>>>> >>> —
>> >>>>>>>>>>>>>> >>> Alexey
>> >>>>>>>>>>>>>> >>>
>> >>>>>>>>>>>>>> >>> On 24 Sep 2021, at 16:17, Alexey Romanenko <ar...@gmail.com> wrote:
>> >>>>>>>>>>>>>> >>>
>> >>>>>>>>>>>>>> >>> I checked with beam-samples [1] and noticed an issue to run some pipelines with Spark 2 runner (Spark 3 seems is ok).
>> >>>>>>>>>>>>>> >>> It looks that it’s caused by new Jackson's version updated recently [2], even if it’s a minor update but it works fine with a previous one.
>> >>>>>>>>>>>>>> >>> I’ll try to find a workaround and get back with a results of this.
>> >>>>>>>>>>>>>> >>>
>> >>>>>>>>>>>>>> >>> [1] https://github.com/Talend/beam-samples/
>> >>>>>>>>>>>>>> >>> [2] https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c
>> >>>>>>>>>>>>>> >>>
>> >>>>>>>>>>>>>> >>> —
>> >>>>>>>>>>>>>> >>> Alexey
>> >>>>>>>>>>>>>> >>>
>> >>>>>>>>>>>>>> >>> On 24 Sep 2021, at 11:17, Jan Lukavský <je...@seznam.cz> wrote:
>> >>>>>>>>>>>>>> >>>
>> >>>>>>>>>>>>>> >>> +1 (non-binding)
>> >>>>>>>>>>>>>> >>>
>> >>>>>>>>>>>>>> >>> Validated several use-cases using non-portable Flink with Java SDK.
>> >>>>>>>>>>>>>> >>>
>> >>>>>>>>>>>>>> >>> On 9/24/21 4:55 AM, Valentyn Tymofieiev wrote:
>> >>>>>>>>>>>>>> >>>
>> >>>>>>>>>>>>>> >>> +1. Ran several Python batch and streaming pipelines on Dataflow and checked that Dataflow containers have required dependencies of Beam.
>> >>>>>>>>>>>>>> >>>
>> >>>>>>>>>>>>>> >>> On Thu, Sep 23, 2021 at 7:03 PM Robert Burke <lo...@apache.org> wrote:
>> >>>>>>>>>>>>>> >>>>
>> >>>>>>>>>>>>>> >>>> +1 (non-binding)
>> >>>>>>>>>>>>>> >>>> I validated the Go Quickstart (wordcount), and my ray tracer against the Go Direct runner, Dataflow, and Spark (ensuring the rc1 tagged container was used) and they executed successfully.
>> >>>>>>>>>>>>>> >>>>
>> >>>>>>>>>>>>>> >>>> I needed to manually synthesize a pseudo-version to ensure I was using the tagged branch version (v2.0.0-20210914211513-b358127f9859) instead of simply using v2.33.0-RC1.
>> >>>>>>>>>>>>>> >>>>
>> >>>>>>>>>>>>>> >>>>  It either can't find the package with the right tagged version, or it can't find the version. It's not clear to me what the issue is, but it's not notionally a release blocker. I'll investigate further once we have a full release, as it's probably some unspecified behavior due to how we transitioned to Go Modules (which strongly recommended doing a major version bump for such transitions, which seems a bit excessive for Beam as a whole...).
>> >>>>>>>>>>>>>> >>>>
>> >>>>>>>>>>>>>> >>>> On 2021/09/23 03:59:18, Ahmet Altay <al...@google.com> wrote:
>> >>>>>>>>>>>>>> >>>> > +1 on the RC. I validated python quick start examples on direct runners.
>> >>>>>>>>>>>>>> >>>> > Thank you Udi.
>> >>>>>>>>>>>>>> >>>> >
>> >>>>>>>>>>>>>> >>>> > On Wed, Sep 22, 2021 at 2:20 PM Robert Burke <ro...@frantil.com> wrote:
>> >>>>>>>>>>>>>> >>>> >
>> >>>>>>>>>>>>>> >>>> > > Just an FYI that intend to validate the Go SDK for this release but can't
>> >>>>>>>>>>>>>> >>>> > > get to it until tomorrow (Thursday). I'm catching up from a week of
>> >>>>>>>>>>>>>> >>>> > > vacation. Apologies for the inconvenience.
>> >>>>>>>>>>>>>> >>>> > >
>> >>>>>>>>>>>>>> >>>> > > On Tue, Sep 21, 2021, 10:59 AM Udi Meiri <eh...@google.com> wrote:
>> >>>>>>>>>>>>>> >>>> > >
>> >>>>>>>>>>>>>> >>>> > >> I updated the affected and fixed version fields for
>> >>>>>>>>>>>>>> >>>> > >> https://issues.apache.org/jira/browse/BEAM-12356.
>> >>>>>>>>>>>>>> >>>> > >>
>> >>>>>>>>>>>>>> >>>> > >>
>> >>>>>>>>>>>>>> >>>> > >> On Tue, Sep 21, 2021 at 10:48 AM Reuven Lax <re...@google.com> wrote:
>> >>>>>>>>>>>>>> >>>> > >>
>> >>>>>>>>>>>>>> >>>> > >>> Unfortunate - I didn't realize that
>> >>>>>>>>>>>>>> >>>> > >>> https://github.com/apache/beam/pull/15480 didn't make the cut.
>> >>>>>>>>>>>>>> >>>> > >>>
>> >>>>>>>>>>>>>> >>>> > >>> This bug was a regression in Beam 2.32.0, and is blocking multiple users
>> >>>>>>>>>>>>>> >>>> > >>> from updating.
>> >>>>>>>>>>>>>> >>>> > >>>
>> >>>>>>>>>>>>>> >>>> > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi Meiri <ud...@apache.org> wrote:
>> >>>>>>>>>>>>>> >>>> > >>>
>> >>>>>>>>>>>>>> >>>> > >>>> Hi everyone,
>> >>>>>>>>>>>>>> >>>> > >>>> Please review and vote on the release candidate #1 for the version
>> >>>>>>>>>>>>>> >>>> > >>>> 2.33.0, as follows:
>> >>>>>>>>>>>>>> >>>> > >>>> [ ] +1, Approve the release
>> >>>>>>>>>>>>>> >>>> > >>>> [ ] -1, Do not approve the release (please provide specific comments)
>> >>>>>>>>>>>>>> >>>> > >>>>
>> >>>>>>>>>>>>>> >>>> > >>>>
>> >>>>>>>>>>>>>> >>>> > >>>> Reviewers are encouraged to test their own use cases with the release
>> >>>>>>>>>>>>>> >>>> > >>>> candidate, and vote +1 if
>> >>>>>>>>>>>>>> >>>> > >>>> no issues are found.
>> >>>>>>>>>>>>>> >>>> > >>>>
>> >>>>>>>>>>>>>> >>>> > >>>> The complete staging area is available for your review, which includes:
>> >>>>>>>>>>>>>> >>>> > >>>> * JIRA release notes [1],
>> >>>>>>>>>>>>>> >>>> > >>>> * the official Apache source release to be deployed to dist.apache.org
>> >>>>>>>>>>>>>> >>>> > >>>> [2], which is signed with the key with fingerprint 587B049C36DAAFE6 [3],
>> >>>>>>>>>>>>>> >>>> > >>>> * all artifacts to be deployed to the Maven Central Repository [4],
>> >>>>>>>>>>>>>> >>>> > >>>> * source code tag "v2.33.0-RC1" [5],
>> >>>>>>>>>>>>>> >>>> > >>>> * website pull request listing the release [6], the blog post [6], and
>> >>>>>>>>>>>>>> >>>> > >>>> publishing the API reference manual [7].
>> >>>>>>>>>>>>>> >>>> > >>>> * Java artifacts were built with Maven 3.6.3 and OpenJDK 1.8.0_181.
>> >>>>>>>>>>>>>> >>>> > >>>> * Python artifacts are deployed along with the source release to the
>> >>>>>>>>>>>>>> >>>> > >>>> dist.apache.org [2] and pypy[8].
>> >>>>>>>>>>>>>> >>>> > >>>> * Validation sheet with a tab for 2.33.0 release to help with
>> >>>>>>>>>>>>>> >>>> > >>>> validation [9].
>> >>>>>>>>>>>>>> >>>> > >>>> * Docker images published to Docker Hub [10].
>> >>>>>>>>>>>>>> >>>> > >>>>
>> >>>>>>>>>>>>>> >>>> > >>>> The vote will be open for at least 72 hours. It is adopted by majority
>> >>>>>>>>>>>>>> >>>> > >>>> approval, with at least 3 PMC affirmative votes.
>> >>>>>>>>>>>>>> >>>> > >>>>
>> >>>>>>>>>>>>>> >>>> > >>>> For guidelines on how to try the release in your projects, check out
>> >>>>>>>>>>>>>> >>>> > >>>> our blog post at https://beam.apache.org/blog/validate-beam-release/.
>> >>>>>>>>>>>>>> >>>> > >>>>
>> >>>>>>>>>>>>>> >>>> > >>>> Thanks,
>> >>>>>>>>>>>>>> >>>> > >>>> Release Manager
>> >>>>>>>>>>>>>> >>>> > >>>>
>> >>>>>>>>>>>>>> >>>> > >>>> [1]
>> >>>>>>>>>>>>>> >>>> > >>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
>> >>>>>>>>>>>>>> >>>> > >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.33.0/
>> >>>>>>>>>>>>>> >>>> > >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>> >>>>>>>>>>>>>> >>>> > >>>> [4]
>> >>>>>>>>>>>>>> >>>> > >>>> https://repository.apache.org/content/repositories/orgapachebeam-1234/
>> >>>>>>>>>>>>>> >>>> > >>>> [5] https://github.com/apache/beam/tree/v2.33.0-RC1
>> >>>>>>>>>>>>>> >>>> > >>>> [6] https://github.com/apache/beam/pull/15543
>> >>>>>>>>>>>>>> >>>> > >>>> [7] https://github.com/apache/beam-site/pull/619
>> >>>>>>>>>>>>>> >>>> > >>>> [8] https://pypi.org/project/apache-beam/2.33.0rc1/
>> >>>>>>>>>>>>>> >>>> > >>>> [9]
>> >>>>>>>>>>>>>> >>>> > >>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
>> >>>>>>>>>>>>>> >>>> > >>>> [10] https://hub.docker.com/search?q=apache%2Fbeam&type=image
>> >>>>>>>>>>>>>> >>>> > >>>>
>> >>>>>>>>>>>>>> >>>> > >>>>
>> >>>>>>>>>>>>>> >>>> >
>> >>>>>>>>>>>>>> >>>
>> >>>>>>>>>>>>>> >>>
>> >>>>>>>>>>>>>> >>>
>> >>>>>>>>>>>>>> >>

Re: [VOTE] Release 2.33.0, release candidate 1

[Kenneth Knowles <ke...@apache.org>]

True. It is the use case of Repeatedly(ProcessingTime(n)) that I would
guess to be the primary case of concern.

Kenn

On Mon, Sep 27, 2021 at 2:38 PM Robert Bradshaw <ro...@google.com> wrote:

> As I read the code, it's only pipelines that use a Repeatedly trigger
> that wrap an already lossy trigger that are declared to be themselves
> lossy. If I'm mistaken, I'll certainly reconsider my vote (and thanks
> for bringing this up).
>
> On Mon, Sep 27, 2021 at 2:21 PM Kenneth Knowles <ke...@apache.org> wrote:
> >
> > My concern is that the error message is incorrect and every user of
> 2.33.0 may be educated wrong, or be worried about data loss in Beam, or
> fail to find the blog post or CHANGES, etc.
> >
> > Kenn
> >
> > On Mon, Sep 27, 2021 at 2:16 PM Udi Meiri <eh...@google.com> wrote:
> >>
> >> I don't know how rare it is, but there is a flag documented in CHANGES
> and blog post that reverts to the old behavior.
> >>
> >> On Mon, Sep 27, 2021 at 2:12 PM Kenneth Knowles <ke...@apache.org>
> wrote:
> >>>
> >>> I guess my vote is -0 since I don't have enough context on this issue.
> A number of people with more awareness of how severe this is have voted +1
> so I will not try to block the release.
> >>>
> >>> Kenn
> >>>
> >>> On Mon, Sep 27, 2021 at 2:11 PM Kenneth Knowles <ke...@apache.org>
> wrote:
> >>>>
> >>>> I have to disagree with the other PMC members here, or at least dig
> in to the question: every pipeline that uses a Repeatedly trigger at the
> top level will be rejected. Is this so rare in Python that it is OK?
> >>>>
> >>>> Kenn
> >>>>
> >>>> On Mon, Sep 27, 2021 at 1:56 PM Robert Burke <re...@google.com> wrote:
> >>>>>
> >>>>> On it. Thanks!
> >>>>>
> >>>>> On Mon, Sep 27, 2021 at 1:18 PM Udi Meiri <eh...@google.com> wrote:
> >>>>>>
> >>>>>> Daniel/Robert, feel free to make changes to this PR:
> https://github.com/apache/beam/pull/15543
> >>>>>>
> >>>>>> On Mon, Sep 27, 2021 at 12:08 PM Daniel Oliveira <
> danoliveira@google.com> wrote:
> >>>>>>>
> >>>>>>> +1
> >>>>>>>
> >>>>>>> I hadn't realized the pipelines were still finishing successfully.
> I retried wordcount with that in mind and confirmed it finishes
> successfully, so this isn't a blocker.
> >>>>>>>
> >>>>>>> Although maybe we should add this to the "Known Issues" because I
> can easily see those messages being interpreted as a pipeline failure.
> >>>>>>>
> >>>>>>> On Sun, Sep 26, 2021 at 7:26 PM Robert Burke <ro...@frantil.com>
> wrote:
> >>>>>>>>
> >>>>>>>> That's on the SDK side, and it just means the PCollection metrics
> are being returned, buy not handled by the SDK. At present the pipeline
> results only handle PTransform metrics.
> >>>>>>>>
> >>>>>>>> As such it's not a regression, as adding those is still under
> development.
> >>>>>>>>
> >>>>>>>> On Sun, Sep 26, 2021, 7:02 PM Daniel Oliveira <
> danoliveira@google.com> wrote:
> >>>>>>>>>
> >>>>>>>>> I tried validating wordcount with the Go SDK on Flink. The
> pipeline failed with a wall of errors like the following. I tried this on
> Flink 1.11, 1.12, and 1.13 job servers built from source at the RC1 commit,
> same errors on all of them.
> >>>>>>>>>
> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
> payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n9"}
> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
> payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n10"}
> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
> payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n8"}
> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
> payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n7"}
> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
> urn:"beam:metric:sampled_byte_size:v1"
> type:"beam:metrics:distribution_int64:v1"
> payload:"\x8d%\xfa\xbf\x05\x04\xa8\x0c"  labels:{key:"PCOLLECTION"
> value:"n7"}
> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
> urn:"beam:metric:sampled_byte_size:v1"
> type:"beam:metrics:distribution_int64:v1"  payload:"\xb9\x05\xcf6\x05\x11"
> labels:{key:"PCOLLECTION"  value:"n9"}
> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
> urn:"beam:metric:sampled_byte_size:v1"
> type:"beam:metrics:distribution_int64:v1"  payload:"\xc5\x05\x8a1\x04\x12"
> labels:{key:"PCOLLECTION"  value:"n8"}
> >>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
> urn:"beam:metric:sampled_byte_size:v1"
> type:"beam:metrics:distribution_int64:v1"
> payload:"\x8d%\xb3\xa7\x07\x14\""  labels:{key:"PCOLLECTION"  value:"n10"}
> >>>>>>>>>> {...}
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>  @Robert Burke I think you might know what's going on here. Is
> this solvable with a cherry-pick and a new RC?
> >>>>>>>>>
> >>>>>>>>> On Fri, Sep 24, 2021 at 4:25 PM Ahmet Altay <al...@google.com>
> wrote:
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Fri, Sep 24, 2021 at 4:21 PM Udi Meiri <eh...@google.com>
> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Fri, Sep 24, 2021 at 4:18 PM Ahmet Altay <al...@google.com>
> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thank you for the update related to allow_unsafe_triggers. My
> vote is still a +1.
> >>>>>>>>>>>>
> >>>>>>>>>>>> If we decide to move forward with this RC, could you please
> include this bug in the known issues list under the changes.md for this
> release?
> >>>>>>>>>>>
> >>>>>>>>>>> Yes, that's included in
> https://github.com/apache/beam/pull/15543/files.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Thank you!
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Fri, Sep 24, 2021 at 3:56 PM Chamikara Jayalath <
> chamikara@google.com> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> +1 (binding)
> >>>>>>>>>>>>> Validated a few scenarios from the spreadsheet.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>> Cham
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Fri, Sep 24, 2021 at 3:07 PM Robert Bradshaw <
> robertwb@google.com> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> +1 (binding)
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> All the artifacts and signatures look good.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I don't think the unsafe trigger check is severe enough to
> block the release.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Fri, Sep 24, 2021 at 2:36 PM Udi Meiri <eh...@google.com>
> wrote:
> >>>>>>>>>>>>>> >
> >>>>>>>>>>>>>> > Hi everyone, we found a bug during testing. It has to do
> with Python SDK's allow_unsafe_triggers check.
> >>>>>>>>>>>>>> > There is a preliminary fix that will go to master.
> >>>>>>>>>>>>>> >
> >>>>>>>>>>>>>> > For the 2.33.0 release, I'm leaning towards not making a
> new RC since there is a workaround: pass the flag --allow_unsafe_triggers.
> >>>>>>>>>>>>>> >
> >>>>>>>>>>>>>> > Please reevaluate your votes accordingly and recast if
> you've changed your vote.
> >>>>>>>>>>>>>> >
> >>>>>>>>>>>>>> > Thanks
> >>>>>>>>>>>>>> >
> >>>>>>>>>>>>>> > On Fri, Sep 24, 2021 at 12:48 PM Alexey Romanenko <
> aromanenko.dev@gmail.com> wrote:
> >>>>>>>>>>>>>> >>
> >>>>>>>>>>>>>> >>
> >>>>>>>>>>>>>> >> On 24 Sep 2021, at 20:45, Udi Meiri <eh...@google.com>
> wrote:
> >>>>>>>>>>>>>> >> Alexey is this something that we should put in the
> release notes, or some other change?
> >>>>>>>>>>>>>> >>
> >>>>>>>>>>>>>> >>
> >>>>>>>>>>>>>> >> Yes, I think it could be helpful to mention that Beam
> Jackson’s deps was bumped and it may require an update of Jackson’s runtime
> deps for Spark 2 users as well.
> >>>>>>>>>>>>>> >>
> >>>>>>>>>>>>>> >> —
> >>>>>>>>>>>>>> >> Alexey
> >>>>>>>>>>>>>> >>
> >>>>>>>>>>>>>> >>
> >>>>>>>>>>>>>> >>
> >>>>>>>>>>>>>> >>>
> >>>>>>>>>>>>>> >>>
> >>>>>>>>>>>>>> >>> —
> >>>>>>>>>>>>>> >>> Alexey
> >>>>>>>>>>>>>> >>>
> >>>>>>>>>>>>>> >>> On 24 Sep 2021, at 16:17, Alexey Romanenko <
> aromanenko.dev@gmail.com> wrote:
> >>>>>>>>>>>>>> >>>
> >>>>>>>>>>>>>> >>> I checked with beam-samples [1] and noticed an issue to
> run some pipelines with Spark 2 runner (Spark 3 seems is ok).
> >>>>>>>>>>>>>> >>> It looks that it’s caused by new Jackson's version
> updated recently [2], even if it’s a minor update but it works fine with a
> previous one.
> >>>>>>>>>>>>>> >>> I’ll try to find a workaround and get back with a
> results of this.
> >>>>>>>>>>>>>> >>>
> >>>>>>>>>>>>>> >>> [1] https://github.com/Talend/beam-samples/
> >>>>>>>>>>>>>> >>> [2]
> https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c
> >>>>>>>>>>>>>> >>>
> >>>>>>>>>>>>>> >>> —
> >>>>>>>>>>>>>> >>> Alexey
> >>>>>>>>>>>>>> >>>
> >>>>>>>>>>>>>> >>> On 24 Sep 2021, at 11:17, Jan Lukavský <je...@seznam.cz>
> wrote:
> >>>>>>>>>>>>>> >>>
> >>>>>>>>>>>>>> >>> +1 (non-binding)
> >>>>>>>>>>>>>> >>>
> >>>>>>>>>>>>>> >>> Validated several use-cases using non-portable Flink
> with Java SDK.
> >>>>>>>>>>>>>> >>>
> >>>>>>>>>>>>>> >>> On 9/24/21 4:55 AM, Valentyn Tymofieiev wrote:
> >>>>>>>>>>>>>> >>>
> >>>>>>>>>>>>>> >>> +1. Ran several Python batch and streaming pipelines on
> Dataflow and checked that Dataflow containers have required dependencies of
> Beam.
> >>>>>>>>>>>>>> >>>
> >>>>>>>>>>>>>> >>> On Thu, Sep 23, 2021 at 7:03 PM Robert Burke <
> lostluck@apache.org> wrote:
> >>>>>>>>>>>>>> >>>>
> >>>>>>>>>>>>>> >>>> +1 (non-binding)
> >>>>>>>>>>>>>> >>>> I validated the Go Quickstart (wordcount), and my ray
> tracer against the Go Direct runner, Dataflow, and Spark (ensuring the rc1
> tagged container was used) and they executed successfully.
> >>>>>>>>>>>>>> >>>>
> >>>>>>>>>>>>>> >>>> I needed to manually synthesize a pseudo-version to
> ensure I was using the tagged branch version
> (v2.0.0-20210914211513-b358127f9859) instead of simply using v2.33.0-RC1.
> >>>>>>>>>>>>>> >>>>
> >>>>>>>>>>>>>> >>>>  It either can't find the package with the right
> tagged version, or it can't find the version. It's not clear to me what the
> issue is, but it's not notionally a release blocker. I'll investigate
> further once we have a full release, as it's probably some unspecified
> behavior due to how we transitioned to Go Modules (which strongly
> recommended doing a major version bump for such transitions, which seems a
> bit excessive for Beam as a whole...).
> >>>>>>>>>>>>>> >>>>
> >>>>>>>>>>>>>> >>>> On 2021/09/23 03:59:18, Ahmet Altay <al...@google.com>
> wrote:
> >>>>>>>>>>>>>> >>>> > +1 on the RC. I validated python quick start
> examples on direct runners.
> >>>>>>>>>>>>>> >>>> > Thank you Udi.
> >>>>>>>>>>>>>> >>>> >
> >>>>>>>>>>>>>> >>>> > On Wed, Sep 22, 2021 at 2:20 PM Robert Burke <
> robert@frantil.com> wrote:
> >>>>>>>>>>>>>> >>>> >
> >>>>>>>>>>>>>> >>>> > > Just an FYI that intend to validate the Go SDK for
> this release but can't
> >>>>>>>>>>>>>> >>>> > > get to it until tomorrow (Thursday). I'm catching
> up from a week of
> >>>>>>>>>>>>>> >>>> > > vacation. Apologies for the inconvenience.
> >>>>>>>>>>>>>> >>>> > >
> >>>>>>>>>>>>>> >>>> > > On Tue, Sep 21, 2021, 10:59 AM Udi Meiri <
> ehudm@google.com> wrote:
> >>>>>>>>>>>>>> >>>> > >
> >>>>>>>>>>>>>> >>>> > >> I updated the affected and fixed version fields
> for
> >>>>>>>>>>>>>> >>>> > >> https://issues.apache.org/jira/browse/BEAM-12356.
> >>>>>>>>>>>>>> >>>> > >>
> >>>>>>>>>>>>>> >>>> > >>
> >>>>>>>>>>>>>> >>>> > >> On Tue, Sep 21, 2021 at 10:48 AM Reuven Lax <
> relax@google.com> wrote:
> >>>>>>>>>>>>>> >>>> > >>
> >>>>>>>>>>>>>> >>>> > >>> Unfortunate - I didn't realize that
> >>>>>>>>>>>>>> >>>> > >>> https://github.com/apache/beam/pull/15480
> didn't make the cut.
> >>>>>>>>>>>>>> >>>> > >>>
> >>>>>>>>>>>>>> >>>> > >>> This bug was a regression in Beam 2.32.0, and is
> blocking multiple users
> >>>>>>>>>>>>>> >>>> > >>> from updating.
> >>>>>>>>>>>>>> >>>> > >>>
> >>>>>>>>>>>>>> >>>> > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi Meiri <
> udim@apache.org> wrote:
> >>>>>>>>>>>>>> >>>> > >>>
> >>>>>>>>>>>>>> >>>> > >>>> Hi everyone,
> >>>>>>>>>>>>>> >>>> > >>>> Please review and vote on the release candidate
> #1 for the version
> >>>>>>>>>>>>>> >>>> > >>>> 2.33.0, as follows:
> >>>>>>>>>>>>>> >>>> > >>>> [ ] +1, Approve the release
> >>>>>>>>>>>>>> >>>> > >>>> [ ] -1, Do not approve the release (please
> provide specific comments)
> >>>>>>>>>>>>>> >>>> > >>>>
> >>>>>>>>>>>>>> >>>> > >>>>
> >>>>>>>>>>>>>> >>>> > >>>> Reviewers are encouraged to test their own use
> cases with the release
> >>>>>>>>>>>>>> >>>> > >>>> candidate, and vote +1 if
> >>>>>>>>>>>>>> >>>> > >>>> no issues are found.
> >>>>>>>>>>>>>> >>>> > >>>>
> >>>>>>>>>>>>>> >>>> > >>>> The complete staging area is available for your
> review, which includes:
> >>>>>>>>>>>>>> >>>> > >>>> * JIRA release notes [1],
> >>>>>>>>>>>>>> >>>> > >>>> * the official Apache source release to be
> deployed to dist.apache.org
> >>>>>>>>>>>>>> >>>> > >>>> [2], which is signed with the key with
> fingerprint 587B049C36DAAFE6 [3],
> >>>>>>>>>>>>>> >>>> > >>>> * all artifacts to be deployed to the Maven
> Central Repository [4],
> >>>>>>>>>>>>>> >>>> > >>>> * source code tag "v2.33.0-RC1" [5],
> >>>>>>>>>>>>>> >>>> > >>>> * website pull request listing the release [6],
> the blog post [6], and
> >>>>>>>>>>>>>> >>>> > >>>> publishing the API reference manual [7].
> >>>>>>>>>>>>>> >>>> > >>>> * Java artifacts were built with Maven 3.6.3
> and OpenJDK 1.8.0_181.
> >>>>>>>>>>>>>> >>>> > >>>> * Python artifacts are deployed along with the
> source release to the
> >>>>>>>>>>>>>> >>>> > >>>> dist.apache.org [2] and pypy[8].
> >>>>>>>>>>>>>> >>>> > >>>> * Validation sheet with a tab for 2.33.0
> release to help with
> >>>>>>>>>>>>>> >>>> > >>>> validation [9].
> >>>>>>>>>>>>>> >>>> > >>>> * Docker images published to Docker Hub [10].
> >>>>>>>>>>>>>> >>>> > >>>>
> >>>>>>>>>>>>>> >>>> > >>>> The vote will be open for at least 72 hours. It
> is adopted by majority
> >>>>>>>>>>>>>> >>>> > >>>> approval, with at least 3 PMC affirmative votes.
> >>>>>>>>>>>>>> >>>> > >>>>
> >>>>>>>>>>>>>> >>>> > >>>> For guidelines on how to try the release in
> your projects, check out
> >>>>>>>>>>>>>> >>>> > >>>> our blog post at
> https://beam.apache.org/blog/validate-beam-release/.
> >>>>>>>>>>>>>> >>>> > >>>>
> >>>>>>>>>>>>>> >>>> > >>>> Thanks,
> >>>>>>>>>>>>>> >>>> > >>>> Release Manager
> >>>>>>>>>>>>>> >>>> > >>>>
> >>>>>>>>>>>>>> >>>> > >>>> [1]
> >>>>>>>>>>>>>> >>>> > >>>>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
> >>>>>>>>>>>>>> >>>> > >>>> [2]
> https://dist.apache.org/repos/dist/dev/beam/2.33.0/
> >>>>>>>>>>>>>> >>>> > >>>> [3]
> https://dist.apache.org/repos/dist/release/beam/KEYS
> >>>>>>>>>>>>>> >>>> > >>>> [4]
> >>>>>>>>>>>>>> >>>> > >>>>
> https://repository.apache.org/content/repositories/orgapachebeam-1234/
> >>>>>>>>>>>>>> >>>> > >>>> [5]
> https://github.com/apache/beam/tree/v2.33.0-RC1
> >>>>>>>>>>>>>> >>>> > >>>> [6] https://github.com/apache/beam/pull/15543
> >>>>>>>>>>>>>> >>>> > >>>> [7]
> https://github.com/apache/beam-site/pull/619
> >>>>>>>>>>>>>> >>>> > >>>> [8]
> https://pypi.org/project/apache-beam/2.33.0rc1/
> >>>>>>>>>>>>>> >>>> > >>>> [9]
> >>>>>>>>>>>>>> >>>> > >>>>
> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
> >>>>>>>>>>>>>> >>>> > >>>> [10]
> https://hub.docker.com/search?q=apache%2Fbeam&type=image
> >>>>>>>>>>>>>> >>>> > >>>>
> >>>>>>>>>>>>>> >>>> > >>>>
> >>>>>>>>>>>>>> >>>> >
> >>>>>>>>>>>>>> >>>
> >>>>>>>>>>>>>> >>>
> >>>>>>>>>>>>>> >>>
> >>>>>>>>>>>>>> >>
>

Re: [VOTE] Release 2.33.0, release candidate 1

[Robert Bradshaw <ro...@google.com>]

As I read the code, it's only pipelines that use a Repeatedly trigger
that wrap an already lossy trigger that are declared to be themselves
lossy. If I'm mistaken, I'll certainly reconsider my vote (and thanks
for bringing this up).

On Mon, Sep 27, 2021 at 2:21 PM Kenneth Knowles <ke...@apache.org> wrote:
>
> My concern is that the error message is incorrect and every user of 2.33.0 may be educated wrong, or be worried about data loss in Beam, or fail to find the blog post or CHANGES, etc.
>
> Kenn
>
> On Mon, Sep 27, 2021 at 2:16 PM Udi Meiri <eh...@google.com> wrote:
>>
>> I don't know how rare it is, but there is a flag documented in CHANGES and blog post that reverts to the old behavior.
>>
>> On Mon, Sep 27, 2021 at 2:12 PM Kenneth Knowles <ke...@apache.org> wrote:
>>>
>>> I guess my vote is -0 since I don't have enough context on this issue. A number of people with more awareness of how severe this is have voted +1 so I will not try to block the release.
>>>
>>> Kenn
>>>
>>> On Mon, Sep 27, 2021 at 2:11 PM Kenneth Knowles <ke...@apache.org> wrote:
>>>>
>>>> I have to disagree with the other PMC members here, or at least dig in to the question: every pipeline that uses a Repeatedly trigger at the top level will be rejected. Is this so rare in Python that it is OK?
>>>>
>>>> Kenn
>>>>
>>>> On Mon, Sep 27, 2021 at 1:56 PM Robert Burke <re...@google.com> wrote:
>>>>>
>>>>> On it. Thanks!
>>>>>
>>>>> On Mon, Sep 27, 2021 at 1:18 PM Udi Meiri <eh...@google.com> wrote:
>>>>>>
>>>>>> Daniel/Robert, feel free to make changes to this PR: https://github.com/apache/beam/pull/15543
>>>>>>
>>>>>> On Mon, Sep 27, 2021 at 12:08 PM Daniel Oliveira <da...@google.com> wrote:
>>>>>>>
>>>>>>> +1
>>>>>>>
>>>>>>> I hadn't realized the pipelines were still finishing successfully. I retried wordcount with that in mind and confirmed it finishes successfully, so this isn't a blocker.
>>>>>>>
>>>>>>> Although maybe we should add this to the "Known Issues" because I can easily see those messages being interpreted as a pipeline failure.
>>>>>>>
>>>>>>> On Sun, Sep 26, 2021 at 7:26 PM Robert Burke <ro...@frantil.com> wrote:
>>>>>>>>
>>>>>>>> That's on the SDK side, and it just means the PCollection metrics are being returned, buy not handled by the SDK. At present the pipeline results only handle PTransform metrics.
>>>>>>>>
>>>>>>>> As such it's not a regression, as adding those is still under development.
>>>>>>>>
>>>>>>>> On Sun, Sep 26, 2021, 7:02 PM Daniel Oliveira <da...@google.com> wrote:
>>>>>>>>>
>>>>>>>>> I tried validating wordcount with the Go SDK on Flink. The pipeline failed with a wall of errors like the following. I tried this on Flink 1.11, 1.12, and 1.13 job servers built from source at the RC1 commit, same errors on all of them.
>>>>>>>>>
>>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n9"}
>>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n10"}
>>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n8"}
>>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n7"}
>>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"  type:"beam:metrics:distribution_int64:v1"  payload:"\x8d%\xfa\xbf\x05\x04\xa8\x0c"  labels:{key:"PCOLLECTION"  value:"n7"}
>>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"  type:"beam:metrics:distribution_int64:v1"  payload:"\xb9\x05\xcf6\x05\x11"  labels:{key:"PCOLLECTION"  value:"n9"}
>>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"  type:"beam:metrics:distribution_int64:v1"  payload:"\xc5\x05\x8a1\x04\x12"  labels:{key:"PCOLLECTION"  value:"n8"}
>>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:sampled_byte_size:v1"  type:"beam:metrics:distribution_int64:v1"  payload:"\x8d%\xb3\xa7\x07\x14\""  labels:{key:"PCOLLECTION"  value:"n10"}
>>>>>>>>>> {...}
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>  @Robert Burke I think you might know what's going on here. Is this solvable with a cherry-pick and a new RC?
>>>>>>>>>
>>>>>>>>> On Fri, Sep 24, 2021 at 4:25 PM Ahmet Altay <al...@google.com> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Fri, Sep 24, 2021 at 4:21 PM Udi Meiri <eh...@google.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Sep 24, 2021 at 4:18 PM Ahmet Altay <al...@google.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Thank you for the update related to allow_unsafe_triggers. My vote is still a +1.
>>>>>>>>>>>>
>>>>>>>>>>>> If we decide to move forward with this RC, could you please include this bug in the known issues list under the changes.md for this release?
>>>>>>>>>>>
>>>>>>>>>>> Yes, that's included in https://github.com/apache/beam/pull/15543/files.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thank you!
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Sep 24, 2021 at 3:56 PM Chamikara Jayalath <ch...@google.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> +1 (binding)
>>>>>>>>>>>>> Validated a few scenarios from the spreadsheet.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Cham
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Sep 24, 2021 at 3:07 PM Robert Bradshaw <ro...@google.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> +1 (binding)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> All the artifacts and signatures look good.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I don't think the unsafe trigger check is severe enough to block the release.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Fri, Sep 24, 2021 at 2:36 PM Udi Meiri <eh...@google.com> wrote:
>>>>>>>>>>>>>> >
>>>>>>>>>>>>>> > Hi everyone, we found a bug during testing. It has to do with Python SDK's allow_unsafe_triggers check.
>>>>>>>>>>>>>> > There is a preliminary fix that will go to master.
>>>>>>>>>>>>>> >
>>>>>>>>>>>>>> > For the 2.33.0 release, I'm leaning towards not making a new RC since there is a workaround: pass the flag --allow_unsafe_triggers.
>>>>>>>>>>>>>> >
>>>>>>>>>>>>>> > Please reevaluate your votes accordingly and recast if you've changed your vote.
>>>>>>>>>>>>>> >
>>>>>>>>>>>>>> > Thanks
>>>>>>>>>>>>>> >
>>>>>>>>>>>>>> > On Fri, Sep 24, 2021 at 12:48 PM Alexey Romanenko <ar...@gmail.com> wrote:
>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>> >> On 24 Sep 2021, at 20:45, Udi Meiri <eh...@google.com> wrote:
>>>>>>>>>>>>>> >> Alexey is this something that we should put in the release notes, or some other change?
>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>> >> Yes, I think it could be helpful to mention that Beam Jackson’s deps was bumped and it may require an update of Jackson’s runtime deps for Spark 2 users as well.
>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>> >> —
>>>>>>>>>>>>>> >> Alexey
>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>> >>>
>>>>>>>>>>>>>> >>>
>>>>>>>>>>>>>> >>> —
>>>>>>>>>>>>>> >>> Alexey
>>>>>>>>>>>>>> >>>
>>>>>>>>>>>>>> >>> On 24 Sep 2021, at 16:17, Alexey Romanenko <ar...@gmail.com> wrote:
>>>>>>>>>>>>>> >>>
>>>>>>>>>>>>>> >>> I checked with beam-samples [1] and noticed an issue to run some pipelines with Spark 2 runner (Spark 3 seems is ok).
>>>>>>>>>>>>>> >>> It looks that it’s caused by new Jackson's version updated recently [2], even if it’s a minor update but it works fine with a previous one.
>>>>>>>>>>>>>> >>> I’ll try to find a workaround and get back with a results of this.
>>>>>>>>>>>>>> >>>
>>>>>>>>>>>>>> >>> [1] https://github.com/Talend/beam-samples/
>>>>>>>>>>>>>> >>> [2] https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c
>>>>>>>>>>>>>> >>>
>>>>>>>>>>>>>> >>> —
>>>>>>>>>>>>>> >>> Alexey
>>>>>>>>>>>>>> >>>
>>>>>>>>>>>>>> >>> On 24 Sep 2021, at 11:17, Jan Lukavský <je...@seznam.cz> wrote:
>>>>>>>>>>>>>> >>>
>>>>>>>>>>>>>> >>> +1 (non-binding)
>>>>>>>>>>>>>> >>>
>>>>>>>>>>>>>> >>> Validated several use-cases using non-portable Flink with Java SDK.
>>>>>>>>>>>>>> >>>
>>>>>>>>>>>>>> >>> On 9/24/21 4:55 AM, Valentyn Tymofieiev wrote:
>>>>>>>>>>>>>> >>>
>>>>>>>>>>>>>> >>> +1. Ran several Python batch and streaming pipelines on Dataflow and checked that Dataflow containers have required dependencies of Beam.
>>>>>>>>>>>>>> >>>
>>>>>>>>>>>>>> >>> On Thu, Sep 23, 2021 at 7:03 PM Robert Burke <lo...@apache.org> wrote:
>>>>>>>>>>>>>> >>>>
>>>>>>>>>>>>>> >>>> +1 (non-binding)
>>>>>>>>>>>>>> >>>> I validated the Go Quickstart (wordcount), and my ray tracer against the Go Direct runner, Dataflow, and Spark (ensuring the rc1 tagged container was used) and they executed successfully.
>>>>>>>>>>>>>> >>>>
>>>>>>>>>>>>>> >>>> I needed to manually synthesize a pseudo-version to ensure I was using the tagged branch version (v2.0.0-20210914211513-b358127f9859) instead of simply using v2.33.0-RC1.
>>>>>>>>>>>>>> >>>>
>>>>>>>>>>>>>> >>>>  It either can't find the package with the right tagged version, or it can't find the version. It's not clear to me what the issue is, but it's not notionally a release blocker. I'll investigate further once we have a full release, as it's probably some unspecified behavior due to how we transitioned to Go Modules (which strongly recommended doing a major version bump for such transitions, which seems a bit excessive for Beam as a whole...).
>>>>>>>>>>>>>> >>>>
>>>>>>>>>>>>>> >>>> On 2021/09/23 03:59:18, Ahmet Altay <al...@google.com> wrote:
>>>>>>>>>>>>>> >>>> > +1 on the RC. I validated python quick start examples on direct runners.
>>>>>>>>>>>>>> >>>> > Thank you Udi.
>>>>>>>>>>>>>> >>>> >
>>>>>>>>>>>>>> >>>> > On Wed, Sep 22, 2021 at 2:20 PM Robert Burke <ro...@frantil.com> wrote:
>>>>>>>>>>>>>> >>>> >
>>>>>>>>>>>>>> >>>> > > Just an FYI that intend to validate the Go SDK for this release but can't
>>>>>>>>>>>>>> >>>> > > get to it until tomorrow (Thursday). I'm catching up from a week of
>>>>>>>>>>>>>> >>>> > > vacation. Apologies for the inconvenience.
>>>>>>>>>>>>>> >>>> > >
>>>>>>>>>>>>>> >>>> > > On Tue, Sep 21, 2021, 10:59 AM Udi Meiri <eh...@google.com> wrote:
>>>>>>>>>>>>>> >>>> > >
>>>>>>>>>>>>>> >>>> > >> I updated the affected and fixed version fields for
>>>>>>>>>>>>>> >>>> > >> https://issues.apache.org/jira/browse/BEAM-12356.
>>>>>>>>>>>>>> >>>> > >>
>>>>>>>>>>>>>> >>>> > >>
>>>>>>>>>>>>>> >>>> > >> On Tue, Sep 21, 2021 at 10:48 AM Reuven Lax <re...@google.com> wrote:
>>>>>>>>>>>>>> >>>> > >>
>>>>>>>>>>>>>> >>>> > >>> Unfortunate - I didn't realize that
>>>>>>>>>>>>>> >>>> > >>> https://github.com/apache/beam/pull/15480 didn't make the cut.
>>>>>>>>>>>>>> >>>> > >>>
>>>>>>>>>>>>>> >>>> > >>> This bug was a regression in Beam 2.32.0, and is blocking multiple users
>>>>>>>>>>>>>> >>>> > >>> from updating.
>>>>>>>>>>>>>> >>>> > >>>
>>>>>>>>>>>>>> >>>> > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi Meiri <ud...@apache.org> wrote:
>>>>>>>>>>>>>> >>>> > >>>
>>>>>>>>>>>>>> >>>> > >>>> Hi everyone,
>>>>>>>>>>>>>> >>>> > >>>> Please review and vote on the release candidate #1 for the version
>>>>>>>>>>>>>> >>>> > >>>> 2.33.0, as follows:
>>>>>>>>>>>>>> >>>> > >>>> [ ] +1, Approve the release
>>>>>>>>>>>>>> >>>> > >>>> [ ] -1, Do not approve the release (please provide specific comments)
>>>>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>>>>> >>>> > >>>> Reviewers are encouraged to test their own use cases with the release
>>>>>>>>>>>>>> >>>> > >>>> candidate, and vote +1 if
>>>>>>>>>>>>>> >>>> > >>>> no issues are found.
>>>>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>>>>> >>>> > >>>> The complete staging area is available for your review, which includes:
>>>>>>>>>>>>>> >>>> > >>>> * JIRA release notes [1],
>>>>>>>>>>>>>> >>>> > >>>> * the official Apache source release to be deployed to dist.apache.org
>>>>>>>>>>>>>> >>>> > >>>> [2], which is signed with the key with fingerprint 587B049C36DAAFE6 [3],
>>>>>>>>>>>>>> >>>> > >>>> * all artifacts to be deployed to the Maven Central Repository [4],
>>>>>>>>>>>>>> >>>> > >>>> * source code tag "v2.33.0-RC1" [5],
>>>>>>>>>>>>>> >>>> > >>>> * website pull request listing the release [6], the blog post [6], and
>>>>>>>>>>>>>> >>>> > >>>> publishing the API reference manual [7].
>>>>>>>>>>>>>> >>>> > >>>> * Java artifacts were built with Maven 3.6.3 and OpenJDK 1.8.0_181.
>>>>>>>>>>>>>> >>>> > >>>> * Python artifacts are deployed along with the source release to the
>>>>>>>>>>>>>> >>>> > >>>> dist.apache.org [2] and pypy[8].
>>>>>>>>>>>>>> >>>> > >>>> * Validation sheet with a tab for 2.33.0 release to help with
>>>>>>>>>>>>>> >>>> > >>>> validation [9].
>>>>>>>>>>>>>> >>>> > >>>> * Docker images published to Docker Hub [10].
>>>>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>>>>> >>>> > >>>> The vote will be open for at least 72 hours. It is adopted by majority
>>>>>>>>>>>>>> >>>> > >>>> approval, with at least 3 PMC affirmative votes.
>>>>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>>>>> >>>> > >>>> For guidelines on how to try the release in your projects, check out
>>>>>>>>>>>>>> >>>> > >>>> our blog post at https://beam.apache.org/blog/validate-beam-release/.
>>>>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>>>>> >>>> > >>>> Thanks,
>>>>>>>>>>>>>> >>>> > >>>> Release Manager
>>>>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>>>>> >>>> > >>>> [1]
>>>>>>>>>>>>>> >>>> > >>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
>>>>>>>>>>>>>> >>>> > >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.33.0/
>>>>>>>>>>>>>> >>>> > >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>>>>>>>>>>>>> >>>> > >>>> [4]
>>>>>>>>>>>>>> >>>> > >>>> https://repository.apache.org/content/repositories/orgapachebeam-1234/
>>>>>>>>>>>>>> >>>> > >>>> [5] https://github.com/apache/beam/tree/v2.33.0-RC1
>>>>>>>>>>>>>> >>>> > >>>> [6] https://github.com/apache/beam/pull/15543
>>>>>>>>>>>>>> >>>> > >>>> [7] https://github.com/apache/beam-site/pull/619
>>>>>>>>>>>>>> >>>> > >>>> [8] https://pypi.org/project/apache-beam/2.33.0rc1/
>>>>>>>>>>>>>> >>>> > >>>> [9]
>>>>>>>>>>>>>> >>>> > >>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
>>>>>>>>>>>>>> >>>> > >>>> [10] https://hub.docker.com/search?q=apache%2Fbeam&type=image
>>>>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>>>>> >>>> >
>>>>>>>>>>>>>> >>>
>>>>>>>>>>>>>> >>>
>>>>>>>>>>>>>> >>>
>>>>>>>>>>>>>> >>

Re: [VOTE] Release 2.33.0, release candidate 1

[Kenneth Knowles <ke...@apache.org>]

My concern is that the error message is incorrect and every user of 2.33.0
may be educated wrong, or be worried about data loss in Beam, or fail to
find the blog post or CHANGES, etc.

Kenn

On Mon, Sep 27, 2021 at 2:16 PM Udi Meiri <eh...@google.com> wrote:

> I don't know how rare it is, but there is a flag documented in CHANGES and
> blog post that reverts to the old behavior.
>
> On Mon, Sep 27, 2021 at 2:12 PM Kenneth Knowles <ke...@apache.org> wrote:
>
>> I guess my vote is -0 since I don't have enough context on this issue. A
>> number of people with more awareness of how severe this is have voted +1 so
>> I will not try to block the release.
>>
>> Kenn
>>
>> On Mon, Sep 27, 2021 at 2:11 PM Kenneth Knowles <ke...@apache.org> wrote:
>>
>>> I have to disagree with the other PMC members here, or at least dig in
>>> to the question: every pipeline that uses a Repeatedly trigger at the top
>>> level will be rejected. Is this so rare in Python that it is OK?
>>>
>>> Kenn
>>>
>>> On Mon, Sep 27, 2021 at 1:56 PM Robert Burke <re...@google.com> wrote:
>>>
>>>> On it. Thanks!
>>>>
>>>> On Mon, Sep 27, 2021 at 1:18 PM Udi Meiri <eh...@google.com> wrote:
>>>>
>>>>> Daniel/Robert, feel free to make changes to this PR:
>>>>> https://github.com/apache/beam/pull/15543
>>>>>
>>>>> On Mon, Sep 27, 2021 at 12:08 PM Daniel Oliveira <
>>>>> danoliveira@google.com> wrote:
>>>>>
>>>>>> +1
>>>>>>
>>>>>> I hadn't realized the pipelines were still finishing successfully. I
>>>>>> retried wordcount with that in mind and confirmed it finishes successfully,
>>>>>> so this isn't a blocker.
>>>>>>
>>>>>> Although maybe we should add this to the "Known Issues" because I can
>>>>>> easily see those messages being interpreted as a pipeline failure.
>>>>>>
>>>>>> On Sun, Sep 26, 2021 at 7:26 PM Robert Burke <ro...@frantil.com>
>>>>>> wrote:
>>>>>>
>>>>>>> That's on the SDK side, and it just means the PCollection metrics
>>>>>>> are being returned, buy not handled by the SDK. At present the pipeline
>>>>>>> results only handle PTransform metrics.
>>>>>>>
>>>>>>> As such it's not a regression, as adding those is still under
>>>>>>> development.
>>>>>>>
>>>>>>> On Sun, Sep 26, 2021, 7:02 PM Daniel Oliveira <
>>>>>>> danoliveira@google.com> wrote:
>>>>>>>
>>>>>>>> I tried validating wordcount with the Go SDK on Flink. The pipeline
>>>>>>>> failed with a wall of errors like the following. I tried this on Flink
>>>>>>>> 1.11, 1.12, and 1.13 job servers built from source at the RC1 commit, same
>>>>>>>> errors on all of them.
>>>>>>>>
>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>>>>>> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>>>>>>>>>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n9"}
>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>>>>>> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>>>>>>>>>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n10"}
>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>>>>>> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>>>>>>>>>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n8"}
>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>>>>>> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>>>>>>>>>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n7"}
>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>>>>>> urn:"beam:metric:sampled_byte_size:v1"
>>>>>>>>>  type:"beam:metrics:distribution_int64:v1"
>>>>>>>>>  payload:"\x8d%\xfa\xbf\x05\x04\xa8\x0c"  labels:{key:"PCOLLECTION"
>>>>>>>>>  value:"n7"}
>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>>>>>> urn:"beam:metric:sampled_byte_size:v1"
>>>>>>>>>  type:"beam:metrics:distribution_int64:v1"  payload:"\xb9\x05\xcf6\x05\x11"
>>>>>>>>>  labels:{key:"PCOLLECTION"  value:"n9"}
>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>>>>>> urn:"beam:metric:sampled_byte_size:v1"
>>>>>>>>>  type:"beam:metrics:distribution_int64:v1"  payload:"\xc5\x05\x8a1\x04\x12"
>>>>>>>>>  labels:{key:"PCOLLECTION"  value:"n8"}
>>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>>>>>> urn:"beam:metric:sampled_byte_size:v1"
>>>>>>>>>  type:"beam:metrics:distribution_int64:v1"
>>>>>>>>>  payload:"\x8d%\xb3\xa7\x07\x14\""  labels:{key:"PCOLLECTION"  value:"n10"}
>>>>>>>>> {...}
>>>>>>>>
>>>>>>>>
>>>>>>>>  @Robert Burke <re...@google.com> I think you might know what's
>>>>>>>> going on here. Is this solvable with a cherry-pick and a new RC?
>>>>>>>>
>>>>>>>> On Fri, Sep 24, 2021 at 4:25 PM Ahmet Altay <al...@google.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Sep 24, 2021 at 4:21 PM Udi Meiri <eh...@google.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Fri, Sep 24, 2021 at 4:18 PM Ahmet Altay <al...@google.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Thank you for the update related to allow_unsafe_triggers. My
>>>>>>>>>>> vote is still a +1.
>>>>>>>>>>>
>>>>>>>>>>> If we decide to move forward with this RC, could you please
>>>>>>>>>>> include this bug in the known issues list under the changes.md for this
>>>>>>>>>>> release?
>>>>>>>>>>>
>>>>>>>>>> Yes, that's included in
>>>>>>>>>> https://github.com/apache/beam/pull/15543/files.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thank you!
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Sep 24, 2021 at 3:56 PM Chamikara Jayalath <
>>>>>>>>>>> chamikara@google.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> +1 (binding)
>>>>>>>>>>>> Validated a few scenarios from the spreadsheet.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Cham
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Sep 24, 2021 at 3:07 PM Robert Bradshaw <
>>>>>>>>>>>> robertwb@google.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> +1 (binding)
>>>>>>>>>>>>>
>>>>>>>>>>>>> All the artifacts and signatures look good.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I don't think the unsafe trigger check is severe enough to
>>>>>>>>>>>>> block the release.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Sep 24, 2021 at 2:36 PM Udi Meiri <eh...@google.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > Hi everyone, we found a bug during testing. It has to do
>>>>>>>>>>>>> with Python SDK's allow_unsafe_triggers check.
>>>>>>>>>>>>> > There is a preliminary fix that will go to master.
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > For the 2.33.0 release, I'm leaning towards not making a new
>>>>>>>>>>>>> RC since there is a workaround: pass the flag --allow_unsafe_triggers.
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > Please reevaluate your votes accordingly and recast if
>>>>>>>>>>>>> you've changed your vote.
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > Thanks
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > On Fri, Sep 24, 2021 at 12:48 PM Alexey Romanenko <
>>>>>>>>>>>>> aromanenko.dev@gmail.com> wrote:
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >> On 24 Sep 2021, at 20:45, Udi Meiri <eh...@google.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> >> Alexey is this something that we should put in the release
>>>>>>>>>>>>> notes, or some other change?
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >> Yes, I think it could be helpful to mention that Beam
>>>>>>>>>>>>> Jackson’s deps was bumped and it may require an update of Jackson’s runtime
>>>>>>>>>>>>> deps for Spark 2 users as well.
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >> —
>>>>>>>>>>>>> >> Alexey
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >>>
>>>>>>>>>>>>> >>>
>>>>>>>>>>>>> >>> —
>>>>>>>>>>>>> >>> Alexey
>>>>>>>>>>>>> >>>
>>>>>>>>>>>>> >>> On 24 Sep 2021, at 16:17, Alexey Romanenko <
>>>>>>>>>>>>> aromanenko.dev@gmail.com> wrote:
>>>>>>>>>>>>> >>>
>>>>>>>>>>>>> >>> I checked with beam-samples [1] and noticed an issue to
>>>>>>>>>>>>> run some pipelines with Spark 2 runner (Spark 3 seems is ok).
>>>>>>>>>>>>> >>> It looks that it’s caused by new Jackson's version updated
>>>>>>>>>>>>> recently [2], even if it’s a minor update but it works fine with a previous
>>>>>>>>>>>>> one.
>>>>>>>>>>>>> >>> I’ll try to find a workaround and get back with a results
>>>>>>>>>>>>> of this.
>>>>>>>>>>>>> >>>
>>>>>>>>>>>>> >>> [1] https://github.com/Talend/beam-samples/
>>>>>>>>>>>>> >>> [2]
>>>>>>>>>>>>> https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c
>>>>>>>>>>>>> >>>
>>>>>>>>>>>>> >>> —
>>>>>>>>>>>>> >>> Alexey
>>>>>>>>>>>>> >>>
>>>>>>>>>>>>> >>> On 24 Sep 2021, at 11:17, Jan Lukavský <je...@seznam.cz>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> >>>
>>>>>>>>>>>>> >>> +1 (non-binding)
>>>>>>>>>>>>> >>>
>>>>>>>>>>>>> >>> Validated several use-cases using non-portable Flink with
>>>>>>>>>>>>> Java SDK.
>>>>>>>>>>>>> >>>
>>>>>>>>>>>>> >>> On 9/24/21 4:55 AM, Valentyn Tymofieiev wrote:
>>>>>>>>>>>>> >>>
>>>>>>>>>>>>> >>> +1. Ran several Python batch and streaming pipelines on
>>>>>>>>>>>>> Dataflow and checked that Dataflow containers have required dependencies of
>>>>>>>>>>>>> Beam.
>>>>>>>>>>>>> >>>
>>>>>>>>>>>>> >>> On Thu, Sep 23, 2021 at 7:03 PM Robert Burke <
>>>>>>>>>>>>> lostluck@apache.org> wrote:
>>>>>>>>>>>>> >>>>
>>>>>>>>>>>>> >>>> +1 (non-binding)
>>>>>>>>>>>>> >>>> I validated the Go Quickstart (wordcount), and my ray
>>>>>>>>>>>>> tracer against the Go Direct runner, Dataflow, and Spark (ensuring the rc1
>>>>>>>>>>>>> tagged container was used) and they executed successfully.
>>>>>>>>>>>>> >>>>
>>>>>>>>>>>>> >>>> I needed to manually synthesize a pseudo-version to
>>>>>>>>>>>>> ensure I was using the tagged branch version
>>>>>>>>>>>>> (v2.0.0-20210914211513-b358127f9859) instead of simply using v2.33.0-RC1.
>>>>>>>>>>>>> >>>>
>>>>>>>>>>>>> >>>>  It either can't find the package with the right tagged
>>>>>>>>>>>>> version, or it can't find the version. It's not clear to me what the issue
>>>>>>>>>>>>> is, but it's not notionally a release blocker. I'll investigate further
>>>>>>>>>>>>> once we have a full release, as it's probably some unspecified behavior due
>>>>>>>>>>>>> to how we transitioned to Go Modules (which strongly recommended doing a
>>>>>>>>>>>>> major version bump for such transitions, which seems a bit excessive for
>>>>>>>>>>>>> Beam as a whole...).
>>>>>>>>>>>>> >>>>
>>>>>>>>>>>>> >>>> On 2021/09/23 03:59:18, Ahmet Altay <al...@google.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> >>>> > +1 on the RC. I validated python quick start examples
>>>>>>>>>>>>> on direct runners.
>>>>>>>>>>>>> >>>> > Thank you Udi.
>>>>>>>>>>>>> >>>> >
>>>>>>>>>>>>> >>>> > On Wed, Sep 22, 2021 at 2:20 PM Robert Burke <
>>>>>>>>>>>>> robert@frantil.com> wrote:
>>>>>>>>>>>>> >>>> >
>>>>>>>>>>>>> >>>> > > Just an FYI that intend to validate the Go SDK for
>>>>>>>>>>>>> this release but can't
>>>>>>>>>>>>> >>>> > > get to it until tomorrow (Thursday). I'm catching up
>>>>>>>>>>>>> from a week of
>>>>>>>>>>>>> >>>> > > vacation. Apologies for the inconvenience.
>>>>>>>>>>>>> >>>> > >
>>>>>>>>>>>>> >>>> > > On Tue, Sep 21, 2021, 10:59 AM Udi Meiri <
>>>>>>>>>>>>> ehudm@google.com> wrote:
>>>>>>>>>>>>> >>>> > >
>>>>>>>>>>>>> >>>> > >> I updated the affected and fixed version fields for
>>>>>>>>>>>>> >>>> > >> https://issues.apache.org/jira/browse/BEAM-12356.
>>>>>>>>>>>>> >>>> > >>
>>>>>>>>>>>>> >>>> > >>
>>>>>>>>>>>>> >>>> > >> On Tue, Sep 21, 2021 at 10:48 AM Reuven Lax <
>>>>>>>>>>>>> relax@google.com> wrote:
>>>>>>>>>>>>> >>>> > >>
>>>>>>>>>>>>> >>>> > >>> Unfortunate - I didn't realize that
>>>>>>>>>>>>> >>>> > >>> https://github.com/apache/beam/pull/15480 didn't
>>>>>>>>>>>>> make the cut.
>>>>>>>>>>>>> >>>> > >>>
>>>>>>>>>>>>> >>>> > >>> This bug was a regression in Beam 2.32.0, and is
>>>>>>>>>>>>> blocking multiple users
>>>>>>>>>>>>> >>>> > >>> from updating.
>>>>>>>>>>>>> >>>> > >>>
>>>>>>>>>>>>> >>>> > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi Meiri <
>>>>>>>>>>>>> udim@apache.org> wrote:
>>>>>>>>>>>>> >>>> > >>>
>>>>>>>>>>>>> >>>> > >>>> Hi everyone,
>>>>>>>>>>>>> >>>> > >>>> Please review and vote on the release candidate #1
>>>>>>>>>>>>> for the version
>>>>>>>>>>>>> >>>> > >>>> 2.33.0, as follows:
>>>>>>>>>>>>> >>>> > >>>> [ ] +1, Approve the release
>>>>>>>>>>>>> >>>> > >>>> [ ] -1, Do not approve the release (please provide
>>>>>>>>>>>>> specific comments)
>>>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>>>> >>>> > >>>> Reviewers are encouraged to test their own use
>>>>>>>>>>>>> cases with the release
>>>>>>>>>>>>> >>>> > >>>> candidate, and vote +1 if
>>>>>>>>>>>>> >>>> > >>>> no issues are found.
>>>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>>>> >>>> > >>>> The complete staging area is available for your
>>>>>>>>>>>>> review, which includes:
>>>>>>>>>>>>> >>>> > >>>> * JIRA release notes [1],
>>>>>>>>>>>>> >>>> > >>>> * the official Apache source release to be
>>>>>>>>>>>>> deployed to dist.apache.org
>>>>>>>>>>>>> >>>> > >>>> [2], which is signed with the key with fingerprint
>>>>>>>>>>>>> 587B049C36DAAFE6 [3],
>>>>>>>>>>>>> >>>> > >>>> * all artifacts to be deployed to the Maven
>>>>>>>>>>>>> Central Repository [4],
>>>>>>>>>>>>> >>>> > >>>> * source code tag "v2.33.0-RC1" [5],
>>>>>>>>>>>>> >>>> > >>>> * website pull request listing the release [6],
>>>>>>>>>>>>> the blog post [6], and
>>>>>>>>>>>>> >>>> > >>>> publishing the API reference manual [7].
>>>>>>>>>>>>> >>>> > >>>> * Java artifacts were built with Maven 3.6.3 and
>>>>>>>>>>>>> OpenJDK 1.8.0_181.
>>>>>>>>>>>>> >>>> > >>>> * Python artifacts are deployed along with the
>>>>>>>>>>>>> source release to the
>>>>>>>>>>>>> >>>> > >>>> dist.apache.org [2] and pypy[8].
>>>>>>>>>>>>> >>>> > >>>> * Validation sheet with a tab for 2.33.0 release
>>>>>>>>>>>>> to help with
>>>>>>>>>>>>> >>>> > >>>> validation [9].
>>>>>>>>>>>>> >>>> > >>>> * Docker images published to Docker Hub [10].
>>>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>>>> >>>> > >>>> The vote will be open for at least 72 hours. It is
>>>>>>>>>>>>> adopted by majority
>>>>>>>>>>>>> >>>> > >>>> approval, with at least 3 PMC affirmative votes.
>>>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>>>> >>>> > >>>> For guidelines on how to try the release in your
>>>>>>>>>>>>> projects, check out
>>>>>>>>>>>>> >>>> > >>>> our blog post at
>>>>>>>>>>>>> https://beam.apache.org/blog/validate-beam-release/.
>>>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>>>> >>>> > >>>> Thanks,
>>>>>>>>>>>>> >>>> > >>>> Release Manager
>>>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>>>> >>>> > >>>> [1]
>>>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
>>>>>>>>>>>>> >>>> > >>>> [2]
>>>>>>>>>>>>> https://dist.apache.org/repos/dist/dev/beam/2.33.0/
>>>>>>>>>>>>> >>>> > >>>> [3]
>>>>>>>>>>>>> https://dist.apache.org/repos/dist/release/beam/KEYS
>>>>>>>>>>>>> >>>> > >>>> [4]
>>>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>>>> https://repository.apache.org/content/repositories/orgapachebeam-1234/
>>>>>>>>>>>>> >>>> > >>>> [5]
>>>>>>>>>>>>> https://github.com/apache/beam/tree/v2.33.0-RC1
>>>>>>>>>>>>> >>>> > >>>> [6] https://github.com/apache/beam/pull/15543
>>>>>>>>>>>>> >>>> > >>>> [7] https://github.com/apache/beam-site/pull/619
>>>>>>>>>>>>> >>>> > >>>> [8]
>>>>>>>>>>>>> https://pypi.org/project/apache-beam/2.33.0rc1/
>>>>>>>>>>>>> >>>> > >>>> [9]
>>>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
>>>>>>>>>>>>> >>>> > >>>> [10]
>>>>>>>>>>>>> https://hub.docker.com/search?q=apache%2Fbeam&type=image
>>>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>>>> >>>> >
>>>>>>>>>>>>> >>>
>>>>>>>>>>>>> >>>
>>>>>>>>>>>>> >>>
>>>>>>>>>>>>> >>
>>>>>>>>>>>>>
>>>>>>>>>>>>

Re: [VOTE] Release 2.33.0, release candidate 1

[Udi Meiri <eh...@google.com>]

I don't know how rare it is, but there is a flag documented in CHANGES and
blog post that reverts to the old behavior.

On Mon, Sep 27, 2021 at 2:12 PM Kenneth Knowles <ke...@apache.org> wrote:

> I guess my vote is -0 since I don't have enough context on this issue. A
> number of people with more awareness of how severe this is have voted +1 so
> I will not try to block the release.
>
> Kenn
>
> On Mon, Sep 27, 2021 at 2:11 PM Kenneth Knowles <ke...@apache.org> wrote:
>
>> I have to disagree with the other PMC members here, or at least dig in to
>> the question: every pipeline that uses a Repeatedly trigger at the top
>> level will be rejected. Is this so rare in Python that it is OK?
>>
>> Kenn
>>
>> On Mon, Sep 27, 2021 at 1:56 PM Robert Burke <re...@google.com> wrote:
>>
>>> On it. Thanks!
>>>
>>> On Mon, Sep 27, 2021 at 1:18 PM Udi Meiri <eh...@google.com> wrote:
>>>
>>>> Daniel/Robert, feel free to make changes to this PR:
>>>> https://github.com/apache/beam/pull/15543
>>>>
>>>> On Mon, Sep 27, 2021 at 12:08 PM Daniel Oliveira <
>>>> danoliveira@google.com> wrote:
>>>>
>>>>> +1
>>>>>
>>>>> I hadn't realized the pipelines were still finishing successfully. I
>>>>> retried wordcount with that in mind and confirmed it finishes successfully,
>>>>> so this isn't a blocker.
>>>>>
>>>>> Although maybe we should add this to the "Known Issues" because I can
>>>>> easily see those messages being interpreted as a pipeline failure.
>>>>>
>>>>> On Sun, Sep 26, 2021 at 7:26 PM Robert Burke <ro...@frantil.com>
>>>>> wrote:
>>>>>
>>>>>> That's on the SDK side, and it just means the PCollection metrics are
>>>>>> being returned, buy not handled by the SDK. At present the pipeline results
>>>>>> only handle PTransform metrics.
>>>>>>
>>>>>> As such it's not a regression, as adding those is still under
>>>>>> development.
>>>>>>
>>>>>> On Sun, Sep 26, 2021, 7:02 PM Daniel Oliveira <da...@google.com>
>>>>>> wrote:
>>>>>>
>>>>>>> I tried validating wordcount with the Go SDK on Flink. The pipeline
>>>>>>> failed with a wall of errors like the following. I tried this on Flink
>>>>>>> 1.11, 1.12, and 1.13 job servers built from source at the RC1 commit, same
>>>>>>> errors on all of them.
>>>>>>>
>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>>>>> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>>>>>>>>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n9"}
>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>>>>> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>>>>>>>>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n10"}
>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>>>>> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>>>>>>>>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n8"}
>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>>>>> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>>>>>>>>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n7"}
>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>>>>> urn:"beam:metric:sampled_byte_size:v1"
>>>>>>>>  type:"beam:metrics:distribution_int64:v1"
>>>>>>>>  payload:"\x8d%\xfa\xbf\x05\x04\xa8\x0c"  labels:{key:"PCOLLECTION"
>>>>>>>>  value:"n7"}
>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>>>>> urn:"beam:metric:sampled_byte_size:v1"
>>>>>>>>  type:"beam:metrics:distribution_int64:v1"  payload:"\xb9\x05\xcf6\x05\x11"
>>>>>>>>  labels:{key:"PCOLLECTION"  value:"n9"}
>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>>>>> urn:"beam:metric:sampled_byte_size:v1"
>>>>>>>>  type:"beam:metrics:distribution_int64:v1"  payload:"\xc5\x05\x8a1\x04\x12"
>>>>>>>>  labels:{key:"PCOLLECTION"  value:"n8"}
>>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>>>>> urn:"beam:metric:sampled_byte_size:v1"
>>>>>>>>  type:"beam:metrics:distribution_int64:v1"
>>>>>>>>  payload:"\x8d%\xb3\xa7\x07\x14\""  labels:{key:"PCOLLECTION"  value:"n10"}
>>>>>>>> {...}
>>>>>>>
>>>>>>>
>>>>>>>  @Robert Burke <re...@google.com> I think you might know what's
>>>>>>> going on here. Is this solvable with a cherry-pick and a new RC?
>>>>>>>
>>>>>>> On Fri, Sep 24, 2021 at 4:25 PM Ahmet Altay <al...@google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Sep 24, 2021 at 4:21 PM Udi Meiri <eh...@google.com> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Sep 24, 2021 at 4:18 PM Ahmet Altay <al...@google.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Thank you for the update related to allow_unsafe_triggers. My
>>>>>>>>>> vote is still a +1.
>>>>>>>>>>
>>>>>>>>>> If we decide to move forward with this RC, could you please
>>>>>>>>>> include this bug in the known issues list under the changes.md for this
>>>>>>>>>> release?
>>>>>>>>>>
>>>>>>>>> Yes, that's included in
>>>>>>>>> https://github.com/apache/beam/pull/15543/files.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Thank you!
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Fri, Sep 24, 2021 at 3:56 PM Chamikara Jayalath <
>>>>>>>>>> chamikara@google.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> +1 (binding)
>>>>>>>>>>> Validated a few scenarios from the spreadsheet.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Cham
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Sep 24, 2021 at 3:07 PM Robert Bradshaw <
>>>>>>>>>>> robertwb@google.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> +1 (binding)
>>>>>>>>>>>>
>>>>>>>>>>>> All the artifacts and signatures look good.
>>>>>>>>>>>>
>>>>>>>>>>>> I don't think the unsafe trigger check is severe enough to
>>>>>>>>>>>> block the release.
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Sep 24, 2021 at 2:36 PM Udi Meiri <eh...@google.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>> >
>>>>>>>>>>>> > Hi everyone, we found a bug during testing. It has to do with
>>>>>>>>>>>> Python SDK's allow_unsafe_triggers check.
>>>>>>>>>>>> > There is a preliminary fix that will go to master.
>>>>>>>>>>>> >
>>>>>>>>>>>> > For the 2.33.0 release, I'm leaning towards not making a new
>>>>>>>>>>>> RC since there is a workaround: pass the flag --allow_unsafe_triggers.
>>>>>>>>>>>> >
>>>>>>>>>>>> > Please reevaluate your votes accordingly and recast if you've
>>>>>>>>>>>> changed your vote.
>>>>>>>>>>>> >
>>>>>>>>>>>> > Thanks
>>>>>>>>>>>> >
>>>>>>>>>>>> > On Fri, Sep 24, 2021 at 12:48 PM Alexey Romanenko <
>>>>>>>>>>>> aromanenko.dev@gmail.com> wrote:
>>>>>>>>>>>> >>
>>>>>>>>>>>> >>
>>>>>>>>>>>> >> On 24 Sep 2021, at 20:45, Udi Meiri <eh...@google.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>> >> Alexey is this something that we should put in the release
>>>>>>>>>>>> notes, or some other change?
>>>>>>>>>>>> >>
>>>>>>>>>>>> >>
>>>>>>>>>>>> >> Yes, I think it could be helpful to mention that Beam
>>>>>>>>>>>> Jackson’s deps was bumped and it may require an update of Jackson’s runtime
>>>>>>>>>>>> deps for Spark 2 users as well.
>>>>>>>>>>>> >>
>>>>>>>>>>>> >> —
>>>>>>>>>>>> >> Alexey
>>>>>>>>>>>> >>
>>>>>>>>>>>> >>
>>>>>>>>>>>> >>
>>>>>>>>>>>> >>>
>>>>>>>>>>>> >>>
>>>>>>>>>>>> >>> —
>>>>>>>>>>>> >>> Alexey
>>>>>>>>>>>> >>>
>>>>>>>>>>>> >>> On 24 Sep 2021, at 16:17, Alexey Romanenko <
>>>>>>>>>>>> aromanenko.dev@gmail.com> wrote:
>>>>>>>>>>>> >>>
>>>>>>>>>>>> >>> I checked with beam-samples [1] and noticed an issue to run
>>>>>>>>>>>> some pipelines with Spark 2 runner (Spark 3 seems is ok).
>>>>>>>>>>>> >>> It looks that it’s caused by new Jackson's version updated
>>>>>>>>>>>> recently [2], even if it’s a minor update but it works fine with a previous
>>>>>>>>>>>> one.
>>>>>>>>>>>> >>> I’ll try to find a workaround and get back with a results
>>>>>>>>>>>> of this.
>>>>>>>>>>>> >>>
>>>>>>>>>>>> >>> [1] https://github.com/Talend/beam-samples/
>>>>>>>>>>>> >>> [2]
>>>>>>>>>>>> https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c
>>>>>>>>>>>> >>>
>>>>>>>>>>>> >>> —
>>>>>>>>>>>> >>> Alexey
>>>>>>>>>>>> >>>
>>>>>>>>>>>> >>> On 24 Sep 2021, at 11:17, Jan Lukavský <je...@seznam.cz>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>> >>>
>>>>>>>>>>>> >>> +1 (non-binding)
>>>>>>>>>>>> >>>
>>>>>>>>>>>> >>> Validated several use-cases using non-portable Flink with
>>>>>>>>>>>> Java SDK.
>>>>>>>>>>>> >>>
>>>>>>>>>>>> >>> On 9/24/21 4:55 AM, Valentyn Tymofieiev wrote:
>>>>>>>>>>>> >>>
>>>>>>>>>>>> >>> +1. Ran several Python batch and streaming pipelines on
>>>>>>>>>>>> Dataflow and checked that Dataflow containers have required dependencies of
>>>>>>>>>>>> Beam.
>>>>>>>>>>>> >>>
>>>>>>>>>>>> >>> On Thu, Sep 23, 2021 at 7:03 PM Robert Burke <
>>>>>>>>>>>> lostluck@apache.org> wrote:
>>>>>>>>>>>> >>>>
>>>>>>>>>>>> >>>> +1 (non-binding)
>>>>>>>>>>>> >>>> I validated the Go Quickstart (wordcount), and my ray
>>>>>>>>>>>> tracer against the Go Direct runner, Dataflow, and Spark (ensuring the rc1
>>>>>>>>>>>> tagged container was used) and they executed successfully.
>>>>>>>>>>>> >>>>
>>>>>>>>>>>> >>>> I needed to manually synthesize a pseudo-version to ensure
>>>>>>>>>>>> I was using the tagged branch version (v2.0.0-20210914211513-b358127f9859)
>>>>>>>>>>>> instead of simply using v2.33.0-RC1.
>>>>>>>>>>>> >>>>
>>>>>>>>>>>> >>>>  It either can't find the package with the right tagged
>>>>>>>>>>>> version, or it can't find the version. It's not clear to me what the issue
>>>>>>>>>>>> is, but it's not notionally a release blocker. I'll investigate further
>>>>>>>>>>>> once we have a full release, as it's probably some unspecified behavior due
>>>>>>>>>>>> to how we transitioned to Go Modules (which strongly recommended doing a
>>>>>>>>>>>> major version bump for such transitions, which seems a bit excessive for
>>>>>>>>>>>> Beam as a whole...).
>>>>>>>>>>>> >>>>
>>>>>>>>>>>> >>>> On 2021/09/23 03:59:18, Ahmet Altay <al...@google.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>> >>>> > +1 on the RC. I validated python quick start examples on
>>>>>>>>>>>> direct runners.
>>>>>>>>>>>> >>>> > Thank you Udi.
>>>>>>>>>>>> >>>> >
>>>>>>>>>>>> >>>> > On Wed, Sep 22, 2021 at 2:20 PM Robert Burke <
>>>>>>>>>>>> robert@frantil.com> wrote:
>>>>>>>>>>>> >>>> >
>>>>>>>>>>>> >>>> > > Just an FYI that intend to validate the Go SDK for
>>>>>>>>>>>> this release but can't
>>>>>>>>>>>> >>>> > > get to it until tomorrow (Thursday). I'm catching up
>>>>>>>>>>>> from a week of
>>>>>>>>>>>> >>>> > > vacation. Apologies for the inconvenience.
>>>>>>>>>>>> >>>> > >
>>>>>>>>>>>> >>>> > > On Tue, Sep 21, 2021, 10:59 AM Udi Meiri <
>>>>>>>>>>>> ehudm@google.com> wrote:
>>>>>>>>>>>> >>>> > >
>>>>>>>>>>>> >>>> > >> I updated the affected and fixed version fields for
>>>>>>>>>>>> >>>> > >> https://issues.apache.org/jira/browse/BEAM-12356.
>>>>>>>>>>>> >>>> > >>
>>>>>>>>>>>> >>>> > >>
>>>>>>>>>>>> >>>> > >> On Tue, Sep 21, 2021 at 10:48 AM Reuven Lax <
>>>>>>>>>>>> relax@google.com> wrote:
>>>>>>>>>>>> >>>> > >>
>>>>>>>>>>>> >>>> > >>> Unfortunate - I didn't realize that
>>>>>>>>>>>> >>>> > >>> https://github.com/apache/beam/pull/15480 didn't
>>>>>>>>>>>> make the cut.
>>>>>>>>>>>> >>>> > >>>
>>>>>>>>>>>> >>>> > >>> This bug was a regression in Beam 2.32.0, and is
>>>>>>>>>>>> blocking multiple users
>>>>>>>>>>>> >>>> > >>> from updating.
>>>>>>>>>>>> >>>> > >>>
>>>>>>>>>>>> >>>> > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi Meiri <
>>>>>>>>>>>> udim@apache.org> wrote:
>>>>>>>>>>>> >>>> > >>>
>>>>>>>>>>>> >>>> > >>>> Hi everyone,
>>>>>>>>>>>> >>>> > >>>> Please review and vote on the release candidate #1
>>>>>>>>>>>> for the version
>>>>>>>>>>>> >>>> > >>>> 2.33.0, as follows:
>>>>>>>>>>>> >>>> > >>>> [ ] +1, Approve the release
>>>>>>>>>>>> >>>> > >>>> [ ] -1, Do not approve the release (please provide
>>>>>>>>>>>> specific comments)
>>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>>> >>>> > >>>> Reviewers are encouraged to test their own use
>>>>>>>>>>>> cases with the release
>>>>>>>>>>>> >>>> > >>>> candidate, and vote +1 if
>>>>>>>>>>>> >>>> > >>>> no issues are found.
>>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>>> >>>> > >>>> The complete staging area is available for your
>>>>>>>>>>>> review, which includes:
>>>>>>>>>>>> >>>> > >>>> * JIRA release notes [1],
>>>>>>>>>>>> >>>> > >>>> * the official Apache source release to be deployed
>>>>>>>>>>>> to dist.apache.org
>>>>>>>>>>>> >>>> > >>>> [2], which is signed with the key with fingerprint
>>>>>>>>>>>> 587B049C36DAAFE6 [3],
>>>>>>>>>>>> >>>> > >>>> * all artifacts to be deployed to the Maven Central
>>>>>>>>>>>> Repository [4],
>>>>>>>>>>>> >>>> > >>>> * source code tag "v2.33.0-RC1" [5],
>>>>>>>>>>>> >>>> > >>>> * website pull request listing the release [6], the
>>>>>>>>>>>> blog post [6], and
>>>>>>>>>>>> >>>> > >>>> publishing the API reference manual [7].
>>>>>>>>>>>> >>>> > >>>> * Java artifacts were built with Maven 3.6.3 and
>>>>>>>>>>>> OpenJDK 1.8.0_181.
>>>>>>>>>>>> >>>> > >>>> * Python artifacts are deployed along with the
>>>>>>>>>>>> source release to the
>>>>>>>>>>>> >>>> > >>>> dist.apache.org [2] and pypy[8].
>>>>>>>>>>>> >>>> > >>>> * Validation sheet with a tab for 2.33.0 release to
>>>>>>>>>>>> help with
>>>>>>>>>>>> >>>> > >>>> validation [9].
>>>>>>>>>>>> >>>> > >>>> * Docker images published to Docker Hub [10].
>>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>>> >>>> > >>>> The vote will be open for at least 72 hours. It is
>>>>>>>>>>>> adopted by majority
>>>>>>>>>>>> >>>> > >>>> approval, with at least 3 PMC affirmative votes.
>>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>>> >>>> > >>>> For guidelines on how to try the release in your
>>>>>>>>>>>> projects, check out
>>>>>>>>>>>> >>>> > >>>> our blog post at
>>>>>>>>>>>> https://beam.apache.org/blog/validate-beam-release/.
>>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>>> >>>> > >>>> Thanks,
>>>>>>>>>>>> >>>> > >>>> Release Manager
>>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>>> >>>> > >>>> [1]
>>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
>>>>>>>>>>>> >>>> > >>>> [2]
>>>>>>>>>>>> https://dist.apache.org/repos/dist/dev/beam/2.33.0/
>>>>>>>>>>>> >>>> > >>>> [3]
>>>>>>>>>>>> https://dist.apache.org/repos/dist/release/beam/KEYS
>>>>>>>>>>>> >>>> > >>>> [4]
>>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>>> https://repository.apache.org/content/repositories/orgapachebeam-1234/
>>>>>>>>>>>> >>>> > >>>> [5] https://github.com/apache/beam/tree/v2.33.0-RC1
>>>>>>>>>>>> >>>> > >>>> [6] https://github.com/apache/beam/pull/15543
>>>>>>>>>>>> >>>> > >>>> [7] https://github.com/apache/beam-site/pull/619
>>>>>>>>>>>> >>>> > >>>> [8] https://pypi.org/project/apache-beam/2.33.0rc1/
>>>>>>>>>>>> >>>> > >>>> [9]
>>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
>>>>>>>>>>>> >>>> > >>>> [10]
>>>>>>>>>>>> https://hub.docker.com/search?q=apache%2Fbeam&type=image
>>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>>> >>>> >
>>>>>>>>>>>> >>>
>>>>>>>>>>>> >>>
>>>>>>>>>>>> >>>
>>>>>>>>>>>> >>
>>>>>>>>>>>>
>>>>>>>>>>>

Re: [VOTE] Release 2.33.0, release candidate 1

[Kenneth Knowles <ke...@apache.org>]

I guess my vote is -0 since I don't have enough context on this issue. A
number of people with more awareness of how severe this is have voted +1 so
I will not try to block the release.

Kenn

On Mon, Sep 27, 2021 at 2:11 PM Kenneth Knowles <ke...@apache.org> wrote:

> I have to disagree with the other PMC members here, or at least dig in to
> the question: every pipeline that uses a Repeatedly trigger at the top
> level will be rejected. Is this so rare in Python that it is OK?
>
> Kenn
>
> On Mon, Sep 27, 2021 at 1:56 PM Robert Burke <re...@google.com> wrote:
>
>> On it. Thanks!
>>
>> On Mon, Sep 27, 2021 at 1:18 PM Udi Meiri <eh...@google.com> wrote:
>>
>>> Daniel/Robert, feel free to make changes to this PR:
>>> https://github.com/apache/beam/pull/15543
>>>
>>> On Mon, Sep 27, 2021 at 12:08 PM Daniel Oliveira <da...@google.com>
>>> wrote:
>>>
>>>> +1
>>>>
>>>> I hadn't realized the pipelines were still finishing successfully. I
>>>> retried wordcount with that in mind and confirmed it finishes successfully,
>>>> so this isn't a blocker.
>>>>
>>>> Although maybe we should add this to the "Known Issues" because I can
>>>> easily see those messages being interpreted as a pipeline failure.
>>>>
>>>> On Sun, Sep 26, 2021 at 7:26 PM Robert Burke <ro...@frantil.com>
>>>> wrote:
>>>>
>>>>> That's on the SDK side, and it just means the PCollection metrics are
>>>>> being returned, buy not handled by the SDK. At present the pipeline results
>>>>> only handle PTransform metrics.
>>>>>
>>>>> As such it's not a regression, as adding those is still under
>>>>> development.
>>>>>
>>>>> On Sun, Sep 26, 2021, 7:02 PM Daniel Oliveira <da...@google.com>
>>>>> wrote:
>>>>>
>>>>>> I tried validating wordcount with the Go SDK on Flink. The pipeline
>>>>>> failed with a wall of errors like the following. I tried this on Flink
>>>>>> 1.11, 1.12, and 1.13 job servers built from source at the RC1 commit, same
>>>>>> errors on all of them.
>>>>>>
>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>>>> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>>>>>>>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n9"}
>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>>>> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>>>>>>>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n10"}
>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>>>> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>>>>>>>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n8"}
>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>>>> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>>>>>>>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n7"}
>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>>>> urn:"beam:metric:sampled_byte_size:v1"
>>>>>>>  type:"beam:metrics:distribution_int64:v1"
>>>>>>>  payload:"\x8d%\xfa\xbf\x05\x04\xa8\x0c"  labels:{key:"PCOLLECTION"
>>>>>>>  value:"n7"}
>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>>>> urn:"beam:metric:sampled_byte_size:v1"
>>>>>>>  type:"beam:metrics:distribution_int64:v1"  payload:"\xb9\x05\xcf6\x05\x11"
>>>>>>>  labels:{key:"PCOLLECTION"  value:"n9"}
>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>>>> urn:"beam:metric:sampled_byte_size:v1"
>>>>>>>  type:"beam:metrics:distribution_int64:v1"  payload:"\xc5\x05\x8a1\x04\x12"
>>>>>>>  labels:{key:"PCOLLECTION"  value:"n8"}
>>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>>>> urn:"beam:metric:sampled_byte_size:v1"
>>>>>>>  type:"beam:metrics:distribution_int64:v1"
>>>>>>>  payload:"\x8d%\xb3\xa7\x07\x14\""  labels:{key:"PCOLLECTION"  value:"n10"}
>>>>>>> {...}
>>>>>>
>>>>>>
>>>>>>  @Robert Burke <re...@google.com> I think you might know what's going
>>>>>> on here. Is this solvable with a cherry-pick and a new RC?
>>>>>>
>>>>>> On Fri, Sep 24, 2021 at 4:25 PM Ahmet Altay <al...@google.com> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Sep 24, 2021 at 4:21 PM Udi Meiri <eh...@google.com> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Sep 24, 2021 at 4:18 PM Ahmet Altay <al...@google.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Thank you for the update related to allow_unsafe_triggers. My vote
>>>>>>>>> is still a +1.
>>>>>>>>>
>>>>>>>>> If we decide to move forward with this RC, could you please
>>>>>>>>> include this bug in the known issues list under the changes.md for this
>>>>>>>>> release?
>>>>>>>>>
>>>>>>>> Yes, that's included in
>>>>>>>> https://github.com/apache/beam/pull/15543/files.
>>>>>>>>
>>>>>>>
>>>>>>> Thank you!
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Sep 24, 2021 at 3:56 PM Chamikara Jayalath <
>>>>>>>>> chamikara@google.com> wrote:
>>>>>>>>>
>>>>>>>>>> +1 (binding)
>>>>>>>>>> Validated a few scenarios from the spreadsheet.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Cham
>>>>>>>>>>
>>>>>>>>>> On Fri, Sep 24, 2021 at 3:07 PM Robert Bradshaw <
>>>>>>>>>> robertwb@google.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> +1 (binding)
>>>>>>>>>>>
>>>>>>>>>>> All the artifacts and signatures look good.
>>>>>>>>>>>
>>>>>>>>>>> I don't think the unsafe trigger check is severe enough to block
>>>>>>>>>>> the release.
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Sep 24, 2021 at 2:36 PM Udi Meiri <eh...@google.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>> >
>>>>>>>>>>> > Hi everyone, we found a bug during testing. It has to do with
>>>>>>>>>>> Python SDK's allow_unsafe_triggers check.
>>>>>>>>>>> > There is a preliminary fix that will go to master.
>>>>>>>>>>> >
>>>>>>>>>>> > For the 2.33.0 release, I'm leaning towards not making a new
>>>>>>>>>>> RC since there is a workaround: pass the flag --allow_unsafe_triggers.
>>>>>>>>>>> >
>>>>>>>>>>> > Please reevaluate your votes accordingly and recast if you've
>>>>>>>>>>> changed your vote.
>>>>>>>>>>> >
>>>>>>>>>>> > Thanks
>>>>>>>>>>> >
>>>>>>>>>>> > On Fri, Sep 24, 2021 at 12:48 PM Alexey Romanenko <
>>>>>>>>>>> aromanenko.dev@gmail.com> wrote:
>>>>>>>>>>> >>
>>>>>>>>>>> >>
>>>>>>>>>>> >> On 24 Sep 2021, at 20:45, Udi Meiri <eh...@google.com> wrote:
>>>>>>>>>>> >> Alexey is this something that we should put in the release
>>>>>>>>>>> notes, or some other change?
>>>>>>>>>>> >>
>>>>>>>>>>> >>
>>>>>>>>>>> >> Yes, I think it could be helpful to mention that Beam
>>>>>>>>>>> Jackson’s deps was bumped and it may require an update of Jackson’s runtime
>>>>>>>>>>> deps for Spark 2 users as well.
>>>>>>>>>>> >>
>>>>>>>>>>> >> —
>>>>>>>>>>> >> Alexey
>>>>>>>>>>> >>
>>>>>>>>>>> >>
>>>>>>>>>>> >>
>>>>>>>>>>> >>>
>>>>>>>>>>> >>>
>>>>>>>>>>> >>> —
>>>>>>>>>>> >>> Alexey
>>>>>>>>>>> >>>
>>>>>>>>>>> >>> On 24 Sep 2021, at 16:17, Alexey Romanenko <
>>>>>>>>>>> aromanenko.dev@gmail.com> wrote:
>>>>>>>>>>> >>>
>>>>>>>>>>> >>> I checked with beam-samples [1] and noticed an issue to run
>>>>>>>>>>> some pipelines with Spark 2 runner (Spark 3 seems is ok).
>>>>>>>>>>> >>> It looks that it’s caused by new Jackson's version updated
>>>>>>>>>>> recently [2], even if it’s a minor update but it works fine with a previous
>>>>>>>>>>> one.
>>>>>>>>>>> >>> I’ll try to find a workaround and get back with a results of
>>>>>>>>>>> this.
>>>>>>>>>>> >>>
>>>>>>>>>>> >>> [1] https://github.com/Talend/beam-samples/
>>>>>>>>>>> >>> [2]
>>>>>>>>>>> https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c
>>>>>>>>>>> >>>
>>>>>>>>>>> >>> —
>>>>>>>>>>> >>> Alexey
>>>>>>>>>>> >>>
>>>>>>>>>>> >>> On 24 Sep 2021, at 11:17, Jan Lukavský <je...@seznam.cz>
>>>>>>>>>>> wrote:
>>>>>>>>>>> >>>
>>>>>>>>>>> >>> +1 (non-binding)
>>>>>>>>>>> >>>
>>>>>>>>>>> >>> Validated several use-cases using non-portable Flink with
>>>>>>>>>>> Java SDK.
>>>>>>>>>>> >>>
>>>>>>>>>>> >>> On 9/24/21 4:55 AM, Valentyn Tymofieiev wrote:
>>>>>>>>>>> >>>
>>>>>>>>>>> >>> +1. Ran several Python batch and streaming pipelines on
>>>>>>>>>>> Dataflow and checked that Dataflow containers have required dependencies of
>>>>>>>>>>> Beam.
>>>>>>>>>>> >>>
>>>>>>>>>>> >>> On Thu, Sep 23, 2021 at 7:03 PM Robert Burke <
>>>>>>>>>>> lostluck@apache.org> wrote:
>>>>>>>>>>> >>>>
>>>>>>>>>>> >>>> +1 (non-binding)
>>>>>>>>>>> >>>> I validated the Go Quickstart (wordcount), and my ray
>>>>>>>>>>> tracer against the Go Direct runner, Dataflow, and Spark (ensuring the rc1
>>>>>>>>>>> tagged container was used) and they executed successfully.
>>>>>>>>>>> >>>>
>>>>>>>>>>> >>>> I needed to manually synthesize a pseudo-version to ensure
>>>>>>>>>>> I was using the tagged branch version (v2.0.0-20210914211513-b358127f9859)
>>>>>>>>>>> instead of simply using v2.33.0-RC1.
>>>>>>>>>>> >>>>
>>>>>>>>>>> >>>>  It either can't find the package with the right tagged
>>>>>>>>>>> version, or it can't find the version. It's not clear to me what the issue
>>>>>>>>>>> is, but it's not notionally a release blocker. I'll investigate further
>>>>>>>>>>> once we have a full release, as it's probably some unspecified behavior due
>>>>>>>>>>> to how we transitioned to Go Modules (which strongly recommended doing a
>>>>>>>>>>> major version bump for such transitions, which seems a bit excessive for
>>>>>>>>>>> Beam as a whole...).
>>>>>>>>>>> >>>>
>>>>>>>>>>> >>>> On 2021/09/23 03:59:18, Ahmet Altay <al...@google.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>> >>>> > +1 on the RC. I validated python quick start examples on
>>>>>>>>>>> direct runners.
>>>>>>>>>>> >>>> > Thank you Udi.
>>>>>>>>>>> >>>> >
>>>>>>>>>>> >>>> > On Wed, Sep 22, 2021 at 2:20 PM Robert Burke <
>>>>>>>>>>> robert@frantil.com> wrote:
>>>>>>>>>>> >>>> >
>>>>>>>>>>> >>>> > > Just an FYI that intend to validate the Go SDK for this
>>>>>>>>>>> release but can't
>>>>>>>>>>> >>>> > > get to it until tomorrow (Thursday). I'm catching up
>>>>>>>>>>> from a week of
>>>>>>>>>>> >>>> > > vacation. Apologies for the inconvenience.
>>>>>>>>>>> >>>> > >
>>>>>>>>>>> >>>> > > On Tue, Sep 21, 2021, 10:59 AM Udi Meiri <
>>>>>>>>>>> ehudm@google.com> wrote:
>>>>>>>>>>> >>>> > >
>>>>>>>>>>> >>>> > >> I updated the affected and fixed version fields for
>>>>>>>>>>> >>>> > >> https://issues.apache.org/jira/browse/BEAM-12356.
>>>>>>>>>>> >>>> > >>
>>>>>>>>>>> >>>> > >>
>>>>>>>>>>> >>>> > >> On Tue, Sep 21, 2021 at 10:48 AM Reuven Lax <
>>>>>>>>>>> relax@google.com> wrote:
>>>>>>>>>>> >>>> > >>
>>>>>>>>>>> >>>> > >>> Unfortunate - I didn't realize that
>>>>>>>>>>> >>>> > >>> https://github.com/apache/beam/pull/15480 didn't
>>>>>>>>>>> make the cut.
>>>>>>>>>>> >>>> > >>>
>>>>>>>>>>> >>>> > >>> This bug was a regression in Beam 2.32.0, and is
>>>>>>>>>>> blocking multiple users
>>>>>>>>>>> >>>> > >>> from updating.
>>>>>>>>>>> >>>> > >>>
>>>>>>>>>>> >>>> > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi Meiri <
>>>>>>>>>>> udim@apache.org> wrote:
>>>>>>>>>>> >>>> > >>>
>>>>>>>>>>> >>>> > >>>> Hi everyone,
>>>>>>>>>>> >>>> > >>>> Please review and vote on the release candidate #1
>>>>>>>>>>> for the version
>>>>>>>>>>> >>>> > >>>> 2.33.0, as follows:
>>>>>>>>>>> >>>> > >>>> [ ] +1, Approve the release
>>>>>>>>>>> >>>> > >>>> [ ] -1, Do not approve the release (please provide
>>>>>>>>>>> specific comments)
>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>> >>>> > >>>> Reviewers are encouraged to test their own use cases
>>>>>>>>>>> with the release
>>>>>>>>>>> >>>> > >>>> candidate, and vote +1 if
>>>>>>>>>>> >>>> > >>>> no issues are found.
>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>> >>>> > >>>> The complete staging area is available for your
>>>>>>>>>>> review, which includes:
>>>>>>>>>>> >>>> > >>>> * JIRA release notes [1],
>>>>>>>>>>> >>>> > >>>> * the official Apache source release to be deployed
>>>>>>>>>>> to dist.apache.org
>>>>>>>>>>> >>>> > >>>> [2], which is signed with the key with fingerprint
>>>>>>>>>>> 587B049C36DAAFE6 [3],
>>>>>>>>>>> >>>> > >>>> * all artifacts to be deployed to the Maven Central
>>>>>>>>>>> Repository [4],
>>>>>>>>>>> >>>> > >>>> * source code tag "v2.33.0-RC1" [5],
>>>>>>>>>>> >>>> > >>>> * website pull request listing the release [6], the
>>>>>>>>>>> blog post [6], and
>>>>>>>>>>> >>>> > >>>> publishing the API reference manual [7].
>>>>>>>>>>> >>>> > >>>> * Java artifacts were built with Maven 3.6.3 and
>>>>>>>>>>> OpenJDK 1.8.0_181.
>>>>>>>>>>> >>>> > >>>> * Python artifacts are deployed along with the
>>>>>>>>>>> source release to the
>>>>>>>>>>> >>>> > >>>> dist.apache.org [2] and pypy[8].
>>>>>>>>>>> >>>> > >>>> * Validation sheet with a tab for 2.33.0 release to
>>>>>>>>>>> help with
>>>>>>>>>>> >>>> > >>>> validation [9].
>>>>>>>>>>> >>>> > >>>> * Docker images published to Docker Hub [10].
>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>> >>>> > >>>> The vote will be open for at least 72 hours. It is
>>>>>>>>>>> adopted by majority
>>>>>>>>>>> >>>> > >>>> approval, with at least 3 PMC affirmative votes.
>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>> >>>> > >>>> For guidelines on how to try the release in your
>>>>>>>>>>> projects, check out
>>>>>>>>>>> >>>> > >>>> our blog post at
>>>>>>>>>>> https://beam.apache.org/blog/validate-beam-release/.
>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>> >>>> > >>>> Thanks,
>>>>>>>>>>> >>>> > >>>> Release Manager
>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>> >>>> > >>>> [1]
>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
>>>>>>>>>>> >>>> > >>>> [2]
>>>>>>>>>>> https://dist.apache.org/repos/dist/dev/beam/2.33.0/
>>>>>>>>>>> >>>> > >>>> [3]
>>>>>>>>>>> https://dist.apache.org/repos/dist/release/beam/KEYS
>>>>>>>>>>> >>>> > >>>> [4]
>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>> https://repository.apache.org/content/repositories/orgapachebeam-1234/
>>>>>>>>>>> >>>> > >>>> [5] https://github.com/apache/beam/tree/v2.33.0-RC1
>>>>>>>>>>> >>>> > >>>> [6] https://github.com/apache/beam/pull/15543
>>>>>>>>>>> >>>> > >>>> [7] https://github.com/apache/beam-site/pull/619
>>>>>>>>>>> >>>> > >>>> [8] https://pypi.org/project/apache-beam/2.33.0rc1/
>>>>>>>>>>> >>>> > >>>> [9]
>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
>>>>>>>>>>> >>>> > >>>> [10]
>>>>>>>>>>> https://hub.docker.com/search?q=apache%2Fbeam&type=image
>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>> >>>> > >>>>
>>>>>>>>>>> >>>> >
>>>>>>>>>>> >>>
>>>>>>>>>>> >>>
>>>>>>>>>>> >>>
>>>>>>>>>>> >>
>>>>>>>>>>>
>>>>>>>>>>

Re: [VOTE] Release 2.33.0, release candidate 1

[Kenneth Knowles <ke...@apache.org>]

I have to disagree with the other PMC members here, or at least dig in to
the question: every pipeline that uses a Repeatedly trigger at the top
level will be rejected. Is this so rare in Python that it is OK?

Kenn

On Mon, Sep 27, 2021 at 1:56 PM Robert Burke <re...@google.com> wrote:

> On it. Thanks!
>
> On Mon, Sep 27, 2021 at 1:18 PM Udi Meiri <eh...@google.com> wrote:
>
>> Daniel/Robert, feel free to make changes to this PR:
>> https://github.com/apache/beam/pull/15543
>>
>> On Mon, Sep 27, 2021 at 12:08 PM Daniel Oliveira <da...@google.com>
>> wrote:
>>
>>> +1
>>>
>>> I hadn't realized the pipelines were still finishing successfully. I
>>> retried wordcount with that in mind and confirmed it finishes successfully,
>>> so this isn't a blocker.
>>>
>>> Although maybe we should add this to the "Known Issues" because I can
>>> easily see those messages being interpreted as a pipeline failure.
>>>
>>> On Sun, Sep 26, 2021 at 7:26 PM Robert Burke <ro...@frantil.com> wrote:
>>>
>>>> That's on the SDK side, and it just means the PCollection metrics are
>>>> being returned, buy not handled by the SDK. At present the pipeline results
>>>> only handle PTransform metrics.
>>>>
>>>> As such it's not a regression, as adding those is still under
>>>> development.
>>>>
>>>> On Sun, Sep 26, 2021, 7:02 PM Daniel Oliveira <da...@google.com>
>>>> wrote:
>>>>
>>>>> I tried validating wordcount with the Go SDK on Flink. The pipeline
>>>>> failed with a wall of errors like the following. I tried this on Flink
>>>>> 1.11, 1.12, and 1.13 job servers built from source at the RC1 commit, same
>>>>> errors on all of them.
>>>>>
>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>>> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>>>>>>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n9"}
>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>>> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>>>>>>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n10"}
>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>>> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>>>>>>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n8"}
>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>>> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>>>>>>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n7"}
>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>>> urn:"beam:metric:sampled_byte_size:v1"
>>>>>>  type:"beam:metrics:distribution_int64:v1"
>>>>>>  payload:"\x8d%\xfa\xbf\x05\x04\xa8\x0c"  labels:{key:"PCOLLECTION"
>>>>>>  value:"n7"}
>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>>> urn:"beam:metric:sampled_byte_size:v1"
>>>>>>  type:"beam:metrics:distribution_int64:v1"  payload:"\xb9\x05\xcf6\x05\x11"
>>>>>>  labels:{key:"PCOLLECTION"  value:"n9"}
>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>>> urn:"beam:metric:sampled_byte_size:v1"
>>>>>>  type:"beam:metrics:distribution_int64:v1"  payload:"\xc5\x05\x8a1\x04\x12"
>>>>>>  labels:{key:"PCOLLECTION"  value:"n8"}
>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>>> urn:"beam:metric:sampled_byte_size:v1"
>>>>>>  type:"beam:metrics:distribution_int64:v1"
>>>>>>  payload:"\x8d%\xb3\xa7\x07\x14\""  labels:{key:"PCOLLECTION"  value:"n10"}
>>>>>> {...}
>>>>>
>>>>>
>>>>>  @Robert Burke <re...@google.com> I think you might know what's going
>>>>> on here. Is this solvable with a cherry-pick and a new RC?
>>>>>
>>>>> On Fri, Sep 24, 2021 at 4:25 PM Ahmet Altay <al...@google.com> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Sep 24, 2021 at 4:21 PM Udi Meiri <eh...@google.com> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Sep 24, 2021 at 4:18 PM Ahmet Altay <al...@google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Thank you for the update related to allow_unsafe_triggers. My vote
>>>>>>>> is still a +1.
>>>>>>>>
>>>>>>>> If we decide to move forward with this RC, could you please include
>>>>>>>> this bug in the known issues list under the changes.md for this release?
>>>>>>>>
>>>>>>> Yes, that's included in
>>>>>>> https://github.com/apache/beam/pull/15543/files.
>>>>>>>
>>>>>>
>>>>>> Thank you!
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Sep 24, 2021 at 3:56 PM Chamikara Jayalath <
>>>>>>>> chamikara@google.com> wrote:
>>>>>>>>
>>>>>>>>> +1 (binding)
>>>>>>>>> Validated a few scenarios from the spreadsheet.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Cham
>>>>>>>>>
>>>>>>>>> On Fri, Sep 24, 2021 at 3:07 PM Robert Bradshaw <
>>>>>>>>> robertwb@google.com> wrote:
>>>>>>>>>
>>>>>>>>>> +1 (binding)
>>>>>>>>>>
>>>>>>>>>> All the artifacts and signatures look good.
>>>>>>>>>>
>>>>>>>>>> I don't think the unsafe trigger check is severe enough to block
>>>>>>>>>> the release.
>>>>>>>>>>
>>>>>>>>>> On Fri, Sep 24, 2021 at 2:36 PM Udi Meiri <eh...@google.com>
>>>>>>>>>> wrote:
>>>>>>>>>> >
>>>>>>>>>> > Hi everyone, we found a bug during testing. It has to do with
>>>>>>>>>> Python SDK's allow_unsafe_triggers check.
>>>>>>>>>> > There is a preliminary fix that will go to master.
>>>>>>>>>> >
>>>>>>>>>> > For the 2.33.0 release, I'm leaning towards not making a new RC
>>>>>>>>>> since there is a workaround: pass the flag --allow_unsafe_triggers.
>>>>>>>>>> >
>>>>>>>>>> > Please reevaluate your votes accordingly and recast if you've
>>>>>>>>>> changed your vote.
>>>>>>>>>> >
>>>>>>>>>> > Thanks
>>>>>>>>>> >
>>>>>>>>>> > On Fri, Sep 24, 2021 at 12:48 PM Alexey Romanenko <
>>>>>>>>>> aromanenko.dev@gmail.com> wrote:
>>>>>>>>>> >>
>>>>>>>>>> >>
>>>>>>>>>> >> On 24 Sep 2021, at 20:45, Udi Meiri <eh...@google.com> wrote:
>>>>>>>>>> >> Alexey is this something that we should put in the release
>>>>>>>>>> notes, or some other change?
>>>>>>>>>> >>
>>>>>>>>>> >>
>>>>>>>>>> >> Yes, I think it could be helpful to mention that Beam
>>>>>>>>>> Jackson’s deps was bumped and it may require an update of Jackson’s runtime
>>>>>>>>>> deps for Spark 2 users as well.
>>>>>>>>>> >>
>>>>>>>>>> >> —
>>>>>>>>>> >> Alexey
>>>>>>>>>> >>
>>>>>>>>>> >>
>>>>>>>>>> >>
>>>>>>>>>> >>>
>>>>>>>>>> >>>
>>>>>>>>>> >>> —
>>>>>>>>>> >>> Alexey
>>>>>>>>>> >>>
>>>>>>>>>> >>> On 24 Sep 2021, at 16:17, Alexey Romanenko <
>>>>>>>>>> aromanenko.dev@gmail.com> wrote:
>>>>>>>>>> >>>
>>>>>>>>>> >>> I checked with beam-samples [1] and noticed an issue to run
>>>>>>>>>> some pipelines with Spark 2 runner (Spark 3 seems is ok).
>>>>>>>>>> >>> It looks that it’s caused by new Jackson's version updated
>>>>>>>>>> recently [2], even if it’s a minor update but it works fine with a previous
>>>>>>>>>> one.
>>>>>>>>>> >>> I’ll try to find a workaround and get back with a results of
>>>>>>>>>> this.
>>>>>>>>>> >>>
>>>>>>>>>> >>> [1] https://github.com/Talend/beam-samples/
>>>>>>>>>> >>> [2]
>>>>>>>>>> https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c
>>>>>>>>>> >>>
>>>>>>>>>> >>> —
>>>>>>>>>> >>> Alexey
>>>>>>>>>> >>>
>>>>>>>>>> >>> On 24 Sep 2021, at 11:17, Jan Lukavský <je...@seznam.cz>
>>>>>>>>>> wrote:
>>>>>>>>>> >>>
>>>>>>>>>> >>> +1 (non-binding)
>>>>>>>>>> >>>
>>>>>>>>>> >>> Validated several use-cases using non-portable Flink with
>>>>>>>>>> Java SDK.
>>>>>>>>>> >>>
>>>>>>>>>> >>> On 9/24/21 4:55 AM, Valentyn Tymofieiev wrote:
>>>>>>>>>> >>>
>>>>>>>>>> >>> +1. Ran several Python batch and streaming pipelines on
>>>>>>>>>> Dataflow and checked that Dataflow containers have required dependencies of
>>>>>>>>>> Beam.
>>>>>>>>>> >>>
>>>>>>>>>> >>> On Thu, Sep 23, 2021 at 7:03 PM Robert Burke <
>>>>>>>>>> lostluck@apache.org> wrote:
>>>>>>>>>> >>>>
>>>>>>>>>> >>>> +1 (non-binding)
>>>>>>>>>> >>>> I validated the Go Quickstart (wordcount), and my ray tracer
>>>>>>>>>> against the Go Direct runner, Dataflow, and Spark (ensuring the rc1 tagged
>>>>>>>>>> container was used) and they executed successfully.
>>>>>>>>>> >>>>
>>>>>>>>>> >>>> I needed to manually synthesize a pseudo-version to ensure I
>>>>>>>>>> was using the tagged branch version (v2.0.0-20210914211513-b358127f9859)
>>>>>>>>>> instead of simply using v2.33.0-RC1.
>>>>>>>>>> >>>>
>>>>>>>>>> >>>>  It either can't find the package with the right tagged
>>>>>>>>>> version, or it can't find the version. It's not clear to me what the issue
>>>>>>>>>> is, but it's not notionally a release blocker. I'll investigate further
>>>>>>>>>> once we have a full release, as it's probably some unspecified behavior due
>>>>>>>>>> to how we transitioned to Go Modules (which strongly recommended doing a
>>>>>>>>>> major version bump for such transitions, which seems a bit excessive for
>>>>>>>>>> Beam as a whole...).
>>>>>>>>>> >>>>
>>>>>>>>>> >>>> On 2021/09/23 03:59:18, Ahmet Altay <al...@google.com>
>>>>>>>>>> wrote:
>>>>>>>>>> >>>> > +1 on the RC. I validated python quick start examples on
>>>>>>>>>> direct runners.
>>>>>>>>>> >>>> > Thank you Udi.
>>>>>>>>>> >>>> >
>>>>>>>>>> >>>> > On Wed, Sep 22, 2021 at 2:20 PM Robert Burke <
>>>>>>>>>> robert@frantil.com> wrote:
>>>>>>>>>> >>>> >
>>>>>>>>>> >>>> > > Just an FYI that intend to validate the Go SDK for this
>>>>>>>>>> release but can't
>>>>>>>>>> >>>> > > get to it until tomorrow (Thursday). I'm catching up
>>>>>>>>>> from a week of
>>>>>>>>>> >>>> > > vacation. Apologies for the inconvenience.
>>>>>>>>>> >>>> > >
>>>>>>>>>> >>>> > > On Tue, Sep 21, 2021, 10:59 AM Udi Meiri <
>>>>>>>>>> ehudm@google.com> wrote:
>>>>>>>>>> >>>> > >
>>>>>>>>>> >>>> > >> I updated the affected and fixed version fields for
>>>>>>>>>> >>>> > >> https://issues.apache.org/jira/browse/BEAM-12356.
>>>>>>>>>> >>>> > >>
>>>>>>>>>> >>>> > >>
>>>>>>>>>> >>>> > >> On Tue, Sep 21, 2021 at 10:48 AM Reuven Lax <
>>>>>>>>>> relax@google.com> wrote:
>>>>>>>>>> >>>> > >>
>>>>>>>>>> >>>> > >>> Unfortunate - I didn't realize that
>>>>>>>>>> >>>> > >>> https://github.com/apache/beam/pull/15480 didn't make
>>>>>>>>>> the cut.
>>>>>>>>>> >>>> > >>>
>>>>>>>>>> >>>> > >>> This bug was a regression in Beam 2.32.0, and is
>>>>>>>>>> blocking multiple users
>>>>>>>>>> >>>> > >>> from updating.
>>>>>>>>>> >>>> > >>>
>>>>>>>>>> >>>> > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi Meiri <
>>>>>>>>>> udim@apache.org> wrote:
>>>>>>>>>> >>>> > >>>
>>>>>>>>>> >>>> > >>>> Hi everyone,
>>>>>>>>>> >>>> > >>>> Please review and vote on the release candidate #1
>>>>>>>>>> for the version
>>>>>>>>>> >>>> > >>>> 2.33.0, as follows:
>>>>>>>>>> >>>> > >>>> [ ] +1, Approve the release
>>>>>>>>>> >>>> > >>>> [ ] -1, Do not approve the release (please provide
>>>>>>>>>> specific comments)
>>>>>>>>>> >>>> > >>>>
>>>>>>>>>> >>>> > >>>>
>>>>>>>>>> >>>> > >>>> Reviewers are encouraged to test their own use cases
>>>>>>>>>> with the release
>>>>>>>>>> >>>> > >>>> candidate, and vote +1 if
>>>>>>>>>> >>>> > >>>> no issues are found.
>>>>>>>>>> >>>> > >>>>
>>>>>>>>>> >>>> > >>>> The complete staging area is available for your
>>>>>>>>>> review, which includes:
>>>>>>>>>> >>>> > >>>> * JIRA release notes [1],
>>>>>>>>>> >>>> > >>>> * the official Apache source release to be deployed
>>>>>>>>>> to dist.apache.org
>>>>>>>>>> >>>> > >>>> [2], which is signed with the key with fingerprint
>>>>>>>>>> 587B049C36DAAFE6 [3],
>>>>>>>>>> >>>> > >>>> * all artifacts to be deployed to the Maven Central
>>>>>>>>>> Repository [4],
>>>>>>>>>> >>>> > >>>> * source code tag "v2.33.0-RC1" [5],
>>>>>>>>>> >>>> > >>>> * website pull request listing the release [6], the
>>>>>>>>>> blog post [6], and
>>>>>>>>>> >>>> > >>>> publishing the API reference manual [7].
>>>>>>>>>> >>>> > >>>> * Java artifacts were built with Maven 3.6.3 and
>>>>>>>>>> OpenJDK 1.8.0_181.
>>>>>>>>>> >>>> > >>>> * Python artifacts are deployed along with the source
>>>>>>>>>> release to the
>>>>>>>>>> >>>> > >>>> dist.apache.org [2] and pypy[8].
>>>>>>>>>> >>>> > >>>> * Validation sheet with a tab for 2.33.0 release to
>>>>>>>>>> help with
>>>>>>>>>> >>>> > >>>> validation [9].
>>>>>>>>>> >>>> > >>>> * Docker images published to Docker Hub [10].
>>>>>>>>>> >>>> > >>>>
>>>>>>>>>> >>>> > >>>> The vote will be open for at least 72 hours. It is
>>>>>>>>>> adopted by majority
>>>>>>>>>> >>>> > >>>> approval, with at least 3 PMC affirmative votes.
>>>>>>>>>> >>>> > >>>>
>>>>>>>>>> >>>> > >>>> For guidelines on how to try the release in your
>>>>>>>>>> projects, check out
>>>>>>>>>> >>>> > >>>> our blog post at
>>>>>>>>>> https://beam.apache.org/blog/validate-beam-release/.
>>>>>>>>>> >>>> > >>>>
>>>>>>>>>> >>>> > >>>> Thanks,
>>>>>>>>>> >>>> > >>>> Release Manager
>>>>>>>>>> >>>> > >>>>
>>>>>>>>>> >>>> > >>>> [1]
>>>>>>>>>> >>>> > >>>>
>>>>>>>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
>>>>>>>>>> >>>> > >>>> [2]
>>>>>>>>>> https://dist.apache.org/repos/dist/dev/beam/2.33.0/
>>>>>>>>>> >>>> > >>>> [3]
>>>>>>>>>> https://dist.apache.org/repos/dist/release/beam/KEYS
>>>>>>>>>> >>>> > >>>> [4]
>>>>>>>>>> >>>> > >>>>
>>>>>>>>>> https://repository.apache.org/content/repositories/orgapachebeam-1234/
>>>>>>>>>> >>>> > >>>> [5] https://github.com/apache/beam/tree/v2.33.0-RC1
>>>>>>>>>> >>>> > >>>> [6] https://github.com/apache/beam/pull/15543
>>>>>>>>>> >>>> > >>>> [7] https://github.com/apache/beam-site/pull/619
>>>>>>>>>> >>>> > >>>> [8] https://pypi.org/project/apache-beam/2.33.0rc1/
>>>>>>>>>> >>>> > >>>> [9]
>>>>>>>>>> >>>> > >>>>
>>>>>>>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
>>>>>>>>>> >>>> > >>>> [10]
>>>>>>>>>> https://hub.docker.com/search?q=apache%2Fbeam&type=image
>>>>>>>>>> >>>> > >>>>
>>>>>>>>>> >>>> > >>>>
>>>>>>>>>> >>>> >
>>>>>>>>>> >>>
>>>>>>>>>> >>>
>>>>>>>>>> >>>
>>>>>>>>>> >>
>>>>>>>>>>
>>>>>>>>>

Re: [VOTE] Release 2.33.0, release candidate 1

[Robert Burke <re...@google.com>]

On it. Thanks!

On Mon, Sep 27, 2021 at 1:18 PM Udi Meiri <eh...@google.com> wrote:

> Daniel/Robert, feel free to make changes to this PR:
> https://github.com/apache/beam/pull/15543
>
> On Mon, Sep 27, 2021 at 12:08 PM Daniel Oliveira <da...@google.com>
> wrote:
>
>> +1
>>
>> I hadn't realized the pipelines were still finishing successfully. I
>> retried wordcount with that in mind and confirmed it finishes successfully,
>> so this isn't a blocker.
>>
>> Although maybe we should add this to the "Known Issues" because I can
>> easily see those messages being interpreted as a pipeline failure.
>>
>> On Sun, Sep 26, 2021 at 7:26 PM Robert Burke <ro...@frantil.com> wrote:
>>
>>> That's on the SDK side, and it just means the PCollection metrics are
>>> being returned, buy not handled by the SDK. At present the pipeline results
>>> only handle PTransform metrics.
>>>
>>> As such it's not a regression, as adding those is still under
>>> development.
>>>
>>> On Sun, Sep 26, 2021, 7:02 PM Daniel Oliveira <da...@google.com>
>>> wrote:
>>>
>>>> I tried validating wordcount with the Go SDK on Flink. The pipeline
>>>> failed with a wall of errors like the following. I tried this on Flink
>>>> 1.11, 1.12, and 1.13 job servers built from source at the RC1 commit, same
>>>> errors on all of them.
>>>>
>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>>>>>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n9"}
>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>>>>>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n10"}
>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>>>>>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n8"}
>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>>>>>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n7"}
>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>> urn:"beam:metric:sampled_byte_size:v1"
>>>>>  type:"beam:metrics:distribution_int64:v1"
>>>>>  payload:"\x8d%\xfa\xbf\x05\x04\xa8\x0c"  labels:{key:"PCOLLECTION"
>>>>>  value:"n7"}
>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>> urn:"beam:metric:sampled_byte_size:v1"
>>>>>  type:"beam:metrics:distribution_int64:v1"  payload:"\xb9\x05\xcf6\x05\x11"
>>>>>  labels:{key:"PCOLLECTION"  value:"n9"}
>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>> urn:"beam:metric:sampled_byte_size:v1"
>>>>>  type:"beam:metrics:distribution_int64:v1"  payload:"\xc5\x05\x8a1\x04\x12"
>>>>>  labels:{key:"PCOLLECTION"  value:"n8"}
>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>>> urn:"beam:metric:sampled_byte_size:v1"
>>>>>  type:"beam:metrics:distribution_int64:v1"
>>>>>  payload:"\x8d%\xb3\xa7\x07\x14\""  labels:{key:"PCOLLECTION"  value:"n10"}
>>>>> {...}
>>>>
>>>>
>>>>  @Robert Burke <re...@google.com> I think you might know what's going
>>>> on here. Is this solvable with a cherry-pick and a new RC?
>>>>
>>>> On Fri, Sep 24, 2021 at 4:25 PM Ahmet Altay <al...@google.com> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Fri, Sep 24, 2021 at 4:21 PM Udi Meiri <eh...@google.com> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Sep 24, 2021 at 4:18 PM Ahmet Altay <al...@google.com> wrote:
>>>>>>
>>>>>>> Thank you for the update related to allow_unsafe_triggers. My vote
>>>>>>> is still a +1.
>>>>>>>
>>>>>>> If we decide to move forward with this RC, could you please include
>>>>>>> this bug in the known issues list under the changes.md for this release?
>>>>>>>
>>>>>> Yes, that's included in
>>>>>> https://github.com/apache/beam/pull/15543/files.
>>>>>>
>>>>>
>>>>> Thank you!
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> On Fri, Sep 24, 2021 at 3:56 PM Chamikara Jayalath <
>>>>>>> chamikara@google.com> wrote:
>>>>>>>
>>>>>>>> +1 (binding)
>>>>>>>> Validated a few scenarios from the spreadsheet.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Cham
>>>>>>>>
>>>>>>>> On Fri, Sep 24, 2021 at 3:07 PM Robert Bradshaw <
>>>>>>>> robertwb@google.com> wrote:
>>>>>>>>
>>>>>>>>> +1 (binding)
>>>>>>>>>
>>>>>>>>> All the artifacts and signatures look good.
>>>>>>>>>
>>>>>>>>> I don't think the unsafe trigger check is severe enough to block
>>>>>>>>> the release.
>>>>>>>>>
>>>>>>>>> On Fri, Sep 24, 2021 at 2:36 PM Udi Meiri <eh...@google.com>
>>>>>>>>> wrote:
>>>>>>>>> >
>>>>>>>>> > Hi everyone, we found a bug during testing. It has to do with
>>>>>>>>> Python SDK's allow_unsafe_triggers check.
>>>>>>>>> > There is a preliminary fix that will go to master.
>>>>>>>>> >
>>>>>>>>> > For the 2.33.0 release, I'm leaning towards not making a new RC
>>>>>>>>> since there is a workaround: pass the flag --allow_unsafe_triggers.
>>>>>>>>> >
>>>>>>>>> > Please reevaluate your votes accordingly and recast if you've
>>>>>>>>> changed your vote.
>>>>>>>>> >
>>>>>>>>> > Thanks
>>>>>>>>> >
>>>>>>>>> > On Fri, Sep 24, 2021 at 12:48 PM Alexey Romanenko <
>>>>>>>>> aromanenko.dev@gmail.com> wrote:
>>>>>>>>> >>
>>>>>>>>> >>
>>>>>>>>> >> On 24 Sep 2021, at 20:45, Udi Meiri <eh...@google.com> wrote:
>>>>>>>>> >> Alexey is this something that we should put in the release
>>>>>>>>> notes, or some other change?
>>>>>>>>> >>
>>>>>>>>> >>
>>>>>>>>> >> Yes, I think it could be helpful to mention that Beam Jackson’s
>>>>>>>>> deps was bumped and it may require an update of Jackson’s runtime deps for
>>>>>>>>> Spark 2 users as well.
>>>>>>>>> >>
>>>>>>>>> >> —
>>>>>>>>> >> Alexey
>>>>>>>>> >>
>>>>>>>>> >>
>>>>>>>>> >>
>>>>>>>>> >>>
>>>>>>>>> >>>
>>>>>>>>> >>> —
>>>>>>>>> >>> Alexey
>>>>>>>>> >>>
>>>>>>>>> >>> On 24 Sep 2021, at 16:17, Alexey Romanenko <
>>>>>>>>> aromanenko.dev@gmail.com> wrote:
>>>>>>>>> >>>
>>>>>>>>> >>> I checked with beam-samples [1] and noticed an issue to run
>>>>>>>>> some pipelines with Spark 2 runner (Spark 3 seems is ok).
>>>>>>>>> >>> It looks that it’s caused by new Jackson's version updated
>>>>>>>>> recently [2], even if it’s a minor update but it works fine with a previous
>>>>>>>>> one.
>>>>>>>>> >>> I’ll try to find a workaround and get back with a results of
>>>>>>>>> this.
>>>>>>>>> >>>
>>>>>>>>> >>> [1] https://github.com/Talend/beam-samples/
>>>>>>>>> >>> [2]
>>>>>>>>> https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c
>>>>>>>>> >>>
>>>>>>>>> >>> —
>>>>>>>>> >>> Alexey
>>>>>>>>> >>>
>>>>>>>>> >>> On 24 Sep 2021, at 11:17, Jan Lukavský <je...@seznam.cz>
>>>>>>>>> wrote:
>>>>>>>>> >>>
>>>>>>>>> >>> +1 (non-binding)
>>>>>>>>> >>>
>>>>>>>>> >>> Validated several use-cases using non-portable Flink with Java
>>>>>>>>> SDK.
>>>>>>>>> >>>
>>>>>>>>> >>> On 9/24/21 4:55 AM, Valentyn Tymofieiev wrote:
>>>>>>>>> >>>
>>>>>>>>> >>> +1. Ran several Python batch and streaming pipelines on
>>>>>>>>> Dataflow and checked that Dataflow containers have required dependencies of
>>>>>>>>> Beam.
>>>>>>>>> >>>
>>>>>>>>> >>> On Thu, Sep 23, 2021 at 7:03 PM Robert Burke <
>>>>>>>>> lostluck@apache.org> wrote:
>>>>>>>>> >>>>
>>>>>>>>> >>>> +1 (non-binding)
>>>>>>>>> >>>> I validated the Go Quickstart (wordcount), and my ray tracer
>>>>>>>>> against the Go Direct runner, Dataflow, and Spark (ensuring the rc1 tagged
>>>>>>>>> container was used) and they executed successfully.
>>>>>>>>> >>>>
>>>>>>>>> >>>> I needed to manually synthesize a pseudo-version to ensure I
>>>>>>>>> was using the tagged branch version (v2.0.0-20210914211513-b358127f9859)
>>>>>>>>> instead of simply using v2.33.0-RC1.
>>>>>>>>> >>>>
>>>>>>>>> >>>>  It either can't find the package with the right tagged
>>>>>>>>> version, or it can't find the version. It's not clear to me what the issue
>>>>>>>>> is, but it's not notionally a release blocker. I'll investigate further
>>>>>>>>> once we have a full release, as it's probably some unspecified behavior due
>>>>>>>>> to how we transitioned to Go Modules (which strongly recommended doing a
>>>>>>>>> major version bump for such transitions, which seems a bit excessive for
>>>>>>>>> Beam as a whole...).
>>>>>>>>> >>>>
>>>>>>>>> >>>> On 2021/09/23 03:59:18, Ahmet Altay <al...@google.com> wrote:
>>>>>>>>> >>>> > +1 on the RC. I validated python quick start examples on
>>>>>>>>> direct runners.
>>>>>>>>> >>>> > Thank you Udi.
>>>>>>>>> >>>> >
>>>>>>>>> >>>> > On Wed, Sep 22, 2021 at 2:20 PM Robert Burke <
>>>>>>>>> robert@frantil.com> wrote:
>>>>>>>>> >>>> >
>>>>>>>>> >>>> > > Just an FYI that intend to validate the Go SDK for this
>>>>>>>>> release but can't
>>>>>>>>> >>>> > > get to it until tomorrow (Thursday). I'm catching up from
>>>>>>>>> a week of
>>>>>>>>> >>>> > > vacation. Apologies for the inconvenience.
>>>>>>>>> >>>> > >
>>>>>>>>> >>>> > > On Tue, Sep 21, 2021, 10:59 AM Udi Meiri <
>>>>>>>>> ehudm@google.com> wrote:
>>>>>>>>> >>>> > >
>>>>>>>>> >>>> > >> I updated the affected and fixed version fields for
>>>>>>>>> >>>> > >> https://issues.apache.org/jira/browse/BEAM-12356.
>>>>>>>>> >>>> > >>
>>>>>>>>> >>>> > >>
>>>>>>>>> >>>> > >> On Tue, Sep 21, 2021 at 10:48 AM Reuven Lax <
>>>>>>>>> relax@google.com> wrote:
>>>>>>>>> >>>> > >>
>>>>>>>>> >>>> > >>> Unfortunate - I didn't realize that
>>>>>>>>> >>>> > >>> https://github.com/apache/beam/pull/15480 didn't make
>>>>>>>>> the cut.
>>>>>>>>> >>>> > >>>
>>>>>>>>> >>>> > >>> This bug was a regression in Beam 2.32.0, and is
>>>>>>>>> blocking multiple users
>>>>>>>>> >>>> > >>> from updating.
>>>>>>>>> >>>> > >>>
>>>>>>>>> >>>> > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi Meiri <
>>>>>>>>> udim@apache.org> wrote:
>>>>>>>>> >>>> > >>>
>>>>>>>>> >>>> > >>>> Hi everyone,
>>>>>>>>> >>>> > >>>> Please review and vote on the release candidate #1 for
>>>>>>>>> the version
>>>>>>>>> >>>> > >>>> 2.33.0, as follows:
>>>>>>>>> >>>> > >>>> [ ] +1, Approve the release
>>>>>>>>> >>>> > >>>> [ ] -1, Do not approve the release (please provide
>>>>>>>>> specific comments)
>>>>>>>>> >>>> > >>>>
>>>>>>>>> >>>> > >>>>
>>>>>>>>> >>>> > >>>> Reviewers are encouraged to test their own use cases
>>>>>>>>> with the release
>>>>>>>>> >>>> > >>>> candidate, and vote +1 if
>>>>>>>>> >>>> > >>>> no issues are found.
>>>>>>>>> >>>> > >>>>
>>>>>>>>> >>>> > >>>> The complete staging area is available for your
>>>>>>>>> review, which includes:
>>>>>>>>> >>>> > >>>> * JIRA release notes [1],
>>>>>>>>> >>>> > >>>> * the official Apache source release to be deployed to
>>>>>>>>> dist.apache.org
>>>>>>>>> >>>> > >>>> [2], which is signed with the key with fingerprint
>>>>>>>>> 587B049C36DAAFE6 [3],
>>>>>>>>> >>>> > >>>> * all artifacts to be deployed to the Maven Central
>>>>>>>>> Repository [4],
>>>>>>>>> >>>> > >>>> * source code tag "v2.33.0-RC1" [5],
>>>>>>>>> >>>> > >>>> * website pull request listing the release [6], the
>>>>>>>>> blog post [6], and
>>>>>>>>> >>>> > >>>> publishing the API reference manual [7].
>>>>>>>>> >>>> > >>>> * Java artifacts were built with Maven 3.6.3 and
>>>>>>>>> OpenJDK 1.8.0_181.
>>>>>>>>> >>>> > >>>> * Python artifacts are deployed along with the source
>>>>>>>>> release to the
>>>>>>>>> >>>> > >>>> dist.apache.org [2] and pypy[8].
>>>>>>>>> >>>> > >>>> * Validation sheet with a tab for 2.33.0 release to
>>>>>>>>> help with
>>>>>>>>> >>>> > >>>> validation [9].
>>>>>>>>> >>>> > >>>> * Docker images published to Docker Hub [10].
>>>>>>>>> >>>> > >>>>
>>>>>>>>> >>>> > >>>> The vote will be open for at least 72 hours. It is
>>>>>>>>> adopted by majority
>>>>>>>>> >>>> > >>>> approval, with at least 3 PMC affirmative votes.
>>>>>>>>> >>>> > >>>>
>>>>>>>>> >>>> > >>>> For guidelines on how to try the release in your
>>>>>>>>> projects, check out
>>>>>>>>> >>>> > >>>> our blog post at
>>>>>>>>> https://beam.apache.org/blog/validate-beam-release/.
>>>>>>>>> >>>> > >>>>
>>>>>>>>> >>>> > >>>> Thanks,
>>>>>>>>> >>>> > >>>> Release Manager
>>>>>>>>> >>>> > >>>>
>>>>>>>>> >>>> > >>>> [1]
>>>>>>>>> >>>> > >>>>
>>>>>>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
>>>>>>>>> >>>> > >>>> [2]
>>>>>>>>> https://dist.apache.org/repos/dist/dev/beam/2.33.0/
>>>>>>>>> >>>> > >>>> [3]
>>>>>>>>> https://dist.apache.org/repos/dist/release/beam/KEYS
>>>>>>>>> >>>> > >>>> [4]
>>>>>>>>> >>>> > >>>>
>>>>>>>>> https://repository.apache.org/content/repositories/orgapachebeam-1234/
>>>>>>>>> >>>> > >>>> [5] https://github.com/apache/beam/tree/v2.33.0-RC1
>>>>>>>>> >>>> > >>>> [6] https://github.com/apache/beam/pull/15543
>>>>>>>>> >>>> > >>>> [7] https://github.com/apache/beam-site/pull/619
>>>>>>>>> >>>> > >>>> [8] https://pypi.org/project/apache-beam/2.33.0rc1/
>>>>>>>>> >>>> > >>>> [9]
>>>>>>>>> >>>> > >>>>
>>>>>>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
>>>>>>>>> >>>> > >>>> [10]
>>>>>>>>> https://hub.docker.com/search?q=apache%2Fbeam&type=image
>>>>>>>>> >>>> > >>>>
>>>>>>>>> >>>> > >>>>
>>>>>>>>> >>>> >
>>>>>>>>> >>>
>>>>>>>>> >>>
>>>>>>>>> >>>
>>>>>>>>> >>
>>>>>>>>>
>>>>>>>>

Re: [VOTE] Release 2.33.0, release candidate 1

[Udi Meiri <eh...@google.com>]

Daniel/Robert, feel free to make changes to this PR:
https://github.com/apache/beam/pull/15543

On Mon, Sep 27, 2021 at 12:08 PM Daniel Oliveira <da...@google.com>
wrote:

> +1
>
> I hadn't realized the pipelines were still finishing successfully. I
> retried wordcount with that in mind and confirmed it finishes successfully,
> so this isn't a blocker.
>
> Although maybe we should add this to the "Known Issues" because I can
> easily see those messages being interpreted as a pipeline failure.
>
> On Sun, Sep 26, 2021 at 7:26 PM Robert Burke <ro...@frantil.com> wrote:
>
>> That's on the SDK side, and it just means the PCollection metrics are
>> being returned, buy not handled by the SDK. At present the pipeline results
>> only handle PTransform metrics.
>>
>> As such it's not a regression, as adding those is still under development.
>>
>> On Sun, Sep 26, 2021, 7:02 PM Daniel Oliveira <da...@google.com>
>> wrote:
>>
>>> I tried validating wordcount with the Go SDK on Flink. The pipeline
>>> failed with a wall of errors like the following. I tried this on Flink
>>> 1.11, 1.12, and 1.13 job servers built from source at the RC1 commit, same
>>> errors on all of them.
>>>
>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>>>>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n9"}
>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>>>>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n10"}
>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>>>>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n8"}
>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>>>>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n7"}
>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>> urn:"beam:metric:sampled_byte_size:v1"
>>>>  type:"beam:metrics:distribution_int64:v1"
>>>>  payload:"\x8d%\xfa\xbf\x05\x04\xa8\x0c"  labels:{key:"PCOLLECTION"
>>>>  value:"n7"}
>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>> urn:"beam:metric:sampled_byte_size:v1"
>>>>  type:"beam:metrics:distribution_int64:v1"  payload:"\xb9\x05\xcf6\x05\x11"
>>>>  labels:{key:"PCOLLECTION"  value:"n9"}
>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>> urn:"beam:metric:sampled_byte_size:v1"
>>>>  type:"beam:metrics:distribution_int64:v1"  payload:"\xc5\x05\x8a1\x04\x12"
>>>>  labels:{key:"PCOLLECTION"  value:"n8"}
>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>>> urn:"beam:metric:sampled_byte_size:v1"
>>>>  type:"beam:metrics:distribution_int64:v1"
>>>>  payload:"\x8d%\xb3\xa7\x07\x14\""  labels:{key:"PCOLLECTION"  value:"n10"}
>>>> {...}
>>>
>>>
>>>  @Robert Burke <re...@google.com> I think you might know what's going on
>>> here. Is this solvable with a cherry-pick and a new RC?
>>>
>>> On Fri, Sep 24, 2021 at 4:25 PM Ahmet Altay <al...@google.com> wrote:
>>>
>>>>
>>>>
>>>> On Fri, Sep 24, 2021 at 4:21 PM Udi Meiri <eh...@google.com> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Fri, Sep 24, 2021 at 4:18 PM Ahmet Altay <al...@google.com> wrote:
>>>>>
>>>>>> Thank you for the update related to allow_unsafe_triggers. My vote is
>>>>>> still a +1.
>>>>>>
>>>>>> If we decide to move forward with this RC, could you please include
>>>>>> this bug in the known issues list under the changes.md for this release?
>>>>>>
>>>>> Yes, that's included in
>>>>> https://github.com/apache/beam/pull/15543/files.
>>>>>
>>>>
>>>> Thank you!
>>>>
>>>>
>>>>>
>>>>>
>>>>>>
>>>>>> On Fri, Sep 24, 2021 at 3:56 PM Chamikara Jayalath <
>>>>>> chamikara@google.com> wrote:
>>>>>>
>>>>>>> +1 (binding)
>>>>>>> Validated a few scenarios from the spreadsheet.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Cham
>>>>>>>
>>>>>>> On Fri, Sep 24, 2021 at 3:07 PM Robert Bradshaw <ro...@google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> +1 (binding)
>>>>>>>>
>>>>>>>> All the artifacts and signatures look good.
>>>>>>>>
>>>>>>>> I don't think the unsafe trigger check is severe enough to block
>>>>>>>> the release.
>>>>>>>>
>>>>>>>> On Fri, Sep 24, 2021 at 2:36 PM Udi Meiri <eh...@google.com> wrote:
>>>>>>>> >
>>>>>>>> > Hi everyone, we found a bug during testing. It has to do with
>>>>>>>> Python SDK's allow_unsafe_triggers check.
>>>>>>>> > There is a preliminary fix that will go to master.
>>>>>>>> >
>>>>>>>> > For the 2.33.0 release, I'm leaning towards not making a new RC
>>>>>>>> since there is a workaround: pass the flag --allow_unsafe_triggers.
>>>>>>>> >
>>>>>>>> > Please reevaluate your votes accordingly and recast if you've
>>>>>>>> changed your vote.
>>>>>>>> >
>>>>>>>> > Thanks
>>>>>>>> >
>>>>>>>> > On Fri, Sep 24, 2021 at 12:48 PM Alexey Romanenko <
>>>>>>>> aromanenko.dev@gmail.com> wrote:
>>>>>>>> >>
>>>>>>>> >>
>>>>>>>> >> On 24 Sep 2021, at 20:45, Udi Meiri <eh...@google.com> wrote:
>>>>>>>> >> Alexey is this something that we should put in the release
>>>>>>>> notes, or some other change?
>>>>>>>> >>
>>>>>>>> >>
>>>>>>>> >> Yes, I think it could be helpful to mention that Beam Jackson’s
>>>>>>>> deps was bumped and it may require an update of Jackson’s runtime deps for
>>>>>>>> Spark 2 users as well.
>>>>>>>> >>
>>>>>>>> >> —
>>>>>>>> >> Alexey
>>>>>>>> >>
>>>>>>>> >>
>>>>>>>> >>
>>>>>>>> >>>
>>>>>>>> >>>
>>>>>>>> >>> —
>>>>>>>> >>> Alexey
>>>>>>>> >>>
>>>>>>>> >>> On 24 Sep 2021, at 16:17, Alexey Romanenko <
>>>>>>>> aromanenko.dev@gmail.com> wrote:
>>>>>>>> >>>
>>>>>>>> >>> I checked with beam-samples [1] and noticed an issue to run
>>>>>>>> some pipelines with Spark 2 runner (Spark 3 seems is ok).
>>>>>>>> >>> It looks that it’s caused by new Jackson's version updated
>>>>>>>> recently [2], even if it’s a minor update but it works fine with a previous
>>>>>>>> one.
>>>>>>>> >>> I’ll try to find a workaround and get back with a results of
>>>>>>>> this.
>>>>>>>> >>>
>>>>>>>> >>> [1] https://github.com/Talend/beam-samples/
>>>>>>>> >>> [2]
>>>>>>>> https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c
>>>>>>>> >>>
>>>>>>>> >>> —
>>>>>>>> >>> Alexey
>>>>>>>> >>>
>>>>>>>> >>> On 24 Sep 2021, at 11:17, Jan Lukavský <je...@seznam.cz> wrote:
>>>>>>>> >>>
>>>>>>>> >>> +1 (non-binding)
>>>>>>>> >>>
>>>>>>>> >>> Validated several use-cases using non-portable Flink with Java
>>>>>>>> SDK.
>>>>>>>> >>>
>>>>>>>> >>> On 9/24/21 4:55 AM, Valentyn Tymofieiev wrote:
>>>>>>>> >>>
>>>>>>>> >>> +1. Ran several Python batch and streaming pipelines on
>>>>>>>> Dataflow and checked that Dataflow containers have required dependencies of
>>>>>>>> Beam.
>>>>>>>> >>>
>>>>>>>> >>> On Thu, Sep 23, 2021 at 7:03 PM Robert Burke <
>>>>>>>> lostluck@apache.org> wrote:
>>>>>>>> >>>>
>>>>>>>> >>>> +1 (non-binding)
>>>>>>>> >>>> I validated the Go Quickstart (wordcount), and my ray tracer
>>>>>>>> against the Go Direct runner, Dataflow, and Spark (ensuring the rc1 tagged
>>>>>>>> container was used) and they executed successfully.
>>>>>>>> >>>>
>>>>>>>> >>>> I needed to manually synthesize a pseudo-version to ensure I
>>>>>>>> was using the tagged branch version (v2.0.0-20210914211513-b358127f9859)
>>>>>>>> instead of simply using v2.33.0-RC1.
>>>>>>>> >>>>
>>>>>>>> >>>>  It either can't find the package with the right tagged
>>>>>>>> version, or it can't find the version. It's not clear to me what the issue
>>>>>>>> is, but it's not notionally a release blocker. I'll investigate further
>>>>>>>> once we have a full release, as it's probably some unspecified behavior due
>>>>>>>> to how we transitioned to Go Modules (which strongly recommended doing a
>>>>>>>> major version bump for such transitions, which seems a bit excessive for
>>>>>>>> Beam as a whole...).
>>>>>>>> >>>>
>>>>>>>> >>>> On 2021/09/23 03:59:18, Ahmet Altay <al...@google.com> wrote:
>>>>>>>> >>>> > +1 on the RC. I validated python quick start examples on
>>>>>>>> direct runners.
>>>>>>>> >>>> > Thank you Udi.
>>>>>>>> >>>> >
>>>>>>>> >>>> > On Wed, Sep 22, 2021 at 2:20 PM Robert Burke <
>>>>>>>> robert@frantil.com> wrote:
>>>>>>>> >>>> >
>>>>>>>> >>>> > > Just an FYI that intend to validate the Go SDK for this
>>>>>>>> release but can't
>>>>>>>> >>>> > > get to it until tomorrow (Thursday). I'm catching up from
>>>>>>>> a week of
>>>>>>>> >>>> > > vacation. Apologies for the inconvenience.
>>>>>>>> >>>> > >
>>>>>>>> >>>> > > On Tue, Sep 21, 2021, 10:59 AM Udi Meiri <eh...@google.com>
>>>>>>>> wrote:
>>>>>>>> >>>> > >
>>>>>>>> >>>> > >> I updated the affected and fixed version fields for
>>>>>>>> >>>> > >> https://issues.apache.org/jira/browse/BEAM-12356.
>>>>>>>> >>>> > >>
>>>>>>>> >>>> > >>
>>>>>>>> >>>> > >> On Tue, Sep 21, 2021 at 10:48 AM Reuven Lax <
>>>>>>>> relax@google.com> wrote:
>>>>>>>> >>>> > >>
>>>>>>>> >>>> > >>> Unfortunate - I didn't realize that
>>>>>>>> >>>> > >>> https://github.com/apache/beam/pull/15480 didn't make
>>>>>>>> the cut.
>>>>>>>> >>>> > >>>
>>>>>>>> >>>> > >>> This bug was a regression in Beam 2.32.0, and is
>>>>>>>> blocking multiple users
>>>>>>>> >>>> > >>> from updating.
>>>>>>>> >>>> > >>>
>>>>>>>> >>>> > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi Meiri <
>>>>>>>> udim@apache.org> wrote:
>>>>>>>> >>>> > >>>
>>>>>>>> >>>> > >>>> Hi everyone,
>>>>>>>> >>>> > >>>> Please review and vote on the release candidate #1 for
>>>>>>>> the version
>>>>>>>> >>>> > >>>> 2.33.0, as follows:
>>>>>>>> >>>> > >>>> [ ] +1, Approve the release
>>>>>>>> >>>> > >>>> [ ] -1, Do not approve the release (please provide
>>>>>>>> specific comments)
>>>>>>>> >>>> > >>>>
>>>>>>>> >>>> > >>>>
>>>>>>>> >>>> > >>>> Reviewers are encouraged to test their own use cases
>>>>>>>> with the release
>>>>>>>> >>>> > >>>> candidate, and vote +1 if
>>>>>>>> >>>> > >>>> no issues are found.
>>>>>>>> >>>> > >>>>
>>>>>>>> >>>> > >>>> The complete staging area is available for your review,
>>>>>>>> which includes:
>>>>>>>> >>>> > >>>> * JIRA release notes [1],
>>>>>>>> >>>> > >>>> * the official Apache source release to be deployed to
>>>>>>>> dist.apache.org
>>>>>>>> >>>> > >>>> [2], which is signed with the key with fingerprint
>>>>>>>> 587B049C36DAAFE6 [3],
>>>>>>>> >>>> > >>>> * all artifacts to be deployed to the Maven Central
>>>>>>>> Repository [4],
>>>>>>>> >>>> > >>>> * source code tag "v2.33.0-RC1" [5],
>>>>>>>> >>>> > >>>> * website pull request listing the release [6], the
>>>>>>>> blog post [6], and
>>>>>>>> >>>> > >>>> publishing the API reference manual [7].
>>>>>>>> >>>> > >>>> * Java artifacts were built with Maven 3.6.3 and
>>>>>>>> OpenJDK 1.8.0_181.
>>>>>>>> >>>> > >>>> * Python artifacts are deployed along with the source
>>>>>>>> release to the
>>>>>>>> >>>> > >>>> dist.apache.org [2] and pypy[8].
>>>>>>>> >>>> > >>>> * Validation sheet with a tab for 2.33.0 release to
>>>>>>>> help with
>>>>>>>> >>>> > >>>> validation [9].
>>>>>>>> >>>> > >>>> * Docker images published to Docker Hub [10].
>>>>>>>> >>>> > >>>>
>>>>>>>> >>>> > >>>> The vote will be open for at least 72 hours. It is
>>>>>>>> adopted by majority
>>>>>>>> >>>> > >>>> approval, with at least 3 PMC affirmative votes.
>>>>>>>> >>>> > >>>>
>>>>>>>> >>>> > >>>> For guidelines on how to try the release in your
>>>>>>>> projects, check out
>>>>>>>> >>>> > >>>> our blog post at
>>>>>>>> https://beam.apache.org/blog/validate-beam-release/.
>>>>>>>> >>>> > >>>>
>>>>>>>> >>>> > >>>> Thanks,
>>>>>>>> >>>> > >>>> Release Manager
>>>>>>>> >>>> > >>>>
>>>>>>>> >>>> > >>>> [1]
>>>>>>>> >>>> > >>>>
>>>>>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
>>>>>>>> >>>> > >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.33.0/
>>>>>>>> >>>> > >>>> [3]
>>>>>>>> https://dist.apache.org/repos/dist/release/beam/KEYS
>>>>>>>> >>>> > >>>> [4]
>>>>>>>> >>>> > >>>>
>>>>>>>> https://repository.apache.org/content/repositories/orgapachebeam-1234/
>>>>>>>> >>>> > >>>> [5] https://github.com/apache/beam/tree/v2.33.0-RC1
>>>>>>>> >>>> > >>>> [6] https://github.com/apache/beam/pull/15543
>>>>>>>> >>>> > >>>> [7] https://github.com/apache/beam-site/pull/619
>>>>>>>> >>>> > >>>> [8] https://pypi.org/project/apache-beam/2.33.0rc1/
>>>>>>>> >>>> > >>>> [9]
>>>>>>>> >>>> > >>>>
>>>>>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
>>>>>>>> >>>> > >>>> [10]
>>>>>>>> https://hub.docker.com/search?q=apache%2Fbeam&type=image
>>>>>>>> >>>> > >>>>
>>>>>>>> >>>> > >>>>
>>>>>>>> >>>> >
>>>>>>>> >>>
>>>>>>>> >>>
>>>>>>>> >>>
>>>>>>>> >>
>>>>>>>>
>>>>>>>

Re: [VOTE] Release 2.33.0, release candidate 1

[Daniel Oliveira <da...@google.com>]

+1

I hadn't realized the pipelines were still finishing successfully. I
retried wordcount with that in mind and confirmed it finishes successfully,
so this isn't a blocker.

Although maybe we should add this to the "Known Issues" because I can
easily see those messages being interpreted as a pipeline failure.

On Sun, Sep 26, 2021 at 7:26 PM Robert Burke <ro...@frantil.com> wrote:

> That's on the SDK side, and it just means the PCollection metrics are
> being returned, buy not handled by the SDK. At present the pipeline results
> only handle PTransform metrics.
>
> As such it's not a regression, as adding those is still under development.
>
> On Sun, Sep 26, 2021, 7:02 PM Daniel Oliveira <da...@google.com>
> wrote:
>
>> I tried validating wordcount with the Go SDK on Flink. The pipeline
>> failed with a wall of errors like the following. I tried this on Flink
>> 1.11, 1.12, and 1.13 job servers built from source at the RC1 commit, same
>> errors on all of them.
>>
>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>>>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n9"}
>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>>>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n10"}
>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>>>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n8"}
>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>>>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n7"}
>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>> urn:"beam:metric:sampled_byte_size:v1"
>>>  type:"beam:metrics:distribution_int64:v1"
>>>  payload:"\x8d%\xfa\xbf\x05\x04\xa8\x0c"  labels:{key:"PCOLLECTION"
>>>  value:"n7"}
>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>> urn:"beam:metric:sampled_byte_size:v1"
>>>  type:"beam:metrics:distribution_int64:v1"  payload:"\xb9\x05\xcf6\x05\x11"
>>>  labels:{key:"PCOLLECTION"  value:"n9"}
>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>> urn:"beam:metric:sampled_byte_size:v1"
>>>  type:"beam:metrics:distribution_int64:v1"  payload:"\xc5\x05\x8a1\x04\x12"
>>>  labels:{key:"PCOLLECTION"  value:"n8"}
>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>>> urn:"beam:metric:sampled_byte_size:v1"
>>>  type:"beam:metrics:distribution_int64:v1"
>>>  payload:"\x8d%\xb3\xa7\x07\x14\""  labels:{key:"PCOLLECTION"  value:"n10"}
>>> {...}
>>
>>
>>  @Robert Burke <re...@google.com> I think you might know what's going on
>> here. Is this solvable with a cherry-pick and a new RC?
>>
>> On Fri, Sep 24, 2021 at 4:25 PM Ahmet Altay <al...@google.com> wrote:
>>
>>>
>>>
>>> On Fri, Sep 24, 2021 at 4:21 PM Udi Meiri <eh...@google.com> wrote:
>>>
>>>>
>>>>
>>>> On Fri, Sep 24, 2021 at 4:18 PM Ahmet Altay <al...@google.com> wrote:
>>>>
>>>>> Thank you for the update related to allow_unsafe_triggers. My vote is
>>>>> still a +1.
>>>>>
>>>>> If we decide to move forward with this RC, could you please include
>>>>> this bug in the known issues list under the changes.md for this release?
>>>>>
>>>> Yes, that's included in https://github.com/apache/beam/pull/15543/files
>>>> .
>>>>
>>>
>>> Thank you!
>>>
>>>
>>>>
>>>>
>>>>>
>>>>> On Fri, Sep 24, 2021 at 3:56 PM Chamikara Jayalath <
>>>>> chamikara@google.com> wrote:
>>>>>
>>>>>> +1 (binding)
>>>>>> Validated a few scenarios from the spreadsheet.
>>>>>>
>>>>>> Thanks,
>>>>>> Cham
>>>>>>
>>>>>> On Fri, Sep 24, 2021 at 3:07 PM Robert Bradshaw <ro...@google.com>
>>>>>> wrote:
>>>>>>
>>>>>>> +1 (binding)
>>>>>>>
>>>>>>> All the artifacts and signatures look good.
>>>>>>>
>>>>>>> I don't think the unsafe trigger check is severe enough to block the
>>>>>>> release.
>>>>>>>
>>>>>>> On Fri, Sep 24, 2021 at 2:36 PM Udi Meiri <eh...@google.com> wrote:
>>>>>>> >
>>>>>>> > Hi everyone, we found a bug during testing. It has to do with
>>>>>>> Python SDK's allow_unsafe_triggers check.
>>>>>>> > There is a preliminary fix that will go to master.
>>>>>>> >
>>>>>>> > For the 2.33.0 release, I'm leaning towards not making a new RC
>>>>>>> since there is a workaround: pass the flag --allow_unsafe_triggers.
>>>>>>> >
>>>>>>> > Please reevaluate your votes accordingly and recast if you've
>>>>>>> changed your vote.
>>>>>>> >
>>>>>>> > Thanks
>>>>>>> >
>>>>>>> > On Fri, Sep 24, 2021 at 12:48 PM Alexey Romanenko <
>>>>>>> aromanenko.dev@gmail.com> wrote:
>>>>>>> >>
>>>>>>> >>
>>>>>>> >> On 24 Sep 2021, at 20:45, Udi Meiri <eh...@google.com> wrote:
>>>>>>> >> Alexey is this something that we should put in the release notes,
>>>>>>> or some other change?
>>>>>>> >>
>>>>>>> >>
>>>>>>> >> Yes, I think it could be helpful to mention that Beam Jackson’s
>>>>>>> deps was bumped and it may require an update of Jackson’s runtime deps for
>>>>>>> Spark 2 users as well.
>>>>>>> >>
>>>>>>> >> —
>>>>>>> >> Alexey
>>>>>>> >>
>>>>>>> >>
>>>>>>> >>
>>>>>>> >>>
>>>>>>> >>>
>>>>>>> >>> —
>>>>>>> >>> Alexey
>>>>>>> >>>
>>>>>>> >>> On 24 Sep 2021, at 16:17, Alexey Romanenko <
>>>>>>> aromanenko.dev@gmail.com> wrote:
>>>>>>> >>>
>>>>>>> >>> I checked with beam-samples [1] and noticed an issue to run some
>>>>>>> pipelines with Spark 2 runner (Spark 3 seems is ok).
>>>>>>> >>> It looks that it’s caused by new Jackson's version updated
>>>>>>> recently [2], even if it’s a minor update but it works fine with a previous
>>>>>>> one.
>>>>>>> >>> I’ll try to find a workaround and get back with a results of
>>>>>>> this.
>>>>>>> >>>
>>>>>>> >>> [1] https://github.com/Talend/beam-samples/
>>>>>>> >>> [2]
>>>>>>> https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c
>>>>>>> >>>
>>>>>>> >>> —
>>>>>>> >>> Alexey
>>>>>>> >>>
>>>>>>> >>> On 24 Sep 2021, at 11:17, Jan Lukavský <je...@seznam.cz> wrote:
>>>>>>> >>>
>>>>>>> >>> +1 (non-binding)
>>>>>>> >>>
>>>>>>> >>> Validated several use-cases using non-portable Flink with Java
>>>>>>> SDK.
>>>>>>> >>>
>>>>>>> >>> On 9/24/21 4:55 AM, Valentyn Tymofieiev wrote:
>>>>>>> >>>
>>>>>>> >>> +1. Ran several Python batch and streaming pipelines on Dataflow
>>>>>>> and checked that Dataflow containers have required dependencies of Beam.
>>>>>>> >>>
>>>>>>> >>> On Thu, Sep 23, 2021 at 7:03 PM Robert Burke <
>>>>>>> lostluck@apache.org> wrote:
>>>>>>> >>>>
>>>>>>> >>>> +1 (non-binding)
>>>>>>> >>>> I validated the Go Quickstart (wordcount), and my ray tracer
>>>>>>> against the Go Direct runner, Dataflow, and Spark (ensuring the rc1 tagged
>>>>>>> container was used) and they executed successfully.
>>>>>>> >>>>
>>>>>>> >>>> I needed to manually synthesize a pseudo-version to ensure I
>>>>>>> was using the tagged branch version (v2.0.0-20210914211513-b358127f9859)
>>>>>>> instead of simply using v2.33.0-RC1.
>>>>>>> >>>>
>>>>>>> >>>>  It either can't find the package with the right tagged
>>>>>>> version, or it can't find the version. It's not clear to me what the issue
>>>>>>> is, but it's not notionally a release blocker. I'll investigate further
>>>>>>> once we have a full release, as it's probably some unspecified behavior due
>>>>>>> to how we transitioned to Go Modules (which strongly recommended doing a
>>>>>>> major version bump for such transitions, which seems a bit excessive for
>>>>>>> Beam as a whole...).
>>>>>>> >>>>
>>>>>>> >>>> On 2021/09/23 03:59:18, Ahmet Altay <al...@google.com> wrote:
>>>>>>> >>>> > +1 on the RC. I validated python quick start examples on
>>>>>>> direct runners.
>>>>>>> >>>> > Thank you Udi.
>>>>>>> >>>> >
>>>>>>> >>>> > On Wed, Sep 22, 2021 at 2:20 PM Robert Burke <
>>>>>>> robert@frantil.com> wrote:
>>>>>>> >>>> >
>>>>>>> >>>> > > Just an FYI that intend to validate the Go SDK for this
>>>>>>> release but can't
>>>>>>> >>>> > > get to it until tomorrow (Thursday). I'm catching up from a
>>>>>>> week of
>>>>>>> >>>> > > vacation. Apologies for the inconvenience.
>>>>>>> >>>> > >
>>>>>>> >>>> > > On Tue, Sep 21, 2021, 10:59 AM Udi Meiri <eh...@google.com>
>>>>>>> wrote:
>>>>>>> >>>> > >
>>>>>>> >>>> > >> I updated the affected and fixed version fields for
>>>>>>> >>>> > >> https://issues.apache.org/jira/browse/BEAM-12356.
>>>>>>> >>>> > >>
>>>>>>> >>>> > >>
>>>>>>> >>>> > >> On Tue, Sep 21, 2021 at 10:48 AM Reuven Lax <
>>>>>>> relax@google.com> wrote:
>>>>>>> >>>> > >>
>>>>>>> >>>> > >>> Unfortunate - I didn't realize that
>>>>>>> >>>> > >>> https://github.com/apache/beam/pull/15480 didn't make
>>>>>>> the cut.
>>>>>>> >>>> > >>>
>>>>>>> >>>> > >>> This bug was a regression in Beam 2.32.0, and is blocking
>>>>>>> multiple users
>>>>>>> >>>> > >>> from updating.
>>>>>>> >>>> > >>>
>>>>>>> >>>> > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi Meiri <
>>>>>>> udim@apache.org> wrote:
>>>>>>> >>>> > >>>
>>>>>>> >>>> > >>>> Hi everyone,
>>>>>>> >>>> > >>>> Please review and vote on the release candidate #1 for
>>>>>>> the version
>>>>>>> >>>> > >>>> 2.33.0, as follows:
>>>>>>> >>>> > >>>> [ ] +1, Approve the release
>>>>>>> >>>> > >>>> [ ] -1, Do not approve the release (please provide
>>>>>>> specific comments)
>>>>>>> >>>> > >>>>
>>>>>>> >>>> > >>>>
>>>>>>> >>>> > >>>> Reviewers are encouraged to test their own use cases
>>>>>>> with the release
>>>>>>> >>>> > >>>> candidate, and vote +1 if
>>>>>>> >>>> > >>>> no issues are found.
>>>>>>> >>>> > >>>>
>>>>>>> >>>> > >>>> The complete staging area is available for your review,
>>>>>>> which includes:
>>>>>>> >>>> > >>>> * JIRA release notes [1],
>>>>>>> >>>> > >>>> * the official Apache source release to be deployed to
>>>>>>> dist.apache.org
>>>>>>> >>>> > >>>> [2], which is signed with the key with fingerprint
>>>>>>> 587B049C36DAAFE6 [3],
>>>>>>> >>>> > >>>> * all artifacts to be deployed to the Maven Central
>>>>>>> Repository [4],
>>>>>>> >>>> > >>>> * source code tag "v2.33.0-RC1" [5],
>>>>>>> >>>> > >>>> * website pull request listing the release [6], the blog
>>>>>>> post [6], and
>>>>>>> >>>> > >>>> publishing the API reference manual [7].
>>>>>>> >>>> > >>>> * Java artifacts were built with Maven 3.6.3 and OpenJDK
>>>>>>> 1.8.0_181.
>>>>>>> >>>> > >>>> * Python artifacts are deployed along with the source
>>>>>>> release to the
>>>>>>> >>>> > >>>> dist.apache.org [2] and pypy[8].
>>>>>>> >>>> > >>>> * Validation sheet with a tab for 2.33.0 release to help
>>>>>>> with
>>>>>>> >>>> > >>>> validation [9].
>>>>>>> >>>> > >>>> * Docker images published to Docker Hub [10].
>>>>>>> >>>> > >>>>
>>>>>>> >>>> > >>>> The vote will be open for at least 72 hours. It is
>>>>>>> adopted by majority
>>>>>>> >>>> > >>>> approval, with at least 3 PMC affirmative votes.
>>>>>>> >>>> > >>>>
>>>>>>> >>>> > >>>> For guidelines on how to try the release in your
>>>>>>> projects, check out
>>>>>>> >>>> > >>>> our blog post at
>>>>>>> https://beam.apache.org/blog/validate-beam-release/.
>>>>>>> >>>> > >>>>
>>>>>>> >>>> > >>>> Thanks,
>>>>>>> >>>> > >>>> Release Manager
>>>>>>> >>>> > >>>>
>>>>>>> >>>> > >>>> [1]
>>>>>>> >>>> > >>>>
>>>>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
>>>>>>> >>>> > >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.33.0/
>>>>>>> >>>> > >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>>>>>> >>>> > >>>> [4]
>>>>>>> >>>> > >>>>
>>>>>>> https://repository.apache.org/content/repositories/orgapachebeam-1234/
>>>>>>> >>>> > >>>> [5] https://github.com/apache/beam/tree/v2.33.0-RC1
>>>>>>> >>>> > >>>> [6] https://github.com/apache/beam/pull/15543
>>>>>>> >>>> > >>>> [7] https://github.com/apache/beam-site/pull/619
>>>>>>> >>>> > >>>> [8] https://pypi.org/project/apache-beam/2.33.0rc1/
>>>>>>> >>>> > >>>> [9]
>>>>>>> >>>> > >>>>
>>>>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
>>>>>>> >>>> > >>>> [10]
>>>>>>> https://hub.docker.com/search?q=apache%2Fbeam&type=image
>>>>>>> >>>> > >>>>
>>>>>>> >>>> > >>>>
>>>>>>> >>>> >
>>>>>>> >>>
>>>>>>> >>>
>>>>>>> >>>
>>>>>>> >>
>>>>>>>
>>>>>>

Re: [VOTE] Release 2.33.0, release candidate 1

[Robert Burke <ro...@frantil.com>]

That's on the SDK side, and it just means the PCollection metrics are being
returned, buy not handled by the SDK. At present the pipeline results only
handle PTransform metrics.

As such it's not a regression, as adding those is still under development.

On Sun, Sep 26, 2021, 7:02 PM Daniel Oliveira <da...@google.com>
wrote:

> I tried validating wordcount with the Go SDK on Flink. The pipeline failed
> with a wall of errors like the following. I tried this on Flink 1.11, 1.12,
> and 1.13 job servers built from source at the RC1 commit, same errors on
> all of them.
>
> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n9"}
>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n10"}
>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n8"}
>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n7"}
>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>> urn:"beam:metric:sampled_byte_size:v1"
>>  type:"beam:metrics:distribution_int64:v1"
>>  payload:"\x8d%\xfa\xbf\x05\x04\xa8\x0c"  labels:{key:"PCOLLECTION"
>>  value:"n7"}
>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>> urn:"beam:metric:sampled_byte_size:v1"
>>  type:"beam:metrics:distribution_int64:v1"  payload:"\xb9\x05\xcf6\x05\x11"
>>  labels:{key:"PCOLLECTION"  value:"n9"}
>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>> urn:"beam:metric:sampled_byte_size:v1"
>>  type:"beam:metrics:distribution_int64:v1"  payload:"\xc5\x05\x8a1\x04\x12"
>>  labels:{key:"PCOLLECTION"  value:"n8"}
>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
>> urn:"beam:metric:sampled_byte_size:v1"
>>  type:"beam:metrics:distribution_int64:v1"
>>  payload:"\x8d%\xb3\xa7\x07\x14\""  labels:{key:"PCOLLECTION"  value:"n10"}
>> {...}
>
>
>  @Robert Burke <re...@google.com> I think you might know what's going on
> here. Is this solvable with a cherry-pick and a new RC?
>
> On Fri, Sep 24, 2021 at 4:25 PM Ahmet Altay <al...@google.com> wrote:
>
>>
>>
>> On Fri, Sep 24, 2021 at 4:21 PM Udi Meiri <eh...@google.com> wrote:
>>
>>>
>>>
>>> On Fri, Sep 24, 2021 at 4:18 PM Ahmet Altay <al...@google.com> wrote:
>>>
>>>> Thank you for the update related to allow_unsafe_triggers. My vote is
>>>> still a +1.
>>>>
>>>> If we decide to move forward with this RC, could you please include
>>>> this bug in the known issues list under the changes.md for this release?
>>>>
>>> Yes, that's included in https://github.com/apache/beam/pull/15543/files.
>>>
>>
>> Thank you!
>>
>>
>>>
>>>
>>>>
>>>> On Fri, Sep 24, 2021 at 3:56 PM Chamikara Jayalath <
>>>> chamikara@google.com> wrote:
>>>>
>>>>> +1 (binding)
>>>>> Validated a few scenarios from the spreadsheet.
>>>>>
>>>>> Thanks,
>>>>> Cham
>>>>>
>>>>> On Fri, Sep 24, 2021 at 3:07 PM Robert Bradshaw <ro...@google.com>
>>>>> wrote:
>>>>>
>>>>>> +1 (binding)
>>>>>>
>>>>>> All the artifacts and signatures look good.
>>>>>>
>>>>>> I don't think the unsafe trigger check is severe enough to block the
>>>>>> release.
>>>>>>
>>>>>> On Fri, Sep 24, 2021 at 2:36 PM Udi Meiri <eh...@google.com> wrote:
>>>>>> >
>>>>>> > Hi everyone, we found a bug during testing. It has to do with
>>>>>> Python SDK's allow_unsafe_triggers check.
>>>>>> > There is a preliminary fix that will go to master.
>>>>>> >
>>>>>> > For the 2.33.0 release, I'm leaning towards not making a new RC
>>>>>> since there is a workaround: pass the flag --allow_unsafe_triggers.
>>>>>> >
>>>>>> > Please reevaluate your votes accordingly and recast if you've
>>>>>> changed your vote.
>>>>>> >
>>>>>> > Thanks
>>>>>> >
>>>>>> > On Fri, Sep 24, 2021 at 12:48 PM Alexey Romanenko <
>>>>>> aromanenko.dev@gmail.com> wrote:
>>>>>> >>
>>>>>> >>
>>>>>> >> On 24 Sep 2021, at 20:45, Udi Meiri <eh...@google.com> wrote:
>>>>>> >> Alexey is this something that we should put in the release notes,
>>>>>> or some other change?
>>>>>> >>
>>>>>> >>
>>>>>> >> Yes, I think it could be helpful to mention that Beam Jackson’s
>>>>>> deps was bumped and it may require an update of Jackson’s runtime deps for
>>>>>> Spark 2 users as well.
>>>>>> >>
>>>>>> >> —
>>>>>> >> Alexey
>>>>>> >>
>>>>>> >>
>>>>>> >>
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> —
>>>>>> >>> Alexey
>>>>>> >>>
>>>>>> >>> On 24 Sep 2021, at 16:17, Alexey Romanenko <
>>>>>> aromanenko.dev@gmail.com> wrote:
>>>>>> >>>
>>>>>> >>> I checked with beam-samples [1] and noticed an issue to run some
>>>>>> pipelines with Spark 2 runner (Spark 3 seems is ok).
>>>>>> >>> It looks that it’s caused by new Jackson's version updated
>>>>>> recently [2], even if it’s a minor update but it works fine with a previous
>>>>>> one.
>>>>>> >>> I’ll try to find a workaround and get back with a results of this.
>>>>>> >>>
>>>>>> >>> [1] https://github.com/Talend/beam-samples/
>>>>>> >>> [2]
>>>>>> https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c
>>>>>> >>>
>>>>>> >>> —
>>>>>> >>> Alexey
>>>>>> >>>
>>>>>> >>> On 24 Sep 2021, at 11:17, Jan Lukavský <je...@seznam.cz> wrote:
>>>>>> >>>
>>>>>> >>> +1 (non-binding)
>>>>>> >>>
>>>>>> >>> Validated several use-cases using non-portable Flink with Java
>>>>>> SDK.
>>>>>> >>>
>>>>>> >>> On 9/24/21 4:55 AM, Valentyn Tymofieiev wrote:
>>>>>> >>>
>>>>>> >>> +1. Ran several Python batch and streaming pipelines on Dataflow
>>>>>> and checked that Dataflow containers have required dependencies of Beam.
>>>>>> >>>
>>>>>> >>> On Thu, Sep 23, 2021 at 7:03 PM Robert Burke <lo...@apache.org>
>>>>>> wrote:
>>>>>> >>>>
>>>>>> >>>> +1 (non-binding)
>>>>>> >>>> I validated the Go Quickstart (wordcount), and my ray tracer
>>>>>> against the Go Direct runner, Dataflow, and Spark (ensuring the rc1 tagged
>>>>>> container was used) and they executed successfully.
>>>>>> >>>>
>>>>>> >>>> I needed to manually synthesize a pseudo-version to ensure I was
>>>>>> using the tagged branch version (v2.0.0-20210914211513-b358127f9859)
>>>>>> instead of simply using v2.33.0-RC1.
>>>>>> >>>>
>>>>>> >>>>  It either can't find the package with the right tagged version,
>>>>>> or it can't find the version. It's not clear to me what the issue is, but
>>>>>> it's not notionally a release blocker. I'll investigate further once we
>>>>>> have a full release, as it's probably some unspecified behavior due to how
>>>>>> we transitioned to Go Modules (which strongly recommended doing a major
>>>>>> version bump for such transitions, which seems a bit excessive for Beam as
>>>>>> a whole...).
>>>>>> >>>>
>>>>>> >>>> On 2021/09/23 03:59:18, Ahmet Altay <al...@google.com> wrote:
>>>>>> >>>> > +1 on the RC. I validated python quick start examples on
>>>>>> direct runners.
>>>>>> >>>> > Thank you Udi.
>>>>>> >>>> >
>>>>>> >>>> > On Wed, Sep 22, 2021 at 2:20 PM Robert Burke <
>>>>>> robert@frantil.com> wrote:
>>>>>> >>>> >
>>>>>> >>>> > > Just an FYI that intend to validate the Go SDK for this
>>>>>> release but can't
>>>>>> >>>> > > get to it until tomorrow (Thursday). I'm catching up from a
>>>>>> week of
>>>>>> >>>> > > vacation. Apologies for the inconvenience.
>>>>>> >>>> > >
>>>>>> >>>> > > On Tue, Sep 21, 2021, 10:59 AM Udi Meiri <eh...@google.com>
>>>>>> wrote:
>>>>>> >>>> > >
>>>>>> >>>> > >> I updated the affected and fixed version fields for
>>>>>> >>>> > >> https://issues.apache.org/jira/browse/BEAM-12356.
>>>>>> >>>> > >>
>>>>>> >>>> > >>
>>>>>> >>>> > >> On Tue, Sep 21, 2021 at 10:48 AM Reuven Lax <
>>>>>> relax@google.com> wrote:
>>>>>> >>>> > >>
>>>>>> >>>> > >>> Unfortunate - I didn't realize that
>>>>>> >>>> > >>> https://github.com/apache/beam/pull/15480 didn't make the
>>>>>> cut.
>>>>>> >>>> > >>>
>>>>>> >>>> > >>> This bug was a regression in Beam 2.32.0, and is blocking
>>>>>> multiple users
>>>>>> >>>> > >>> from updating.
>>>>>> >>>> > >>>
>>>>>> >>>> > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi Meiri <
>>>>>> udim@apache.org> wrote:
>>>>>> >>>> > >>>
>>>>>> >>>> > >>>> Hi everyone,
>>>>>> >>>> > >>>> Please review and vote on the release candidate #1 for
>>>>>> the version
>>>>>> >>>> > >>>> 2.33.0, as follows:
>>>>>> >>>> > >>>> [ ] +1, Approve the release
>>>>>> >>>> > >>>> [ ] -1, Do not approve the release (please provide
>>>>>> specific comments)
>>>>>> >>>> > >>>>
>>>>>> >>>> > >>>>
>>>>>> >>>> > >>>> Reviewers are encouraged to test their own use cases with
>>>>>> the release
>>>>>> >>>> > >>>> candidate, and vote +1 if
>>>>>> >>>> > >>>> no issues are found.
>>>>>> >>>> > >>>>
>>>>>> >>>> > >>>> The complete staging area is available for your review,
>>>>>> which includes:
>>>>>> >>>> > >>>> * JIRA release notes [1],
>>>>>> >>>> > >>>> * the official Apache source release to be deployed to
>>>>>> dist.apache.org
>>>>>> >>>> > >>>> [2], which is signed with the key with fingerprint
>>>>>> 587B049C36DAAFE6 [3],
>>>>>> >>>> > >>>> * all artifacts to be deployed to the Maven Central
>>>>>> Repository [4],
>>>>>> >>>> > >>>> * source code tag "v2.33.0-RC1" [5],
>>>>>> >>>> > >>>> * website pull request listing the release [6], the blog
>>>>>> post [6], and
>>>>>> >>>> > >>>> publishing the API reference manual [7].
>>>>>> >>>> > >>>> * Java artifacts were built with Maven 3.6.3 and OpenJDK
>>>>>> 1.8.0_181.
>>>>>> >>>> > >>>> * Python artifacts are deployed along with the source
>>>>>> release to the
>>>>>> >>>> > >>>> dist.apache.org [2] and pypy[8].
>>>>>> >>>> > >>>> * Validation sheet with a tab for 2.33.0 release to help
>>>>>> with
>>>>>> >>>> > >>>> validation [9].
>>>>>> >>>> > >>>> * Docker images published to Docker Hub [10].
>>>>>> >>>> > >>>>
>>>>>> >>>> > >>>> The vote will be open for at least 72 hours. It is
>>>>>> adopted by majority
>>>>>> >>>> > >>>> approval, with at least 3 PMC affirmative votes.
>>>>>> >>>> > >>>>
>>>>>> >>>> > >>>> For guidelines on how to try the release in your
>>>>>> projects, check out
>>>>>> >>>> > >>>> our blog post at
>>>>>> https://beam.apache.org/blog/validate-beam-release/.
>>>>>> >>>> > >>>>
>>>>>> >>>> > >>>> Thanks,
>>>>>> >>>> > >>>> Release Manager
>>>>>> >>>> > >>>>
>>>>>> >>>> > >>>> [1]
>>>>>> >>>> > >>>>
>>>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
>>>>>> >>>> > >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.33.0/
>>>>>> >>>> > >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>>>>> >>>> > >>>> [4]
>>>>>> >>>> > >>>>
>>>>>> https://repository.apache.org/content/repositories/orgapachebeam-1234/
>>>>>> >>>> > >>>> [5] https://github.com/apache/beam/tree/v2.33.0-RC1
>>>>>> >>>> > >>>> [6] https://github.com/apache/beam/pull/15543
>>>>>> >>>> > >>>> [7] https://github.com/apache/beam-site/pull/619
>>>>>> >>>> > >>>> [8] https://pypi.org/project/apache-beam/2.33.0rc1/
>>>>>> >>>> > >>>> [9]
>>>>>> >>>> > >>>>
>>>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
>>>>>> >>>> > >>>> [10]
>>>>>> https://hub.docker.com/search?q=apache%2Fbeam&type=image
>>>>>> >>>> > >>>>
>>>>>> >>>> > >>>>
>>>>>> >>>> >
>>>>>> >>>
>>>>>> >>>
>>>>>> >>>
>>>>>> >>
>>>>>>
>>>>>

Re: [VOTE] Release 2.33.0, release candidate 1

[Daniel Oliveira <da...@google.com>]

I tried validating wordcount with the Go SDK on Flink. The pipeline failed
with a wall of errors like the following. I tried this on Flink 1.11, 1.12,
and 1.13 job servers built from source at the RC1 commit, same errors on
all of them.

2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n9"}
> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n10"}
> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n8"}
> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
> urn:"beam:metric:element_count:v1"  type:"beam:metrics:sum_int64:v1"
>  payload:"\x8d%"  labels:{key:"PCOLLECTION"  value:"n7"}
> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
> urn:"beam:metric:sampled_byte_size:v1"
>  type:"beam:metrics:distribution_int64:v1"
>  payload:"\x8d%\xfa\xbf\x05\x04\xa8\x0c"  labels:{key:"PCOLLECTION"
>  value:"n7"}
> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
> urn:"beam:metric:sampled_byte_size:v1"
>  type:"beam:metrics:distribution_int64:v1"  payload:"\xb9\x05\xcf6\x05\x11"
>  labels:{key:"PCOLLECTION"  value:"n9"}
> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
> urn:"beam:metric:sampled_byte_size:v1"
>  type:"beam:metrics:distribution_int64:v1"  payload:"\xc5\x05\x8a1\x04\x12"
>  labels:{key:"PCOLLECTION"  value:"n8"}
> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo:
> urn:"beam:metric:sampled_byte_size:v1"
>  type:"beam:metrics:distribution_int64:v1"
>  payload:"\x8d%\xb3\xa7\x07\x14\""  labels:{key:"PCOLLECTION"  value:"n10"}
> {...}


 @Robert Burke <re...@google.com> I think you might know what's going on
here. Is this solvable with a cherry-pick and a new RC?

On Fri, Sep 24, 2021 at 4:25 PM Ahmet Altay <al...@google.com> wrote:

>
>
> On Fri, Sep 24, 2021 at 4:21 PM Udi Meiri <eh...@google.com> wrote:
>
>>
>>
>> On Fri, Sep 24, 2021 at 4:18 PM Ahmet Altay <al...@google.com> wrote:
>>
>>> Thank you for the update related to allow_unsafe_triggers. My vote is
>>> still a +1.
>>>
>>> If we decide to move forward with this RC, could you please include this
>>> bug in the known issues list under the changes.md for this release?
>>>
>> Yes, that's included in https://github.com/apache/beam/pull/15543/files.
>>
>
> Thank you!
>
>
>>
>>
>>>
>>> On Fri, Sep 24, 2021 at 3:56 PM Chamikara Jayalath <ch...@google.com>
>>> wrote:
>>>
>>>> +1 (binding)
>>>> Validated a few scenarios from the spreadsheet.
>>>>
>>>> Thanks,
>>>> Cham
>>>>
>>>> On Fri, Sep 24, 2021 at 3:07 PM Robert Bradshaw <ro...@google.com>
>>>> wrote:
>>>>
>>>>> +1 (binding)
>>>>>
>>>>> All the artifacts and signatures look good.
>>>>>
>>>>> I don't think the unsafe trigger check is severe enough to block the
>>>>> release.
>>>>>
>>>>> On Fri, Sep 24, 2021 at 2:36 PM Udi Meiri <eh...@google.com> wrote:
>>>>> >
>>>>> > Hi everyone, we found a bug during testing. It has to do with Python
>>>>> SDK's allow_unsafe_triggers check.
>>>>> > There is a preliminary fix that will go to master.
>>>>> >
>>>>> > For the 2.33.0 release, I'm leaning towards not making a new RC
>>>>> since there is a workaround: pass the flag --allow_unsafe_triggers.
>>>>> >
>>>>> > Please reevaluate your votes accordingly and recast if you've
>>>>> changed your vote.
>>>>> >
>>>>> > Thanks
>>>>> >
>>>>> > On Fri, Sep 24, 2021 at 12:48 PM Alexey Romanenko <
>>>>> aromanenko.dev@gmail.com> wrote:
>>>>> >>
>>>>> >>
>>>>> >> On 24 Sep 2021, at 20:45, Udi Meiri <eh...@google.com> wrote:
>>>>> >> Alexey is this something that we should put in the release notes,
>>>>> or some other change?
>>>>> >>
>>>>> >>
>>>>> >> Yes, I think it could be helpful to mention that Beam Jackson’s
>>>>> deps was bumped and it may require an update of Jackson’s runtime deps for
>>>>> Spark 2 users as well.
>>>>> >>
>>>>> >> —
>>>>> >> Alexey
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >>>
>>>>> >>>
>>>>> >>> —
>>>>> >>> Alexey
>>>>> >>>
>>>>> >>> On 24 Sep 2021, at 16:17, Alexey Romanenko <
>>>>> aromanenko.dev@gmail.com> wrote:
>>>>> >>>
>>>>> >>> I checked with beam-samples [1] and noticed an issue to run some
>>>>> pipelines with Spark 2 runner (Spark 3 seems is ok).
>>>>> >>> It looks that it’s caused by new Jackson's version updated
>>>>> recently [2], even if it’s a minor update but it works fine with a previous
>>>>> one.
>>>>> >>> I’ll try to find a workaround and get back with a results of this.
>>>>> >>>
>>>>> >>> [1] https://github.com/Talend/beam-samples/
>>>>> >>> [2]
>>>>> https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c
>>>>> >>>
>>>>> >>> —
>>>>> >>> Alexey
>>>>> >>>
>>>>> >>> On 24 Sep 2021, at 11:17, Jan Lukavský <je...@seznam.cz> wrote:
>>>>> >>>
>>>>> >>> +1 (non-binding)
>>>>> >>>
>>>>> >>> Validated several use-cases using non-portable Flink with Java SDK.
>>>>> >>>
>>>>> >>> On 9/24/21 4:55 AM, Valentyn Tymofieiev wrote:
>>>>> >>>
>>>>> >>> +1. Ran several Python batch and streaming pipelines on Dataflow
>>>>> and checked that Dataflow containers have required dependencies of Beam.
>>>>> >>>
>>>>> >>> On Thu, Sep 23, 2021 at 7:03 PM Robert Burke <lo...@apache.org>
>>>>> wrote:
>>>>> >>>>
>>>>> >>>> +1 (non-binding)
>>>>> >>>> I validated the Go Quickstart (wordcount), and my ray tracer
>>>>> against the Go Direct runner, Dataflow, and Spark (ensuring the rc1 tagged
>>>>> container was used) and they executed successfully.
>>>>> >>>>
>>>>> >>>> I needed to manually synthesize a pseudo-version to ensure I was
>>>>> using the tagged branch version (v2.0.0-20210914211513-b358127f9859)
>>>>> instead of simply using v2.33.0-RC1.
>>>>> >>>>
>>>>> >>>>  It either can't find the package with the right tagged version,
>>>>> or it can't find the version. It's not clear to me what the issue is, but
>>>>> it's not notionally a release blocker. I'll investigate further once we
>>>>> have a full release, as it's probably some unspecified behavior due to how
>>>>> we transitioned to Go Modules (which strongly recommended doing a major
>>>>> version bump for such transitions, which seems a bit excessive for Beam as
>>>>> a whole...).
>>>>> >>>>
>>>>> >>>> On 2021/09/23 03:59:18, Ahmet Altay <al...@google.com> wrote:
>>>>> >>>> > +1 on the RC. I validated python quick start examples on direct
>>>>> runners.
>>>>> >>>> > Thank you Udi.
>>>>> >>>> >
>>>>> >>>> > On Wed, Sep 22, 2021 at 2:20 PM Robert Burke <
>>>>> robert@frantil.com> wrote:
>>>>> >>>> >
>>>>> >>>> > > Just an FYI that intend to validate the Go SDK for this
>>>>> release but can't
>>>>> >>>> > > get to it until tomorrow (Thursday). I'm catching up from a
>>>>> week of
>>>>> >>>> > > vacation. Apologies for the inconvenience.
>>>>> >>>> > >
>>>>> >>>> > > On Tue, Sep 21, 2021, 10:59 AM Udi Meiri <eh...@google.com>
>>>>> wrote:
>>>>> >>>> > >
>>>>> >>>> > >> I updated the affected and fixed version fields for
>>>>> >>>> > >> https://issues.apache.org/jira/browse/BEAM-12356.
>>>>> >>>> > >>
>>>>> >>>> > >>
>>>>> >>>> > >> On Tue, Sep 21, 2021 at 10:48 AM Reuven Lax <
>>>>> relax@google.com> wrote:
>>>>> >>>> > >>
>>>>> >>>> > >>> Unfortunate - I didn't realize that
>>>>> >>>> > >>> https://github.com/apache/beam/pull/15480 didn't make the
>>>>> cut.
>>>>> >>>> > >>>
>>>>> >>>> > >>> This bug was a regression in Beam 2.32.0, and is blocking
>>>>> multiple users
>>>>> >>>> > >>> from updating.
>>>>> >>>> > >>>
>>>>> >>>> > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi Meiri <ud...@apache.org>
>>>>> wrote:
>>>>> >>>> > >>>
>>>>> >>>> > >>>> Hi everyone,
>>>>> >>>> > >>>> Please review and vote on the release candidate #1 for the
>>>>> version
>>>>> >>>> > >>>> 2.33.0, as follows:
>>>>> >>>> > >>>> [ ] +1, Approve the release
>>>>> >>>> > >>>> [ ] -1, Do not approve the release (please provide
>>>>> specific comments)
>>>>> >>>> > >>>>
>>>>> >>>> > >>>>
>>>>> >>>> > >>>> Reviewers are encouraged to test their own use cases with
>>>>> the release
>>>>> >>>> > >>>> candidate, and vote +1 if
>>>>> >>>> > >>>> no issues are found.
>>>>> >>>> > >>>>
>>>>> >>>> > >>>> The complete staging area is available for your review,
>>>>> which includes:
>>>>> >>>> > >>>> * JIRA release notes [1],
>>>>> >>>> > >>>> * the official Apache source release to be deployed to
>>>>> dist.apache.org
>>>>> >>>> > >>>> [2], which is signed with the key with fingerprint
>>>>> 587B049C36DAAFE6 [3],
>>>>> >>>> > >>>> * all artifacts to be deployed to the Maven Central
>>>>> Repository [4],
>>>>> >>>> > >>>> * source code tag "v2.33.0-RC1" [5],
>>>>> >>>> > >>>> * website pull request listing the release [6], the blog
>>>>> post [6], and
>>>>> >>>> > >>>> publishing the API reference manual [7].
>>>>> >>>> > >>>> * Java artifacts were built with Maven 3.6.3 and OpenJDK
>>>>> 1.8.0_181.
>>>>> >>>> > >>>> * Python artifacts are deployed along with the source
>>>>> release to the
>>>>> >>>> > >>>> dist.apache.org [2] and pypy[8].
>>>>> >>>> > >>>> * Validation sheet with a tab for 2.33.0 release to help
>>>>> with
>>>>> >>>> > >>>> validation [9].
>>>>> >>>> > >>>> * Docker images published to Docker Hub [10].
>>>>> >>>> > >>>>
>>>>> >>>> > >>>> The vote will be open for at least 72 hours. It is adopted
>>>>> by majority
>>>>> >>>> > >>>> approval, with at least 3 PMC affirmative votes.
>>>>> >>>> > >>>>
>>>>> >>>> > >>>> For guidelines on how to try the release in your projects,
>>>>> check out
>>>>> >>>> > >>>> our blog post at
>>>>> https://beam.apache.org/blog/validate-beam-release/.
>>>>> >>>> > >>>>
>>>>> >>>> > >>>> Thanks,
>>>>> >>>> > >>>> Release Manager
>>>>> >>>> > >>>>
>>>>> >>>> > >>>> [1]
>>>>> >>>> > >>>>
>>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
>>>>> >>>> > >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.33.0/
>>>>> >>>> > >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>>>> >>>> > >>>> [4]
>>>>> >>>> > >>>>
>>>>> https://repository.apache.org/content/repositories/orgapachebeam-1234/
>>>>> >>>> > >>>> [5] https://github.com/apache/beam/tree/v2.33.0-RC1
>>>>> >>>> > >>>> [6] https://github.com/apache/beam/pull/15543
>>>>> >>>> > >>>> [7] https://github.com/apache/beam-site/pull/619
>>>>> >>>> > >>>> [8] https://pypi.org/project/apache-beam/2.33.0rc1/
>>>>> >>>> > >>>> [9]
>>>>> >>>> > >>>>
>>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
>>>>> >>>> > >>>> [10]
>>>>> https://hub.docker.com/search?q=apache%2Fbeam&type=image
>>>>> >>>> > >>>>
>>>>> >>>> > >>>>
>>>>> >>>> >
>>>>> >>>
>>>>> >>>
>>>>> >>>
>>>>> >>
>>>>>
>>>>

Re: [VOTE] Release 2.33.0, release candidate 1

[Ahmet Altay <al...@google.com>]

On Fri, Sep 24, 2021 at 4:21 PM Udi Meiri <eh...@google.com> wrote:

>
>
> On Fri, Sep 24, 2021 at 4:18 PM Ahmet Altay <al...@google.com> wrote:
>
>> Thank you for the update related to allow_unsafe_triggers. My vote is
>> still a +1.
>>
>> If we decide to move forward with this RC, could you please include this
>> bug in the known issues list under the changes.md for this release?
>>
> Yes, that's included in https://github.com/apache/beam/pull/15543/files.
>

Thank you!


>
>
>>
>> On Fri, Sep 24, 2021 at 3:56 PM Chamikara Jayalath <ch...@google.com>
>> wrote:
>>
>>> +1 (binding)
>>> Validated a few scenarios from the spreadsheet.
>>>
>>> Thanks,
>>> Cham
>>>
>>> On Fri, Sep 24, 2021 at 3:07 PM Robert Bradshaw <ro...@google.com>
>>> wrote:
>>>
>>>> +1 (binding)
>>>>
>>>> All the artifacts and signatures look good.
>>>>
>>>> I don't think the unsafe trigger check is severe enough to block the
>>>> release.
>>>>
>>>> On Fri, Sep 24, 2021 at 2:36 PM Udi Meiri <eh...@google.com> wrote:
>>>> >
>>>> > Hi everyone, we found a bug during testing. It has to do with Python
>>>> SDK's allow_unsafe_triggers check.
>>>> > There is a preliminary fix that will go to master.
>>>> >
>>>> > For the 2.33.0 release, I'm leaning towards not making a new RC since
>>>> there is a workaround: pass the flag --allow_unsafe_triggers.
>>>> >
>>>> > Please reevaluate your votes accordingly and recast if you've changed
>>>> your vote.
>>>> >
>>>> > Thanks
>>>> >
>>>> > On Fri, Sep 24, 2021 at 12:48 PM Alexey Romanenko <
>>>> aromanenko.dev@gmail.com> wrote:
>>>> >>
>>>> >>
>>>> >> On 24 Sep 2021, at 20:45, Udi Meiri <eh...@google.com> wrote:
>>>> >> Alexey is this something that we should put in the release notes, or
>>>> some other change?
>>>> >>
>>>> >>
>>>> >> Yes, I think it could be helpful to mention that Beam Jackson’s deps
>>>> was bumped and it may require an update of Jackson’s runtime deps for Spark
>>>> 2 users as well.
>>>> >>
>>>> >> —
>>>> >> Alexey
>>>> >>
>>>> >>
>>>> >>
>>>> >>>
>>>> >>>
>>>> >>> —
>>>> >>> Alexey
>>>> >>>
>>>> >>> On 24 Sep 2021, at 16:17, Alexey Romanenko <
>>>> aromanenko.dev@gmail.com> wrote:
>>>> >>>
>>>> >>> I checked with beam-samples [1] and noticed an issue to run some
>>>> pipelines with Spark 2 runner (Spark 3 seems is ok).
>>>> >>> It looks that it’s caused by new Jackson's version updated recently
>>>> [2], even if it’s a minor update but it works fine with a previous one.
>>>> >>> I’ll try to find a workaround and get back with a results of this.
>>>> >>>
>>>> >>> [1] https://github.com/Talend/beam-samples/
>>>> >>> [2]
>>>> https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c
>>>> >>>
>>>> >>> —
>>>> >>> Alexey
>>>> >>>
>>>> >>> On 24 Sep 2021, at 11:17, Jan Lukavský <je...@seznam.cz> wrote:
>>>> >>>
>>>> >>> +1 (non-binding)
>>>> >>>
>>>> >>> Validated several use-cases using non-portable Flink with Java SDK.
>>>> >>>
>>>> >>> On 9/24/21 4:55 AM, Valentyn Tymofieiev wrote:
>>>> >>>
>>>> >>> +1. Ran several Python batch and streaming pipelines on Dataflow
>>>> and checked that Dataflow containers have required dependencies of Beam.
>>>> >>>
>>>> >>> On Thu, Sep 23, 2021 at 7:03 PM Robert Burke <lo...@apache.org>
>>>> wrote:
>>>> >>>>
>>>> >>>> +1 (non-binding)
>>>> >>>> I validated the Go Quickstart (wordcount), and my ray tracer
>>>> against the Go Direct runner, Dataflow, and Spark (ensuring the rc1 tagged
>>>> container was used) and they executed successfully.
>>>> >>>>
>>>> >>>> I needed to manually synthesize a pseudo-version to ensure I was
>>>> using the tagged branch version (v2.0.0-20210914211513-b358127f9859)
>>>> instead of simply using v2.33.0-RC1.
>>>> >>>>
>>>> >>>>  It either can't find the package with the right tagged version,
>>>> or it can't find the version. It's not clear to me what the issue is, but
>>>> it's not notionally a release blocker. I'll investigate further once we
>>>> have a full release, as it's probably some unspecified behavior due to how
>>>> we transitioned to Go Modules (which strongly recommended doing a major
>>>> version bump for such transitions, which seems a bit excessive for Beam as
>>>> a whole...).
>>>> >>>>
>>>> >>>> On 2021/09/23 03:59:18, Ahmet Altay <al...@google.com> wrote:
>>>> >>>> > +1 on the RC. I validated python quick start examples on direct
>>>> runners.
>>>> >>>> > Thank you Udi.
>>>> >>>> >
>>>> >>>> > On Wed, Sep 22, 2021 at 2:20 PM Robert Burke <ro...@frantil.com>
>>>> wrote:
>>>> >>>> >
>>>> >>>> > > Just an FYI that intend to validate the Go SDK for this
>>>> release but can't
>>>> >>>> > > get to it until tomorrow (Thursday). I'm catching up from a
>>>> week of
>>>> >>>> > > vacation. Apologies for the inconvenience.
>>>> >>>> > >
>>>> >>>> > > On Tue, Sep 21, 2021, 10:59 AM Udi Meiri <eh...@google.com>
>>>> wrote:
>>>> >>>> > >
>>>> >>>> > >> I updated the affected and fixed version fields for
>>>> >>>> > >> https://issues.apache.org/jira/browse/BEAM-12356.
>>>> >>>> > >>
>>>> >>>> > >>
>>>> >>>> > >> On Tue, Sep 21, 2021 at 10:48 AM Reuven Lax <re...@google.com>
>>>> wrote:
>>>> >>>> > >>
>>>> >>>> > >>> Unfortunate - I didn't realize that
>>>> >>>> > >>> https://github.com/apache/beam/pull/15480 didn't make the
>>>> cut.
>>>> >>>> > >>>
>>>> >>>> > >>> This bug was a regression in Beam 2.32.0, and is blocking
>>>> multiple users
>>>> >>>> > >>> from updating.
>>>> >>>> > >>>
>>>> >>>> > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi Meiri <ud...@apache.org>
>>>> wrote:
>>>> >>>> > >>>
>>>> >>>> > >>>> Hi everyone,
>>>> >>>> > >>>> Please review and vote on the release candidate #1 for the
>>>> version
>>>> >>>> > >>>> 2.33.0, as follows:
>>>> >>>> > >>>> [ ] +1, Approve the release
>>>> >>>> > >>>> [ ] -1, Do not approve the release (please provide specific
>>>> comments)
>>>> >>>> > >>>>
>>>> >>>> > >>>>
>>>> >>>> > >>>> Reviewers are encouraged to test their own use cases with
>>>> the release
>>>> >>>> > >>>> candidate, and vote +1 if
>>>> >>>> > >>>> no issues are found.
>>>> >>>> > >>>>
>>>> >>>> > >>>> The complete staging area is available for your review,
>>>> which includes:
>>>> >>>> > >>>> * JIRA release notes [1],
>>>> >>>> > >>>> * the official Apache source release to be deployed to
>>>> dist.apache.org
>>>> >>>> > >>>> [2], which is signed with the key with fingerprint
>>>> 587B049C36DAAFE6 [3],
>>>> >>>> > >>>> * all artifacts to be deployed to the Maven Central
>>>> Repository [4],
>>>> >>>> > >>>> * source code tag "v2.33.0-RC1" [5],
>>>> >>>> > >>>> * website pull request listing the release [6], the blog
>>>> post [6], and
>>>> >>>> > >>>> publishing the API reference manual [7].
>>>> >>>> > >>>> * Java artifacts were built with Maven 3.6.3 and OpenJDK
>>>> 1.8.0_181.
>>>> >>>> > >>>> * Python artifacts are deployed along with the source
>>>> release to the
>>>> >>>> > >>>> dist.apache.org [2] and pypy[8].
>>>> >>>> > >>>> * Validation sheet with a tab for 2.33.0 release to help
>>>> with
>>>> >>>> > >>>> validation [9].
>>>> >>>> > >>>> * Docker images published to Docker Hub [10].
>>>> >>>> > >>>>
>>>> >>>> > >>>> The vote will be open for at least 72 hours. It is adopted
>>>> by majority
>>>> >>>> > >>>> approval, with at least 3 PMC affirmative votes.
>>>> >>>> > >>>>
>>>> >>>> > >>>> For guidelines on how to try the release in your projects,
>>>> check out
>>>> >>>> > >>>> our blog post at
>>>> https://beam.apache.org/blog/validate-beam-release/.
>>>> >>>> > >>>>
>>>> >>>> > >>>> Thanks,
>>>> >>>> > >>>> Release Manager
>>>> >>>> > >>>>
>>>> >>>> > >>>> [1]
>>>> >>>> > >>>>
>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
>>>> >>>> > >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.33.0/
>>>> >>>> > >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>>> >>>> > >>>> [4]
>>>> >>>> > >>>>
>>>> https://repository.apache.org/content/repositories/orgapachebeam-1234/
>>>> >>>> > >>>> [5] https://github.com/apache/beam/tree/v2.33.0-RC1
>>>> >>>> > >>>> [6] https://github.com/apache/beam/pull/15543
>>>> >>>> > >>>> [7] https://github.com/apache/beam-site/pull/619
>>>> >>>> > >>>> [8] https://pypi.org/project/apache-beam/2.33.0rc1/
>>>> >>>> > >>>> [9]
>>>> >>>> > >>>>
>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
>>>> >>>> > >>>> [10]
>>>> https://hub.docker.com/search?q=apache%2Fbeam&type=image
>>>> >>>> > >>>>
>>>> >>>> > >>>>
>>>> >>>> >
>>>> >>>
>>>> >>>
>>>> >>>
>>>> >>
>>>>
>>>

Re: [VOTE] Release 2.33.0, release candidate 1

[Udi Meiri <eh...@google.com>]

On Fri, Sep 24, 2021 at 4:18 PM Ahmet Altay <al...@google.com> wrote:

> Thank you for the update related to allow_unsafe_triggers. My vote is
> still a +1.
>
> If we decide to move forward with this RC, could you please include this
> bug in the known issues list under the changes.md for this release?
>
Yes, that's included in https://github.com/apache/beam/pull/15543/files.


>
> On Fri, Sep 24, 2021 at 3:56 PM Chamikara Jayalath <ch...@google.com>
> wrote:
>
>> +1 (binding)
>> Validated a few scenarios from the spreadsheet.
>>
>> Thanks,
>> Cham
>>
>> On Fri, Sep 24, 2021 at 3:07 PM Robert Bradshaw <ro...@google.com>
>> wrote:
>>
>>> +1 (binding)
>>>
>>> All the artifacts and signatures look good.
>>>
>>> I don't think the unsafe trigger check is severe enough to block the
>>> release.
>>>
>>> On Fri, Sep 24, 2021 at 2:36 PM Udi Meiri <eh...@google.com> wrote:
>>> >
>>> > Hi everyone, we found a bug during testing. It has to do with Python
>>> SDK's allow_unsafe_triggers check.
>>> > There is a preliminary fix that will go to master.
>>> >
>>> > For the 2.33.0 release, I'm leaning towards not making a new RC since
>>> there is a workaround: pass the flag --allow_unsafe_triggers.
>>> >
>>> > Please reevaluate your votes accordingly and recast if you've changed
>>> your vote.
>>> >
>>> > Thanks
>>> >
>>> > On Fri, Sep 24, 2021 at 12:48 PM Alexey Romanenko <
>>> aromanenko.dev@gmail.com> wrote:
>>> >>
>>> >>
>>> >> On 24 Sep 2021, at 20:45, Udi Meiri <eh...@google.com> wrote:
>>> >> Alexey is this something that we should put in the release notes, or
>>> some other change?
>>> >>
>>> >>
>>> >> Yes, I think it could be helpful to mention that Beam Jackson’s deps
>>> was bumped and it may require an update of Jackson’s runtime deps for Spark
>>> 2 users as well.
>>> >>
>>> >> —
>>> >> Alexey
>>> >>
>>> >>
>>> >>
>>> >>>
>>> >>>
>>> >>> —
>>> >>> Alexey
>>> >>>
>>> >>> On 24 Sep 2021, at 16:17, Alexey Romanenko <ar...@gmail.com>
>>> wrote:
>>> >>>
>>> >>> I checked with beam-samples [1] and noticed an issue to run some
>>> pipelines with Spark 2 runner (Spark 3 seems is ok).
>>> >>> It looks that it’s caused by new Jackson's version updated recently
>>> [2], even if it’s a minor update but it works fine with a previous one.
>>> >>> I’ll try to find a workaround and get back with a results of this.
>>> >>>
>>> >>> [1] https://github.com/Talend/beam-samples/
>>> >>> [2]
>>> https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c
>>> >>>
>>> >>> —
>>> >>> Alexey
>>> >>>
>>> >>> On 24 Sep 2021, at 11:17, Jan Lukavský <je...@seznam.cz> wrote:
>>> >>>
>>> >>> +1 (non-binding)
>>> >>>
>>> >>> Validated several use-cases using non-portable Flink with Java SDK.
>>> >>>
>>> >>> On 9/24/21 4:55 AM, Valentyn Tymofieiev wrote:
>>> >>>
>>> >>> +1. Ran several Python batch and streaming pipelines on Dataflow and
>>> checked that Dataflow containers have required dependencies of Beam.
>>> >>>
>>> >>> On Thu, Sep 23, 2021 at 7:03 PM Robert Burke <lo...@apache.org>
>>> wrote:
>>> >>>>
>>> >>>> +1 (non-binding)
>>> >>>> I validated the Go Quickstart (wordcount), and my ray tracer
>>> against the Go Direct runner, Dataflow, and Spark (ensuring the rc1 tagged
>>> container was used) and they executed successfully.
>>> >>>>
>>> >>>> I needed to manually synthesize a pseudo-version to ensure I was
>>> using the tagged branch version (v2.0.0-20210914211513-b358127f9859)
>>> instead of simply using v2.33.0-RC1.
>>> >>>>
>>> >>>>  It either can't find the package with the right tagged version, or
>>> it can't find the version. It's not clear to me what the issue is, but it's
>>> not notionally a release blocker. I'll investigate further once we have a
>>> full release, as it's probably some unspecified behavior due to how we
>>> transitioned to Go Modules (which strongly recommended doing a major
>>> version bump for such transitions, which seems a bit excessive for Beam as
>>> a whole...).
>>> >>>>
>>> >>>> On 2021/09/23 03:59:18, Ahmet Altay <al...@google.com> wrote:
>>> >>>> > +1 on the RC. I validated python quick start examples on direct
>>> runners.
>>> >>>> > Thank you Udi.
>>> >>>> >
>>> >>>> > On Wed, Sep 22, 2021 at 2:20 PM Robert Burke <ro...@frantil.com>
>>> wrote:
>>> >>>> >
>>> >>>> > > Just an FYI that intend to validate the Go SDK for this release
>>> but can't
>>> >>>> > > get to it until tomorrow (Thursday). I'm catching up from a
>>> week of
>>> >>>> > > vacation. Apologies for the inconvenience.
>>> >>>> > >
>>> >>>> > > On Tue, Sep 21, 2021, 10:59 AM Udi Meiri <eh...@google.com>
>>> wrote:
>>> >>>> > >
>>> >>>> > >> I updated the affected and fixed version fields for
>>> >>>> > >> https://issues.apache.org/jira/browse/BEAM-12356.
>>> >>>> > >>
>>> >>>> > >>
>>> >>>> > >> On Tue, Sep 21, 2021 at 10:48 AM Reuven Lax <re...@google.com>
>>> wrote:
>>> >>>> > >>
>>> >>>> > >>> Unfortunate - I didn't realize that
>>> >>>> > >>> https://github.com/apache/beam/pull/15480 didn't make the
>>> cut.
>>> >>>> > >>>
>>> >>>> > >>> This bug was a regression in Beam 2.32.0, and is blocking
>>> multiple users
>>> >>>> > >>> from updating.
>>> >>>> > >>>
>>> >>>> > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi Meiri <ud...@apache.org>
>>> wrote:
>>> >>>> > >>>
>>> >>>> > >>>> Hi everyone,
>>> >>>> > >>>> Please review and vote on the release candidate #1 for the
>>> version
>>> >>>> > >>>> 2.33.0, as follows:
>>> >>>> > >>>> [ ] +1, Approve the release
>>> >>>> > >>>> [ ] -1, Do not approve the release (please provide specific
>>> comments)
>>> >>>> > >>>>
>>> >>>> > >>>>
>>> >>>> > >>>> Reviewers are encouraged to test their own use cases with
>>> the release
>>> >>>> > >>>> candidate, and vote +1 if
>>> >>>> > >>>> no issues are found.
>>> >>>> > >>>>
>>> >>>> > >>>> The complete staging area is available for your review,
>>> which includes:
>>> >>>> > >>>> * JIRA release notes [1],
>>> >>>> > >>>> * the official Apache source release to be deployed to
>>> dist.apache.org
>>> >>>> > >>>> [2], which is signed with the key with fingerprint
>>> 587B049C36DAAFE6 [3],
>>> >>>> > >>>> * all artifacts to be deployed to the Maven Central
>>> Repository [4],
>>> >>>> > >>>> * source code tag "v2.33.0-RC1" [5],
>>> >>>> > >>>> * website pull request listing the release [6], the blog
>>> post [6], and
>>> >>>> > >>>> publishing the API reference manual [7].
>>> >>>> > >>>> * Java artifacts were built with Maven 3.6.3 and OpenJDK
>>> 1.8.0_181.
>>> >>>> > >>>> * Python artifacts are deployed along with the source
>>> release to the
>>> >>>> > >>>> dist.apache.org [2] and pypy[8].
>>> >>>> > >>>> * Validation sheet with a tab for 2.33.0 release to help with
>>> >>>> > >>>> validation [9].
>>> >>>> > >>>> * Docker images published to Docker Hub [10].
>>> >>>> > >>>>
>>> >>>> > >>>> The vote will be open for at least 72 hours. It is adopted
>>> by majority
>>> >>>> > >>>> approval, with at least 3 PMC affirmative votes.
>>> >>>> > >>>>
>>> >>>> > >>>> For guidelines on how to try the release in your projects,
>>> check out
>>> >>>> > >>>> our blog post at
>>> https://beam.apache.org/blog/validate-beam-release/.
>>> >>>> > >>>>
>>> >>>> > >>>> Thanks,
>>> >>>> > >>>> Release Manager
>>> >>>> > >>>>
>>> >>>> > >>>> [1]
>>> >>>> > >>>>
>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
>>> >>>> > >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.33.0/
>>> >>>> > >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>> >>>> > >>>> [4]
>>> >>>> > >>>>
>>> https://repository.apache.org/content/repositories/orgapachebeam-1234/
>>> >>>> > >>>> [5] https://github.com/apache/beam/tree/v2.33.0-RC1
>>> >>>> > >>>> [6] https://github.com/apache/beam/pull/15543
>>> >>>> > >>>> [7] https://github.com/apache/beam-site/pull/619
>>> >>>> > >>>> [8] https://pypi.org/project/apache-beam/2.33.0rc1/
>>> >>>> > >>>> [9]
>>> >>>> > >>>>
>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
>>> >>>> > >>>> [10]
>>> https://hub.docker.com/search?q=apache%2Fbeam&type=image
>>> >>>> > >>>>
>>> >>>> > >>>>
>>> >>>> >
>>> >>>
>>> >>>
>>> >>>
>>> >>
>>>
>>

Re: [VOTE] Release 2.33.0, release candidate 1

[Ahmet Altay <al...@google.com>]

Thank you for the update related to allow_unsafe_triggers. My vote is still
a +1.

If we decide to move forward with this RC, could you please include this
bug in the known issues list under the changes.md for this release?

On Fri, Sep 24, 2021 at 3:56 PM Chamikara Jayalath <ch...@google.com>
wrote:

> +1 (binding)
> Validated a few scenarios from the spreadsheet.
>
> Thanks,
> Cham
>
> On Fri, Sep 24, 2021 at 3:07 PM Robert Bradshaw <ro...@google.com>
> wrote:
>
>> +1 (binding)
>>
>> All the artifacts and signatures look good.
>>
>> I don't think the unsafe trigger check is severe enough to block the
>> release.
>>
>> On Fri, Sep 24, 2021 at 2:36 PM Udi Meiri <eh...@google.com> wrote:
>> >
>> > Hi everyone, we found a bug during testing. It has to do with Python
>> SDK's allow_unsafe_triggers check.
>> > There is a preliminary fix that will go to master.
>> >
>> > For the 2.33.0 release, I'm leaning towards not making a new RC since
>> there is a workaround: pass the flag --allow_unsafe_triggers.
>> >
>> > Please reevaluate your votes accordingly and recast if you've changed
>> your vote.
>> >
>> > Thanks
>> >
>> > On Fri, Sep 24, 2021 at 12:48 PM Alexey Romanenko <
>> aromanenko.dev@gmail.com> wrote:
>> >>
>> >>
>> >> On 24 Sep 2021, at 20:45, Udi Meiri <eh...@google.com> wrote:
>> >> Alexey is this something that we should put in the release notes, or
>> some other change?
>> >>
>> >>
>> >> Yes, I think it could be helpful to mention that Beam Jackson’s deps
>> was bumped and it may require an update of Jackson’s runtime deps for Spark
>> 2 users as well.
>> >>
>> >> —
>> >> Alexey
>> >>
>> >>
>> >>
>> >>>
>> >>>
>> >>> —
>> >>> Alexey
>> >>>
>> >>> On 24 Sep 2021, at 16:17, Alexey Romanenko <ar...@gmail.com>
>> wrote:
>> >>>
>> >>> I checked with beam-samples [1] and noticed an issue to run some
>> pipelines with Spark 2 runner (Spark 3 seems is ok).
>> >>> It looks that it’s caused by new Jackson's version updated recently
>> [2], even if it’s a minor update but it works fine with a previous one.
>> >>> I’ll try to find a workaround and get back with a results of this.
>> >>>
>> >>> [1] https://github.com/Talend/beam-samples/
>> >>> [2]
>> https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c
>> >>>
>> >>> —
>> >>> Alexey
>> >>>
>> >>> On 24 Sep 2021, at 11:17, Jan Lukavský <je...@seznam.cz> wrote:
>> >>>
>> >>> +1 (non-binding)
>> >>>
>> >>> Validated several use-cases using non-portable Flink with Java SDK.
>> >>>
>> >>> On 9/24/21 4:55 AM, Valentyn Tymofieiev wrote:
>> >>>
>> >>> +1. Ran several Python batch and streaming pipelines on Dataflow and
>> checked that Dataflow containers have required dependencies of Beam.
>> >>>
>> >>> On Thu, Sep 23, 2021 at 7:03 PM Robert Burke <lo...@apache.org>
>> wrote:
>> >>>>
>> >>>> +1 (non-binding)
>> >>>> I validated the Go Quickstart (wordcount), and my ray tracer against
>> the Go Direct runner, Dataflow, and Spark (ensuring the rc1 tagged
>> container was used) and they executed successfully.
>> >>>>
>> >>>> I needed to manually synthesize a pseudo-version to ensure I was
>> using the tagged branch version (v2.0.0-20210914211513-b358127f9859)
>> instead of simply using v2.33.0-RC1.
>> >>>>
>> >>>>  It either can't find the package with the right tagged version, or
>> it can't find the version. It's not clear to me what the issue is, but it's
>> not notionally a release blocker. I'll investigate further once we have a
>> full release, as it's probably some unspecified behavior due to how we
>> transitioned to Go Modules (which strongly recommended doing a major
>> version bump for such transitions, which seems a bit excessive for Beam as
>> a whole...).
>> >>>>
>> >>>> On 2021/09/23 03:59:18, Ahmet Altay <al...@google.com> wrote:
>> >>>> > +1 on the RC. I validated python quick start examples on direct
>> runners.
>> >>>> > Thank you Udi.
>> >>>> >
>> >>>> > On Wed, Sep 22, 2021 at 2:20 PM Robert Burke <ro...@frantil.com>
>> wrote:
>> >>>> >
>> >>>> > > Just an FYI that intend to validate the Go SDK for this release
>> but can't
>> >>>> > > get to it until tomorrow (Thursday). I'm catching up from a week
>> of
>> >>>> > > vacation. Apologies for the inconvenience.
>> >>>> > >
>> >>>> > > On Tue, Sep 21, 2021, 10:59 AM Udi Meiri <eh...@google.com>
>> wrote:
>> >>>> > >
>> >>>> > >> I updated the affected and fixed version fields for
>> >>>> > >> https://issues.apache.org/jira/browse/BEAM-12356.
>> >>>> > >>
>> >>>> > >>
>> >>>> > >> On Tue, Sep 21, 2021 at 10:48 AM Reuven Lax <re...@google.com>
>> wrote:
>> >>>> > >>
>> >>>> > >>> Unfortunate - I didn't realize that
>> >>>> > >>> https://github.com/apache/beam/pull/15480 didn't make the cut.
>> >>>> > >>>
>> >>>> > >>> This bug was a regression in Beam 2.32.0, and is blocking
>> multiple users
>> >>>> > >>> from updating.
>> >>>> > >>>
>> >>>> > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi Meiri <ud...@apache.org>
>> wrote:
>> >>>> > >>>
>> >>>> > >>>> Hi everyone,
>> >>>> > >>>> Please review and vote on the release candidate #1 for the
>> version
>> >>>> > >>>> 2.33.0, as follows:
>> >>>> > >>>> [ ] +1, Approve the release
>> >>>> > >>>> [ ] -1, Do not approve the release (please provide specific
>> comments)
>> >>>> > >>>>
>> >>>> > >>>>
>> >>>> > >>>> Reviewers are encouraged to test their own use cases with the
>> release
>> >>>> > >>>> candidate, and vote +1 if
>> >>>> > >>>> no issues are found.
>> >>>> > >>>>
>> >>>> > >>>> The complete staging area is available for your review, which
>> includes:
>> >>>> > >>>> * JIRA release notes [1],
>> >>>> > >>>> * the official Apache source release to be deployed to
>> dist.apache.org
>> >>>> > >>>> [2], which is signed with the key with fingerprint
>> 587B049C36DAAFE6 [3],
>> >>>> > >>>> * all artifacts to be deployed to the Maven Central
>> Repository [4],
>> >>>> > >>>> * source code tag "v2.33.0-RC1" [5],
>> >>>> > >>>> * website pull request listing the release [6], the blog post
>> [6], and
>> >>>> > >>>> publishing the API reference manual [7].
>> >>>> > >>>> * Java artifacts were built with Maven 3.6.3 and OpenJDK
>> 1.8.0_181.
>> >>>> > >>>> * Python artifacts are deployed along with the source release
>> to the
>> >>>> > >>>> dist.apache.org [2] and pypy[8].
>> >>>> > >>>> * Validation sheet with a tab for 2.33.0 release to help with
>> >>>> > >>>> validation [9].
>> >>>> > >>>> * Docker images published to Docker Hub [10].
>> >>>> > >>>>
>> >>>> > >>>> The vote will be open for at least 72 hours. It is adopted by
>> majority
>> >>>> > >>>> approval, with at least 3 PMC affirmative votes.
>> >>>> > >>>>
>> >>>> > >>>> For guidelines on how to try the release in your projects,
>> check out
>> >>>> > >>>> our blog post at
>> https://beam.apache.org/blog/validate-beam-release/.
>> >>>> > >>>>
>> >>>> > >>>> Thanks,
>> >>>> > >>>> Release Manager
>> >>>> > >>>>
>> >>>> > >>>> [1]
>> >>>> > >>>>
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
>> >>>> > >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.33.0/
>> >>>> > >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>> >>>> > >>>> [4]
>> >>>> > >>>>
>> https://repository.apache.org/content/repositories/orgapachebeam-1234/
>> >>>> > >>>> [5] https://github.com/apache/beam/tree/v2.33.0-RC1
>> >>>> > >>>> [6] https://github.com/apache/beam/pull/15543
>> >>>> > >>>> [7] https://github.com/apache/beam-site/pull/619
>> >>>> > >>>> [8] https://pypi.org/project/apache-beam/2.33.0rc1/
>> >>>> > >>>> [9]
>> >>>> > >>>>
>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
>> >>>> > >>>> [10] https://hub.docker.com/search?q=apache%2Fbeam&type=image
>> >>>> > >>>>
>> >>>> > >>>>
>> >>>> >
>> >>>
>> >>>
>> >>>
>> >>
>>
>

Re: [VOTE] Release 2.33.0, release candidate 1

[Chamikara Jayalath <ch...@google.com>]

+1 (binding)
Validated a few scenarios from the spreadsheet.

Thanks,
Cham

On Fri, Sep 24, 2021 at 3:07 PM Robert Bradshaw <ro...@google.com> wrote:

> +1 (binding)
>
> All the artifacts and signatures look good.
>
> I don't think the unsafe trigger check is severe enough to block the
> release.
>
> On Fri, Sep 24, 2021 at 2:36 PM Udi Meiri <eh...@google.com> wrote:
> >
> > Hi everyone, we found a bug during testing. It has to do with Python
> SDK's allow_unsafe_triggers check.
> > There is a preliminary fix that will go to master.
> >
> > For the 2.33.0 release, I'm leaning towards not making a new RC since
> there is a workaround: pass the flag --allow_unsafe_triggers.
> >
> > Please reevaluate your votes accordingly and recast if you've changed
> your vote.
> >
> > Thanks
> >
> > On Fri, Sep 24, 2021 at 12:48 PM Alexey Romanenko <
> aromanenko.dev@gmail.com> wrote:
> >>
> >>
> >> On 24 Sep 2021, at 20:45, Udi Meiri <eh...@google.com> wrote:
> >> Alexey is this something that we should put in the release notes, or
> some other change?
> >>
> >>
> >> Yes, I think it could be helpful to mention that Beam Jackson’s deps
> was bumped and it may require an update of Jackson’s runtime deps for Spark
> 2 users as well.
> >>
> >> —
> >> Alexey
> >>
> >>
> >>
> >>>
> >>>
> >>> —
> >>> Alexey
> >>>
> >>> On 24 Sep 2021, at 16:17, Alexey Romanenko <ar...@gmail.com>
> wrote:
> >>>
> >>> I checked with beam-samples [1] and noticed an issue to run some
> pipelines with Spark 2 runner (Spark 3 seems is ok).
> >>> It looks that it’s caused by new Jackson's version updated recently
> [2], even if it’s a minor update but it works fine with a previous one.
> >>> I’ll try to find a workaround and get back with a results of this.
> >>>
> >>> [1] https://github.com/Talend/beam-samples/
> >>> [2]
> https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c
> >>>
> >>> —
> >>> Alexey
> >>>
> >>> On 24 Sep 2021, at 11:17, Jan Lukavský <je...@seznam.cz> wrote:
> >>>
> >>> +1 (non-binding)
> >>>
> >>> Validated several use-cases using non-portable Flink with Java SDK.
> >>>
> >>> On 9/24/21 4:55 AM, Valentyn Tymofieiev wrote:
> >>>
> >>> +1. Ran several Python batch and streaming pipelines on Dataflow and
> checked that Dataflow containers have required dependencies of Beam.
> >>>
> >>> On Thu, Sep 23, 2021 at 7:03 PM Robert Burke <lo...@apache.org>
> wrote:
> >>>>
> >>>> +1 (non-binding)
> >>>> I validated the Go Quickstart (wordcount), and my ray tracer against
> the Go Direct runner, Dataflow, and Spark (ensuring the rc1 tagged
> container was used) and they executed successfully.
> >>>>
> >>>> I needed to manually synthesize a pseudo-version to ensure I was
> using the tagged branch version (v2.0.0-20210914211513-b358127f9859)
> instead of simply using v2.33.0-RC1.
> >>>>
> >>>>  It either can't find the package with the right tagged version, or
> it can't find the version. It's not clear to me what the issue is, but it's
> not notionally a release blocker. I'll investigate further once we have a
> full release, as it's probably some unspecified behavior due to how we
> transitioned to Go Modules (which strongly recommended doing a major
> version bump for such transitions, which seems a bit excessive for Beam as
> a whole...).
> >>>>
> >>>> On 2021/09/23 03:59:18, Ahmet Altay <al...@google.com> wrote:
> >>>> > +1 on the RC. I validated python quick start examples on direct
> runners.
> >>>> > Thank you Udi.
> >>>> >
> >>>> > On Wed, Sep 22, 2021 at 2:20 PM Robert Burke <ro...@frantil.com>
> wrote:
> >>>> >
> >>>> > > Just an FYI that intend to validate the Go SDK for this release
> but can't
> >>>> > > get to it until tomorrow (Thursday). I'm catching up from a week
> of
> >>>> > > vacation. Apologies for the inconvenience.
> >>>> > >
> >>>> > > On Tue, Sep 21, 2021, 10:59 AM Udi Meiri <eh...@google.com>
> wrote:
> >>>> > >
> >>>> > >> I updated the affected and fixed version fields for
> >>>> > >> https://issues.apache.org/jira/browse/BEAM-12356.
> >>>> > >>
> >>>> > >>
> >>>> > >> On Tue, Sep 21, 2021 at 10:48 AM Reuven Lax <re...@google.com>
> wrote:
> >>>> > >>
> >>>> > >>> Unfortunate - I didn't realize that
> >>>> > >>> https://github.com/apache/beam/pull/15480 didn't make the cut.
> >>>> > >>>
> >>>> > >>> This bug was a regression in Beam 2.32.0, and is blocking
> multiple users
> >>>> > >>> from updating.
> >>>> > >>>
> >>>> > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi Meiri <ud...@apache.org>
> wrote:
> >>>> > >>>
> >>>> > >>>> Hi everyone,
> >>>> > >>>> Please review and vote on the release candidate #1 for the
> version
> >>>> > >>>> 2.33.0, as follows:
> >>>> > >>>> [ ] +1, Approve the release
> >>>> > >>>> [ ] -1, Do not approve the release (please provide specific
> comments)
> >>>> > >>>>
> >>>> > >>>>
> >>>> > >>>> Reviewers are encouraged to test their own use cases with the
> release
> >>>> > >>>> candidate, and vote +1 if
> >>>> > >>>> no issues are found.
> >>>> > >>>>
> >>>> > >>>> The complete staging area is available for your review, which
> includes:
> >>>> > >>>> * JIRA release notes [1],
> >>>> > >>>> * the official Apache source release to be deployed to
> dist.apache.org
> >>>> > >>>> [2], which is signed with the key with fingerprint
> 587B049C36DAAFE6 [3],
> >>>> > >>>> * all artifacts to be deployed to the Maven Central Repository
> [4],
> >>>> > >>>> * source code tag "v2.33.0-RC1" [5],
> >>>> > >>>> * website pull request listing the release [6], the blog post
> [6], and
> >>>> > >>>> publishing the API reference manual [7].
> >>>> > >>>> * Java artifacts were built with Maven 3.6.3 and OpenJDK
> 1.8.0_181.
> >>>> > >>>> * Python artifacts are deployed along with the source release
> to the
> >>>> > >>>> dist.apache.org [2] and pypy[8].
> >>>> > >>>> * Validation sheet with a tab for 2.33.0 release to help with
> >>>> > >>>> validation [9].
> >>>> > >>>> * Docker images published to Docker Hub [10].
> >>>> > >>>>
> >>>> > >>>> The vote will be open for at least 72 hours. It is adopted by
> majority
> >>>> > >>>> approval, with at least 3 PMC affirmative votes.
> >>>> > >>>>
> >>>> > >>>> For guidelines on how to try the release in your projects,
> check out
> >>>> > >>>> our blog post at
> https://beam.apache.org/blog/validate-beam-release/.
> >>>> > >>>>
> >>>> > >>>> Thanks,
> >>>> > >>>> Release Manager
> >>>> > >>>>
> >>>> > >>>> [1]
> >>>> > >>>>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
> >>>> > >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.33.0/
> >>>> > >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> >>>> > >>>> [4]
> >>>> > >>>>
> https://repository.apache.org/content/repositories/orgapachebeam-1234/
> >>>> > >>>> [5] https://github.com/apache/beam/tree/v2.33.0-RC1
> >>>> > >>>> [6] https://github.com/apache/beam/pull/15543
> >>>> > >>>> [7] https://github.com/apache/beam-site/pull/619
> >>>> > >>>> [8] https://pypi.org/project/apache-beam/2.33.0rc1/
> >>>> > >>>> [9]
> >>>> > >>>>
> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
> >>>> > >>>> [10] https://hub.docker.com/search?q=apache%2Fbeam&type=image
> >>>> > >>>>
> >>>> > >>>>
> >>>> >
> >>>
> >>>
> >>>
> >>
>

Re: [VOTE] Release 2.33.0, release candidate 1

[Robert Bradshaw <ro...@google.com>]

+1 (binding)

All the artifacts and signatures look good.

I don't think the unsafe trigger check is severe enough to block the release.

On Fri, Sep 24, 2021 at 2:36 PM Udi Meiri <eh...@google.com> wrote:
>
> Hi everyone, we found a bug during testing. It has to do with Python SDK's allow_unsafe_triggers check.
> There is a preliminary fix that will go to master.
>
> For the 2.33.0 release, I'm leaning towards not making a new RC since there is a workaround: pass the flag --allow_unsafe_triggers.
>
> Please reevaluate your votes accordingly and recast if you've changed your vote.
>
> Thanks
>
> On Fri, Sep 24, 2021 at 12:48 PM Alexey Romanenko <ar...@gmail.com> wrote:
>>
>>
>> On 24 Sep 2021, at 20:45, Udi Meiri <eh...@google.com> wrote:
>> Alexey is this something that we should put in the release notes, or some other change?
>>
>>
>> Yes, I think it could be helpful to mention that Beam Jackson’s deps was bumped and it may require an update of Jackson’s runtime deps for Spark 2 users as well.
>>
>> —
>> Alexey
>>
>>
>>
>>>
>>>
>>> —
>>> Alexey
>>>
>>> On 24 Sep 2021, at 16:17, Alexey Romanenko <ar...@gmail.com> wrote:
>>>
>>> I checked with beam-samples [1] and noticed an issue to run some pipelines with Spark 2 runner (Spark 3 seems is ok).
>>> It looks that it’s caused by new Jackson's version updated recently [2], even if it’s a minor update but it works fine with a previous one.
>>> I’ll try to find a workaround and get back with a results of this.
>>>
>>> [1] https://github.com/Talend/beam-samples/
>>> [2] https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c
>>>
>>> —
>>> Alexey
>>>
>>> On 24 Sep 2021, at 11:17, Jan Lukavský <je...@seznam.cz> wrote:
>>>
>>> +1 (non-binding)
>>>
>>> Validated several use-cases using non-portable Flink with Java SDK.
>>>
>>> On 9/24/21 4:55 AM, Valentyn Tymofieiev wrote:
>>>
>>> +1. Ran several Python batch and streaming pipelines on Dataflow and checked that Dataflow containers have required dependencies of Beam.
>>>
>>> On Thu, Sep 23, 2021 at 7:03 PM Robert Burke <lo...@apache.org> wrote:
>>>>
>>>> +1 (non-binding)
>>>> I validated the Go Quickstart (wordcount), and my ray tracer against the Go Direct runner, Dataflow, and Spark (ensuring the rc1 tagged container was used) and they executed successfully.
>>>>
>>>> I needed to manually synthesize a pseudo-version to ensure I was using the tagged branch version (v2.0.0-20210914211513-b358127f9859) instead of simply using v2.33.0-RC1.
>>>>
>>>>  It either can't find the package with the right tagged version, or it can't find the version. It's not clear to me what the issue is, but it's not notionally a release blocker. I'll investigate further once we have a full release, as it's probably some unspecified behavior due to how we transitioned to Go Modules (which strongly recommended doing a major version bump for such transitions, which seems a bit excessive for Beam as a whole...).
>>>>
>>>> On 2021/09/23 03:59:18, Ahmet Altay <al...@google.com> wrote:
>>>> > +1 on the RC. I validated python quick start examples on direct runners.
>>>> > Thank you Udi.
>>>> >
>>>> > On Wed, Sep 22, 2021 at 2:20 PM Robert Burke <ro...@frantil.com> wrote:
>>>> >
>>>> > > Just an FYI that intend to validate the Go SDK for this release but can't
>>>> > > get to it until tomorrow (Thursday). I'm catching up from a week of
>>>> > > vacation. Apologies for the inconvenience.
>>>> > >
>>>> > > On Tue, Sep 21, 2021, 10:59 AM Udi Meiri <eh...@google.com> wrote:
>>>> > >
>>>> > >> I updated the affected and fixed version fields for
>>>> > >> https://issues.apache.org/jira/browse/BEAM-12356.
>>>> > >>
>>>> > >>
>>>> > >> On Tue, Sep 21, 2021 at 10:48 AM Reuven Lax <re...@google.com> wrote:
>>>> > >>
>>>> > >>> Unfortunate - I didn't realize that
>>>> > >>> https://github.com/apache/beam/pull/15480 didn't make the cut.
>>>> > >>>
>>>> > >>> This bug was a regression in Beam 2.32.0, and is blocking multiple users
>>>> > >>> from updating.
>>>> > >>>
>>>> > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi Meiri <ud...@apache.org> wrote:
>>>> > >>>
>>>> > >>>> Hi everyone,
>>>> > >>>> Please review and vote on the release candidate #1 for the version
>>>> > >>>> 2.33.0, as follows:
>>>> > >>>> [ ] +1, Approve the release
>>>> > >>>> [ ] -1, Do not approve the release (please provide specific comments)
>>>> > >>>>
>>>> > >>>>
>>>> > >>>> Reviewers are encouraged to test their own use cases with the release
>>>> > >>>> candidate, and vote +1 if
>>>> > >>>> no issues are found.
>>>> > >>>>
>>>> > >>>> The complete staging area is available for your review, which includes:
>>>> > >>>> * JIRA release notes [1],
>>>> > >>>> * the official Apache source release to be deployed to dist.apache.org
>>>> > >>>> [2], which is signed with the key with fingerprint 587B049C36DAAFE6 [3],
>>>> > >>>> * all artifacts to be deployed to the Maven Central Repository [4],
>>>> > >>>> * source code tag "v2.33.0-RC1" [5],
>>>> > >>>> * website pull request listing the release [6], the blog post [6], and
>>>> > >>>> publishing the API reference manual [7].
>>>> > >>>> * Java artifacts were built with Maven 3.6.3 and OpenJDK 1.8.0_181.
>>>> > >>>> * Python artifacts are deployed along with the source release to the
>>>> > >>>> dist.apache.org [2] and pypy[8].
>>>> > >>>> * Validation sheet with a tab for 2.33.0 release to help with
>>>> > >>>> validation [9].
>>>> > >>>> * Docker images published to Docker Hub [10].
>>>> > >>>>
>>>> > >>>> The vote will be open for at least 72 hours. It is adopted by majority
>>>> > >>>> approval, with at least 3 PMC affirmative votes.
>>>> > >>>>
>>>> > >>>> For guidelines on how to try the release in your projects, check out
>>>> > >>>> our blog post at https://beam.apache.org/blog/validate-beam-release/.
>>>> > >>>>
>>>> > >>>> Thanks,
>>>> > >>>> Release Manager
>>>> > >>>>
>>>> > >>>> [1]
>>>> > >>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
>>>> > >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.33.0/
>>>> > >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>>> > >>>> [4]
>>>> > >>>> https://repository.apache.org/content/repositories/orgapachebeam-1234/
>>>> > >>>> [5] https://github.com/apache/beam/tree/v2.33.0-RC1
>>>> > >>>> [6] https://github.com/apache/beam/pull/15543
>>>> > >>>> [7] https://github.com/apache/beam-site/pull/619
>>>> > >>>> [8] https://pypi.org/project/apache-beam/2.33.0rc1/
>>>> > >>>> [9]
>>>> > >>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
>>>> > >>>> [10] https://hub.docker.com/search?q=apache%2Fbeam&type=image
>>>> > >>>>
>>>> > >>>>
>>>> >
>>>
>>>
>>>
>>

Re: [VOTE] Release 2.33.0, release candidate 1

[Udi Meiri <eh...@google.com>]

Hi everyone, we found a bug during testing. It has to do with Python
SDK's allow_unsafe_triggers check.
There is a preliminary fix <https://github.com/apache/beam/pull/15586> that
will go to master.

For the 2.33.0 release, I'm leaning towards not making a new RC since there
is a workaround: pass the flag --allow_unsafe_triggers.

Please reevaluate your votes accordingly and recast if you've changed your
vote.

Thanks

On Fri, Sep 24, 2021 at 12:48 PM Alexey Romanenko <ar...@gmail.com>
wrote:

>
> On 24 Sep 2021, at 20:45, Udi Meiri <eh...@google.com> wrote:
> Alexey is this something that we should put in the release notes, or some
> other change?
>
>
> Yes, I think it could be helpful to mention that Beam Jackson’s deps was
> bumped and it may require an update of Jackson’s runtime deps for Spark 2
> users as well.
>
> —
> Alexey
>
>
>
>
>>
>> —
>> Alexey
>>
>> On 24 Sep 2021, at 16:17, Alexey Romanenko <ar...@gmail.com>
>> wrote:
>>
>> I checked with beam-samples [1] and noticed an issue to run some
>> pipelines with Spark 2 runner (Spark 3 seems is ok).
>> It looks that it’s caused by new Jackson's version updated recently [2],
>> even if it’s a minor update but it works fine with a previous one.
>> I’ll try to find a workaround and get back with a results of this.
>>
>> [1] https://github.com/Talend/beam-samples/
>> [2]
>> https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c
>>
>> —
>> Alexey
>>
>> On 24 Sep 2021, at 11:17, Jan Lukavský <je...@seznam.cz> wrote:
>>
>> +1 (non-binding)
>>
>> Validated several use-cases using non-portable Flink with Java SDK.
>> On 9/24/21 4:55 AM, Valentyn Tymofieiev wrote:
>>
>> +1. Ran several Python batch and streaming pipelines on Dataflow and
>> checked that Dataflow containers have required dependencies of Beam.
>>
>> On Thu, Sep 23, 2021 at 7:03 PM Robert Burke <lo...@apache.org> wrote:
>>
>>> +1 (non-binding)
>>> I validated the Go Quickstart (wordcount), and my ray tracer against the
>>> Go Direct runner, Dataflow, and Spark (ensuring the rc1 tagged container
>>> was used) and they executed successfully.
>>>
>>> I needed to manually synthesize a pseudo-version to ensure I was using
>>> the tagged branch version (v2.0.0-20210914211513-b358127f9859) instead of
>>> simply using v2.33.0-RC1.
>>>
>>>  It either can't find the package with the right tagged version, or it
>>> can't find the version. It's not clear to me what the issue is, but it's
>>> not notionally a release blocker. I'll investigate further once we have a
>>> full release, as it's probably some unspecified behavior due to how we
>>> transitioned to Go Modules (which strongly recommended doing a major
>>> version bump for such transitions, which seems a bit excessive for Beam as
>>> a whole...).
>>>
>>> On 2021/09/23 03:59:18, Ahmet Altay <al...@google.com> wrote:
>>> > +1 on the RC. I validated python quick start examples on direct
>>> runners.
>>> > Thank you Udi.
>>> >
>>> > On Wed, Sep 22, 2021 at 2:20 PM Robert Burke <ro...@frantil.com>
>>> wrote:
>>> >
>>> > > Just an FYI that intend to validate the Go SDK for this release but
>>> can't
>>> > > get to it until tomorrow (Thursday). I'm catching up from a week of
>>> > > vacation. Apologies for the inconvenience.
>>> > >
>>> > > On Tue, Sep 21, 2021, 10:59 AM Udi Meiri <eh...@google.com> wrote:
>>> > >
>>> > >> I updated the affected and fixed version fields for
>>> > >> https://issues.apache.org/jira/browse/BEAM-12356.
>>> > >>
>>> > >>
>>> > >> On Tue, Sep 21, 2021 at 10:48 AM Reuven Lax <re...@google.com>
>>> wrote:
>>> > >>
>>> > >>> Unfortunate - I didn't realize that
>>> > >>> https://github.com/apache/beam/pull/15480 didn't make the cut.
>>> > >>>
>>> > >>> This bug was a regression in Beam 2.32.0, and is blocking multiple
>>> users
>>> > >>> from updating.
>>> > >>>
>>> > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi Meiri <ud...@apache.org>
>>> wrote:
>>> > >>>
>>> > >>>> Hi everyone,
>>> > >>>> Please review and vote on the release candidate #1 for the version
>>> > >>>> 2.33.0, as follows:
>>> > >>>> [ ] +1, Approve the release
>>> > >>>> [ ] -1, Do not approve the release (please provide specific
>>> comments)
>>> > >>>>
>>> > >>>>
>>> > >>>> Reviewers are encouraged to test their own use cases with the
>>> release
>>> > >>>> candidate, and vote +1 if
>>> > >>>> no issues are found.
>>> > >>>>
>>> > >>>> The complete staging area is available for your review, which
>>> includes:
>>> > >>>> * JIRA release notes [1],
>>> > >>>> * the official Apache source release to be deployed to
>>> dist.apache.org
>>> > >>>> [2], which is signed with the key with fingerprint
>>> 587B049C36DAAFE6 [3],
>>> > >>>> * all artifacts to be deployed to the Maven Central Repository
>>> [4],
>>> > >>>> * source code tag "v2.33.0-RC1" [5],
>>> > >>>> * website pull request listing the release [6], the blog post
>>> [6], and
>>> > >>>> publishing the API reference manual [7].
>>> > >>>> * Java artifacts were built with Maven 3.6.3 and OpenJDK
>>> 1.8.0_181.
>>> > >>>> * Python artifacts are deployed along with the source release to
>>> the
>>> > >>>> dist.apache.org [2] and pypy[8].
>>> > >>>> * Validation sheet with a tab for 2.33.0 release to help with
>>> > >>>> validation [9].
>>> > >>>> * Docker images published to Docker Hub [10].
>>> > >>>>
>>> > >>>> The vote will be open for at least 72 hours. It is adopted by
>>> majority
>>> > >>>> approval, with at least 3 PMC affirmative votes.
>>> > >>>>
>>> > >>>> For guidelines on how to try the release in your projects, check
>>> out
>>> > >>>> our blog post at
>>> https://beam.apache.org/blog/validate-beam-release/.
>>> > >>>>
>>> > >>>> Thanks,
>>> > >>>> Release Manager
>>> > >>>>
>>> > >>>> [1]
>>> > >>>>
>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
>>> > >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.33.0/
>>> > >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>> > >>>> [4]
>>> > >>>>
>>> https://repository.apache.org/content/repositories/orgapachebeam-1234/
>>> > >>>> [5] https://github.com/apache/beam/tree/v2.33.0-RC1
>>> > >>>> [6] https://github.com/apache/beam/pull/15543
>>> > >>>> [7] https://github.com/apache/beam-site/pull/619
>>> > >>>> [8] https://pypi.org/project/apache-beam/2.33.0rc1/
>>> > >>>> [9]
>>> > >>>>
>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
>>> > >>>> [10] https://hub.docker.com/search?q=apache%2Fbeam&type=image
>>> > >>>>
>>> > >>>>
>>> >
>>>
>>
>>
>>
>

Re: [VOTE] Release 2.33.0, release candidate 1

[Alexey Romanenko <ar...@gmail.com>]

> On 24 Sep 2021, at 20:45, Udi Meiri <eh...@google.com> wrote:
> Alexey is this something that we should put in the release notes, or some other change?

Yes, I think it could be helpful to mention that Beam Jackson’s deps was bumped and it may require an update of Jackson’s runtime deps for Spark 2 users as well.

—
Alexey


>  
> 
> —
> Alexey
> 
>> On 24 Sep 2021, at 16:17, Alexey Romanenko <aromanenko.dev@gmail.com <ma...@gmail.com>> wrote:
>> 
>> I checked with beam-samples [1] and noticed an issue to run some pipelines with Spark 2 runner (Spark 3 seems is ok). 
>> It looks that it’s caused by new Jackson's version updated recently [2], even if it’s a minor update but it works fine with a previous one. 
>> I’ll try to find a workaround and get back with a results of this.
>> 
>> [1] https://github.com/Talend/beam-samples/ <https://github.com/Talend/beam-samples/>
>> [2] https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c <https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c>
>> 
>> —
>> Alexey
>> 
>>> On 24 Sep 2021, at 11:17, Jan Lukavský <je.ik@seznam.cz <ma...@seznam.cz>> wrote:
>>> 
>>> +1 (non-binding)
>>> 
>>> Validated several use-cases using non-portable Flink with Java SDK.
>>> 
>>> On 9/24/21 4:55 AM, Valentyn Tymofieiev wrote:
>>>> +1. Ran several Python batch and streaming pipelines on Dataflow and checked that Dataflow containers have required dependencies of Beam.
>>>> 
>>>> On Thu, Sep 23, 2021 at 7:03 PM Robert Burke <lostluck@apache.org <ma...@apache.org>> wrote:
>>>> +1 (non-binding) 
>>>> I validated the Go Quickstart (wordcount), and my ray tracer against the Go Direct runner, Dataflow, and Spark (ensuring the rc1 tagged container was used) and they executed successfully.
>>>> 
>>>> I needed to manually synthesize a pseudo-version to ensure I was using the tagged branch version (v2.0.0-20210914211513-b358127f9859) instead of simply using v2.33.0-RC1. 
>>>> 
>>>>  It either can't find the package with the right tagged version, or it can't find the version. It's not clear to me what the issue is, but it's not notionally a release blocker. I'll investigate further once we have a full release, as it's probably some unspecified behavior due to how we transitioned to Go Modules (which strongly recommended doing a major version bump for such transitions, which seems a bit excessive for Beam as a whole...).
>>>> 
>>>> On 2021/09/23 03:59:18, Ahmet Altay <altay@google.com <ma...@google.com>> wrote: 
>>>> > +1 on the RC. I validated python quick start examples on direct runners.
>>>> > Thank you Udi.
>>>> > 
>>>> > On Wed, Sep 22, 2021 at 2:20 PM Robert Burke <robert@frantil.com <ma...@frantil.com>> wrote:
>>>> > 
>>>> > > Just an FYI that intend to validate the Go SDK for this release but can't
>>>> > > get to it until tomorrow (Thursday). I'm catching up from a week of
>>>> > > vacation. Apologies for the inconvenience.
>>>> > >
>>>> > > On Tue, Sep 21, 2021, 10:59 AM Udi Meiri <ehudm@google.com <ma...@google.com>> wrote:
>>>> > >
>>>> > >> I updated the affected and fixed version fields for
>>>> > >> https://issues.apache.org/jira/browse/BEAM-12356 <https://issues.apache.org/jira/browse/BEAM-12356>.
>>>> > >>
>>>> > >>
>>>> > >> On Tue, Sep 21, 2021 at 10:48 AM Reuven Lax <relax@google.com <ma...@google.com>> wrote:
>>>> > >>
>>>> > >>> Unfortunate - I didn't realize that
>>>> > >>> https://github.com/apache/beam/pull/15480 <https://github.com/apache/beam/pull/15480> didn't make the cut.
>>>> > >>>
>>>> > >>> This bug was a regression in Beam 2.32.0, and is blocking multiple users
>>>> > >>> from updating.
>>>> > >>>
>>>> > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi Meiri <udim@apache.org <ma...@apache.org>> wrote:
>>>> > >>>
>>>> > >>>> Hi everyone,
>>>> > >>>> Please review and vote on the release candidate #1 for the version
>>>> > >>>> 2.33.0, as follows:
>>>> > >>>> [ ] +1, Approve the release
>>>> > >>>> [ ] -1, Do not approve the release (please provide specific comments)
>>>> > >>>>
>>>> > >>>>
>>>> > >>>> Reviewers are encouraged to test their own use cases with the release
>>>> > >>>> candidate, and vote +1 if
>>>> > >>>> no issues are found.
>>>> > >>>>
>>>> > >>>> The complete staging area is available for your review, which includes:
>>>> > >>>> * JIRA release notes [1],
>>>> > >>>> * the official Apache source release to be deployed to dist.apache.org <http://dist.apache.org/>
>>>> > >>>> [2], which is signed with the key with fingerprint 587B049C36DAAFE6 [3],
>>>> > >>>> * all artifacts to be deployed to the Maven Central Repository [4],
>>>> > >>>> * source code tag "v2.33.0-RC1" [5],
>>>> > >>>> * website pull request listing the release [6], the blog post [6], and
>>>> > >>>> publishing the API reference manual [7].
>>>> > >>>> * Java artifacts were built with Maven 3.6.3 and OpenJDK 1.8.0_181.
>>>> > >>>> * Python artifacts are deployed along with the source release to the
>>>> > >>>> dist.apache.org <http://dist.apache.org/> [2] and pypy[8].
>>>> > >>>> * Validation sheet with a tab for 2.33.0 release to help with
>>>> > >>>> validation [9].
>>>> > >>>> * Docker images published to Docker Hub [10].
>>>> > >>>>
>>>> > >>>> The vote will be open for at least 72 hours. It is adopted by majority
>>>> > >>>> approval, with at least 3 PMC affirmative votes.
>>>> > >>>>
>>>> > >>>> For guidelines on how to try the release in your projects, check out
>>>> > >>>> our blog post at https://beam.apache.org/blog/validate-beam-release/ <https://beam.apache.org/blog/validate-beam-release/>.
>>>> > >>>>
>>>> > >>>> Thanks,
>>>> > >>>> Release Manager
>>>> > >>>>
>>>> > >>>> [1]
>>>> > >>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404 <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404>
>>>> > >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.33.0/ <https://dist.apache.org/repos/dist/dev/beam/2.33.0/>
>>>> > >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS <https://dist.apache.org/repos/dist/release/beam/KEYS>
>>>> > >>>> [4]
>>>> > >>>> https://repository.apache.org/content/repositories/orgapachebeam-1234/ <https://repository.apache.org/content/repositories/orgapachebeam-1234/>
>>>> > >>>> [5] https://github.com/apache/beam/tree/v2.33.0-RC1 <https://github.com/apache/beam/tree/v2.33.0-RC1>
>>>> > >>>> [6] https://github.com/apache/beam/pull/15543 <https://github.com/apache/beam/pull/15543>
>>>> > >>>> [7] https://github.com/apache/beam-site/pull/619 <https://github.com/apache/beam-site/pull/619>
>>>> > >>>> [8] https://pypi.org/project/apache-beam/2.33.0rc1/ <https://pypi.org/project/apache-beam/2.33.0rc1/>
>>>> > >>>> [9]
>>>> > >>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493 <https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493>
>>>> > >>>> [10] https://hub.docker.com/search?q=apache%2Fbeam&type=image <https://hub.docker.com/search?q=apache%2Fbeam&type=image>
>>>> > >>>>
>>>> > >>>>
>>>> > 
>> 
> 

Re: [VOTE] Release 2.33.0, release candidate 1

[Udi Meiri <eh...@google.com>]

I'm currently running additional validations.

On Fri, Sep 24, 2021 at 9:52 AM Alexey Romanenko <ar...@gmail.com>
wrote:

> +1 (binding)
>
> Ok, seems that I found a workaround - jackson runtime dependencies should
> be updated (at least to 2.9.2) in case of using Spark 2.x
> Other than that all look fine for me.
>

Alexey is this something that we should put in the release notes, or some
other change?


>
> —
> Alexey
>
> On 24 Sep 2021, at 16:17, Alexey Romanenko <ar...@gmail.com>
> wrote:
>
> I checked with beam-samples [1] and noticed an issue to run some pipelines
> with Spark 2 runner (Spark 3 seems is ok).
> It looks that it’s caused by new Jackson's version updated recently [2],
> even if it’s a minor update but it works fine with a previous one.
> I’ll try to find a workaround and get back with a results of this.
>
> [1] https://github.com/Talend/beam-samples/
> [2]
> https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c
>
> —
> Alexey
>
> On 24 Sep 2021, at 11:17, Jan Lukavský <je...@seznam.cz> wrote:
>
> +1 (non-binding)
>
> Validated several use-cases using non-portable Flink with Java SDK.
> On 9/24/21 4:55 AM, Valentyn Tymofieiev wrote:
>
> +1. Ran several Python batch and streaming pipelines on Dataflow and
> checked that Dataflow containers have required dependencies of Beam.
>
> On Thu, Sep 23, 2021 at 7:03 PM Robert Burke <lo...@apache.org> wrote:
>
>> +1 (non-binding)
>> I validated the Go Quickstart (wordcount), and my ray tracer against the
>> Go Direct runner, Dataflow, and Spark (ensuring the rc1 tagged container
>> was used) and they executed successfully.
>>
>> I needed to manually synthesize a pseudo-version to ensure I was using
>> the tagged branch version (v2.0.0-20210914211513-b358127f9859) instead of
>> simply using v2.33.0-RC1.
>>
>>  It either can't find the package with the right tagged version, or it
>> can't find the version. It's not clear to me what the issue is, but it's
>> not notionally a release blocker. I'll investigate further once we have a
>> full release, as it's probably some unspecified behavior due to how we
>> transitioned to Go Modules (which strongly recommended doing a major
>> version bump for such transitions, which seems a bit excessive for Beam as
>> a whole...).
>>
>> On 2021/09/23 03:59:18, Ahmet Altay <al...@google.com> wrote:
>> > +1 on the RC. I validated python quick start examples on direct runners.
>> > Thank you Udi.
>> >
>> > On Wed, Sep 22, 2021 at 2:20 PM Robert Burke <ro...@frantil.com>
>> wrote:
>> >
>> > > Just an FYI that intend to validate the Go SDK for this release but
>> can't
>> > > get to it until tomorrow (Thursday). I'm catching up from a week of
>> > > vacation. Apologies for the inconvenience.
>> > >
>> > > On Tue, Sep 21, 2021, 10:59 AM Udi Meiri <eh...@google.com> wrote:
>> > >
>> > >> I updated the affected and fixed version fields for
>> > >> https://issues.apache.org/jira/browse/BEAM-12356.
>> > >>
>> > >>
>> > >> On Tue, Sep 21, 2021 at 10:48 AM Reuven Lax <re...@google.com>
>> wrote:
>> > >>
>> > >>> Unfortunate - I didn't realize that
>> > >>> https://github.com/apache/beam/pull/15480 didn't make the cut.
>> > >>>
>> > >>> This bug was a regression in Beam 2.32.0, and is blocking multiple
>> users
>> > >>> from updating.
>> > >>>
>> > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi Meiri <ud...@apache.org> wrote:
>> > >>>
>> > >>>> Hi everyone,
>> > >>>> Please review and vote on the release candidate #1 for the version
>> > >>>> 2.33.0, as follows:
>> > >>>> [ ] +1, Approve the release
>> > >>>> [ ] -1, Do not approve the release (please provide specific
>> comments)
>> > >>>>
>> > >>>>
>> > >>>> Reviewers are encouraged to test their own use cases with the
>> release
>> > >>>> candidate, and vote +1 if
>> > >>>> no issues are found.
>> > >>>>
>> > >>>> The complete staging area is available for your review, which
>> includes:
>> > >>>> * JIRA release notes [1],
>> > >>>> * the official Apache source release to be deployed to
>> dist.apache.org
>> > >>>> [2], which is signed with the key with fingerprint
>> 587B049C36DAAFE6 [3],
>> > >>>> * all artifacts to be deployed to the Maven Central Repository [4],
>> > >>>> * source code tag "v2.33.0-RC1" [5],
>> > >>>> * website pull request listing the release [6], the blog post [6],
>> and
>> > >>>> publishing the API reference manual [7].
>> > >>>> * Java artifacts were built with Maven 3.6.3 and OpenJDK 1.8.0_181.
>> > >>>> * Python artifacts are deployed along with the source release to
>> the
>> > >>>> dist.apache.org [2] and pypy[8].
>> > >>>> * Validation sheet with a tab for 2.33.0 release to help with
>> > >>>> validation [9].
>> > >>>> * Docker images published to Docker Hub [10].
>> > >>>>
>> > >>>> The vote will be open for at least 72 hours. It is adopted by
>> majority
>> > >>>> approval, with at least 3 PMC affirmative votes.
>> > >>>>
>> > >>>> For guidelines on how to try the release in your projects, check
>> out
>> > >>>> our blog post at
>> https://beam.apache.org/blog/validate-beam-release/.
>> > >>>>
>> > >>>> Thanks,
>> > >>>> Release Manager
>> > >>>>
>> > >>>> [1]
>> > >>>>
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
>> > >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.33.0/
>> > >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>> > >>>> [4]
>> > >>>>
>> https://repository.apache.org/content/repositories/orgapachebeam-1234/
>> > >>>> [5] https://github.com/apache/beam/tree/v2.33.0-RC1
>> > >>>> [6] https://github.com/apache/beam/pull/15543
>> > >>>> [7] https://github.com/apache/beam-site/pull/619
>> > >>>> [8] https://pypi.org/project/apache-beam/2.33.0rc1/
>> > >>>> [9]
>> > >>>>
>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
>> > >>>> [10] https://hub.docker.com/search?q=apache%2Fbeam&type=image
>> > >>>>
>> > >>>>
>> >
>>
>
>
>

Re: [VOTE] Release 2.33.0, release candidate 1

[Alexey Romanenko <ar...@gmail.com>]

+1 (binding)

Ok, seems that I found a workaround - jackson runtime dependencies should be updated (at least to 2.9.2) in case of using Spark 2.x
Other than that all look fine for me.

—
Alexey

> On 24 Sep 2021, at 16:17, Alexey Romanenko <ar...@gmail.com> wrote:
> 
> I checked with beam-samples [1] and noticed an issue to run some pipelines with Spark 2 runner (Spark 3 seems is ok). 
> It looks that it’s caused by new Jackson's version updated recently [2], even if it’s a minor update but it works fine with a previous one. 
> I’ll try to find a workaround and get back with a results of this.
> 
> [1] https://github.com/Talend/beam-samples/ <https://github.com/Talend/beam-samples/>
> [2] https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c <https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c>
> 
> —
> Alexey
> 
>> On 24 Sep 2021, at 11:17, Jan Lukavský <je.ik@seznam.cz <ma...@seznam.cz>> wrote:
>> 
>> +1 (non-binding)
>> 
>> Validated several use-cases using non-portable Flink with Java SDK.
>> 
>> On 9/24/21 4:55 AM, Valentyn Tymofieiev wrote:
>>> +1. Ran several Python batch and streaming pipelines on Dataflow and checked that Dataflow containers have required dependencies of Beam.
>>> 
>>> On Thu, Sep 23, 2021 at 7:03 PM Robert Burke <lostluck@apache.org <ma...@apache.org>> wrote:
>>> +1 (non-binding) 
>>> I validated the Go Quickstart (wordcount), and my ray tracer against the Go Direct runner, Dataflow, and Spark (ensuring the rc1 tagged container was used) and they executed successfully.
>>> 
>>> I needed to manually synthesize a pseudo-version to ensure I was using the tagged branch version (v2.0.0-20210914211513-b358127f9859) instead of simply using v2.33.0-RC1. 
>>> 
>>>  It either can't find the package with the right tagged version, or it can't find the version. It's not clear to me what the issue is, but it's not notionally a release blocker. I'll investigate further once we have a full release, as it's probably some unspecified behavior due to how we transitioned to Go Modules (which strongly recommended doing a major version bump for such transitions, which seems a bit excessive for Beam as a whole...).
>>> 
>>> On 2021/09/23 03:59:18, Ahmet Altay <altay@google.com <ma...@google.com>> wrote: 
>>> > +1 on the RC. I validated python quick start examples on direct runners.
>>> > Thank you Udi.
>>> > 
>>> > On Wed, Sep 22, 2021 at 2:20 PM Robert Burke <robert@frantil.com <ma...@frantil.com>> wrote:
>>> > 
>>> > > Just an FYI that intend to validate the Go SDK for this release but can't
>>> > > get to it until tomorrow (Thursday). I'm catching up from a week of
>>> > > vacation. Apologies for the inconvenience.
>>> > >
>>> > > On Tue, Sep 21, 2021, 10:59 AM Udi Meiri <ehudm@google.com <ma...@google.com>> wrote:
>>> > >
>>> > >> I updated the affected and fixed version fields for
>>> > >> https://issues.apache.org/jira/browse/BEAM-12356 <https://issues.apache.org/jira/browse/BEAM-12356>.
>>> > >>
>>> > >>
>>> > >> On Tue, Sep 21, 2021 at 10:48 AM Reuven Lax <relax@google.com <ma...@google.com>> wrote:
>>> > >>
>>> > >>> Unfortunate - I didn't realize that
>>> > >>> https://github.com/apache/beam/pull/15480 <https://github.com/apache/beam/pull/15480> didn't make the cut.
>>> > >>>
>>> > >>> This bug was a regression in Beam 2.32.0, and is blocking multiple users
>>> > >>> from updating.
>>> > >>>
>>> > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi Meiri <udim@apache.org <ma...@apache.org>> wrote:
>>> > >>>
>>> > >>>> Hi everyone,
>>> > >>>> Please review and vote on the release candidate #1 for the version
>>> > >>>> 2.33.0, as follows:
>>> > >>>> [ ] +1, Approve the release
>>> > >>>> [ ] -1, Do not approve the release (please provide specific comments)
>>> > >>>>
>>> > >>>>
>>> > >>>> Reviewers are encouraged to test their own use cases with the release
>>> > >>>> candidate, and vote +1 if
>>> > >>>> no issues are found.
>>> > >>>>
>>> > >>>> The complete staging area is available for your review, which includes:
>>> > >>>> * JIRA release notes [1],
>>> > >>>> * the official Apache source release to be deployed to dist.apache.org <http://dist.apache.org/>
>>> > >>>> [2], which is signed with the key with fingerprint 587B049C36DAAFE6 [3],
>>> > >>>> * all artifacts to be deployed to the Maven Central Repository [4],
>>> > >>>> * source code tag "v2.33.0-RC1" [5],
>>> > >>>> * website pull request listing the release [6], the blog post [6], and
>>> > >>>> publishing the API reference manual [7].
>>> > >>>> * Java artifacts were built with Maven 3.6.3 and OpenJDK 1.8.0_181.
>>> > >>>> * Python artifacts are deployed along with the source release to the
>>> > >>>> dist.apache.org <http://dist.apache.org/> [2] and pypy[8].
>>> > >>>> * Validation sheet with a tab for 2.33.0 release to help with
>>> > >>>> validation [9].
>>> > >>>> * Docker images published to Docker Hub [10].
>>> > >>>>
>>> > >>>> The vote will be open for at least 72 hours. It is adopted by majority
>>> > >>>> approval, with at least 3 PMC affirmative votes.
>>> > >>>>
>>> > >>>> For guidelines on how to try the release in your projects, check out
>>> > >>>> our blog post at https://beam.apache.org/blog/validate-beam-release/ <https://beam.apache.org/blog/validate-beam-release/>.
>>> > >>>>
>>> > >>>> Thanks,
>>> > >>>> Release Manager
>>> > >>>>
>>> > >>>> [1]
>>> > >>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404 <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404>
>>> > >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.33.0/ <https://dist.apache.org/repos/dist/dev/beam/2.33.0/>
>>> > >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS <https://dist.apache.org/repos/dist/release/beam/KEYS>
>>> > >>>> [4]
>>> > >>>> https://repository.apache.org/content/repositories/orgapachebeam-1234/ <https://repository.apache.org/content/repositories/orgapachebeam-1234/>
>>> > >>>> [5] https://github.com/apache/beam/tree/v2.33.0-RC1 <https://github.com/apache/beam/tree/v2.33.0-RC1>
>>> > >>>> [6] https://github.com/apache/beam/pull/15543 <https://github.com/apache/beam/pull/15543>
>>> > >>>> [7] https://github.com/apache/beam-site/pull/619 <https://github.com/apache/beam-site/pull/619>
>>> > >>>> [8] https://pypi.org/project/apache-beam/2.33.0rc1/ <https://pypi.org/project/apache-beam/2.33.0rc1/>
>>> > >>>> [9]
>>> > >>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493 <https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493>
>>> > >>>> [10] https://hub.docker.com/search?q=apache%2Fbeam&type=image <https://hub.docker.com/search?q=apache%2Fbeam&type=image>
>>> > >>>>
>>> > >>>>
>>> > 
> 

Re: [VOTE] Release 2.33.0, release candidate 1

[Alexey Romanenko <ar...@gmail.com>]

I checked with beam-samples [1] and noticed an issue to run some pipelines with Spark 2 runner (Spark 3 seems is ok). 
It looks that it’s caused by new Jackson's version updated recently [2], even if it’s a minor update but it works fine with a previous one. 
I’ll try to find a workaround and get back with a results of this.

[1] https://github.com/Talend/beam-samples/
[2] https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c

—
Alexey

> On 24 Sep 2021, at 11:17, Jan Lukavský <je...@seznam.cz> wrote:
> 
> +1 (non-binding)
> 
> Validated several use-cases using non-portable Flink with Java SDK.
> 
> On 9/24/21 4:55 AM, Valentyn Tymofieiev wrote:
>> +1. Ran several Python batch and streaming pipelines on Dataflow and checked that Dataflow containers have required dependencies of Beam.
>> 
>> On Thu, Sep 23, 2021 at 7:03 PM Robert Burke <lostluck@apache.org <ma...@apache.org>> wrote:
>> +1 (non-binding) 
>> I validated the Go Quickstart (wordcount), and my ray tracer against the Go Direct runner, Dataflow, and Spark (ensuring the rc1 tagged container was used) and they executed successfully.
>> 
>> I needed to manually synthesize a pseudo-version to ensure I was using the tagged branch version (v2.0.0-20210914211513-b358127f9859) instead of simply using v2.33.0-RC1. 
>> 
>>  It either can't find the package with the right tagged version, or it can't find the version. It's not clear to me what the issue is, but it's not notionally a release blocker. I'll investigate further once we have a full release, as it's probably some unspecified behavior due to how we transitioned to Go Modules (which strongly recommended doing a major version bump for such transitions, which seems a bit excessive for Beam as a whole...).
>> 
>> On 2021/09/23 03:59:18, Ahmet Altay <altay@google.com <ma...@google.com>> wrote: 
>> > +1 on the RC. I validated python quick start examples on direct runners.
>> > Thank you Udi.
>> > 
>> > On Wed, Sep 22, 2021 at 2:20 PM Robert Burke <robert@frantil.com <ma...@frantil.com>> wrote:
>> > 
>> > > Just an FYI that intend to validate the Go SDK for this release but can't
>> > > get to it until tomorrow (Thursday). I'm catching up from a week of
>> > > vacation. Apologies for the inconvenience.
>> > >
>> > > On Tue, Sep 21, 2021, 10:59 AM Udi Meiri <ehudm@google.com <ma...@google.com>> wrote:
>> > >
>> > >> I updated the affected and fixed version fields for
>> > >> https://issues.apache.org/jira/browse/BEAM-12356 <https://issues.apache.org/jira/browse/BEAM-12356>.
>> > >>
>> > >>
>> > >> On Tue, Sep 21, 2021 at 10:48 AM Reuven Lax <relax@google.com <ma...@google.com>> wrote:
>> > >>
>> > >>> Unfortunate - I didn't realize that
>> > >>> https://github.com/apache/beam/pull/15480 <https://github.com/apache/beam/pull/15480> didn't make the cut.
>> > >>>
>> > >>> This bug was a regression in Beam 2.32.0, and is blocking multiple users
>> > >>> from updating.
>> > >>>
>> > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi Meiri <udim@apache.org <ma...@apache.org>> wrote:
>> > >>>
>> > >>>> Hi everyone,
>> > >>>> Please review and vote on the release candidate #1 for the version
>> > >>>> 2.33.0, as follows:
>> > >>>> [ ] +1, Approve the release
>> > >>>> [ ] -1, Do not approve the release (please provide specific comments)
>> > >>>>
>> > >>>>
>> > >>>> Reviewers are encouraged to test their own use cases with the release
>> > >>>> candidate, and vote +1 if
>> > >>>> no issues are found.
>> > >>>>
>> > >>>> The complete staging area is available for your review, which includes:
>> > >>>> * JIRA release notes [1],
>> > >>>> * the official Apache source release to be deployed to dist.apache.org <http://dist.apache.org/>
>> > >>>> [2], which is signed with the key with fingerprint 587B049C36DAAFE6 [3],
>> > >>>> * all artifacts to be deployed to the Maven Central Repository [4],
>> > >>>> * source code tag "v2.33.0-RC1" [5],
>> > >>>> * website pull request listing the release [6], the blog post [6], and
>> > >>>> publishing the API reference manual [7].
>> > >>>> * Java artifacts were built with Maven 3.6.3 and OpenJDK 1.8.0_181.
>> > >>>> * Python artifacts are deployed along with the source release to the
>> > >>>> dist.apache.org <http://dist.apache.org/> [2] and pypy[8].
>> > >>>> * Validation sheet with a tab for 2.33.0 release to help with
>> > >>>> validation [9].
>> > >>>> * Docker images published to Docker Hub [10].
>> > >>>>
>> > >>>> The vote will be open for at least 72 hours. It is adopted by majority
>> > >>>> approval, with at least 3 PMC affirmative votes.
>> > >>>>
>> > >>>> For guidelines on how to try the release in your projects, check out
>> > >>>> our blog post at https://beam.apache.org/blog/validate-beam-release/ <https://beam.apache.org/blog/validate-beam-release/>.
>> > >>>>
>> > >>>> Thanks,
>> > >>>> Release Manager
>> > >>>>
>> > >>>> [1]
>> > >>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404 <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404>
>> > >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.33.0/ <https://dist.apache.org/repos/dist/dev/beam/2.33.0/>
>> > >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS <https://dist.apache.org/repos/dist/release/beam/KEYS>
>> > >>>> [4]
>> > >>>> https://repository.apache.org/content/repositories/orgapachebeam-1234/ <https://repository.apache.org/content/repositories/orgapachebeam-1234/>
>> > >>>> [5] https://github.com/apache/beam/tree/v2.33.0-RC1 <https://github.com/apache/beam/tree/v2.33.0-RC1>
>> > >>>> [6] https://github.com/apache/beam/pull/15543 <https://github.com/apache/beam/pull/15543>
>> > >>>> [7] https://github.com/apache/beam-site/pull/619 <https://github.com/apache/beam-site/pull/619>
>> > >>>> [8] https://pypi.org/project/apache-beam/2.33.0rc1/ <https://pypi.org/project/apache-beam/2.33.0rc1/>
>> > >>>> [9]
>> > >>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493 <https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493>
>> > >>>> [10] https://hub.docker.com/search?q=apache%2Fbeam&type=image <https://hub.docker.com/search?q=apache%2Fbeam&type=image>
>> > >>>>
>> > >>>>
>> > 

Re: [VOTE] Release 2.33.0, release candidate 1

[Jan Lukavský <je...@seznam.cz>]

+1 (non-binding)

Validated several use-cases using non-portable Flink with Java SDK.

On 9/24/21 4:55 AM, Valentyn Tymofieiev wrote:
> +1. Ran several Python batch and streaming pipelines on Dataflow and 
> checked that Dataflow containers have required dependencies of Beam.
>
> On Thu, Sep 23, 2021 at 7:03 PM Robert Burke <lostluck@apache.org 
> <ma...@apache.org>> wrote:
>
>     +1 (non-binding)
>     I validated the Go Quickstart (wordcount), and my ray tracer
>     against the Go Direct runner, Dataflow, and Spark (ensuring the
>     rc1 tagged container was used) and they executed successfully.
>
>     I needed to manually synthesize a pseudo-version to ensure I was
>     using the tagged branch version
>     (v2.0.0-20210914211513-b358127f9859) instead of simply using
>     v2.33.0-RC1.
>
>      It either can't find the package with the right tagged version,
>     or it can't find the version. It's not clear to me what the issue
>     is, but it's not notionally a release blocker. I'll investigate
>     further once we have a full release, as it's probably some
>     unspecified behavior due to how we transitioned to Go Modules
>     (which strongly recommended doing a major version bump for such
>     transitions, which seems a bit excessive for Beam as a whole...).
>
>     On 2021/09/23 03:59:18, Ahmet Altay <altay@google.com
>     <ma...@google.com>> wrote:
>     > +1 on the RC. I validated python quick start examples on direct
>     runners.
>     > Thank you Udi.
>     >
>     > On Wed, Sep 22, 2021 at 2:20 PM Robert Burke <robert@frantil.com
>     <ma...@frantil.com>> wrote:
>     >
>     > > Just an FYI that intend to validate the Go SDK for this
>     release but can't
>     > > get to it until tomorrow (Thursday). I'm catching up from a
>     week of
>     > > vacation. Apologies for the inconvenience.
>     > >
>     > > On Tue, Sep 21, 2021, 10:59 AM Udi Meiri <ehudm@google.com
>     <ma...@google.com>> wrote:
>     > >
>     > >> I updated the affected and fixed version fields for
>     > >> https://issues.apache.org/jira/browse/BEAM-12356
>     <https://issues.apache.org/jira/browse/BEAM-12356>.
>     > >>
>     > >>
>     > >> On Tue, Sep 21, 2021 at 10:48 AM Reuven Lax <relax@google.com
>     <ma...@google.com>> wrote:
>     > >>
>     > >>> Unfortunate - I didn't realize that
>     > >>> https://github.com/apache/beam/pull/15480
>     <https://github.com/apache/beam/pull/15480> didn't make the cut.
>     > >>>
>     > >>> This bug was a regression in Beam 2.32.0, and is blocking
>     multiple users
>     > >>> from updating.
>     > >>>
>     > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi Meiri <udim@apache.org
>     <ma...@apache.org>> wrote:
>     > >>>
>     > >>>> Hi everyone,
>     > >>>> Please review and vote on the release candidate #1 for the
>     version
>     > >>>> 2.33.0, as follows:
>     > >>>> [ ] +1, Approve the release
>     > >>>> [ ] -1, Do not approve the release (please provide specific
>     comments)
>     > >>>>
>     > >>>>
>     > >>>> Reviewers are encouraged to test their own use cases with
>     the release
>     > >>>> candidate, and vote +1 if
>     > >>>> no issues are found.
>     > >>>>
>     > >>>> The complete staging area is available for your review,
>     which includes:
>     > >>>> * JIRA release notes [1],
>     > >>>> * the official Apache source release to be deployed to
>     dist.apache.org <http://dist.apache.org>
>     > >>>> [2], which is signed with the key with fingerprint
>     587B049C36DAAFE6 [3],
>     > >>>> * all artifacts to be deployed to the Maven Central
>     Repository [4],
>     > >>>> * source code tag "v2.33.0-RC1" [5],
>     > >>>> * website pull request listing the release [6], the blog
>     post [6], and
>     > >>>> publishing the API reference manual [7].
>     > >>>> * Java artifacts were built with Maven 3.6.3 and OpenJDK
>     1.8.0_181.
>     > >>>> * Python artifacts are deployed along with the source
>     release to the
>     > >>>> dist.apache.org <http://dist.apache.org> [2] and pypy[8].
>     > >>>> * Validation sheet with a tab for 2.33.0 release to help with
>     > >>>> validation [9].
>     > >>>> * Docker images published to Docker Hub [10].
>     > >>>>
>     > >>>> The vote will be open for at least 72 hours. It is adopted
>     by majority
>     > >>>> approval, with at least 3 PMC affirmative votes.
>     > >>>>
>     > >>>> For guidelines on how to try the release in your projects,
>     check out
>     > >>>> our blog post at
>     https://beam.apache.org/blog/validate-beam-release/
>     <https://beam.apache.org/blog/validate-beam-release/>.
>     > >>>>
>     > >>>> Thanks,
>     > >>>> Release Manager
>     > >>>>
>     > >>>> [1]
>     > >>>>
>     https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
>     <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404>
>     > >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.33.0/
>     <https://dist.apache.org/repos/dist/dev/beam/2.33.0/>
>     > >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>     <https://dist.apache.org/repos/dist/release/beam/KEYS>
>     > >>>> [4]
>     > >>>>
>     https://repository.apache.org/content/repositories/orgapachebeam-1234/
>     <https://repository.apache.org/content/repositories/orgapachebeam-1234/>
>     > >>>> [5] https://github.com/apache/beam/tree/v2.33.0-RC1
>     <https://github.com/apache/beam/tree/v2.33.0-RC1>
>     > >>>> [6] https://github.com/apache/beam/pull/15543
>     <https://github.com/apache/beam/pull/15543>
>     > >>>> [7] https://github.com/apache/beam-site/pull/619
>     <https://github.com/apache/beam-site/pull/619>
>     > >>>> [8] https://pypi.org/project/apache-beam/2.33.0rc1/
>     <https://pypi.org/project/apache-beam/2.33.0rc1/>
>     > >>>> [9]
>     > >>>>
>     https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
>     <https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493>
>     > >>>> [10]
>     https://hub.docker.com/search?q=apache%2Fbeam&type=image
>     <https://hub.docker.com/search?q=apache%2Fbeam&type=image>
>     > >>>>
>     > >>>>
>     >
>

Re: [VOTE] Release 2.33.0, release candidate 1

[Valentyn Tymofieiev <va...@google.com>]

+1. Ran several Python batch and streaming pipelines on Dataflow and
checked that Dataflow containers have required dependencies of Beam.

On Thu, Sep 23, 2021 at 7:03 PM Robert Burke <lo...@apache.org> wrote:

> +1 (non-binding)
> I validated the Go Quickstart (wordcount), and my ray tracer against the
> Go Direct runner, Dataflow, and Spark (ensuring the rc1 tagged container
> was used) and they executed successfully.
>
> I needed to manually synthesize a pseudo-version to ensure I was using the
> tagged branch version (v2.0.0-20210914211513-b358127f9859) instead of
> simply using v2.33.0-RC1.
>
>  It either can't find the package with the right tagged version, or it
> can't find the version. It's not clear to me what the issue is, but it's
> not notionally a release blocker. I'll investigate further once we have a
> full release, as it's probably some unspecified behavior due to how we
> transitioned to Go Modules (which strongly recommended doing a major
> version bump for such transitions, which seems a bit excessive for Beam as
> a whole...).
>
> On 2021/09/23 03:59:18, Ahmet Altay <al...@google.com> wrote:
> > +1 on the RC. I validated python quick start examples on direct runners.
> > Thank you Udi.
> >
> > On Wed, Sep 22, 2021 at 2:20 PM Robert Burke <ro...@frantil.com> wrote:
> >
> > > Just an FYI that intend to validate the Go SDK for this release but
> can't
> > > get to it until tomorrow (Thursday). I'm catching up from a week of
> > > vacation. Apologies for the inconvenience.
> > >
> > > On Tue, Sep 21, 2021, 10:59 AM Udi Meiri <eh...@google.com> wrote:
> > >
> > >> I updated the affected and fixed version fields for
> > >> https://issues.apache.org/jira/browse/BEAM-12356.
> > >>
> > >>
> > >> On Tue, Sep 21, 2021 at 10:48 AM Reuven Lax <re...@google.com> wrote:
> > >>
> > >>> Unfortunate - I didn't realize that
> > >>> https://github.com/apache/beam/pull/15480 didn't make the cut.
> > >>>
> > >>> This bug was a regression in Beam 2.32.0, and is blocking multiple
> users
> > >>> from updating.
> > >>>
> > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi Meiri <ud...@apache.org> wrote:
> > >>>
> > >>>> Hi everyone,
> > >>>> Please review and vote on the release candidate #1 for the version
> > >>>> 2.33.0, as follows:
> > >>>> [ ] +1, Approve the release
> > >>>> [ ] -1, Do not approve the release (please provide specific
> comments)
> > >>>>
> > >>>>
> > >>>> Reviewers are encouraged to test their own use cases with the
> release
> > >>>> candidate, and vote +1 if
> > >>>> no issues are found.
> > >>>>
> > >>>> The complete staging area is available for your review, which
> includes:
> > >>>> * JIRA release notes [1],
> > >>>> * the official Apache source release to be deployed to
> dist.apache.org
> > >>>> [2], which is signed with the key with fingerprint 587B049C36DAAFE6
> [3],
> > >>>> * all artifacts to be deployed to the Maven Central Repository [4],
> > >>>> * source code tag "v2.33.0-RC1" [5],
> > >>>> * website pull request listing the release [6], the blog post [6],
> and
> > >>>> publishing the API reference manual [7].
> > >>>> * Java artifacts were built with Maven 3.6.3 and OpenJDK 1.8.0_181.
> > >>>> * Python artifacts are deployed along with the source release to the
> > >>>> dist.apache.org [2] and pypy[8].
> > >>>> * Validation sheet with a tab for 2.33.0 release to help with
> > >>>> validation [9].
> > >>>> * Docker images published to Docker Hub [10].
> > >>>>
> > >>>> The vote will be open for at least 72 hours. It is adopted by
> majority
> > >>>> approval, with at least 3 PMC affirmative votes.
> > >>>>
> > >>>> For guidelines on how to try the release in your projects, check out
> > >>>> our blog post at
> https://beam.apache.org/blog/validate-beam-release/.
> > >>>>
> > >>>> Thanks,
> > >>>> Release Manager
> > >>>>
> > >>>> [1]
> > >>>>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
> > >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.33.0/
> > >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> > >>>> [4]
> > >>>>
> https://repository.apache.org/content/repositories/orgapachebeam-1234/
> > >>>> [5] https://github.com/apache/beam/tree/v2.33.0-RC1
> > >>>> [6] https://github.com/apache/beam/pull/15543
> > >>>> [7] https://github.com/apache/beam-site/pull/619
> > >>>> [8] https://pypi.org/project/apache-beam/2.33.0rc1/
> > >>>> [9]
> > >>>>
> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
> > >>>> [10] https://hub.docker.com/search?q=apache%2Fbeam&type=image
> > >>>>
> > >>>>
> >
>

Re: [VOTE] Release 2.33.0, release candidate 1

[Robert Burke <lo...@apache.org>]

+1 (non-binding) 
I validated the Go Quickstart (wordcount), and my ray tracer against the Go Direct runner, Dataflow, and Spark (ensuring the rc1 tagged container was used) and they executed successfully.

I needed to manually synthesize a pseudo-version to ensure I was using the tagged branch version (v2.0.0-20210914211513-b358127f9859) instead of simply using v2.33.0-RC1. 

 It either can't find the package with the right tagged version, or it can't find the version. It's not clear to me what the issue is, but it's not notionally a release blocker. I'll investigate further once we have a full release, as it's probably some unspecified behavior due to how we transitioned to Go Modules (which strongly recommended doing a major version bump for such transitions, which seems a bit excessive for Beam as a whole...).

On 2021/09/23 03:59:18, Ahmet Altay <al...@google.com> wrote: 
> +1 on the RC. I validated python quick start examples on direct runners.
> Thank you Udi.
> 
> On Wed, Sep 22, 2021 at 2:20 PM Robert Burke <ro...@frantil.com> wrote:
> 
> > Just an FYI that intend to validate the Go SDK for this release but can't
> > get to it until tomorrow (Thursday). I'm catching up from a week of
> > vacation. Apologies for the inconvenience.
> >
> > On Tue, Sep 21, 2021, 10:59 AM Udi Meiri <eh...@google.com> wrote:
> >
> >> I updated the affected and fixed version fields for
> >> https://issues.apache.org/jira/browse/BEAM-12356.
> >>
> >>
> >> On Tue, Sep 21, 2021 at 10:48 AM Reuven Lax <re...@google.com> wrote:
> >>
> >>> Unfortunate - I didn't realize that
> >>> https://github.com/apache/beam/pull/15480 didn't make the cut.
> >>>
> >>> This bug was a regression in Beam 2.32.0, and is blocking multiple users
> >>> from updating.
> >>>
> >>> On Tue, Sep 21, 2021 at 10:33 AM Udi Meiri <ud...@apache.org> wrote:
> >>>
> >>>> Hi everyone,
> >>>> Please review and vote on the release candidate #1 for the version
> >>>> 2.33.0, as follows:
> >>>> [ ] +1, Approve the release
> >>>> [ ] -1, Do not approve the release (please provide specific comments)
> >>>>
> >>>>
> >>>> Reviewers are encouraged to test their own use cases with the release
> >>>> candidate, and vote +1 if
> >>>> no issues are found.
> >>>>
> >>>> The complete staging area is available for your review, which includes:
> >>>> * JIRA release notes [1],
> >>>> * the official Apache source release to be deployed to dist.apache.org
> >>>> [2], which is signed with the key with fingerprint 587B049C36DAAFE6 [3],
> >>>> * all artifacts to be deployed to the Maven Central Repository [4],
> >>>> * source code tag "v2.33.0-RC1" [5],
> >>>> * website pull request listing the release [6], the blog post [6], and
> >>>> publishing the API reference manual [7].
> >>>> * Java artifacts were built with Maven 3.6.3 and OpenJDK 1.8.0_181.
> >>>> * Python artifacts are deployed along with the source release to the
> >>>> dist.apache.org [2] and pypy[8].
> >>>> * Validation sheet with a tab for 2.33.0 release to help with
> >>>> validation [9].
> >>>> * Docker images published to Docker Hub [10].
> >>>>
> >>>> The vote will be open for at least 72 hours. It is adopted by majority
> >>>> approval, with at least 3 PMC affirmative votes.
> >>>>
> >>>> For guidelines on how to try the release in your projects, check out
> >>>> our blog post at https://beam.apache.org/blog/validate-beam-release/.
> >>>>
> >>>> Thanks,
> >>>> Release Manager
> >>>>
> >>>> [1]
> >>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
> >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.33.0/
> >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> >>>> [4]
> >>>> https://repository.apache.org/content/repositories/orgapachebeam-1234/
> >>>> [5] https://github.com/apache/beam/tree/v2.33.0-RC1
> >>>> [6] https://github.com/apache/beam/pull/15543
> >>>> [7] https://github.com/apache/beam-site/pull/619
> >>>> [8] https://pypi.org/project/apache-beam/2.33.0rc1/
> >>>> [9]
> >>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
> >>>> [10] https://hub.docker.com/search?q=apache%2Fbeam&type=image
> >>>>
> >>>>
> 

Re: [VOTE] Release 2.33.0, release candidate 1

[Ahmet Altay <al...@google.com>]

+1 on the RC. I validated python quick start examples on direct runners.
Thank you Udi.

On Wed, Sep 22, 2021 at 2:20 PM Robert Burke <ro...@frantil.com> wrote:

> Just an FYI that intend to validate the Go SDK for this release but can't
> get to it until tomorrow (Thursday). I'm catching up from a week of
> vacation. Apologies for the inconvenience.
>
> On Tue, Sep 21, 2021, 10:59 AM Udi Meiri <eh...@google.com> wrote:
>
>> I updated the affected and fixed version fields for
>> https://issues.apache.org/jira/browse/BEAM-12356.
>>
>>
>> On Tue, Sep 21, 2021 at 10:48 AM Reuven Lax <re...@google.com> wrote:
>>
>>> Unfortunate - I didn't realize that
>>> https://github.com/apache/beam/pull/15480 didn't make the cut.
>>>
>>> This bug was a regression in Beam 2.32.0, and is blocking multiple users
>>> from updating.
>>>
>>> On Tue, Sep 21, 2021 at 10:33 AM Udi Meiri <ud...@apache.org> wrote:
>>>
>>>> Hi everyone,
>>>> Please review and vote on the release candidate #1 for the version
>>>> 2.33.0, as follows:
>>>> [ ] +1, Approve the release
>>>> [ ] -1, Do not approve the release (please provide specific comments)
>>>>
>>>>
>>>> Reviewers are encouraged to test their own use cases with the release
>>>> candidate, and vote +1 if
>>>> no issues are found.
>>>>
>>>> The complete staging area is available for your review, which includes:
>>>> * JIRA release notes [1],
>>>> * the official Apache source release to be deployed to dist.apache.org
>>>> [2], which is signed with the key with fingerprint 587B049C36DAAFE6 [3],
>>>> * all artifacts to be deployed to the Maven Central Repository [4],
>>>> * source code tag "v2.33.0-RC1" [5],
>>>> * website pull request listing the release [6], the blog post [6], and
>>>> publishing the API reference manual [7].
>>>> * Java artifacts were built with Maven 3.6.3 and OpenJDK 1.8.0_181.
>>>> * Python artifacts are deployed along with the source release to the
>>>> dist.apache.org [2] and pypy[8].
>>>> * Validation sheet with a tab for 2.33.0 release to help with
>>>> validation [9].
>>>> * Docker images published to Docker Hub [10].
>>>>
>>>> The vote will be open for at least 72 hours. It is adopted by majority
>>>> approval, with at least 3 PMC affirmative votes.
>>>>
>>>> For guidelines on how to try the release in your projects, check out
>>>> our blog post at https://beam.apache.org/blog/validate-beam-release/.
>>>>
>>>> Thanks,
>>>> Release Manager
>>>>
>>>> [1]
>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
>>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.33.0/
>>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>>> [4]
>>>> https://repository.apache.org/content/repositories/orgapachebeam-1234/
>>>> [5] https://github.com/apache/beam/tree/v2.33.0-RC1
>>>> [6] https://github.com/apache/beam/pull/15543
>>>> [7] https://github.com/apache/beam-site/pull/619
>>>> [8] https://pypi.org/project/apache-beam/2.33.0rc1/
>>>> [9]
>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
>>>> [10] https://hub.docker.com/search?q=apache%2Fbeam&amp;type=image
>>>>
>>>>

Re: [VOTE] Release 2.33.0, release candidate 1

[Robert Burke <ro...@frantil.com>]

Just an FYI that intend to validate the Go SDK for this release but can't
get to it until tomorrow (Thursday). I'm catching up from a week of
vacation. Apologies for the inconvenience.

On Tue, Sep 21, 2021, 10:59 AM Udi Meiri <eh...@google.com> wrote:

> I updated the affected and fixed version fields for
> https://issues.apache.org/jira/browse/BEAM-12356.
>
>
> On Tue, Sep 21, 2021 at 10:48 AM Reuven Lax <re...@google.com> wrote:
>
>> Unfortunate - I didn't realize that
>> https://github.com/apache/beam/pull/15480 didn't make the cut.
>>
>> This bug was a regression in Beam 2.32.0, and is blocking multiple users
>> from updating.
>>
>> On Tue, Sep 21, 2021 at 10:33 AM Udi Meiri <ud...@apache.org> wrote:
>>
>>> Hi everyone,
>>> Please review and vote on the release candidate #1 for the version
>>> 2.33.0, as follows:
>>> [ ] +1, Approve the release
>>> [ ] -1, Do not approve the release (please provide specific comments)
>>>
>>>
>>> Reviewers are encouraged to test their own use cases with the release
>>> candidate, and vote +1 if
>>> no issues are found.
>>>
>>> The complete staging area is available for your review, which includes:
>>> * JIRA release notes [1],
>>> * the official Apache source release to be deployed to dist.apache.org
>>> [2], which is signed with the key with fingerprint 587B049C36DAAFE6 [3],
>>> * all artifacts to be deployed to the Maven Central Repository [4],
>>> * source code tag "v2.33.0-RC1" [5],
>>> * website pull request listing the release [6], the blog post [6], and
>>> publishing the API reference manual [7].
>>> * Java artifacts were built with Maven 3.6.3 and OpenJDK 1.8.0_181.
>>> * Python artifacts are deployed along with the source release to the
>>> dist.apache.org [2] and pypy[8].
>>> * Validation sheet with a tab for 2.33.0 release to help with validation
>>> [9].
>>> * Docker images published to Docker Hub [10].
>>>
>>> The vote will be open for at least 72 hours. It is adopted by majority
>>> approval, with at least 3 PMC affirmative votes.
>>>
>>> For guidelines on how to try the release in your projects, check out our
>>> blog post at https://beam.apache.org/blog/validate-beam-release/.
>>>
>>> Thanks,
>>> Release Manager
>>>
>>> [1]
>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.33.0/
>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>> [4]
>>> https://repository.apache.org/content/repositories/orgapachebeam-1234/
>>> [5] https://github.com/apache/beam/tree/v2.33.0-RC1
>>> [6] https://github.com/apache/beam/pull/15543
>>> [7] https://github.com/apache/beam-site/pull/619
>>> [8] https://pypi.org/project/apache-beam/2.33.0rc1/
>>> [9]
>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
>>> [10] https://hub.docker.com/search?q=apache%2Fbeam&amp;type=image
>>>
>>>

Re: [VOTE] Release 2.33.0, release candidate 1

[Udi Meiri <eh...@google.com>]

I updated the affected and fixed version fields for
https://issues.apache.org/jira/browse/BEAM-12356.


On Tue, Sep 21, 2021 at 10:48 AM Reuven Lax <re...@google.com> wrote:

> Unfortunate - I didn't realize that
> https://github.com/apache/beam/pull/15480 didn't make the cut.
>
> This bug was a regression in Beam 2.32.0, and is blocking multiple users
> from updating.
>
> On Tue, Sep 21, 2021 at 10:33 AM Udi Meiri <ud...@apache.org> wrote:
>
>> Hi everyone,
>> Please review and vote on the release candidate #1 for the version
>> 2.33.0, as follows:
>> [ ] +1, Approve the release
>> [ ] -1, Do not approve the release (please provide specific comments)
>>
>>
>> Reviewers are encouraged to test their own use cases with the release
>> candidate, and vote +1 if
>> no issues are found.
>>
>> The complete staging area is available for your review, which includes:
>> * JIRA release notes [1],
>> * the official Apache source release to be deployed to dist.apache.org
>> [2], which is signed with the key with fingerprint 587B049C36DAAFE6 [3],
>> * all artifacts to be deployed to the Maven Central Repository [4],
>> * source code tag "v2.33.0-RC1" [5],
>> * website pull request listing the release [6], the blog post [6], and
>> publishing the API reference manual [7].
>> * Java artifacts were built with Maven 3.6.3 and OpenJDK 1.8.0_181.
>> * Python artifacts are deployed along with the source release to the
>> dist.apache.org [2] and pypy[8].
>> * Validation sheet with a tab for 2.33.0 release to help with validation
>> [9].
>> * Docker images published to Docker Hub [10].
>>
>> The vote will be open for at least 72 hours. It is adopted by majority
>> approval, with at least 3 PMC affirmative votes.
>>
>> For guidelines on how to try the release in your projects, check out our
>> blog post at https://beam.apache.org/blog/validate-beam-release/.
>>
>> Thanks,
>> Release Manager
>>
>> [1]
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
>> [2] https://dist.apache.org/repos/dist/dev/beam/2.33.0/
>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>> [4]
>> https://repository.apache.org/content/repositories/orgapachebeam-1234/
>> [5] https://github.com/apache/beam/tree/v2.33.0-RC1
>> [6] https://github.com/apache/beam/pull/15543
>> [7] https://github.com/apache/beam-site/pull/619
>> [8] https://pypi.org/project/apache-beam/2.33.0rc1/
>> [9]
>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
>> [10] https://hub.docker.com/search?q=apache%2Fbeam&amp;type=image
>>
>>

Re: [VOTE] Release 2.33.0, release candidate 1

[Reuven Lax <re...@google.com>]

Unfortunate - I didn't realize that
https://github.com/apache/beam/pull/15480 didn't make the cut.

This bug was a regression in Beam 2.32.0, and is blocking multiple users
from updating.

On Tue, Sep 21, 2021 at 10:33 AM Udi Meiri <ud...@apache.org> wrote:

> Hi everyone,
> Please review and vote on the release candidate #1 for the version 2.33.0,
> as follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
>
> Reviewers are encouraged to test their own use cases with the release
> candidate, and vote +1 if
> no issues are found.
>
> The complete staging area is available for your review, which includes:
> * JIRA release notes [1],
> * the official Apache source release to be deployed to dist.apache.org
> [2], which is signed with the key with fingerprint 587B049C36DAAFE6 [3],
> * all artifacts to be deployed to the Maven Central Repository [4],
> * source code tag "v2.33.0-RC1" [5],
> * website pull request listing the release [6], the blog post [6], and
> publishing the API reference manual [7].
> * Java artifacts were built with Maven 3.6.3 and OpenJDK 1.8.0_181.
> * Python artifacts are deployed along with the source release to the
> dist.apache.org [2] and pypy[8].
> * Validation sheet with a tab for 2.33.0 release to help with validation
> [9].
> * Docker images published to Docker Hub [10].
>
> The vote will be open for at least 72 hours. It is adopted by majority
> approval, with at least 3 PMC affirmative votes.
>
> For guidelines on how to try the release in your projects, check out our
> blog post at https://beam.apache.org/blog/validate-beam-release/.
>
> Thanks,
> Release Manager
>
> [1]
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404
> [2] https://dist.apache.org/repos/dist/dev/beam/2.33.0/
> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> [4] https://repository.apache.org/content/repositories/orgapachebeam-1234/
> [5] https://github.com/apache/beam/tree/v2.33.0-RC1
> [6] https://github.com/apache/beam/pull/15543
> [7] https://github.com/apache/beam-site/pull/619
> [8] https://pypi.org/project/apache-beam/2.33.0rc1/
> [9]
> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493
> [10] https://hub.docker.com/search?q=apache%2Fbeam&amp;type=image
>
>