You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Andrew Pilloud <ap...@google.com> on 2019/04/15 22:50:27 UTC
[VOTE] Release 2.12.0, release candidate #4
Hi everyone,
Please review and vote on the release candidate #4 for the version 2.12.0,
as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)
The complete staging area is available for your review, which includes:
* JIRA release notes [1],
* the official Apache source release to be deployed to dist.apache.org [2],
which is signed with the key with fingerprint 9E7CEC0661EFD610B6
32C610AE8FE17F9F8AE3D4 [3],
* all artifacts to be deployed to the Maven Central Repository [4],
* source code tag "v2.12.0-RC4" [5],
* website pull request listing the release [6], publishing the API
reference manual [7], and the blog post [8].
* Java artifacts were built with Gradle/5.2.1 and OpenJDK/Oracle JDK
1.8.0_181.
* Python artifacts are deployed along with the source release to the
dist.apache.org [2].
* Validation sheet with a tab for 2.12.0 release to help with validation
[9].
The vote will be open for at least 72 hours. It is adopted by majority
approval, with at least 3 PMC affirmative votes.
Thanks,
Andrew
1]
https://jira.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12344944
[2] https://dist.apache.org/repos/dist/dev/beam/2.12.0/
[3] https://dist.apache.org/repos/dist/release/beam/KEYS
[4] https://repository.apache.org/content/repositories/orgapachebeam-1068/
[5] https://github.com/apache/beam/tree/v2.12.0-RC4
[6] https://github.com/apache/beam/pull/8215
[7] https://github.com/apache/beam-site/pull/588
[8] https://github.com/apache/beam/pull/8314
[9]
https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Kenneth Knowles <ke...@apache.org>.
+1
Ran the verification scripts.
Caveats:
- I input a GCS bucket that did not exist, expecting it to be created, so
the Dataflow tests failed.
- I also skipped the Python tests that asked to write to GitHub.
- You also have not built, staged, & signed the Python wheels. It is a bit
hidden in the release guide under the "run all steps manually" section
(even though manual is the only way to do it). Here:
https://beam.apache.org/contribute/release-guide/#stage-python-binaries-on-distapacheorg.
It requires inputting your credentials to Travis-CI, so do what you need to
do to feel OK about that (I changed my credentials to a UUID and changed
back, trusting Travis not to capture my account instantly). It may be
possible to quickly change the target for the multibuild upload to another
destination where you can create a limited key for it to use.
Kenn
On Mon, Apr 15, 2019 at 3:51 PM Andrew Pilloud <ap...@google.com> wrote:
> Hi everyone,
>
> Please review and vote on the release candidate #4 for the version 2.12.0,
> as follows:
>
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
> The complete staging area is available for your review, which includes:
> * JIRA release notes [1],
> * the official Apache source release to be deployed to dist.apache.org
> [2], which is signed with the key with fingerprint 9E7CEC0661EFD610B6
> 32C610AE8FE17F9F8AE3D4 [3],
> * all artifacts to be deployed to the Maven Central Repository [4],
> * source code tag "v2.12.0-RC4" [5],
> * website pull request listing the release [6], publishing the API
> reference manual [7], and the blog post [8].
> * Java artifacts were built with Gradle/5.2.1 and OpenJDK/Oracle JDK
> 1.8.0_181.
> * Python artifacts are deployed along with the source release to the
> dist.apache.org [2].
> * Validation sheet with a tab for 2.12.0 release to help with validation
> [9].
>
> The vote will be open for at least 72 hours. It is adopted by majority
> approval, with at least 3 PMC affirmative votes.
>
> Thanks,
> Andrew
>
> 1]
> https://jira.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12344944
> [2] https://dist.apache.org/repos/dist/dev/beam/2.12.0/
> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> [4] https://repository.apache.org/content/repositories/orgapachebeam-1068/
> [5] https://github.com/apache/beam/tree/v2.12.0-RC4
> [6] https://github.com/apache/beam/pull/8215
> [7] https://github.com/apache/beam-site/pull/588
> [8] https://github.com/apache/beam/pull/8314
> [9]
> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984
>
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Etienne Chauchot <ec...@apache.org>.
Hi, I agree that checking Nexmark should be a mandatory task of the release process, I think it is already mentioned in
the spreadsheet. Indeed it detects both functional and performances regressions and on all the beam model scope.The only
lacking things in Nexmark are with 2 runners:- gearpump is not integrated because streaming pipelines (there is no batch
mode) never end (timeout in awaitTermination is not respected)- spark runner in streaming mode is not integrated for the
same reason
Etienne
Le mardi 23 avril 2019 à 15:02 -0700, Andrew Pilloud a écrit :
> It looks like Java Nexmark tests are on the validation sheet but we've missed it the last few releases. Thanks for
> checking it Etienne! Does the current release process require everything to be tested before making the release final?
> I fully agree with you on point 2. All of these issues were in RC1 and could have been fixed for RC2.
>
> Andrew
> On Tue, Apr 23, 2019 at 2:58 PM Ahmet Altay <al...@google.com> wrote:
> > Thank you Andrew. I will suggest two improvements to the release process:1. We can include benchmarks in the
> > validation sheet ("Apache Beam Release Acceptance Criteria"). They are used part of the validation process and we
> > can ensure that we check those for each release.
> > 2. For RC validation, we can continue to exhaustively validate each RC even after the first -1 vote. Otherwise we
> > end up with not discovering all issues in a given RC and find them a successive RC, increasing the number of
> > iterations required.
> >
> >
> > On Tue, Apr 23, 2019 at 2:11 PM Andrew Pilloud <ap...@google.com> wrote:
> > > Please consider the vote for RC4 canceled. I'll quickly follow up with a new RC.
> > > Thanks for the complete testing everyone!
> > > Andrew
> > > On Tue, Apr 23, 2019 at 2:06 PM Reuven Lax <re...@google.com> wrote:
> > > > -1
> > > > we need to cherry pick pr/8325 and pr/8385 to fix the above issue
> > > > On Tue, Apr 23, 2019 at 1:48 PM Andrew Pilloud <ap...@google.com> wrote:
> > > > > I believe the breakage of Nexmark on Dataflow is https://issues.apache.org/jira/browse/BEAM-7002, which went
> > > > > in before the release was cut. It looks like this might be a release blocker based on the fix:
> > > > > https://github.com/apache/beam/pull/8325.
> > > > >
> > > > > The degraded performance is after the release is cut, so we should be good there.
> > > > >
> > > > > Andrew
> > > > > On Tue, Apr 23, 2019 at 8:44 AM Ismaël Mejía <ie...@gmail.com> wrote:
> > > > > > Etienne RC1 vote happened in 04/03 and there have not been any cherry
> > > > > >
> > > > > > picks on the spark runner afterwards so if there is a commit that
> > > > > >
> > > > > > degraded performance around 04/10 it is not part of the release we are
> > > > > >
> > > > > > voting, so please consider reverting your -1.
> > > > > >
> > > > > >
> > > > > >
> > > > > > However the issue you are reporting looks important, from a quick look
> > > > > >
> > > > > > I am guessing it could be related to BEAM-5775 that was merged on
> > > > > >
> > > > > > 12/04 however the performance regressions started happening since
> > > > > >
> > > > > > 09/04 so it could be unrelated. Maybe it could be due to changes in
> > > > > >
> > > > > > our infrastructure. Maybe the change in the workers to be tracked, but
> > > > > >
> > > > > > definitely not a release blocker at least for the Spark runner.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Tue, Apr 23, 2019 at 5:12 PM Etienne Chauchot <ec...@apache.org> wrote:
> > > > > >
> > > > > > >
> > > > > >
> > > > > > > Hi guys ,
> > > > > >
> > > > > > > I will vote -1 (binding) on this RC (although degradation is before RC4 cut date). I took a look at
> > > > > > Nexmark graphs for the 3 major runners :
> > > > > >
> > > > > > > - there seem to have functional regressions on Dataflow:
> > > > > > https://apache-beam-testing.appspot.com/explore?dashboard=5647201107705856 . 13 queries fail in batch mode
> > > > > > starting on 04/17
> > > > > >
> > > > > > > - there is a perf degradation (+200%) in spark runner starting on 04/10 for all the queries:
> > > > > > https://apache-beam-testing.appspot.com/explore?dashboard=5138380291571712
> > > > > >
> > > > > > >
> > > > > >
> > > > > > > Sorry Andrew for the added work
> > > > > >
> > > > > > >
> > > > > >
> > > > > > > Etienne
> > > > > >
> > > > > > >
> > > > > >
> > > > > > > Le lundi 22 avril 2019 à 12:21 -0700, Andrew Pilloud a écrit :
> > > > > >
> > > > > > >
> > > > > >
> > > > > > > I signed the wheels files and updated the build process to not require giving travis apache credentials.
> > > > > > (You should probably change your password if you haven't already.)
> > > > > >
> > > > > > >
> > > > > >
> > > > > > > Andrew
> > > > > >
> > > > > > >
> > > > > >
> > > > > > > On Mon, Apr 22, 2019 at 12:18 PM Ahmet Altay <al...@google.com> wrote:
> > > > > >
> > > > > > >
> > > > > >
> > > > > > > +1 (binding)
> > > > > >
> > > > > > >
> > > > > >
> > > > > > > Verified the python 2 wheel files with quick start examples.
> > > > > >
> > > > > > >
> > > > > >
> > > > > > > On Mon, Apr 22, 2019 at 11:26 AM Ahmet Altay <al...@google.com> wrote:
> > > > > >
> > > > > > >
> > > > > >
> > > > > > > I built the wheel files. They are in the usual place along with other python artifacts. I will test them a
> > > > > > bit and update here. Could someone else please try the wheel files as well?
> > > > > >
> > > > > > >
> > > > > >
> > > > > > > Andrew, could you sign and hash the wheel files?
> > > > > >
> > > > > > >
> > > > > >
> > > > > > > On Mon, Apr 22, 2019 at 10:11 AM Ahmet Altay <al...@google.com> wrote:
> > > > > >
> > > > > > >
> > > > > >
> > > > > > > I verified
> > > > > >
> > > > > > > - signatures and hashes.
> > > > > >
> > > > > > > - python streaming quickstart guide
> > > > > >
> > > > > > >
> > > > > >
> > > > > > > I would like to verify the wheel files before voting. Please let us know when they are ready. Also, if you
> > > > > > need help with building wheel files I can help/build.
> > > > > >
> > > > > > >
> > > > > >
> > > > > > > Ahmet
> > > > > >
> > > > > > >
> > > > > >
> > > > > > > On Mon, Apr 22, 2019 at 3:33 AM Maximilian Michels <mx...@apache.org> wrote:
> > > > > >
> > > > > > >
> > > > > >
> > > > > > > +1 (binding)
> > > > > >
> > > > > > >
> > > > > >
> > > > > > > Found a minor bug while testing, but not a blocker:
> > > > > >
> > > > > > > https://jira.apache.org/jira/browse/BEAM-7128
> > > > > >
> > > > > > >
> > > > > >
> > > > > > > Thanks,
> > > > > >
> > > > > > > Max
> > > > > >
> > > > > > >
> > > > > >
> > > > > > > On 20.04.19 23:02, Pablo Estrada wrote:
> > > > > >
> > > > > > > > +1
> > > > > >
> > > > > > > > Ran SQL postcommit, and Dataflow Portability Java validatesrunner tests.
> > > > > >
> > > > > > > >
> > > > > >
> > > > > > > > -P.
> > > > > >
> > > > > > > >
> > > > > >
> > > > > > > > On Wed, Apr 17, 2019 at 1:38 AM Jean-Baptiste Onofré <jb@nanthrax.net
> > > > > >
> > > > > > > > <ma...@nanthrax.net>> wrote:
> > > > > >
> > > > > > > >
> > > > > >
> > > > > > > > +1 (binding)
> > > > > >
> > > > > > > >
> > > > > >
> > > > > > > > Quickly checked with beam-samples.
> > > > > >
> > > > > > > >
> > > > > >
> > > > > > > > Regards
> > > > > >
> > > > > > > > JB
> > > > > >
> > > > > > > >
> > > > > >
> > > > > > > > On 16/04/2019 00:50, Andrew Pilloud wrote:
> > > > > >
> > > > > > > > > Hi everyone,
> > > > > >
> > > > > > > > >
> > > > > >
> > > > > > > > > Please review and vote on the release candidate #4 for the version
> > > > > >
> > > > > > > > > 2.12.0, as follows:
> > > > > >
> > > > > > > > >
> > > > > >
> > > > > > > > > [ ] +1, Approve the release
> > > > > >
> > > > > > > > > [ ] -1, Do not approve the release (please provide specific comments)
> > > > > >
> > > > > > > > >
> > > > > >
> > > > > > > > > The complete staging area is available for your review, which
> > > > > >
> > > > > > > > includes:
> > > > > >
> > > > > > > > > * JIRA release notes [1],
> > > > > >
> > > > > > > > > * the official Apache source release to be deployed to
> > > > > >
> > > > > > > > dist.apache.org <http://dist.apache.org>
> > > > > >
> > > > > > > > > <http://dist.apache.org> [2], which is signed with the key with
> > > > > >
> > > > > > > > > fingerprint 9E7CEC0661EFD610B632C610AE8FE17F9F8AE3D4 [3],
> > > > > >
> > > > > > > > > * all artifacts to be deployed to the Maven Central Repository [4],
> > > > > >
> > > > > > > > > * source code tag "v2.12.0-RC4" [5],
> > > > > >
> > > > > > > > > * website pull request listing the release [6], publishing the API
> > > > > >
> > > > > > > > > reference manual [7], and the blog post [8].
> > > > > >
> > > > > > > > > * Java artifacts were built with Gradle/5.2.1 and OpenJDK/Oracle JDK
> > > > > >
> > > > > > > > > 1.8.0_181.
> > > > > >
> > > > > > > > > * Python artifacts are deployed along with the source release to the
> > > > > >
> > > > > > > > > dist.apache.org <http://dist.apache.org> <http://dist.apache.org>
> > > > > >
> > > > > > > > [2].
> > > > > >
> > > > > > > > > * Validation sheet with a tab for 2.12.0 release to help with
> > > > > >
> > > > > > > > validation
> > > > > >
> > > > > > > > > [9].
> > > > > >
> > > > > > > > >
> > > > > >
> > > > > > > > > The vote will be open for at least 72 hours. It is adopted by
> > > > > >
> > > > > > > > majority
> > > > > >
> > > > > > > > > approval, with at least 3 PMC affirmative votes.
> > > > > >
> > > > > > > > >
> > > > > >
> > > > > > > > > Thanks,
> > > > > >
> > > > > > > > > Andrew
> > > > > >
> > > > > > > > >
> > > > > >
> > > > > > > > > 1]
> > > > > >
> > > > > > > > https://jira.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12344944
> > > > > >
> > > > > > > > > [2] https://dist.apache.org/repos/dist/dev/beam/2.12.0/
> > > > > >
> > > > > > > > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> > > > > >
> > > > > > > > > [4]
> > > > > >
> > > > > > > > https://repository.apache.org/content/repositories/orgapachebeam-1068/
> > > > > >
> > > > > > > > > [5] https://github.com/apache/beam/tree/v2.12.0-RC4
> > > > > >
> > > > > > > > > [6] https://github.com/apache/beam/pull/8215
> > > > > >
> > > > > > > > > [7] https://github.com/apache/beam-site/pull/588
> > > > > >
> > > > > > > > > [8] https://github.com/apache/beam/pull/8314
> > > > > >
> > > > > > > > > [9]
> > > > > >
> > > > > > > >
> > > > > > https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984
> > > > > >
> > > > > > > >
> > > > > >
> > > > > > > > --
> > > > > >
> > > > > > > > Jean-Baptiste Onofré
> > > > > >
> > > > > > > > jbonofre@apache.org <ma...@apache.org>
> > > > > >
> > > > > > > > http://blog.nanthrax.net
> > > > > >
> > > > > > > > Talend - http://www.talend.com
> > > > > >
> > > > > > > >
> > > > > >
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Valentyn Tymofieiev <va...@google.com>.
I think we should also leverage/invest in the automation for RC
validation. We have some validation scripts, but last time I looked at
them they worked only partially and had several usability issues.
On Tue, Apr 23, 2019 at 3:24 PM Ahmet Altay <al...@google.com> wrote:
>
>
> On Tue, Apr 23, 2019 at 3:21 PM Kenneth Knowles <ke...@apache.org> wrote:
>
>> What can we do to make this part of day-to-day workflow instead of
>> finding out during release validation? Was this just a failing test that
>> was missed?
>>
>> Kenn
>>
>> On Tue, Apr 23, 2019 at 3:02 PM Andrew Pilloud <ap...@google.com>
>> wrote:
>>
>>> It looks like Java Nexmark tests are on the validation sheet but we've
>>> missed it the last few releases. Thanks for checking it Etienne!
>>>
>>
> I only see nexmark perf on direct runner. Is this even a thing? Should we
> also add major runners to that list?
>
>
>> Does the current release process require everything to be tested before
>>> making the release final?
>>>
>>
> I do not think we agreed on this. Previous releases went out with partial
> validations. It would be good to validate everything for the final release.
>
>
>>
>>> I fully agree with you on point 2. All of these issues were in RC1 and
>>> could have been fixed for RC2.
>>>
>>> Andrew
>>>
>>> On Tue, Apr 23, 2019 at 2:58 PM Ahmet Altay <al...@google.com> wrote:
>>>
>>>> Thank you Andrew. I will suggest two improvements to the release
>>>> process:
>>>> 1. We can include benchmarks in the validation sheet ("Apache Beam
>>>> Release Acceptance Criteria"). They are used part of the validation process
>>>> and we can ensure that we check those for each release.
>>>> 2. For RC validation, we can continue to exhaustively validate each RC
>>>> even after the first -1 vote. Otherwise we end up with not discovering all
>>>> issues in a given RC and find them a successive RC, increasing the number
>>>> of iterations required.
>>>>
>>>>
>>>> On Tue, Apr 23, 2019 at 2:11 PM Andrew Pilloud <ap...@google.com>
>>>> wrote:
>>>>
>>>>> Please consider the vote for RC4 canceled. I'll quickly follow up with
>>>>> a new RC.
>>>>>
>>>>> Thanks for the complete testing everyone!
>>>>>
>>>>> Andrew
>>>>>
>>>>> On Tue, Apr 23, 2019 at 2:06 PM Reuven Lax <re...@google.com> wrote:
>>>>>
>>>>>> -1
>>>>>>
>>>>>> we need to cherry pick pr/8325 and pr/8385 to fix the above issue
>>>>>>
>>>>>> On Tue, Apr 23, 2019 at 1:48 PM Andrew Pilloud <ap...@google.com>
>>>>>> wrote:
>>>>>>
>>>>>>> I believe the breakage of Nexmark on Dataflow is
>>>>>>> https://issues.apache.org/jira/browse/BEAM-7002, which went in
>>>>>>> before the release was cut. It looks like this might be a release blocker
>>>>>>> based on the fix: https://github.com/apache/beam/pull/8325.
>>>>>>>
>>>>>>> The degraded performance is after the release is cut, so we should
>>>>>>> be good there.
>>>>>>>
>>>>>>> Andrew
>>>>>>>
>>>>>>> On Tue, Apr 23, 2019 at 8:44 AM Ismaël Mejía <ie...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Etienne RC1 vote happened in 04/03 and there have not been any
>>>>>>>> cherry
>>>>>>>> picks on the spark runner afterwards so if there is a commit that
>>>>>>>> degraded performance around 04/10 it is not part of the release we
>>>>>>>> are
>>>>>>>> voting, so please consider reverting your -1.
>>>>>>>>
>>>>>>>> However the issue you are reporting looks important, from a quick
>>>>>>>> look
>>>>>>>> I am guessing it could be related to BEAM-5775 that was merged on
>>>>>>>> 12/04 however the performance regressions started happening since
>>>>>>>> 09/04 so it could be unrelated. Maybe it could be due to changes in
>>>>>>>> our infrastructure. Maybe the change in the workers to be tracked,
>>>>>>>> but
>>>>>>>> definitely not a release blocker at least for the Spark runner.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Apr 23, 2019 at 5:12 PM Etienne Chauchot <
>>>>>>>> echauchot@apache.org> wrote:
>>>>>>>> >
>>>>>>>> > Hi guys ,
>>>>>>>> > I will vote -1 (binding) on this RC (although degradation is
>>>>>>>> before RC4 cut date). I took a look at Nexmark graphs for the 3 major
>>>>>>>> runners :
>>>>>>>> > - there seem to have functional regressions on Dataflow:
>>>>>>>> https://apache-beam-testing.appspot.com/explore?dashboard=5647201107705856
>>>>>>>> . 13 queries fail in batch mode starting on 04/17
>>>>>>>> > - there is a perf degradation (+200%) in spark runner starting on
>>>>>>>> 04/10 for all the queries:
>>>>>>>> https://apache-beam-testing.appspot.com/explore?dashboard=5138380291571712
>>>>>>>> >
>>>>>>>> > Sorry Andrew for the added work
>>>>>>>> >
>>>>>>>> > Etienne
>>>>>>>> >
>>>>>>>> > Le lundi 22 avril 2019 à 12:21 -0700, Andrew Pilloud a écrit :
>>>>>>>> >
>>>>>>>> > I signed the wheels files and updated the build process to not
>>>>>>>> require giving travis apache credentials. (You should probably change your
>>>>>>>> password if you haven't already.)
>>>>>>>> >
>>>>>>>> > Andrew
>>>>>>>> >
>>>>>>>> > On Mon, Apr 22, 2019 at 12:18 PM Ahmet Altay <al...@google.com>
>>>>>>>> wrote:
>>>>>>>> >
>>>>>>>> > +1 (binding)
>>>>>>>> >
>>>>>>>> > Verified the python 2 wheel files with quick start examples.
>>>>>>>> >
>>>>>>>> > On Mon, Apr 22, 2019 at 11:26 AM Ahmet Altay <al...@google.com>
>>>>>>>> wrote:
>>>>>>>> >
>>>>>>>> > I built the wheel files. They are in the usual place along with
>>>>>>>> other python artifacts. I will test them a bit and update here. Could
>>>>>>>> someone else please try the wheel files as well?
>>>>>>>> >
>>>>>>>> > Andrew, could you sign and hash the wheel files?
>>>>>>>> >
>>>>>>>> > On Mon, Apr 22, 2019 at 10:11 AM Ahmet Altay <al...@google.com>
>>>>>>>> wrote:
>>>>>>>> >
>>>>>>>> > I verified
>>>>>>>> > - signatures and hashes.
>>>>>>>> > - python streaming quickstart guide
>>>>>>>> >
>>>>>>>> > I would like to verify the wheel files before voting. Please let
>>>>>>>> us know when they are ready. Also, if you need help with building wheel
>>>>>>>> files I can help/build.
>>>>>>>> >
>>>>>>>> > Ahmet
>>>>>>>> >
>>>>>>>> > On Mon, Apr 22, 2019 at 3:33 AM Maximilian Michels <
>>>>>>>> mxm@apache.org> wrote:
>>>>>>>> >
>>>>>>>> > +1 (binding)
>>>>>>>> >
>>>>>>>> > Found a minor bug while testing, but not a blocker:
>>>>>>>> > https://jira.apache.org/jira/browse/BEAM-7128
>>>>>>>> >
>>>>>>>> > Thanks,
>>>>>>>> > Max
>>>>>>>> >
>>>>>>>> > On 20.04.19 23:02, Pablo Estrada wrote:
>>>>>>>> > > +1
>>>>>>>> > > Ran SQL postcommit, and Dataflow Portability Java
>>>>>>>> validatesrunner tests.
>>>>>>>> > >
>>>>>>>> > > -P.
>>>>>>>> > >
>>>>>>>> > > On Wed, Apr 17, 2019 at 1:38 AM Jean-Baptiste Onofré <
>>>>>>>> jb@nanthrax.net
>>>>>>>> > > <ma...@nanthrax.net>> wrote:
>>>>>>>> > >
>>>>>>>> > > +1 (binding)
>>>>>>>> > >
>>>>>>>> > > Quickly checked with beam-samples.
>>>>>>>> > >
>>>>>>>> > > Regards
>>>>>>>> > > JB
>>>>>>>> > >
>>>>>>>> > > On 16/04/2019 00:50, Andrew Pilloud wrote:
>>>>>>>> > > > Hi everyone,
>>>>>>>> > > >
>>>>>>>> > > > Please review and vote on the release candidate #4 for
>>>>>>>> the version
>>>>>>>> > > > 2.12.0, as follows:
>>>>>>>> > > >
>>>>>>>> > > > [ ] +1, Approve the release
>>>>>>>> > > > [ ] -1, Do not approve the release (please provide
>>>>>>>> specific comments)
>>>>>>>> > > >
>>>>>>>> > > > The complete staging area is available for your review,
>>>>>>>> which
>>>>>>>> > > includes:
>>>>>>>> > > > * JIRA release notes [1],
>>>>>>>> > > > * the official Apache source release to be deployed to
>>>>>>>> > > dist.apache.org <http://dist.apache.org>
>>>>>>>> > > > <http://dist.apache.org> [2], which is signed with the
>>>>>>>> key with
>>>>>>>> > > > fingerprint 9E7CEC0661EFD610B632C610AE8FE17F9F8AE3D4 [3],
>>>>>>>> > > > * all artifacts to be deployed to the Maven Central
>>>>>>>> Repository [4],
>>>>>>>> > > > * source code tag "v2.12.0-RC4" [5],
>>>>>>>> > > > * website pull request listing the release [6],
>>>>>>>> publishing the API
>>>>>>>> > > > reference manual [7], and the blog post [8].
>>>>>>>> > > > * Java artifacts were built with Gradle/5.2.1 and
>>>>>>>> OpenJDK/Oracle JDK
>>>>>>>> > > > 1.8.0_181.
>>>>>>>> > > > * Python artifacts are deployed along with the source
>>>>>>>> release to the
>>>>>>>> > > > dist.apache.org <http://dist.apache.org> <
>>>>>>>> http://dist.apache.org>
>>>>>>>> > > [2].
>>>>>>>> > > > * Validation sheet with a tab for 2.12.0 release to help
>>>>>>>> with
>>>>>>>> > > validation
>>>>>>>> > > > [9].
>>>>>>>> > > >
>>>>>>>> > > > The vote will be open for at least 72 hours. It is
>>>>>>>> adopted by
>>>>>>>> > > majority
>>>>>>>> > > > approval, with at least 3 PMC affirmative votes.
>>>>>>>> > > >
>>>>>>>> > > > Thanks,
>>>>>>>> > > > Andrew
>>>>>>>> > > >
>>>>>>>> > > > 1]
>>>>>>>> > >
>>>>>>>> https://jira.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12344944
>>>>>>>> > > > [2] https://dist.apache.org/repos/dist/dev/beam/2.12.0/
>>>>>>>> > > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>>>>>>> > > > [4]
>>>>>>>> > >
>>>>>>>> https://repository.apache.org/content/repositories/orgapachebeam-1068/
>>>>>>>> > > > [5] https://github.com/apache/beam/tree/v2.12.0-RC4
>>>>>>>> > > > [6] https://github.com/apache/beam/pull/8215
>>>>>>>> > > > [7] https://github.com/apache/beam-site/pull/588
>>>>>>>> > > > [8] https://github.com/apache/beam/pull/8314
>>>>>>>> > > > [9]
>>>>>>>> > >
>>>>>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984
>>>>>>>> > >
>>>>>>>> > > --
>>>>>>>> > > Jean-Baptiste Onofré
>>>>>>>> > > jbonofre@apache.org <ma...@apache.org>
>>>>>>>> > > http://blog.nanthrax.net
>>>>>>>> > > Talend - http://www.talend.com
>>>>>>>> > >
>>>>>>>>
>>>>>>>
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Etienne Chauchot <ec...@apache.org>.
Ahmet +1 to your suggestion. I did not know the spreadsheet indicated only DR. I add all the other runners. I also added
details on what needs to be checkedEtienne
Le mercredi 24 avril 2019 à 08:48 -0700, Ahmet Altay a écrit :
> Yes, I see these benchmarks. Thank you.
>
> My point was, the validation sheet (
> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984) only mentions
> Nextmark for Direct runner as something to be validated. I am guessing it means for RCs validate that nexmark can run
> on DirectRunner. I am suggesting adding the check of the dashboard to the validation spreadsheet.
>
> On Wed, Apr 24, 2019 at 2:03 AM Etienne Chauchot <ec...@apache.org> wrote:
> > Ahmet,Don't you see all Nexmark dashboards visible in the joint image at
> > https://apache-beam-testing.appspot.com/dashboard-admin?
> > EtienneLe mardi 23 avril 2019 à 15:24 -0700, Ahmet Altay a écrit :
> > > On Tue, Apr 23, 2019 at 3:21 PM Kenneth Knowles <ke...@apache.org> wrote:
> > > > What can we do to make this part of day-to-day workflow instead of finding out during release validation? Was
> > > > this just a failing test that was missed?
> > > > Kenn
> > > > On Tue, Apr 23, 2019 at 3:02 PM Andrew Pilloud <ap...@google.com> wrote:
> > > > > It looks like Java Nexmark tests are on the validation sheet but we've missed it the last few releases. Thanks
> > > > > for checking it Etienne!
> > >
> > > I only see nexmark perf on direct runner. Is this even a thing? Should we also add major runners to that list?
> > > > > Does the current release process require everything to be tested before making the release final?
> > >
> > > I do not think we agreed on this. Previous releases went out with partial validations. It would be good to
> > > validate everything for the final release.
> > > > > I fully agree with you on point 2. All of these issues were in RC1 and could have been fixed for RC2.
> > > > >
> > > > > Andrew
> > > > > On Tue, Apr 23, 2019 at 2:58 PM Ahmet Altay <al...@google.com> wrote:
> > > > > > Thank you Andrew. I will suggest two improvements to the release process:1. We can include benchmarks in the
> > > > > > validation sheet ("Apache Beam Release Acceptance Criteria"). They are used part of the validation process
> > > > > > and we can ensure that we check those for each release.
> > > > > > 2. For RC validation, we can continue to exhaustively validate each RC even after the first -1 vote.
> > > > > > Otherwise we end up with not discovering all issues in a given RC and find them a successive RC, increasing
> > > > > > the number of iterations required.
> > > > > >
> > > > > >
> > > > > > On Tue, Apr 23, 2019 at 2:11 PM Andrew Pilloud <ap...@google.com> wrote:
> > > > > > > Please consider the vote for RC4 canceled. I'll quickly follow up with a new RC.
> > > > > > > Thanks for the complete testing everyone!
> > > > > > > Andrew
> > > > > > > On Tue, Apr 23, 2019 at 2:06 PM Reuven Lax <re...@google.com> wrote:
> > > > > > > > -1
> > > > > > > > we need to cherry pick pr/8325 and pr/8385 to fix the above issue
> > > > > > > > On Tue, Apr 23, 2019 at 1:48 PM Andrew Pilloud <ap...@google.com> wrote:
> > > > > > > > > I believe the breakage of Nexmark on Dataflow is https://issues.apache.org/jira/browse/BEAM-7002,
> > > > > > > > > which went in before the release was cut. It looks like this might be a release blocker based on the
> > > > > > > > > fix: https://github.com/apache/beam/pull/8325.
> > > > > > > > >
> > > > > > > > > The degraded performance is after the release is cut, so we should be good there.
> > > > > > > > >
> > > > > > > > > Andrew
> > > > > > > > > On Tue, Apr 23, 2019 at 8:44 AM Ismaël Mejía <ie...@gmail.com> wrote:
> > > > > > > > > > Etienne RC1 vote happened in 04/03 and there have not been any cherry
> > > > > > > > > >
> > > > > > > > > > picks on the spark runner afterwards so if there is a commit that
> > > > > > > > > >
> > > > > > > > > > degraded performance around 04/10 it is not part of the release we are
> > > > > > > > > >
> > > > > > > > > > voting, so please consider reverting your -1.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > However the issue you are reporting looks important, from a quick look
> > > > > > > > > >
> > > > > > > > > > I am guessing it could be related to BEAM-5775 that was merged on
> > > > > > > > > >
> > > > > > > > > > 12/04 however the performance regressions started happening since
> > > > > > > > > >
> > > > > > > > > > 09/04 so it could be unrelated. Maybe it could be due to changes in
> > > > > > > > > >
> > > > > > > > > > our infrastructure. Maybe the change in the workers to be tracked, but
> > > > > > > > > >
> > > > > > > > > > definitely not a release blocker at least for the Spark runner.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Tue, Apr 23, 2019 at 5:12 PM Etienne Chauchot <ec...@apache.org> wrote:
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > Hi guys ,
> > > > > > > > > >
> > > > > > > > > > > I will vote -1 (binding) on this RC (although degradation is before RC4 cut date). I took a look
> > > > > > > > > > at Nexmark graphs for the 3 major runners :
> > > > > > > > > >
> > > > > > > > > > > - there seem to have functional regressions on Dataflow:
> > > > > > > > > > https://apache-beam-testing.appspot.com/explore?dashboard=5647201107705856 . 13 queries fail in
> > > > > > > > > > batch mode starting on 04/17
> > > > > > > > > >
> > > > > > > > > > > - there is a perf degradation (+200%) in spark runner starting on 04/10 for all the queries:
> > > > > > > > > > https://apache-beam-testing.appspot.com/explore?dashboard=5138380291571712
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > Sorry Andrew for the added work
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > Etienne
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > Le lundi 22 avril 2019 à 12:21 -0700, Andrew Pilloud a écrit :
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > I signed the wheels files and updated the build process to not require giving travis apache
> > > > > > > > > > credentials. (You should probably change your password if you haven't already.)
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > Andrew
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > On Mon, Apr 22, 2019 at 12:18 PM Ahmet Altay <al...@google.com> wrote:
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > +1 (binding)
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > Verified the python 2 wheel files with quick start examples.
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > On Mon, Apr 22, 2019 at 11:26 AM Ahmet Altay <al...@google.com> wrote:
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > I built the wheel files. They are in the usual place along with other python artifacts. I will
> > > > > > > > > > test them a bit and update here. Could someone else please try the wheel files as well?
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > Andrew, could you sign and hash the wheel files?
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > On Mon, Apr 22, 2019 at 10:11 AM Ahmet Altay <al...@google.com> wrote:
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > I verified
> > > > > > > > > >
> > > > > > > > > > > - signatures and hashes.
> > > > > > > > > >
> > > > > > > > > > > - python streaming quickstart guide
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > I would like to verify the wheel files before voting. Please let us know when they are ready.
> > > > > > > > > > Also, if you need help with building wheel files I can help/build.
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > Ahmet
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > On Mon, Apr 22, 2019 at 3:33 AM Maximilian Michels <mx...@apache.org> wrote:
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > +1 (binding)
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > Found a minor bug while testing, but not a blocker:
> > > > > > > > > >
> > > > > > > > > > > https://jira.apache.org/jira/browse/BEAM-7128
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > Thanks,
> > > > > > > > > >
> > > > > > > > > > > Max
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > On 20.04.19 23:02, Pablo Estrada wrote:
> > > > > > > > > >
> > > > > > > > > > > > +1
> > > > > > > > > >
> > > > > > > > > > > > Ran SQL postcommit, and Dataflow Portability Java validatesrunner tests.
> > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > -P.
> > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > On Wed, Apr 17, 2019 at 1:38 AM Jean-Baptiste Onofré <jb@nanthrax.net
> > > > > > > > > >
> > > > > > > > > > > > <ma...@nanthrax.net>> wrote:
> > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > +1 (binding)
> > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > Quickly checked with beam-samples.
> > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > Regards
> > > > > > > > > >
> > > > > > > > > > > > JB
> > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > On 16/04/2019 00:50, Andrew Pilloud wrote:
> > > > > > > > > >
> > > > > > > > > > > > > Hi everyone,
> > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > > Please review and vote on the release candidate #4 for the version
> > > > > > > > > >
> > > > > > > > > > > > > 2.12.0, as follows:
> > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > > [ ] +1, Approve the release
> > > > > > > > > >
> > > > > > > > > > > > > [ ] -1, Do not approve the release (please provide specific comments)
> > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > > The complete staging area is available for your review, which
> > > > > > > > > >
> > > > > > > > > > > > includes:
> > > > > > > > > >
> > > > > > > > > > > > > * JIRA release notes [1],
> > > > > > > > > >
> > > > > > > > > > > > > * the official Apache source release to be deployed to
> > > > > > > > > >
> > > > > > > > > > > > dist.apache.org <http://dist.apache.org>
> > > > > > > > > >
> > > > > > > > > > > > > <http://dist.apache.org> [2], which is signed with the key with
> > > > > > > > > >
> > > > > > > > > > > > > fingerprint 9E7CEC0661EFD610B632C610AE8FE17F9F8AE3D4 [3],
> > > > > > > > > >
> > > > > > > > > > > > > * all artifacts to be deployed to the Maven Central Repository [4],
> > > > > > > > > >
> > > > > > > > > > > > > * source code tag "v2.12.0-RC4" [5],
> > > > > > > > > >
> > > > > > > > > > > > > * website pull request listing the release [6], publishing the API
> > > > > > > > > >
> > > > > > > > > > > > > reference manual [7], and the blog post [8].
> > > > > > > > > >
> > > > > > > > > > > > > * Java artifacts were built with Gradle/5.2.1 and OpenJDK/Oracle JDK
> > > > > > > > > >
> > > > > > > > > > > > > 1.8.0_181.
> > > > > > > > > >
> > > > > > > > > > > > > * Python artifacts are deployed along with the source release to the
> > > > > > > > > >
> > > > > > > > > > > > > dist.apache.org <http://dist.apache.org> <http://dist.apache.org>
> > > > > > > > > >
> > > > > > > > > > > > [2].
> > > > > > > > > >
> > > > > > > > > > > > > * Validation sheet with a tab for 2.12.0 release to help with
> > > > > > > > > >
> > > > > > > > > > > > validation
> > > > > > > > > >
> > > > > > > > > > > > > [9].
> > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > > The vote will be open for at least 72 hours. It is adopted by
> > > > > > > > > >
> > > > > > > > > > > > majority
> > > > > > > > > >
> > > > > > > > > > > > > approval, with at least 3 PMC affirmative votes.
> > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > > Thanks,
> > > > > > > > > >
> > > > > > > > > > > > > Andrew
> > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > > 1]
> > > > > > > > > >
> > > > > > > > > > > > https://jira.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12344944
> > > > > > > > > >
> > > > > > > > > > > > > [2] https://dist.apache.org/repos/dist/dev/beam/2.12.0/
> > > > > > > > > >
> > > > > > > > > > > > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> > > > > > > > > >
> > > > > > > > > > > > > [4]
> > > > > > > > > >
> > > > > > > > > > > > https://repository.apache.org/content/repositories/orgapachebeam-1068/
> > > > > > > > > >
> > > > > > > > > > > > > [5] https://github.com/apache/beam/tree/v2.12.0-RC4
> > > > > > > > > >
> > > > > > > > > > > > > [6] https://github.com/apache/beam/pull/8215
> > > > > > > > > >
> > > > > > > > > > > > > [7] https://github.com/apache/beam-site/pull/588
> > > > > > > > > >
> > > > > > > > > > > > > [8] https://github.com/apache/beam/pull/8314
> > > > > > > > > >
> > > > > > > > > > > > > [9]
> > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984
> > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > --
> > > > > > > > > >
> > > > > > > > > > > > Jean-Baptiste Onofré
> > > > > > > > > >
> > > > > > > > > > > > jbonofre@apache.org <ma...@apache.org>
> > > > > > > > > >
> > > > > > > > > > > > http://blog.nanthrax.net
> > > > > > > > > >
> > > > > > > > > > > > Talend - http://www.talend.com
> > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > >
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Connell O'Callaghan <co...@google.com>.
+1 thank you Andrew!!!
On Thu, Apr 25, 2019 at 11:44 AM Tim Robertson <ti...@gmail.com>
wrote:
> Thank you for running the release Andrew
>
> On Thu, Apr 25, 2019 at 8:24 PM Andrew Pilloud <ap...@google.com>
> wrote:
>
>> I reran the Nexmark tests, each runner passed. I compared the numbers
>> on the direct runner to the dashboard and they are where they should
>> be.
>>
>> With that, I'm happy to announce that we have unanimously approved this
>> release.
>>
>> There are 8 approving votes, 4 of which are binding:
>> * Jean-Baptiste Onofré
>> * Lukasz Cwik
>> * Maximilian Michels
>> * Ahmet Altay
>>
>> There are no disapproving votes.
>>
>> Thanks everyone!
>>
>
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Tim Robertson <ti...@gmail.com>.
Thank you for running the release Andrew
On Thu, Apr 25, 2019 at 8:24 PM Andrew Pilloud <ap...@google.com> wrote:
> I reran the Nexmark tests, each runner passed. I compared the numbers
> on the direct runner to the dashboard and they are where they should
> be.
>
> With that, I'm happy to announce that we have unanimously approved this
> release.
>
> There are 8 approving votes, 4 of which are binding:
> * Jean-Baptiste Onofré
> * Lukasz Cwik
> * Maximilian Michels
> * Ahmet Altay
>
> There are no disapproving votes.
>
> Thanks everyone!
>
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Connell O'Callaghan <co...@google.com>.
Thank you Andrew!!!
On Mon, Apr 29, 2019 at 9:21 AM Andrew Pilloud <ap...@google.com> wrote:
> Yes, they were moved and are now at
> https://dist.apache.org/repos/dist/release/beam/2.12.0/
>
> On Fri, Apr 26, 2019 at 2:02 AM Robert Bradshaw <ro...@google.com>
> wrote:
> >
> > Thanks for all the hard work!
> >
> > https://dist.apache.org/repos/dist/dev/beam/2.12.0/ seems empty; were
> > the artifacts already moved?
> >
> > On Fri, Apr 26, 2019 at 10:31 AM Etienne Chauchot <ec...@apache.org>
> wrote:
> > >
> > > Hi,
> > > Thanks for all your work and patience Andrew !
> > >
> > > PS: as a side note, there were 5 binding votes (I voted +1)
> > >
> > > Etienne
> > >
> > > Le jeudi 25 avril 2019 à 11:16 -0700, Andrew Pilloud a écrit :
> > >
> > > I reran the Nexmark tests, each runner passed. I compared the numbers
> > >
> > > on the direct runner to the dashboard and they are where they should
> > >
> > > be.
> > >
> > >
> > > With that, I'm happy to announce that we have unanimously approved
> this release.
> > >
> > >
> > > There are 8 approving votes, 4 of which are binding:
> > >
> > > * Jean-Baptiste Onofré
> > >
> > > * Lukasz Cwik
> > >
> > > * Maximilian Michels
> > >
> > > * Ahmet Altay
> > >
> > >
> > > There are no disapproving votes.
> > >
> > >
> > > Thanks everyone!
> > >
> > >
>
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Andrew Pilloud <ap...@google.com>.
Yes, they were moved and are now at
https://dist.apache.org/repos/dist/release/beam/2.12.0/
On Fri, Apr 26, 2019 at 2:02 AM Robert Bradshaw <ro...@google.com> wrote:
>
> Thanks for all the hard work!
>
> https://dist.apache.org/repos/dist/dev/beam/2.12.0/ seems empty; were
> the artifacts already moved?
>
> On Fri, Apr 26, 2019 at 10:31 AM Etienne Chauchot <ec...@apache.org> wrote:
> >
> > Hi,
> > Thanks for all your work and patience Andrew !
> >
> > PS: as a side note, there were 5 binding votes (I voted +1)
> >
> > Etienne
> >
> > Le jeudi 25 avril 2019 à 11:16 -0700, Andrew Pilloud a écrit :
> >
> > I reran the Nexmark tests, each runner passed. I compared the numbers
> >
> > on the direct runner to the dashboard and they are where they should
> >
> > be.
> >
> >
> > With that, I'm happy to announce that we have unanimously approved this release.
> >
> >
> > There are 8 approving votes, 4 of which are binding:
> >
> > * Jean-Baptiste Onofré
> >
> > * Lukasz Cwik
> >
> > * Maximilian Michels
> >
> > * Ahmet Altay
> >
> >
> > There are no disapproving votes.
> >
> >
> > Thanks everyone!
> >
> >
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Robert Bradshaw <ro...@google.com>.
Thanks for all the hard work!
https://dist.apache.org/repos/dist/dev/beam/2.12.0/ seems empty; were
the artifacts already moved?
On Fri, Apr 26, 2019 at 10:31 AM Etienne Chauchot <ec...@apache.org> wrote:
>
> Hi,
> Thanks for all your work and patience Andrew !
>
> PS: as a side note, there were 5 binding votes (I voted +1)
>
> Etienne
>
> Le jeudi 25 avril 2019 à 11:16 -0700, Andrew Pilloud a écrit :
>
> I reran the Nexmark tests, each runner passed. I compared the numbers
>
> on the direct runner to the dashboard and they are where they should
>
> be.
>
>
> With that, I'm happy to announce that we have unanimously approved this release.
>
>
> There are 8 approving votes, 4 of which are binding:
>
> * Jean-Baptiste Onofré
>
> * Lukasz Cwik
>
> * Maximilian Michels
>
> * Ahmet Altay
>
>
> There are no disapproving votes.
>
>
> Thanks everyone!
>
>
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Etienne Chauchot <ec...@apache.org>.
Hi,
Thanks for all your work and patience Andrew !
PS: as a side note, there were 5 binding votes (I voted +1)
Etienne
Le jeudi 25 avril 2019 à 11:16 -0700, Andrew Pilloud a écrit :
> I reran the Nexmark tests, each runner passed. I compared the numbers
> on the direct runner to the dashboard and they are where they should
> be.
>
> With that, I'm happy to announce that we have unanimously approved this release.
>
> There are 8 approving votes, 4 of which are binding:
> * Jean-Baptiste Onofré
> * Lukasz Cwik
> * Maximilian Michels
> * Ahmet Altay
>
> There are no disapproving votes.
>
> Thanks everyone!
>
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Andrew Pilloud <ap...@google.com>.
I reran the Nexmark tests, each runner passed. I compared the numbers
on the direct runner to the dashboard and they are where they should
be.
With that, I'm happy to announce that we have unanimously approved this release.
There are 8 approving votes, 4 of which are binding:
* Jean-Baptiste Onofré
* Lukasz Cwik
* Maximilian Michels
* Ahmet Altay
There are no disapproving votes.
Thanks everyone!
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Etienne Chauchot <ec...@apache.org>.
Hi All,
Andrew, based on this [1] email I thought the RC4 cut date was 04/15. As nexmark regressions started before 04/15, I
voted -1.But it seems I had the wrong RC4 cut date and the correct cut date is before the regressions, then consider my
vote +1.
In that case, all we need is be careful to not cherry pick the regressions commits for 2.12 and for 2.13 to fix these
regressions.
[1]
https://lists.apache.org/thread.html/e5c8d484ad id="-x-evo-selection-start-marker">93b70790e532241b68325bd39056c99ad2a6ee03b4f758@%3Cdev.beam.apache.org%3E
Etienne
Le mercredi 24 avril 2019 à 15:00 -0700, Andrew Pilloud a écrit :
> Please consider the vote for RC4 resumed.
> We currently have two -1 votes based on Nexmark issues that did not start until after the release branch was cut[1].
>
> Etienne and Reuven: Can you verify and update your votes as appropriate?
>
> Andrew
>
> [1] https://github.com/apache/beam/commit/1d9daf1aca101fa5a194cbbba969886734e08902
> On Wed, Apr 24, 2019 at 10:52 AM Andrew Pilloud <ap...@google.com> wrote:
> > Ok, it looks like I messed up here. I went to revert Reuven's change and it wasn't on the branch. I was previously
> > looking at RC1, which was a week after actual cut off of master.
> >
> > The actual cut was the morning of March 28, with the last commit being March 27. We have good performance runs for
> > the commit of the cut.
> >
> > Can we reopen the vote on RC4 and revisit the -1 votes?
> >
> > Andrew
> > On Wed, Apr 24, 2019 at 8:59 AM Ahmet Altay <al...@google.com> wrote:
> > > Yes, I see these benchmarks. Thank you.
> > >
> > > My point was, the validation sheet (
> > > https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984) only
> > > mentions Nextmark for Direct runner as something to be validated. I am guessing it means for RCs validate that
> > > nexmark can run on DirectRunner. I am suggesting adding the check of the dashboard to the validation spreadsheet.
> > >
> > > On Wed, Apr 24, 2019 at 2:03 AM Etienne Chauchot <ec...@apache.org> wrote:
> > > > Ahmet,Don't you see all Nexmark dashboards visible in the joint image at
> > > > https://apache-beam-testing.appspot.com/dashboard-admin?
> > > > EtienneLe mardi 23 avril 2019 à 15:24 -0700, Ahmet Altay a écrit :
> > > > > On Tue, Apr 23, 2019 at 3:21 PM Kenneth Knowles <ke...@apache.org> wrote:
> > > > > > What can we do to make this part of day-to-day workflow instead of finding out during release validation?
> > > > > > Was this just a failing test that was missed?
> > > > > > Kenn
> > > > > > On Tue, Apr 23, 2019 at 3:02 PM Andrew Pilloud <ap...@google.com> wrote:
> > > > > > > It looks like Java Nexmark tests are on the validation sheet but we've missed it the last few releases.
> > > > > > > Thanks for checking it Etienne!
> > > > >
> > > > > I only see nexmark perf on direct runner. Is this even a thing? Should we also add major runners to that
> > > > > list?
> > > > > > > Does the current release process require everything to be tested before making the release final?
> > > > >
> > > > > I do not think we agreed on this. Previous releases went out with partial validations. It would be good to
> > > > > validate everything for the final release.
> > > > > > > I fully agree with you on point 2. All of these issues were in RC1 and could have been fixed for RC2.
> > > > > > >
> > > > > > > Andrew
> > > > > > > On Tue, Apr 23, 2019 at 2:58 PM Ahmet Altay <al...@google.com> wrote:
> > > > > > > > Thank you Andrew. I will suggest two improvements to the release process:1. We can include benchmarks in
> > > > > > > > the validation sheet ("Apache Beam Release Acceptance Criteria"). They are used part of the validation
> > > > > > > > process and we can ensure that we check those for each release.
> > > > > > > > 2. For RC validation, we can continue to exhaustively validate each RC even after the first -1 vote.
> > > > > > > > Otherwise we end up with not discovering all issues in a given RC and find them a successive RC,
> > > > > > > > increasing the number of iterations required.
> > > > > > > >
> > > > > > > >
> > > > > > > > On Tue, Apr 23, 2019 at 2:11 PM Andrew Pilloud <ap...@google.com> wrote:
> > > > > > > > > Please consider the vote for RC4 canceled. I'll quickly follow up with a new RC.
> > > > > > > > > Thanks for the complete testing everyone!
> > > > > > > > > Andrew
> > > > > > > > > On Tue, Apr 23, 2019 at 2:06 PM Reuven Lax <re...@google.com> wrote:
> > > > > > > > > > -1
> > > > > > > > > > we need to cherry pick pr/8325 and pr/8385 to fix the above issue
> > > > > > > > > > On Tue, Apr 23, 2019 at 1:48 PM Andrew Pilloud <ap...@google.com> wrote:
> > > > > > > > > > > I believe the breakage of Nexmark on Dataflow is https://issues.apache.org/jira/browse/BEAM-7002,
> > > > > > > > > > > which went in before the release was cut. It looks like this might be a release blocker based on
> > > > > > > > > > > the fix: https://github.com/apache/beam/pull/8325.
> > > > > > > > > > >
> > > > > > > > > > > The degraded performance is after the release is cut, so we should be good there.
> > > > > > > > > > >
> > > > > > > > > > > Andrew
> > > > > > > > > > > On Tue, Apr 23, 2019 at 8:44 AM Ismaël Mejía <ie...@gmail.com> wrote:
> > > > > > > > > > > > Etienne RC1 vote happened in 04/03 and there have not been any cherry
> > > > > > > > > > > >
> > > > > > > > > > > > picks on the spark runner afterwards so if there is a commit that
> > > > > > > > > > > >
> > > > > > > > > > > > degraded performance around 04/10 it is not part of the release we are
> > > > > > > > > > > >
> > > > > > > > > > > > voting, so please consider reverting your -1.
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > However the issue you are reporting looks important, from a quick look
> > > > > > > > > > > >
> > > > > > > > > > > > I am guessing it could be related to BEAM-5775 that was merged on
> > > > > > > > > > > >
> > > > > > > > > > > > 12/04 however the performance regressions started happening since
> > > > > > > > > > > >
> > > > > > > > > > > > 09/04 so it could be unrelated. Maybe it could be due to changes in
> > > > > > > > > > > >
> > > > > > > > > > > > our infrastructure. Maybe the change in the workers to be tracked, but
> > > > > > > > > > > >
> > > > > > > > > > > > definitely not a release blocker at least for the Spark runner.
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > On Tue, Apr 23, 2019 at 5:12 PM Etienne Chauchot <ec...@apache.org> wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > Hi guys ,
> > > > > > > > > > > >
> > > > > > > > > > > > > I will vote -1 (binding) on this RC (although degradation is before RC4 cut date). I took a
> > > > > > > > > > > > look at Nexmark graphs for the 3 major runners :
> > > > > > > > > > > >
> > > > > > > > > > > > > - there seem to have functional regressions on Dataflow:
> > > > > > > > > > > > https://apache-beam-testing.appspot.com/explore?dashboard=5647201107705856 . 13 queries fail in
> > > > > > > > > > > > batch mode starting on 04/17
> > > > > > > > > > > >
> > > > > > > > > > > > > - there is a perf degradation (+200%) in spark runner starting on 04/10 for all the queries:
> > > > > > > > > > > > https://apache-beam-testing.appspot.com/explore?dashboard=5138380291571712
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > Sorry Andrew for the added work
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > Etienne
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > Le lundi 22 avril 2019 à 12:21 -0700, Andrew Pilloud a écrit :
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > I signed the wheels files and updated the build process to not require giving travis apache
> > > > > > > > > > > > credentials. (You should probably change your password if you haven't already.)
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > Andrew
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > On Mon, Apr 22, 2019 at 12:18 PM Ahmet Altay <al...@google.com> wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > +1 (binding)
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > Verified the python 2 wheel files with quick start examples.
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > On Mon, Apr 22, 2019 at 11:26 AM Ahmet Altay <al...@google.com> wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > I built the wheel files. They are in the usual place along with other python artifacts. I will
> > > > > > > > > > > > test them a bit and update here. Could someone else please try the wheel files as well?
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > Andrew, could you sign and hash the wheel files?
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > On Mon, Apr 22, 2019 at 10:11 AM Ahmet Altay <al...@google.com> wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > I verified
> > > > > > > > > > > >
> > > > > > > > > > > > > - signatures and hashes.
> > > > > > > > > > > >
> > > > > > > > > > > > > - python streaming quickstart guide
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > I would like to verify the wheel files before voting. Please let us know when they are ready.
> > > > > > > > > > > > Also, if you need help with building wheel files I can help/build.
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > Ahmet
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > On Mon, Apr 22, 2019 at 3:33 AM Maximilian Michels <mx...@apache.org> wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > +1 (binding)
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > Found a minor bug while testing, but not a blocker:
> > > > > > > > > > > >
> > > > > > > > > > > > > https://jira.apache.org/jira/browse/BEAM-7128
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > Thanks,
> > > > > > > > > > > >
> > > > > > > > > > > > > Max
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > On 20.04.19 23:02, Pablo Estrada wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > > +1
> > > > > > > > > > > >
> > > > > > > > > > > > > > Ran SQL postcommit, and Dataflow Portability Java validatesrunner tests.
> > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > > -P.
> > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > > On Wed, Apr 17, 2019 at 1:38 AM Jean-Baptiste Onofré <jb@nanthrax.net
> > > > > > > > > > > >
> > > > > > > > > > > > > > <ma...@nanthrax.net>> wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > > +1 (binding)
> > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > > Quickly checked with beam-samples.
> > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > > Regards
> > > > > > > > > > > >
> > > > > > > > > > > > > > JB
> > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > > On 16/04/2019 00:50, Andrew Pilloud wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > > > Hi everyone,
> > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > > > Please review and vote on the release candidate #4 for the version
> > > > > > > > > > > >
> > > > > > > > > > > > > > > 2.12.0, as follows:
> > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > > > [ ] +1, Approve the release
> > > > > > > > > > > >
> > > > > > > > > > > > > > > [ ] -1, Do not approve the release (please provide specific comments)
> > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > > > The complete staging area is available for your review, which
> > > > > > > > > > > >
> > > > > > > > > > > > > > includes:
> > > > > > > > > > > >
> > > > > > > > > > > > > > > * JIRA release notes [1],
> > > > > > > > > > > >
> > > > > > > > > > > > > > > * the official Apache source release to be deployed to
> > > > > > > > > > > >
> > > > > > > > > > > > > > dist.apache.org <http://dist.apache.org>
> > > > > > > > > > > >
> > > > > > > > > > > > > > > <http://dist.apache.org> [2], which is signed with the key with
> > > > > > > > > > > >
> > > > > > > > > > > > > > > fingerprint 9E7CEC0661EFD610B632C610AE8FE17F9F8AE3D4 [3],
> > > > > > > > > > > >
> > > > > > > > > > > > > > > * all artifacts to be deployed to the Maven Central Repository [4],
> > > > > > > > > > > >
> > > > > > > > > > > > > > > * source code tag "v2.12.0-RC4" [5],
> > > > > > > > > > > >
> > > > > > > > > > > > > > > * website pull request listing the release [6], publishing the API
> > > > > > > > > > > >
> > > > > > > > > > > > > > > reference manual [7], and the blog post [8].
> > > > > > > > > > > >
> > > > > > > > > > > > > > > * Java artifacts were built with Gradle/5.2.1 and OpenJDK/Oracle JDK
> > > > > > > > > > > >
> > > > > > > > > > > > > > > 1.8.0_181.
> > > > > > > > > > > >
> > > > > > > > > > > > > > > * Python artifacts are deployed along with the source release to the
> > > > > > > > > > > >
> > > > > > > > > > > > > > > dist.apache.org <http://dist.apache.org> <http://dist.apache.org>
> > > > > > > > > > > >
> > > > > > > > > > > > > > [2].
> > > > > > > > > > > >
> > > > > > > > > > > > > > > * Validation sheet with a tab for 2.12.0 release to help with
> > > > > > > > > > > >
> > > > > > > > > > > > > > validation
> > > > > > > > > > > >
> > > > > > > > > > > > > > > [9].
> > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > > > The vote will be open for at least 72 hours. It is adopted by
> > > > > > > > > > > >
> > > > > > > > > > > > > > majority
> > > > > > > > > > > >
> > > > > > > > > > > > > > > approval, with at least 3 PMC affirmative votes.
> > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > > > Thanks,
> > > > > > > > > > > >
> > > > > > > > > > > > > > > Andrew
> > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > > > 1]
> > > > > > > > > > > >
> > > > > > > > > > > > > > https://jira.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12344944
> > > > > > > > > > > >
> > > > > > > > > > > > > > > [2] https://dist.apache.org/repos/dist/dev/beam/2.12.0/
> > > > > > > > > > > >
> > > > > > > > > > > > > > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> > > > > > > > > > > >
> > > > > > > > > > > > > > > [4]
> > > > > > > > > > > >
> > > > > > > > > > > > > > https://repository.apache.org/content/repositories/orgapachebeam-1068/
> > > > > > > > > > > >
> > > > > > > > > > > > > > > [5] https://github.com/apache/beam/tree/v2.12.0-RC4
> > > > > > > > > > > >
> > > > > > > > > > > > > > > [6] https://github.com/apache/beam/pull/8215
> > > > > > > > > > > >
> > > > > > > > > > > > > > > [7] https://github.com/apache/beam-site/pull/588
> > > > > > > > > > > >
> > > > > > > > > > > > > > > [8] https://github.com/apache/beam/pull/8314
> > > > > > > > > > > >
> > > > > > > > > > > > > > > [9]
> > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984
> > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > > --
> > > > > > > > > > > >
> > > > > > > > > > > > > > Jean-Baptiste Onofré
> > > > > > > > > > > >
> > > > > > > > > > > > > > jbonofre@apache.org <ma...@apache.org>
> > > > > > > > > > > >
> > > > > > > > > > > > > > http://blog.nanthrax.net
> > > > > > > > > > > >
> > > > > > > > > > > > > > Talend - http://www.talend.com
> > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > >
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Reuven Lax <re...@google.com>.
So the regression is not in RC4? In that case consider me a +1
On Wed, Apr 24, 2019 at 3:00 PM Andrew Pilloud <ap...@google.com> wrote:
> Please consider the vote for RC4 resumed.
>
> We currently have two -1 votes based on Nexmark issues that did not start
> until after the release branch was cut[1].
>
> Etienne and Reuven: Can you verify and update your votes as appropriate?
>
> Andrew
>
> [1]
> https://github.com/apache/beam/commit/1d9daf1aca101fa5a194cbbba969886734e08902
>
> On Wed, Apr 24, 2019 at 10:52 AM Andrew Pilloud <ap...@google.com>
> wrote:
>
>> Ok, it looks like I messed up here. I went to revert Reuven's change and
>> it wasn't on the branch. I was previously looking at RC1, which was a week
>> after actual cut off of master.
>>
>> The actual cut was the morning of March 28, with the last commit being
>> March 27. We have good performance runs for the commit of the cut.
>>
>> Can we reopen the vote on RC4 and revisit the -1 votes?
>>
>> Andrew
>>
>> On Wed, Apr 24, 2019 at 8:59 AM Ahmet Altay <al...@google.com> wrote:
>>
>>> Yes, I see these benchmarks. Thank you.
>>>
>>> My point was, the validation sheet (
>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984)
>>> only mentions Nextmark for Direct runner as something to be validated. I am
>>> guessing it means for RCs validate that nexmark can run on DirectRunner. I
>>> am suggesting adding the check of the dashboard to the validation
>>> spreadsheet.
>>>
>>> On Wed, Apr 24, 2019 at 2:03 AM Etienne Chauchot <ec...@apache.org>
>>> wrote:
>>>
>>>> Ahmet,
>>>> Don't you see all Nexmark dashboards visible in the joint image at
>>>> <https://apache-beam-testing.appspot.com/dashboard-admin>
>>>> https://apache-beam-testing.appspot.com/dashboard-admin?
>>>>
>>>> Etienne
>>>> Le mardi 23 avril 2019 à 15:24 -0700, Ahmet Altay a écrit :
>>>>
>>>>
>>>>
>>>> On Tue, Apr 23, 2019 at 3:21 PM Kenneth Knowles <ke...@apache.org>
>>>> wrote:
>>>>
>>>> What can we do to make this part of day-to-day workflow instead of
>>>> finding out during release validation? Was this just a failing test that
>>>> was missed?
>>>>
>>>> Kenn
>>>>
>>>> On Tue, Apr 23, 2019 at 3:02 PM Andrew Pilloud <ap...@google.com>
>>>> wrote:
>>>>
>>>> It looks like Java Nexmark tests are on the validation sheet but we've
>>>> missed it the last few releases. Thanks for checking it Etienne!
>>>>
>>>>
>>>>
>>>> I only see nexmark perf on direct runner. Is this even a thing? Should
>>>> we also add major runners to that list?
>>>>
>>>>
>>>> Does the current release process require everything to be tested before
>>>> making the release final?
>>>>
>>>>
>>>>
>>>> I do not think we agreed on this. Previous releases went out with
>>>> partial validations. It would be good to validate everything for the final
>>>> release.
>>>>
>>>>
>>>>
>>>> I fully agree with you on point 2. All of these issues were in RC1 and
>>>> could have been fixed for RC2.
>>>>
>>>> Andrew
>>>>
>>>> On Tue, Apr 23, 2019 at 2:58 PM Ahmet Altay <al...@google.com> wrote:
>>>>
>>>> Thank you Andrew. I will suggest two improvements to the release
>>>> process:
>>>> 1. We can include benchmarks in the validation sheet ("Apache Beam
>>>> Release Acceptance Criteria"). They are used part of the validation process
>>>> and we can ensure that we check those for each release.
>>>> 2. For RC validation, we can continue to exhaustively validate each RC
>>>> even after the first -1 vote. Otherwise we end up with not discovering all
>>>> issues in a given RC and find them a successive RC, increasing the number
>>>> of iterations required.
>>>>
>>>>
>>>> On Tue, Apr 23, 2019 at 2:11 PM Andrew Pilloud <ap...@google.com>
>>>> wrote:
>>>>
>>>> Please consider the vote for RC4 canceled. I'll quickly follow up with
>>>> a new RC.
>>>>
>>>> Thanks for the complete testing everyone!
>>>>
>>>> Andrew
>>>>
>>>> On Tue, Apr 23, 2019 at 2:06 PM Reuven Lax <re...@google.com> wrote:
>>>>
>>>> -1
>>>>
>>>> we need to cherry pick pr/8325 and pr/8385 to fix the above issue
>>>>
>>>> On Tue, Apr 23, 2019 at 1:48 PM Andrew Pilloud <ap...@google.com>
>>>> wrote:
>>>>
>>>> I believe the breakage of Nexmark on Dataflow is
>>>> https://issues.apache.org/jira/browse/BEAM-7002, which went in before
>>>> the release was cut. It looks like this might be a release blocker based on
>>>> the fix: https://github.com/apache/beam/pull/8325.
>>>>
>>>> The degraded performance is after the release is cut, so we should be
>>>> good there.
>>>>
>>>> Andrew
>>>>
>>>> On Tue, Apr 23, 2019 at 8:44 AM Ismaël Mejía <ie...@gmail.com> wrote:
>>>>
>>>> Etienne RC1 vote happened in 04/03 and there have not been any cherry
>>>> picks on the spark runner afterwards so if there is a commit that
>>>> degraded performance around 04/10 it is not part of the release we are
>>>> voting, so please consider reverting your -1.
>>>>
>>>> However the issue you are reporting looks important, from a quick look
>>>> I am guessing it could be related to BEAM-5775 that was merged on
>>>> 12/04 however the performance regressions started happening since
>>>> 09/04 so it could be unrelated. Maybe it could be due to changes in
>>>> our infrastructure. Maybe the change in the workers to be tracked, but
>>>> definitely not a release blocker at least for the Spark runner.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, Apr 23, 2019 at 5:12 PM Etienne Chauchot <ec...@apache.org>
>>>> wrote:
>>>> >
>>>> > Hi guys ,
>>>> > I will vote -1 (binding) on this RC (although degradation is before
>>>> RC4 cut date). I took a look at Nexmark graphs for the 3 major runners :
>>>> > - there seem to have functional regressions on Dataflow:
>>>> https://apache-beam-testing.appspot.com/explore?dashboard=5647201107705856
>>>> . 13 queries fail in batch mode starting on 04/17
>>>> > - there is a perf degradation (+200%) in spark runner starting on
>>>> 04/10 for all the queries:
>>>> https://apache-beam-testing.appspot.com/explore?dashboard=5138380291571712
>>>> >
>>>> > Sorry Andrew for the added work
>>>> >
>>>> > Etienne
>>>> >
>>>> > Le lundi 22 avril 2019 à 12:21 -0700, Andrew Pilloud a écrit :
>>>> >
>>>> > I signed the wheels files and updated the build process to not
>>>> require giving travis apache credentials. (You should probably change your
>>>> password if you haven't already.)
>>>> >
>>>> > Andrew
>>>> >
>>>> > On Mon, Apr 22, 2019 at 12:18 PM Ahmet Altay <al...@google.com>
>>>> wrote:
>>>> >
>>>> > +1 (binding)
>>>> >
>>>> > Verified the python 2 wheel files with quick start examples.
>>>> >
>>>> > On Mon, Apr 22, 2019 at 11:26 AM Ahmet Altay <al...@google.com>
>>>> wrote:
>>>> >
>>>> > I built the wheel files. They are in the usual place along with other
>>>> python artifacts. I will test them a bit and update here. Could someone
>>>> else please try the wheel files as well?
>>>> >
>>>> > Andrew, could you sign and hash the wheel files?
>>>> >
>>>> > On Mon, Apr 22, 2019 at 10:11 AM Ahmet Altay <al...@google.com>
>>>> wrote:
>>>> >
>>>> > I verified
>>>> > - signatures and hashes.
>>>> > - python streaming quickstart guide
>>>> >
>>>> > I would like to verify the wheel files before voting. Please let us
>>>> know when they are ready. Also, if you need help with building wheel files
>>>> I can help/build.
>>>> >
>>>> > Ahmet
>>>> >
>>>> > On Mon, Apr 22, 2019 at 3:33 AM Maximilian Michels <mx...@apache.org>
>>>> wrote:
>>>> >
>>>> > +1 (binding)
>>>> >
>>>> > Found a minor bug while testing, but not a blocker:
>>>> > https://jira.apache.org/jira/browse/BEAM-7128
>>>> >
>>>> > Thanks,
>>>> > Max
>>>> >
>>>> > On 20.04.19 23:02, Pablo Estrada wrote:
>>>> > > +1
>>>> > > Ran SQL postcommit, and Dataflow Portability Java validatesrunner
>>>> tests.
>>>> > >
>>>> > > -P.
>>>> > >
>>>> > > On Wed, Apr 17, 2019 at 1:38 AM Jean-Baptiste Onofré <
>>>> jb@nanthrax.net
>>>> > > <ma...@nanthrax.net>> wrote:
>>>> > >
>>>> > > +1 (binding)
>>>> > >
>>>> > > Quickly checked with beam-samples.
>>>> > >
>>>> > > Regards
>>>> > > JB
>>>> > >
>>>> > > On 16/04/2019 00:50, Andrew Pilloud wrote:
>>>> > > > Hi everyone,
>>>> > > >
>>>> > > > Please review and vote on the release candidate #4 for the
>>>> version
>>>> > > > 2.12.0, as follows:
>>>> > > >
>>>> > > > [ ] +1, Approve the release
>>>> > > > [ ] -1, Do not approve the release (please provide specific
>>>> comments)
>>>> > > >
>>>> > > > The complete staging area is available for your review, which
>>>> > > includes:
>>>> > > > * JIRA release notes [1],
>>>> > > > * the official Apache source release to be deployed to
>>>> > > dist.apache.org <http://dist.apache.org>
>>>> > > > <http://dist.apache.org> [2], which is signed with the key
>>>> with
>>>> > > > fingerprint 9E7CEC0661EFD610B632C610AE8FE17F9F8AE3D4 [3],
>>>> > > > * all artifacts to be deployed to the Maven Central
>>>> Repository [4],
>>>> > > > * source code tag "v2.12.0-RC4" [5],
>>>> > > > * website pull request listing the release [6], publishing
>>>> the API
>>>> > > > reference manual [7], and the blog post [8].
>>>> > > > * Java artifacts were built with Gradle/5.2.1 and
>>>> OpenJDK/Oracle JDK
>>>> > > > 1.8.0_181.
>>>> > > > * Python artifacts are deployed along with the source
>>>> release to the
>>>> > > > dist.apache.org <http://dist.apache.org> <
>>>> http://dist.apache.org>
>>>> > > [2].
>>>> > > > * Validation sheet with a tab for 2.12.0 release to help with
>>>> > > validation
>>>> > > > [9].
>>>> > > >
>>>> > > > The vote will be open for at least 72 hours. It is adopted by
>>>> > > majority
>>>> > > > approval, with at least 3 PMC affirmative votes.
>>>> > > >
>>>> > > > Thanks,
>>>> > > > Andrew
>>>> > > >
>>>> > > > 1]
>>>> > >
>>>> https://jira.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12344944
>>>> > > > [2] https://dist.apache.org/repos/dist/dev/beam/2.12.0/
>>>> > > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>>> > > > [4]
>>>> > >
>>>> https://repository.apache.org/content/repositories/orgapachebeam-1068/
>>>> > > > [5] https://github.com/apache/beam/tree/v2.12.0-RC4
>>>> > > > [6] https://github.com/apache/beam/pull/8215
>>>> > > > [7] https://github.com/apache/beam-site/pull/588
>>>> > > > [8] https://github.com/apache/beam/pull/8314
>>>> > > > [9]
>>>> > >
>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984
>>>> > >
>>>> > > --
>>>> > > Jean-Baptiste Onofré
>>>> > > jbonofre@apache.org <ma...@apache.org>
>>>> > > http://blog.nanthrax.net
>>>> > > Talend - http://www.talend.com
>>>> > >
>>>>
>>>>
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Andrew Pilloud <ap...@google.com>.
Please consider the vote for RC4 resumed.
We currently have two -1 votes based on Nexmark issues that did not start
until after the release branch was cut[1].
Etienne and Reuven: Can you verify and update your votes as appropriate?
Andrew
[1]
https://github.com/apache/beam/commit/1d9daf1aca101fa5a194cbbba969886734e08902
On Wed, Apr 24, 2019 at 10:52 AM Andrew Pilloud <ap...@google.com> wrote:
> Ok, it looks like I messed up here. I went to revert Reuven's change and
> it wasn't on the branch. I was previously looking at RC1, which was a week
> after actual cut off of master.
>
> The actual cut was the morning of March 28, with the last commit being
> March 27. We have good performance runs for the commit of the cut.
>
> Can we reopen the vote on RC4 and revisit the -1 votes?
>
> Andrew
>
> On Wed, Apr 24, 2019 at 8:59 AM Ahmet Altay <al...@google.com> wrote:
>
>> Yes, I see these benchmarks. Thank you.
>>
>> My point was, the validation sheet (
>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984)
>> only mentions Nextmark for Direct runner as something to be validated. I am
>> guessing it means for RCs validate that nexmark can run on DirectRunner. I
>> am suggesting adding the check of the dashboard to the validation
>> spreadsheet.
>>
>> On Wed, Apr 24, 2019 at 2:03 AM Etienne Chauchot <ec...@apache.org>
>> wrote:
>>
>>> Ahmet,
>>> Don't you see all Nexmark dashboards visible in the joint image at
>>> <https://apache-beam-testing.appspot.com/dashboard-admin>
>>> https://apache-beam-testing.appspot.com/dashboard-admin?
>>>
>>> Etienne
>>> Le mardi 23 avril 2019 à 15:24 -0700, Ahmet Altay a écrit :
>>>
>>>
>>>
>>> On Tue, Apr 23, 2019 at 3:21 PM Kenneth Knowles <ke...@apache.org> wrote:
>>>
>>> What can we do to make this part of day-to-day workflow instead of
>>> finding out during release validation? Was this just a failing test that
>>> was missed?
>>>
>>> Kenn
>>>
>>> On Tue, Apr 23, 2019 at 3:02 PM Andrew Pilloud <ap...@google.com>
>>> wrote:
>>>
>>> It looks like Java Nexmark tests are on the validation sheet but we've
>>> missed it the last few releases. Thanks for checking it Etienne!
>>>
>>>
>>>
>>> I only see nexmark perf on direct runner. Is this even a thing? Should
>>> we also add major runners to that list?
>>>
>>>
>>> Does the current release process require everything to be tested before
>>> making the release final?
>>>
>>>
>>>
>>> I do not think we agreed on this. Previous releases went out with
>>> partial validations. It would be good to validate everything for the final
>>> release.
>>>
>>>
>>>
>>> I fully agree with you on point 2. All of these issues were in RC1 and
>>> could have been fixed for RC2.
>>>
>>> Andrew
>>>
>>> On Tue, Apr 23, 2019 at 2:58 PM Ahmet Altay <al...@google.com> wrote:
>>>
>>> Thank you Andrew. I will suggest two improvements to the release process:
>>> 1. We can include benchmarks in the validation sheet ("Apache Beam
>>> Release Acceptance Criteria"). They are used part of the validation process
>>> and we can ensure that we check those for each release.
>>> 2. For RC validation, we can continue to exhaustively validate each RC
>>> even after the first -1 vote. Otherwise we end up with not discovering all
>>> issues in a given RC and find them a successive RC, increasing the number
>>> of iterations required.
>>>
>>>
>>> On Tue, Apr 23, 2019 at 2:11 PM Andrew Pilloud <ap...@google.com>
>>> wrote:
>>>
>>> Please consider the vote for RC4 canceled. I'll quickly follow up with a
>>> new RC.
>>>
>>> Thanks for the complete testing everyone!
>>>
>>> Andrew
>>>
>>> On Tue, Apr 23, 2019 at 2:06 PM Reuven Lax <re...@google.com> wrote:
>>>
>>> -1
>>>
>>> we need to cherry pick pr/8325 and pr/8385 to fix the above issue
>>>
>>> On Tue, Apr 23, 2019 at 1:48 PM Andrew Pilloud <ap...@google.com>
>>> wrote:
>>>
>>> I believe the breakage of Nexmark on Dataflow is
>>> https://issues.apache.org/jira/browse/BEAM-7002, which went in before
>>> the release was cut. It looks like this might be a release blocker based on
>>> the fix: https://github.com/apache/beam/pull/8325.
>>>
>>> The degraded performance is after the release is cut, so we should be
>>> good there.
>>>
>>> Andrew
>>>
>>> On Tue, Apr 23, 2019 at 8:44 AM Ismaël Mejía <ie...@gmail.com> wrote:
>>>
>>> Etienne RC1 vote happened in 04/03 and there have not been any cherry
>>> picks on the spark runner afterwards so if there is a commit that
>>> degraded performance around 04/10 it is not part of the release we are
>>> voting, so please consider reverting your -1.
>>>
>>> However the issue you are reporting looks important, from a quick look
>>> I am guessing it could be related to BEAM-5775 that was merged on
>>> 12/04 however the performance regressions started happening since
>>> 09/04 so it could be unrelated. Maybe it could be due to changes in
>>> our infrastructure. Maybe the change in the workers to be tracked, but
>>> definitely not a release blocker at least for the Spark runner.
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Apr 23, 2019 at 5:12 PM Etienne Chauchot <ec...@apache.org>
>>> wrote:
>>> >
>>> > Hi guys ,
>>> > I will vote -1 (binding) on this RC (although degradation is before
>>> RC4 cut date). I took a look at Nexmark graphs for the 3 major runners :
>>> > - there seem to have functional regressions on Dataflow:
>>> https://apache-beam-testing.appspot.com/explore?dashboard=5647201107705856
>>> . 13 queries fail in batch mode starting on 04/17
>>> > - there is a perf degradation (+200%) in spark runner starting on
>>> 04/10 for all the queries:
>>> https://apache-beam-testing.appspot.com/explore?dashboard=5138380291571712
>>> >
>>> > Sorry Andrew for the added work
>>> >
>>> > Etienne
>>> >
>>> > Le lundi 22 avril 2019 à 12:21 -0700, Andrew Pilloud a écrit :
>>> >
>>> > I signed the wheels files and updated the build process to not require
>>> giving travis apache credentials. (You should probably change your password
>>> if you haven't already.)
>>> >
>>> > Andrew
>>> >
>>> > On Mon, Apr 22, 2019 at 12:18 PM Ahmet Altay <al...@google.com> wrote:
>>> >
>>> > +1 (binding)
>>> >
>>> > Verified the python 2 wheel files with quick start examples.
>>> >
>>> > On Mon, Apr 22, 2019 at 11:26 AM Ahmet Altay <al...@google.com> wrote:
>>> >
>>> > I built the wheel files. They are in the usual place along with other
>>> python artifacts. I will test them a bit and update here. Could someone
>>> else please try the wheel files as well?
>>> >
>>> > Andrew, could you sign and hash the wheel files?
>>> >
>>> > On Mon, Apr 22, 2019 at 10:11 AM Ahmet Altay <al...@google.com> wrote:
>>> >
>>> > I verified
>>> > - signatures and hashes.
>>> > - python streaming quickstart guide
>>> >
>>> > I would like to verify the wheel files before voting. Please let us
>>> know when they are ready. Also, if you need help with building wheel files
>>> I can help/build.
>>> >
>>> > Ahmet
>>> >
>>> > On Mon, Apr 22, 2019 at 3:33 AM Maximilian Michels <mx...@apache.org>
>>> wrote:
>>> >
>>> > +1 (binding)
>>> >
>>> > Found a minor bug while testing, but not a blocker:
>>> > https://jira.apache.org/jira/browse/BEAM-7128
>>> >
>>> > Thanks,
>>> > Max
>>> >
>>> > On 20.04.19 23:02, Pablo Estrada wrote:
>>> > > +1
>>> > > Ran SQL postcommit, and Dataflow Portability Java validatesrunner
>>> tests.
>>> > >
>>> > > -P.
>>> > >
>>> > > On Wed, Apr 17, 2019 at 1:38 AM Jean-Baptiste Onofré <
>>> jb@nanthrax.net
>>> > > <ma...@nanthrax.net>> wrote:
>>> > >
>>> > > +1 (binding)
>>> > >
>>> > > Quickly checked with beam-samples.
>>> > >
>>> > > Regards
>>> > > JB
>>> > >
>>> > > On 16/04/2019 00:50, Andrew Pilloud wrote:
>>> > > > Hi everyone,
>>> > > >
>>> > > > Please review and vote on the release candidate #4 for the
>>> version
>>> > > > 2.12.0, as follows:
>>> > > >
>>> > > > [ ] +1, Approve the release
>>> > > > [ ] -1, Do not approve the release (please provide specific
>>> comments)
>>> > > >
>>> > > > The complete staging area is available for your review, which
>>> > > includes:
>>> > > > * JIRA release notes [1],
>>> > > > * the official Apache source release to be deployed to
>>> > > dist.apache.org <http://dist.apache.org>
>>> > > > <http://dist.apache.org> [2], which is signed with the key
>>> with
>>> > > > fingerprint 9E7CEC0661EFD610B632C610AE8FE17F9F8AE3D4 [3],
>>> > > > * all artifacts to be deployed to the Maven Central
>>> Repository [4],
>>> > > > * source code tag "v2.12.0-RC4" [5],
>>> > > > * website pull request listing the release [6], publishing
>>> the API
>>> > > > reference manual [7], and the blog post [8].
>>> > > > * Java artifacts were built with Gradle/5.2.1 and
>>> OpenJDK/Oracle JDK
>>> > > > 1.8.0_181.
>>> > > > * Python artifacts are deployed along with the source release
>>> to the
>>> > > > dist.apache.org <http://dist.apache.org> <
>>> http://dist.apache.org>
>>> > > [2].
>>> > > > * Validation sheet with a tab for 2.12.0 release to help with
>>> > > validation
>>> > > > [9].
>>> > > >
>>> > > > The vote will be open for at least 72 hours. It is adopted by
>>> > > majority
>>> > > > approval, with at least 3 PMC affirmative votes.
>>> > > >
>>> > > > Thanks,
>>> > > > Andrew
>>> > > >
>>> > > > 1]
>>> > >
>>> https://jira.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12344944
>>> > > > [2] https://dist.apache.org/repos/dist/dev/beam/2.12.0/
>>> > > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>> > > > [4]
>>> > >
>>> https://repository.apache.org/content/repositories/orgapachebeam-1068/
>>> > > > [5] https://github.com/apache/beam/tree/v2.12.0-RC4
>>> > > > [6] https://github.com/apache/beam/pull/8215
>>> > > > [7] https://github.com/apache/beam-site/pull/588
>>> > > > [8] https://github.com/apache/beam/pull/8314
>>> > > > [9]
>>> > >
>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984
>>> > >
>>> > > --
>>> > > Jean-Baptiste Onofré
>>> > > jbonofre@apache.org <ma...@apache.org>
>>> > > http://blog.nanthrax.net
>>> > > Talend - http://www.talend.com
>>> > >
>>>
>>>
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Andrew Pilloud <ap...@google.com>.
Ok, it looks like I messed up here. I went to revert Reuven's change and it
wasn't on the branch. I was previously looking at RC1, which was a week
after actual cut off of master.
The actual cut was the morning of March 28, with the last commit being
March 27. We have good performance runs for the commit of the cut.
Can we reopen the vote on RC4 and revisit the -1 votes?
Andrew
On Wed, Apr 24, 2019 at 8:59 AM Ahmet Altay <al...@google.com> wrote:
> Yes, I see these benchmarks. Thank you.
>
> My point was, the validation sheet (
> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984)
> only mentions Nextmark for Direct runner as something to be validated. I am
> guessing it means for RCs validate that nexmark can run on DirectRunner. I
> am suggesting adding the check of the dashboard to the validation
> spreadsheet.
>
> On Wed, Apr 24, 2019 at 2:03 AM Etienne Chauchot <ec...@apache.org>
> wrote:
>
>> Ahmet,
>> Don't you see all Nexmark dashboards visible in the joint image at
>> <https://apache-beam-testing.appspot.com/dashboard-admin>
>> https://apache-beam-testing.appspot.com/dashboard-admin?
>>
>> Etienne
>> Le mardi 23 avril 2019 à 15:24 -0700, Ahmet Altay a écrit :
>>
>>
>>
>> On Tue, Apr 23, 2019 at 3:21 PM Kenneth Knowles <ke...@apache.org> wrote:
>>
>> What can we do to make this part of day-to-day workflow instead of
>> finding out during release validation? Was this just a failing test that
>> was missed?
>>
>> Kenn
>>
>> On Tue, Apr 23, 2019 at 3:02 PM Andrew Pilloud <ap...@google.com>
>> wrote:
>>
>> It looks like Java Nexmark tests are on the validation sheet but we've
>> missed it the last few releases. Thanks for checking it Etienne!
>>
>>
>>
>> I only see nexmark perf on direct runner. Is this even a thing? Should we
>> also add major runners to that list?
>>
>>
>> Does the current release process require everything to be tested before
>> making the release final?
>>
>>
>>
>> I do not think we agreed on this. Previous releases went out with partial
>> validations. It would be good to validate everything for the final release.
>>
>>
>>
>> I fully agree with you on point 2. All of these issues were in RC1 and
>> could have been fixed for RC2.
>>
>> Andrew
>>
>> On Tue, Apr 23, 2019 at 2:58 PM Ahmet Altay <al...@google.com> wrote:
>>
>> Thank you Andrew. I will suggest two improvements to the release process:
>> 1. We can include benchmarks in the validation sheet ("Apache Beam
>> Release Acceptance Criteria"). They are used part of the validation process
>> and we can ensure that we check those for each release.
>> 2. For RC validation, we can continue to exhaustively validate each RC
>> even after the first -1 vote. Otherwise we end up with not discovering all
>> issues in a given RC and find them a successive RC, increasing the number
>> of iterations required.
>>
>>
>> On Tue, Apr 23, 2019 at 2:11 PM Andrew Pilloud <ap...@google.com>
>> wrote:
>>
>> Please consider the vote for RC4 canceled. I'll quickly follow up with a
>> new RC.
>>
>> Thanks for the complete testing everyone!
>>
>> Andrew
>>
>> On Tue, Apr 23, 2019 at 2:06 PM Reuven Lax <re...@google.com> wrote:
>>
>> -1
>>
>> we need to cherry pick pr/8325 and pr/8385 to fix the above issue
>>
>> On Tue, Apr 23, 2019 at 1:48 PM Andrew Pilloud <ap...@google.com>
>> wrote:
>>
>> I believe the breakage of Nexmark on Dataflow is
>> https://issues.apache.org/jira/browse/BEAM-7002, which went in before
>> the release was cut. It looks like this might be a release blocker based on
>> the fix: https://github.com/apache/beam/pull/8325.
>>
>> The degraded performance is after the release is cut, so we should be
>> good there.
>>
>> Andrew
>>
>> On Tue, Apr 23, 2019 at 8:44 AM Ismaël Mejía <ie...@gmail.com> wrote:
>>
>> Etienne RC1 vote happened in 04/03 and there have not been any cherry
>> picks on the spark runner afterwards so if there is a commit that
>> degraded performance around 04/10 it is not part of the release we are
>> voting, so please consider reverting your -1.
>>
>> However the issue you are reporting looks important, from a quick look
>> I am guessing it could be related to BEAM-5775 that was merged on
>> 12/04 however the performance regressions started happening since
>> 09/04 so it could be unrelated. Maybe it could be due to changes in
>> our infrastructure. Maybe the change in the workers to be tracked, but
>> definitely not a release blocker at least for the Spark runner.
>>
>>
>>
>>
>>
>> On Tue, Apr 23, 2019 at 5:12 PM Etienne Chauchot <ec...@apache.org>
>> wrote:
>> >
>> > Hi guys ,
>> > I will vote -1 (binding) on this RC (although degradation is before RC4
>> cut date). I took a look at Nexmark graphs for the 3 major runners :
>> > - there seem to have functional regressions on Dataflow:
>> https://apache-beam-testing.appspot.com/explore?dashboard=5647201107705856
>> . 13 queries fail in batch mode starting on 04/17
>> > - there is a perf degradation (+200%) in spark runner starting on 04/10
>> for all the queries:
>> https://apache-beam-testing.appspot.com/explore?dashboard=5138380291571712
>> >
>> > Sorry Andrew for the added work
>> >
>> > Etienne
>> >
>> > Le lundi 22 avril 2019 à 12:21 -0700, Andrew Pilloud a écrit :
>> >
>> > I signed the wheels files and updated the build process to not require
>> giving travis apache credentials. (You should probably change your password
>> if you haven't already.)
>> >
>> > Andrew
>> >
>> > On Mon, Apr 22, 2019 at 12:18 PM Ahmet Altay <al...@google.com> wrote:
>> >
>> > +1 (binding)
>> >
>> > Verified the python 2 wheel files with quick start examples.
>> >
>> > On Mon, Apr 22, 2019 at 11:26 AM Ahmet Altay <al...@google.com> wrote:
>> >
>> > I built the wheel files. They are in the usual place along with other
>> python artifacts. I will test them a bit and update here. Could someone
>> else please try the wheel files as well?
>> >
>> > Andrew, could you sign and hash the wheel files?
>> >
>> > On Mon, Apr 22, 2019 at 10:11 AM Ahmet Altay <al...@google.com> wrote:
>> >
>> > I verified
>> > - signatures and hashes.
>> > - python streaming quickstart guide
>> >
>> > I would like to verify the wheel files before voting. Please let us
>> know when they are ready. Also, if you need help with building wheel files
>> I can help/build.
>> >
>> > Ahmet
>> >
>> > On Mon, Apr 22, 2019 at 3:33 AM Maximilian Michels <mx...@apache.org>
>> wrote:
>> >
>> > +1 (binding)
>> >
>> > Found a minor bug while testing, but not a blocker:
>> > https://jira.apache.org/jira/browse/BEAM-7128
>> >
>> > Thanks,
>> > Max
>> >
>> > On 20.04.19 23:02, Pablo Estrada wrote:
>> > > +1
>> > > Ran SQL postcommit, and Dataflow Portability Java validatesrunner
>> tests.
>> > >
>> > > -P.
>> > >
>> > > On Wed, Apr 17, 2019 at 1:38 AM Jean-Baptiste Onofré <jb@nanthrax.net
>> > > <ma...@nanthrax.net>> wrote:
>> > >
>> > > +1 (binding)
>> > >
>> > > Quickly checked with beam-samples.
>> > >
>> > > Regards
>> > > JB
>> > >
>> > > On 16/04/2019 00:50, Andrew Pilloud wrote:
>> > > > Hi everyone,
>> > > >
>> > > > Please review and vote on the release candidate #4 for the
>> version
>> > > > 2.12.0, as follows:
>> > > >
>> > > > [ ] +1, Approve the release
>> > > > [ ] -1, Do not approve the release (please provide specific
>> comments)
>> > > >
>> > > > The complete staging area is available for your review, which
>> > > includes:
>> > > > * JIRA release notes [1],
>> > > > * the official Apache source release to be deployed to
>> > > dist.apache.org <http://dist.apache.org>
>> > > > <http://dist.apache.org> [2], which is signed with the key
>> with
>> > > > fingerprint 9E7CEC0661EFD610B632C610AE8FE17F9F8AE3D4 [3],
>> > > > * all artifacts to be deployed to the Maven Central Repository
>> [4],
>> > > > * source code tag "v2.12.0-RC4" [5],
>> > > > * website pull request listing the release [6], publishing the
>> API
>> > > > reference manual [7], and the blog post [8].
>> > > > * Java artifacts were built with Gradle/5.2.1 and
>> OpenJDK/Oracle JDK
>> > > > 1.8.0_181.
>> > > > * Python artifacts are deployed along with the source release
>> to the
>> > > > dist.apache.org <http://dist.apache.org> <
>> http://dist.apache.org>
>> > > [2].
>> > > > * Validation sheet with a tab for 2.12.0 release to help with
>> > > validation
>> > > > [9].
>> > > >
>> > > > The vote will be open for at least 72 hours. It is adopted by
>> > > majority
>> > > > approval, with at least 3 PMC affirmative votes.
>> > > >
>> > > > Thanks,
>> > > > Andrew
>> > > >
>> > > > 1]
>> > >
>> https://jira.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12344944
>> > > > [2] https://dist.apache.org/repos/dist/dev/beam/2.12.0/
>> > > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>> > > > [4]
>> > >
>> https://repository.apache.org/content/repositories/orgapachebeam-1068/
>> > > > [5] https://github.com/apache/beam/tree/v2.12.0-RC4
>> > > > [6] https://github.com/apache/beam/pull/8215
>> > > > [7] https://github.com/apache/beam-site/pull/588
>> > > > [8] https://github.com/apache/beam/pull/8314
>> > > > [9]
>> > >
>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984
>> > >
>> > > --
>> > > Jean-Baptiste Onofré
>> > > jbonofre@apache.org <ma...@apache.org>
>> > > http://blog.nanthrax.net
>> > > Talend - http://www.talend.com
>> > >
>>
>>
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Ahmet Altay <al...@google.com>.
Yes, I see these benchmarks. Thank you.
My point was, the validation sheet (
https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984)
only mentions Nextmark for Direct runner as something to be validated. I am
guessing it means for RCs validate that nexmark can run on DirectRunner. I
am suggesting adding the check of the dashboard to the validation
spreadsheet.
On Wed, Apr 24, 2019 at 2:03 AM Etienne Chauchot <ec...@apache.org>
wrote:
> Ahmet,
> Don't you see all Nexmark dashboards visible in the joint image at
> <https://apache-beam-testing.appspot.com/dashboard-admin>
> https://apache-beam-testing.appspot.com/dashboard-admin?
>
> Etienne
> Le mardi 23 avril 2019 à 15:24 -0700, Ahmet Altay a écrit :
>
>
>
> On Tue, Apr 23, 2019 at 3:21 PM Kenneth Knowles <ke...@apache.org> wrote:
>
> What can we do to make this part of day-to-day workflow instead of finding
> out during release validation? Was this just a failing test that was missed?
>
> Kenn
>
> On Tue, Apr 23, 2019 at 3:02 PM Andrew Pilloud <ap...@google.com>
> wrote:
>
> It looks like Java Nexmark tests are on the validation sheet but we've
> missed it the last few releases. Thanks for checking it Etienne!
>
>
>
> I only see nexmark perf on direct runner. Is this even a thing? Should we
> also add major runners to that list?
>
>
> Does the current release process require everything to be tested before
> making the release final?
>
>
>
> I do not think we agreed on this. Previous releases went out with partial
> validations. It would be good to validate everything for the final release.
>
>
>
> I fully agree with you on point 2. All of these issues were in RC1 and
> could have been fixed for RC2.
>
> Andrew
>
> On Tue, Apr 23, 2019 at 2:58 PM Ahmet Altay <al...@google.com> wrote:
>
> Thank you Andrew. I will suggest two improvements to the release process:
> 1. We can include benchmarks in the validation sheet ("Apache Beam Release
> Acceptance Criteria"). They are used part of the validation process and we
> can ensure that we check those for each release.
> 2. For RC validation, we can continue to exhaustively validate each RC
> even after the first -1 vote. Otherwise we end up with not discovering all
> issues in a given RC and find them a successive RC, increasing the number
> of iterations required.
>
>
> On Tue, Apr 23, 2019 at 2:11 PM Andrew Pilloud <ap...@google.com>
> wrote:
>
> Please consider the vote for RC4 canceled. I'll quickly follow up with a
> new RC.
>
> Thanks for the complete testing everyone!
>
> Andrew
>
> On Tue, Apr 23, 2019 at 2:06 PM Reuven Lax <re...@google.com> wrote:
>
> -1
>
> we need to cherry pick pr/8325 and pr/8385 to fix the above issue
>
> On Tue, Apr 23, 2019 at 1:48 PM Andrew Pilloud <ap...@google.com>
> wrote:
>
> I believe the breakage of Nexmark on Dataflow is
> https://issues.apache.org/jira/browse/BEAM-7002, which went in before the
> release was cut. It looks like this might be a release blocker based on the
> fix: https://github.com/apache/beam/pull/8325.
>
> The degraded performance is after the release is cut, so we should be good
> there.
>
> Andrew
>
> On Tue, Apr 23, 2019 at 8:44 AM Ismaël Mejía <ie...@gmail.com> wrote:
>
> Etienne RC1 vote happened in 04/03 and there have not been any cherry
> picks on the spark runner afterwards so if there is a commit that
> degraded performance around 04/10 it is not part of the release we are
> voting, so please consider reverting your -1.
>
> However the issue you are reporting looks important, from a quick look
> I am guessing it could be related to BEAM-5775 that was merged on
> 12/04 however the performance regressions started happening since
> 09/04 so it could be unrelated. Maybe it could be due to changes in
> our infrastructure. Maybe the change in the workers to be tracked, but
> definitely not a release blocker at least for the Spark runner.
>
>
>
>
>
> On Tue, Apr 23, 2019 at 5:12 PM Etienne Chauchot <ec...@apache.org>
> wrote:
> >
> > Hi guys ,
> > I will vote -1 (binding) on this RC (although degradation is before RC4
> cut date). I took a look at Nexmark graphs for the 3 major runners :
> > - there seem to have functional regressions on Dataflow:
> https://apache-beam-testing.appspot.com/explore?dashboard=5647201107705856
> . 13 queries fail in batch mode starting on 04/17
> > - there is a perf degradation (+200%) in spark runner starting on 04/10
> for all the queries:
> https://apache-beam-testing.appspot.com/explore?dashboard=5138380291571712
> >
> > Sorry Andrew for the added work
> >
> > Etienne
> >
> > Le lundi 22 avril 2019 à 12:21 -0700, Andrew Pilloud a écrit :
> >
> > I signed the wheels files and updated the build process to not require
> giving travis apache credentials. (You should probably change your password
> if you haven't already.)
> >
> > Andrew
> >
> > On Mon, Apr 22, 2019 at 12:18 PM Ahmet Altay <al...@google.com> wrote:
> >
> > +1 (binding)
> >
> > Verified the python 2 wheel files with quick start examples.
> >
> > On Mon, Apr 22, 2019 at 11:26 AM Ahmet Altay <al...@google.com> wrote:
> >
> > I built the wheel files. They are in the usual place along with other
> python artifacts. I will test them a bit and update here. Could someone
> else please try the wheel files as well?
> >
> > Andrew, could you sign and hash the wheel files?
> >
> > On Mon, Apr 22, 2019 at 10:11 AM Ahmet Altay <al...@google.com> wrote:
> >
> > I verified
> > - signatures and hashes.
> > - python streaming quickstart guide
> >
> > I would like to verify the wheel files before voting. Please let us know
> when they are ready. Also, if you need help with building wheel files I can
> help/build.
> >
> > Ahmet
> >
> > On Mon, Apr 22, 2019 at 3:33 AM Maximilian Michels <mx...@apache.org>
> wrote:
> >
> > +1 (binding)
> >
> > Found a minor bug while testing, but not a blocker:
> > https://jira.apache.org/jira/browse/BEAM-7128
> >
> > Thanks,
> > Max
> >
> > On 20.04.19 23:02, Pablo Estrada wrote:
> > > +1
> > > Ran SQL postcommit, and Dataflow Portability Java validatesrunner
> tests.
> > >
> > > -P.
> > >
> > > On Wed, Apr 17, 2019 at 1:38 AM Jean-Baptiste Onofré <jb@nanthrax.net
> > > <ma...@nanthrax.net>> wrote:
> > >
> > > +1 (binding)
> > >
> > > Quickly checked with beam-samples.
> > >
> > > Regards
> > > JB
> > >
> > > On 16/04/2019 00:50, Andrew Pilloud wrote:
> > > > Hi everyone,
> > > >
> > > > Please review and vote on the release candidate #4 for the
> version
> > > > 2.12.0, as follows:
> > > >
> > > > [ ] +1, Approve the release
> > > > [ ] -1, Do not approve the release (please provide specific
> comments)
> > > >
> > > > The complete staging area is available for your review, which
> > > includes:
> > > > * JIRA release notes [1],
> > > > * the official Apache source release to be deployed to
> > > dist.apache.org <http://dist.apache.org>
> > > > <http://dist.apache.org> [2], which is signed with the key with
> > > > fingerprint 9E7CEC0661EFD610B632C610AE8FE17F9F8AE3D4 [3],
> > > > * all artifacts to be deployed to the Maven Central Repository
> [4],
> > > > * source code tag "v2.12.0-RC4" [5],
> > > > * website pull request listing the release [6], publishing the
> API
> > > > reference manual [7], and the blog post [8].
> > > > * Java artifacts were built with Gradle/5.2.1 and
> OpenJDK/Oracle JDK
> > > > 1.8.0_181.
> > > > * Python artifacts are deployed along with the source release
> to the
> > > > dist.apache.org <http://dist.apache.org> <
> http://dist.apache.org>
> > > [2].
> > > > * Validation sheet with a tab for 2.12.0 release to help with
> > > validation
> > > > [9].
> > > >
> > > > The vote will be open for at least 72 hours. It is adopted by
> > > majority
> > > > approval, with at least 3 PMC affirmative votes.
> > > >
> > > > Thanks,
> > > > Andrew
> > > >
> > > > 1]
> > >
> https://jira.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12344944
> > > > [2] https://dist.apache.org/repos/dist/dev/beam/2.12.0/
> > > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> > > > [4]
> > >
> https://repository.apache.org/content/repositories/orgapachebeam-1068/
> > > > [5] https://github.com/apache/beam/tree/v2.12.0-RC4
> > > > [6] https://github.com/apache/beam/pull/8215
> > > > [7] https://github.com/apache/beam-site/pull/588
> > > > [8] https://github.com/apache/beam/pull/8314
> > > > [9]
> > >
> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984
> > >
> > > --
> > > Jean-Baptiste Onofré
> > > jbonofre@apache.org <ma...@apache.org>
> > > http://blog.nanthrax.net
> > > Talend - http://www.talend.com
> > >
>
>
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Etienne Chauchot <ec...@apache.org>.
Ahmet,Don't you see all Nexmark dashboards visible in the joint image at
https://apache-beam-testing.appspot.com/dashboard-admin?
EtienneLe mardi 23 avril 2019 à 15:24 -0700, Ahmet Altay a écrit :
> On Tue, Apr 23, 2019 at 3:21 PM Kenneth Knowles <ke...@apache.org> wrote:
> > What can we do to make this part of day-to-day workflow instead of finding out during release validation? Was this
> > just a failing test that was missed?
> > Kenn
> > On Tue, Apr 23, 2019 at 3:02 PM Andrew Pilloud <ap...@google.com> wrote:
> > > It looks like Java Nexmark tests are on the validation sheet but we've missed it the last few releases. Thanks for
> > > checking it Etienne!
>
> I only see nexmark perf on direct runner. Is this even a thing? Should we also add major runners to that list?
> > > Does the current release process require everything to be tested before making the release final?
>
> I do not think we agreed on this. Previous releases went out with partial validations. It would be good to validate
> everything for the final release.
> > > I fully agree with you on point 2. All of these issues were in RC1 and could have been fixed for RC2.
> > >
> > > Andrew
> > > On Tue, Apr 23, 2019 at 2:58 PM Ahmet Altay <al...@google.com> wrote:
> > > > Thank you Andrew. I will suggest two improvements to the release process:1. We can include benchmarks in the
> > > > validation sheet ("Apache Beam Release Acceptance Criteria"). They are used part of the validation process and
> > > > we can ensure that we check those for each release.
> > > > 2. For RC validation, we can continue to exhaustively validate each RC even after the first -1 vote. Otherwise
> > > > we end up with not discovering all issues in a given RC and find them a successive RC, increasing the number of
> > > > iterations required.
> > > >
> > > >
> > > > On Tue, Apr 23, 2019 at 2:11 PM Andrew Pilloud <ap...@google.com> wrote:
> > > > > Please consider the vote for RC4 canceled. I'll quickly follow up with a new RC.
> > > > > Thanks for the complete testing everyone!
> > > > > Andrew
> > > > > On Tue, Apr 23, 2019 at 2:06 PM Reuven Lax <re...@google.com> wrote:
> > > > > > -1
> > > > > > we need to cherry pick pr/8325 and pr/8385 to fix the above issue
> > > > > > On Tue, Apr 23, 2019 at 1:48 PM Andrew Pilloud <ap...@google.com> wrote:
> > > > > > > I believe the breakage of Nexmark on Dataflow is https://issues.apache.org/jira/browse/BEAM-7002, which
> > > > > > > went in before the release was cut. It looks like this might be a release blocker based on the fix:
> > > > > > > https://github.com/apache/beam/pull/8325.
> > > > > > >
> > > > > > > The degraded performance is after the release is cut, so we should be good there.
> > > > > > >
> > > > > > > Andrew
> > > > > > > On Tue, Apr 23, 2019 at 8:44 AM Ismaël Mejía <ie...@gmail.com> wrote:
> > > > > > > > Etienne RC1 vote happened in 04/03 and there have not been any cherry
> > > > > > > >
> > > > > > > > picks on the spark runner afterwards so if there is a commit that
> > > > > > > >
> > > > > > > > degraded performance around 04/10 it is not part of the release we are
> > > > > > > >
> > > > > > > > voting, so please consider reverting your -1.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > However the issue you are reporting looks important, from a quick look
> > > > > > > >
> > > > > > > > I am guessing it could be related to BEAM-5775 that was merged on
> > > > > > > >
> > > > > > > > 12/04 however the performance regressions started happening since
> > > > > > > >
> > > > > > > > 09/04 so it could be unrelated. Maybe it could be due to changes in
> > > > > > > >
> > > > > > > > our infrastructure. Maybe the change in the workers to be tracked, but
> > > > > > > >
> > > > > > > > definitely not a release blocker at least for the Spark runner.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Tue, Apr 23, 2019 at 5:12 PM Etienne Chauchot <ec...@apache.org> wrote:
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > Hi guys ,
> > > > > > > >
> > > > > > > > > I will vote -1 (binding) on this RC (although degradation is before RC4 cut date). I took a look at
> > > > > > > > Nexmark graphs for the 3 major runners :
> > > > > > > >
> > > > > > > > > - there seem to have functional regressions on Dataflow:
> > > > > > > > https://apache-beam-testing.appspot.com/explore?dashboard=5647201107705856 . 13 queries fail in batch
> > > > > > > > mode starting on 04/17
> > > > > > > >
> > > > > > > > > - there is a perf degradation (+200%) in spark runner starting on 04/10 for all the queries:
> > > > > > > > https://apache-beam-testing.appspot.com/explore?dashboard=5138380291571712
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > Sorry Andrew for the added work
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > Etienne
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > Le lundi 22 avril 2019 à 12:21 -0700, Andrew Pilloud a écrit :
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > I signed the wheels files and updated the build process to not require giving travis apache
> > > > > > > > credentials. (You should probably change your password if you haven't already.)
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > Andrew
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > On Mon, Apr 22, 2019 at 12:18 PM Ahmet Altay <al...@google.com> wrote:
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > +1 (binding)
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > Verified the python 2 wheel files with quick start examples.
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > On Mon, Apr 22, 2019 at 11:26 AM Ahmet Altay <al...@google.com> wrote:
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > I built the wheel files. They are in the usual place along with other python artifacts. I will test
> > > > > > > > them a bit and update here. Could someone else please try the wheel files as well?
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > Andrew, could you sign and hash the wheel files?
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > On Mon, Apr 22, 2019 at 10:11 AM Ahmet Altay <al...@google.com> wrote:
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > I verified
> > > > > > > >
> > > > > > > > > - signatures and hashes.
> > > > > > > >
> > > > > > > > > - python streaming quickstart guide
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > I would like to verify the wheel files before voting. Please let us know when they are ready. Also, if
> > > > > > > > you need help with building wheel files I can help/build.
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > Ahmet
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > On Mon, Apr 22, 2019 at 3:33 AM Maximilian Michels <mx...@apache.org> wrote:
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > +1 (binding)
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > Found a minor bug while testing, but not a blocker:
> > > > > > > >
> > > > > > > > > https://jira.apache.org/jira/browse/BEAM-7128
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > > Max
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > On 20.04.19 23:02, Pablo Estrada wrote:
> > > > > > > >
> > > > > > > > > > +1
> > > > > > > >
> > > > > > > > > > Ran SQL postcommit, and Dataflow Portability Java validatesrunner tests.
> > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > > > > -P.
> > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > > > > On Wed, Apr 17, 2019 at 1:38 AM Jean-Baptiste Onofré <jb@nanthrax.net
> > > > > > > >
> > > > > > > > > > <ma...@nanthrax.net>> wrote:
> > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > > > > +1 (binding)
> > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > > > > Quickly checked with beam-samples.
> > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > > > > Regards
> > > > > > > >
> > > > > > > > > > JB
> > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > > > > On 16/04/2019 00:50, Andrew Pilloud wrote:
> > > > > > > >
> > > > > > > > > > > Hi everyone,
> > > > > > > >
> > > > > > > > > > >
> > > > > > > >
> > > > > > > > > > > Please review and vote on the release candidate #4 for the version
> > > > > > > >
> > > > > > > > > > > 2.12.0, as follows:
> > > > > > > >
> > > > > > > > > > >
> > > > > > > >
> > > > > > > > > > > [ ] +1, Approve the release
> > > > > > > >
> > > > > > > > > > > [ ] -1, Do not approve the release (please provide specific comments)
> > > > > > > >
> > > > > > > > > > >
> > > > > > > >
> > > > > > > > > > > The complete staging area is available for your review, which
> > > > > > > >
> > > > > > > > > > includes:
> > > > > > > >
> > > > > > > > > > > * JIRA release notes [1],
> > > > > > > >
> > > > > > > > > > > * the official Apache source release to be deployed to
> > > > > > > >
> > > > > > > > > > dist.apache.org <http://dist.apache.org>
> > > > > > > >
> > > > > > > > > > > <http://dist.apache.org> [2], which is signed with the key with
> > > > > > > >
> > > > > > > > > > > fingerprint 9E7CEC0661EFD610B632C610AE8FE17F9F8AE3D4 [3],
> > > > > > > >
> > > > > > > > > > > * all artifacts to be deployed to the Maven Central Repository [4],
> > > > > > > >
> > > > > > > > > > > * source code tag "v2.12.0-RC4" [5],
> > > > > > > >
> > > > > > > > > > > * website pull request listing the release [6], publishing the API
> > > > > > > >
> > > > > > > > > > > reference manual [7], and the blog post [8].
> > > > > > > >
> > > > > > > > > > > * Java artifacts were built with Gradle/5.2.1 and OpenJDK/Oracle JDK
> > > > > > > >
> > > > > > > > > > > 1.8.0_181.
> > > > > > > >
> > > > > > > > > > > * Python artifacts are deployed along with the source release to the
> > > > > > > >
> > > > > > > > > > > dist.apache.org <http://dist.apache.org> <http://dist.apache.org>
> > > > > > > >
> > > > > > > > > > [2].
> > > > > > > >
> > > > > > > > > > > * Validation sheet with a tab for 2.12.0 release to help with
> > > > > > > >
> > > > > > > > > > validation
> > > > > > > >
> > > > > > > > > > > [9].
> > > > > > > >
> > > > > > > > > > >
> > > > > > > >
> > > > > > > > > > > The vote will be open for at least 72 hours. It is adopted by
> > > > > > > >
> > > > > > > > > > majority
> > > > > > > >
> > > > > > > > > > > approval, with at least 3 PMC affirmative votes.
> > > > > > > >
> > > > > > > > > > >
> > > > > > > >
> > > > > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > > > > Andrew
> > > > > > > >
> > > > > > > > > > >
> > > > > > > >
> > > > > > > > > > > 1]
> > > > > > > >
> > > > > > > > > > https://jira.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12344944
> > > > > > > >
> > > > > > > > > > > [2] https://dist.apache.org/repos/dist/dev/beam/2.12.0/
> > > > > > > >
> > > > > > > > > > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> > > > > > > >
> > > > > > > > > > > [4]
> > > > > > > >
> > > > > > > > > > https://repository.apache.org/content/repositories/orgapachebeam-1068/
> > > > > > > >
> > > > > > > > > > > [5] https://github.com/apache/beam/tree/v2.12.0-RC4
> > > > > > > >
> > > > > > > > > > > [6] https://github.com/apache/beam/pull/8215
> > > > > > > >
> > > > > > > > > > > [7] https://github.com/apache/beam-site/pull/588
> > > > > > > >
> > > > > > > > > > > [8] https://github.com/apache/beam/pull/8314
> > > > > > > >
> > > > > > > > > > > [9]
> > > > > > > >
> > > > > > > > > >
> > > > > > > > https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984
> > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > > > > --
> > > > > > > >
> > > > > > > > > > Jean-Baptiste Onofré
> > > > > > > >
> > > > > > > > > > jbonofre@apache.org <ma...@apache.org>
> > > > > > > >
> > > > > > > > > > http://blog.nanthrax.net
> > > > > > > >
> > > > > > > > > > Talend - http://www.talend.com
> > > > > > > >
> > > > > > > > > >
> > > > > > > >
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Ahmet Altay <al...@google.com>.
On Tue, Apr 23, 2019 at 3:21 PM Kenneth Knowles <ke...@apache.org> wrote:
> What can we do to make this part of day-to-day workflow instead of finding
> out during release validation? Was this just a failing test that was missed?
>
> Kenn
>
> On Tue, Apr 23, 2019 at 3:02 PM Andrew Pilloud <ap...@google.com>
> wrote:
>
>> It looks like Java Nexmark tests are on the validation sheet but we've
>> missed it the last few releases. Thanks for checking it Etienne!
>>
>
I only see nexmark perf on direct runner. Is this even a thing? Should we
also add major runners to that list?
> Does the current release process require everything to be tested before
>> making the release final?
>>
>
I do not think we agreed on this. Previous releases went out with partial
validations. It would be good to validate everything for the final release.
>
>> I fully agree with you on point 2. All of these issues were in RC1 and
>> could have been fixed for RC2.
>>
>> Andrew
>>
>> On Tue, Apr 23, 2019 at 2:58 PM Ahmet Altay <al...@google.com> wrote:
>>
>>> Thank you Andrew. I will suggest two improvements to the release process:
>>> 1. We can include benchmarks in the validation sheet ("Apache Beam
>>> Release Acceptance Criteria"). They are used part of the validation process
>>> and we can ensure that we check those for each release.
>>> 2. For RC validation, we can continue to exhaustively validate each RC
>>> even after the first -1 vote. Otherwise we end up with not discovering all
>>> issues in a given RC and find them a successive RC, increasing the number
>>> of iterations required.
>>>
>>>
>>> On Tue, Apr 23, 2019 at 2:11 PM Andrew Pilloud <ap...@google.com>
>>> wrote:
>>>
>>>> Please consider the vote for RC4 canceled. I'll quickly follow up with
>>>> a new RC.
>>>>
>>>> Thanks for the complete testing everyone!
>>>>
>>>> Andrew
>>>>
>>>> On Tue, Apr 23, 2019 at 2:06 PM Reuven Lax <re...@google.com> wrote:
>>>>
>>>>> -1
>>>>>
>>>>> we need to cherry pick pr/8325 and pr/8385 to fix the above issue
>>>>>
>>>>> On Tue, Apr 23, 2019 at 1:48 PM Andrew Pilloud <ap...@google.com>
>>>>> wrote:
>>>>>
>>>>>> I believe the breakage of Nexmark on Dataflow is
>>>>>> https://issues.apache.org/jira/browse/BEAM-7002, which went in
>>>>>> before the release was cut. It looks like this might be a release blocker
>>>>>> based on the fix: https://github.com/apache/beam/pull/8325.
>>>>>>
>>>>>> The degraded performance is after the release is cut, so we should be
>>>>>> good there.
>>>>>>
>>>>>> Andrew
>>>>>>
>>>>>> On Tue, Apr 23, 2019 at 8:44 AM Ismaël Mejía <ie...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Etienne RC1 vote happened in 04/03 and there have not been any cherry
>>>>>>> picks on the spark runner afterwards so if there is a commit that
>>>>>>> degraded performance around 04/10 it is not part of the release we
>>>>>>> are
>>>>>>> voting, so please consider reverting your -1.
>>>>>>>
>>>>>>> However the issue you are reporting looks important, from a quick
>>>>>>> look
>>>>>>> I am guessing it could be related to BEAM-5775 that was merged on
>>>>>>> 12/04 however the performance regressions started happening since
>>>>>>> 09/04 so it could be unrelated. Maybe it could be due to changes in
>>>>>>> our infrastructure. Maybe the change in the workers to be tracked,
>>>>>>> but
>>>>>>> definitely not a release blocker at least for the Spark runner.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Apr 23, 2019 at 5:12 PM Etienne Chauchot <
>>>>>>> echauchot@apache.org> wrote:
>>>>>>> >
>>>>>>> > Hi guys ,
>>>>>>> > I will vote -1 (binding) on this RC (although degradation is
>>>>>>> before RC4 cut date). I took a look at Nexmark graphs for the 3 major
>>>>>>> runners :
>>>>>>> > - there seem to have functional regressions on Dataflow:
>>>>>>> https://apache-beam-testing.appspot.com/explore?dashboard=5647201107705856
>>>>>>> . 13 queries fail in batch mode starting on 04/17
>>>>>>> > - there is a perf degradation (+200%) in spark runner starting on
>>>>>>> 04/10 for all the queries:
>>>>>>> https://apache-beam-testing.appspot.com/explore?dashboard=5138380291571712
>>>>>>> >
>>>>>>> > Sorry Andrew for the added work
>>>>>>> >
>>>>>>> > Etienne
>>>>>>> >
>>>>>>> > Le lundi 22 avril 2019 à 12:21 -0700, Andrew Pilloud a écrit :
>>>>>>> >
>>>>>>> > I signed the wheels files and updated the build process to not
>>>>>>> require giving travis apache credentials. (You should probably change your
>>>>>>> password if you haven't already.)
>>>>>>> >
>>>>>>> > Andrew
>>>>>>> >
>>>>>>> > On Mon, Apr 22, 2019 at 12:18 PM Ahmet Altay <al...@google.com>
>>>>>>> wrote:
>>>>>>> >
>>>>>>> > +1 (binding)
>>>>>>> >
>>>>>>> > Verified the python 2 wheel files with quick start examples.
>>>>>>> >
>>>>>>> > On Mon, Apr 22, 2019 at 11:26 AM Ahmet Altay <al...@google.com>
>>>>>>> wrote:
>>>>>>> >
>>>>>>> > I built the wheel files. They are in the usual place along with
>>>>>>> other python artifacts. I will test them a bit and update here. Could
>>>>>>> someone else please try the wheel files as well?
>>>>>>> >
>>>>>>> > Andrew, could you sign and hash the wheel files?
>>>>>>> >
>>>>>>> > On Mon, Apr 22, 2019 at 10:11 AM Ahmet Altay <al...@google.com>
>>>>>>> wrote:
>>>>>>> >
>>>>>>> > I verified
>>>>>>> > - signatures and hashes.
>>>>>>> > - python streaming quickstart guide
>>>>>>> >
>>>>>>> > I would like to verify the wheel files before voting. Please let
>>>>>>> us know when they are ready. Also, if you need help with building wheel
>>>>>>> files I can help/build.
>>>>>>> >
>>>>>>> > Ahmet
>>>>>>> >
>>>>>>> > On Mon, Apr 22, 2019 at 3:33 AM Maximilian Michels <mx...@apache.org>
>>>>>>> wrote:
>>>>>>> >
>>>>>>> > +1 (binding)
>>>>>>> >
>>>>>>> > Found a minor bug while testing, but not a blocker:
>>>>>>> > https://jira.apache.org/jira/browse/BEAM-7128
>>>>>>> >
>>>>>>> > Thanks,
>>>>>>> > Max
>>>>>>> >
>>>>>>> > On 20.04.19 23:02, Pablo Estrada wrote:
>>>>>>> > > +1
>>>>>>> > > Ran SQL postcommit, and Dataflow Portability Java
>>>>>>> validatesrunner tests.
>>>>>>> > >
>>>>>>> > > -P.
>>>>>>> > >
>>>>>>> > > On Wed, Apr 17, 2019 at 1:38 AM Jean-Baptiste Onofré <
>>>>>>> jb@nanthrax.net
>>>>>>> > > <ma...@nanthrax.net>> wrote:
>>>>>>> > >
>>>>>>> > > +1 (binding)
>>>>>>> > >
>>>>>>> > > Quickly checked with beam-samples.
>>>>>>> > >
>>>>>>> > > Regards
>>>>>>> > > JB
>>>>>>> > >
>>>>>>> > > On 16/04/2019 00:50, Andrew Pilloud wrote:
>>>>>>> > > > Hi everyone,
>>>>>>> > > >
>>>>>>> > > > Please review and vote on the release candidate #4 for
>>>>>>> the version
>>>>>>> > > > 2.12.0, as follows:
>>>>>>> > > >
>>>>>>> > > > [ ] +1, Approve the release
>>>>>>> > > > [ ] -1, Do not approve the release (please provide
>>>>>>> specific comments)
>>>>>>> > > >
>>>>>>> > > > The complete staging area is available for your review,
>>>>>>> which
>>>>>>> > > includes:
>>>>>>> > > > * JIRA release notes [1],
>>>>>>> > > > * the official Apache source release to be deployed to
>>>>>>> > > dist.apache.org <http://dist.apache.org>
>>>>>>> > > > <http://dist.apache.org> [2], which is signed with the
>>>>>>> key with
>>>>>>> > > > fingerprint 9E7CEC0661EFD610B632C610AE8FE17F9F8AE3D4 [3],
>>>>>>> > > > * all artifacts to be deployed to the Maven Central
>>>>>>> Repository [4],
>>>>>>> > > > * source code tag "v2.12.0-RC4" [5],
>>>>>>> > > > * website pull request listing the release [6],
>>>>>>> publishing the API
>>>>>>> > > > reference manual [7], and the blog post [8].
>>>>>>> > > > * Java artifacts were built with Gradle/5.2.1 and
>>>>>>> OpenJDK/Oracle JDK
>>>>>>> > > > 1.8.0_181.
>>>>>>> > > > * Python artifacts are deployed along with the source
>>>>>>> release to the
>>>>>>> > > > dist.apache.org <http://dist.apache.org> <
>>>>>>> http://dist.apache.org>
>>>>>>> > > [2].
>>>>>>> > > > * Validation sheet with a tab for 2.12.0 release to help
>>>>>>> with
>>>>>>> > > validation
>>>>>>> > > > [9].
>>>>>>> > > >
>>>>>>> > > > The vote will be open for at least 72 hours. It is
>>>>>>> adopted by
>>>>>>> > > majority
>>>>>>> > > > approval, with at least 3 PMC affirmative votes.
>>>>>>> > > >
>>>>>>> > > > Thanks,
>>>>>>> > > > Andrew
>>>>>>> > > >
>>>>>>> > > > 1]
>>>>>>> > >
>>>>>>> https://jira.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12344944
>>>>>>> > > > [2] https://dist.apache.org/repos/dist/dev/beam/2.12.0/
>>>>>>> > > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>>>>>> > > > [4]
>>>>>>> > >
>>>>>>> https://repository.apache.org/content/repositories/orgapachebeam-1068/
>>>>>>> > > > [5] https://github.com/apache/beam/tree/v2.12.0-RC4
>>>>>>> > > > [6] https://github.com/apache/beam/pull/8215
>>>>>>> > > > [7] https://github.com/apache/beam-site/pull/588
>>>>>>> > > > [8] https://github.com/apache/beam/pull/8314
>>>>>>> > > > [9]
>>>>>>> > >
>>>>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984
>>>>>>> > >
>>>>>>> > > --
>>>>>>> > > Jean-Baptiste Onofré
>>>>>>> > > jbonofre@apache.org <ma...@apache.org>
>>>>>>> > > http://blog.nanthrax.net
>>>>>>> > > Talend - http://www.talend.com
>>>>>>> > >
>>>>>>>
>>>>>>
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Reuven Lax <re...@google.com>.
The Nexmark dataflow runs don't seem to be triggered by Run Java
PostCommit.
On Wed, Apr 24, 2019 at 1:58 AM Etienne Chauchot <ec...@apache.org>
wrote:
> Reuven,
>
> Nexmark tests are indeed run as PostCommits (each commit on master). I
> guess we have been flooded with jenkins notification emails.
>
> Etienne
>
> Le mardi 23 avril 2019 à 15:24 -0700, Reuven Lax a écrit :
>
> I mistakenly though that Java PostCommit would run these tests, and I
> merged based on PostCommit passing. That's how the bug got into master.
>
> On Tue, Apr 23, 2019 at 3:21 PM Kenneth Knowles <ke...@apache.org> wrote:
>
> What can we do to make this part of day-to-day workflow instead of finding
> out during release validation? Was this just a failing test that was missed?
>
> Kenn
>
> On Tue, Apr 23, 2019 at 3:02 PM Andrew Pilloud <ap...@google.com>
> wrote:
>
> It looks like Java Nexmark tests are on the validation sheet but we've
> missed it the last few releases. Thanks for checking it Etienne! Does the
> current release process require everything to be tested before making the
> release final?
>
> I fully agree with you on point 2. All of these issues were in RC1 and
> could have been fixed for RC2.
>
> Andrew
>
> On Tue, Apr 23, 2019 at 2:58 PM Ahmet Altay <al...@google.com> wrote:
>
> Thank you Andrew. I will suggest two improvements to the release process:
> 1. We can include benchmarks in the validation sheet ("Apache Beam Release
> Acceptance Criteria"). They are used part of the validation process and we
> can ensure that we check those for each release.
> 2. For RC validation, we can continue to exhaustively validate each RC
> even after the first -1 vote. Otherwise we end up with not discovering all
> issues in a given RC and find them a successive RC, increasing the number
> of iterations required.
>
>
> On Tue, Apr 23, 2019 at 2:11 PM Andrew Pilloud <ap...@google.com>
> wrote:
>
> Please consider the vote for RC4 canceled. I'll quickly follow up with a
> new RC.
>
> Thanks for the complete testing everyone!
>
> Andrew
>
> On Tue, Apr 23, 2019 at 2:06 PM Reuven Lax <re...@google.com> wrote:
>
> -1
>
> we need to cherry pick pr/8325 and pr/8385 to fix the above issue
>
> On Tue, Apr 23, 2019 at 1:48 PM Andrew Pilloud <ap...@google.com>
> wrote:
>
> I believe the breakage of Nexmark on Dataflow is
> https://issues.apache.org/jira/browse/BEAM-7002, which went in before the
> release was cut. It looks like this might be a release blocker based on the
> fix: https://github.com/apache/beam/pull/8325.
>
> The degraded performance is after the release is cut, so we should be good
> there.
>
> Andrew
>
> On Tue, Apr 23, 2019 at 8:44 AM Ismaël Mejía <ie...@gmail.com> wrote:
>
> Etienne RC1 vote happened in 04/03 and there have not been any cherry
> picks on the spark runner afterwards so if there is a commit that
> degraded performance around 04/10 it is not part of the release we are
> voting, so please consider reverting your -1.
>
> However the issue you are reporting looks important, from a quick look
> I am guessing it could be related to BEAM-5775 that was merged on
> 12/04 however the performance regressions started happening since
> 09/04 so it could be unrelated. Maybe it could be due to changes in
> our infrastructure. Maybe the change in the workers to be tracked, but
> definitely not a release blocker at least for the Spark runner.
>
>
>
>
>
> On Tue, Apr 23, 2019 at 5:12 PM Etienne Chauchot <ec...@apache.org>
> wrote:
> >
> > Hi guys ,
> > I will vote -1 (binding) on this RC (although degradation is before RC4
> cut date). I took a look at Nexmark graphs for the 3 major runners :
> > - there seem to have functional regressions on Dataflow:
> https://apache-beam-testing.appspot.com/explore?dashboard=5647201107705856
> . 13 queries fail in batch mode starting on 04/17
> > - there is a perf degradation (+200%) in spark runner starting on 04/10
> for all the queries:
> https://apache-beam-testing.appspot.com/explore?dashboard=5138380291571712
> >
> > Sorry Andrew for the added work
> >
> > Etienne
> >
> > Le lundi 22 avril 2019 à 12:21 -0700, Andrew Pilloud a écrit :
> >
> > I signed the wheels files and updated the build process to not require
> giving travis apache credentials. (You should probably change your password
> if you haven't already.)
> >
> > Andrew
> >
> > On Mon, Apr 22, 2019 at 12:18 PM Ahmet Altay <al...@google.com> wrote:
> >
> > +1 (binding)
> >
> > Verified the python 2 wheel files with quick start examples.
> >
> > On Mon, Apr 22, 2019 at 11:26 AM Ahmet Altay <al...@google.com> wrote:
> >
> > I built the wheel files. They are in the usual place along with other
> python artifacts. I will test them a bit and update here. Could someone
> else please try the wheel files as well?
> >
> > Andrew, could you sign and hash the wheel files?
> >
> > On Mon, Apr 22, 2019 at 10:11 AM Ahmet Altay <al...@google.com> wrote:
> >
> > I verified
> > - signatures and hashes.
> > - python streaming quickstart guide
> >
> > I would like to verify the wheel files before voting. Please let us know
> when they are ready. Also, if you need help with building wheel files I can
> help/build.
> >
> > Ahmet
> >
> > On Mon, Apr 22, 2019 at 3:33 AM Maximilian Michels <mx...@apache.org>
> wrote:
> >
> > +1 (binding)
> >
> > Found a minor bug while testing, but not a blocker:
> > https://jira.apache.org/jira/browse/BEAM-7128
> >
> > Thanks,
> > Max
> >
> > On 20.04.19 23:02, Pablo Estrada wrote:
> > > +1
> > > Ran SQL postcommit, and Dataflow Portability Java validatesrunner
> tests.
> > >
> > > -P.
> > >
> > > On Wed, Apr 17, 2019 at 1:38 AM Jean-Baptiste Onofré <jb@nanthrax.net
> > > <ma...@nanthrax.net>> wrote:
> > >
> > > +1 (binding)
> > >
> > > Quickly checked with beam-samples.
> > >
> > > Regards
> > > JB
> > >
> > > On 16/04/2019 00:50, Andrew Pilloud wrote:
> > > > Hi everyone,
> > > >
> > > > Please review and vote on the release candidate #4 for the
> version
> > > > 2.12.0, as follows:
> > > >
> > > > [ ] +1, Approve the release
> > > > [ ] -1, Do not approve the release (please provide specific
> comments)
> > > >
> > > > The complete staging area is available for your review, which
> > > includes:
> > > > * JIRA release notes [1],
> > > > * the official Apache source release to be deployed to
> > > dist.apache.org <http://dist.apache.org>
> > > > <http://dist.apache.org> [2], which is signed with the key with
> > > > fingerprint 9E7CEC0661EFD610B632C610AE8FE17F9F8AE3D4 [3],
> > > > * all artifacts to be deployed to the Maven Central Repository
> [4],
> > > > * source code tag "v2.12.0-RC4" [5],
> > > > * website pull request listing the release [6], publishing the
> API
> > > > reference manual [7], and the blog post [8].
> > > > * Java artifacts were built with Gradle/5.2.1 and
> OpenJDK/Oracle JDK
> > > > 1.8.0_181.
> > > > * Python artifacts are deployed along with the source release
> to the
> > > > dist.apache.org <http://dist.apache.org> <
> http://dist.apache.org>
> > > [2].
> > > > * Validation sheet with a tab for 2.12.0 release to help with
> > > validation
> > > > [9].
> > > >
> > > > The vote will be open for at least 72 hours. It is adopted by
> > > majority
> > > > approval, with at least 3 PMC affirmative votes.
> > > >
> > > > Thanks,
> > > > Andrew
> > > >
> > > > 1]
> > >
> https://jira.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12344944
> > > > [2] https://dist.apache.org/repos/dist/dev/beam/2.12.0/
> > > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> > > > [4]
> > >
> https://repository.apache.org/content/repositories/orgapachebeam-1068/
> > > > [5] https://github.com/apache/beam/tree/v2.12.0-RC4
> > > > [6] https://github.com/apache/beam/pull/8215
> > > > [7] https://github.com/apache/beam-site/pull/588
> > > > [8] https://github.com/apache/beam/pull/8314
> > > > [9]
> > >
> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984
> > >
> > > --
> > > Jean-Baptiste Onofré
> > > jbonofre@apache.org <ma...@apache.org>
> > > http://blog.nanthrax.net
> > > Talend - http://www.talend.com
> > >
>
>
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Etienne Chauchot <ec...@apache.org>.
Reuven,
Nexmark tests are indeed run as PostCommits (each commit on master). I guess we have been flooded with jenkins
notification emails.
Etienne
Le mardi 23 avril 2019 à 15:24 -0700, Reuven Lax a écrit :
> I mistakenly though that Java PostCommit would run these tests, and I merged based on PostCommit passing. That's how
> the bug got into master.
> On Tue, Apr 23, 2019 at 3:21 PM Kenneth Knowles <ke...@apache.org> wrote:
> > What can we do to make this part of day-to-day workflow instead of finding out during release validation? Was this
> > just a failing test that was missed?
> > Kenn
> > On Tue, Apr 23, 2019 at 3:02 PM Andrew Pilloud <ap...@google.com> wrote:
> > > It looks like Java Nexmark tests are on the validation sheet but we've missed it the last few releases. Thanks for
> > > checking it Etienne! Does the current release process require everything to be tested before making the release
> > > final?
> > > I fully agree with you on point 2. All of these issues were in RC1 and could have been fixed for RC2.
> > >
> > > Andrew
> > > On Tue, Apr 23, 2019 at 2:58 PM Ahmet Altay <al...@google.com> wrote:
> > > > Thank you Andrew. I will suggest two improvements to the release process:1. We can include benchmarks in the
> > > > validation sheet ("Apache Beam Release Acceptance Criteria"). They are used part of the validation process and
> > > > we can ensure that we check those for each release.
> > > > 2. For RC validation, we can continue to exhaustively validate each RC even after the first -1 vote. Otherwise
> > > > we end up with not discovering all issues in a given RC and find them a successive RC, increasing the number of
> > > > iterations required.
> > > >
> > > >
> > > > On Tue, Apr 23, 2019 at 2:11 PM Andrew Pilloud <ap...@google.com> wrote:
> > > > > Please consider the vote for RC4 canceled. I'll quickly follow up with a new RC.
> > > > > Thanks for the complete testing everyone!
> > > > > Andrew
> > > > > On Tue, Apr 23, 2019 at 2:06 PM Reuven Lax <re...@google.com> wrote:
> > > > > > -1
> > > > > > we need to cherry pick pr/8325 and pr/8385 to fix the above issue
> > > > > > On Tue, Apr 23, 2019 at 1:48 PM Andrew Pilloud <ap...@google.com> wrote:
> > > > > > > I believe the breakage of Nexmark on Dataflow is https://issues.apache.org/jira/browse/BEAM-7002, which
> > > > > > > went in before the release was cut. It looks like this might be a release blocker based on the fix:
> > > > > > > https://github.com/apache/beam/pull/8325.
> > > > > > >
> > > > > > > The degraded performance is after the release is cut, so we should be good there.
> > > > > > >
> > > > > > > Andrew
> > > > > > > On Tue, Apr 23, 2019 at 8:44 AM Ismaël Mejía <ie...@gmail.com> wrote:
> > > > > > > > Etienne RC1 vote happened in 04/03 and there have not been any cherry
> > > > > > > >
> > > > > > > > picks on the spark runner afterwards so if there is a commit that
> > > > > > > >
> > > > > > > > degraded performance around 04/10 it is not part of the release we are
> > > > > > > >
> > > > > > > > voting, so please consider reverting your -1.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > However the issue you are reporting looks important, from a quick look
> > > > > > > >
> > > > > > > > I am guessing it could be related to BEAM-5775 that was merged on
> > > > > > > >
> > > > > > > > 12/04 however the performance regressions started happening since
> > > > > > > >
> > > > > > > > 09/04 so it could be unrelated. Maybe it could be due to changes in
> > > > > > > >
> > > > > > > > our infrastructure. Maybe the change in the workers to be tracked, but
> > > > > > > >
> > > > > > > > definitely not a release blocker at least for the Spark runner.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Tue, Apr 23, 2019 at 5:12 PM Etienne Chauchot <ec...@apache.org> wrote:
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > Hi guys ,
> > > > > > > >
> > > > > > > > > I will vote -1 (binding) on this RC (although degradation is before RC4 cut date). I took a look at
> > > > > > > > Nexmark graphs for the 3 major runners :
> > > > > > > >
> > > > > > > > > - there seem to have functional regressions on Dataflow:
> > > > > > > > https://apache-beam-testing.appspot.com/explore?dashboard=5647201107705856 . 13 queries fail in batch
> > > > > > > > mode starting on 04/17
> > > > > > > >
> > > > > > > > > - there is a perf degradation (+200%) in spark runner starting on 04/10 for all the queries:
> > > > > > > > https://apache-beam-testing.appspot.com/explore?dashboard=5138380291571712
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > Sorry Andrew for the added work
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > Etienne
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > Le lundi 22 avril 2019 à 12:21 -0700, Andrew Pilloud a écrit :
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > I signed the wheels files and updated the build process to not require giving travis apache
> > > > > > > > credentials. (You should probably change your password if you haven't already.)
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > Andrew
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > On Mon, Apr 22, 2019 at 12:18 PM Ahmet Altay <al...@google.com> wrote:
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > +1 (binding)
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > Verified the python 2 wheel files with quick start examples.
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > On Mon, Apr 22, 2019 at 11:26 AM Ahmet Altay <al...@google.com> wrote:
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > I built the wheel files. They are in the usual place along with other python artifacts. I will test
> > > > > > > > them a bit and update here. Could someone else please try the wheel files as well?
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > Andrew, could you sign and hash the wheel files?
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > On Mon, Apr 22, 2019 at 10:11 AM Ahmet Altay <al...@google.com> wrote:
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > I verified
> > > > > > > >
> > > > > > > > > - signatures and hashes.
> > > > > > > >
> > > > > > > > > - python streaming quickstart guide
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > I would like to verify the wheel files before voting. Please let us know when they are ready. Also, if
> > > > > > > > you need help with building wheel files I can help/build.
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > Ahmet
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > On Mon, Apr 22, 2019 at 3:33 AM Maximilian Michels <mx...@apache.org> wrote:
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > +1 (binding)
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > Found a minor bug while testing, but not a blocker:
> > > > > > > >
> > > > > > > > > https://jira.apache.org/jira/browse/BEAM-7128
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > > Max
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > On 20.04.19 23:02, Pablo Estrada wrote:
> > > > > > > >
> > > > > > > > > > +1
> > > > > > > >
> > > > > > > > > > Ran SQL postcommit, and Dataflow Portability Java validatesrunner tests.
> > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > > > > -P.
> > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > > > > On Wed, Apr 17, 2019 at 1:38 AM Jean-Baptiste Onofré <jb@nanthrax.net
> > > > > > > >
> > > > > > > > > > <ma...@nanthrax.net>> wrote:
> > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > > > > +1 (binding)
> > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > > > > Quickly checked with beam-samples.
> > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > > > > Regards
> > > > > > > >
> > > > > > > > > > JB
> > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > > > > On 16/04/2019 00:50, Andrew Pilloud wrote:
> > > > > > > >
> > > > > > > > > > > Hi everyone,
> > > > > > > >
> > > > > > > > > > >
> > > > > > > >
> > > > > > > > > > > Please review and vote on the release candidate #4 for the version
> > > > > > > >
> > > > > > > > > > > 2.12.0, as follows:
> > > > > > > >
> > > > > > > > > > >
> > > > > > > >
> > > > > > > > > > > [ ] +1, Approve the release
> > > > > > > >
> > > > > > > > > > > [ ] -1, Do not approve the release (please provide specific comments)
> > > > > > > >
> > > > > > > > > > >
> > > > > > > >
> > > > > > > > > > > The complete staging area is available for your review, which
> > > > > > > >
> > > > > > > > > > includes:
> > > > > > > >
> > > > > > > > > > > * JIRA release notes [1],
> > > > > > > >
> > > > > > > > > > > * the official Apache source release to be deployed to
> > > > > > > >
> > > > > > > > > > dist.apache.org <http://dist.apache.org>
> > > > > > > >
> > > > > > > > > > > <http://dist.apache.org> [2], which is signed with the key with
> > > > > > > >
> > > > > > > > > > > fingerprint 9E7CEC0661EFD610B632C610AE8FE17F9F8AE3D4 [3],
> > > > > > > >
> > > > > > > > > > > * all artifacts to be deployed to the Maven Central Repository [4],
> > > > > > > >
> > > > > > > > > > > * source code tag "v2.12.0-RC4" [5],
> > > > > > > >
> > > > > > > > > > > * website pull request listing the release [6], publishing the API
> > > > > > > >
> > > > > > > > > > > reference manual [7], and the blog post [8].
> > > > > > > >
> > > > > > > > > > > * Java artifacts were built with Gradle/5.2.1 and OpenJDK/Oracle JDK
> > > > > > > >
> > > > > > > > > > > 1.8.0_181.
> > > > > > > >
> > > > > > > > > > > * Python artifacts are deployed along with the source release to the
> > > > > > > >
> > > > > > > > > > > dist.apache.org <http://dist.apache.org> <http://dist.apache.org>
> > > > > > > >
> > > > > > > > > > [2].
> > > > > > > >
> > > > > > > > > > > * Validation sheet with a tab for 2.12.0 release to help with
> > > > > > > >
> > > > > > > > > > validation
> > > > > > > >
> > > > > > > > > > > [9].
> > > > > > > >
> > > > > > > > > > >
> > > > > > > >
> > > > > > > > > > > The vote will be open for at least 72 hours. It is adopted by
> > > > > > > >
> > > > > > > > > > majority
> > > > > > > >
> > > > > > > > > > > approval, with at least 3 PMC affirmative votes.
> > > > > > > >
> > > > > > > > > > >
> > > > > > > >
> > > > > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > > > > Andrew
> > > > > > > >
> > > > > > > > > > >
> > > > > > > >
> > > > > > > > > > > 1]
> > > > > > > >
> > > > > > > > > > https://jira.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12344944
> > > > > > > >
> > > > > > > > > > > [2] https://dist.apache.org/repos/dist/dev/beam/2.12.0/
> > > > > > > >
> > > > > > > > > > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> > > > > > > >
> > > > > > > > > > > [4]
> > > > > > > >
> > > > > > > > > > https://repository.apache.org/content/repositories/orgapachebeam-1068/
> > > > > > > >
> > > > > > > > > > > [5] https://github.com/apache/beam/tree/v2.12.0-RC4
> > > > > > > >
> > > > > > > > > > > [6] https://github.com/apache/beam/pull/8215
> > > > > > > >
> > > > > > > > > > > [7] https://github.com/apache/beam-site/pull/588
> > > > > > > >
> > > > > > > > > > > [8] https://github.com/apache/beam/pull/8314
> > > > > > > >
> > > > > > > > > > > [9]
> > > > > > > >
> > > > > > > > > >
> > > > > > > > https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984
> > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > > > > --
> > > > > > > >
> > > > > > > > > > Jean-Baptiste Onofré
> > > > > > > >
> > > > > > > > > > jbonofre@apache.org <ma...@apache.org>
> > > > > > > >
> > > > > > > > > > http://blog.nanthrax.net
> > > > > > > >
> > > > > > > > > > Talend - http://www.talend.com
> > > > > > > >
> > > > > > > > > >
> > > > > > > >
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Reuven Lax <re...@google.com>.
I mistakenly though that Java PostCommit would run these tests, and I
merged based on PostCommit passing. That's how the bug got into master.
On Tue, Apr 23, 2019 at 3:21 PM Kenneth Knowles <ke...@apache.org> wrote:
> What can we do to make this part of day-to-day workflow instead of finding
> out during release validation? Was this just a failing test that was missed?
>
> Kenn
>
> On Tue, Apr 23, 2019 at 3:02 PM Andrew Pilloud <ap...@google.com>
> wrote:
>
>> It looks like Java Nexmark tests are on the validation sheet but we've
>> missed it the last few releases. Thanks for checking it Etienne! Does the
>> current release process require everything to be tested before making the
>> release final?
>>
>> I fully agree with you on point 2. All of these issues were in RC1 and
>> could have been fixed for RC2.
>>
>> Andrew
>>
>> On Tue, Apr 23, 2019 at 2:58 PM Ahmet Altay <al...@google.com> wrote:
>>
>>> Thank you Andrew. I will suggest two improvements to the release process:
>>> 1. We can include benchmarks in the validation sheet ("Apache Beam
>>> Release Acceptance Criteria"). They are used part of the validation process
>>> and we can ensure that we check those for each release.
>>> 2. For RC validation, we can continue to exhaustively validate each RC
>>> even after the first -1 vote. Otherwise we end up with not discovering all
>>> issues in a given RC and find them a successive RC, increasing the number
>>> of iterations required.
>>>
>>>
>>> On Tue, Apr 23, 2019 at 2:11 PM Andrew Pilloud <ap...@google.com>
>>> wrote:
>>>
>>>> Please consider the vote for RC4 canceled. I'll quickly follow up with
>>>> a new RC.
>>>>
>>>> Thanks for the complete testing everyone!
>>>>
>>>> Andrew
>>>>
>>>> On Tue, Apr 23, 2019 at 2:06 PM Reuven Lax <re...@google.com> wrote:
>>>>
>>>>> -1
>>>>>
>>>>> we need to cherry pick pr/8325 and pr/8385 to fix the above issue
>>>>>
>>>>> On Tue, Apr 23, 2019 at 1:48 PM Andrew Pilloud <ap...@google.com>
>>>>> wrote:
>>>>>
>>>>>> I believe the breakage of Nexmark on Dataflow is
>>>>>> https://issues.apache.org/jira/browse/BEAM-7002, which went in
>>>>>> before the release was cut. It looks like this might be a release blocker
>>>>>> based on the fix: https://github.com/apache/beam/pull/8325.
>>>>>>
>>>>>> The degraded performance is after the release is cut, so we should be
>>>>>> good there.
>>>>>>
>>>>>> Andrew
>>>>>>
>>>>>> On Tue, Apr 23, 2019 at 8:44 AM Ismaël Mejía <ie...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Etienne RC1 vote happened in 04/03 and there have not been any cherry
>>>>>>> picks on the spark runner afterwards so if there is a commit that
>>>>>>> degraded performance around 04/10 it is not part of the release we
>>>>>>> are
>>>>>>> voting, so please consider reverting your -1.
>>>>>>>
>>>>>>> However the issue you are reporting looks important, from a quick
>>>>>>> look
>>>>>>> I am guessing it could be related to BEAM-5775 that was merged on
>>>>>>> 12/04 however the performance regressions started happening since
>>>>>>> 09/04 so it could be unrelated. Maybe it could be due to changes in
>>>>>>> our infrastructure. Maybe the change in the workers to be tracked,
>>>>>>> but
>>>>>>> definitely not a release blocker at least for the Spark runner.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Apr 23, 2019 at 5:12 PM Etienne Chauchot <
>>>>>>> echauchot@apache.org> wrote:
>>>>>>> >
>>>>>>> > Hi guys ,
>>>>>>> > I will vote -1 (binding) on this RC (although degradation is
>>>>>>> before RC4 cut date). I took a look at Nexmark graphs for the 3 major
>>>>>>> runners :
>>>>>>> > - there seem to have functional regressions on Dataflow:
>>>>>>> https://apache-beam-testing.appspot.com/explore?dashboard=5647201107705856
>>>>>>> . 13 queries fail in batch mode starting on 04/17
>>>>>>> > - there is a perf degradation (+200%) in spark runner starting on
>>>>>>> 04/10 for all the queries:
>>>>>>> https://apache-beam-testing.appspot.com/explore?dashboard=5138380291571712
>>>>>>> >
>>>>>>> > Sorry Andrew for the added work
>>>>>>> >
>>>>>>> > Etienne
>>>>>>> >
>>>>>>> > Le lundi 22 avril 2019 à 12:21 -0700, Andrew Pilloud a écrit :
>>>>>>> >
>>>>>>> > I signed the wheels files and updated the build process to not
>>>>>>> require giving travis apache credentials. (You should probably change your
>>>>>>> password if you haven't already.)
>>>>>>> >
>>>>>>> > Andrew
>>>>>>> >
>>>>>>> > On Mon, Apr 22, 2019 at 12:18 PM Ahmet Altay <al...@google.com>
>>>>>>> wrote:
>>>>>>> >
>>>>>>> > +1 (binding)
>>>>>>> >
>>>>>>> > Verified the python 2 wheel files with quick start examples.
>>>>>>> >
>>>>>>> > On Mon, Apr 22, 2019 at 11:26 AM Ahmet Altay <al...@google.com>
>>>>>>> wrote:
>>>>>>> >
>>>>>>> > I built the wheel files. They are in the usual place along with
>>>>>>> other python artifacts. I will test them a bit and update here. Could
>>>>>>> someone else please try the wheel files as well?
>>>>>>> >
>>>>>>> > Andrew, could you sign and hash the wheel files?
>>>>>>> >
>>>>>>> > On Mon, Apr 22, 2019 at 10:11 AM Ahmet Altay <al...@google.com>
>>>>>>> wrote:
>>>>>>> >
>>>>>>> > I verified
>>>>>>> > - signatures and hashes.
>>>>>>> > - python streaming quickstart guide
>>>>>>> >
>>>>>>> > I would like to verify the wheel files before voting. Please let
>>>>>>> us know when they are ready. Also, if you need help with building wheel
>>>>>>> files I can help/build.
>>>>>>> >
>>>>>>> > Ahmet
>>>>>>> >
>>>>>>> > On Mon, Apr 22, 2019 at 3:33 AM Maximilian Michels <mx...@apache.org>
>>>>>>> wrote:
>>>>>>> >
>>>>>>> > +1 (binding)
>>>>>>> >
>>>>>>> > Found a minor bug while testing, but not a blocker:
>>>>>>> > https://jira.apache.org/jira/browse/BEAM-7128
>>>>>>> >
>>>>>>> > Thanks,
>>>>>>> > Max
>>>>>>> >
>>>>>>> > On 20.04.19 23:02, Pablo Estrada wrote:
>>>>>>> > > +1
>>>>>>> > > Ran SQL postcommit, and Dataflow Portability Java
>>>>>>> validatesrunner tests.
>>>>>>> > >
>>>>>>> > > -P.
>>>>>>> > >
>>>>>>> > > On Wed, Apr 17, 2019 at 1:38 AM Jean-Baptiste Onofré <
>>>>>>> jb@nanthrax.net
>>>>>>> > > <ma...@nanthrax.net>> wrote:
>>>>>>> > >
>>>>>>> > > +1 (binding)
>>>>>>> > >
>>>>>>> > > Quickly checked with beam-samples.
>>>>>>> > >
>>>>>>> > > Regards
>>>>>>> > > JB
>>>>>>> > >
>>>>>>> > > On 16/04/2019 00:50, Andrew Pilloud wrote:
>>>>>>> > > > Hi everyone,
>>>>>>> > > >
>>>>>>> > > > Please review and vote on the release candidate #4 for
>>>>>>> the version
>>>>>>> > > > 2.12.0, as follows:
>>>>>>> > > >
>>>>>>> > > > [ ] +1, Approve the release
>>>>>>> > > > [ ] -1, Do not approve the release (please provide
>>>>>>> specific comments)
>>>>>>> > > >
>>>>>>> > > > The complete staging area is available for your review,
>>>>>>> which
>>>>>>> > > includes:
>>>>>>> > > > * JIRA release notes [1],
>>>>>>> > > > * the official Apache source release to be deployed to
>>>>>>> > > dist.apache.org <http://dist.apache.org>
>>>>>>> > > > <http://dist.apache.org> [2], which is signed with the
>>>>>>> key with
>>>>>>> > > > fingerprint 9E7CEC0661EFD610B632C610AE8FE17F9F8AE3D4 [3],
>>>>>>> > > > * all artifacts to be deployed to the Maven Central
>>>>>>> Repository [4],
>>>>>>> > > > * source code tag "v2.12.0-RC4" [5],
>>>>>>> > > > * website pull request listing the release [6],
>>>>>>> publishing the API
>>>>>>> > > > reference manual [7], and the blog post [8].
>>>>>>> > > > * Java artifacts were built with Gradle/5.2.1 and
>>>>>>> OpenJDK/Oracle JDK
>>>>>>> > > > 1.8.0_181.
>>>>>>> > > > * Python artifacts are deployed along with the source
>>>>>>> release to the
>>>>>>> > > > dist.apache.org <http://dist.apache.org> <
>>>>>>> http://dist.apache.org>
>>>>>>> > > [2].
>>>>>>> > > > * Validation sheet with a tab for 2.12.0 release to help
>>>>>>> with
>>>>>>> > > validation
>>>>>>> > > > [9].
>>>>>>> > > >
>>>>>>> > > > The vote will be open for at least 72 hours. It is
>>>>>>> adopted by
>>>>>>> > > majority
>>>>>>> > > > approval, with at least 3 PMC affirmative votes.
>>>>>>> > > >
>>>>>>> > > > Thanks,
>>>>>>> > > > Andrew
>>>>>>> > > >
>>>>>>> > > > 1]
>>>>>>> > >
>>>>>>> https://jira.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12344944
>>>>>>> > > > [2] https://dist.apache.org/repos/dist/dev/beam/2.12.0/
>>>>>>> > > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>>>>>> > > > [4]
>>>>>>> > >
>>>>>>> https://repository.apache.org/content/repositories/orgapachebeam-1068/
>>>>>>> > > > [5] https://github.com/apache/beam/tree/v2.12.0-RC4
>>>>>>> > > > [6] https://github.com/apache/beam/pull/8215
>>>>>>> > > > [7] https://github.com/apache/beam-site/pull/588
>>>>>>> > > > [8] https://github.com/apache/beam/pull/8314
>>>>>>> > > > [9]
>>>>>>> > >
>>>>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984
>>>>>>> > >
>>>>>>> > > --
>>>>>>> > > Jean-Baptiste Onofré
>>>>>>> > > jbonofre@apache.org <ma...@apache.org>
>>>>>>> > > http://blog.nanthrax.net
>>>>>>> > > Talend - http://www.talend.com
>>>>>>> > >
>>>>>>>
>>>>>>
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Kenneth Knowles <ke...@apache.org>.
What can we do to make this part of day-to-day workflow instead of finding
out during release validation? Was this just a failing test that was missed?
Kenn
On Tue, Apr 23, 2019 at 3:02 PM Andrew Pilloud <ap...@google.com> wrote:
> It looks like Java Nexmark tests are on the validation sheet but we've
> missed it the last few releases. Thanks for checking it Etienne! Does the
> current release process require everything to be tested before making the
> release final?
>
> I fully agree with you on point 2. All of these issues were in RC1 and
> could have been fixed for RC2.
>
> Andrew
>
> On Tue, Apr 23, 2019 at 2:58 PM Ahmet Altay <al...@google.com> wrote:
>
>> Thank you Andrew. I will suggest two improvements to the release process:
>> 1. We can include benchmarks in the validation sheet ("Apache Beam
>> Release Acceptance Criteria"). They are used part of the validation process
>> and we can ensure that we check those for each release.
>> 2. For RC validation, we can continue to exhaustively validate each RC
>> even after the first -1 vote. Otherwise we end up with not discovering all
>> issues in a given RC and find them a successive RC, increasing the number
>> of iterations required.
>>
>>
>> On Tue, Apr 23, 2019 at 2:11 PM Andrew Pilloud <ap...@google.com>
>> wrote:
>>
>>> Please consider the vote for RC4 canceled. I'll quickly follow up with a
>>> new RC.
>>>
>>> Thanks for the complete testing everyone!
>>>
>>> Andrew
>>>
>>> On Tue, Apr 23, 2019 at 2:06 PM Reuven Lax <re...@google.com> wrote:
>>>
>>>> -1
>>>>
>>>> we need to cherry pick pr/8325 and pr/8385 to fix the above issue
>>>>
>>>> On Tue, Apr 23, 2019 at 1:48 PM Andrew Pilloud <ap...@google.com>
>>>> wrote:
>>>>
>>>>> I believe the breakage of Nexmark on Dataflow is
>>>>> https://issues.apache.org/jira/browse/BEAM-7002, which went in before
>>>>> the release was cut. It looks like this might be a release blocker based on
>>>>> the fix: https://github.com/apache/beam/pull/8325.
>>>>>
>>>>> The degraded performance is after the release is cut, so we should be
>>>>> good there.
>>>>>
>>>>> Andrew
>>>>>
>>>>> On Tue, Apr 23, 2019 at 8:44 AM Ismaël Mejía <ie...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Etienne RC1 vote happened in 04/03 and there have not been any cherry
>>>>>> picks on the spark runner afterwards so if there is a commit that
>>>>>> degraded performance around 04/10 it is not part of the release we are
>>>>>> voting, so please consider reverting your -1.
>>>>>>
>>>>>> However the issue you are reporting looks important, from a quick look
>>>>>> I am guessing it could be related to BEAM-5775 that was merged on
>>>>>> 12/04 however the performance regressions started happening since
>>>>>> 09/04 so it could be unrelated. Maybe it could be due to changes in
>>>>>> our infrastructure. Maybe the change in the workers to be tracked, but
>>>>>> definitely not a release blocker at least for the Spark runner.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Apr 23, 2019 at 5:12 PM Etienne Chauchot <
>>>>>> echauchot@apache.org> wrote:
>>>>>> >
>>>>>> > Hi guys ,
>>>>>> > I will vote -1 (binding) on this RC (although degradation is before
>>>>>> RC4 cut date). I took a look at Nexmark graphs for the 3 major runners :
>>>>>> > - there seem to have functional regressions on Dataflow:
>>>>>> https://apache-beam-testing.appspot.com/explore?dashboard=5647201107705856
>>>>>> . 13 queries fail in batch mode starting on 04/17
>>>>>> > - there is a perf degradation (+200%) in spark runner starting on
>>>>>> 04/10 for all the queries:
>>>>>> https://apache-beam-testing.appspot.com/explore?dashboard=5138380291571712
>>>>>> >
>>>>>> > Sorry Andrew for the added work
>>>>>> >
>>>>>> > Etienne
>>>>>> >
>>>>>> > Le lundi 22 avril 2019 à 12:21 -0700, Andrew Pilloud a écrit :
>>>>>> >
>>>>>> > I signed the wheels files and updated the build process to not
>>>>>> require giving travis apache credentials. (You should probably change your
>>>>>> password if you haven't already.)
>>>>>> >
>>>>>> > Andrew
>>>>>> >
>>>>>> > On Mon, Apr 22, 2019 at 12:18 PM Ahmet Altay <al...@google.com>
>>>>>> wrote:
>>>>>> >
>>>>>> > +1 (binding)
>>>>>> >
>>>>>> > Verified the python 2 wheel files with quick start examples.
>>>>>> >
>>>>>> > On Mon, Apr 22, 2019 at 11:26 AM Ahmet Altay <al...@google.com>
>>>>>> wrote:
>>>>>> >
>>>>>> > I built the wheel files. They are in the usual place along with
>>>>>> other python artifacts. I will test them a bit and update here. Could
>>>>>> someone else please try the wheel files as well?
>>>>>> >
>>>>>> > Andrew, could you sign and hash the wheel files?
>>>>>> >
>>>>>> > On Mon, Apr 22, 2019 at 10:11 AM Ahmet Altay <al...@google.com>
>>>>>> wrote:
>>>>>> >
>>>>>> > I verified
>>>>>> > - signatures and hashes.
>>>>>> > - python streaming quickstart guide
>>>>>> >
>>>>>> > I would like to verify the wheel files before voting. Please let us
>>>>>> know when they are ready. Also, if you need help with building wheel files
>>>>>> I can help/build.
>>>>>> >
>>>>>> > Ahmet
>>>>>> >
>>>>>> > On Mon, Apr 22, 2019 at 3:33 AM Maximilian Michels <mx...@apache.org>
>>>>>> wrote:
>>>>>> >
>>>>>> > +1 (binding)
>>>>>> >
>>>>>> > Found a minor bug while testing, but not a blocker:
>>>>>> > https://jira.apache.org/jira/browse/BEAM-7128
>>>>>> >
>>>>>> > Thanks,
>>>>>> > Max
>>>>>> >
>>>>>> > On 20.04.19 23:02, Pablo Estrada wrote:
>>>>>> > > +1
>>>>>> > > Ran SQL postcommit, and Dataflow Portability Java validatesrunner
>>>>>> tests.
>>>>>> > >
>>>>>> > > -P.
>>>>>> > >
>>>>>> > > On Wed, Apr 17, 2019 at 1:38 AM Jean-Baptiste Onofré <
>>>>>> jb@nanthrax.net
>>>>>> > > <ma...@nanthrax.net>> wrote:
>>>>>> > >
>>>>>> > > +1 (binding)
>>>>>> > >
>>>>>> > > Quickly checked with beam-samples.
>>>>>> > >
>>>>>> > > Regards
>>>>>> > > JB
>>>>>> > >
>>>>>> > > On 16/04/2019 00:50, Andrew Pilloud wrote:
>>>>>> > > > Hi everyone,
>>>>>> > > >
>>>>>> > > > Please review and vote on the release candidate #4 for the
>>>>>> version
>>>>>> > > > 2.12.0, as follows:
>>>>>> > > >
>>>>>> > > > [ ] +1, Approve the release
>>>>>> > > > [ ] -1, Do not approve the release (please provide
>>>>>> specific comments)
>>>>>> > > >
>>>>>> > > > The complete staging area is available for your review,
>>>>>> which
>>>>>> > > includes:
>>>>>> > > > * JIRA release notes [1],
>>>>>> > > > * the official Apache source release to be deployed to
>>>>>> > > dist.apache.org <http://dist.apache.org>
>>>>>> > > > <http://dist.apache.org> [2], which is signed with the
>>>>>> key with
>>>>>> > > > fingerprint 9E7CEC0661EFD610B632C610AE8FE17F9F8AE3D4 [3],
>>>>>> > > > * all artifacts to be deployed to the Maven Central
>>>>>> Repository [4],
>>>>>> > > > * source code tag "v2.12.0-RC4" [5],
>>>>>> > > > * website pull request listing the release [6], publishing
>>>>>> the API
>>>>>> > > > reference manual [7], and the blog post [8].
>>>>>> > > > * Java artifacts were built with Gradle/5.2.1 and
>>>>>> OpenJDK/Oracle JDK
>>>>>> > > > 1.8.0_181.
>>>>>> > > > * Python artifacts are deployed along with the source
>>>>>> release to the
>>>>>> > > > dist.apache.org <http://dist.apache.org> <
>>>>>> http://dist.apache.org>
>>>>>> > > [2].
>>>>>> > > > * Validation sheet with a tab for 2.12.0 release to help
>>>>>> with
>>>>>> > > validation
>>>>>> > > > [9].
>>>>>> > > >
>>>>>> > > > The vote will be open for at least 72 hours. It is adopted
>>>>>> by
>>>>>> > > majority
>>>>>> > > > approval, with at least 3 PMC affirmative votes.
>>>>>> > > >
>>>>>> > > > Thanks,
>>>>>> > > > Andrew
>>>>>> > > >
>>>>>> > > > 1]
>>>>>> > >
>>>>>> https://jira.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12344944
>>>>>> > > > [2] https://dist.apache.org/repos/dist/dev/beam/2.12.0/
>>>>>> > > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>>>>> > > > [4]
>>>>>> > >
>>>>>> https://repository.apache.org/content/repositories/orgapachebeam-1068/
>>>>>> > > > [5] https://github.com/apache/beam/tree/v2.12.0-RC4
>>>>>> > > > [6] https://github.com/apache/beam/pull/8215
>>>>>> > > > [7] https://github.com/apache/beam-site/pull/588
>>>>>> > > > [8] https://github.com/apache/beam/pull/8314
>>>>>> > > > [9]
>>>>>> > >
>>>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984
>>>>>> > >
>>>>>> > > --
>>>>>> > > Jean-Baptiste Onofré
>>>>>> > > jbonofre@apache.org <ma...@apache.org>
>>>>>> > > http://blog.nanthrax.net
>>>>>> > > Talend - http://www.talend.com
>>>>>> > >
>>>>>>
>>>>>
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Andrew Pilloud <ap...@google.com>.
It looks like Java Nexmark tests are on the validation sheet but we've
missed it the last few releases. Thanks for checking it Etienne! Does the
current release process require everything to be tested before making the
release final?
I fully agree with you on point 2. All of these issues were in RC1 and
could have been fixed for RC2.
Andrew
On Tue, Apr 23, 2019 at 2:58 PM Ahmet Altay <al...@google.com> wrote:
> Thank you Andrew. I will suggest two improvements to the release process:
> 1. We can include benchmarks in the validation sheet ("Apache Beam Release
> Acceptance Criteria"). They are used part of the validation process and we
> can ensure that we check those for each release.
> 2. For RC validation, we can continue to exhaustively validate each RC
> even after the first -1 vote. Otherwise we end up with not discovering all
> issues in a given RC and find them a successive RC, increasing the number
> of iterations required.
>
>
> On Tue, Apr 23, 2019 at 2:11 PM Andrew Pilloud <ap...@google.com>
> wrote:
>
>> Please consider the vote for RC4 canceled. I'll quickly follow up with a
>> new RC.
>>
>> Thanks for the complete testing everyone!
>>
>> Andrew
>>
>> On Tue, Apr 23, 2019 at 2:06 PM Reuven Lax <re...@google.com> wrote:
>>
>>> -1
>>>
>>> we need to cherry pick pr/8325 and pr/8385 to fix the above issue
>>>
>>> On Tue, Apr 23, 2019 at 1:48 PM Andrew Pilloud <ap...@google.com>
>>> wrote:
>>>
>>>> I believe the breakage of Nexmark on Dataflow is
>>>> https://issues.apache.org/jira/browse/BEAM-7002, which went in before
>>>> the release was cut. It looks like this might be a release blocker based on
>>>> the fix: https://github.com/apache/beam/pull/8325.
>>>>
>>>> The degraded performance is after the release is cut, so we should be
>>>> good there.
>>>>
>>>> Andrew
>>>>
>>>> On Tue, Apr 23, 2019 at 8:44 AM Ismaël Mejía <ie...@gmail.com> wrote:
>>>>
>>>>> Etienne RC1 vote happened in 04/03 and there have not been any cherry
>>>>> picks on the spark runner afterwards so if there is a commit that
>>>>> degraded performance around 04/10 it is not part of the release we are
>>>>> voting, so please consider reverting your -1.
>>>>>
>>>>> However the issue you are reporting looks important, from a quick look
>>>>> I am guessing it could be related to BEAM-5775 that was merged on
>>>>> 12/04 however the performance regressions started happening since
>>>>> 09/04 so it could be unrelated. Maybe it could be due to changes in
>>>>> our infrastructure. Maybe the change in the workers to be tracked, but
>>>>> definitely not a release blocker at least for the Spark runner.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Apr 23, 2019 at 5:12 PM Etienne Chauchot <ec...@apache.org>
>>>>> wrote:
>>>>> >
>>>>> > Hi guys ,
>>>>> > I will vote -1 (binding) on this RC (although degradation is before
>>>>> RC4 cut date). I took a look at Nexmark graphs for the 3 major runners :
>>>>> > - there seem to have functional regressions on Dataflow:
>>>>> https://apache-beam-testing.appspot.com/explore?dashboard=5647201107705856
>>>>> . 13 queries fail in batch mode starting on 04/17
>>>>> > - there is a perf degradation (+200%) in spark runner starting on
>>>>> 04/10 for all the queries:
>>>>> https://apache-beam-testing.appspot.com/explore?dashboard=5138380291571712
>>>>> >
>>>>> > Sorry Andrew for the added work
>>>>> >
>>>>> > Etienne
>>>>> >
>>>>> > Le lundi 22 avril 2019 à 12:21 -0700, Andrew Pilloud a écrit :
>>>>> >
>>>>> > I signed the wheels files and updated the build process to not
>>>>> require giving travis apache credentials. (You should probably change your
>>>>> password if you haven't already.)
>>>>> >
>>>>> > Andrew
>>>>> >
>>>>> > On Mon, Apr 22, 2019 at 12:18 PM Ahmet Altay <al...@google.com>
>>>>> wrote:
>>>>> >
>>>>> > +1 (binding)
>>>>> >
>>>>> > Verified the python 2 wheel files with quick start examples.
>>>>> >
>>>>> > On Mon, Apr 22, 2019 at 11:26 AM Ahmet Altay <al...@google.com>
>>>>> wrote:
>>>>> >
>>>>> > I built the wheel files. They are in the usual place along with
>>>>> other python artifacts. I will test them a bit and update here. Could
>>>>> someone else please try the wheel files as well?
>>>>> >
>>>>> > Andrew, could you sign and hash the wheel files?
>>>>> >
>>>>> > On Mon, Apr 22, 2019 at 10:11 AM Ahmet Altay <al...@google.com>
>>>>> wrote:
>>>>> >
>>>>> > I verified
>>>>> > - signatures and hashes.
>>>>> > - python streaming quickstart guide
>>>>> >
>>>>> > I would like to verify the wheel files before voting. Please let us
>>>>> know when they are ready. Also, if you need help with building wheel files
>>>>> I can help/build.
>>>>> >
>>>>> > Ahmet
>>>>> >
>>>>> > On Mon, Apr 22, 2019 at 3:33 AM Maximilian Michels <mx...@apache.org>
>>>>> wrote:
>>>>> >
>>>>> > +1 (binding)
>>>>> >
>>>>> > Found a minor bug while testing, but not a blocker:
>>>>> > https://jira.apache.org/jira/browse/BEAM-7128
>>>>> >
>>>>> > Thanks,
>>>>> > Max
>>>>> >
>>>>> > On 20.04.19 23:02, Pablo Estrada wrote:
>>>>> > > +1
>>>>> > > Ran SQL postcommit, and Dataflow Portability Java validatesrunner
>>>>> tests.
>>>>> > >
>>>>> > > -P.
>>>>> > >
>>>>> > > On Wed, Apr 17, 2019 at 1:38 AM Jean-Baptiste Onofré <
>>>>> jb@nanthrax.net
>>>>> > > <ma...@nanthrax.net>> wrote:
>>>>> > >
>>>>> > > +1 (binding)
>>>>> > >
>>>>> > > Quickly checked with beam-samples.
>>>>> > >
>>>>> > > Regards
>>>>> > > JB
>>>>> > >
>>>>> > > On 16/04/2019 00:50, Andrew Pilloud wrote:
>>>>> > > > Hi everyone,
>>>>> > > >
>>>>> > > > Please review and vote on the release candidate #4 for the
>>>>> version
>>>>> > > > 2.12.0, as follows:
>>>>> > > >
>>>>> > > > [ ] +1, Approve the release
>>>>> > > > [ ] -1, Do not approve the release (please provide specific
>>>>> comments)
>>>>> > > >
>>>>> > > > The complete staging area is available for your review,
>>>>> which
>>>>> > > includes:
>>>>> > > > * JIRA release notes [1],
>>>>> > > > * the official Apache source release to be deployed to
>>>>> > > dist.apache.org <http://dist.apache.org>
>>>>> > > > <http://dist.apache.org> [2], which is signed with the key
>>>>> with
>>>>> > > > fingerprint 9E7CEC0661EFD610B632C610AE8FE17F9F8AE3D4 [3],
>>>>> > > > * all artifacts to be deployed to the Maven Central
>>>>> Repository [4],
>>>>> > > > * source code tag "v2.12.0-RC4" [5],
>>>>> > > > * website pull request listing the release [6], publishing
>>>>> the API
>>>>> > > > reference manual [7], and the blog post [8].
>>>>> > > > * Java artifacts were built with Gradle/5.2.1 and
>>>>> OpenJDK/Oracle JDK
>>>>> > > > 1.8.0_181.
>>>>> > > > * Python artifacts are deployed along with the source
>>>>> release to the
>>>>> > > > dist.apache.org <http://dist.apache.org> <
>>>>> http://dist.apache.org>
>>>>> > > [2].
>>>>> > > > * Validation sheet with a tab for 2.12.0 release to help
>>>>> with
>>>>> > > validation
>>>>> > > > [9].
>>>>> > > >
>>>>> > > > The vote will be open for at least 72 hours. It is adopted
>>>>> by
>>>>> > > majority
>>>>> > > > approval, with at least 3 PMC affirmative votes.
>>>>> > > >
>>>>> > > > Thanks,
>>>>> > > > Andrew
>>>>> > > >
>>>>> > > > 1]
>>>>> > >
>>>>> https://jira.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12344944
>>>>> > > > [2] https://dist.apache.org/repos/dist/dev/beam/2.12.0/
>>>>> > > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>>>> > > > [4]
>>>>> > >
>>>>> https://repository.apache.org/content/repositories/orgapachebeam-1068/
>>>>> > > > [5] https://github.com/apache/beam/tree/v2.12.0-RC4
>>>>> > > > [6] https://github.com/apache/beam/pull/8215
>>>>> > > > [7] https://github.com/apache/beam-site/pull/588
>>>>> > > > [8] https://github.com/apache/beam/pull/8314
>>>>> > > > [9]
>>>>> > >
>>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984
>>>>> > >
>>>>> > > --
>>>>> > > Jean-Baptiste Onofré
>>>>> > > jbonofre@apache.org <ma...@apache.org>
>>>>> > > http://blog.nanthrax.net
>>>>> > > Talend - http://www.talend.com
>>>>> > >
>>>>>
>>>>
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Ahmet Altay <al...@google.com>.
Thank you Andrew. I will suggest two improvements to the release process:
1. We can include benchmarks in the validation sheet ("Apache Beam Release
Acceptance Criteria"). They are used part of the validation process and we
can ensure that we check those for each release.
2. For RC validation, we can continue to exhaustively validate each RC even
after the first -1 vote. Otherwise we end up with not discovering all
issues in a given RC and find them a successive RC, increasing the number
of iterations required.
On Tue, Apr 23, 2019 at 2:11 PM Andrew Pilloud <ap...@google.com> wrote:
> Please consider the vote for RC4 canceled. I'll quickly follow up with a
> new RC.
>
> Thanks for the complete testing everyone!
>
> Andrew
>
> On Tue, Apr 23, 2019 at 2:06 PM Reuven Lax <re...@google.com> wrote:
>
>> -1
>>
>> we need to cherry pick pr/8325 and pr/8385 to fix the above issue
>>
>> On Tue, Apr 23, 2019 at 1:48 PM Andrew Pilloud <ap...@google.com>
>> wrote:
>>
>>> I believe the breakage of Nexmark on Dataflow is
>>> https://issues.apache.org/jira/browse/BEAM-7002, which went in before
>>> the release was cut. It looks like this might be a release blocker based on
>>> the fix: https://github.com/apache/beam/pull/8325.
>>>
>>> The degraded performance is after the release is cut, so we should be
>>> good there.
>>>
>>> Andrew
>>>
>>> On Tue, Apr 23, 2019 at 8:44 AM Ismaël Mejía <ie...@gmail.com> wrote:
>>>
>>>> Etienne RC1 vote happened in 04/03 and there have not been any cherry
>>>> picks on the spark runner afterwards so if there is a commit that
>>>> degraded performance around 04/10 it is not part of the release we are
>>>> voting, so please consider reverting your -1.
>>>>
>>>> However the issue you are reporting looks important, from a quick look
>>>> I am guessing it could be related to BEAM-5775 that was merged on
>>>> 12/04 however the performance regressions started happening since
>>>> 09/04 so it could be unrelated. Maybe it could be due to changes in
>>>> our infrastructure. Maybe the change in the workers to be tracked, but
>>>> definitely not a release blocker at least for the Spark runner.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, Apr 23, 2019 at 5:12 PM Etienne Chauchot <ec...@apache.org>
>>>> wrote:
>>>> >
>>>> > Hi guys ,
>>>> > I will vote -1 (binding) on this RC (although degradation is before
>>>> RC4 cut date). I took a look at Nexmark graphs for the 3 major runners :
>>>> > - there seem to have functional regressions on Dataflow:
>>>> https://apache-beam-testing.appspot.com/explore?dashboard=5647201107705856
>>>> . 13 queries fail in batch mode starting on 04/17
>>>> > - there is a perf degradation (+200%) in spark runner starting on
>>>> 04/10 for all the queries:
>>>> https://apache-beam-testing.appspot.com/explore?dashboard=5138380291571712
>>>> >
>>>> > Sorry Andrew for the added work
>>>> >
>>>> > Etienne
>>>> >
>>>> > Le lundi 22 avril 2019 à 12:21 -0700, Andrew Pilloud a écrit :
>>>> >
>>>> > I signed the wheels files and updated the build process to not
>>>> require giving travis apache credentials. (You should probably change your
>>>> password if you haven't already.)
>>>> >
>>>> > Andrew
>>>> >
>>>> > On Mon, Apr 22, 2019 at 12:18 PM Ahmet Altay <al...@google.com>
>>>> wrote:
>>>> >
>>>> > +1 (binding)
>>>> >
>>>> > Verified the python 2 wheel files with quick start examples.
>>>> >
>>>> > On Mon, Apr 22, 2019 at 11:26 AM Ahmet Altay <al...@google.com>
>>>> wrote:
>>>> >
>>>> > I built the wheel files. They are in the usual place along with other
>>>> python artifacts. I will test them a bit and update here. Could someone
>>>> else please try the wheel files as well?
>>>> >
>>>> > Andrew, could you sign and hash the wheel files?
>>>> >
>>>> > On Mon, Apr 22, 2019 at 10:11 AM Ahmet Altay <al...@google.com>
>>>> wrote:
>>>> >
>>>> > I verified
>>>> > - signatures and hashes.
>>>> > - python streaming quickstart guide
>>>> >
>>>> > I would like to verify the wheel files before voting. Please let us
>>>> know when they are ready. Also, if you need help with building wheel files
>>>> I can help/build.
>>>> >
>>>> > Ahmet
>>>> >
>>>> > On Mon, Apr 22, 2019 at 3:33 AM Maximilian Michels <mx...@apache.org>
>>>> wrote:
>>>> >
>>>> > +1 (binding)
>>>> >
>>>> > Found a minor bug while testing, but not a blocker:
>>>> > https://jira.apache.org/jira/browse/BEAM-7128
>>>> >
>>>> > Thanks,
>>>> > Max
>>>> >
>>>> > On 20.04.19 23:02, Pablo Estrada wrote:
>>>> > > +1
>>>> > > Ran SQL postcommit, and Dataflow Portability Java validatesrunner
>>>> tests.
>>>> > >
>>>> > > -P.
>>>> > >
>>>> > > On Wed, Apr 17, 2019 at 1:38 AM Jean-Baptiste Onofré <
>>>> jb@nanthrax.net
>>>> > > <ma...@nanthrax.net>> wrote:
>>>> > >
>>>> > > +1 (binding)
>>>> > >
>>>> > > Quickly checked with beam-samples.
>>>> > >
>>>> > > Regards
>>>> > > JB
>>>> > >
>>>> > > On 16/04/2019 00:50, Andrew Pilloud wrote:
>>>> > > > Hi everyone,
>>>> > > >
>>>> > > > Please review and vote on the release candidate #4 for the
>>>> version
>>>> > > > 2.12.0, as follows:
>>>> > > >
>>>> > > > [ ] +1, Approve the release
>>>> > > > [ ] -1, Do not approve the release (please provide specific
>>>> comments)
>>>> > > >
>>>> > > > The complete staging area is available for your review, which
>>>> > > includes:
>>>> > > > * JIRA release notes [1],
>>>> > > > * the official Apache source release to be deployed to
>>>> > > dist.apache.org <http://dist.apache.org>
>>>> > > > <http://dist.apache.org> [2], which is signed with the key
>>>> with
>>>> > > > fingerprint 9E7CEC0661EFD610B632C610AE8FE17F9F8AE3D4 [3],
>>>> > > > * all artifacts to be deployed to the Maven Central
>>>> Repository [4],
>>>> > > > * source code tag "v2.12.0-RC4" [5],
>>>> > > > * website pull request listing the release [6], publishing
>>>> the API
>>>> > > > reference manual [7], and the blog post [8].
>>>> > > > * Java artifacts were built with Gradle/5.2.1 and
>>>> OpenJDK/Oracle JDK
>>>> > > > 1.8.0_181.
>>>> > > > * Python artifacts are deployed along with the source
>>>> release to the
>>>> > > > dist.apache.org <http://dist.apache.org> <
>>>> http://dist.apache.org>
>>>> > > [2].
>>>> > > > * Validation sheet with a tab for 2.12.0 release to help with
>>>> > > validation
>>>> > > > [9].
>>>> > > >
>>>> > > > The vote will be open for at least 72 hours. It is adopted by
>>>> > > majority
>>>> > > > approval, with at least 3 PMC affirmative votes.
>>>> > > >
>>>> > > > Thanks,
>>>> > > > Andrew
>>>> > > >
>>>> > > > 1]
>>>> > >
>>>> https://jira.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12344944
>>>> > > > [2] https://dist.apache.org/repos/dist/dev/beam/2.12.0/
>>>> > > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>>> > > > [4]
>>>> > >
>>>> https://repository.apache.org/content/repositories/orgapachebeam-1068/
>>>> > > > [5] https://github.com/apache/beam/tree/v2.12.0-RC4
>>>> > > > [6] https://github.com/apache/beam/pull/8215
>>>> > > > [7] https://github.com/apache/beam-site/pull/588
>>>> > > > [8] https://github.com/apache/beam/pull/8314
>>>> > > > [9]
>>>> > >
>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984
>>>> > >
>>>> > > --
>>>> > > Jean-Baptiste Onofré
>>>> > > jbonofre@apache.org <ma...@apache.org>
>>>> > > http://blog.nanthrax.net
>>>> > > Talend - http://www.talend.com
>>>> > >
>>>>
>>>
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Andrew Pilloud <ap...@google.com>.
Please consider the vote for RC4 canceled. I'll quickly follow up with a
new RC.
Thanks for the complete testing everyone!
Andrew
On Tue, Apr 23, 2019 at 2:06 PM Reuven Lax <re...@google.com> wrote:
> -1
>
> we need to cherry pick pr/8325 and pr/8385 to fix the above issue
>
> On Tue, Apr 23, 2019 at 1:48 PM Andrew Pilloud <ap...@google.com>
> wrote:
>
>> I believe the breakage of Nexmark on Dataflow is
>> https://issues.apache.org/jira/browse/BEAM-7002, which went in before
>> the release was cut. It looks like this might be a release blocker based on
>> the fix: https://github.com/apache/beam/pull/8325.
>>
>> The degraded performance is after the release is cut, so we should be
>> good there.
>>
>> Andrew
>>
>> On Tue, Apr 23, 2019 at 8:44 AM Ismaël Mejía <ie...@gmail.com> wrote:
>>
>>> Etienne RC1 vote happened in 04/03 and there have not been any cherry
>>> picks on the spark runner afterwards so if there is a commit that
>>> degraded performance around 04/10 it is not part of the release we are
>>> voting, so please consider reverting your -1.
>>>
>>> However the issue you are reporting looks important, from a quick look
>>> I am guessing it could be related to BEAM-5775 that was merged on
>>> 12/04 however the performance regressions started happening since
>>> 09/04 so it could be unrelated. Maybe it could be due to changes in
>>> our infrastructure. Maybe the change in the workers to be tracked, but
>>> definitely not a release blocker at least for the Spark runner.
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Apr 23, 2019 at 5:12 PM Etienne Chauchot <ec...@apache.org>
>>> wrote:
>>> >
>>> > Hi guys ,
>>> > I will vote -1 (binding) on this RC (although degradation is before
>>> RC4 cut date). I took a look at Nexmark graphs for the 3 major runners :
>>> > - there seem to have functional regressions on Dataflow:
>>> https://apache-beam-testing.appspot.com/explore?dashboard=5647201107705856
>>> . 13 queries fail in batch mode starting on 04/17
>>> > - there is a perf degradation (+200%) in spark runner starting on
>>> 04/10 for all the queries:
>>> https://apache-beam-testing.appspot.com/explore?dashboard=5138380291571712
>>> >
>>> > Sorry Andrew for the added work
>>> >
>>> > Etienne
>>> >
>>> > Le lundi 22 avril 2019 à 12:21 -0700, Andrew Pilloud a écrit :
>>> >
>>> > I signed the wheels files and updated the build process to not require
>>> giving travis apache credentials. (You should probably change your password
>>> if you haven't already.)
>>> >
>>> > Andrew
>>> >
>>> > On Mon, Apr 22, 2019 at 12:18 PM Ahmet Altay <al...@google.com> wrote:
>>> >
>>> > +1 (binding)
>>> >
>>> > Verified the python 2 wheel files with quick start examples.
>>> >
>>> > On Mon, Apr 22, 2019 at 11:26 AM Ahmet Altay <al...@google.com> wrote:
>>> >
>>> > I built the wheel files. They are in the usual place along with other
>>> python artifacts. I will test them a bit and update here. Could someone
>>> else please try the wheel files as well?
>>> >
>>> > Andrew, could you sign and hash the wheel files?
>>> >
>>> > On Mon, Apr 22, 2019 at 10:11 AM Ahmet Altay <al...@google.com> wrote:
>>> >
>>> > I verified
>>> > - signatures and hashes.
>>> > - python streaming quickstart guide
>>> >
>>> > I would like to verify the wheel files before voting. Please let us
>>> know when they are ready. Also, if you need help with building wheel files
>>> I can help/build.
>>> >
>>> > Ahmet
>>> >
>>> > On Mon, Apr 22, 2019 at 3:33 AM Maximilian Michels <mx...@apache.org>
>>> wrote:
>>> >
>>> > +1 (binding)
>>> >
>>> > Found a minor bug while testing, but not a blocker:
>>> > https://jira.apache.org/jira/browse/BEAM-7128
>>> >
>>> > Thanks,
>>> > Max
>>> >
>>> > On 20.04.19 23:02, Pablo Estrada wrote:
>>> > > +1
>>> > > Ran SQL postcommit, and Dataflow Portability Java validatesrunner
>>> tests.
>>> > >
>>> > > -P.
>>> > >
>>> > > On Wed, Apr 17, 2019 at 1:38 AM Jean-Baptiste Onofré <
>>> jb@nanthrax.net
>>> > > <ma...@nanthrax.net>> wrote:
>>> > >
>>> > > +1 (binding)
>>> > >
>>> > > Quickly checked with beam-samples.
>>> > >
>>> > > Regards
>>> > > JB
>>> > >
>>> > > On 16/04/2019 00:50, Andrew Pilloud wrote:
>>> > > > Hi everyone,
>>> > > >
>>> > > > Please review and vote on the release candidate #4 for the
>>> version
>>> > > > 2.12.0, as follows:
>>> > > >
>>> > > > [ ] +1, Approve the release
>>> > > > [ ] -1, Do not approve the release (please provide specific
>>> comments)
>>> > > >
>>> > > > The complete staging area is available for your review, which
>>> > > includes:
>>> > > > * JIRA release notes [1],
>>> > > > * the official Apache source release to be deployed to
>>> > > dist.apache.org <http://dist.apache.org>
>>> > > > <http://dist.apache.org> [2], which is signed with the key
>>> with
>>> > > > fingerprint 9E7CEC0661EFD610B632C610AE8FE17F9F8AE3D4 [3],
>>> > > > * all artifacts to be deployed to the Maven Central
>>> Repository [4],
>>> > > > * source code tag "v2.12.0-RC4" [5],
>>> > > > * website pull request listing the release [6], publishing
>>> the API
>>> > > > reference manual [7], and the blog post [8].
>>> > > > * Java artifacts were built with Gradle/5.2.1 and
>>> OpenJDK/Oracle JDK
>>> > > > 1.8.0_181.
>>> > > > * Python artifacts are deployed along with the source release
>>> to the
>>> > > > dist.apache.org <http://dist.apache.org> <
>>> http://dist.apache.org>
>>> > > [2].
>>> > > > * Validation sheet with a tab for 2.12.0 release to help with
>>> > > validation
>>> > > > [9].
>>> > > >
>>> > > > The vote will be open for at least 72 hours. It is adopted by
>>> > > majority
>>> > > > approval, with at least 3 PMC affirmative votes.
>>> > > >
>>> > > > Thanks,
>>> > > > Andrew
>>> > > >
>>> > > > 1]
>>> > >
>>> https://jira.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12344944
>>> > > > [2] https://dist.apache.org/repos/dist/dev/beam/2.12.0/
>>> > > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>> > > > [4]
>>> > >
>>> https://repository.apache.org/content/repositories/orgapachebeam-1068/
>>> > > > [5] https://github.com/apache/beam/tree/v2.12.0-RC4
>>> > > > [6] https://github.com/apache/beam/pull/8215
>>> > > > [7] https://github.com/apache/beam-site/pull/588
>>> > > > [8] https://github.com/apache/beam/pull/8314
>>> > > > [9]
>>> > >
>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984
>>> > >
>>> > > --
>>> > > Jean-Baptiste Onofré
>>> > > jbonofre@apache.org <ma...@apache.org>
>>> > > http://blog.nanthrax.net
>>> > > Talend - http://www.talend.com
>>> > >
>>>
>>
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Reuven Lax <re...@google.com>.
-1
we need to cherry pick pr/8325 and pr/8385 to fix the above issue
On Tue, Apr 23, 2019 at 1:48 PM Andrew Pilloud <ap...@google.com> wrote:
> I believe the breakage of Nexmark on Dataflow is
> https://issues.apache.org/jira/browse/BEAM-7002, which went in before the
> release was cut. It looks like this might be a release blocker based on the
> fix: https://github.com/apache/beam/pull/8325.
>
> The degraded performance is after the release is cut, so we should be good
> there.
>
> Andrew
>
> On Tue, Apr 23, 2019 at 8:44 AM Ismaël Mejía <ie...@gmail.com> wrote:
>
>> Etienne RC1 vote happened in 04/03 and there have not been any cherry
>> picks on the spark runner afterwards so if there is a commit that
>> degraded performance around 04/10 it is not part of the release we are
>> voting, so please consider reverting your -1.
>>
>> However the issue you are reporting looks important, from a quick look
>> I am guessing it could be related to BEAM-5775 that was merged on
>> 12/04 however the performance regressions started happening since
>> 09/04 so it could be unrelated. Maybe it could be due to changes in
>> our infrastructure. Maybe the change in the workers to be tracked, but
>> definitely not a release blocker at least for the Spark runner.
>>
>>
>>
>>
>>
>> On Tue, Apr 23, 2019 at 5:12 PM Etienne Chauchot <ec...@apache.org>
>> wrote:
>> >
>> > Hi guys ,
>> > I will vote -1 (binding) on this RC (although degradation is before RC4
>> cut date). I took a look at Nexmark graphs for the 3 major runners :
>> > - there seem to have functional regressions on Dataflow:
>> https://apache-beam-testing.appspot.com/explore?dashboard=5647201107705856
>> . 13 queries fail in batch mode starting on 04/17
>> > - there is a perf degradation (+200%) in spark runner starting on 04/10
>> for all the queries:
>> https://apache-beam-testing.appspot.com/explore?dashboard=5138380291571712
>> >
>> > Sorry Andrew for the added work
>> >
>> > Etienne
>> >
>> > Le lundi 22 avril 2019 à 12:21 -0700, Andrew Pilloud a écrit :
>> >
>> > I signed the wheels files and updated the build process to not require
>> giving travis apache credentials. (You should probably change your password
>> if you haven't already.)
>> >
>> > Andrew
>> >
>> > On Mon, Apr 22, 2019 at 12:18 PM Ahmet Altay <al...@google.com> wrote:
>> >
>> > +1 (binding)
>> >
>> > Verified the python 2 wheel files with quick start examples.
>> >
>> > On Mon, Apr 22, 2019 at 11:26 AM Ahmet Altay <al...@google.com> wrote:
>> >
>> > I built the wheel files. They are in the usual place along with other
>> python artifacts. I will test them a bit and update here. Could someone
>> else please try the wheel files as well?
>> >
>> > Andrew, could you sign and hash the wheel files?
>> >
>> > On Mon, Apr 22, 2019 at 10:11 AM Ahmet Altay <al...@google.com> wrote:
>> >
>> > I verified
>> > - signatures and hashes.
>> > - python streaming quickstart guide
>> >
>> > I would like to verify the wheel files before voting. Please let us
>> know when they are ready. Also, if you need help with building wheel files
>> I can help/build.
>> >
>> > Ahmet
>> >
>> > On Mon, Apr 22, 2019 at 3:33 AM Maximilian Michels <mx...@apache.org>
>> wrote:
>> >
>> > +1 (binding)
>> >
>> > Found a minor bug while testing, but not a blocker:
>> > https://jira.apache.org/jira/browse/BEAM-7128
>> >
>> > Thanks,
>> > Max
>> >
>> > On 20.04.19 23:02, Pablo Estrada wrote:
>> > > +1
>> > > Ran SQL postcommit, and Dataflow Portability Java validatesrunner
>> tests.
>> > >
>> > > -P.
>> > >
>> > > On Wed, Apr 17, 2019 at 1:38 AM Jean-Baptiste Onofré <jb@nanthrax.net
>> > > <ma...@nanthrax.net>> wrote:
>> > >
>> > > +1 (binding)
>> > >
>> > > Quickly checked with beam-samples.
>> > >
>> > > Regards
>> > > JB
>> > >
>> > > On 16/04/2019 00:50, Andrew Pilloud wrote:
>> > > > Hi everyone,
>> > > >
>> > > > Please review and vote on the release candidate #4 for the
>> version
>> > > > 2.12.0, as follows:
>> > > >
>> > > > [ ] +1, Approve the release
>> > > > [ ] -1, Do not approve the release (please provide specific
>> comments)
>> > > >
>> > > > The complete staging area is available for your review, which
>> > > includes:
>> > > > * JIRA release notes [1],
>> > > > * the official Apache source release to be deployed to
>> > > dist.apache.org <http://dist.apache.org>
>> > > > <http://dist.apache.org> [2], which is signed with the key
>> with
>> > > > fingerprint 9E7CEC0661EFD610B632C610AE8FE17F9F8AE3D4 [3],
>> > > > * all artifacts to be deployed to the Maven Central Repository
>> [4],
>> > > > * source code tag "v2.12.0-RC4" [5],
>> > > > * website pull request listing the release [6], publishing the
>> API
>> > > > reference manual [7], and the blog post [8].
>> > > > * Java artifacts were built with Gradle/5.2.1 and
>> OpenJDK/Oracle JDK
>> > > > 1.8.0_181.
>> > > > * Python artifacts are deployed along with the source release
>> to the
>> > > > dist.apache.org <http://dist.apache.org> <
>> http://dist.apache.org>
>> > > [2].
>> > > > * Validation sheet with a tab for 2.12.0 release to help with
>> > > validation
>> > > > [9].
>> > > >
>> > > > The vote will be open for at least 72 hours. It is adopted by
>> > > majority
>> > > > approval, with at least 3 PMC affirmative votes.
>> > > >
>> > > > Thanks,
>> > > > Andrew
>> > > >
>> > > > 1]
>> > >
>> https://jira.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12344944
>> > > > [2] https://dist.apache.org/repos/dist/dev/beam/2.12.0/
>> > > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>> > > > [4]
>> > >
>> https://repository.apache.org/content/repositories/orgapachebeam-1068/
>> > > > [5] https://github.com/apache/beam/tree/v2.12.0-RC4
>> > > > [6] https://github.com/apache/beam/pull/8215
>> > > > [7] https://github.com/apache/beam-site/pull/588
>> > > > [8] https://github.com/apache/beam/pull/8314
>> > > > [9]
>> > >
>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984
>> > >
>> > > --
>> > > Jean-Baptiste Onofré
>> > > jbonofre@apache.org <ma...@apache.org>
>> > > http://blog.nanthrax.net
>> > > Talend - http://www.talend.com
>> > >
>>
>
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Andrew Pilloud <ap...@google.com>.
I believe the breakage of Nexmark on Dataflow is
https://issues.apache.org/jira/browse/BEAM-7002, which went in before the
release was cut. It looks like this might be a release blocker based on the
fix: https://github.com/apache/beam/pull/8325.
The degraded performance is after the release is cut, so we should be good
there.
Andrew
On Tue, Apr 23, 2019 at 8:44 AM Ismaël Mejía <ie...@gmail.com> wrote:
> Etienne RC1 vote happened in 04/03 and there have not been any cherry
> picks on the spark runner afterwards so if there is a commit that
> degraded performance around 04/10 it is not part of the release we are
> voting, so please consider reverting your -1.
>
> However the issue you are reporting looks important, from a quick look
> I am guessing it could be related to BEAM-5775 that was merged on
> 12/04 however the performance regressions started happening since
> 09/04 so it could be unrelated. Maybe it could be due to changes in
> our infrastructure. Maybe the change in the workers to be tracked, but
> definitely not a release blocker at least for the Spark runner.
>
>
>
>
>
> On Tue, Apr 23, 2019 at 5:12 PM Etienne Chauchot <ec...@apache.org>
> wrote:
> >
> > Hi guys ,
> > I will vote -1 (binding) on this RC (although degradation is before RC4
> cut date). I took a look at Nexmark graphs for the 3 major runners :
> > - there seem to have functional regressions on Dataflow:
> https://apache-beam-testing.appspot.com/explore?dashboard=5647201107705856
> . 13 queries fail in batch mode starting on 04/17
> > - there is a perf degradation (+200%) in spark runner starting on 04/10
> for all the queries:
> https://apache-beam-testing.appspot.com/explore?dashboard=5138380291571712
> >
> > Sorry Andrew for the added work
> >
> > Etienne
> >
> > Le lundi 22 avril 2019 à 12:21 -0700, Andrew Pilloud a écrit :
> >
> > I signed the wheels files and updated the build process to not require
> giving travis apache credentials. (You should probably change your password
> if you haven't already.)
> >
> > Andrew
> >
> > On Mon, Apr 22, 2019 at 12:18 PM Ahmet Altay <al...@google.com> wrote:
> >
> > +1 (binding)
> >
> > Verified the python 2 wheel files with quick start examples.
> >
> > On Mon, Apr 22, 2019 at 11:26 AM Ahmet Altay <al...@google.com> wrote:
> >
> > I built the wheel files. They are in the usual place along with other
> python artifacts. I will test them a bit and update here. Could someone
> else please try the wheel files as well?
> >
> > Andrew, could you sign and hash the wheel files?
> >
> > On Mon, Apr 22, 2019 at 10:11 AM Ahmet Altay <al...@google.com> wrote:
> >
> > I verified
> > - signatures and hashes.
> > - python streaming quickstart guide
> >
> > I would like to verify the wheel files before voting. Please let us know
> when they are ready. Also, if you need help with building wheel files I can
> help/build.
> >
> > Ahmet
> >
> > On Mon, Apr 22, 2019 at 3:33 AM Maximilian Michels <mx...@apache.org>
> wrote:
> >
> > +1 (binding)
> >
> > Found a minor bug while testing, but not a blocker:
> > https://jira.apache.org/jira/browse/BEAM-7128
> >
> > Thanks,
> > Max
> >
> > On 20.04.19 23:02, Pablo Estrada wrote:
> > > +1
> > > Ran SQL postcommit, and Dataflow Portability Java validatesrunner
> tests.
> > >
> > > -P.
> > >
> > > On Wed, Apr 17, 2019 at 1:38 AM Jean-Baptiste Onofré <jb@nanthrax.net
> > > <ma...@nanthrax.net>> wrote:
> > >
> > > +1 (binding)
> > >
> > > Quickly checked with beam-samples.
> > >
> > > Regards
> > > JB
> > >
> > > On 16/04/2019 00:50, Andrew Pilloud wrote:
> > > > Hi everyone,
> > > >
> > > > Please review and vote on the release candidate #4 for the
> version
> > > > 2.12.0, as follows:
> > > >
> > > > [ ] +1, Approve the release
> > > > [ ] -1, Do not approve the release (please provide specific
> comments)
> > > >
> > > > The complete staging area is available for your review, which
> > > includes:
> > > > * JIRA release notes [1],
> > > > * the official Apache source release to be deployed to
> > > dist.apache.org <http://dist.apache.org>
> > > > <http://dist.apache.org> [2], which is signed with the key with
> > > > fingerprint 9E7CEC0661EFD610B632C610AE8FE17F9F8AE3D4 [3],
> > > > * all artifacts to be deployed to the Maven Central Repository
> [4],
> > > > * source code tag "v2.12.0-RC4" [5],
> > > > * website pull request listing the release [6], publishing the
> API
> > > > reference manual [7], and the blog post [8].
> > > > * Java artifacts were built with Gradle/5.2.1 and
> OpenJDK/Oracle JDK
> > > > 1.8.0_181.
> > > > * Python artifacts are deployed along with the source release
> to the
> > > > dist.apache.org <http://dist.apache.org> <
> http://dist.apache.org>
> > > [2].
> > > > * Validation sheet with a tab for 2.12.0 release to help with
> > > validation
> > > > [9].
> > > >
> > > > The vote will be open for at least 72 hours. It is adopted by
> > > majority
> > > > approval, with at least 3 PMC affirmative votes.
> > > >
> > > > Thanks,
> > > > Andrew
> > > >
> > > > 1]
> > >
> https://jira.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12344944
> > > > [2] https://dist.apache.org/repos/dist/dev/beam/2.12.0/
> > > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> > > > [4]
> > >
> https://repository.apache.org/content/repositories/orgapachebeam-1068/
> > > > [5] https://github.com/apache/beam/tree/v2.12.0-RC4
> > > > [6] https://github.com/apache/beam/pull/8215
> > > > [7] https://github.com/apache/beam-site/pull/588
> > > > [8] https://github.com/apache/beam/pull/8314
> > > > [9]
> > >
> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984
> > >
> > > --
> > > Jean-Baptiste Onofré
> > > jbonofre@apache.org <ma...@apache.org>
> > > http://blog.nanthrax.net
> > > Talend - http://www.talend.com
> > >
>
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Ismaël Mejía <ie...@gmail.com>.
Etienne RC1 vote happened in 04/03 and there have not been any cherry
picks on the spark runner afterwards so if there is a commit that
degraded performance around 04/10 it is not part of the release we are
voting, so please consider reverting your -1.
However the issue you are reporting looks important, from a quick look
I am guessing it could be related to BEAM-5775 that was merged on
12/04 however the performance regressions started happening since
09/04 so it could be unrelated. Maybe it could be due to changes in
our infrastructure. Maybe the change in the workers to be tracked, but
definitely not a release blocker at least for the Spark runner.
On Tue, Apr 23, 2019 at 5:12 PM Etienne Chauchot <ec...@apache.org> wrote:
>
> Hi guys ,
> I will vote -1 (binding) on this RC (although degradation is before RC4 cut date). I took a look at Nexmark graphs for the 3 major runners :
> - there seem to have functional regressions on Dataflow: https://apache-beam-testing.appspot.com/explore?dashboard=5647201107705856 . 13 queries fail in batch mode starting on 04/17
> - there is a perf degradation (+200%) in spark runner starting on 04/10 for all the queries: https://apache-beam-testing.appspot.com/explore?dashboard=5138380291571712
>
> Sorry Andrew for the added work
>
> Etienne
>
> Le lundi 22 avril 2019 à 12:21 -0700, Andrew Pilloud a écrit :
>
> I signed the wheels files and updated the build process to not require giving travis apache credentials. (You should probably change your password if you haven't already.)
>
> Andrew
>
> On Mon, Apr 22, 2019 at 12:18 PM Ahmet Altay <al...@google.com> wrote:
>
> +1 (binding)
>
> Verified the python 2 wheel files with quick start examples.
>
> On Mon, Apr 22, 2019 at 11:26 AM Ahmet Altay <al...@google.com> wrote:
>
> I built the wheel files. They are in the usual place along with other python artifacts. I will test them a bit and update here. Could someone else please try the wheel files as well?
>
> Andrew, could you sign and hash the wheel files?
>
> On Mon, Apr 22, 2019 at 10:11 AM Ahmet Altay <al...@google.com> wrote:
>
> I verified
> - signatures and hashes.
> - python streaming quickstart guide
>
> I would like to verify the wheel files before voting. Please let us know when they are ready. Also, if you need help with building wheel files I can help/build.
>
> Ahmet
>
> On Mon, Apr 22, 2019 at 3:33 AM Maximilian Michels <mx...@apache.org> wrote:
>
> +1 (binding)
>
> Found a minor bug while testing, but not a blocker:
> https://jira.apache.org/jira/browse/BEAM-7128
>
> Thanks,
> Max
>
> On 20.04.19 23:02, Pablo Estrada wrote:
> > +1
> > Ran SQL postcommit, and Dataflow Portability Java validatesrunner tests.
> >
> > -P.
> >
> > On Wed, Apr 17, 2019 at 1:38 AM Jean-Baptiste Onofré <jb@nanthrax.net
> > <ma...@nanthrax.net>> wrote:
> >
> > +1 (binding)
> >
> > Quickly checked with beam-samples.
> >
> > Regards
> > JB
> >
> > On 16/04/2019 00:50, Andrew Pilloud wrote:
> > > Hi everyone,
> > >
> > > Please review and vote on the release candidate #4 for the version
> > > 2.12.0, as follows:
> > >
> > > [ ] +1, Approve the release
> > > [ ] -1, Do not approve the release (please provide specific comments)
> > >
> > > The complete staging area is available for your review, which
> > includes:
> > > * JIRA release notes [1],
> > > * the official Apache source release to be deployed to
> > dist.apache.org <http://dist.apache.org>
> > > <http://dist.apache.org> [2], which is signed with the key with
> > > fingerprint 9E7CEC0661EFD610B632C610AE8FE17F9F8AE3D4 [3],
> > > * all artifacts to be deployed to the Maven Central Repository [4],
> > > * source code tag "v2.12.0-RC4" [5],
> > > * website pull request listing the release [6], publishing the API
> > > reference manual [7], and the blog post [8].
> > > * Java artifacts were built with Gradle/5.2.1 and OpenJDK/Oracle JDK
> > > 1.8.0_181.
> > > * Python artifacts are deployed along with the source release to the
> > > dist.apache.org <http://dist.apache.org> <http://dist.apache.org>
> > [2].
> > > * Validation sheet with a tab for 2.12.0 release to help with
> > validation
> > > [9].
> > >
> > > The vote will be open for at least 72 hours. It is adopted by
> > majority
> > > approval, with at least 3 PMC affirmative votes.
> > >
> > > Thanks,
> > > Andrew
> > >
> > > 1]
> > https://jira.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12344944
> > > [2] https://dist.apache.org/repos/dist/dev/beam/2.12.0/
> > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> > > [4]
> > https://repository.apache.org/content/repositories/orgapachebeam-1068/
> > > [5] https://github.com/apache/beam/tree/v2.12.0-RC4
> > > [6] https://github.com/apache/beam/pull/8215
> > > [7] https://github.com/apache/beam-site/pull/588
> > > [8] https://github.com/apache/beam/pull/8314
> > > [9]
> > https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984
> >
> > --
> > Jean-Baptiste Onofré
> > jbonofre@apache.org <ma...@apache.org>
> > http://blog.nanthrax.net
> > Talend - http://www.talend.com
> >
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Etienne Chauchot <ec...@apache.org>.
Hi guys ,I will vote -1 (binding) on this RC (although degradation is before RC4 cut date). I took a look at Nexmark
graphs for the 3 major runners :- there seem to have functional regressions on Dataflow:
https://apache-beam-testing.appspot.com/explore?dashboard=5647201107705856 . 13 queries fail in batch mode starting on
04/17- there is a perf degradation (+200%) in spark runner starting on 04/10 for all the queries:
https://apache-beam-testing.appspot.com/explore?dashboard=5138380291571712
Sorry Andrew for the added work
Etienne
Le lundi 22 avril 2019 à 12:21 -0700, Andrew Pilloud a écrit :
> I signed the wheels files and updated the build process to not require giving travis apache credentials. (You should
> probably change your password if you haven't already.)
> Andrew
> On Mon, Apr 22, 2019 at 12:18 PM Ahmet Altay <al...@google.com> wrote:
> > +1 (binding)
> >
> > Verified the python 2 wheel files with quick start examples.
> > On Mon, Apr 22, 2019 at 11:26 AM Ahmet Altay <al...@google.com> wrote:
> > > I built the wheel files. They are in the usual place along with other python artifacts. I will test them a bit and
> > > update here. Could someone else please try the wheel files as well?
> > > Andrew, could you sign and hash the wheel files?
> > > On Mon, Apr 22, 2019 at 10:11 AM Ahmet Altay <al...@google.com> wrote:
> > > > I verified- signatures and hashes. - python streaming quickstart guide
> > > >
> > > > I would like to verify the wheel files before voting. Please let us know when they are ready. Also, if you need
> > > > help with building wheel files I can help/build.
> > > >
> > > > Ahmet
> > > > On Mon, Apr 22, 2019 at 3:33 AM Maximilian Michels <mx...@apache.org> wrote:
> > > > > +1 (binding)
> > > > >
> > > > >
> > > > >
> > > > > Found a minor bug while testing, but not a blocker:
> > > > >
> > > > > https://jira.apache.org/jira/browse/BEAM-7128
> > > > >
> > > > >
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Max
> > > > >
> > > > >
> > > > >
> > > > > On 20.04.19 23:02, Pablo Estrada wrote:
> > > > >
> > > > > > +1
> > > > >
> > > > > > Ran SQL postcommit, and Dataflow Portability Java validatesrunner tests.
> > > > >
> > > > > >
> > > > >
> > > > > > -P.
> > > > >
> > > > > >
> > > > >
> > > > > > On Wed, Apr 17, 2019 at 1:38 AM Jean-Baptiste Onofré <jb@nanthrax.net
> > > > >
> > > > > > <ma...@nanthrax.net>> wrote:
> > > > >
> > > > > >
> > > > >
> > > > > > +1 (binding)
> > > > >
> > > > > >
> > > > >
> > > > > > Quickly checked with beam-samples.
> > > > >
> > > > > >
> > > > >
> > > > > > Regards
> > > > >
> > > > > > JB
> > > > >
> > > > > >
> > > > >
> > > > > > On 16/04/2019 00:50, Andrew Pilloud wrote:
> > > > >
> > > > > > > Hi everyone,
> > > > >
> > > > > > >
> > > > >
> > > > > > > Please review and vote on the release candidate #4 for the version
> > > > >
> > > > > > > 2.12.0, as follows:
> > > > >
> > > > > > >
> > > > >
> > > > > > > [ ] +1, Approve the release
> > > > >
> > > > > > > [ ] -1, Do not approve the release (please provide specific comments)
> > > > >
> > > > > > >
> > > > >
> > > > > > > The complete staging area is available for your review, which
> > > > >
> > > > > > includes:
> > > > >
> > > > > > > * JIRA release notes [1],
> > > > >
> > > > > > > * the official Apache source release to be deployed to
> > > > >
> > > > > > dist.apache.org <http://dist.apache.org>
> > > > >
> > > > > > > <http://dist.apache.org> [2], which is signed with the key with
> > > > >
> > > > > > > fingerprint 9E7CEC0661EFD610B632C610AE8FE17F9F8AE3D4 [3],
> > > > >
> > > > > > > * all artifacts to be deployed to the Maven Central Repository [4],
> > > > >
> > > > > > > * source code tag "v2.12.0-RC4" [5],
> > > > >
> > > > > > > * website pull request listing the release [6], publishing the API
> > > > >
> > > > > > > reference manual [7], and the blog post [8].
> > > > >
> > > > > > > * Java artifacts were built with Gradle/5.2.1 and OpenJDK/Oracle JDK
> > > > >
> > > > > > > 1.8.0_181.
> > > > >
> > > > > > > * Python artifacts are deployed along with the source release to the
> > > > >
> > > > > > > dist.apache.org <http://dist.apache.org> <http://dist.apache.org>
> > > > >
> > > > > > [2].
> > > > >
> > > > > > > * Validation sheet with a tab for 2.12.0 release to help with
> > > > >
> > > > > > validation
> > > > >
> > > > > > > [9].
> > > > >
> > > > > > >
> > > > >
> > > > > > > The vote will be open for at least 72 hours. It is adopted by
> > > > >
> > > > > > majority
> > > > >
> > > > > > > approval, with at least 3 PMC affirmative votes.
> > > > >
> > > > > > >
> > > > >
> > > > > > > Thanks,
> > > > >
> > > > > > > Andrew
> > > > >
> > > > > > >
> > > > >
> > > > > > > 1]
> > > > >
> > > > > > https://jira.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12344944
> > > > >
> > > > > > > [2] https://dist.apache.org/repos/dist/dev/beam/2.12.0/
> > > > >
> > > > > > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> > > > >
> > > > > > > [4]
> > > > >
> > > > > > https://repository.apache.org/content/repositories/orgapachebeam-1068/
> > > > >
> > > > > > > [5] https://github.com/apache/beam/tree/v2.12.0-RC4
> > > > >
> > > > > > > [6] https://github.com/apache/beam/pull/8215
> > > > >
> > > > > > > [7] https://github.com/apache/beam-site/pull/588
> > > > >
> > > > > > > [8] https://github.com/apache/beam/pull/8314
> > > > >
> > > > > > > [9]
> > > > >
> > > > > > https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984
> > > > >
> > > > > >
> > > > >
> > > > > > --
> > > > >
> > > > > > Jean-Baptiste Onofré
> > > > >
> > > > > > jbonofre@apache.org <ma...@apache.org>
> > > > >
> > > > > > http://blog.nanthrax.net
> > > > >
> > > > > > Talend - http://www.talend.com
> > > > >
> > > > > >
> > > > >
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Andrew Pilloud <ap...@google.com>.
I signed the wheels files and updated the build process to not require
giving travis apache credentials. (You should probably change your password
if you haven't already.)
Andrew
On Mon, Apr 22, 2019 at 12:18 PM Ahmet Altay <al...@google.com> wrote:
> +1 (binding)
>
> Verified the python 2 wheel files with quick start examples.
>
> On Mon, Apr 22, 2019 at 11:26 AM Ahmet Altay <al...@google.com> wrote:
>
>> I built the wheel files. They are in the usual place along with other
>> python artifacts. I will test them a bit and update here. Could someone
>> else please try the wheel files as well?
>>
>> Andrew, could you sign and hash the wheel files?
>>
>> On Mon, Apr 22, 2019 at 10:11 AM Ahmet Altay <al...@google.com> wrote:
>>
>>> I verified
>>> - signatures and hashes.
>>> - python streaming quickstart guide
>>>
>>> I would like to verify the wheel files before voting. Please let us know
>>> when they are ready. Also, if you need help with building wheel files I can
>>> help/build.
>>>
>>> Ahmet
>>>
>>> On Mon, Apr 22, 2019 at 3:33 AM Maximilian Michels <mx...@apache.org>
>>> wrote:
>>>
>>>> +1 (binding)
>>>>
>>>> Found a minor bug while testing, but not a blocker:
>>>> https://jira.apache.org/jira/browse/BEAM-7128
>>>>
>>>> Thanks,
>>>> Max
>>>>
>>>> On 20.04.19 23:02, Pablo Estrada wrote:
>>>> > +1
>>>> > Ran SQL postcommit, and Dataflow Portability Java validatesrunner
>>>> tests.
>>>> >
>>>> > -P.
>>>> >
>>>> > On Wed, Apr 17, 2019 at 1:38 AM Jean-Baptiste Onofré <jb@nanthrax.net
>>>> > <ma...@nanthrax.net>> wrote:
>>>> >
>>>> > +1 (binding)
>>>> >
>>>> > Quickly checked with beam-samples.
>>>> >
>>>> > Regards
>>>> > JB
>>>> >
>>>> > On 16/04/2019 00:50, Andrew Pilloud wrote:
>>>> > > Hi everyone,
>>>> > >
>>>> > > Please review and vote on the release candidate #4 for the
>>>> version
>>>> > > 2.12.0, as follows:
>>>> > >
>>>> > > [ ] +1, Approve the release
>>>> > > [ ] -1, Do not approve the release (please provide specific
>>>> comments)
>>>> > >
>>>> > > The complete staging area is available for your review, which
>>>> > includes:
>>>> > > * JIRA release notes [1],
>>>> > > * the official Apache source release to be deployed to
>>>> > dist.apache.org <http://dist.apache.org>
>>>> > > <http://dist.apache.org> [2], which is signed with the key
>>>> with
>>>> > > fingerprint 9E7CEC0661EFD610B632C610AE8FE17F9F8AE3D4 [3],
>>>> > > * all artifacts to be deployed to the Maven Central Repository
>>>> [4],
>>>> > > * source code tag "v2.12.0-RC4" [5],
>>>> > > * website pull request listing the release [6], publishing the
>>>> API
>>>> > > reference manual [7], and the blog post [8].
>>>> > > * Java artifacts were built with Gradle/5.2.1 and
>>>> OpenJDK/Oracle JDK
>>>> > > 1.8.0_181.
>>>> > > * Python artifacts are deployed along with the source release
>>>> to the
>>>> > > dist.apache.org <http://dist.apache.org> <
>>>> http://dist.apache.org>
>>>> > [2].
>>>> > > * Validation sheet with a tab for 2.12.0 release to help with
>>>> > validation
>>>> > > [9].
>>>> > >
>>>> > > The vote will be open for at least 72 hours. It is adopted by
>>>> > majority
>>>> > > approval, with at least 3 PMC affirmative votes.
>>>> > >
>>>> > > Thanks,
>>>> > > Andrew
>>>> > >
>>>> > > 1]
>>>> >
>>>> https://jira.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12344944
>>>> > > [2] https://dist.apache.org/repos/dist/dev/beam/2.12.0/
>>>> > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>>> > > [4]
>>>> >
>>>> https://repository.apache.org/content/repositories/orgapachebeam-1068/
>>>> > > [5] https://github.com/apache/beam/tree/v2.12.0-RC4
>>>> > > [6] https://github.com/apache/beam/pull/8215
>>>> > > [7] https://github.com/apache/beam-site/pull/588
>>>> > > [8] https://github.com/apache/beam/pull/8314
>>>> > > [9]
>>>> >
>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984
>>>> >
>>>> > --
>>>> > Jean-Baptiste Onofré
>>>> > jbonofre@apache.org <ma...@apache.org>
>>>> > http://blog.nanthrax.net
>>>> > Talend - http://www.talend.com
>>>> >
>>>>
>>>
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Ahmet Altay <al...@google.com>.
+1 (binding)
Verified the python 2 wheel files with quick start examples.
On Mon, Apr 22, 2019 at 11:26 AM Ahmet Altay <al...@google.com> wrote:
> I built the wheel files. They are in the usual place along with other
> python artifacts. I will test them a bit and update here. Could someone
> else please try the wheel files as well?
>
> Andrew, could you sign and hash the wheel files?
>
> On Mon, Apr 22, 2019 at 10:11 AM Ahmet Altay <al...@google.com> wrote:
>
>> I verified
>> - signatures and hashes.
>> - python streaming quickstart guide
>>
>> I would like to verify the wheel files before voting. Please let us know
>> when they are ready. Also, if you need help with building wheel files I can
>> help/build.
>>
>> Ahmet
>>
>> On Mon, Apr 22, 2019 at 3:33 AM Maximilian Michels <mx...@apache.org>
>> wrote:
>>
>>> +1 (binding)
>>>
>>> Found a minor bug while testing, but not a blocker:
>>> https://jira.apache.org/jira/browse/BEAM-7128
>>>
>>> Thanks,
>>> Max
>>>
>>> On 20.04.19 23:02, Pablo Estrada wrote:
>>> > +1
>>> > Ran SQL postcommit, and Dataflow Portability Java validatesrunner
>>> tests.
>>> >
>>> > -P.
>>> >
>>> > On Wed, Apr 17, 2019 at 1:38 AM Jean-Baptiste Onofré <jb@nanthrax.net
>>> > <ma...@nanthrax.net>> wrote:
>>> >
>>> > +1 (binding)
>>> >
>>> > Quickly checked with beam-samples.
>>> >
>>> > Regards
>>> > JB
>>> >
>>> > On 16/04/2019 00:50, Andrew Pilloud wrote:
>>> > > Hi everyone,
>>> > >
>>> > > Please review and vote on the release candidate #4 for the
>>> version
>>> > > 2.12.0, as follows:
>>> > >
>>> > > [ ] +1, Approve the release
>>> > > [ ] -1, Do not approve the release (please provide specific
>>> comments)
>>> > >
>>> > > The complete staging area is available for your review, which
>>> > includes:
>>> > > * JIRA release notes [1],
>>> > > * the official Apache source release to be deployed to
>>> > dist.apache.org <http://dist.apache.org>
>>> > > <http://dist.apache.org> [2], which is signed with the key with
>>> > > fingerprint 9E7CEC0661EFD610B632C610AE8FE17F9F8AE3D4 [3],
>>> > > * all artifacts to be deployed to the Maven Central Repository
>>> [4],
>>> > > * source code tag "v2.12.0-RC4" [5],
>>> > > * website pull request listing the release [6], publishing the
>>> API
>>> > > reference manual [7], and the blog post [8].
>>> > > * Java artifacts were built with Gradle/5.2.1 and
>>> OpenJDK/Oracle JDK
>>> > > 1.8.0_181.
>>> > > * Python artifacts are deployed along with the source release
>>> to the
>>> > > dist.apache.org <http://dist.apache.org> <
>>> http://dist.apache.org>
>>> > [2].
>>> > > * Validation sheet with a tab for 2.12.0 release to help with
>>> > validation
>>> > > [9].
>>> > >
>>> > > The vote will be open for at least 72 hours. It is adopted by
>>> > majority
>>> > > approval, with at least 3 PMC affirmative votes.
>>> > >
>>> > > Thanks,
>>> > > Andrew
>>> > >
>>> > > 1]
>>> >
>>> https://jira.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12344944
>>> > > [2] https://dist.apache.org/repos/dist/dev/beam/2.12.0/
>>> > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>> > > [4]
>>> >
>>> https://repository.apache.org/content/repositories/orgapachebeam-1068/
>>> > > [5] https://github.com/apache/beam/tree/v2.12.0-RC4
>>> > > [6] https://github.com/apache/beam/pull/8215
>>> > > [7] https://github.com/apache/beam-site/pull/588
>>> > > [8] https://github.com/apache/beam/pull/8314
>>> > > [9]
>>> >
>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984
>>> >
>>> > --
>>> > Jean-Baptiste Onofré
>>> > jbonofre@apache.org <ma...@apache.org>
>>> > http://blog.nanthrax.net
>>> > Talend - http://www.talend.com
>>> >
>>>
>>
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Ahmet Altay <al...@google.com>.
I built the wheel files. They are in the usual place along with other
python artifacts. I will test them a bit and update here. Could someone
else please try the wheel files as well?
Andrew, could you sign and hash the wheel files?
On Mon, Apr 22, 2019 at 10:11 AM Ahmet Altay <al...@google.com> wrote:
> I verified
> - signatures and hashes.
> - python streaming quickstart guide
>
> I would like to verify the wheel files before voting. Please let us know
> when they are ready. Also, if you need help with building wheel files I can
> help/build.
>
> Ahmet
>
> On Mon, Apr 22, 2019 at 3:33 AM Maximilian Michels <mx...@apache.org> wrote:
>
>> +1 (binding)
>>
>> Found a minor bug while testing, but not a blocker:
>> https://jira.apache.org/jira/browse/BEAM-7128
>>
>> Thanks,
>> Max
>>
>> On 20.04.19 23:02, Pablo Estrada wrote:
>> > +1
>> > Ran SQL postcommit, and Dataflow Portability Java validatesrunner tests.
>> >
>> > -P.
>> >
>> > On Wed, Apr 17, 2019 at 1:38 AM Jean-Baptiste Onofré <jb@nanthrax.net
>> > <ma...@nanthrax.net>> wrote:
>> >
>> > +1 (binding)
>> >
>> > Quickly checked with beam-samples.
>> >
>> > Regards
>> > JB
>> >
>> > On 16/04/2019 00:50, Andrew Pilloud wrote:
>> > > Hi everyone,
>> > >
>> > > Please review and vote on the release candidate #4 for the
>> version
>> > > 2.12.0, as follows:
>> > >
>> > > [ ] +1, Approve the release
>> > > [ ] -1, Do not approve the release (please provide specific
>> comments)
>> > >
>> > > The complete staging area is available for your review, which
>> > includes:
>> > > * JIRA release notes [1],
>> > > * the official Apache source release to be deployed to
>> > dist.apache.org <http://dist.apache.org>
>> > > <http://dist.apache.org> [2], which is signed with the key with
>> > > fingerprint 9E7CEC0661EFD610B632C610AE8FE17F9F8AE3D4 [3],
>> > > * all artifacts to be deployed to the Maven Central Repository
>> [4],
>> > > * source code tag "v2.12.0-RC4" [5],
>> > > * website pull request listing the release [6], publishing the
>> API
>> > > reference manual [7], and the blog post [8].
>> > > * Java artifacts were built with Gradle/5.2.1 and OpenJDK/Oracle
>> JDK
>> > > 1.8.0_181.
>> > > * Python artifacts are deployed along with the source release to
>> the
>> > > dist.apache.org <http://dist.apache.org> <http://dist.apache.org
>> >
>> > [2].
>> > > * Validation sheet with a tab for 2.12.0 release to help with
>> > validation
>> > > [9].
>> > >
>> > > The vote will be open for at least 72 hours. It is adopted by
>> > majority
>> > > approval, with at least 3 PMC affirmative votes.
>> > >
>> > > Thanks,
>> > > Andrew
>> > >
>> > > 1]
>> >
>> https://jira.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12344944
>> > > [2] https://dist.apache.org/repos/dist/dev/beam/2.12.0/
>> > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>> > > [4]
>> >
>> https://repository.apache.org/content/repositories/orgapachebeam-1068/
>> > > [5] https://github.com/apache/beam/tree/v2.12.0-RC4
>> > > [6] https://github.com/apache/beam/pull/8215
>> > > [7] https://github.com/apache/beam-site/pull/588
>> > > [8] https://github.com/apache/beam/pull/8314
>> > > [9]
>> >
>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984
>> >
>> > --
>> > Jean-Baptiste Onofré
>> > jbonofre@apache.org <ma...@apache.org>
>> > http://blog.nanthrax.net
>> > Talend - http://www.talend.com
>> >
>>
>
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Ahmet Altay <al...@google.com>.
I verified
- signatures and hashes.
- python streaming quickstart guide
I would like to verify the wheel files before voting. Please let us know
when they are ready. Also, if you need help with building wheel files I can
help/build.
Ahmet
On Mon, Apr 22, 2019 at 3:33 AM Maximilian Michels <mx...@apache.org> wrote:
> +1 (binding)
>
> Found a minor bug while testing, but not a blocker:
> https://jira.apache.org/jira/browse/BEAM-7128
>
> Thanks,
> Max
>
> On 20.04.19 23:02, Pablo Estrada wrote:
> > +1
> > Ran SQL postcommit, and Dataflow Portability Java validatesrunner tests.
> >
> > -P.
> >
> > On Wed, Apr 17, 2019 at 1:38 AM Jean-Baptiste Onofré <jb@nanthrax.net
> > <ma...@nanthrax.net>> wrote:
> >
> > +1 (binding)
> >
> > Quickly checked with beam-samples.
> >
> > Regards
> > JB
> >
> > On 16/04/2019 00:50, Andrew Pilloud wrote:
> > > Hi everyone,
> > >
> > > Please review and vote on the release candidate #4 for the version
> > > 2.12.0, as follows:
> > >
> > > [ ] +1, Approve the release
> > > [ ] -1, Do not approve the release (please provide specific
> comments)
> > >
> > > The complete staging area is available for your review, which
> > includes:
> > > * JIRA release notes [1],
> > > * the official Apache source release to be deployed to
> > dist.apache.org <http://dist.apache.org>
> > > <http://dist.apache.org> [2], which is signed with the key with
> > > fingerprint 9E7CEC0661EFD610B632C610AE8FE17F9F8AE3D4 [3],
> > > * all artifacts to be deployed to the Maven Central Repository
> [4],
> > > * source code tag "v2.12.0-RC4" [5],
> > > * website pull request listing the release [6], publishing the API
> > > reference manual [7], and the blog post [8].
> > > * Java artifacts were built with Gradle/5.2.1 and OpenJDK/Oracle
> JDK
> > > 1.8.0_181.
> > > * Python artifacts are deployed along with the source release to
> the
> > > dist.apache.org <http://dist.apache.org> <http://dist.apache.org>
> > [2].
> > > * Validation sheet with a tab for 2.12.0 release to help with
> > validation
> > > [9].
> > >
> > > The vote will be open for at least 72 hours. It is adopted by
> > majority
> > > approval, with at least 3 PMC affirmative votes.
> > >
> > > Thanks,
> > > Andrew
> > >
> > > 1]
> >
> https://jira.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12344944
> > > [2] https://dist.apache.org/repos/dist/dev/beam/2.12.0/
> > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> > > [4]
> >
> https://repository.apache.org/content/repositories/orgapachebeam-1068/
> > > [5] https://github.com/apache/beam/tree/v2.12.0-RC4
> > > [6] https://github.com/apache/beam/pull/8215
> > > [7] https://github.com/apache/beam-site/pull/588
> > > [8] https://github.com/apache/beam/pull/8314
> > > [9]
> >
> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984
> >
> > --
> > Jean-Baptiste Onofré
> > jbonofre@apache.org <ma...@apache.org>
> > http://blog.nanthrax.net
> > Talend - http://www.talend.com
> >
>
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Maximilian Michels <mx...@apache.org>.
+1 (binding)
Found a minor bug while testing, but not a blocker:
https://jira.apache.org/jira/browse/BEAM-7128
Thanks,
Max
On 20.04.19 23:02, Pablo Estrada wrote:
> +1
> Ran SQL postcommit, and Dataflow Portability Java validatesrunner tests.
>
> -P.
>
> On Wed, Apr 17, 2019 at 1:38 AM Jean-Baptiste Onofré <jb@nanthrax.net
> <ma...@nanthrax.net>> wrote:
>
> +1 (binding)
>
> Quickly checked with beam-samples.
>
> Regards
> JB
>
> On 16/04/2019 00:50, Andrew Pilloud wrote:
> > Hi everyone,
> >
> > Please review and vote on the release candidate #4 for the version
> > 2.12.0, as follows:
> >
> > [ ] +1, Approve the release
> > [ ] -1, Do not approve the release (please provide specific comments)
> >
> > The complete staging area is available for your review, which
> includes:
> > * JIRA release notes [1],
> > * the official Apache source release to be deployed to
> dist.apache.org <http://dist.apache.org>
> > <http://dist.apache.org> [2], which is signed with the key with
> > fingerprint 9E7CEC0661EFD610B632C610AE8FE17F9F8AE3D4 [3],
> > * all artifacts to be deployed to the Maven Central Repository [4],
> > * source code tag "v2.12.0-RC4" [5],
> > * website pull request listing the release [6], publishing the API
> > reference manual [7], and the blog post [8].
> > * Java artifacts were built with Gradle/5.2.1 and OpenJDK/Oracle JDK
> > 1.8.0_181.
> > * Python artifacts are deployed along with the source release to the
> > dist.apache.org <http://dist.apache.org> <http://dist.apache.org>
> [2].
> > * Validation sheet with a tab for 2.12.0 release to help with
> validation
> > [9].
> >
> > The vote will be open for at least 72 hours. It is adopted by
> majority
> > approval, with at least 3 PMC affirmative votes.
> >
> > Thanks,
> > Andrew
> >
> > 1]
> https://jira.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12344944
> > [2] https://dist.apache.org/repos/dist/dev/beam/2.12.0/
> > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> > [4]
> https://repository.apache.org/content/repositories/orgapachebeam-1068/
> > [5] https://github.com/apache/beam/tree/v2.12.0-RC4
> > [6] https://github.com/apache/beam/pull/8215
> > [7] https://github.com/apache/beam-site/pull/588
> > [8] https://github.com/apache/beam/pull/8314
> > [9]
> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org <ma...@apache.org>
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Pablo Estrada <pa...@google.com>.
+1
Ran SQL postcommit, and Dataflow Portability Java validatesrunner tests.
-P.
On Wed, Apr 17, 2019 at 1:38 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> +1 (binding)
>
> Quickly checked with beam-samples.
>
> Regards
> JB
>
> On 16/04/2019 00:50, Andrew Pilloud wrote:
> > Hi everyone,
> >
> > Please review and vote on the release candidate #4 for the version
> > 2.12.0, as follows:
> >
> > [ ] +1, Approve the release
> > [ ] -1, Do not approve the release (please provide specific comments)
> >
> > The complete staging area is available for your review, which includes:
> > * JIRA release notes [1],
> > * the official Apache source release to be deployed to dist.apache.org
> > <http://dist.apache.org> [2], which is signed with the key with
> > fingerprint 9E7CEC0661EFD610B632C610AE8FE17F9F8AE3D4 [3],
> > * all artifacts to be deployed to the Maven Central Repository [4],
> > * source code tag "v2.12.0-RC4" [5],
> > * website pull request listing the release [6], publishing the API
> > reference manual [7], and the blog post [8].
> > * Java artifacts were built with Gradle/5.2.1 and OpenJDK/Oracle JDK
> > 1.8.0_181.
> > * Python artifacts are deployed along with the source release to the
> > dist.apache.org <http://dist.apache.org> [2].
> > * Validation sheet with a tab for 2.12.0 release to help with validation
> > [9].
> >
> > The vote will be open for at least 72 hours. It is adopted by majority
> > approval, with at least 3 PMC affirmative votes.
> >
> > Thanks,
> > Andrew
> >
> > 1]
> https://jira.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12344944
> > [2] https://dist.apache.org/repos/dist/dev/beam/2.12.0/
> > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> > [4]
> https://repository.apache.org/content/repositories/orgapachebeam-1068/
> > [5] https://github.com/apache/beam/tree/v2.12.0-RC4
> > [6] https://github.com/apache/beam/pull/8215
> > [7] https://github.com/apache/beam-site/pull/588
> > [8] https://github.com/apache/beam/pull/8314
> > [9]
> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
Re: [VOTE] Release 2.12.0, release candidate #4
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
+1 (binding)
Quickly checked with beam-samples.
Regards
JB
On 16/04/2019 00:50, Andrew Pilloud wrote:
> Hi everyone,
>
> Please review and vote on the release candidate #4 for the version
> 2.12.0, as follows:
>
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
> The complete staging area is available for your review, which includes:
> * JIRA release notes [1],
> * the official Apache source release to be deployed to dist.apache.org
> <http://dist.apache.org> [2], which is signed with the key with
> fingerprint 9E7CEC0661EFD610B632C610AE8FE17F9F8AE3D4 [3],
> * all artifacts to be deployed to the Maven Central Repository [4],
> * source code tag "v2.12.0-RC4" [5],
> * website pull request listing the release [6], publishing the API
> reference manual [7], and the blog post [8].
> * Java artifacts were built with Gradle/5.2.1 and OpenJDK/Oracle JDK
> 1.8.0_181.
> * Python artifacts are deployed along with the source release to the
> dist.apache.org <http://dist.apache.org> [2].
> * Validation sheet with a tab for 2.12.0 release to help with validation
> [9].
>
> The vote will be open for at least 72 hours. It is adopted by majority
> approval, with at least 3 PMC affirmative votes.
>
> Thanks,
> Andrew
>
> 1] https://jira.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12344944
> [2] https://dist.apache.org/repos/dist/dev/beam/2.12.0/
> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> [4] https://repository.apache.org/content/repositories/orgapachebeam-1068/
> [5] https://github.com/apache/beam/tree/v2.12.0-RC4
> [6] https://github.com/apache/beam/pull/8215
> [7] https://github.com/apache/beam-site/pull/588
> [8] https://github.com/apache/beam/pull/8314
> [9] https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com