You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Jean-Baptiste Onofré <jb...@nanthrax.net> on 2018/01/30 08:04:27 UTC
[VOTE] Release 2.3.0, release candidate #1
Hi everyone,
Please review and vote on the release candidate #1 for the version 2.3.0, as
follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)
The complete staging area is available for your review, which includes:
* JIRA release notes [1],
* the official Apache source release to be deployed to dist.apache.org [2],
which is signed with the key with fingerprint C8282E76 [3],
* all artifacts to be deployed to the Maven Central Repository [4],
* source code tag "v2.3.0-RC1" [5],
* website pull request listing the release and publishing the API reference
manual [6].
* Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
* Python artifacts are deployed along with the source release to the
dist.apache.org [2].
The vote will be open for at least 72 hours. It is adopted by majority approval,
with at least 3 PMC affirmative votes.
Thanks,
JB
[1]
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
[2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
[3] https://dist.apache.org/repos/dist/release/beam/KEYS
[4] https://repository.apache.org/content/repositories/orgapachebeam-1026/
[5] https://github.com/apache/beam/tree/v2.3.0-RC1
[6] https://github.com/apache/beam-site/pull/381
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: [VOTE] Release 2.3.0, release candidate #1
Posted by Romain Manni-Bucau <rm...@gmail.com>.
+1 (non-binding), upgraded to spark 2 in several test suites and
integrations and works very well. Good job guys!
Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> | Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau>
2018-01-30 12:16 GMT+01:00 Colm O hEigeartaigh <co...@apache.org>:
>
> +1 (non-binding), tested digests, signatures, built the source repo + tag.
>
> Colm.
>
> On Tue, Jan 30, 2018 at 8:04 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
>
>> Hi everyone,
>>
>> Please review and vote on the release candidate #1 for the version 2.3.0,
>> as
>> follows:
>>
>> [ ] +1, Approve the release
>> [ ] -1, Do not approve the release (please provide specific comments)
>>
>>
>> The complete staging area is available for your review, which includes:
>> * JIRA release notes [1],
>> * the official Apache source release to be deployed to dist.apache.org
>> [2],
>> which is signed with the key with fingerprint C8282E76 [3],
>> * all artifacts to be deployed to the Maven Central Repository [4],
>> * source code tag "v2.3.0-RC1" [5],
>> * website pull request listing the release and publishing the API
>> reference
>> manual [6].
>> * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
>> * Python artifacts are deployed along with the source release to the
>> dist.apache.org [2].
>>
>> The vote will be open for at least 72 hours. It is adopted by majority
>> approval,
>> with at least 3 PMC affirmative votes.
>>
>> Thanks,
>> JB
>>
>> [1]
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje
>> ctId=12319527&version=12341608
>> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>> [4] https://repository.apache.org/content/repositories/orgapache
>> beam-1026/
>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>> [6] https://github.com/apache/beam-site/pull/381
>> --
>> Jean-Baptiste Onofré
>> jbonofre@apache.org
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>
>
>
>
> --
> Colm O hEigeartaigh
>
> Talend Community Coder
> http://coders.talend.com
>
Re: [VOTE] Release 2.3.0, release candidate #1
Posted by Colm O hEigeartaigh <co...@apache.org>.
+1 (non-binding), tested digests, signatures, built the source repo + tag.
Colm.
On Tue, Jan 30, 2018 at 8:04 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> Hi everyone,
>
> Please review and vote on the release candidate #1 for the version 2.3.0,
> as
> follows:
>
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
>
> The complete staging area is available for your review, which includes:
> * JIRA release notes [1],
> * the official Apache source release to be deployed to dist.apache.org
> [2],
> which is signed with the key with fingerprint C8282E76 [3],
> * all artifacts to be deployed to the Maven Central Repository [4],
> * source code tag "v2.3.0-RC1" [5],
> * website pull request listing the release and publishing the API reference
> manual [6].
> * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
> * Python artifacts are deployed along with the source release to the
> dist.apache.org [2].
>
> The vote will be open for at least 72 hours. It is adopted by majority
> approval,
> with at least 3 PMC affirmative votes.
>
> Thanks,
> JB
>
> [1]
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12319527&version=12341608
> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> [4] https://repository.apache.org/content/repositories/orgapachebeam-1026/
> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
> [6] https://github.com/apache/beam-site/pull/381
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
--
Colm O hEigeartaigh
Talend Community Coder
http://coders.talend.com
Re: [VOTE] Release 2.3.0, release candidate #1
Posted by Romain Manni-Bucau <rm...@gmail.com>.
@ismael: any vote can be changes from -1 to +1 (or +-0) without additional
delay
Le 1 févr. 2018 03:15, "Lukasz Cwik" <lc...@google.com> a écrit :
> Note that a user reported TextIO being broken on Flink.
> Thread is here: https://lists.apache.org/thread.html/
> 47b16c94032392782505415e010970fd2a9480891c55c2f7b5de92bd@%
> 3Cuser.beam.apache.org%3E
> Can someone confirm/refute?
>
> On Wed, Jan 31, 2018 at 3:36 PM, Konstantinos Katsiapis <
> katsiapis@google.com> wrote:
>
>> +1 (non-binding). tensorflow.transform
>> <https://github.com/tensorflow/transform> 0.5.0 is blocked on Apache
>> Beam 2.3
>>
>>
>> On Wed, Jan 31, 2018 at 5:59 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
>> wrote:
>>
>>> +1 (binding)
>>>
>>> Casting my own +1 ;)
>>>
>>> Regards
>>> JB
>>>
>>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>>> > Hi everyone,
>>> >
>>> > Please review and vote on the release candidate #1 for the version
>>> 2.3.0, as
>>> > follows:
>>> >
>>> > [ ] +1, Approve the release
>>> > [ ] -1, Do not approve the release (please provide specific comments)
>>> >
>>> >
>>> > The complete staging area is available for your review, which includes:
>>> > * JIRA release notes [1],
>>> > * the official Apache source release to be deployed to dist.apache.org
>>> [2],
>>> > which is signed with the key with fingerprint C8282E76 [3],
>>> > * all artifacts to be deployed to the Maven Central Repository [4],
>>> > * source code tag "v2.3.0-RC1" [5],
>>> > * website pull request listing the release and publishing the API
>>> reference
>>> > manual [6].
>>> > * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
>>> > * Python artifacts are deployed along with the source release to the
>>> > dist.apache.org [2].
>>> >
>>> > The vote will be open for at least 72 hours. It is adopted by majority
>>> approval,
>>> > with at least 3 PMC affirmative votes.
>>> >
>>> > Thanks,
>>> > JB
>>> >
>>> > [1]
>>> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje
>>> ctId=12319527&version=12341608
>>> > [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>>> > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>> > [4] https://repository.apache.org/content/repositories/orgapache
>>> beam-1026/
>>> > [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>>> > [6] https://github.com/apache/beam-site/pull/381
>>> >
>>>
>>> --
>>> Jean-Baptiste Onofré
>>> jbonofre@apache.org
>>> http://blog.nanthrax.net
>>> Talend - http://www.talend.com
>>>
>>
>>
>>
>> --
>> Gus Katsiapis | Software Engineer | katsiapis@google.com | 650-918-7487
>> <(650)%20918-7487>
>>
>
>
Re: [VOTE] Release 2.3.0, release candidate #1
Posted by Lukasz Cwik <lc...@google.com>.
Note that a user reported TextIO being broken on Flink.
Thread is here:
https://lists.apache.org/thread.html/47b16c94032392782505415e010970fd2a9480891c55c2f7b5de92bd@%3Cuser.beam.apache.org%3E
Can someone confirm/refute?
On Wed, Jan 31, 2018 at 3:36 PM, Konstantinos Katsiapis <
katsiapis@google.com> wrote:
> +1 (non-binding). tensorflow.transform
> <https://github.com/tensorflow/transform> 0.5.0 is blocked on Apache Beam
> 2.3
>
>
> On Wed, Jan 31, 2018 at 5:59 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
>
>> +1 (binding)
>>
>> Casting my own +1 ;)
>>
>> Regards
>> JB
>>
>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>> > Hi everyone,
>> >
>> > Please review and vote on the release candidate #1 for the version
>> 2.3.0, as
>> > follows:
>> >
>> > [ ] +1, Approve the release
>> > [ ] -1, Do not approve the release (please provide specific comments)
>> >
>> >
>> > The complete staging area is available for your review, which includes:
>> > * JIRA release notes [1],
>> > * the official Apache source release to be deployed to dist.apache.org
>> [2],
>> > which is signed with the key with fingerprint C8282E76 [3],
>> > * all artifacts to be deployed to the Maven Central Repository [4],
>> > * source code tag "v2.3.0-RC1" [5],
>> > * website pull request listing the release and publishing the API
>> reference
>> > manual [6].
>> > * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
>> > * Python artifacts are deployed along with the source release to the
>> > dist.apache.org [2].
>> >
>> > The vote will be open for at least 72 hours. It is adopted by majority
>> approval,
>> > with at least 3 PMC affirmative votes.
>> >
>> > Thanks,
>> > JB
>> >
>> > [1]
>> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje
>> ctId=12319527&version=12341608
>> > [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>> > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>> > [4] https://repository.apache.org/content/repositories/orgapache
>> beam-1026/
>> > [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>> > [6] https://github.com/apache/beam-site/pull/381
>> >
>>
>> --
>> Jean-Baptiste Onofré
>> jbonofre@apache.org
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>
>
>
>
> --
> Gus Katsiapis | Software Engineer | katsiapis@google.com | 650-918-7487
> <(650)%20918-7487>
>
Re: [VOTE] Release 2.3.0, release candidate #1
Posted by Konstantinos Katsiapis <ka...@google.com>.
+1 (non-binding). tensorflow.transform
<https://github.com/tensorflow/transform> 0.5.0 is blocked on Apache Beam
2.3
On Wed, Jan 31, 2018 at 5:59 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> +1 (binding)
>
> Casting my own +1 ;)
>
> Regards
> JB
>
> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
> > Hi everyone,
> >
> > Please review and vote on the release candidate #1 for the version
> 2.3.0, as
> > follows:
> >
> > [ ] +1, Approve the release
> > [ ] -1, Do not approve the release (please provide specific comments)
> >
> >
> > The complete staging area is available for your review, which includes:
> > * JIRA release notes [1],
> > * the official Apache source release to be deployed to dist.apache.org
> [2],
> > which is signed with the key with fingerprint C8282E76 [3],
> > * all artifacts to be deployed to the Maven Central Repository [4],
> > * source code tag "v2.3.0-RC1" [5],
> > * website pull request listing the release and publishing the API
> reference
> > manual [6].
> > * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
> > * Python artifacts are deployed along with the source release to the
> > dist.apache.org [2].
> >
> > The vote will be open for at least 72 hours. It is adopted by majority
> approval,
> > with at least 3 PMC affirmative votes.
> >
> > Thanks,
> > JB
> >
> > [1]
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12319527&version=12341608
> > [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> > [4] https://repository.apache.org/content/repositories/
> orgapachebeam-1026/
> > [5] https://github.com/apache/beam/tree/v2.3.0-RC1
> > [6] https://github.com/apache/beam-site/pull/381
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
--
Gus Katsiapis | Software Engineer | katsiapis@google.com | 650-918-7487
Re: [VOTE] Release 2.3.0, release candidate #1
Posted by Ahmet Altay <al...@google.com>.
On Wed, Jan 31, 2018 at 2:09 PM, Ahmet Altay <al...@google.com> wrote:
> This will require a change in the Beam code, because image names are
> hardcoded in to code (python) and configuration (java). RC1 as it is will
> not work correctly with Cloud Dataflow.
>
Please ignore this. Looking at the issues Alan created this does not seem
to be correct. And RC1 should work as expected once Google related step is
completed.
>
> On Wed, Jan 31, 2018 at 2:08 PM, Reuven Lax <re...@google.com> wrote:
>
>> Hopefully we can validate soon. I believe some of the delays are because
>> of integrating major changes done over the last week (e.g. Java 8
>> migration).
>>
>> On Wed, Jan 31, 2018 at 2:04 PM, Ismaël Mejía <ie...@gmail.com> wrote:
>>
>>> What is the common procedure in cases like this ? Because it doesn't
>>> seems that it needs a re-vote, just an extra day or two for
>>> validation, any ideas JB ?
>>>
>>> On Wed, Jan 31, 2018 at 10:41 PM, Alan Myrvold <am...@google.com>
>>> wrote:
>>> > Yes, it is a dataflow step. Happy to test this again when they are
>>> > available.
>>> >
>>> > On Wed, Jan 31, 2018 at 1:39 PM, Jean-Baptiste Onofré <jb@nanthrax.net
>>> >
>>> > wrote:
>>> >>
>>> >> OK, I think I understood ;)
>>> >>
>>> >> So it's not "directly" related to Beam itself (it's more a Dataflow
>>> step
>>> >> to perform).
>>> >>
>>> >> @Alan, I think it's better to test first and then cast the vote. This
>>> kind
>>> >> of tests are valuable to validate the release and make sense. But vote
>>> >> should represent the state of the Beam release. So I think -1 vote is
>>> a bit
>>> >> too early before the test.
>>> >>
>>> >> Thanks !
>>> >> Regards
>>> >> JB
>>> >>
>>> >> On 31/01/2018 22:33, Reuven Lax wrote:
>>> >>>
>>> >>> It's just a step that needs to be peformed before the new release
>>> works
>>> >>> on Dataflow. Alan is saying that we've been unable to validate
>>> Dataflow so
>>> >>> far, as worker images are not yet built. Hopefully they'll be built
>>> soon,
>>> >>> and we'll be able to validate.
>>> >>>
>>> >>> On Wed, Jan 31, 2018 at 1:31 PM, Jean-Baptiste Onofré <
>>> jb@nanthrax.net
>>> >>> <ma...@nanthrax.net>> wrote:
>>> >>>
>>> >>> Hi Alan
>>> >>>
>>> >>> does it related to change in the codebase or in a
>>> dependency/related
>>> >>> project ?
>>> >>>
>>> >>> I mean: is it something we have to fix/change in Beam ?
>>> >>>
>>> >>> Just curious as I'm not sure what you mean by "worker images" ;)
>>> >>>
>>> >>> Thanks !
>>> >>> Regards
>>> >>> JB
>>> >>>
>>> >>> On 31/01/2018 22:18, Alan Myrvold wrote:
>>> >>>
>>> >>> -1 (for now, hope to change this)
>>> >>>
>>> >>> Dataflow runner jobs are failing for me with 2.3.0 RC1, for
>>> both
>>> >>> Java and Python.
>>> >>>
>>> >>> This is not an issues with the 2.3.0 RC1 SDK, we (google)
>>> need
>>> >>> to release worker images.
>>> >>>
>>> >>> I have assigned these bugs to myself, and will be working to
>>> >>> help get these workers released.
>>> >>>
>>> >>> [BEAM-3584] Java dataflow job fails with 2.3.0 RC1, due to
>>> >>> missing worker image
>>> >>> [BEAM-3585] Python dataflow job fails with 2.3.0 RC1, due to
>>> >>> missing worker image
>>> >>>
>>> >>> On Wed, Jan 31, 2018 at 6:12 AM, Jean-Baptiste Onofré
>>> >>> <jb@nanthrax.net <ma...@nanthrax.net>
>>> >>> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
>>> >>>
>>> >>> Thanks Kenn,
>>> >>>
>>> >>> I prepared the list of tasks I did. I will complete
>>> where
>>> >>> release is
>>> >>> out.
>>> >>>
>>> >>> Regards
>>> >>> JB
>>> >>>
>>> >>> On 01/31/2018 03:07 PM, Kenneth Knowles wrote:
>>> >>> > I've cloned the release validation spreadsheet:
>>> >>> >
>>> >>> > https://s.apache.org/beam-2.3.0-release-validation
>>> >>> <https://s.apache.org/beam-2.3.0-release-validation>
>>> >>> <https://s.apache.org/beam-2.3.0-release-validation
>>> >>> <https://s.apache.org/beam-2.3.0-release-validation>>
>>> >>> >
>>> >>> > If you plan to perform a manual validation task,
>>> please
>>> >>> sign up so multiple
>>> >>> > people don't waste effort.
>>> >>> >
>>> >>> > Alan & JB, as far as your pairing up to automate more,
>>> >>> anything manual on this
>>> >>> > sheet should be considered.
>>> >>> >
>>> >>> > Kenn
>>> >>> >
>>> >>> > On Wed, Jan 31, 2018 at 5:59 AM, Jean-Baptiste Onofré
>>> >>> <jb@nanthrax.net <ma...@nanthrax.net>
>>> >>> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>
>>> >>> > <mailto:jb@nanthrax.net <ma...@nanthrax.net>
>>> >>> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>>> wrote:
>>> >>> >
>>> >>> > +1 (binding)
>>> >>> >
>>> >>> > Casting my own +1 ;)
>>> >>> >
>>> >>> > Regards
>>> >>> > JB
>>> >>> >
>>> >>> > On 01/30/2018 09:04 AM, Jean-Baptiste Onofré
>>> wrote:
>>> >>> > > Hi everyone,
>>> >>> > >
>>> >>> > > Please review and vote on the release candidate
>>> #1
>>> >>> for the version 2.3.0, as
>>> >>> > > follows:
>>> >>> > >
>>> >>> > > [ ] +1, Approve the release
>>> >>> > > [ ] -1, Do not approve the release (please
>>> provide
>>> >>> specific comments)
>>> >>> > >
>>> >>> > >
>>> >>> > > The complete staging area is available for your
>>> >>> review, which includes:
>>> >>> > > * JIRA release notes [1],
>>> >>> > > * the official Apache source release to be
>>> deployed
>>> >>> to dist.apache.org <http://dist.apache.org>
>>> >>> <http://dist.apache.org>
>>> >>> > <http://dist.apache.org> [2],
>>> >>> > > which is signed with the key with fingerprint
>>> >>> C8282E76 [3],
>>> >>> > > * all artifacts to be deployed to the Maven
>>> Central
>>> >>> Repository [4],
>>> >>> > > * source code tag "v2.3.0-RC1" [5],
>>> >>> > > * website pull request listing the release and
>>> >>> publishing the API reference
>>> >>> > > manual [6].
>>> >>> > > * Java artifacts were built with Maven 3.3.9 and
>>> >>> Oracle JDK 1.8.0_111.
>>> >>> > > * Python artifacts are deployed along with the
>>> >>> source release to the
>>> >>> > > dist.apache.org <http://dist.apache.org>
>>> >>> <http://dist.apache.org>
>>> >>> <http://dist.apache.org> [2].
>>> >>> > >
>>> >>> > > The vote will be open for at least 72 hours. It
>>> is
>>> >>> adopted by majority
>>> >>> > approval,
>>> >>> > > with at least 3 PMC affirmative votes.
>>> >>> > >
>>> >>> > > Thanks,
>>> >>> > > JB
>>> >>> > >
>>> >>> > > [1]
>>> >>> > >
>>> >>> >
>>> >>>
>>> >>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje
>>> ctId=12319527&version=12341608
>>> >>>
>>> >>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?proj
>>> ectId=12319527&version=12341608>
>>> >>>
>>> >>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?proj
>>> ectId=12319527&version=12341608
>>> >>>
>>> >>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?proj
>>> ectId=12319527&version=12341608>>
>>> >>> >
>>> >>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?proj
>>> ectId=12319527&version=12341608
>>> >>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?proj
>>> ectId=12319527&version=12341608>
>>> >>>
>>> >>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?proj
>>> ectId=12319527&version=12341608
>>> >>>
>>> >>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?proj
>>> ectId=12319527&version=12341608>>>
>>> >>> > > [2]
>>> >>> https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>>> >>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
>>> >>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>>> >>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>
>>> >>> > <https://dist.apache.org/repo
>>> s/dist/dev/beam/2.3.0/
>>> >>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
>>> >>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>>> >>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>>
>>> >>> > > [3]
>>> >>> https://dist.apache.org/repos/dist/release/beam/KEYS
>>> >>> <https://dist.apache.org/repos/dist/release/beam/KEYS>
>>> >>> <https://dist.apache.org/repos/dist/release/beam/KEYS
>>> >>> <https://dist.apache.org/repos/dist/release/beam/KEYS>>
>>> >>> > <https://dist.apache.org/repo
>>> s/dist/release/beam/KEYS
>>> >>> <https://dist.apache.org/repos/dist/release/beam/KEYS>
>>> >>> <https://dist.apache.org/repos/dist/release/beam/KEYS
>>> >>> <https://dist.apache.org/repos/dist/release/beam/KEYS>>>
>>> >>> > > [4]
>>> >>>
>>> >>> https://repository.apache.org/content/repositories/orgapache
>>> beam-1026/
>>> >>>
>>> >>> <https://repository.apache.org/content/repositories/orgapach
>>> ebeam-1026/>
>>> >>>
>>> >>> <https://repository.apache.org/content/repositories/orgapach
>>> ebeam-1026/
>>> >>>
>>> >>> <https://repository.apache.org/content/repositories/orgapach
>>> ebeam-1026/>>
>>> >>> >
>>> >>> <https://repository.apache.org/content/repositories/orgapach
>>> ebeam-1026/
>>> >>> <https://repository.apache.org/content/repositories/orgapach
>>> ebeam-1026/>
>>> >>>
>>> >>> <https://repository.apache.org/content/repositories/orgapach
>>> ebeam-1026/
>>> >>>
>>> >>> <https://repository.apache.org/content/repositories/orgapach
>>> ebeam-1026/>>>
>>> >>> > > [5] https://github.com/apache/beam
>>> /tree/v2.3.0-RC1
>>> >>> <https://github.com/apache/beam/tree/v2.3.0-RC1>
>>> >>> <https://github.com/apache/beam/tree/v2.3.0-RC1
>>> >>> <https://github.com/apache/beam/tree/v2.3.0-RC1>>
>>> >>> > <https://github.com/apache/beam/tree/v2.3.0-RC1
>>> >>> <https://github.com/apache/beam/tree/v2.3.0-RC1>
>>> >>> <https://github.com/apache/beam/tree/v2.3.0-RC1
>>> >>> <https://github.com/apache/beam/tree/v2.3.0-RC1>>>
>>> >>> > > [6] https://github.com/apache/beam
>>> -site/pull/381
>>> >>> <https://github.com/apache/beam-site/pull/381>
>>> >>> <https://github.com/apache/beam-site/pull/381
>>> >>> <https://github.com/apache/beam-site/pull/381>>
>>> >>> > <https://github.com/apache/beam-site/pull/381
>>> >>> <https://github.com/apache/beam-site/pull/381>
>>> >>> <https://github.com/apache/beam-site/pull/381
>>> >>> <https://github.com/apache/beam-site/pull/381>>>
>>> >>> > >
>>> >>> >
>>> >>> > --
>>> >>> > Jean-Baptiste Onofré
>>> >>> > jbonofre@apache.org <ma...@apache.org>
>>> >>> <mailto:jbonofre@apache.org <ma...@apache.org>>
>>> >>> <mailto:jbonofre@apache.org <mailto:jbonofre@apache.org
>>> >
>>> >>> <mailto:jbonofre@apache.org <ma...@apache.org>>>
>>> >>> > http://blog.nanthrax.net
>>> >>> > Talend - http://www.talend.com
>>> >>> >
>>> >>> >
>>> >>>
>>> >>> --
>>> >>> Jean-Baptiste Onofré
>>> >>> jbonofre@apache.org <ma...@apache.org>
>>> >>> <mailto:jbonofre@apache.org <ma...@apache.org>>
>>> >>> http://blog.nanthrax.net
>>> >>> Talend - http://www.talend.com
>>> >>>
>>> >>>
>>> >>>
>>> >
>>>
>>
>>
>
Re: [VOTE] Release 2.3.0, release candidate #1
Posted by Ahmet Altay <al...@google.com>.
This will require a change in the Beam code, because image names are
hardcoded in to code (python) and configuration (java). RC1 as it is will
not work correctly with Cloud Dataflow.
On Wed, Jan 31, 2018 at 2:08 PM, Reuven Lax <re...@google.com> wrote:
> Hopefully we can validate soon. I believe some of the delays are because
> of integrating major changes done over the last week (e.g. Java 8
> migration).
>
> On Wed, Jan 31, 2018 at 2:04 PM, Ismaël Mejía <ie...@gmail.com> wrote:
>
>> What is the common procedure in cases like this ? Because it doesn't
>> seems that it needs a re-vote, just an extra day or two for
>> validation, any ideas JB ?
>>
>> On Wed, Jan 31, 2018 at 10:41 PM, Alan Myrvold <am...@google.com>
>> wrote:
>> > Yes, it is a dataflow step. Happy to test this again when they are
>> > available.
>> >
>> > On Wed, Jan 31, 2018 at 1:39 PM, Jean-Baptiste Onofré <jb...@nanthrax.net>
>> > wrote:
>> >>
>> >> OK, I think I understood ;)
>> >>
>> >> So it's not "directly" related to Beam itself (it's more a Dataflow
>> step
>> >> to perform).
>> >>
>> >> @Alan, I think it's better to test first and then cast the vote. This
>> kind
>> >> of tests are valuable to validate the release and make sense. But vote
>> >> should represent the state of the Beam release. So I think -1 vote is
>> a bit
>> >> too early before the test.
>> >>
>> >> Thanks !
>> >> Regards
>> >> JB
>> >>
>> >> On 31/01/2018 22:33, Reuven Lax wrote:
>> >>>
>> >>> It's just a step that needs to be peformed before the new release
>> works
>> >>> on Dataflow. Alan is saying that we've been unable to validate
>> Dataflow so
>> >>> far, as worker images are not yet built. Hopefully they'll be built
>> soon,
>> >>> and we'll be able to validate.
>> >>>
>> >>> On Wed, Jan 31, 2018 at 1:31 PM, Jean-Baptiste Onofré <
>> jb@nanthrax.net
>> >>> <ma...@nanthrax.net>> wrote:
>> >>>
>> >>> Hi Alan
>> >>>
>> >>> does it related to change in the codebase or in a
>> dependency/related
>> >>> project ?
>> >>>
>> >>> I mean: is it something we have to fix/change in Beam ?
>> >>>
>> >>> Just curious as I'm not sure what you mean by "worker images" ;)
>> >>>
>> >>> Thanks !
>> >>> Regards
>> >>> JB
>> >>>
>> >>> On 31/01/2018 22:18, Alan Myrvold wrote:
>> >>>
>> >>> -1 (for now, hope to change this)
>> >>>
>> >>> Dataflow runner jobs are failing for me with 2.3.0 RC1, for
>> both
>> >>> Java and Python.
>> >>>
>> >>> This is not an issues with the 2.3.0 RC1 SDK, we (google) need
>> >>> to release worker images.
>> >>>
>> >>> I have assigned these bugs to myself, and will be working to
>> >>> help get these workers released.
>> >>>
>> >>> [BEAM-3584] Java dataflow job fails with 2.3.0 RC1, due to
>> >>> missing worker image
>> >>> [BEAM-3585] Python dataflow job fails with 2.3.0 RC1, due to
>> >>> missing worker image
>> >>>
>> >>> On Wed, Jan 31, 2018 at 6:12 AM, Jean-Baptiste Onofré
>> >>> <jb@nanthrax.net <ma...@nanthrax.net>
>> >>> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
>> >>>
>> >>> Thanks Kenn,
>> >>>
>> >>> I prepared the list of tasks I did. I will complete where
>> >>> release is
>> >>> out.
>> >>>
>> >>> Regards
>> >>> JB
>> >>>
>> >>> On 01/31/2018 03:07 PM, Kenneth Knowles wrote:
>> >>> > I've cloned the release validation spreadsheet:
>> >>> >
>> >>> > https://s.apache.org/beam-2.3.0-release-validation
>> >>> <https://s.apache.org/beam-2.3.0-release-validation>
>> >>> <https://s.apache.org/beam-2.3.0-release-validation
>> >>> <https://s.apache.org/beam-2.3.0-release-validation>>
>> >>> >
>> >>> > If you plan to perform a manual validation task, please
>> >>> sign up so multiple
>> >>> > people don't waste effort.
>> >>> >
>> >>> > Alan & JB, as far as your pairing up to automate more,
>> >>> anything manual on this
>> >>> > sheet should be considered.
>> >>> >
>> >>> > Kenn
>> >>> >
>> >>> > On Wed, Jan 31, 2018 at 5:59 AM, Jean-Baptiste Onofré
>> >>> <jb@nanthrax.net <ma...@nanthrax.net>
>> >>> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>
>> >>> > <mailto:jb@nanthrax.net <ma...@nanthrax.net>
>> >>> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>>> wrote:
>> >>> >
>> >>> > +1 (binding)
>> >>> >
>> >>> > Casting my own +1 ;)
>> >>> >
>> >>> > Regards
>> >>> > JB
>> >>> >
>> >>> > On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>> >>> > > Hi everyone,
>> >>> > >
>> >>> > > Please review and vote on the release candidate
>> #1
>> >>> for the version 2.3.0, as
>> >>> > > follows:
>> >>> > >
>> >>> > > [ ] +1, Approve the release
>> >>> > > [ ] -1, Do not approve the release (please
>> provide
>> >>> specific comments)
>> >>> > >
>> >>> > >
>> >>> > > The complete staging area is available for your
>> >>> review, which includes:
>> >>> > > * JIRA release notes [1],
>> >>> > > * the official Apache source release to be
>> deployed
>> >>> to dist.apache.org <http://dist.apache.org>
>> >>> <http://dist.apache.org>
>> >>> > <http://dist.apache.org> [2],
>> >>> > > which is signed with the key with fingerprint
>> >>> C8282E76 [3],
>> >>> > > * all artifacts to be deployed to the Maven
>> Central
>> >>> Repository [4],
>> >>> > > * source code tag "v2.3.0-RC1" [5],
>> >>> > > * website pull request listing the release and
>> >>> publishing the API reference
>> >>> > > manual [6].
>> >>> > > * Java artifacts were built with Maven 3.3.9 and
>> >>> Oracle JDK 1.8.0_111.
>> >>> > > * Python artifacts are deployed along with the
>> >>> source release to the
>> >>> > > dist.apache.org <http://dist.apache.org>
>> >>> <http://dist.apache.org>
>> >>> <http://dist.apache.org> [2].
>> >>> > >
>> >>> > > The vote will be open for at least 72 hours. It
>> is
>> >>> adopted by majority
>> >>> > approval,
>> >>> > > with at least 3 PMC affirmative votes.
>> >>> > >
>> >>> > > Thanks,
>> >>> > > JB
>> >>> > >
>> >>> > > [1]
>> >>> > >
>> >>> >
>> >>>
>> >>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje
>> ctId=12319527&version=12341608
>> >>>
>> >>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?proj
>> ectId=12319527&version=12341608>
>> >>>
>> >>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?proj
>> ectId=12319527&version=12341608
>> >>>
>> >>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?proj
>> ectId=12319527&version=12341608>>
>> >>> >
>> >>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?proj
>> ectId=12319527&version=12341608
>> >>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?proj
>> ectId=12319527&version=12341608>
>> >>>
>> >>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?proj
>> ectId=12319527&version=12341608
>> >>>
>> >>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?proj
>> ectId=12319527&version=12341608>>>
>> >>> > > [2]
>> >>> https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>> >>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
>> >>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>> >>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>
>> >>> > <https://dist.apache.org/repo
>> s/dist/dev/beam/2.3.0/
>> >>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
>> >>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>> >>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>>
>> >>> > > [3]
>> >>> https://dist.apache.org/repos/dist/release/beam/KEYS
>> >>> <https://dist.apache.org/repos/dist/release/beam/KEYS>
>> >>> <https://dist.apache.org/repos/dist/release/beam/KEYS
>> >>> <https://dist.apache.org/repos/dist/release/beam/KEYS>>
>> >>> > <https://dist.apache.org/repo
>> s/dist/release/beam/KEYS
>> >>> <https://dist.apache.org/repos/dist/release/beam/KEYS>
>> >>> <https://dist.apache.org/repos/dist/release/beam/KEYS
>> >>> <https://dist.apache.org/repos/dist/release/beam/KEYS>>>
>> >>> > > [4]
>> >>>
>> >>> https://repository.apache.org/content/repositories/orgapache
>> beam-1026/
>> >>>
>> >>> <https://repository.apache.org/content/repositories/orgapach
>> ebeam-1026/>
>> >>>
>> >>> <https://repository.apache.org/content/repositories/orgapach
>> ebeam-1026/
>> >>>
>> >>> <https://repository.apache.org/content/repositories/orgapach
>> ebeam-1026/>>
>> >>> >
>> >>> <https://repository.apache.org/content/repositories/orgapach
>> ebeam-1026/
>> >>> <https://repository.apache.org/content/repositories/orgapach
>> ebeam-1026/>
>> >>>
>> >>> <https://repository.apache.org/content/repositories/orgapach
>> ebeam-1026/
>> >>>
>> >>> <https://repository.apache.org/content/repositories/orgapach
>> ebeam-1026/>>>
>> >>> > > [5] https://github.com/apache/beam
>> /tree/v2.3.0-RC1
>> >>> <https://github.com/apache/beam/tree/v2.3.0-RC1>
>> >>> <https://github.com/apache/beam/tree/v2.3.0-RC1
>> >>> <https://github.com/apache/beam/tree/v2.3.0-RC1>>
>> >>> > <https://github.com/apache/beam/tree/v2.3.0-RC1
>> >>> <https://github.com/apache/beam/tree/v2.3.0-RC1>
>> >>> <https://github.com/apache/beam/tree/v2.3.0-RC1
>> >>> <https://github.com/apache/beam/tree/v2.3.0-RC1>>>
>> >>> > > [6] https://github.com/apache/beam-site/pull/381
>> >>> <https://github.com/apache/beam-site/pull/381>
>> >>> <https://github.com/apache/beam-site/pull/381
>> >>> <https://github.com/apache/beam-site/pull/381>>
>> >>> > <https://github.com/apache/beam-site/pull/381
>> >>> <https://github.com/apache/beam-site/pull/381>
>> >>> <https://github.com/apache/beam-site/pull/381
>> >>> <https://github.com/apache/beam-site/pull/381>>>
>> >>> > >
>> >>> >
>> >>> > --
>> >>> > Jean-Baptiste Onofré
>> >>> > jbonofre@apache.org <ma...@apache.org>
>> >>> <mailto:jbonofre@apache.org <ma...@apache.org>>
>> >>> <mailto:jbonofre@apache.org <ma...@apache.org>
>> >>> <mailto:jbonofre@apache.org <ma...@apache.org>>>
>> >>> > http://blog.nanthrax.net
>> >>> > Talend - http://www.talend.com
>> >>> >
>> >>> >
>> >>>
>> >>> --
>> >>> Jean-Baptiste Onofré
>> >>> jbonofre@apache.org <ma...@apache.org>
>> >>> <mailto:jbonofre@apache.org <ma...@apache.org>>
>> >>> http://blog.nanthrax.net
>> >>> Talend - http://www.talend.com
>> >>>
>> >>>
>> >>>
>> >
>>
>
>
Re: [VOTE] Release 2.3.0, release candidate #1
Posted by Reuven Lax <re...@google.com>.
Hopefully we can validate soon. I believe some of the delays are because of
integrating major changes done over the last week (e.g. Java 8 migration).
On Wed, Jan 31, 2018 at 2:04 PM, Ismaël Mejía <ie...@gmail.com> wrote:
> What is the common procedure in cases like this ? Because it doesn't
> seems that it needs a re-vote, just an extra day or two for
> validation, any ideas JB ?
>
> On Wed, Jan 31, 2018 at 10:41 PM, Alan Myrvold <am...@google.com>
> wrote:
> > Yes, it is a dataflow step. Happy to test this again when they are
> > available.
> >
> > On Wed, Jan 31, 2018 at 1:39 PM, Jean-Baptiste Onofré <jb...@nanthrax.net>
> > wrote:
> >>
> >> OK, I think I understood ;)
> >>
> >> So it's not "directly" related to Beam itself (it's more a Dataflow step
> >> to perform).
> >>
> >> @Alan, I think it's better to test first and then cast the vote. This
> kind
> >> of tests are valuable to validate the release and make sense. But vote
> >> should represent the state of the Beam release. So I think -1 vote is a
> bit
> >> too early before the test.
> >>
> >> Thanks !
> >> Regards
> >> JB
> >>
> >> On 31/01/2018 22:33, Reuven Lax wrote:
> >>>
> >>> It's just a step that needs to be peformed before the new release works
> >>> on Dataflow. Alan is saying that we've been unable to validate
> Dataflow so
> >>> far, as worker images are not yet built. Hopefully they'll be built
> soon,
> >>> and we'll be able to validate.
> >>>
> >>> On Wed, Jan 31, 2018 at 1:31 PM, Jean-Baptiste Onofré <jb@nanthrax.net
> >>> <ma...@nanthrax.net>> wrote:
> >>>
> >>> Hi Alan
> >>>
> >>> does it related to change in the codebase or in a
> dependency/related
> >>> project ?
> >>>
> >>> I mean: is it something we have to fix/change in Beam ?
> >>>
> >>> Just curious as I'm not sure what you mean by "worker images" ;)
> >>>
> >>> Thanks !
> >>> Regards
> >>> JB
> >>>
> >>> On 31/01/2018 22:18, Alan Myrvold wrote:
> >>>
> >>> -1 (for now, hope to change this)
> >>>
> >>> Dataflow runner jobs are failing for me with 2.3.0 RC1, for
> both
> >>> Java and Python.
> >>>
> >>> This is not an issues with the 2.3.0 RC1 SDK, we (google) need
> >>> to release worker images.
> >>>
> >>> I have assigned these bugs to myself, and will be working to
> >>> help get these workers released.
> >>>
> >>> [BEAM-3584] Java dataflow job fails with 2.3.0 RC1, due to
> >>> missing worker image
> >>> [BEAM-3585] Python dataflow job fails with 2.3.0 RC1, due to
> >>> missing worker image
> >>>
> >>> On Wed, Jan 31, 2018 at 6:12 AM, Jean-Baptiste Onofré
> >>> <jb@nanthrax.net <ma...@nanthrax.net>
> >>> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
> >>>
> >>> Thanks Kenn,
> >>>
> >>> I prepared the list of tasks I did. I will complete where
> >>> release is
> >>> out.
> >>>
> >>> Regards
> >>> JB
> >>>
> >>> On 01/31/2018 03:07 PM, Kenneth Knowles wrote:
> >>> > I've cloned the release validation spreadsheet:
> >>> >
> >>> > https://s.apache.org/beam-2.3.0-release-validation
> >>> <https://s.apache.org/beam-2.3.0-release-validation>
> >>> <https://s.apache.org/beam-2.3.0-release-validation
> >>> <https://s.apache.org/beam-2.3.0-release-validation>>
> >>> >
> >>> > If you plan to perform a manual validation task, please
> >>> sign up so multiple
> >>> > people don't waste effort.
> >>> >
> >>> > Alan & JB, as far as your pairing up to automate more,
> >>> anything manual on this
> >>> > sheet should be considered.
> >>> >
> >>> > Kenn
> >>> >
> >>> > On Wed, Jan 31, 2018 at 5:59 AM, Jean-Baptiste Onofré
> >>> <jb@nanthrax.net <ma...@nanthrax.net>
> >>> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>
> >>> > <mailto:jb@nanthrax.net <ma...@nanthrax.net>
> >>> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>>> wrote:
> >>> >
> >>> > +1 (binding)
> >>> >
> >>> > Casting my own +1 ;)
> >>> >
> >>> > Regards
> >>> > JB
> >>> >
> >>> > On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
> >>> > > Hi everyone,
> >>> > >
> >>> > > Please review and vote on the release candidate #1
> >>> for the version 2.3.0, as
> >>> > > follows:
> >>> > >
> >>> > > [ ] +1, Approve the release
> >>> > > [ ] -1, Do not approve the release (please provide
> >>> specific comments)
> >>> > >
> >>> > >
> >>> > > The complete staging area is available for your
> >>> review, which includes:
> >>> > > * JIRA release notes [1],
> >>> > > * the official Apache source release to be
> deployed
> >>> to dist.apache.org <http://dist.apache.org>
> >>> <http://dist.apache.org>
> >>> > <http://dist.apache.org> [2],
> >>> > > which is signed with the key with fingerprint
> >>> C8282E76 [3],
> >>> > > * all artifacts to be deployed to the Maven
> Central
> >>> Repository [4],
> >>> > > * source code tag "v2.3.0-RC1" [5],
> >>> > > * website pull request listing the release and
> >>> publishing the API reference
> >>> > > manual [6].
> >>> > > * Java artifacts were built with Maven 3.3.9 and
> >>> Oracle JDK 1.8.0_111.
> >>> > > * Python artifacts are deployed along with the
> >>> source release to the
> >>> > > dist.apache.org <http://dist.apache.org>
> >>> <http://dist.apache.org>
> >>> <http://dist.apache.org> [2].
> >>> > >
> >>> > > The vote will be open for at least 72 hours. It is
> >>> adopted by majority
> >>> > approval,
> >>> > > with at least 3 PMC affirmative votes.
> >>> > >
> >>> > > Thanks,
> >>> > > JB
> >>> > >
> >>> > > [1]
> >>> > >
> >>> >
> >>>
> >>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12319527&version=12341608
> >>>
> >>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12319527&version=12341608>
> >>>
> >>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12319527&version=12341608
> >>>
> >>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12319527&version=12341608>>
> >>> >
> >>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12319527&version=12341608
> >>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12319527&version=12341608>
> >>>
> >>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12319527&version=12341608
> >>>
> >>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12319527&version=12341608>>>
> >>> > > [2]
> >>> https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> >>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
> >>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> >>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>
> >>> > <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> >>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
> >>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> >>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>>
> >>> > > [3]
> >>> https://dist.apache.org/repos/dist/release/beam/KEYS
> >>> <https://dist.apache.org/repos/dist/release/beam/KEYS>
> >>> <https://dist.apache.org/repos/dist/release/beam/KEYS
> >>> <https://dist.apache.org/repos/dist/release/beam/KEYS>>
> >>> > <https://dist.apache.org/
> repos/dist/release/beam/KEYS
> >>> <https://dist.apache.org/repos/dist/release/beam/KEYS>
> >>> <https://dist.apache.org/repos/dist/release/beam/KEYS
> >>> <https://dist.apache.org/repos/dist/release/beam/KEYS>>>
> >>> > > [4]
> >>>
> >>> https://repository.apache.org/content/repositories/orgapachebeam-1026/
> >>>
> >>> <https://repository.apache.org/content/repositories/
> orgapachebeam-1026/>
> >>>
> >>> <https://repository.apache.org/content/repositories/
> orgapachebeam-1026/
> >>>
> >>> <https://repository.apache.org/content/repositories/
> orgapachebeam-1026/>>
> >>> >
> >>> <https://repository.apache.org/content/repositories/
> orgapachebeam-1026/
> >>> <https://repository.apache.org/content/repositories/
> orgapachebeam-1026/>
> >>>
> >>> <https://repository.apache.org/content/repositories/
> orgapachebeam-1026/
> >>>
> >>> <https://repository.apache.org/content/repositories/
> orgapachebeam-1026/>>>
> >>> > > [5] https://github.com/apache/
> beam/tree/v2.3.0-RC1
> >>> <https://github.com/apache/beam/tree/v2.3.0-RC1>
> >>> <https://github.com/apache/beam/tree/v2.3.0-RC1
> >>> <https://github.com/apache/beam/tree/v2.3.0-RC1>>
> >>> > <https://github.com/apache/beam/tree/v2.3.0-RC1
> >>> <https://github.com/apache/beam/tree/v2.3.0-RC1>
> >>> <https://github.com/apache/beam/tree/v2.3.0-RC1
> >>> <https://github.com/apache/beam/tree/v2.3.0-RC1>>>
> >>> > > [6] https://github.com/apache/beam-site/pull/381
> >>> <https://github.com/apache/beam-site/pull/381>
> >>> <https://github.com/apache/beam-site/pull/381
> >>> <https://github.com/apache/beam-site/pull/381>>
> >>> > <https://github.com/apache/beam-site/pull/381
> >>> <https://github.com/apache/beam-site/pull/381>
> >>> <https://github.com/apache/beam-site/pull/381
> >>> <https://github.com/apache/beam-site/pull/381>>>
> >>> > >
> >>> >
> >>> > --
> >>> > Jean-Baptiste Onofré
> >>> > jbonofre@apache.org <ma...@apache.org>
> >>> <mailto:jbonofre@apache.org <ma...@apache.org>>
> >>> <mailto:jbonofre@apache.org <ma...@apache.org>
> >>> <mailto:jbonofre@apache.org <ma...@apache.org>>>
> >>> > http://blog.nanthrax.net
> >>> > Talend - http://www.talend.com
> >>> >
> >>> >
> >>>
> >>> --
> >>> Jean-Baptiste Onofré
> >>> jbonofre@apache.org <ma...@apache.org>
> >>> <mailto:jbonofre@apache.org <ma...@apache.org>>
> >>> http://blog.nanthrax.net
> >>> Talend - http://www.talend.com
> >>>
> >>>
> >>>
> >
>
Re: [VOTE] Release 2.3.0, release candidate #1
Posted by Ismaël Mejía <ie...@gmail.com>.
What is the common procedure in cases like this ? Because it doesn't
seems that it needs a re-vote, just an extra day or two for
validation, any ideas JB ?
On Wed, Jan 31, 2018 at 10:41 PM, Alan Myrvold <am...@google.com> wrote:
> Yes, it is a dataflow step. Happy to test this again when they are
> available.
>
> On Wed, Jan 31, 2018 at 1:39 PM, Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
>>
>> OK, I think I understood ;)
>>
>> So it's not "directly" related to Beam itself (it's more a Dataflow step
>> to perform).
>>
>> @Alan, I think it's better to test first and then cast the vote. This kind
>> of tests are valuable to validate the release and make sense. But vote
>> should represent the state of the Beam release. So I think -1 vote is a bit
>> too early before the test.
>>
>> Thanks !
>> Regards
>> JB
>>
>> On 31/01/2018 22:33, Reuven Lax wrote:
>>>
>>> It's just a step that needs to be peformed before the new release works
>>> on Dataflow. Alan is saying that we've been unable to validate Dataflow so
>>> far, as worker images are not yet built. Hopefully they'll be built soon,
>>> and we'll be able to validate.
>>>
>>> On Wed, Jan 31, 2018 at 1:31 PM, Jean-Baptiste Onofré <jb@nanthrax.net
>>> <ma...@nanthrax.net>> wrote:
>>>
>>> Hi Alan
>>>
>>> does it related to change in the codebase or in a dependency/related
>>> project ?
>>>
>>> I mean: is it something we have to fix/change in Beam ?
>>>
>>> Just curious as I'm not sure what you mean by "worker images" ;)
>>>
>>> Thanks !
>>> Regards
>>> JB
>>>
>>> On 31/01/2018 22:18, Alan Myrvold wrote:
>>>
>>> -1 (for now, hope to change this)
>>>
>>> Dataflow runner jobs are failing for me with 2.3.0 RC1, for both
>>> Java and Python.
>>>
>>> This is not an issues with the 2.3.0 RC1 SDK, we (google) need
>>> to release worker images.
>>>
>>> I have assigned these bugs to myself, and will be working to
>>> help get these workers released.
>>>
>>> [BEAM-3584] Java dataflow job fails with 2.3.0 RC1, due to
>>> missing worker image
>>> [BEAM-3585] Python dataflow job fails with 2.3.0 RC1, due to
>>> missing worker image
>>>
>>> On Wed, Jan 31, 2018 at 6:12 AM, Jean-Baptiste Onofré
>>> <jb@nanthrax.net <ma...@nanthrax.net>
>>> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
>>>
>>> Thanks Kenn,
>>>
>>> I prepared the list of tasks I did. I will complete where
>>> release is
>>> out.
>>>
>>> Regards
>>> JB
>>>
>>> On 01/31/2018 03:07 PM, Kenneth Knowles wrote:
>>> > I've cloned the release validation spreadsheet:
>>> >
>>> > https://s.apache.org/beam-2.3.0-release-validation
>>> <https://s.apache.org/beam-2.3.0-release-validation>
>>> <https://s.apache.org/beam-2.3.0-release-validation
>>> <https://s.apache.org/beam-2.3.0-release-validation>>
>>> >
>>> > If you plan to perform a manual validation task, please
>>> sign up so multiple
>>> > people don't waste effort.
>>> >
>>> > Alan & JB, as far as your pairing up to automate more,
>>> anything manual on this
>>> > sheet should be considered.
>>> >
>>> > Kenn
>>> >
>>> > On Wed, Jan 31, 2018 at 5:59 AM, Jean-Baptiste Onofré
>>> <jb@nanthrax.net <ma...@nanthrax.net>
>>> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>
>>> > <mailto:jb@nanthrax.net <ma...@nanthrax.net>
>>> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>>> wrote:
>>> >
>>> > +1 (binding)
>>> >
>>> > Casting my own +1 ;)
>>> >
>>> > Regards
>>> > JB
>>> >
>>> > On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>>> > > Hi everyone,
>>> > >
>>> > > Please review and vote on the release candidate #1
>>> for the version 2.3.0, as
>>> > > follows:
>>> > >
>>> > > [ ] +1, Approve the release
>>> > > [ ] -1, Do not approve the release (please provide
>>> specific comments)
>>> > >
>>> > >
>>> > > The complete staging area is available for your
>>> review, which includes:
>>> > > * JIRA release notes [1],
>>> > > * the official Apache source release to be deployed
>>> to dist.apache.org <http://dist.apache.org>
>>> <http://dist.apache.org>
>>> > <http://dist.apache.org> [2],
>>> > > which is signed with the key with fingerprint
>>> C8282E76 [3],
>>> > > * all artifacts to be deployed to the Maven Central
>>> Repository [4],
>>> > > * source code tag "v2.3.0-RC1" [5],
>>> > > * website pull request listing the release and
>>> publishing the API reference
>>> > > manual [6].
>>> > > * Java artifacts were built with Maven 3.3.9 and
>>> Oracle JDK 1.8.0_111.
>>> > > * Python artifacts are deployed along with the
>>> source release to the
>>> > > dist.apache.org <http://dist.apache.org>
>>> <http://dist.apache.org>
>>> <http://dist.apache.org> [2].
>>> > >
>>> > > The vote will be open for at least 72 hours. It is
>>> adopted by majority
>>> > approval,
>>> > > with at least 3 PMC affirmative votes.
>>> > >
>>> > > Thanks,
>>> > > JB
>>> > >
>>> > > [1]
>>> > >
>>> >
>>>
>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>>>
>>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>
>>>
>>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>>>
>>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>>
>>> >
>>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>
>>>
>>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>>>
>>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>>>
>>> > > [2]
>>> https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
>>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>
>>> > <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
>>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>>
>>> > > [3]
>>> https://dist.apache.org/repos/dist/release/beam/KEYS
>>> <https://dist.apache.org/repos/dist/release/beam/KEYS>
>>> <https://dist.apache.org/repos/dist/release/beam/KEYS
>>> <https://dist.apache.org/repos/dist/release/beam/KEYS>>
>>> > <https://dist.apache.org/repos/dist/release/beam/KEYS
>>> <https://dist.apache.org/repos/dist/release/beam/KEYS>
>>> <https://dist.apache.org/repos/dist/release/beam/KEYS
>>> <https://dist.apache.org/repos/dist/release/beam/KEYS>>>
>>> > > [4]
>>>
>>> https://repository.apache.org/content/repositories/orgapachebeam-1026/
>>>
>>> <https://repository.apache.org/content/repositories/orgapachebeam-1026/>
>>>
>>> <https://repository.apache.org/content/repositories/orgapachebeam-1026/
>>>
>>> <https://repository.apache.org/content/repositories/orgapachebeam-1026/>>
>>> >
>>> <https://repository.apache.org/content/repositories/orgapachebeam-1026/
>>> <https://repository.apache.org/content/repositories/orgapachebeam-1026/>
>>>
>>> <https://repository.apache.org/content/repositories/orgapachebeam-1026/
>>>
>>> <https://repository.apache.org/content/repositories/orgapachebeam-1026/>>>
>>> > > [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>>> <https://github.com/apache/beam/tree/v2.3.0-RC1>
>>> <https://github.com/apache/beam/tree/v2.3.0-RC1
>>> <https://github.com/apache/beam/tree/v2.3.0-RC1>>
>>> > <https://github.com/apache/beam/tree/v2.3.0-RC1
>>> <https://github.com/apache/beam/tree/v2.3.0-RC1>
>>> <https://github.com/apache/beam/tree/v2.3.0-RC1
>>> <https://github.com/apache/beam/tree/v2.3.0-RC1>>>
>>> > > [6] https://github.com/apache/beam-site/pull/381
>>> <https://github.com/apache/beam-site/pull/381>
>>> <https://github.com/apache/beam-site/pull/381
>>> <https://github.com/apache/beam-site/pull/381>>
>>> > <https://github.com/apache/beam-site/pull/381
>>> <https://github.com/apache/beam-site/pull/381>
>>> <https://github.com/apache/beam-site/pull/381
>>> <https://github.com/apache/beam-site/pull/381>>>
>>> > >
>>> >
>>> > --
>>> > Jean-Baptiste Onofré
>>> > jbonofre@apache.org <ma...@apache.org>
>>> <mailto:jbonofre@apache.org <ma...@apache.org>>
>>> <mailto:jbonofre@apache.org <ma...@apache.org>
>>> <mailto:jbonofre@apache.org <ma...@apache.org>>>
>>> > http://blog.nanthrax.net
>>> > Talend - http://www.talend.com
>>> >
>>> >
>>>
>>> --
>>> Jean-Baptiste Onofré
>>> jbonofre@apache.org <ma...@apache.org>
>>> <mailto:jbonofre@apache.org <ma...@apache.org>>
>>> http://blog.nanthrax.net
>>> Talend - http://www.talend.com
>>>
>>>
>>>
>
Re: [VOTE] Release 2.3.0, release candidate #1
Posted by Alan Myrvold <am...@google.com>.
Yes, it is a dataflow step. Happy to test this again when they are
available.
On Wed, Jan 31, 2018 at 1:39 PM, Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> OK, I think I understood ;)
>
> So it's not "directly" related to Beam itself (it's more a Dataflow step
> to perform).
>
> @Alan, I think it's better to test first and then cast the vote. This kind
> of tests are valuable to validate the release and make sense. But vote
> should represent the state of the Beam release. So I think -1 vote is a bit
> too early before the test.
>
> Thanks !
> Regards
> JB
>
> On 31/01/2018 22:33, Reuven Lax wrote:
>
>> It's just a step that needs to be peformed before the new release works
>> on Dataflow. Alan is saying that we've been unable to validate Dataflow so
>> far, as worker images are not yet built. Hopefully they'll be built soon,
>> and we'll be able to validate.
>>
>> On Wed, Jan 31, 2018 at 1:31 PM, Jean-Baptiste Onofré <jb@nanthrax.net
>> <ma...@nanthrax.net>> wrote:
>>
>> Hi Alan
>>
>> does it related to change in the codebase or in a dependency/related
>> project ?
>>
>> I mean: is it something we have to fix/change in Beam ?
>>
>> Just curious as I'm not sure what you mean by "worker images" ;)
>>
>> Thanks !
>> Regards
>> JB
>>
>> On 31/01/2018 22:18, Alan Myrvold wrote:
>>
>> -1 (for now, hope to change this)
>>
>> Dataflow runner jobs are failing for me with 2.3.0 RC1, for both
>> Java and Python.
>>
>> This is not an issues with the 2.3.0 RC1 SDK, we (google) need
>> to release worker images.
>>
>> I have assigned these bugs to myself, and will be working to
>> help get these workers released.
>>
>> [BEAM-3584] Java dataflow job fails with 2.3.0 RC1, due to
>> missing worker image
>> [BEAM-3585] Python dataflow job fails with 2.3.0 RC1, due to
>> missing worker image
>>
>> On Wed, Jan 31, 2018 at 6:12 AM, Jean-Baptiste Onofré
>> <jb@nanthrax.net <ma...@nanthrax.net>
>> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
>>
>> Thanks Kenn,
>>
>> I prepared the list of tasks I did. I will complete where
>> release is
>> out.
>>
>> Regards
>> JB
>>
>> On 01/31/2018 03:07 PM, Kenneth Knowles wrote:
>> > I've cloned the release validation spreadsheet:
>> >
>> > https://s.apache.org/beam-2.3.0-release-validation
>> <https://s.apache.org/beam-2.3.0-release-validation>
>> <https://s.apache.org/beam-2.3.0-release-validation
>> <https://s.apache.org/beam-2.3.0-release-validation>>
>> >
>> > If you plan to perform a manual validation task, please
>> sign up so multiple
>> > people don't waste effort.
>> >
>> > Alan & JB, as far as your pairing up to automate more,
>> anything manual on this
>> > sheet should be considered.
>> >
>> > Kenn
>> >
>> > On Wed, Jan 31, 2018 at 5:59 AM, Jean-Baptiste Onofré
>> <jb@nanthrax.net <ma...@nanthrax.net>
>> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>
>> > <mailto:jb@nanthrax.net <ma...@nanthrax.net>
>> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>>> wrote:
>> >
>> > +1 (binding)
>> >
>> > Casting my own +1 ;)
>> >
>> > Regards
>> > JB
>> >
>> > On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>> > > Hi everyone,
>> > >
>> > > Please review and vote on the release candidate #1
>> for the version 2.3.0, as
>> > > follows:
>> > >
>> > > [ ] +1, Approve the release
>> > > [ ] -1, Do not approve the release (please provide
>> specific comments)
>> > >
>> > >
>> > > The complete staging area is available for your
>> review, which includes:
>> > > * JIRA release notes [1],
>> > > * the official Apache source release to be deployed
>> to dist.apache.org <http://dist.apache.org> <
>> http://dist.apache.org>
>> > <http://dist.apache.org> [2],
>> > > which is signed with the key with fingerprint
>> C8282E76 [3],
>> > > * all artifacts to be deployed to the Maven Central
>> Repository [4],
>> > > * source code tag "v2.3.0-RC1" [5],
>> > > * website pull request listing the release and
>> publishing the API reference
>> > > manual [6].
>> > > * Java artifacts were built with Maven 3.3.9 and
>> Oracle JDK 1.8.0_111.
>> > > * Python artifacts are deployed along with the
>> source release to the
>> > > dist.apache.org <http://dist.apache.org>
>> <http://dist.apache.org>
>> <http://dist.apache.org> [2].
>> > >
>> > > The vote will be open for at least 72 hours. It is
>> adopted by majority
>> > approval,
>> > > with at least 3 PMC affirmative votes.
>> > >
>> > > Thanks,
>> > > JB
>> > >
>> > > [1]
>> > >
>> >
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje
>> ctId=12319527&version=12341608
>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?proj
>> ectId=12319527&version=12341608>
>> <https://issues.apache.org/jir
>> a/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?proj
>> ectId=12319527&version=12341608>>
>> > <https://issues.apache.org/ji
>> ra/secure/ReleaseNote.jspa?projectId=12319527&version=12341608 <
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?proj
>> ectId=12319527&version=12341608>
>> <https://issues.apache.org/jir
>> a/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?proj
>> ectId=12319527&version=12341608>>>
>> > > [2]
>> https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>
>> > <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>>
>> > > [3]
>> https://dist.apache.org/repos/dist/release/beam/KEYS
>> <https://dist.apache.org/repos/dist/release/beam/KEYS>
>> <https://dist.apache.org/repos/dist/release/beam/KEYS
>> <https://dist.apache.org/repos/dist/release/beam/KEYS>>
>> > <https://dist.apache.org/repos/dist/release/beam/KEYS
>> <https://dist.apache.org/repos/dist/release/beam/KEYS>
>> <https://dist.apache.org/repos/dist/release/beam/KEYS
>> <https://dist.apache.org/repos/dist/release/beam/KEYS>>>
>> > > [4]
>> https://repository.apache.org/content/repositories/orgapache
>> beam-1026/
>> <https://repository.apache.org/content/repositories/orgapach
>> ebeam-1026/>
>> <https://repository.apache.org
>> /content/repositories/orgapachebeam-1026/
>> <https://repository.apache.org/content/repositories/orgapach
>> ebeam-1026/>>
>> > <https://repository.apache.or
>> g/content/repositories/orgapachebeam-1026/ <https://repository.apache.org
>> /content/repositories/orgapachebeam-1026/>
>> <https://repository.apache.org
>> /content/repositories/orgapachebeam-1026/
>> <https://repository.apache.org/content/repositories/orgapach
>> ebeam-1026/>>>
>> > > [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>> <https://github.com/apache/beam/tree/v2.3.0-RC1>
>> <https://github.com/apache/beam/tree/v2.3.0-RC1
>> <https://github.com/apache/beam/tree/v2.3.0-RC1>>
>> > <https://github.com/apache/beam/tree/v2.3.0-RC1
>> <https://github.com/apache/beam/tree/v2.3.0-RC1>
>> <https://github.com/apache/beam/tree/v2.3.0-RC1
>> <https://github.com/apache/beam/tree/v2.3.0-RC1>>>
>> > > [6] https://github.com/apache/beam-site/pull/381
>> <https://github.com/apache/beam-site/pull/381>
>> <https://github.com/apache/beam-site/pull/381
>> <https://github.com/apache/beam-site/pull/381>>
>> > <https://github.com/apache/beam-site/pull/381
>> <https://github.com/apache/beam-site/pull/381>
>> <https://github.com/apache/beam-site/pull/381
>> <https://github.com/apache/beam-site/pull/381>>>
>> > >
>> >
>> > --
>> > Jean-Baptiste Onofré
>> > jbonofre@apache.org <ma...@apache.org>
>> <mailto:jbonofre@apache.org <ma...@apache.org>>
>> <mailto:jbonofre@apache.org <ma...@apache.org>
>> <mailto:jbonofre@apache.org <ma...@apache.org>>>
>> > http://blog.nanthrax.net
>> > Talend - http://www.talend.com
>> >
>> >
>>
>> --
>> Jean-Baptiste Onofré
>> jbonofre@apache.org <ma...@apache.org>
>> <mailto:jbonofre@apache.org <ma...@apache.org>>
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>
>>
>>
>>
Re: [VOTE] Release 2.3.0, release candidate #1
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
OK, I think I understood ;)
So it's not "directly" related to Beam itself (it's more a Dataflow step
to perform).
@Alan, I think it's better to test first and then cast the vote. This
kind of tests are valuable to validate the release and make sense. But
vote should represent the state of the Beam release. So I think -1 vote
is a bit too early before the test.
Thanks !
Regards
JB
On 31/01/2018 22:33, Reuven Lax wrote:
> It's just a step that needs to be peformed before the new release works
> on Dataflow. Alan is saying that we've been unable to validate Dataflow
> so far, as worker images are not yet built. Hopefully they'll be built
> soon, and we'll be able to validate.
>
> On Wed, Jan 31, 2018 at 1:31 PM, Jean-Baptiste Onofré <jb@nanthrax.net
> <ma...@nanthrax.net>> wrote:
>
> Hi Alan
>
> does it related to change in the codebase or in a dependency/related
> project ?
>
> I mean: is it something we have to fix/change in Beam ?
>
> Just curious as I'm not sure what you mean by "worker images" ;)
>
> Thanks !
> Regards
> JB
>
> On 31/01/2018 22:18, Alan Myrvold wrote:
>
> -1 (for now, hope to change this)
>
> Dataflow runner jobs are failing for me with 2.3.0 RC1, for both
> Java and Python.
>
> This is not an issues with the 2.3.0 RC1 SDK, we (google) need
> to release worker images.
>
> I have assigned these bugs to myself, and will be working to
> help get these workers released.
>
> [BEAM-3584] Java dataflow job fails with 2.3.0 RC1, due to
> missing worker image
> [BEAM-3585] Python dataflow job fails with 2.3.0 RC1, due to
> missing worker image
>
> On Wed, Jan 31, 2018 at 6:12 AM, Jean-Baptiste Onofré
> <jb@nanthrax.net <ma...@nanthrax.net>
> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
>
> Thanks Kenn,
>
> I prepared the list of tasks I did. I will complete where
> release is
> out.
>
> Regards
> JB
>
> On 01/31/2018 03:07 PM, Kenneth Knowles wrote:
> > I've cloned the release validation spreadsheet:
> >
> > https://s.apache.org/beam-2.3.0-release-validation
> <https://s.apache.org/beam-2.3.0-release-validation>
> <https://s.apache.org/beam-2.3.0-release-validation
> <https://s.apache.org/beam-2.3.0-release-validation>>
> >
> > If you plan to perform a manual validation task, please
> sign up so multiple
> > people don't waste effort.
> >
> > Alan & JB, as far as your pairing up to automate more,
> anything manual on this
> > sheet should be considered.
> >
> > Kenn
> >
> > On Wed, Jan 31, 2018 at 5:59 AM, Jean-Baptiste Onofré
> <jb@nanthrax.net <ma...@nanthrax.net>
> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>
> > <mailto:jb@nanthrax.net <ma...@nanthrax.net>
> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>>> wrote:
> >
> > +1 (binding)
> >
> > Casting my own +1 ;)
> >
> > Regards
> > JB
> >
> > On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
> > > Hi everyone,
> > >
> > > Please review and vote on the release candidate #1
> for the version 2.3.0, as
> > > follows:
> > >
> > > [ ] +1, Approve the release
> > > [ ] -1, Do not approve the release (please provide
> specific comments)
> > >
> > >
> > > The complete staging area is available for your
> review, which includes:
> > > * JIRA release notes [1],
> > > * the official Apache source release to be deployed
> to dist.apache.org <http://dist.apache.org> <http://dist.apache.org>
> > <http://dist.apache.org> [2],
> > > which is signed with the key with fingerprint
> C8282E76 [3],
> > > * all artifacts to be deployed to the Maven Central
> Repository [4],
> > > * source code tag "v2.3.0-RC1" [5],
> > > * website pull request listing the release and
> publishing the API reference
> > > manual [6].
> > > * Java artifacts were built with Maven 3.3.9 and
> Oracle JDK 1.8.0_111.
> > > * Python artifacts are deployed along with the
> source release to the
> > > dist.apache.org <http://dist.apache.org>
> <http://dist.apache.org>
> <http://dist.apache.org> [2].
> > >
> > > The vote will be open for at least 72 hours. It is
> adopted by majority
> > approval,
> > > with at least 3 PMC affirmative votes.
> > >
> > > Thanks,
> > > JB
> > >
> > > [1]
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>
>
> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>>
> >
> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608 <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>
>
> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>>>
> > > [2]
> https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>
> > <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>>
> > > [3]
> https://dist.apache.org/repos/dist/release/beam/KEYS
> <https://dist.apache.org/repos/dist/release/beam/KEYS>
> <https://dist.apache.org/repos/dist/release/beam/KEYS
> <https://dist.apache.org/repos/dist/release/beam/KEYS>>
> > <https://dist.apache.org/repos/dist/release/beam/KEYS
> <https://dist.apache.org/repos/dist/release/beam/KEYS>
> <https://dist.apache.org/repos/dist/release/beam/KEYS
> <https://dist.apache.org/repos/dist/release/beam/KEYS>>>
> > > [4]
> https://repository.apache.org/content/repositories/orgapachebeam-1026/
> <https://repository.apache.org/content/repositories/orgapachebeam-1026/>
>
> <https://repository.apache.org/content/repositories/orgapachebeam-1026/
> <https://repository.apache.org/content/repositories/orgapachebeam-1026/>>
> >
> <https://repository.apache.org/content/repositories/orgapachebeam-1026/ <https://repository.apache.org/content/repositories/orgapachebeam-1026/>
>
> <https://repository.apache.org/content/repositories/orgapachebeam-1026/
> <https://repository.apache.org/content/repositories/orgapachebeam-1026/>>>
> > > [5] https://github.com/apache/beam/tree/v2.3.0-RC1
> <https://github.com/apache/beam/tree/v2.3.0-RC1>
> <https://github.com/apache/beam/tree/v2.3.0-RC1
> <https://github.com/apache/beam/tree/v2.3.0-RC1>>
> > <https://github.com/apache/beam/tree/v2.3.0-RC1
> <https://github.com/apache/beam/tree/v2.3.0-RC1>
> <https://github.com/apache/beam/tree/v2.3.0-RC1
> <https://github.com/apache/beam/tree/v2.3.0-RC1>>>
> > > [6] https://github.com/apache/beam-site/pull/381
> <https://github.com/apache/beam-site/pull/381>
> <https://github.com/apache/beam-site/pull/381
> <https://github.com/apache/beam-site/pull/381>>
> > <https://github.com/apache/beam-site/pull/381
> <https://github.com/apache/beam-site/pull/381>
> <https://github.com/apache/beam-site/pull/381
> <https://github.com/apache/beam-site/pull/381>>>
> > >
> >
> > --
> > Jean-Baptiste Onofré
> > jbonofre@apache.org <ma...@apache.org>
> <mailto:jbonofre@apache.org <ma...@apache.org>>
> <mailto:jbonofre@apache.org <ma...@apache.org>
> <mailto:jbonofre@apache.org <ma...@apache.org>>>
> > http://blog.nanthrax.net
> > Talend - http://www.talend.com
> >
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org <ma...@apache.org>
> <mailto:jbonofre@apache.org <ma...@apache.org>>
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
>
>
Re: [VOTE] Release 2.3.0, release candidate #1
Posted by Reuven Lax <re...@google.com>.
It's just a step that needs to be peformed before the new release works on
Dataflow. Alan is saying that we've been unable to validate Dataflow so
far, as worker images are not yet built. Hopefully they'll be built soon,
and we'll be able to validate.
On Wed, Jan 31, 2018 at 1:31 PM, Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> Hi Alan
>
> does it related to change in the codebase or in a dependency/related
> project ?
>
> I mean: is it something we have to fix/change in Beam ?
>
> Just curious as I'm not sure what you mean by "worker images" ;)
>
> Thanks !
> Regards
> JB
>
> On 31/01/2018 22:18, Alan Myrvold wrote:
>
>> -1 (for now, hope to change this)
>>
>> Dataflow runner jobs are failing for me with 2.3.0 RC1, for both Java and
>> Python.
>>
>> This is not an issues with the 2.3.0 RC1 SDK, we (google) need to release
>> worker images.
>>
>> I have assigned these bugs to myself, and will be working to help get
>> these workers released.
>>
>> [BEAM-3584] Java dataflow job fails with 2.3.0 RC1, due to missing worker
>> image
>> [BEAM-3585] Python dataflow job fails with 2.3.0 RC1, due to missing
>> worker image
>>
>> On Wed, Jan 31, 2018 at 6:12 AM, Jean-Baptiste Onofré <jb@nanthrax.net
>> <ma...@nanthrax.net>> wrote:
>>
>> Thanks Kenn,
>>
>> I prepared the list of tasks I did. I will complete where release is
>> out.
>>
>> Regards
>> JB
>>
>> On 01/31/2018 03:07 PM, Kenneth Knowles wrote:
>> > I've cloned the release validation spreadsheet:
>> >
>> > https://s.apache.org/beam-2.3.0-release-validation
>> <https://s.apache.org/beam-2.3.0-release-validation>
>> >
>> > If you plan to perform a manual validation task, please sign up so
>> multiple
>> > people don't waste effort.
>> >
>> > Alan & JB, as far as your pairing up to automate more, anything
>> manual on this
>> > sheet should be considered.
>> >
>> > Kenn
>> >
>> > On Wed, Jan 31, 2018 at 5:59 AM, Jean-Baptiste Onofré <
>> jb@nanthrax.net <ma...@nanthrax.net>
>> > <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
>> >
>> > +1 (binding)
>> >
>> > Casting my own +1 ;)
>> >
>> > Regards
>> > JB
>> >
>> > On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>> > > Hi everyone,
>> > >
>> > > Please review and vote on the release candidate #1 for the
>> version 2.3.0, as
>> > > follows:
>> > >
>> > > [ ] +1, Approve the release
>> > > [ ] -1, Do not approve the release (please provide specific
>> comments)
>> > >
>> > >
>> > > The complete staging area is available for your review, which
>> includes:
>> > > * JIRA release notes [1],
>> > > * the official Apache source release to be deployed to
>> dist.apache.org <http://dist.apache.org>
>> > <http://dist.apache.org> [2],
>> > > which is signed with the key with fingerprint C8282E76 [3],
>> > > * all artifacts to be deployed to the Maven Central
>> Repository [4],
>> > > * source code tag "v2.3.0-RC1" [5],
>> > > * website pull request listing the release and publishing the
>> API reference
>> > > manual [6].
>> > > * Java artifacts were built with Maven 3.3.9 and Oracle JDK
>> 1.8.0_111.
>> > > * Python artifacts are deployed along with the source release
>> to the
>> > > dist.apache.org <http://dist.apache.org>
>> <http://dist.apache.org> [2].
>> > >
>> > > The vote will be open for at least 72 hours. It is adopted by
>> majority
>> > approval,
>> > > with at least 3 PMC affirmative votes.
>> > >
>> > > Thanks,
>> > > JB
>> > >
>> > > [1]
>> > >
>> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?proj
>> ectId=12319527&version=12341608
>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?proj
>> ectId=12319527&version=12341608>
>> > <https://issues.apache.org/jira/secure/ReleaseNote.jspa?pro
>> jectId=12319527&version=12341608
>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?proj
>> ectId=12319527&version=12341608>>
>> > > [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
>> > <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>
>> > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>> <https://dist.apache.org/repos/dist/release/beam/KEYS>
>> > <https://dist.apache.org/repos/dist/release/beam/KEYS
>> <https://dist.apache.org/repos/dist/release/beam/KEYS>>
>> > > [4] https://repository.apache.org/
>> content/repositories/orgapachebeam-1026/
>> <https://repository.apache.org/content/repositories/orgapach
>> ebeam-1026/>
>> > <https://repository.apache.org/content/repositories/orgapac
>> hebeam-1026/
>> <https://repository.apache.org/content/repositories/orgapach
>> ebeam-1026/>>
>> > > [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>> <https://github.com/apache/beam/tree/v2.3.0-RC1>
>> > <https://github.com/apache/beam/tree/v2.3.0-RC1
>> <https://github.com/apache/beam/tree/v2.3.0-RC1>>
>> > > [6] https://github.com/apache/beam-site/pull/381
>> <https://github.com/apache/beam-site/pull/381>
>> > <https://github.com/apache/beam-site/pull/381
>> <https://github.com/apache/beam-site/pull/381>>
>> > >
>> >
>> > --
>> > Jean-Baptiste Onofré
>> > jbonofre@apache.org <ma...@apache.org>
>> <mailto:jbonofre@apache.org <ma...@apache.org>>
>> > http://blog.nanthrax.net
>> > Talend - http://www.talend.com
>> >
>> >
>>
>> --
>> Jean-Baptiste Onofré
>> jbonofre@apache.org <ma...@apache.org>
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>
>>
>>
Re: [VOTE] Release 2.3.0, release candidate #1
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi Alan
does it related to change in the codebase or in a dependency/related
project ?
I mean: is it something we have to fix/change in Beam ?
Just curious as I'm not sure what you mean by "worker images" ;)
Thanks !
Regards
JB
On 31/01/2018 22:18, Alan Myrvold wrote:
> -1 (for now, hope to change this)
>
> Dataflow runner jobs are failing for me with 2.3.0 RC1, for both Java
> and Python.
>
> This is not an issues with the 2.3.0 RC1 SDK, we (google) need to
> release worker images.
>
> I have assigned these bugs to myself, and will be working to help get
> these workers released.
>
> [BEAM-3584] Java dataflow job fails with 2.3.0 RC1, due to missing
> worker image
> [BEAM-3585] Python dataflow job fails with 2.3.0 RC1, due to missing
> worker image
>
> On Wed, Jan 31, 2018 at 6:12 AM, Jean-Baptiste Onofré <jb@nanthrax.net
> <ma...@nanthrax.net>> wrote:
>
> Thanks Kenn,
>
> I prepared the list of tasks I did. I will complete where release is
> out.
>
> Regards
> JB
>
> On 01/31/2018 03:07 PM, Kenneth Knowles wrote:
> > I've cloned the release validation spreadsheet:
> >
> > https://s.apache.org/beam-2.3.0-release-validation
> <https://s.apache.org/beam-2.3.0-release-validation>
> >
> > If you plan to perform a manual validation task, please sign up so multiple
> > people don't waste effort.
> >
> > Alan & JB, as far as your pairing up to automate more, anything manual on this
> > sheet should be considered.
> >
> > Kenn
> >
> > On Wed, Jan 31, 2018 at 5:59 AM, Jean-Baptiste Onofré <jb@nanthrax.net <ma...@nanthrax.net>
> > <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
> >
> > +1 (binding)
> >
> > Casting my own +1 ;)
> >
> > Regards
> > JB
> >
> > On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
> > > Hi everyone,
> > >
> > > Please review and vote on the release candidate #1 for the version 2.3.0, as
> > > follows:
> > >
> > > [ ] +1, Approve the release
> > > [ ] -1, Do not approve the release (please provide specific comments)
> > >
> > >
> > > The complete staging area is available for your review, which includes:
> > > * JIRA release notes [1],
> > > * the official Apache source release to be deployed to dist.apache.org <http://dist.apache.org>
> > <http://dist.apache.org> [2],
> > > which is signed with the key with fingerprint C8282E76 [3],
> > > * all artifacts to be deployed to the Maven Central Repository [4],
> > > * source code tag "v2.3.0-RC1" [5],
> > > * website pull request listing the release and publishing the API reference
> > > manual [6].
> > > * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
> > > * Python artifacts are deployed along with the source release to the
> > > dist.apache.org <http://dist.apache.org>
> <http://dist.apache.org> [2].
> > >
> > > The vote will be open for at least 72 hours. It is adopted by majority
> > approval,
> > > with at least 3 PMC affirmative votes.
> > >
> > > Thanks,
> > > JB
> > >
> > > [1]
> > >
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>
> > <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>>
> > > [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
> > <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>
> > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> <https://dist.apache.org/repos/dist/release/beam/KEYS>
> > <https://dist.apache.org/repos/dist/release/beam/KEYS
> <https://dist.apache.org/repos/dist/release/beam/KEYS>>
> > > [4] https://repository.apache.org/content/repositories/orgapachebeam-1026/
> <https://repository.apache.org/content/repositories/orgapachebeam-1026/>
> > <https://repository.apache.org/content/repositories/orgapachebeam-1026/
> <https://repository.apache.org/content/repositories/orgapachebeam-1026/>>
> > > [5] https://github.com/apache/beam/tree/v2.3.0-RC1
> <https://github.com/apache/beam/tree/v2.3.0-RC1>
> > <https://github.com/apache/beam/tree/v2.3.0-RC1
> <https://github.com/apache/beam/tree/v2.3.0-RC1>>
> > > [6] https://github.com/apache/beam-site/pull/381
> <https://github.com/apache/beam-site/pull/381>
> > <https://github.com/apache/beam-site/pull/381
> <https://github.com/apache/beam-site/pull/381>>
> > >
> >
> > --
> > Jean-Baptiste Onofré
> > jbonofre@apache.org <ma...@apache.org>
> <mailto:jbonofre@apache.org <ma...@apache.org>>
> > http://blog.nanthrax.net
> > Talend - http://www.talend.com
> >
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org <ma...@apache.org>
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
>
Re: [VOTE] Release 2.3.0, release candidate #1
Posted by Alan Myrvold <am...@google.com>.
-1 (for now, hope to change this)
Dataflow runner jobs are failing for me with 2.3.0 RC1, for both Java and
Python.
This is not an issues with the 2.3.0 RC1 SDK, we (google) need to release
worker images.
I have assigned these bugs to myself, and will be working to help get these
workers released.
[BEAM-3584] Java dataflow job fails with 2.3.0 RC1, due to missing worker
image
[BEAM-3585] Python dataflow job fails with 2.3.0 RC1, due to missing worker
image
On Wed, Jan 31, 2018 at 6:12 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> Thanks Kenn,
>
> I prepared the list of tasks I did. I will complete where release is out.
>
> Regards
> JB
>
> On 01/31/2018 03:07 PM, Kenneth Knowles wrote:
> > I've cloned the release validation spreadsheet:
> >
> > https://s.apache.org/beam-2.3.0-release-validation
> >
> > If you plan to perform a manual validation task, please sign up so
> multiple
> > people don't waste effort.
> >
> > Alan & JB, as far as your pairing up to automate more, anything manual
> on this
> > sheet should be considered.
> >
> > Kenn
> >
> > On Wed, Jan 31, 2018 at 5:59 AM, Jean-Baptiste Onofré <jb@nanthrax.net
> > <ma...@nanthrax.net>> wrote:
> >
> > +1 (binding)
> >
> > Casting my own +1 ;)
> >
> > Regards
> > JB
> >
> > On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
> > > Hi everyone,
> > >
> > > Please review and vote on the release candidate #1 for the version
> 2.3.0, as
> > > follows:
> > >
> > > [ ] +1, Approve the release
> > > [ ] -1, Do not approve the release (please provide specific
> comments)
> > >
> > >
> > > The complete staging area is available for your review, which
> includes:
> > > * JIRA release notes [1],
> > > * the official Apache source release to be deployed to
> dist.apache.org
> > <http://dist.apache.org> [2],
> > > which is signed with the key with fingerprint C8282E76 [3],
> > > * all artifacts to be deployed to the Maven Central Repository [4],
> > > * source code tag "v2.3.0-RC1" [5],
> > > * website pull request listing the release and publishing the API
> reference
> > > manual [6].
> > > * Java artifacts were built with Maven 3.3.9 and Oracle JDK
> 1.8.0_111.
> > > * Python artifacts are deployed along with the source release to
> the
> > > dist.apache.org <http://dist.apache.org> [2].
> > >
> > > The vote will be open for at least 72 hours. It is adopted by
> majority
> > approval,
> > > with at least 3 PMC affirmative votes.
> > >
> > > Thanks,
> > > JB
> > >
> > > [1]
> > >
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12319527&version=12341608
> > <https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12319527&version=12341608>
> > > [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> > <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
> > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> > <https://dist.apache.org/repos/dist/release/beam/KEYS>
> > > [4] https://repository.apache.org/content/repositories/
> orgapachebeam-1026/
> > <https://repository.apache.org/content/repositories/
> orgapachebeam-1026/>
> > > [5] https://github.com/apache/beam/tree/v2.3.0-RC1
> > <https://github.com/apache/beam/tree/v2.3.0-RC1>
> > > [6] https://github.com/apache/beam-site/pull/381
> > <https://github.com/apache/beam-site/pull/381>
> > >
> >
> > --
> > Jean-Baptiste Onofré
> > jbonofre@apache.org <ma...@apache.org>
> > http://blog.nanthrax.net
> > Talend - http://www.talend.com
> >
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
Re: [VOTE] Release 2.3.0, release candidate #1
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Thanks Kenn,
I prepared the list of tasks I did. I will complete where release is out.
Regards
JB
On 01/31/2018 03:07 PM, Kenneth Knowles wrote:
> I've cloned the release validation spreadsheet:
>
> https://s.apache.org/beam-2.3.0-release-validation
>
> If you plan to perform a manual validation task, please sign up so multiple
> people don't waste effort.
>
> Alan & JB, as far as your pairing up to automate more, anything manual on this
> sheet should be considered.
>
> Kenn
>
> On Wed, Jan 31, 2018 at 5:59 AM, Jean-Baptiste Onofré <jb@nanthrax.net
> <ma...@nanthrax.net>> wrote:
>
> +1 (binding)
>
> Casting my own +1 ;)
>
> Regards
> JB
>
> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
> > Hi everyone,
> >
> > Please review and vote on the release candidate #1 for the version 2.3.0, as
> > follows:
> >
> > [ ] +1, Approve the release
> > [ ] -1, Do not approve the release (please provide specific comments)
> >
> >
> > The complete staging area is available for your review, which includes:
> > * JIRA release notes [1],
> > * the official Apache source release to be deployed to dist.apache.org
> <http://dist.apache.org> [2],
> > which is signed with the key with fingerprint C8282E76 [3],
> > * all artifacts to be deployed to the Maven Central Repository [4],
> > * source code tag "v2.3.0-RC1" [5],
> > * website pull request listing the release and publishing the API reference
> > manual [6].
> > * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
> > * Python artifacts are deployed along with the source release to the
> > dist.apache.org <http://dist.apache.org> [2].
> >
> > The vote will be open for at least 72 hours. It is adopted by majority
> approval,
> > with at least 3 PMC affirmative votes.
> >
> > Thanks,
> > JB
> >
> > [1]
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>
> > [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
> > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> <https://dist.apache.org/repos/dist/release/beam/KEYS>
> > [4] https://repository.apache.org/content/repositories/orgapachebeam-1026/
> <https://repository.apache.org/content/repositories/orgapachebeam-1026/>
> > [5] https://github.com/apache/beam/tree/v2.3.0-RC1
> <https://github.com/apache/beam/tree/v2.3.0-RC1>
> > [6] https://github.com/apache/beam-site/pull/381
> <https://github.com/apache/beam-site/pull/381>
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org <ma...@apache.org>
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: [VOTE] Release 2.3.0, release candidate #1
Posted by Kenneth Knowles <kl...@google.com>.
I've cloned the release validation spreadsheet:
https://s.apache.org/beam-2.3.0-release-validation
If you plan to perform a manual validation task, please sign up so multiple
people don't waste effort.
Alan & JB, as far as your pairing up to automate more, anything manual on
this sheet should be considered.
Kenn
On Wed, Jan 31, 2018 at 5:59 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> +1 (binding)
>
> Casting my own +1 ;)
>
> Regards
> JB
>
> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
> > Hi everyone,
> >
> > Please review and vote on the release candidate #1 for the version
> 2.3.0, as
> > follows:
> >
> > [ ] +1, Approve the release
> > [ ] -1, Do not approve the release (please provide specific comments)
> >
> >
> > The complete staging area is available for your review, which includes:
> > * JIRA release notes [1],
> > * the official Apache source release to be deployed to dist.apache.org
> [2],
> > which is signed with the key with fingerprint C8282E76 [3],
> > * all artifacts to be deployed to the Maven Central Repository [4],
> > * source code tag "v2.3.0-RC1" [5],
> > * website pull request listing the release and publishing the API
> reference
> > manual [6].
> > * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
> > * Python artifacts are deployed along with the source release to the
> > dist.apache.org [2].
> >
> > The vote will be open for at least 72 hours. It is adopted by majority
> approval,
> > with at least 3 PMC affirmative votes.
> >
> > Thanks,
> > JB
> >
> > [1]
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12319527&version=12341608
> > [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> > [4] https://repository.apache.org/content/repositories/
> orgapachebeam-1026/
> > [5] https://github.com/apache/beam/tree/v2.3.0-RC1
> > [6] https://github.com/apache/beam-site/pull/381
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
Re: [VOTE] Release 2.3.0, release candidate #1
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
+1 (binding)
Casting my own +1 ;)
Regards
JB
On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
> Hi everyone,
>
> Please review and vote on the release candidate #1 for the version 2.3.0, as
> follows:
>
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
>
> The complete staging area is available for your review, which includes:
> * JIRA release notes [1],
> * the official Apache source release to be deployed to dist.apache.org [2],
> which is signed with the key with fingerprint C8282E76 [3],
> * all artifacts to be deployed to the Maven Central Repository [4],
> * source code tag "v2.3.0-RC1" [5],
> * website pull request listing the release and publishing the API reference
> manual [6].
> * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
> * Python artifacts are deployed along with the source release to the
> dist.apache.org [2].
>
> The vote will be open for at least 72 hours. It is adopted by majority approval,
> with at least 3 PMC affirmative votes.
>
> Thanks,
> JB
>
> [1]
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> [4] https://repository.apache.org/content/repositories/orgapachebeam-1026/
> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
> [6] https://github.com/apache/beam-site/pull/381
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Thanks to Kenn, I'm able to move forward on RC2.
I will start the RC2 release process in the coming hour.
Regards
JB
On 02/06/2018 11:17 PM, Kenneth Knowles wrote:
> I've cherry-picked JB's commit to https://github.com/apache/beam/pull/4621
>
> On master it seems that HBase tests are failing reliably, but actually mostly
> isolated to this PR. Hopefully the changes on the release branch will go green.
>
> On Tue, Feb 6, 2018 at 2:12 PM, Kenneth Knowles <klk@google.com
> <ma...@google.com>> wrote:
>
> Changes are cherrypicked to the release-2.3.0 branch, or applied directly there.
>
> On Tue, Feb 6, 2018 at 2:08 PM, Eugene Kirpichov <kirpichov@google.com
> <ma...@google.com>> wrote:
>
> Possibly stupid question: are new RCs created from master, or by
> cherrypicks of desired changes on top of previous RCs?
> (it affects whether or not my recently merged change on "master" will be
> in 2.3.0; I'd like it to be :) )
>
> On Tue, Feb 6, 2018 at 1:49 PM Lukasz Cwik <lcwik@google.com
> <ma...@google.com>> wrote:
>
> +1 for rollback as well.
>
> On Tue, Feb 6, 2018 at 12:37 PM, Reuven Lax <relax@google.com
> <ma...@google.com>> wrote:
>
> +1 for the rollback.
>
> On Tue, Feb 6, 2018 at 9:24 AM, Kenneth Knowles <klk@google.com
> <ma...@google.com>> wrote:
>
> +1 for option 1. It is almost certainly repeated proto
> parsing. That is fixed for ParDo via generalized caching but
> not for other transforms. Flink also has a proto round trip,
> but the design might avoid the problem anyhow. We should
> investigate before releasing, or just preemptively roll it
> back too. The proto round trips are helpful for maturing
> portability and for preventing hacks from returning but
> otherwise don't matter so much yet.
>
> Kenn
>
> On Tue, Feb 6, 2018 at 8:54 AM, Jean-Baptiste Onofré
> <jb@nanthrax.net <ma...@nanthrax.net>> wrote:
>
> Hi all,
>
> As you might have seen, I found the cause of performance
> degradation on the
> direct runner (BEAM-3617).
>
> We have basically three options for RC2:
>
> 1. We revert the change for 2.3.0. I think it's the
> fastest and more secure way.
> I created a PR for that
> (https://github.com/apache/beam/pull/4609
> <https://github.com/apache/beam/pull/4609>). It gives us
> time to investigate and provide a better fix in Runner
> API for 2.4.0.
>
> 2. If the fix is easy and with limited impact in Runner
> API, we do that.
>
> 3. We leave as it is, adding a note that direct runner
> has worse performance in
> 2.3.0 compared to 2.2.0.
>
> In any case, I would like to cut RC2 later tonight or
> tomorrow morning (my time).
>
> Thoughts ?
>
> Regards
> JB
>
> On 02/06/2018 09:42 AM, Jean-Baptiste Onofré wrote:
> > Hi Reuven,
> >
> > it's what I'm suspecting. git bisect should give us
> more information (still in
> > progress, 3 more steps to complete ;)). I keep you posted.
> >
> > Regards
> > JB
> >
> > On 02/06/2018 08:36 AM, Reuven Lax wrote:
> >> Could this be related to any of the portability changes?
> >>
> >> On Mon, Feb 5, 2018 at 7:51 AM, Jean-Baptiste Onofré
> <jb@nanthrax.net <ma...@nanthrax.net>
> >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
> >>
> >> Created:
> >>
> >> https://issues.apache.org/jira/browse/BEAM-3617
> <https://issues.apache.org/jira/browse/BEAM-3617>
> >> <https://issues.apache.org/jira/browse/BEAM-3617
> <https://issues.apache.org/jira/browse/BEAM-3617>>
> >>
> >> Regards
> >> JB
> >>
> >> On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
> >> > What is the Jira for direct runner perf?
> >> >
> >> > On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste
> Onofré <jb@nanthrax.net <ma...@nanthrax.net>
> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>
> >> > <mailto:jb@nanthrax.net
> <ma...@nanthrax.net> <mailto:jb@nanthrax.net
> <ma...@nanthrax.net>>>> wrote:
> >> >
> >> > Thanks !
> >> >
> >> > I cherry-pick on release-2.3.0 branch.
> >> >
> >> > I'm on the direct runner perf test in the
> mean time.
> >> >
> >> > Thanks again !
> >> >
> >> > Regards
> >> > JB
> >> >
> >> > On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
> >> > > I merged fixes for:
> >> > >
> - https://issues.apache.org/jira/browse/BEAM-3186
> <https://issues.apache.org/jira/browse/BEAM-3186>
> >> <https://issues.apache.org/jira/browse/BEAM-3186
> <https://issues.apache.org/jira/browse/BEAM-3186>>
> >> >
> <https://issues.apache.org/jira/browse/BEAM-3186
> <https://issues.apache.org/jira/browse/BEAM-3186>
> >> <https://issues.apache.org/jira/browse/BEAM-3186
> <https://issues.apache.org/jira/browse/BEAM-3186>>>
> >> > >
> - https://issues.apache.org/jira/browse/BEAM-3589
> <https://issues.apache.org/jira/browse/BEAM-3589>
> >> <https://issues.apache.org/jira/browse/BEAM-3589
> <https://issues.apache.org/jira/browse/BEAM-3589>>
> >> >
> <https://issues.apache.org/jira/browse/BEAM-3589
> <https://issues.apache.org/jira/browse/BEAM-3589>
> >> <https://issues.apache.org/jira/browse/BEAM-3589
> <https://issues.apache.org/jira/browse/BEAM-3589>>>
> >> > >
> >> > > @JB I didn't yet merge them on the 2.3.0
> branch, though, but I can or you can go
> >> > > ahead.
> >> > >
> >> > >> On 5. Feb 2018, at 06:30, Jean-Baptiste
> Onofré <jb@nanthrax.net <ma...@nanthrax.net>
> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>
> <mailto:jb@nanthrax.net <ma...@nanthrax.net>
> >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>>
> >> > >> <mailto:jb@nanthrax.net
> <ma...@nanthrax.net> <mailto:jb@nanthrax.net
> <ma...@nanthrax.net>>
> >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>
> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>>>> wrote:
> >> > >>
> >> > >> Hi guys,
> >> > >>
> >> > >> Quick update on the RC2 preparation:
> >> > >>
> >> > >> * BEAM-3587 (TextIO with Flink) seems
> related to a custom build with
> >> > Gradle (not
> >> > >> using artifacts created by Maven).
> Anyway, I will take a look today.
> >> > >> * BEAM-3186 has a PR. Aljoscha will do
> the review pretty soon.
> >> > >> * I'm also taking a look on the direct
> runner performance change.
> >> > >>
> >> > >> The goal is to cut RC2 tomorrow or the
> day after.
> >> > >>
> >> > >> Regards
> >> > >> JB
> >> > >>
> >> > >> On 02/01/2018 04:16 PM, Jean-Baptiste
> Onofré wrote:
> >> > >>> Hi guys,
> >> > >>>
> >> > >>> Especially due to BEAM-3587 & BEAM-3186
> regressions, I cancel RC1.
> >> > >>>
> >> > >>> We will cherry-pick fixes on
> release-2.3.0 branch.
> >> > >>>
> >> > >>> I'm updating Jira right now. When the
> fixes will be cherry-picked, I
> >> > will submit
> >> > >>> a RC2 to vote.
> >> > >>>
> >> > >>> Thanks !
> >> > >>> Regards
> >> > >>> JB
> >> > >>>
> >> > >>> On 01/30/2018 09:04 AM, Jean-Baptiste
> Onofré wrote:
> >> > >>>> Hi everyone,
> >> > >>>>
> >> > >>>> Please review and vote on the release
> candidate #1 for the version
> >> > 2.3.0, as
> >> > >>>> follows:
> >> > >>>>
> >> > >>>> [ ] +1, Approve the release
> >> > >>>> [ ] -1, Do not approve the release
> (please provide specific comments)
> >> > >>>>
> >> > >>>>
> >> > >>>> The complete staging area is available
> for your review, which
> >> includes:
> >> > >>>> * JIRA release notes [1],
> >> > >>>> * the official Apache source release
> to be deployed to
> >> dist.apache.org <http://dist.apache.org>
> <http://dist.apache.org>
> >> > <http://dist.apache.org>
> >> > >>>> <http://dist.apache.org> [2],
> >> > >>>> which is signed with the key with
> fingerprint C8282E76 [3],
> >> > >>>> * all artifacts to be deployed to the
> Maven Central Repository [4],
> >> > >>>> * source code tag "v2.3.0-RC1" [5],
> >> > >>>> * website pull request listing the
> release and publishing the API
> >> reference
> >> > >>>> manual [6].
> >> > >>>> * Java artifacts were built with Maven
> 3.3.9 and Oracle JDK
> >> 1.8.0_111.
> >> > >>>> * Python artifacts are deployed along
> with the source release to the
> >> > >>>> dist.apache.org
> <http://dist.apache.org> <http://dist.apache.org>
> <http://dist.apache.org>
> >> <http://dist.apache.org> [2].
> >> > >>>>
> >> > >>>> The vote will be open for at least 72
> hours. It is adopted by majority approval,
> >> > >>>> with at least 3 PMC affirmative votes.
> >> > >>>>
> >> > >>>> Thanks,
> >> > >>>> JB
> >> > >>>>
> >> > >>>> [1]
> >> > >>>>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>
> >>
> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>>
> >> >
> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>
> >>
> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>>>
> >> > >>>> [2]
> https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
> >>
> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>
> >> >
> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
> >>
> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>>
> >> > >>>> [3]
> https://dist.apache.org/repos/dist/release/beam/KEYS
> <https://dist.apache.org/repos/dist/release/beam/KEYS>
> >>
> <https://dist.apache.org/repos/dist/release/beam/KEYS
> <https://dist.apache.org/repos/dist/release/beam/KEYS>>
> >> >
> <https://dist.apache.org/repos/dist/release/beam/KEYS
> <https://dist.apache.org/repos/dist/release/beam/KEYS>
> >>
> <https://dist.apache.org/repos/dist/release/beam/KEYS
> <https://dist.apache.org/repos/dist/release/beam/KEYS>>>
> >> > >>>> [4]
> https://repository.apache.org/content/repositories/orgapachebeam-1026/
> <https://repository.apache.org/content/repositories/orgapachebeam-1026/>
> >>
> <https://repository.apache.org/content/repositories/orgapachebeam-1026/
> <https://repository.apache.org/content/repositories/orgapachebeam-1026/>>
> >> >
> <https://repository.apache.org/content/repositories/orgapachebeam-1026/
> <https://repository.apache.org/content/repositories/orgapachebeam-1026/>
> >>
> <https://repository.apache.org/content/repositories/orgapachebeam-1026/
> <https://repository.apache.org/content/repositories/orgapachebeam-1026/>>>
> >> > >>>> [5]
> https://github.com/apache/beam/tree/v2.3.0-RC1
> <https://github.com/apache/beam/tree/v2.3.0-RC1>
> >> <https://github.com/apache/beam/tree/v2.3.0-RC1
> <https://github.com/apache/beam/tree/v2.3.0-RC1>>
> >> >
> <https://github.com/apache/beam/tree/v2.3.0-RC1
> <https://github.com/apache/beam/tree/v2.3.0-RC1>
> >> <https://github.com/apache/beam/tree/v2.3.0-RC1
> <https://github.com/apache/beam/tree/v2.3.0-RC1>>>
> >> > >>>> [6]
> https://github.com/apache/beam-site/pull/381
> <https://github.com/apache/beam-site/pull/381>
> >> <https://github.com/apache/beam-site/pull/381
> <https://github.com/apache/beam-site/pull/381>>
> >> >
> <https://github.com/apache/beam-site/pull/381
> <https://github.com/apache/beam-site/pull/381>
> >> <https://github.com/apache/beam-site/pull/381
> <https://github.com/apache/beam-site/pull/381>>>
> >> > >>>>
> >> > >>>
> >> > >>
> >> > >> --
> >> > >> Jean-Baptiste Onofré
> >> > >> jbonofre@apache.org
> <ma...@apache.org> <mailto:jbonofre@apache.org
> <ma...@apache.org>>
> <mailto:jbonofre@apache.org <ma...@apache.org>
> >> <mailto:jbonofre@apache.org
> <ma...@apache.org>>>
> >> > <mailto:jbonofre@apache.org
> <ma...@apache.org> <mailto:jbonofre@apache.org
> <ma...@apache.org>>
> >> <mailto:jbonofre@apache.org
> <ma...@apache.org> <mailto:jbonofre@apache.org
> <ma...@apache.org>>>>
> >> > >> http://blog.nanthrax.net
> >> > >> Talend - http://www.talend.com
> >> > >
> >> >
> >> > --
> >> > Jean-Baptiste Onofré
> >> > jbonofre@apache.org
> <ma...@apache.org> <mailto:jbonofre@apache.org
> <ma...@apache.org>>
> >> <mailto:jbonofre@apache.org
> <ma...@apache.org> <mailto:jbonofre@apache.org
> <ma...@apache.org>>>
> >> > http://blog.nanthrax.net
> >> > Talend - http://www.talend.com
> >> >
> >> >
> >>
> >> --
> >> Jean-Baptiste Onofré
> >> jbonofre@apache.org <ma...@apache.org>
> <mailto:jbonofre@apache.org <ma...@apache.org>>
> >> http://blog.nanthrax.net
> >> Talend - http://www.talend.com
> >>
> >>
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org <ma...@apache.org>
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
>
>
>
>
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Kenneth Knowles <kl...@google.com>.
I've cherry-picked JB's commit to https://github.com/apache/beam/pull/4621
On master it seems that HBase tests are failing reliably, but actually
mostly isolated to this PR. Hopefully the changes on the release branch
will go green.
On Tue, Feb 6, 2018 at 2:12 PM, Kenneth Knowles <kl...@google.com> wrote:
> Changes are cherrypicked to the release-2.3.0 branch, or applied directly
> there.
>
> On Tue, Feb 6, 2018 at 2:08 PM, Eugene Kirpichov <ki...@google.com>
> wrote:
>
>> Possibly stupid question: are new RCs created from master, or by
>> cherrypicks of desired changes on top of previous RCs?
>> (it affects whether or not my recently merged change on "master" will be
>> in 2.3.0; I'd like it to be :) )
>>
>> On Tue, Feb 6, 2018 at 1:49 PM Lukasz Cwik <lc...@google.com> wrote:
>>
>>> +1 for rollback as well.
>>>
>>> On Tue, Feb 6, 2018 at 12:37 PM, Reuven Lax <re...@google.com> wrote:
>>>
>>>> +1 for the rollback.
>>>>
>>>> On Tue, Feb 6, 2018 at 9:24 AM, Kenneth Knowles <kl...@google.com> wrote:
>>>>
>>>>> +1 for option 1. It is almost certainly repeated proto parsing. That
>>>>> is fixed for ParDo via generalized caching but not for other transforms.
>>>>> Flink also has a proto round trip, but the design might avoid the problem
>>>>> anyhow. We should investigate before releasing, or just preemptively roll
>>>>> it back too. The proto round trips are helpful for maturing portability and
>>>>> for preventing hacks from returning but otherwise don't matter so much yet.
>>>>>
>>>>> Kenn
>>>>>
>>>>> On Tue, Feb 6, 2018 at 8:54 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
>>>>> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> As you might have seen, I found the cause of performance degradation
>>>>>> on the
>>>>>> direct runner (BEAM-3617).
>>>>>>
>>>>>> We have basically three options for RC2:
>>>>>>
>>>>>> 1. We revert the change for 2.3.0. I think it's the fastest and more
>>>>>> secure way.
>>>>>> I created a PR for that (https://github.com/apache/beam/pull/4609).
>>>>>> It gives us
>>>>>> time to investigate and provide a better fix in Runner API for 2.4.0.
>>>>>>
>>>>>> 2. If the fix is easy and with limited impact in Runner API, we do
>>>>>> that.
>>>>>>
>>>>>> 3. We leave as it is, adding a note that direct runner has worse
>>>>>> performance in
>>>>>> 2.3.0 compared to 2.2.0.
>>>>>>
>>>>>> In any case, I would like to cut RC2 later tonight or tomorrow
>>>>>> morning (my time).
>>>>>>
>>>>>> Thoughts ?
>>>>>>
>>>>>> Regards
>>>>>> JB
>>>>>>
>>>>>> On 02/06/2018 09:42 AM, Jean-Baptiste Onofré wrote:
>>>>>> > Hi Reuven,
>>>>>> >
>>>>>> > it's what I'm suspecting. git bisect should give us more
>>>>>> information (still in
>>>>>> > progress, 3 more steps to complete ;)). I keep you posted.
>>>>>> >
>>>>>> > Regards
>>>>>> > JB
>>>>>> >
>>>>>> > On 02/06/2018 08:36 AM, Reuven Lax wrote:
>>>>>> >> Could this be related to any of the portability changes?
>>>>>> >>
>>>>>> >> On Mon, Feb 5, 2018 at 7:51 AM, Jean-Baptiste Onofré <
>>>>>> jb@nanthrax.net
>>>>>> >> <ma...@nanthrax.net>> wrote:
>>>>>> >>
>>>>>> >> Created:
>>>>>> >>
>>>>>> >> https://issues.apache.org/jira/browse/BEAM-3617
>>>>>> >> <https://issues.apache.org/jira/browse/BEAM-3617>
>>>>>> >>
>>>>>> >> Regards
>>>>>> >> JB
>>>>>> >>
>>>>>> >> On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
>>>>>> >> > What is the Jira for direct runner perf?
>>>>>> >> >
>>>>>> >> > On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré <
>>>>>> jb@nanthrax.net <ma...@nanthrax.net>
>>>>>> >> > <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
>>>>>> >> >
>>>>>> >> > Thanks !
>>>>>> >> >
>>>>>> >> > I cherry-pick on release-2.3.0 branch.
>>>>>> >> >
>>>>>> >> > I'm on the direct runner perf test in the mean time.
>>>>>> >> >
>>>>>> >> > Thanks again !
>>>>>> >> >
>>>>>> >> > Regards
>>>>>> >> > JB
>>>>>> >> >
>>>>>> >> > On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
>>>>>> >> > > I merged fixes for:
>>>>>> >> > > - https://issues.apache.org/jira/browse/BEAM-3186
>>>>>> >> <https://issues.apache.org/jira/browse/BEAM-3186>
>>>>>> >> > <https://issues.apache.org/jira/browse/BEAM-3186
>>>>>> >> <https://issues.apache.org/jira/browse/BEAM-3186>>
>>>>>> >> > > - https://issues.apache.org/jira/browse/BEAM-3589
>>>>>> >> <https://issues.apache.org/jira/browse/BEAM-3589>
>>>>>> >> > <https://issues.apache.org/jira/browse/BEAM-3589
>>>>>> >> <https://issues.apache.org/jira/browse/BEAM-3589>>
>>>>>> >> > >
>>>>>> >> > > @JB I didn't yet merge them on the 2.3.0 branch,
>>>>>> though, but I can or you can go
>>>>>> >> > > ahead.
>>>>>> >> > >
>>>>>> >> > >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <
>>>>>> jb@nanthrax.net <ma...@nanthrax.net> <mailto:jb@nanthrax.net
>>>>>> >> <ma...@nanthrax.net>>
>>>>>> >> > >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>
>>>>>> >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>>> wrote:
>>>>>> >> > >>
>>>>>> >> > >> Hi guys,
>>>>>> >> > >>
>>>>>> >> > >> Quick update on the RC2 preparation:
>>>>>> >> > >>
>>>>>> >> > >> * BEAM-3587 (TextIO with Flink) seems related to a
>>>>>> custom build with
>>>>>> >> > Gradle (not
>>>>>> >> > >> using artifacts created by Maven). Anyway, I will
>>>>>> take a look today.
>>>>>> >> > >> * BEAM-3186 has a PR. Aljoscha will do the review
>>>>>> pretty soon.
>>>>>> >> > >> * I'm also taking a look on the direct runner
>>>>>> performance change.
>>>>>> >> > >>
>>>>>> >> > >> The goal is to cut RC2 tomorrow or the day after.
>>>>>> >> > >>
>>>>>> >> > >> Regards
>>>>>> >> > >> JB
>>>>>> >> > >>
>>>>>> >> > >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
>>>>>> >> > >>> Hi guys,
>>>>>> >> > >>>
>>>>>> >> > >>> Especially due to BEAM-3587 & BEAM-3186 regressions,
>>>>>> I cancel RC1.
>>>>>> >> > >>>
>>>>>> >> > >>> We will cherry-pick fixes on release-2.3.0 branch.
>>>>>> >> > >>>
>>>>>> >> > >>> I'm updating Jira right now. When the fixes will be
>>>>>> cherry-picked, I
>>>>>> >> > will submit
>>>>>> >> > >>> a RC2 to vote.
>>>>>> >> > >>>
>>>>>> >> > >>> Thanks !
>>>>>> >> > >>> Regards
>>>>>> >> > >>> JB
>>>>>> >> > >>>
>>>>>> >> > >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>>>>>> >> > >>>> Hi everyone,
>>>>>> >> > >>>>
>>>>>> >> > >>>> Please review and vote on the release candidate #1
>>>>>> for the version
>>>>>> >> > 2.3.0, as
>>>>>> >> > >>>> follows:
>>>>>> >> > >>>>
>>>>>> >> > >>>> [ ] +1, Approve the release
>>>>>> >> > >>>> [ ] -1, Do not approve the release (please provide
>>>>>> specific comments)
>>>>>> >> > >>>>
>>>>>> >> > >>>>
>>>>>> >> > >>>> The complete staging area is available for your
>>>>>> review, which
>>>>>> >> includes:
>>>>>> >> > >>>> * JIRA release notes [1],
>>>>>> >> > >>>> * the official Apache source release to be deployed
>>>>>> to
>>>>>> >> dist.apache.org <http://dist.apache.org>
>>>>>> >> > <http://dist.apache.org>
>>>>>> >> > >>>> <http://dist.apache.org> [2],
>>>>>> >> > >>>> which is signed with the key with fingerprint
>>>>>> C8282E76 [3],
>>>>>> >> > >>>> * all artifacts to be deployed to the Maven Central
>>>>>> Repository [4],
>>>>>> >> > >>>> * source code tag "v2.3.0-RC1" [5],
>>>>>> >> > >>>> * website pull request listing the release and
>>>>>> publishing the API
>>>>>> >> reference
>>>>>> >> > >>>> manual [6].
>>>>>> >> > >>>> * Java artifacts were built with Maven 3.3.9 and
>>>>>> Oracle JDK
>>>>>> >> 1.8.0_111.
>>>>>> >> > >>>> * Python artifacts are deployed along with the
>>>>>> source release to the
>>>>>> >> > >>>> dist.apache.org <http://dist.apache.org> <
>>>>>> http://dist.apache.org>
>>>>>> >> <http://dist.apache.org> [2].
>>>>>> >> > >>>>
>>>>>> >> > >>>> The vote will be open for at least 72 hours. It is
>>>>>> adopted by majority approval,
>>>>>> >> > >>>> with at least 3 PMC affirmative votes.
>>>>>> >> > >>>>
>>>>>> >> > >>>> Thanks,
>>>>>> >> > >>>> JB
>>>>>> >> > >>>>
>>>>>> >> > >>>> [1]
>>>>>> >> > >>>> https://issues.apache.org/jira
>>>>>> /secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>>>>>> >> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?pro
>>>>>> jectId=12319527&version=12341608>
>>>>>> >> > <https://issues.apache.org/ji
>>>>>> ra/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>>>>>> >> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?pro
>>>>>> jectId=12319527&version=12341608>>
>>>>>> >> > >>>> [2] https://dist.apache.org/repos/
>>>>>> dist/dev/beam/2.3.0/
>>>>>> >> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
>>>>>> >> > <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>>>>>> >> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>
>>>>>> >> > >>>> [3] https://dist.apache.org/repos/
>>>>>> dist/release/beam/KEYS
>>>>>> >> <https://dist.apache.org/repos/dist/release/beam/KEYS>
>>>>>> >> > <https://dist.apache.org/repos/dist/release/beam/KEYS
>>>>>> >> <https://dist.apache.org/repos/dist/release/beam/KEYS>>
>>>>>> >> > >>>> [4] https://repository.apache.org/
>>>>>> content/repositories/orgapachebeam-1026/
>>>>>> >> <https://repository.apache.org/content/repositories/orgapac
>>>>>> hebeam-1026/>
>>>>>> >> > <https://repository.apache.or
>>>>>> g/content/repositories/orgapachebeam-1026/
>>>>>> >> <https://repository.apache.org/content/repositories/orgapac
>>>>>> hebeam-1026/>>
>>>>>> >> > >>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>>>>>> >> <https://github.com/apache/beam/tree/v2.3.0-RC1>
>>>>>> >> > <https://github.com/apache/beam/tree/v2.3.0-RC1
>>>>>> >> <https://github.com/apache/beam/tree/v2.3.0-RC1>>
>>>>>> >> > >>>> [6] https://github.com/apache/beam-site/pull/381
>>>>>> >> <https://github.com/apache/beam-site/pull/381>
>>>>>> >> > <https://github.com/apache/beam-site/pull/381
>>>>>> >> <https://github.com/apache/beam-site/pull/381>>
>>>>>> >> > >>>>
>>>>>> >> > >>>
>>>>>> >> > >>
>>>>>> >> > >> --
>>>>>> >> > >> Jean-Baptiste Onofré
>>>>>> >> > >> jbonofre@apache.org <ma...@apache.org>
>>>>>> <mailto:jbonofre@apache.org
>>>>>> >> <ma...@apache.org>>
>>>>>> >> > <mailto:jbonofre@apache.org <ma...@apache.org>
>>>>>> >> <mailto:jbonofre@apache.org <ma...@apache.org>>>
>>>>>> >> > >> http://blog.nanthrax.net
>>>>>> >> > >> Talend - http://www.talend.com
>>>>>> >> > >
>>>>>> >> >
>>>>>> >> > --
>>>>>> >> > Jean-Baptiste Onofré
>>>>>> >> > jbonofre@apache.org <ma...@apache.org>
>>>>>> >> <mailto:jbonofre@apache.org <ma...@apache.org>>
>>>>>> >> > http://blog.nanthrax.net
>>>>>> >> > Talend - http://www.talend.com
>>>>>> >> >
>>>>>> >> >
>>>>>> >>
>>>>>> >> --
>>>>>> >> Jean-Baptiste Onofré
>>>>>> >> jbonofre@apache.org <ma...@apache.org>
>>>>>> >> http://blog.nanthrax.net
>>>>>> >> Talend - http://www.talend.com
>>>>>> >>
>>>>>> >>
>>>>>> >
>>>>>>
>>>>>> --
>>>>>> Jean-Baptiste Onofré
>>>>>> jbonofre@apache.org
>>>>>> http://blog.nanthrax.net
>>>>>> Talend - http://www.talend.com
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Kenneth Knowles <kl...@google.com>.
Changes are cherrypicked to the release-2.3.0 branch, or applied directly
there.
On Tue, Feb 6, 2018 at 2:08 PM, Eugene Kirpichov <ki...@google.com>
wrote:
> Possibly stupid question: are new RCs created from master, or by
> cherrypicks of desired changes on top of previous RCs?
> (it affects whether or not my recently merged change on "master" will be
> in 2.3.0; I'd like it to be :) )
>
> On Tue, Feb 6, 2018 at 1:49 PM Lukasz Cwik <lc...@google.com> wrote:
>
>> +1 for rollback as well.
>>
>> On Tue, Feb 6, 2018 at 12:37 PM, Reuven Lax <re...@google.com> wrote:
>>
>>> +1 for the rollback.
>>>
>>> On Tue, Feb 6, 2018 at 9:24 AM, Kenneth Knowles <kl...@google.com> wrote:
>>>
>>>> +1 for option 1. It is almost certainly repeated proto parsing. That is
>>>> fixed for ParDo via generalized caching but not for other transforms. Flink
>>>> also has a proto round trip, but the design might avoid the problem anyhow.
>>>> We should investigate before releasing, or just preemptively roll it back
>>>> too. The proto round trips are helpful for maturing portability and for
>>>> preventing hacks from returning but otherwise don't matter so much yet.
>>>>
>>>> Kenn
>>>>
>>>> On Tue, Feb 6, 2018 at 8:54 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
>>>> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> As you might have seen, I found the cause of performance degradation
>>>>> on the
>>>>> direct runner (BEAM-3617).
>>>>>
>>>>> We have basically three options for RC2:
>>>>>
>>>>> 1. We revert the change for 2.3.0. I think it's the fastest and more
>>>>> secure way.
>>>>> I created a PR for that (https://github.com/apache/beam/pull/4609).
>>>>> It gives us
>>>>> time to investigate and provide a better fix in Runner API for 2.4.0.
>>>>>
>>>>> 2. If the fix is easy and with limited impact in Runner API, we do
>>>>> that.
>>>>>
>>>>> 3. We leave as it is, adding a note that direct runner has worse
>>>>> performance in
>>>>> 2.3.0 compared to 2.2.0.
>>>>>
>>>>> In any case, I would like to cut RC2 later tonight or tomorrow morning
>>>>> (my time).
>>>>>
>>>>> Thoughts ?
>>>>>
>>>>> Regards
>>>>> JB
>>>>>
>>>>> On 02/06/2018 09:42 AM, Jean-Baptiste Onofré wrote:
>>>>> > Hi Reuven,
>>>>> >
>>>>> > it's what I'm suspecting. git bisect should give us more information
>>>>> (still in
>>>>> > progress, 3 more steps to complete ;)). I keep you posted.
>>>>> >
>>>>> > Regards
>>>>> > JB
>>>>> >
>>>>> > On 02/06/2018 08:36 AM, Reuven Lax wrote:
>>>>> >> Could this be related to any of the portability changes?
>>>>> >>
>>>>> >> On Mon, Feb 5, 2018 at 7:51 AM, Jean-Baptiste Onofré <
>>>>> jb@nanthrax.net
>>>>> >> <ma...@nanthrax.net>> wrote:
>>>>> >>
>>>>> >> Created:
>>>>> >>
>>>>> >> https://issues.apache.org/jira/browse/BEAM-3617
>>>>> >> <https://issues.apache.org/jira/browse/BEAM-3617>
>>>>> >>
>>>>> >> Regards
>>>>> >> JB
>>>>> >>
>>>>> >> On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
>>>>> >> > What is the Jira for direct runner perf?
>>>>> >> >
>>>>> >> > On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré <
>>>>> jb@nanthrax.net <ma...@nanthrax.net>
>>>>> >> > <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
>>>>> >> >
>>>>> >> > Thanks !
>>>>> >> >
>>>>> >> > I cherry-pick on release-2.3.0 branch.
>>>>> >> >
>>>>> >> > I'm on the direct runner perf test in the mean time.
>>>>> >> >
>>>>> >> > Thanks again !
>>>>> >> >
>>>>> >> > Regards
>>>>> >> > JB
>>>>> >> >
>>>>> >> > On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
>>>>> >> > > I merged fixes for:
>>>>> >> > > - https://issues.apache.org/jira/browse/BEAM-3186
>>>>> >> <https://issues.apache.org/jira/browse/BEAM-3186>
>>>>> >> > <https://issues.apache.org/jira/browse/BEAM-3186
>>>>> >> <https://issues.apache.org/jira/browse/BEAM-3186>>
>>>>> >> > > - https://issues.apache.org/jira/browse/BEAM-3589
>>>>> >> <https://issues.apache.org/jira/browse/BEAM-3589>
>>>>> >> > <https://issues.apache.org/jira/browse/BEAM-3589
>>>>> >> <https://issues.apache.org/jira/browse/BEAM-3589>>
>>>>> >> > >
>>>>> >> > > @JB I didn't yet merge them on the 2.3.0 branch,
>>>>> though, but I can or you can go
>>>>> >> > > ahead.
>>>>> >> > >
>>>>> >> > >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <
>>>>> jb@nanthrax.net <ma...@nanthrax.net> <mailto:jb@nanthrax.net
>>>>> >> <ma...@nanthrax.net>>
>>>>> >> > >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>
>>>>> >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>>> wrote:
>>>>> >> > >>
>>>>> >> > >> Hi guys,
>>>>> >> > >>
>>>>> >> > >> Quick update on the RC2 preparation:
>>>>> >> > >>
>>>>> >> > >> * BEAM-3587 (TextIO with Flink) seems related to a
>>>>> custom build with
>>>>> >> > Gradle (not
>>>>> >> > >> using artifacts created by Maven). Anyway, I will take
>>>>> a look today.
>>>>> >> > >> * BEAM-3186 has a PR. Aljoscha will do the review
>>>>> pretty soon.
>>>>> >> > >> * I'm also taking a look on the direct runner
>>>>> performance change.
>>>>> >> > >>
>>>>> >> > >> The goal is to cut RC2 tomorrow or the day after.
>>>>> >> > >>
>>>>> >> > >> Regards
>>>>> >> > >> JB
>>>>> >> > >>
>>>>> >> > >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
>>>>> >> > >>> Hi guys,
>>>>> >> > >>>
>>>>> >> > >>> Especially due to BEAM-3587 & BEAM-3186 regressions,
>>>>> I cancel RC1.
>>>>> >> > >>>
>>>>> >> > >>> We will cherry-pick fixes on release-2.3.0 branch.
>>>>> >> > >>>
>>>>> >> > >>> I'm updating Jira right now. When the fixes will be
>>>>> cherry-picked, I
>>>>> >> > will submit
>>>>> >> > >>> a RC2 to vote.
>>>>> >> > >>>
>>>>> >> > >>> Thanks !
>>>>> >> > >>> Regards
>>>>> >> > >>> JB
>>>>> >> > >>>
>>>>> >> > >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>>>>> >> > >>>> Hi everyone,
>>>>> >> > >>>>
>>>>> >> > >>>> Please review and vote on the release candidate #1
>>>>> for the version
>>>>> >> > 2.3.0, as
>>>>> >> > >>>> follows:
>>>>> >> > >>>>
>>>>> >> > >>>> [ ] +1, Approve the release
>>>>> >> > >>>> [ ] -1, Do not approve the release (please provide
>>>>> specific comments)
>>>>> >> > >>>>
>>>>> >> > >>>>
>>>>> >> > >>>> The complete staging area is available for your
>>>>> review, which
>>>>> >> includes:
>>>>> >> > >>>> * JIRA release notes [1],
>>>>> >> > >>>> * the official Apache source release to be deployed
>>>>> to
>>>>> >> dist.apache.org <http://dist.apache.org>
>>>>> >> > <http://dist.apache.org>
>>>>> >> > >>>> <http://dist.apache.org> [2],
>>>>> >> > >>>> which is signed with the key with fingerprint
>>>>> C8282E76 [3],
>>>>> >> > >>>> * all artifacts to be deployed to the Maven Central
>>>>> Repository [4],
>>>>> >> > >>>> * source code tag "v2.3.0-RC1" [5],
>>>>> >> > >>>> * website pull request listing the release and
>>>>> publishing the API
>>>>> >> reference
>>>>> >> > >>>> manual [6].
>>>>> >> > >>>> * Java artifacts were built with Maven 3.3.9 and
>>>>> Oracle JDK
>>>>> >> 1.8.0_111.
>>>>> >> > >>>> * Python artifacts are deployed along with the
>>>>> source release to the
>>>>> >> > >>>> dist.apache.org <http://dist.apache.org> <
>>>>> http://dist.apache.org>
>>>>> >> <http://dist.apache.org> [2].
>>>>> >> > >>>>
>>>>> >> > >>>> The vote will be open for at least 72 hours. It is
>>>>> adopted by majority approval,
>>>>> >> > >>>> with at least 3 PMC affirmative votes.
>>>>> >> > >>>>
>>>>> >> > >>>> Thanks,
>>>>> >> > >>>> JB
>>>>> >> > >>>>
>>>>> >> > >>>> [1]
>>>>> >> > >>>> https://issues.apache.org/
>>>>> jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>>>>> >> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?
>>>>> projectId=12319527&version=12341608>
>>>>> >> > <https://issues.apache.org/jira/secure/ReleaseNote.jspa?
>>>>> projectId=12319527&version=12341608
>>>>> >> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?
>>>>> projectId=12319527&version=12341608>>
>>>>> >> > >>>> [2] https://dist.apache.org/repos/
>>>>> dist/dev/beam/2.3.0/
>>>>> >> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
>>>>> >> > <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>>>>> >> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>
>>>>> >> > >>>> [3] https://dist.apache.org/repos/
>>>>> dist/release/beam/KEYS
>>>>> >> <https://dist.apache.org/repos/dist/release/beam/KEYS>
>>>>> >> > <https://dist.apache.org/repos/dist/release/beam/KEYS
>>>>> >> <https://dist.apache.org/repos/dist/release/beam/KEYS>>
>>>>> >> > >>>> [4] https://repository.apache.org/
>>>>> content/repositories/orgapachebeam-1026/
>>>>> >> <https://repository.apache.org/content/repositories/
>>>>> orgapachebeam-1026/>
>>>>> >> > <https://repository.apache.org/content/repositories/
>>>>> orgapachebeam-1026/
>>>>> >> <https://repository.apache.org/content/repositories/
>>>>> orgapachebeam-1026/>>
>>>>> >> > >>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>>>>> >> <https://github.com/apache/beam/tree/v2.3.0-RC1>
>>>>> >> > <https://github.com/apache/beam/tree/v2.3.0-RC1
>>>>> >> <https://github.com/apache/beam/tree/v2.3.0-RC1>>
>>>>> >> > >>>> [6] https://github.com/apache/beam-site/pull/381
>>>>> >> <https://github.com/apache/beam-site/pull/381>
>>>>> >> > <https://github.com/apache/beam-site/pull/381
>>>>> >> <https://github.com/apache/beam-site/pull/381>>
>>>>> >> > >>>>
>>>>> >> > >>>
>>>>> >> > >>
>>>>> >> > >> --
>>>>> >> > >> Jean-Baptiste Onofré
>>>>> >> > >> jbonofre@apache.org <ma...@apache.org>
>>>>> <mailto:jbonofre@apache.org
>>>>> >> <ma...@apache.org>>
>>>>> >> > <mailto:jbonofre@apache.org <ma...@apache.org>
>>>>> >> <mailto:jbonofre@apache.org <ma...@apache.org>>>
>>>>> >> > >> http://blog.nanthrax.net
>>>>> >> > >> Talend - http://www.talend.com
>>>>> >> > >
>>>>> >> >
>>>>> >> > --
>>>>> >> > Jean-Baptiste Onofré
>>>>> >> > jbonofre@apache.org <ma...@apache.org>
>>>>> >> <mailto:jbonofre@apache.org <ma...@apache.org>>
>>>>> >> > http://blog.nanthrax.net
>>>>> >> > Talend - http://www.talend.com
>>>>> >> >
>>>>> >> >
>>>>> >>
>>>>> >> --
>>>>> >> Jean-Baptiste Onofré
>>>>> >> jbonofre@apache.org <ma...@apache.org>
>>>>> >> http://blog.nanthrax.net
>>>>> >> Talend - http://www.talend.com
>>>>> >>
>>>>> >>
>>>>> >
>>>>>
>>>>> --
>>>>> Jean-Baptiste Onofré
>>>>> jbonofre@apache.org
>>>>> http://blog.nanthrax.net
>>>>> Talend - http://www.talend.com
>>>>>
>>>>
>>>>
>>>
>>
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Eugene Kirpichov <ki...@google.com>.
Possibly stupid question: are new RCs created from master, or by
cherrypicks of desired changes on top of previous RCs?
(it affects whether or not my recently merged change on "master" will be in
2.3.0; I'd like it to be :) )
On Tue, Feb 6, 2018 at 1:49 PM Lukasz Cwik <lc...@google.com> wrote:
> +1 for rollback as well.
>
> On Tue, Feb 6, 2018 at 12:37 PM, Reuven Lax <re...@google.com> wrote:
>
>> +1 for the rollback.
>>
>> On Tue, Feb 6, 2018 at 9:24 AM, Kenneth Knowles <kl...@google.com> wrote:
>>
>>> +1 for option 1. It is almost certainly repeated proto parsing. That is
>>> fixed for ParDo via generalized caching but not for other transforms. Flink
>>> also has a proto round trip, but the design might avoid the problem anyhow.
>>> We should investigate before releasing, or just preemptively roll it back
>>> too. The proto round trips are helpful for maturing portability and for
>>> preventing hacks from returning but otherwise don't matter so much yet.
>>>
>>> Kenn
>>>
>>> On Tue, Feb 6, 2018 at 8:54 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
>>> wrote:
>>>
>>>> Hi all,
>>>>
>>>> As you might have seen, I found the cause of performance degradation on
>>>> the
>>>> direct runner (BEAM-3617).
>>>>
>>>> We have basically three options for RC2:
>>>>
>>>> 1. We revert the change for 2.3.0. I think it's the fastest and more
>>>> secure way.
>>>> I created a PR for that (https://github.com/apache/beam/pull/4609). It
>>>> gives us
>>>> time to investigate and provide a better fix in Runner API for 2.4.0.
>>>>
>>>> 2. If the fix is easy and with limited impact in Runner API, we do that.
>>>>
>>>> 3. We leave as it is, adding a note that direct runner has worse
>>>> performance in
>>>> 2.3.0 compared to 2.2.0.
>>>>
>>>> In any case, I would like to cut RC2 later tonight or tomorrow morning
>>>> (my time).
>>>>
>>>> Thoughts ?
>>>>
>>>> Regards
>>>> JB
>>>>
>>>> On 02/06/2018 09:42 AM, Jean-Baptiste Onofré wrote:
>>>> > Hi Reuven,
>>>> >
>>>> > it's what I'm suspecting. git bisect should give us more information
>>>> (still in
>>>> > progress, 3 more steps to complete ;)). I keep you posted.
>>>> >
>>>> > Regards
>>>> > JB
>>>> >
>>>> > On 02/06/2018 08:36 AM, Reuven Lax wrote:
>>>> >> Could this be related to any of the portability changes?
>>>> >>
>>>> >> On Mon, Feb 5, 2018 at 7:51 AM, Jean-Baptiste Onofré <
>>>> jb@nanthrax.net
>>>> >> <ma...@nanthrax.net>> wrote:
>>>> >>
>>>> >> Created:
>>>> >>
>>>> >> https://issues.apache.org/jira/browse/BEAM-3617
>>>> >> <https://issues.apache.org/jira/browse/BEAM-3617>
>>>> >>
>>>> >> Regards
>>>> >> JB
>>>> >>
>>>> >> On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
>>>> >> > What is the Jira for direct runner perf?
>>>> >> >
>>>> >> > On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré <
>>>> jb@nanthrax.net <ma...@nanthrax.net>
>>>> >> > <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
>>>> >> >
>>>> >> > Thanks !
>>>> >> >
>>>> >> > I cherry-pick on release-2.3.0 branch.
>>>> >> >
>>>> >> > I'm on the direct runner perf test in the mean time.
>>>> >> >
>>>> >> > Thanks again !
>>>> >> >
>>>> >> > Regards
>>>> >> > JB
>>>> >> >
>>>> >> > On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
>>>> >> > > I merged fixes for:
>>>> >> > > - https://issues.apache.org/jira/browse/BEAM-3186
>>>> >> <https://issues.apache.org/jira/browse/BEAM-3186>
>>>> >> > <https://issues.apache.org/jira/browse/BEAM-3186
>>>> >> <https://issues.apache.org/jira/browse/BEAM-3186>>
>>>> >> > > - https://issues.apache.org/jira/browse/BEAM-3589
>>>> >> <https://issues.apache.org/jira/browse/BEAM-3589>
>>>> >> > <https://issues.apache.org/jira/browse/BEAM-3589
>>>> >> <https://issues.apache.org/jira/browse/BEAM-3589>>
>>>> >> > >
>>>> >> > > @JB I didn't yet merge them on the 2.3.0 branch, though,
>>>> but I can or you can go
>>>> >> > > ahead.
>>>> >> > >
>>>> >> > >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <
>>>> jb@nanthrax.net <ma...@nanthrax.net> <mailto:jb@nanthrax.net
>>>> >> <ma...@nanthrax.net>>
>>>> >> > >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>
>>>> >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>>> wrote:
>>>> >> > >>
>>>> >> > >> Hi guys,
>>>> >> > >>
>>>> >> > >> Quick update on the RC2 preparation:
>>>> >> > >>
>>>> >> > >> * BEAM-3587 (TextIO with Flink) seems related to a
>>>> custom build with
>>>> >> > Gradle (not
>>>> >> > >> using artifacts created by Maven). Anyway, I will take
>>>> a look today.
>>>> >> > >> * BEAM-3186 has a PR. Aljoscha will do the review
>>>> pretty soon.
>>>> >> > >> * I'm also taking a look on the direct runner
>>>> performance change.
>>>> >> > >>
>>>> >> > >> The goal is to cut RC2 tomorrow or the day after.
>>>> >> > >>
>>>> >> > >> Regards
>>>> >> > >> JB
>>>> >> > >>
>>>> >> > >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
>>>> >> > >>> Hi guys,
>>>> >> > >>>
>>>> >> > >>> Especially due to BEAM-3587 & BEAM-3186 regressions, I
>>>> cancel RC1.
>>>> >> > >>>
>>>> >> > >>> We will cherry-pick fixes on release-2.3.0 branch.
>>>> >> > >>>
>>>> >> > >>> I'm updating Jira right now. When the fixes will be
>>>> cherry-picked, I
>>>> >> > will submit
>>>> >> > >>> a RC2 to vote.
>>>> >> > >>>
>>>> >> > >>> Thanks !
>>>> >> > >>> Regards
>>>> >> > >>> JB
>>>> >> > >>>
>>>> >> > >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>>>> >> > >>>> Hi everyone,
>>>> >> > >>>>
>>>> >> > >>>> Please review and vote on the release candidate #1
>>>> for the version
>>>> >> > 2.3.0, as
>>>> >> > >>>> follows:
>>>> >> > >>>>
>>>> >> > >>>> [ ] +1, Approve the release
>>>> >> > >>>> [ ] -1, Do not approve the release (please provide
>>>> specific comments)
>>>> >> > >>>>
>>>> >> > >>>>
>>>> >> > >>>> The complete staging area is available for your
>>>> review, which
>>>> >> includes:
>>>> >> > >>>> * JIRA release notes [1],
>>>> >> > >>>> * the official Apache source release to be deployed to
>>>> >> dist.apache.org <http://dist.apache.org>
>>>> >> > <http://dist.apache.org>
>>>> >> > >>>> <http://dist.apache.org> [2],
>>>> >> > >>>> which is signed with the key with fingerprint
>>>> C8282E76 [3],
>>>> >> > >>>> * all artifacts to be deployed to the Maven Central
>>>> Repository [4],
>>>> >> > >>>> * source code tag "v2.3.0-RC1" [5],
>>>> >> > >>>> * website pull request listing the release and
>>>> publishing the API
>>>> >> reference
>>>> >> > >>>> manual [6].
>>>> >> > >>>> * Java artifacts were built with Maven 3.3.9 and
>>>> Oracle JDK
>>>> >> 1.8.0_111.
>>>> >> > >>>> * Python artifacts are deployed along with the source
>>>> release to the
>>>> >> > >>>> dist.apache.org <http://dist.apache.org> <
>>>> http://dist.apache.org>
>>>> >> <http://dist.apache.org> [2].
>>>> >> > >>>>
>>>> >> > >>>> The vote will be open for at least 72 hours. It is
>>>> adopted by majority approval,
>>>> >> > >>>> with at least 3 PMC affirmative votes.
>>>> >> > >>>>
>>>> >> > >>>> Thanks,
>>>> >> > >>>> JB
>>>> >> > >>>>
>>>> >> > >>>> [1]
>>>> >> > >>>>
>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>>>> >> <
>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>>>> >
>>>> >> > <
>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>>>> >> <
>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>>>> >>
>>>> >> > >>>> [2]
>>>> https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>>>> >> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
>>>> >> > <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>>>> >> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>
>>>> >> > >>>> [3]
>>>> https://dist.apache.org/repos/dist/release/beam/KEYS
>>>> >> <https://dist.apache.org/repos/dist/release/beam/KEYS>
>>>> >> > <https://dist.apache.org/repos/dist/release/beam/KEYS
>>>> >> <https://dist.apache.org/repos/dist/release/beam/KEYS>>
>>>> >> > >>>> [4]
>>>> https://repository.apache.org/content/repositories/orgapachebeam-1026/
>>>> >> <
>>>> https://repository.apache.org/content/repositories/orgapachebeam-1026/>
>>>> >> > <
>>>> https://repository.apache.org/content/repositories/orgapachebeam-1026/
>>>> >> <
>>>> https://repository.apache.org/content/repositories/orgapachebeam-1026/
>>>> >>
>>>> >> > >>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>>>> >> <https://github.com/apache/beam/tree/v2.3.0-RC1>
>>>> >> > <https://github.com/apache/beam/tree/v2.3.0-RC1
>>>> >> <https://github.com/apache/beam/tree/v2.3.0-RC1>>
>>>> >> > >>>> [6] https://github.com/apache/beam-site/pull/381
>>>> >> <https://github.com/apache/beam-site/pull/381>
>>>> >> > <https://github.com/apache/beam-site/pull/381
>>>> >> <https://github.com/apache/beam-site/pull/381>>
>>>> >> > >>>>
>>>> >> > >>>
>>>> >> > >>
>>>> >> > >> --
>>>> >> > >> Jean-Baptiste Onofré
>>>> >> > >> jbonofre@apache.org <ma...@apache.org>
>>>> <mailto:jbonofre@apache.org
>>>> >> <ma...@apache.org>>
>>>> >> > <mailto:jbonofre@apache.org <ma...@apache.org>
>>>> >> <mailto:jbonofre@apache.org <ma...@apache.org>>>
>>>> >> > >> http://blog.nanthrax.net
>>>> >> > >> Talend - http://www.talend.com
>>>> >> > >
>>>> >> >
>>>> >> > --
>>>> >> > Jean-Baptiste Onofré
>>>> >> > jbonofre@apache.org <ma...@apache.org>
>>>> >> <mailto:jbonofre@apache.org <ma...@apache.org>>
>>>> >> > http://blog.nanthrax.net
>>>> >> > Talend - http://www.talend.com
>>>> >> >
>>>> >> >
>>>> >>
>>>> >> --
>>>> >> Jean-Baptiste Onofré
>>>> >> jbonofre@apache.org <ma...@apache.org>
>>>> >> http://blog.nanthrax.net
>>>> >> Talend - http://www.talend.com
>>>> >>
>>>> >>
>>>> >
>>>>
>>>> --
>>>> Jean-Baptiste Onofré
>>>> jbonofre@apache.org
>>>> http://blog.nanthrax.net
>>>> Talend - http://www.talend.com
>>>>
>>>
>>>
>>
>
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Lukasz Cwik <lc...@google.com>.
+1 for rollback as well.
On Tue, Feb 6, 2018 at 12:37 PM, Reuven Lax <re...@google.com> wrote:
> +1 for the rollback.
>
> On Tue, Feb 6, 2018 at 9:24 AM, Kenneth Knowles <kl...@google.com> wrote:
>
>> +1 for option 1. It is almost certainly repeated proto parsing. That is
>> fixed for ParDo via generalized caching but not for other transforms. Flink
>> also has a proto round trip, but the design might avoid the problem anyhow.
>> We should investigate before releasing, or just preemptively roll it back
>> too. The proto round trips are helpful for maturing portability and for
>> preventing hacks from returning but otherwise don't matter so much yet.
>>
>> Kenn
>>
>> On Tue, Feb 6, 2018 at 8:54 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
>> wrote:
>>
>>> Hi all,
>>>
>>> As you might have seen, I found the cause of performance degradation on
>>> the
>>> direct runner (BEAM-3617).
>>>
>>> We have basically three options for RC2:
>>>
>>> 1. We revert the change for 2.3.0. I think it's the fastest and more
>>> secure way.
>>> I created a PR for that (https://github.com/apache/beam/pull/4609). It
>>> gives us
>>> time to investigate and provide a better fix in Runner API for 2.4.0.
>>>
>>> 2. If the fix is easy and with limited impact in Runner API, we do that.
>>>
>>> 3. We leave as it is, adding a note that direct runner has worse
>>> performance in
>>> 2.3.0 compared to 2.2.0.
>>>
>>> In any case, I would like to cut RC2 later tonight or tomorrow morning
>>> (my time).
>>>
>>> Thoughts ?
>>>
>>> Regards
>>> JB
>>>
>>> On 02/06/2018 09:42 AM, Jean-Baptiste Onofré wrote:
>>> > Hi Reuven,
>>> >
>>> > it's what I'm suspecting. git bisect should give us more information
>>> (still in
>>> > progress, 3 more steps to complete ;)). I keep you posted.
>>> >
>>> > Regards
>>> > JB
>>> >
>>> > On 02/06/2018 08:36 AM, Reuven Lax wrote:
>>> >> Could this be related to any of the portability changes?
>>> >>
>>> >> On Mon, Feb 5, 2018 at 7:51 AM, Jean-Baptiste Onofré <jb@nanthrax.net
>>> >> <ma...@nanthrax.net>> wrote:
>>> >>
>>> >> Created:
>>> >>
>>> >> https://issues.apache.org/jira/browse/BEAM-3617
>>> >> <https://issues.apache.org/jira/browse/BEAM-3617>
>>> >>
>>> >> Regards
>>> >> JB
>>> >>
>>> >> On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
>>> >> > What is the Jira for direct runner perf?
>>> >> >
>>> >> > On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré <
>>> jb@nanthrax.net <ma...@nanthrax.net>
>>> >> > <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
>>> >> >
>>> >> > Thanks !
>>> >> >
>>> >> > I cherry-pick on release-2.3.0 branch.
>>> >> >
>>> >> > I'm on the direct runner perf test in the mean time.
>>> >> >
>>> >> > Thanks again !
>>> >> >
>>> >> > Regards
>>> >> > JB
>>> >> >
>>> >> > On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
>>> >> > > I merged fixes for:
>>> >> > > - https://issues.apache.org/jira/browse/BEAM-3186
>>> >> <https://issues.apache.org/jira/browse/BEAM-3186>
>>> >> > <https://issues.apache.org/jira/browse/BEAM-3186
>>> >> <https://issues.apache.org/jira/browse/BEAM-3186>>
>>> >> > > - https://issues.apache.org/jira/browse/BEAM-3589
>>> >> <https://issues.apache.org/jira/browse/BEAM-3589>
>>> >> > <https://issues.apache.org/jira/browse/BEAM-3589
>>> >> <https://issues.apache.org/jira/browse/BEAM-3589>>
>>> >> > >
>>> >> > > @JB I didn't yet merge them on the 2.3.0 branch, though,
>>> but I can or you can go
>>> >> > > ahead.
>>> >> > >
>>> >> > >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <
>>> jb@nanthrax.net <ma...@nanthrax.net> <mailto:jb@nanthrax.net
>>> >> <ma...@nanthrax.net>>
>>> >> > >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>
>>> >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>>> wrote:
>>> >> > >>
>>> >> > >> Hi guys,
>>> >> > >>
>>> >> > >> Quick update on the RC2 preparation:
>>> >> > >>
>>> >> > >> * BEAM-3587 (TextIO with Flink) seems related to a
>>> custom build with
>>> >> > Gradle (not
>>> >> > >> using artifacts created by Maven). Anyway, I will take a
>>> look today.
>>> >> > >> * BEAM-3186 has a PR. Aljoscha will do the review pretty
>>> soon.
>>> >> > >> * I'm also taking a look on the direct runner
>>> performance change.
>>> >> > >>
>>> >> > >> The goal is to cut RC2 tomorrow or the day after.
>>> >> > >>
>>> >> > >> Regards
>>> >> > >> JB
>>> >> > >>
>>> >> > >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
>>> >> > >>> Hi guys,
>>> >> > >>>
>>> >> > >>> Especially due to BEAM-3587 & BEAM-3186 regressions, I
>>> cancel RC1.
>>> >> > >>>
>>> >> > >>> We will cherry-pick fixes on release-2.3.0 branch.
>>> >> > >>>
>>> >> > >>> I'm updating Jira right now. When the fixes will be
>>> cherry-picked, I
>>> >> > will submit
>>> >> > >>> a RC2 to vote.
>>> >> > >>>
>>> >> > >>> Thanks !
>>> >> > >>> Regards
>>> >> > >>> JB
>>> >> > >>>
>>> >> > >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>>> >> > >>>> Hi everyone,
>>> >> > >>>>
>>> >> > >>>> Please review and vote on the release candidate #1 for
>>> the version
>>> >> > 2.3.0, as
>>> >> > >>>> follows:
>>> >> > >>>>
>>> >> > >>>> [ ] +1, Approve the release
>>> >> > >>>> [ ] -1, Do not approve the release (please provide
>>> specific comments)
>>> >> > >>>>
>>> >> > >>>>
>>> >> > >>>> The complete staging area is available for your
>>> review, which
>>> >> includes:
>>> >> > >>>> * JIRA release notes [1],
>>> >> > >>>> * the official Apache source release to be deployed to
>>> >> dist.apache.org <http://dist.apache.org>
>>> >> > <http://dist.apache.org>
>>> >> > >>>> <http://dist.apache.org> [2],
>>> >> > >>>> which is signed with the key with fingerprint C8282E76
>>> [3],
>>> >> > >>>> * all artifacts to be deployed to the Maven Central
>>> Repository [4],
>>> >> > >>>> * source code tag "v2.3.0-RC1" [5],
>>> >> > >>>> * website pull request listing the release and
>>> publishing the API
>>> >> reference
>>> >> > >>>> manual [6].
>>> >> > >>>> * Java artifacts were built with Maven 3.3.9 and
>>> Oracle JDK
>>> >> 1.8.0_111.
>>> >> > >>>> * Python artifacts are deployed along with the source
>>> release to the
>>> >> > >>>> dist.apache.org <http://dist.apache.org> <
>>> http://dist.apache.org>
>>> >> <http://dist.apache.org> [2].
>>> >> > >>>>
>>> >> > >>>> The vote will be open for at least 72 hours. It is
>>> adopted by majority approval,
>>> >> > >>>> with at least 3 PMC affirmative votes.
>>> >> > >>>>
>>> >> > >>>> Thanks,
>>> >> > >>>> JB
>>> >> > >>>>
>>> >> > >>>> [1]
>>> >> > >>>> https://issues.apache.org/jira
>>> /secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>>> >> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?pro
>>> jectId=12319527&version=12341608>
>>> >> > <https://issues.apache.org/jira/secure/ReleaseNote.jspa?pro
>>> jectId=12319527&version=12341608
>>> >> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?pro
>>> jectId=12319527&version=12341608>>
>>> >> > >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>>> >> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
>>> >> > <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>>> >> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>
>>> >> > >>>> [3] https://dist.apache.org/repos/
>>> dist/release/beam/KEYS
>>> >> <https://dist.apache.org/repos/dist/release/beam/KEYS>
>>> >> > <https://dist.apache.org/repos/dist/release/beam/KEYS
>>> >> <https://dist.apache.org/repos/dist/release/beam/KEYS>>
>>> >> > >>>> [4] https://repository.apache.org/
>>> content/repositories/orgapachebeam-1026/
>>> >> <https://repository.apache.org/content/repositories/orgapac
>>> hebeam-1026/>
>>> >> > <https://repository.apache.org/content/repositories/orgapac
>>> hebeam-1026/
>>> >> <https://repository.apache.org/content/repositories/orgapac
>>> hebeam-1026/>>
>>> >> > >>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>>> >> <https://github.com/apache/beam/tree/v2.3.0-RC1>
>>> >> > <https://github.com/apache/beam/tree/v2.3.0-RC1
>>> >> <https://github.com/apache/beam/tree/v2.3.0-RC1>>
>>> >> > >>>> [6] https://github.com/apache/beam-site/pull/381
>>> >> <https://github.com/apache/beam-site/pull/381>
>>> >> > <https://github.com/apache/beam-site/pull/381
>>> >> <https://github.com/apache/beam-site/pull/381>>
>>> >> > >>>>
>>> >> > >>>
>>> >> > >>
>>> >> > >> --
>>> >> > >> Jean-Baptiste Onofré
>>> >> > >> jbonofre@apache.org <ma...@apache.org>
>>> <mailto:jbonofre@apache.org
>>> >> <ma...@apache.org>>
>>> >> > <mailto:jbonofre@apache.org <ma...@apache.org>
>>> >> <mailto:jbonofre@apache.org <ma...@apache.org>>>
>>> >> > >> http://blog.nanthrax.net
>>> >> > >> Talend - http://www.talend.com
>>> >> > >
>>> >> >
>>> >> > --
>>> >> > Jean-Baptiste Onofré
>>> >> > jbonofre@apache.org <ma...@apache.org>
>>> >> <mailto:jbonofre@apache.org <ma...@apache.org>>
>>> >> > http://blog.nanthrax.net
>>> >> > Talend - http://www.talend.com
>>> >> >
>>> >> >
>>> >>
>>> >> --
>>> >> Jean-Baptiste Onofré
>>> >> jbonofre@apache.org <ma...@apache.org>
>>> >> http://blog.nanthrax.net
>>> >> Talend - http://www.talend.com
>>> >>
>>> >>
>>> >
>>>
>>> --
>>> Jean-Baptiste Onofré
>>> jbonofre@apache.org
>>> http://blog.nanthrax.net
>>> Talend - http://www.talend.com
>>>
>>
>>
>
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Reuven Lax <re...@google.com>.
+1 for the rollback.
On Tue, Feb 6, 2018 at 9:24 AM, Kenneth Knowles <kl...@google.com> wrote:
> +1 for option 1. It is almost certainly repeated proto parsing. That is
> fixed for ParDo via generalized caching but not for other transforms. Flink
> also has a proto round trip, but the design might avoid the problem anyhow.
> We should investigate before releasing, or just preemptively roll it back
> too. The proto round trips are helpful for maturing portability and for
> preventing hacks from returning but otherwise don't matter so much yet.
>
> Kenn
>
> On Tue, Feb 6, 2018 at 8:54 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
>
>> Hi all,
>>
>> As you might have seen, I found the cause of performance degradation on
>> the
>> direct runner (BEAM-3617).
>>
>> We have basically three options for RC2:
>>
>> 1. We revert the change for 2.3.0. I think it's the fastest and more
>> secure way.
>> I created a PR for that (https://github.com/apache/beam/pull/4609). It
>> gives us
>> time to investigate and provide a better fix in Runner API for 2.4.0.
>>
>> 2. If the fix is easy and with limited impact in Runner API, we do that.
>>
>> 3. We leave as it is, adding a note that direct runner has worse
>> performance in
>> 2.3.0 compared to 2.2.0.
>>
>> In any case, I would like to cut RC2 later tonight or tomorrow morning
>> (my time).
>>
>> Thoughts ?
>>
>> Regards
>> JB
>>
>> On 02/06/2018 09:42 AM, Jean-Baptiste Onofré wrote:
>> > Hi Reuven,
>> >
>> > it's what I'm suspecting. git bisect should give us more information
>> (still in
>> > progress, 3 more steps to complete ;)). I keep you posted.
>> >
>> > Regards
>> > JB
>> >
>> > On 02/06/2018 08:36 AM, Reuven Lax wrote:
>> >> Could this be related to any of the portability changes?
>> >>
>> >> On Mon, Feb 5, 2018 at 7:51 AM, Jean-Baptiste Onofré <jb@nanthrax.net
>> >> <ma...@nanthrax.net>> wrote:
>> >>
>> >> Created:
>> >>
>> >> https://issues.apache.org/jira/browse/BEAM-3617
>> >> <https://issues.apache.org/jira/browse/BEAM-3617>
>> >>
>> >> Regards
>> >> JB
>> >>
>> >> On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
>> >> > What is the Jira for direct runner perf?
>> >> >
>> >> > On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré <
>> jb@nanthrax.net <ma...@nanthrax.net>
>> >> > <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
>> >> >
>> >> > Thanks !
>> >> >
>> >> > I cherry-pick on release-2.3.0 branch.
>> >> >
>> >> > I'm on the direct runner perf test in the mean time.
>> >> >
>> >> > Thanks again !
>> >> >
>> >> > Regards
>> >> > JB
>> >> >
>> >> > On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
>> >> > > I merged fixes for:
>> >> > > - https://issues.apache.org/jira/browse/BEAM-3186
>> >> <https://issues.apache.org/jira/browse/BEAM-3186>
>> >> > <https://issues.apache.org/jira/browse/BEAM-3186
>> >> <https://issues.apache.org/jira/browse/BEAM-3186>>
>> >> > > - https://issues.apache.org/jira/browse/BEAM-3589
>> >> <https://issues.apache.org/jira/browse/BEAM-3589>
>> >> > <https://issues.apache.org/jira/browse/BEAM-3589
>> >> <https://issues.apache.org/jira/browse/BEAM-3589>>
>> >> > >
>> >> > > @JB I didn't yet merge them on the 2.3.0 branch, though,
>> but I can or you can go
>> >> > > ahead.
>> >> > >
>> >> > >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <
>> jb@nanthrax.net <ma...@nanthrax.net> <mailto:jb@nanthrax.net
>> >> <ma...@nanthrax.net>>
>> >> > >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>
>> >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>>> wrote:
>> >> > >>
>> >> > >> Hi guys,
>> >> > >>
>> >> > >> Quick update on the RC2 preparation:
>> >> > >>
>> >> > >> * BEAM-3587 (TextIO with Flink) seems related to a custom
>> build with
>> >> > Gradle (not
>> >> > >> using artifacts created by Maven). Anyway, I will take a
>> look today.
>> >> > >> * BEAM-3186 has a PR. Aljoscha will do the review pretty
>> soon.
>> >> > >> * I'm also taking a look on the direct runner performance
>> change.
>> >> > >>
>> >> > >> The goal is to cut RC2 tomorrow or the day after.
>> >> > >>
>> >> > >> Regards
>> >> > >> JB
>> >> > >>
>> >> > >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
>> >> > >>> Hi guys,
>> >> > >>>
>> >> > >>> Especially due to BEAM-3587 & BEAM-3186 regressions, I
>> cancel RC1.
>> >> > >>>
>> >> > >>> We will cherry-pick fixes on release-2.3.0 branch.
>> >> > >>>
>> >> > >>> I'm updating Jira right now. When the fixes will be
>> cherry-picked, I
>> >> > will submit
>> >> > >>> a RC2 to vote.
>> >> > >>>
>> >> > >>> Thanks !
>> >> > >>> Regards
>> >> > >>> JB
>> >> > >>>
>> >> > >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>> >> > >>>> Hi everyone,
>> >> > >>>>
>> >> > >>>> Please review and vote on the release candidate #1 for
>> the version
>> >> > 2.3.0, as
>> >> > >>>> follows:
>> >> > >>>>
>> >> > >>>> [ ] +1, Approve the release
>> >> > >>>> [ ] -1, Do not approve the release (please provide
>> specific comments)
>> >> > >>>>
>> >> > >>>>
>> >> > >>>> The complete staging area is available for your review,
>> which
>> >> includes:
>> >> > >>>> * JIRA release notes [1],
>> >> > >>>> * the official Apache source release to be deployed to
>> >> dist.apache.org <http://dist.apache.org>
>> >> > <http://dist.apache.org>
>> >> > >>>> <http://dist.apache.org> [2],
>> >> > >>>> which is signed with the key with fingerprint C8282E76
>> [3],
>> >> > >>>> * all artifacts to be deployed to the Maven Central
>> Repository [4],
>> >> > >>>> * source code tag "v2.3.0-RC1" [5],
>> >> > >>>> * website pull request listing the release and
>> publishing the API
>> >> reference
>> >> > >>>> manual [6].
>> >> > >>>> * Java artifacts were built with Maven 3.3.9 and Oracle
>> JDK
>> >> 1.8.0_111.
>> >> > >>>> * Python artifacts are deployed along with the source
>> release to the
>> >> > >>>> dist.apache.org <http://dist.apache.org> <
>> http://dist.apache.org>
>> >> <http://dist.apache.org> [2].
>> >> > >>>>
>> >> > >>>> The vote will be open for at least 72 hours. It is
>> adopted by majority approval,
>> >> > >>>> with at least 3 PMC affirmative votes.
>> >> > >>>>
>> >> > >>>> Thanks,
>> >> > >>>> JB
>> >> > >>>>
>> >> > >>>> [1]
>> >> > >>>> https://issues.apache.org/jira
>> /secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>> >> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?pro
>> jectId=12319527&version=12341608>
>> >> > <https://issues.apache.org/jira/secure/ReleaseNote.jspa?pro
>> jectId=12319527&version=12341608
>> >> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?pro
>> jectId=12319527&version=12341608>>
>> >> > >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>> >> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
>> >> > <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>> >> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>
>> >> > >>>> [3] https://dist.apache.org/repos/
>> dist/release/beam/KEYS
>> >> <https://dist.apache.org/repos/dist/release/beam/KEYS>
>> >> > <https://dist.apache.org/repos/dist/release/beam/KEYS
>> >> <https://dist.apache.org/repos/dist/release/beam/KEYS>>
>> >> > >>>> [4] https://repository.apache.org/
>> content/repositories/orgapachebeam-1026/
>> >> <https://repository.apache.org/content/repositories/orgapac
>> hebeam-1026/>
>> >> > <https://repository.apache.org/content/repositories/orgapac
>> hebeam-1026/
>> >> <https://repository.apache.org/content/repositories/orgapac
>> hebeam-1026/>>
>> >> > >>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>> >> <https://github.com/apache/beam/tree/v2.3.0-RC1>
>> >> > <https://github.com/apache/beam/tree/v2.3.0-RC1
>> >> <https://github.com/apache/beam/tree/v2.3.0-RC1>>
>> >> > >>>> [6] https://github.com/apache/beam-site/pull/381
>> >> <https://github.com/apache/beam-site/pull/381>
>> >> > <https://github.com/apache/beam-site/pull/381
>> >> <https://github.com/apache/beam-site/pull/381>>
>> >> > >>>>
>> >> > >>>
>> >> > >>
>> >> > >> --
>> >> > >> Jean-Baptiste Onofré
>> >> > >> jbonofre@apache.org <ma...@apache.org> <mailto:
>> jbonofre@apache.org
>> >> <ma...@apache.org>>
>> >> > <mailto:jbonofre@apache.org <ma...@apache.org>
>> >> <mailto:jbonofre@apache.org <ma...@apache.org>>>
>> >> > >> http://blog.nanthrax.net
>> >> > >> Talend - http://www.talend.com
>> >> > >
>> >> >
>> >> > --
>> >> > Jean-Baptiste Onofré
>> >> > jbonofre@apache.org <ma...@apache.org>
>> >> <mailto:jbonofre@apache.org <ma...@apache.org>>
>> >> > http://blog.nanthrax.net
>> >> > Talend - http://www.talend.com
>> >> >
>> >> >
>> >>
>> >> --
>> >> Jean-Baptiste Onofré
>> >> jbonofre@apache.org <ma...@apache.org>
>> >> http://blog.nanthrax.net
>> >> Talend - http://www.talend.com
>> >>
>> >>
>> >
>>
>> --
>> Jean-Baptiste Onofré
>> jbonofre@apache.org
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>
>
>
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Kenneth Knowles <kl...@google.com>.
+1 for option 1. It is almost certainly repeated proto parsing. That is
fixed for ParDo via generalized caching but not for other transforms. Flink
also has a proto round trip, but the design might avoid the problem anyhow.
We should investigate before releasing, or just preemptively roll it back
too. The proto round trips are helpful for maturing portability and for
preventing hacks from returning but otherwise don't matter so much yet.
Kenn
On Tue, Feb 6, 2018 at 8:54 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> Hi all,
>
> As you might have seen, I found the cause of performance degradation on the
> direct runner (BEAM-3617).
>
> We have basically three options for RC2:
>
> 1. We revert the change for 2.3.0. I think it's the fastest and more
> secure way.
> I created a PR for that (https://github.com/apache/beam/pull/4609). It
> gives us
> time to investigate and provide a better fix in Runner API for 2.4.0.
>
> 2. If the fix is easy and with limited impact in Runner API, we do that.
>
> 3. We leave as it is, adding a note that direct runner has worse
> performance in
> 2.3.0 compared to 2.2.0.
>
> In any case, I would like to cut RC2 later tonight or tomorrow morning (my
> time).
>
> Thoughts ?
>
> Regards
> JB
>
> On 02/06/2018 09:42 AM, Jean-Baptiste Onofré wrote:
> > Hi Reuven,
> >
> > it's what I'm suspecting. git bisect should give us more information
> (still in
> > progress, 3 more steps to complete ;)). I keep you posted.
> >
> > Regards
> > JB
> >
> > On 02/06/2018 08:36 AM, Reuven Lax wrote:
> >> Could this be related to any of the portability changes?
> >>
> >> On Mon, Feb 5, 2018 at 7:51 AM, Jean-Baptiste Onofré <jb@nanthrax.net
> >> <ma...@nanthrax.net>> wrote:
> >>
> >> Created:
> >>
> >> https://issues.apache.org/jira/browse/BEAM-3617
> >> <https://issues.apache.org/jira/browse/BEAM-3617>
> >>
> >> Regards
> >> JB
> >>
> >> On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
> >> > What is the Jira for direct runner perf?
> >> >
> >> > On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré <
> jb@nanthrax.net <ma...@nanthrax.net>
> >> > <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
> >> >
> >> > Thanks !
> >> >
> >> > I cherry-pick on release-2.3.0 branch.
> >> >
> >> > I'm on the direct runner perf test in the mean time.
> >> >
> >> > Thanks again !
> >> >
> >> > Regards
> >> > JB
> >> >
> >> > On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
> >> > > I merged fixes for:
> >> > > - https://issues.apache.org/jira/browse/BEAM-3186
> >> <https://issues.apache.org/jira/browse/BEAM-3186>
> >> > <https://issues.apache.org/jira/browse/BEAM-3186
> >> <https://issues.apache.org/jira/browse/BEAM-3186>>
> >> > > - https://issues.apache.org/jira/browse/BEAM-3589
> >> <https://issues.apache.org/jira/browse/BEAM-3589>
> >> > <https://issues.apache.org/jira/browse/BEAM-3589
> >> <https://issues.apache.org/jira/browse/BEAM-3589>>
> >> > >
> >> > > @JB I didn't yet merge them on the 2.3.0 branch, though,
> but I can or you can go
> >> > > ahead.
> >> > >
> >> > >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <
> jb@nanthrax.net <ma...@nanthrax.net> <mailto:jb@nanthrax.net
> >> <ma...@nanthrax.net>>
> >> > >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>
> >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>>> wrote:
> >> > >>
> >> > >> Hi guys,
> >> > >>
> >> > >> Quick update on the RC2 preparation:
> >> > >>
> >> > >> * BEAM-3587 (TextIO with Flink) seems related to a custom
> build with
> >> > Gradle (not
> >> > >> using artifacts created by Maven). Anyway, I will take a
> look today.
> >> > >> * BEAM-3186 has a PR. Aljoscha will do the review pretty
> soon.
> >> > >> * I'm also taking a look on the direct runner performance
> change.
> >> > >>
> >> > >> The goal is to cut RC2 tomorrow or the day after.
> >> > >>
> >> > >> Regards
> >> > >> JB
> >> > >>
> >> > >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
> >> > >>> Hi guys,
> >> > >>>
> >> > >>> Especially due to BEAM-3587 & BEAM-3186 regressions, I
> cancel RC1.
> >> > >>>
> >> > >>> We will cherry-pick fixes on release-2.3.0 branch.
> >> > >>>
> >> > >>> I'm updating Jira right now. When the fixes will be
> cherry-picked, I
> >> > will submit
> >> > >>> a RC2 to vote.
> >> > >>>
> >> > >>> Thanks !
> >> > >>> Regards
> >> > >>> JB
> >> > >>>
> >> > >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
> >> > >>>> Hi everyone,
> >> > >>>>
> >> > >>>> Please review and vote on the release candidate #1 for
> the version
> >> > 2.3.0, as
> >> > >>>> follows:
> >> > >>>>
> >> > >>>> [ ] +1, Approve the release
> >> > >>>> [ ] -1, Do not approve the release (please provide
> specific comments)
> >> > >>>>
> >> > >>>>
> >> > >>>> The complete staging area is available for your review,
> which
> >> includes:
> >> > >>>> * JIRA release notes [1],
> >> > >>>> * the official Apache source release to be deployed to
> >> dist.apache.org <http://dist.apache.org>
> >> > <http://dist.apache.org>
> >> > >>>> <http://dist.apache.org> [2],
> >> > >>>> which is signed with the key with fingerprint C8282E76
> [3],
> >> > >>>> * all artifacts to be deployed to the Maven Central
> Repository [4],
> >> > >>>> * source code tag "v2.3.0-RC1" [5],
> >> > >>>> * website pull request listing the release and
> publishing the API
> >> reference
> >> > >>>> manual [6].
> >> > >>>> * Java artifacts were built with Maven 3.3.9 and Oracle
> JDK
> >> 1.8.0_111.
> >> > >>>> * Python artifacts are deployed along with the source
> release to the
> >> > >>>> dist.apache.org <http://dist.apache.org> <
> http://dist.apache.org>
> >> <http://dist.apache.org> [2].
> >> > >>>>
> >> > >>>> The vote will be open for at least 72 hours. It is
> adopted by majority approval,
> >> > >>>> with at least 3 PMC affirmative votes.
> >> > >>>>
> >> > >>>> Thanks,
> >> > >>>> JB
> >> > >>>>
> >> > >>>> [1]
> >> > >>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12319527&version=12341608
> >> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12319527&version=12341608>
> >> > <https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12319527&version=12341608
> >> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12319527&version=12341608>>
> >> > >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> >> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
> >> > <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> >> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>
> >> > >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> >> <https://dist.apache.org/repos/dist/release/beam/KEYS>
> >> > <https://dist.apache.org/repos/dist/release/beam/KEYS
> >> <https://dist.apache.org/repos/dist/release/beam/KEYS>>
> >> > >>>> [4] https://repository.apache.org/content/repositories/
> orgapachebeam-1026/
> >> <https://repository.apache.org/content/repositories/
> orgapachebeam-1026/>
> >> > <https://repository.apache.org/content/repositories/
> orgapachebeam-1026/
> >> <https://repository.apache.org/content/repositories/
> orgapachebeam-1026/>>
> >> > >>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
> >> <https://github.com/apache/beam/tree/v2.3.0-RC1>
> >> > <https://github.com/apache/beam/tree/v2.3.0-RC1
> >> <https://github.com/apache/beam/tree/v2.3.0-RC1>>
> >> > >>>> [6] https://github.com/apache/beam-site/pull/381
> >> <https://github.com/apache/beam-site/pull/381>
> >> > <https://github.com/apache/beam-site/pull/381
> >> <https://github.com/apache/beam-site/pull/381>>
> >> > >>>>
> >> > >>>
> >> > >>
> >> > >> --
> >> > >> Jean-Baptiste Onofré
> >> > >> jbonofre@apache.org <ma...@apache.org> <mailto:
> jbonofre@apache.org
> >> <ma...@apache.org>>
> >> > <mailto:jbonofre@apache.org <ma...@apache.org>
> >> <mailto:jbonofre@apache.org <ma...@apache.org>>>
> >> > >> http://blog.nanthrax.net
> >> > >> Talend - http://www.talend.com
> >> > >
> >> >
> >> > --
> >> > Jean-Baptiste Onofré
> >> > jbonofre@apache.org <ma...@apache.org>
> >> <mailto:jbonofre@apache.org <ma...@apache.org>>
> >> > http://blog.nanthrax.net
> >> > Talend - http://www.talend.com
> >> >
> >> >
> >>
> >> --
> >> Jean-Baptiste Onofré
> >> jbonofre@apache.org <ma...@apache.org>
> >> http://blog.nanthrax.net
> >> Talend - http://www.talend.com
> >>
> >>
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi all,
As you might have seen, I found the cause of performance degradation on the
direct runner (BEAM-3617).
We have basically three options for RC2:
1. We revert the change for 2.3.0. I think it's the fastest and more secure way.
I created a PR for that (https://github.com/apache/beam/pull/4609). It gives us
time to investigate and provide a better fix in Runner API for 2.4.0.
2. If the fix is easy and with limited impact in Runner API, we do that.
3. We leave as it is, adding a note that direct runner has worse performance in
2.3.0 compared to 2.2.0.
In any case, I would like to cut RC2 later tonight or tomorrow morning (my time).
Thoughts ?
Regards
JB
On 02/06/2018 09:42 AM, Jean-Baptiste Onofré wrote:
> Hi Reuven,
>
> it's what I'm suspecting. git bisect should give us more information (still in
> progress, 3 more steps to complete ;)). I keep you posted.
>
> Regards
> JB
>
> On 02/06/2018 08:36 AM, Reuven Lax wrote:
>> Could this be related to any of the portability changes?
>>
>> On Mon, Feb 5, 2018 at 7:51 AM, Jean-Baptiste Onofré <jb@nanthrax.net
>> <ma...@nanthrax.net>> wrote:
>>
>> Created:
>>
>> https://issues.apache.org/jira/browse/BEAM-3617
>> <https://issues.apache.org/jira/browse/BEAM-3617>
>>
>> Regards
>> JB
>>
>> On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
>> > What is the Jira for direct runner perf?
>> >
>> > On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré <jb@nanthrax.net <ma...@nanthrax.net>
>> > <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
>> >
>> > Thanks !
>> >
>> > I cherry-pick on release-2.3.0 branch.
>> >
>> > I'm on the direct runner perf test in the mean time.
>> >
>> > Thanks again !
>> >
>> > Regards
>> > JB
>> >
>> > On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
>> > > I merged fixes for:
>> > > - https://issues.apache.org/jira/browse/BEAM-3186
>> <https://issues.apache.org/jira/browse/BEAM-3186>
>> > <https://issues.apache.org/jira/browse/BEAM-3186
>> <https://issues.apache.org/jira/browse/BEAM-3186>>
>> > > - https://issues.apache.org/jira/browse/BEAM-3589
>> <https://issues.apache.org/jira/browse/BEAM-3589>
>> > <https://issues.apache.org/jira/browse/BEAM-3589
>> <https://issues.apache.org/jira/browse/BEAM-3589>>
>> > >
>> > > @JB I didn't yet merge them on the 2.3.0 branch, though, but I can or you can go
>> > > ahead.
>> > >
>> > >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <jb@nanthrax.net <ma...@nanthrax.net> <mailto:jb@nanthrax.net
>> <ma...@nanthrax.net>>
>> > >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>
>> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>>> wrote:
>> > >>
>> > >> Hi guys,
>> > >>
>> > >> Quick update on the RC2 preparation:
>> > >>
>> > >> * BEAM-3587 (TextIO with Flink) seems related to a custom build with
>> > Gradle (not
>> > >> using artifacts created by Maven). Anyway, I will take a look today.
>> > >> * BEAM-3186 has a PR. Aljoscha will do the review pretty soon.
>> > >> * I'm also taking a look on the direct runner performance change.
>> > >>
>> > >> The goal is to cut RC2 tomorrow or the day after.
>> > >>
>> > >> Regards
>> > >> JB
>> > >>
>> > >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
>> > >>> Hi guys,
>> > >>>
>> > >>> Especially due to BEAM-3587 & BEAM-3186 regressions, I cancel RC1.
>> > >>>
>> > >>> We will cherry-pick fixes on release-2.3.0 branch.
>> > >>>
>> > >>> I'm updating Jira right now. When the fixes will be cherry-picked, I
>> > will submit
>> > >>> a RC2 to vote.
>> > >>>
>> > >>> Thanks !
>> > >>> Regards
>> > >>> JB
>> > >>>
>> > >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>> > >>>> Hi everyone,
>> > >>>>
>> > >>>> Please review and vote on the release candidate #1 for the version
>> > 2.3.0, as
>> > >>>> follows:
>> > >>>>
>> > >>>> [ ] +1, Approve the release
>> > >>>> [ ] -1, Do not approve the release (please provide specific comments)
>> > >>>>
>> > >>>>
>> > >>>> The complete staging area is available for your review, which
>> includes:
>> > >>>> * JIRA release notes [1],
>> > >>>> * the official Apache source release to be deployed to
>> dist.apache.org <http://dist.apache.org>
>> > <http://dist.apache.org>
>> > >>>> <http://dist.apache.org> [2],
>> > >>>> which is signed with the key with fingerprint C8282E76 [3],
>> > >>>> * all artifacts to be deployed to the Maven Central Repository [4],
>> > >>>> * source code tag "v2.3.0-RC1" [5],
>> > >>>> * website pull request listing the release and publishing the API
>> reference
>> > >>>> manual [6].
>> > >>>> * Java artifacts were built with Maven 3.3.9 and Oracle JDK
>> 1.8.0_111.
>> > >>>> * Python artifacts are deployed along with the source release to the
>> > >>>> dist.apache.org <http://dist.apache.org> <http://dist.apache.org>
>> <http://dist.apache.org> [2].
>> > >>>>
>> > >>>> The vote will be open for at least 72 hours. It is adopted by majority approval,
>> > >>>> with at least 3 PMC affirmative votes.
>> > >>>>
>> > >>>> Thanks,
>> > >>>> JB
>> > >>>>
>> > >>>> [1]
>> > >>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>
>> > <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>>
>> > >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
>> > <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>
>> > >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>> <https://dist.apache.org/repos/dist/release/beam/KEYS>
>> > <https://dist.apache.org/repos/dist/release/beam/KEYS
>> <https://dist.apache.org/repos/dist/release/beam/KEYS>>
>> > >>>> [4] https://repository.apache.org/content/repositories/orgapachebeam-1026/
>> <https://repository.apache.org/content/repositories/orgapachebeam-1026/>
>> > <https://repository.apache.org/content/repositories/orgapachebeam-1026/
>> <https://repository.apache.org/content/repositories/orgapachebeam-1026/>>
>> > >>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>> <https://github.com/apache/beam/tree/v2.3.0-RC1>
>> > <https://github.com/apache/beam/tree/v2.3.0-RC1
>> <https://github.com/apache/beam/tree/v2.3.0-RC1>>
>> > >>>> [6] https://github.com/apache/beam-site/pull/381
>> <https://github.com/apache/beam-site/pull/381>
>> > <https://github.com/apache/beam-site/pull/381
>> <https://github.com/apache/beam-site/pull/381>>
>> > >>>>
>> > >>>
>> > >>
>> > >> --
>> > >> Jean-Baptiste Onofré
>> > >> jbonofre@apache.org <ma...@apache.org> <mailto:jbonofre@apache.org
>> <ma...@apache.org>>
>> > <mailto:jbonofre@apache.org <ma...@apache.org>
>> <mailto:jbonofre@apache.org <ma...@apache.org>>>
>> > >> http://blog.nanthrax.net
>> > >> Talend - http://www.talend.com
>> > >
>> >
>> > --
>> > Jean-Baptiste Onofré
>> > jbonofre@apache.org <ma...@apache.org>
>> <mailto:jbonofre@apache.org <ma...@apache.org>>
>> > http://blog.nanthrax.net
>> > Talend - http://www.talend.com
>> >
>> >
>>
>> --
>> Jean-Baptiste Onofré
>> jbonofre@apache.org <ma...@apache.org>
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>
>>
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi Reuven,
it's what I'm suspecting. git bisect should give us more information (still in
progress, 3 more steps to complete ;)). I keep you posted.
Regards
JB
On 02/06/2018 08:36 AM, Reuven Lax wrote:
> Could this be related to any of the portability changes?
>
> On Mon, Feb 5, 2018 at 7:51 AM, Jean-Baptiste Onofré <jb@nanthrax.net
> <ma...@nanthrax.net>> wrote:
>
> Created:
>
> https://issues.apache.org/jira/browse/BEAM-3617
> <https://issues.apache.org/jira/browse/BEAM-3617>
>
> Regards
> JB
>
> On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
> > What is the Jira for direct runner perf?
> >
> > On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré <jb@nanthrax.net <ma...@nanthrax.net>
> > <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
> >
> > Thanks !
> >
> > I cherry-pick on release-2.3.0 branch.
> >
> > I'm on the direct runner perf test in the mean time.
> >
> > Thanks again !
> >
> > Regards
> > JB
> >
> > On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
> > > I merged fixes for:
> > > - https://issues.apache.org/jira/browse/BEAM-3186
> <https://issues.apache.org/jira/browse/BEAM-3186>
> > <https://issues.apache.org/jira/browse/BEAM-3186
> <https://issues.apache.org/jira/browse/BEAM-3186>>
> > > - https://issues.apache.org/jira/browse/BEAM-3589
> <https://issues.apache.org/jira/browse/BEAM-3589>
> > <https://issues.apache.org/jira/browse/BEAM-3589
> <https://issues.apache.org/jira/browse/BEAM-3589>>
> > >
> > > @JB I didn't yet merge them on the 2.3.0 branch, though, but I can or you can go
> > > ahead.
> > >
> > >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <jb@nanthrax.net <ma...@nanthrax.net> <mailto:jb@nanthrax.net
> <ma...@nanthrax.net>>
> > >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>
> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>>> wrote:
> > >>
> > >> Hi guys,
> > >>
> > >> Quick update on the RC2 preparation:
> > >>
> > >> * BEAM-3587 (TextIO with Flink) seems related to a custom build with
> > Gradle (not
> > >> using artifacts created by Maven). Anyway, I will take a look today.
> > >> * BEAM-3186 has a PR. Aljoscha will do the review pretty soon.
> > >> * I'm also taking a look on the direct runner performance change.
> > >>
> > >> The goal is to cut RC2 tomorrow or the day after.
> > >>
> > >> Regards
> > >> JB
> > >>
> > >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
> > >>> Hi guys,
> > >>>
> > >>> Especially due to BEAM-3587 & BEAM-3186 regressions, I cancel RC1.
> > >>>
> > >>> We will cherry-pick fixes on release-2.3.0 branch.
> > >>>
> > >>> I'm updating Jira right now. When the fixes will be cherry-picked, I
> > will submit
> > >>> a RC2 to vote.
> > >>>
> > >>> Thanks !
> > >>> Regards
> > >>> JB
> > >>>
> > >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
> > >>>> Hi everyone,
> > >>>>
> > >>>> Please review and vote on the release candidate #1 for the version
> > 2.3.0, as
> > >>>> follows:
> > >>>>
> > >>>> [ ] +1, Approve the release
> > >>>> [ ] -1, Do not approve the release (please provide specific comments)
> > >>>>
> > >>>>
> > >>>> The complete staging area is available for your review, which
> includes:
> > >>>> * JIRA release notes [1],
> > >>>> * the official Apache source release to be deployed to
> dist.apache.org <http://dist.apache.org>
> > <http://dist.apache.org>
> > >>>> <http://dist.apache.org> [2],
> > >>>> which is signed with the key with fingerprint C8282E76 [3],
> > >>>> * all artifacts to be deployed to the Maven Central Repository [4],
> > >>>> * source code tag "v2.3.0-RC1" [5],
> > >>>> * website pull request listing the release and publishing the API
> reference
> > >>>> manual [6].
> > >>>> * Java artifacts were built with Maven 3.3.9 and Oracle JDK
> 1.8.0_111.
> > >>>> * Python artifacts are deployed along with the source release to the
> > >>>> dist.apache.org <http://dist.apache.org> <http://dist.apache.org>
> <http://dist.apache.org> [2].
> > >>>>
> > >>>> The vote will be open for at least 72 hours. It is adopted by majority approval,
> > >>>> with at least 3 PMC affirmative votes.
> > >>>>
> > >>>> Thanks,
> > >>>> JB
> > >>>>
> > >>>> [1]
> > >>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>
> > <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>>
> > >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
> > <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>
> > >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> <https://dist.apache.org/repos/dist/release/beam/KEYS>
> > <https://dist.apache.org/repos/dist/release/beam/KEYS
> <https://dist.apache.org/repos/dist/release/beam/KEYS>>
> > >>>> [4] https://repository.apache.org/content/repositories/orgapachebeam-1026/
> <https://repository.apache.org/content/repositories/orgapachebeam-1026/>
> > <https://repository.apache.org/content/repositories/orgapachebeam-1026/
> <https://repository.apache.org/content/repositories/orgapachebeam-1026/>>
> > >>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
> <https://github.com/apache/beam/tree/v2.3.0-RC1>
> > <https://github.com/apache/beam/tree/v2.3.0-RC1
> <https://github.com/apache/beam/tree/v2.3.0-RC1>>
> > >>>> [6] https://github.com/apache/beam-site/pull/381
> <https://github.com/apache/beam-site/pull/381>
> > <https://github.com/apache/beam-site/pull/381
> <https://github.com/apache/beam-site/pull/381>>
> > >>>>
> > >>>
> > >>
> > >> --
> > >> Jean-Baptiste Onofré
> > >> jbonofre@apache.org <ma...@apache.org> <mailto:jbonofre@apache.org
> <ma...@apache.org>>
> > <mailto:jbonofre@apache.org <ma...@apache.org>
> <mailto:jbonofre@apache.org <ma...@apache.org>>>
> > >> http://blog.nanthrax.net
> > >> Talend - http://www.talend.com
> > >
> >
> > --
> > Jean-Baptiste Onofré
> > jbonofre@apache.org <ma...@apache.org>
> <mailto:jbonofre@apache.org <ma...@apache.org>>
> > http://blog.nanthrax.net
> > Talend - http://www.talend.com
> >
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org <ma...@apache.org>
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Reuven Lax <re...@google.com>.
Could this be related to any of the portability changes?
On Mon, Feb 5, 2018 at 7:51 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> Created:
>
> https://issues.apache.org/jira/browse/BEAM-3617
>
> Regards
> JB
>
> On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
> > What is the Jira for direct runner perf?
> >
> > On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré <jb@nanthrax.net
> > <ma...@nanthrax.net>> wrote:
> >
> > Thanks !
> >
> > I cherry-pick on release-2.3.0 branch.
> >
> > I'm on the direct runner perf test in the mean time.
> >
> > Thanks again !
> >
> > Regards
> > JB
> >
> > On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
> > > I merged fixes for:
> > > - https://issues.apache.org/jira/browse/BEAM-3186
> > <https://issues.apache.org/jira/browse/BEAM-3186>
> > > - https://issues.apache.org/jira/browse/BEAM-3589
> > <https://issues.apache.org/jira/browse/BEAM-3589>
> > >
> > > @JB I didn't yet merge them on the 2.3.0 branch, though, but I can
> or you can go
> > > ahead.
> > >
> > >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <jb@nanthrax.net
> <ma...@nanthrax.net>
> > >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
> > >>
> > >> Hi guys,
> > >>
> > >> Quick update on the RC2 preparation:
> > >>
> > >> * BEAM-3587 (TextIO with Flink) seems related to a custom build
> with
> > Gradle (not
> > >> using artifacts created by Maven). Anyway, I will take a look
> today.
> > >> * BEAM-3186 has a PR. Aljoscha will do the review pretty soon.
> > >> * I'm also taking a look on the direct runner performance change.
> > >>
> > >> The goal is to cut RC2 tomorrow or the day after.
> > >>
> > >> Regards
> > >> JB
> > >>
> > >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
> > >>> Hi guys,
> > >>>
> > >>> Especially due to BEAM-3587 & BEAM-3186 regressions, I cancel
> RC1.
> > >>>
> > >>> We will cherry-pick fixes on release-2.3.0 branch.
> > >>>
> > >>> I'm updating Jira right now. When the fixes will be
> cherry-picked, I
> > will submit
> > >>> a RC2 to vote.
> > >>>
> > >>> Thanks !
> > >>> Regards
> > >>> JB
> > >>>
> > >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
> > >>>> Hi everyone,
> > >>>>
> > >>>> Please review and vote on the release candidate #1 for the
> version
> > 2.3.0, as
> > >>>> follows:
> > >>>>
> > >>>> [ ] +1, Approve the release
> > >>>> [ ] -1, Do not approve the release (please provide specific
> comments)
> > >>>>
> > >>>>
> > >>>> The complete staging area is available for your review, which
> includes:
> > >>>> * JIRA release notes [1],
> > >>>> * the official Apache source release to be deployed to
> dist.apache.org
> > <http://dist.apache.org>
> > >>>> <http://dist.apache.org> [2],
> > >>>> which is signed with the key with fingerprint C8282E76 [3],
> > >>>> * all artifacts to be deployed to the Maven Central Repository
> [4],
> > >>>> * source code tag "v2.3.0-RC1" [5],
> > >>>> * website pull request listing the release and publishing the
> API reference
> > >>>> manual [6].
> > >>>> * Java artifacts were built with Maven 3.3.9 and Oracle JDK
> 1.8.0_111.
> > >>>> * Python artifacts are deployed along with the source release
> to the
> > >>>> dist.apache.org <http://dist.apache.org> <
> http://dist.apache.org> [2].
> > >>>>
> > >>>> The vote will be open for at least 72 hours. It is adopted by
> majority approval,
> > >>>> with at least 3 PMC affirmative votes.
> > >>>>
> > >>>> Thanks,
> > >>>> JB
> > >>>>
> > >>>> [1]
> > >>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12319527&version=12341608
> > <https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12319527&version=12341608>
> > >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> > <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
> > >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> > <https://dist.apache.org/repos/dist/release/beam/KEYS>
> > >>>> [4] https://repository.apache.org/content/repositories/
> orgapachebeam-1026/
> > <https://repository.apache.org/content/repositories/
> orgapachebeam-1026/>
> > >>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
> > <https://github.com/apache/beam/tree/v2.3.0-RC1>
> > >>>> [6] https://github.com/apache/beam-site/pull/381
> > <https://github.com/apache/beam-site/pull/381>
> > >>>>
> > >>>
> > >>
> > >> --
> > >> Jean-Baptiste Onofré
> > >> jbonofre@apache.org <ma...@apache.org>
> > <mailto:jbonofre@apache.org <ma...@apache.org>>
> > >> http://blog.nanthrax.net
> > >> Talend - http://www.talend.com
> > >
> >
> > --
> > Jean-Baptiste Onofré
> > jbonofre@apache.org <ma...@apache.org>
> > http://blog.nanthrax.net
> > Talend - http://www.talend.com
> >
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi,
quick update about RC2: BEAM-3617 is the only Jira pending for the release.
I'm doing a git bisect to identify the commit who caused the performance
degradation.
Depending of the result, if it's a easy fix than we will try to do it for RC2,
else I will start the RC2 as now.
I will keep you posted.
Regards
JB
On 02/05/2018 04:51 PM, Jean-Baptiste Onofré wrote:
> Created:
>
> https://issues.apache.org/jira/browse/BEAM-3617
>
> Regards
> JB
>
> On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
>> What is the Jira for direct runner perf?
>>
>> On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré <jb@nanthrax.net
>> <ma...@nanthrax.net>> wrote:
>>
>> Thanks !
>>
>> I cherry-pick on release-2.3.0 branch.
>>
>> I'm on the direct runner perf test in the mean time.
>>
>> Thanks again !
>>
>> Regards
>> JB
>>
>> On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
>> > I merged fixes for:
>> > - https://issues.apache.org/jira/browse/BEAM-3186
>> <https://issues.apache.org/jira/browse/BEAM-3186>
>> > - https://issues.apache.org/jira/browse/BEAM-3589
>> <https://issues.apache.org/jira/browse/BEAM-3589>
>> >
>> > @JB I didn't yet merge them on the 2.3.0 branch, though, but I can or you can go
>> > ahead.
>> >
>> >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <jb@nanthrax.net <ma...@nanthrax.net>
>> >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
>> >>
>> >> Hi guys,
>> >>
>> >> Quick update on the RC2 preparation:
>> >>
>> >> * BEAM-3587 (TextIO with Flink) seems related to a custom build with
>> Gradle (not
>> >> using artifacts created by Maven). Anyway, I will take a look today.
>> >> * BEAM-3186 has a PR. Aljoscha will do the review pretty soon.
>> >> * I'm also taking a look on the direct runner performance change.
>> >>
>> >> The goal is to cut RC2 tomorrow or the day after.
>> >>
>> >> Regards
>> >> JB
>> >>
>> >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
>> >>> Hi guys,
>> >>>
>> >>> Especially due to BEAM-3587 & BEAM-3186 regressions, I cancel RC1.
>> >>>
>> >>> We will cherry-pick fixes on release-2.3.0 branch.
>> >>>
>> >>> I'm updating Jira right now. When the fixes will be cherry-picked, I
>> will submit
>> >>> a RC2 to vote.
>> >>>
>> >>> Thanks !
>> >>> Regards
>> >>> JB
>> >>>
>> >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>> >>>> Hi everyone,
>> >>>>
>> >>>> Please review and vote on the release candidate #1 for the version
>> 2.3.0, as
>> >>>> follows:
>> >>>>
>> >>>> [ ] +1, Approve the release
>> >>>> [ ] -1, Do not approve the release (please provide specific comments)
>> >>>>
>> >>>>
>> >>>> The complete staging area is available for your review, which includes:
>> >>>> * JIRA release notes [1],
>> >>>> * the official Apache source release to be deployed to dist.apache.org
>> <http://dist.apache.org>
>> >>>> <http://dist.apache.org> [2],
>> >>>> which is signed with the key with fingerprint C8282E76 [3],
>> >>>> * all artifacts to be deployed to the Maven Central Repository [4],
>> >>>> * source code tag "v2.3.0-RC1" [5],
>> >>>> * website pull request listing the release and publishing the API reference
>> >>>> manual [6].
>> >>>> * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
>> >>>> * Python artifacts are deployed along with the source release to the
>> >>>> dist.apache.org <http://dist.apache.org> <http://dist.apache.org> [2].
>> >>>>
>> >>>> The vote will be open for at least 72 hours. It is adopted by majority approval,
>> >>>> with at least 3 PMC affirmative votes.
>> >>>>
>> >>>> Thanks,
>> >>>> JB
>> >>>>
>> >>>> [1]
>> >>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>
>> >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
>> >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>> <https://dist.apache.org/repos/dist/release/beam/KEYS>
>> >>>> [4] https://repository.apache.org/content/repositories/orgapachebeam-1026/
>> <https://repository.apache.org/content/repositories/orgapachebeam-1026/>
>> >>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>> <https://github.com/apache/beam/tree/v2.3.0-RC1>
>> >>>> [6] https://github.com/apache/beam-site/pull/381
>> <https://github.com/apache/beam-site/pull/381>
>> >>>>
>> >>>
>> >>
>> >> --
>> >> Jean-Baptiste Onofré
>> >> jbonofre@apache.org <ma...@apache.org>
>> <mailto:jbonofre@apache.org <ma...@apache.org>>
>> >> http://blog.nanthrax.net
>> >> Talend - http://www.talend.com
>> >
>>
>> --
>> Jean-Baptiste Onofré
>> jbonofre@apache.org <ma...@apache.org>
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>
>>
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Created:
https://issues.apache.org/jira/browse/BEAM-3617
Regards
JB
On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
> What is the Jira for direct runner perf?
>
> On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré <jb@nanthrax.net
> <ma...@nanthrax.net>> wrote:
>
> Thanks !
>
> I cherry-pick on release-2.3.0 branch.
>
> I'm on the direct runner perf test in the mean time.
>
> Thanks again !
>
> Regards
> JB
>
> On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
> > I merged fixes for:
> > - https://issues.apache.org/jira/browse/BEAM-3186
> <https://issues.apache.org/jira/browse/BEAM-3186>
> > - https://issues.apache.org/jira/browse/BEAM-3589
> <https://issues.apache.org/jira/browse/BEAM-3589>
> >
> > @JB I didn't yet merge them on the 2.3.0 branch, though, but I can or you can go
> > ahead.
> >
> >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <jb@nanthrax.net <ma...@nanthrax.net>
> >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
> >>
> >> Hi guys,
> >>
> >> Quick update on the RC2 preparation:
> >>
> >> * BEAM-3587 (TextIO with Flink) seems related to a custom build with
> Gradle (not
> >> using artifacts created by Maven). Anyway, I will take a look today.
> >> * BEAM-3186 has a PR. Aljoscha will do the review pretty soon.
> >> * I'm also taking a look on the direct runner performance change.
> >>
> >> The goal is to cut RC2 tomorrow or the day after.
> >>
> >> Regards
> >> JB
> >>
> >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
> >>> Hi guys,
> >>>
> >>> Especially due to BEAM-3587 & BEAM-3186 regressions, I cancel RC1.
> >>>
> >>> We will cherry-pick fixes on release-2.3.0 branch.
> >>>
> >>> I'm updating Jira right now. When the fixes will be cherry-picked, I
> will submit
> >>> a RC2 to vote.
> >>>
> >>> Thanks !
> >>> Regards
> >>> JB
> >>>
> >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
> >>>> Hi everyone,
> >>>>
> >>>> Please review and vote on the release candidate #1 for the version
> 2.3.0, as
> >>>> follows:
> >>>>
> >>>> [ ] +1, Approve the release
> >>>> [ ] -1, Do not approve the release (please provide specific comments)
> >>>>
> >>>>
> >>>> The complete staging area is available for your review, which includes:
> >>>> * JIRA release notes [1],
> >>>> * the official Apache source release to be deployed to dist.apache.org
> <http://dist.apache.org>
> >>>> <http://dist.apache.org> [2],
> >>>> which is signed with the key with fingerprint C8282E76 [3],
> >>>> * all artifacts to be deployed to the Maven Central Repository [4],
> >>>> * source code tag "v2.3.0-RC1" [5],
> >>>> * website pull request listing the release and publishing the API reference
> >>>> manual [6].
> >>>> * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
> >>>> * Python artifacts are deployed along with the source release to the
> >>>> dist.apache.org <http://dist.apache.org> <http://dist.apache.org> [2].
> >>>>
> >>>> The vote will be open for at least 72 hours. It is adopted by majority approval,
> >>>> with at least 3 PMC affirmative votes.
> >>>>
> >>>> Thanks,
> >>>> JB
> >>>>
> >>>> [1]
> >>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>
> >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
> >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> <https://dist.apache.org/repos/dist/release/beam/KEYS>
> >>>> [4] https://repository.apache.org/content/repositories/orgapachebeam-1026/
> <https://repository.apache.org/content/repositories/orgapachebeam-1026/>
> >>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
> <https://github.com/apache/beam/tree/v2.3.0-RC1>
> >>>> [6] https://github.com/apache/beam-site/pull/381
> <https://github.com/apache/beam-site/pull/381>
> >>>>
> >>>
> >>
> >> --
> >> Jean-Baptiste Onofré
> >> jbonofre@apache.org <ma...@apache.org>
> <mailto:jbonofre@apache.org <ma...@apache.org>>
> >> http://blog.nanthrax.net
> >> Talend - http://www.talend.com
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org <ma...@apache.org>
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi Kenn,
my bad, I didn't create one yet (I was busy on the TextIO with flink runner, now
identify \o/ ;)).
Let me create it right now.
Thanks !
Regards
JB
On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
> What is the Jira for direct runner perf?
>
> On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré <jb@nanthrax.net
> <ma...@nanthrax.net>> wrote:
>
> Thanks !
>
> I cherry-pick on release-2.3.0 branch.
>
> I'm on the direct runner perf test in the mean time.
>
> Thanks again !
>
> Regards
> JB
>
> On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
> > I merged fixes for:
> > - https://issues.apache.org/jira/browse/BEAM-3186
> <https://issues.apache.org/jira/browse/BEAM-3186>
> > - https://issues.apache.org/jira/browse/BEAM-3589
> <https://issues.apache.org/jira/browse/BEAM-3589>
> >
> > @JB I didn't yet merge them on the 2.3.0 branch, though, but I can or you can go
> > ahead.
> >
> >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <jb@nanthrax.net <ma...@nanthrax.net>
> >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
> >>
> >> Hi guys,
> >>
> >> Quick update on the RC2 preparation:
> >>
> >> * BEAM-3587 (TextIO with Flink) seems related to a custom build with
> Gradle (not
> >> using artifacts created by Maven). Anyway, I will take a look today.
> >> * BEAM-3186 has a PR. Aljoscha will do the review pretty soon.
> >> * I'm also taking a look on the direct runner performance change.
> >>
> >> The goal is to cut RC2 tomorrow or the day after.
> >>
> >> Regards
> >> JB
> >>
> >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
> >>> Hi guys,
> >>>
> >>> Especially due to BEAM-3587 & BEAM-3186 regressions, I cancel RC1.
> >>>
> >>> We will cherry-pick fixes on release-2.3.0 branch.
> >>>
> >>> I'm updating Jira right now. When the fixes will be cherry-picked, I
> will submit
> >>> a RC2 to vote.
> >>>
> >>> Thanks !
> >>> Regards
> >>> JB
> >>>
> >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
> >>>> Hi everyone,
> >>>>
> >>>> Please review and vote on the release candidate #1 for the version
> 2.3.0, as
> >>>> follows:
> >>>>
> >>>> [ ] +1, Approve the release
> >>>> [ ] -1, Do not approve the release (please provide specific comments)
> >>>>
> >>>>
> >>>> The complete staging area is available for your review, which includes:
> >>>> * JIRA release notes [1],
> >>>> * the official Apache source release to be deployed to dist.apache.org
> <http://dist.apache.org>
> >>>> <http://dist.apache.org> [2],
> >>>> which is signed with the key with fingerprint C8282E76 [3],
> >>>> * all artifacts to be deployed to the Maven Central Repository [4],
> >>>> * source code tag "v2.3.0-RC1" [5],
> >>>> * website pull request listing the release and publishing the API reference
> >>>> manual [6].
> >>>> * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
> >>>> * Python artifacts are deployed along with the source release to the
> >>>> dist.apache.org <http://dist.apache.org> <http://dist.apache.org> [2].
> >>>>
> >>>> The vote will be open for at least 72 hours. It is adopted by majority approval,
> >>>> with at least 3 PMC affirmative votes.
> >>>>
> >>>> Thanks,
> >>>> JB
> >>>>
> >>>> [1]
> >>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>
> >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
> >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> <https://dist.apache.org/repos/dist/release/beam/KEYS>
> >>>> [4] https://repository.apache.org/content/repositories/orgapachebeam-1026/
> <https://repository.apache.org/content/repositories/orgapachebeam-1026/>
> >>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
> <https://github.com/apache/beam/tree/v2.3.0-RC1>
> >>>> [6] https://github.com/apache/beam-site/pull/381
> <https://github.com/apache/beam-site/pull/381>
> >>>>
> >>>
> >>
> >> --
> >> Jean-Baptiste Onofré
> >> jbonofre@apache.org <ma...@apache.org>
> <mailto:jbonofre@apache.org <ma...@apache.org>>
> >> http://blog.nanthrax.net
> >> Talend - http://www.talend.com
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org <ma...@apache.org>
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Kenneth Knowles <kl...@google.com>.
What is the Jira for direct runner perf?
On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> Thanks !
>
> I cherry-pick on release-2.3.0 branch.
>
> I'm on the direct runner perf test in the mean time.
>
> Thanks again !
>
> Regards
> JB
>
> On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
> > I merged fixes for:
> > - https://issues.apache.org/jira/browse/BEAM-3186
> > - https://issues.apache.org/jira/browse/BEAM-3589
> >
> > @JB I didn't yet merge them on the 2.3.0 branch, though, but I can or
> you can go
> > ahead.
> >
> >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <jb@nanthrax.net
> >> <ma...@nanthrax.net>> wrote:
> >>
> >> Hi guys,
> >>
> >> Quick update on the RC2 preparation:
> >>
> >> * BEAM-3587 (TextIO with Flink) seems related to a custom build with
> Gradle (not
> >> using artifacts created by Maven). Anyway, I will take a look today.
> >> * BEAM-3186 has a PR. Aljoscha will do the review pretty soon.
> >> * I'm also taking a look on the direct runner performance change.
> >>
> >> The goal is to cut RC2 tomorrow or the day after.
> >>
> >> Regards
> >> JB
> >>
> >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
> >>> Hi guys,
> >>>
> >>> Especially due to BEAM-3587 & BEAM-3186 regressions, I cancel RC1.
> >>>
> >>> We will cherry-pick fixes on release-2.3.0 branch.
> >>>
> >>> I'm updating Jira right now. When the fixes will be cherry-picked, I
> will submit
> >>> a RC2 to vote.
> >>>
> >>> Thanks !
> >>> Regards
> >>> JB
> >>>
> >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
> >>>> Hi everyone,
> >>>>
> >>>> Please review and vote on the release candidate #1 for the version
> 2.3.0, as
> >>>> follows:
> >>>>
> >>>> [ ] +1, Approve the release
> >>>> [ ] -1, Do not approve the release (please provide specific comments)
> >>>>
> >>>>
> >>>> The complete staging area is available for your review, which
> includes:
> >>>> * JIRA release notes [1],
> >>>> * the official Apache source release to be deployed to
> dist.apache.org
> >>>> <http://dist.apache.org> [2],
> >>>> which is signed with the key with fingerprint C8282E76 [3],
> >>>> * all artifacts to be deployed to the Maven Central Repository [4],
> >>>> * source code tag "v2.3.0-RC1" [5],
> >>>> * website pull request listing the release and publishing the API
> reference
> >>>> manual [6].
> >>>> * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
> >>>> * Python artifacts are deployed along with the source release to the
> >>>> dist.apache.org <http://dist.apache.org> [2].
> >>>>
> >>>> The vote will be open for at least 72 hours. It is adopted by
> majority approval,
> >>>> with at least 3 PMC affirmative votes.
> >>>>
> >>>> Thanks,
> >>>> JB
> >>>>
> >>>> [1]
> >>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12319527&version=12341608
> >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> >>>> [4] https://repository.apache.org/content/repositories/
> orgapachebeam-1026/
> >>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
> >>>> [6] https://github.com/apache/beam-site/pull/381
> >>>>
> >>>
> >>
> >> --
> >> Jean-Baptiste Onofré
> >> jbonofre@apache.org <ma...@apache.org>
> >> http://blog.nanthrax.net
> >> Talend - http://www.talend.com
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Thanks !
I cherry-pick on release-2.3.0 branch.
I'm on the direct runner perf test in the mean time.
Thanks again !
Regards
JB
On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
> I merged fixes for:
> - https://issues.apache.org/jira/browse/BEAM-3186
> - https://issues.apache.org/jira/browse/BEAM-3589
>
> @JB I didn't yet merge them on the 2.3.0 branch, though, but I can or you can go
> ahead.
>
>> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <jb@nanthrax.net
>> <ma...@nanthrax.net>> wrote:
>>
>> Hi guys,
>>
>> Quick update on the RC2 preparation:
>>
>> * BEAM-3587 (TextIO with Flink) seems related to a custom build with Gradle (not
>> using artifacts created by Maven). Anyway, I will take a look today.
>> * BEAM-3186 has a PR. Aljoscha will do the review pretty soon.
>> * I'm also taking a look on the direct runner performance change.
>>
>> The goal is to cut RC2 tomorrow or the day after.
>>
>> Regards
>> JB
>>
>> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
>>> Hi guys,
>>>
>>> Especially due to BEAM-3587 & BEAM-3186 regressions, I cancel RC1.
>>>
>>> We will cherry-pick fixes on release-2.3.0 branch.
>>>
>>> I'm updating Jira right now. When the fixes will be cherry-picked, I will submit
>>> a RC2 to vote.
>>>
>>> Thanks !
>>> Regards
>>> JB
>>>
>>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>>>> Hi everyone,
>>>>
>>>> Please review and vote on the release candidate #1 for the version 2.3.0, as
>>>> follows:
>>>>
>>>> [ ] +1, Approve the release
>>>> [ ] -1, Do not approve the release (please provide specific comments)
>>>>
>>>>
>>>> The complete staging area is available for your review, which includes:
>>>> * JIRA release notes [1],
>>>> * the official Apache source release to be deployed to dist.apache.org
>>>> <http://dist.apache.org> [2],
>>>> which is signed with the key with fingerprint C8282E76 [3],
>>>> * all artifacts to be deployed to the Maven Central Repository [4],
>>>> * source code tag "v2.3.0-RC1" [5],
>>>> * website pull request listing the release and publishing the API reference
>>>> manual [6].
>>>> * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
>>>> * Python artifacts are deployed along with the source release to the
>>>> dist.apache.org <http://dist.apache.org> [2].
>>>>
>>>> The vote will be open for at least 72 hours. It is adopted by majority approval,
>>>> with at least 3 PMC affirmative votes.
>>>>
>>>> Thanks,
>>>> JB
>>>>
>>>> [1]
>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>>> [4] https://repository.apache.org/content/repositories/orgapachebeam-1026/
>>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>>>> [6] https://github.com/apache/beam-site/pull/381
>>>>
>>>
>>
>> --
>> Jean-Baptiste Onofré
>> jbonofre@apache.org <ma...@apache.org>
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Aljoscha Krettek <al...@apache.org>.
I merged fixes for:
- https://issues.apache.org/jira/browse/BEAM-3186 <https://issues.apache.org/jira/browse/BEAM-3186>
- https://issues.apache.org/jira/browse/BEAM-3589 <https://issues.apache.org/jira/browse/BEAM-3589>
@JB I didn't yet merge them on the 2.3.0 branch, though, but I can or you can go ahead.
> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
>
> Hi guys,
>
> Quick update on the RC2 preparation:
>
> * BEAM-3587 (TextIO with Flink) seems related to a custom build with Gradle (not
> using artifacts created by Maven). Anyway, I will take a look today.
> * BEAM-3186 has a PR. Aljoscha will do the review pretty soon.
> * I'm also taking a look on the direct runner performance change.
>
> The goal is to cut RC2 tomorrow or the day after.
>
> Regards
> JB
>
> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
>> Hi guys,
>>
>> Especially due to BEAM-3587 & BEAM-3186 regressions, I cancel RC1.
>>
>> We will cherry-pick fixes on release-2.3.0 branch.
>>
>> I'm updating Jira right now. When the fixes will be cherry-picked, I will submit
>> a RC2 to vote.
>>
>> Thanks !
>> Regards
>> JB
>>
>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>>> Hi everyone,
>>>
>>> Please review and vote on the release candidate #1 for the version 2.3.0, as
>>> follows:
>>>
>>> [ ] +1, Approve the release
>>> [ ] -1, Do not approve the release (please provide specific comments)
>>>
>>>
>>> The complete staging area is available for your review, which includes:
>>> * JIRA release notes [1],
>>> * the official Apache source release to be deployed to dist.apache.org [2],
>>> which is signed with the key with fingerprint C8282E76 [3],
>>> * all artifacts to be deployed to the Maven Central Repository [4],
>>> * source code tag "v2.3.0-RC1" [5],
>>> * website pull request listing the release and publishing the API reference
>>> manual [6].
>>> * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
>>> * Python artifacts are deployed along with the source release to the
>>> dist.apache.org [2].
>>>
>>> The vote will be open for at least 72 hours. It is adopted by majority approval,
>>> with at least 3 PMC affirmative votes.
>>>
>>> Thanks,
>>> JB
>>>
>>> [1]
>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>> [4] https://repository.apache.org/content/repositories/orgapachebeam-1026/
>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>>> [6] https://github.com/apache/beam-site/pull/381
>>>
>>
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi guys,
Quick update on the RC2 preparation:
* BEAM-3587 (TextIO with Flink) seems related to a custom build with Gradle (not
using artifacts created by Maven). Anyway, I will take a look today.
* BEAM-3186 has a PR. Aljoscha will do the review pretty soon.
* I'm also taking a look on the direct runner performance change.
The goal is to cut RC2 tomorrow or the day after.
Regards
JB
On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
> Hi guys,
>
> Especially due to BEAM-3587 & BEAM-3186 regressions, I cancel RC1.
>
> We will cherry-pick fixes on release-2.3.0 branch.
>
> I'm updating Jira right now. When the fixes will be cherry-picked, I will submit
> a RC2 to vote.
>
> Thanks !
> Regards
> JB
>
> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>> Hi everyone,
>>
>> Please review and vote on the release candidate #1 for the version 2.3.0, as
>> follows:
>>
>> [ ] +1, Approve the release
>> [ ] -1, Do not approve the release (please provide specific comments)
>>
>>
>> The complete staging area is available for your review, which includes:
>> * JIRA release notes [1],
>> * the official Apache source release to be deployed to dist.apache.org [2],
>> which is signed with the key with fingerprint C8282E76 [3],
>> * all artifacts to be deployed to the Maven Central Repository [4],
>> * source code tag "v2.3.0-RC1" [5],
>> * website pull request listing the release and publishing the API reference
>> manual [6].
>> * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
>> * Python artifacts are deployed along with the source release to the
>> dist.apache.org [2].
>>
>> The vote will be open for at least 72 hours. It is adopted by majority approval,
>> with at least 3 PMC affirmative votes.
>>
>> Thanks,
>> JB
>>
>> [1]
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>> [4] https://repository.apache.org/content/repositories/orgapachebeam-1026/
>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>> [6] https://github.com/apache/beam-site/pull/381
>>
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
[CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi guys,
Especially due to BEAM-3587 & BEAM-3186 regressions, I cancel RC1.
We will cherry-pick fixes on release-2.3.0 branch.
I'm updating Jira right now. When the fixes will be cherry-picked, I will submit
a RC2 to vote.
Thanks !
Regards
JB
On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
> Hi everyone,
>
> Please review and vote on the release candidate #1 for the version 2.3.0, as
> follows:
>
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
>
> The complete staging area is available for your review, which includes:
> * JIRA release notes [1],
> * the official Apache source release to be deployed to dist.apache.org [2],
> which is signed with the key with fingerprint C8282E76 [3],
> * all artifacts to be deployed to the Maven Central Repository [4],
> * source code tag "v2.3.0-RC1" [5],
> * website pull request listing the release and publishing the API reference
> manual [6].
> * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
> * Python artifacts are deployed along with the source release to the
> dist.apache.org [2].
>
> The vote will be open for at least 72 hours. It is adopted by majority approval,
> with at least 3 PMC affirmative votes.
>
> Thanks,
> JB
>
> [1]
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> [4] https://repository.apache.org/content/repositories/orgapachebeam-1026/
> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
> [6] https://github.com/apache/beam-site/pull/381
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: [VOTE] Release 2.3.0, release candidate #1
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Fully agree. Good catch.
I cancel RC1 to prepare a RC2 including at least this fix.
Thanks !
Regards
JB
On 02/01/2018 04:11 PM, Aljoscha Krettek wrote:
> -1
>
> I think the issue discovered with unbounded sources on Flink Streaming Runner is a serious regression. Good news is that there is already a fix for that: https://github.com/apache/beam/pull/4558/files
>
> And BEAM-3587 also seems serious enough, IMHO.
>
> Btw, BEAM-3186, which seems quite serious, was also finally figured out.
>
> Best,
> Aljoscha
>
>> On 1. Feb 2018, at 16:00, Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
>>
>> So, are you casting -1 vote ?
>>
>> I guess so.
>>
>> Regards
>> JB
>>
>> On 02/01/2018 03:53 PM, Ismaël Mejía wrote:
>>> Hi,
>>>
>>> I started to test the release with Nexmark and found three issues
>>> (from minor to more important):
>>>
>>> 1. Small issues to run Nexmark with the release (BEAM-3531 fixed,
>>> BEAM-3592 in PR):
>>>
>>> BEAM-3531 Nexmark failed with NPE with DEFAULT suite
>>> BEAM-3592 Spark-runner profile is broken on Nexmark after move to Spark 2.x
>>>
>>> 2. Flink is broken with Unbounded sources, big enough to deserve a new
>>> vote. Pred:
>>> BEAM-3589 Flink runner breaks with ClassCastException on UnboundedSource
>>>
>>> 3. Direct runner has a relatively big performance regression when
>>> dealing with UnboundedSources. in particular the impact on query 7 of
>>> Nexmark is considerable.
>>>
>>> Just with the small SMOKE suite in my machine I get:
>>>
>>> ========================================
>>> Beam 2.2.0 Beam 2.3.0
>>> Query Runtime(sec) Runtime(sec)
>>> ========================================
>>> 0000 6.4 10.6
>>> 0001 5.1 10.2
>>> 0002 3.0 5.8
>>> 0003 3.8 6.2
>>> 0004 0.9 1.4
>>> 0005 5.8 11.4
>>> 0006 0.8 1.4
>>> 0007 193.8 1249.1
>>> 0008 3.9 6.9
>>> 0009 0.9 1.3
>>> 0010 6.4 8.2
>>> 0011 5.0 9.4
>>> 0012 4.7 9.1
>>>
>>> This can be reproduced by running this command:
>>>
>>> mvn exec:java -Dexec.mainClass=org.apache.beam.sdk.nexmark.Main
>>> -Pdirect-runner -Dexec.args="--runner=DirectRunner --suite=SMOKE
>>> --streaming=true --manageResources=false --monitorJobs=true
>>> --enforceEncodability=true --enforceImmutability=true" -pl
>>> 'sdks/java/nexmark'
>>>
>>> I think 2 and 3 deserve to be fixed or at least evaluated as important
>>> enough to cancel the vote. And if possible I would love to cherry-pick
>>> the Nexmark fixes for a future RC.
>>>
>>> On Thu, Feb 1, 2018 at 7:20 AM, Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
>>>> Hi all,
>>>>
>>>> just a quick reminder about the vote process:
>>>>
>>>> 1. Any vote can be changed during the vote period. A -1 vote has to be argued
>>>> (especially if there's not change to do in the project codebase).
>>>> 2. For convenience to the release manager, please inform if your vote is binding
>>>> or non-binding (the vote from PMC members are binding)
>>>> 3. It's not possible to "veto" a release: if we have at least 3 binding votes,
>>>> the vote can pass.
>>>> 4. Please, keep only vote in the thread. If you have some tests in progress,
>>>> please use another thread. It would be great if the thread only contains
>>>> concrete votes.
>>>> 5. The vote duration can be extended on request.
>>>>
>>>> So, I'm extending this vote to 72 more hours to give us time to review
>>>> especially the dataflow worker images test and the Flink TextIO potential issue.
>>>>
>>>> Thanks !
>>>> Regards
>>>> JB
>>>>
>>>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>>>>> Hi everyone,
>>>>>
>>>>> Please review and vote on the release candidate #1 for the version 2.3.0, as
>>>>> follows:
>>>>>
>>>>> [ ] +1, Approve the release
>>>>> [ ] -1, Do not approve the release (please provide specific comments)
>>>>>
>>>>>
>>>>> The complete staging area is available for your review, which includes:
>>>>> * JIRA release notes [1],
>>>>> * the official Apache source release to be deployed to dist.apache.org [2],
>>>>> which is signed with the key with fingerprint C8282E76 [3],
>>>>> * all artifacts to be deployed to the Maven Central Repository [4],
>>>>> * source code tag "v2.3.0-RC1" [5],
>>>>> * website pull request listing the release and publishing the API reference
>>>>> manual [6].
>>>>> * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
>>>>> * Python artifacts are deployed along with the source release to the
>>>>> dist.apache.org [2].
>>>>>
>>>>> The vote will be open for at least 72 hours. It is adopted by majority approval,
>>>>> with at least 3 PMC affirmative votes.
>>>>>
>>>>> Thanks,
>>>>> JB
>>>>>
>>>>> [1]
>>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>>>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>>>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>>>> [4] https://repository.apache.org/content/repositories/orgapachebeam-1026/
>>>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>>>>> [6] https://github.com/apache/beam-site/pull/381
>>>>>
>>>>
>>>> --
>>>> Jean-Baptiste Onofré
>>>> jbonofre@apache.org
>>>> http://blog.nanthrax.net
>>>> Talend - http://www.talend.com
>>
>> --
>> Jean-Baptiste Onofré
>> jbonofre@apache.org
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: [VOTE] Release 2.3.0, release candidate #1
Posted by Aljoscha Krettek <al...@apache.org>.
-1
I think the issue discovered with unbounded sources on Flink Streaming Runner is a serious regression. Good news is that there is already a fix for that: https://github.com/apache/beam/pull/4558/files
And BEAM-3587 also seems serious enough, IMHO.
Btw, BEAM-3186, which seems quite serious, was also finally figured out.
Best,
Aljoscha
> On 1. Feb 2018, at 16:00, Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
>
> So, are you casting -1 vote ?
>
> I guess so.
>
> Regards
> JB
>
> On 02/01/2018 03:53 PM, Ismaël Mejía wrote:
>> Hi,
>>
>> I started to test the release with Nexmark and found three issues
>> (from minor to more important):
>>
>> 1. Small issues to run Nexmark with the release (BEAM-3531 fixed,
>> BEAM-3592 in PR):
>>
>> BEAM-3531 Nexmark failed with NPE with DEFAULT suite
>> BEAM-3592 Spark-runner profile is broken on Nexmark after move to Spark 2.x
>>
>> 2. Flink is broken with Unbounded sources, big enough to deserve a new
>> vote. Pred:
>> BEAM-3589 Flink runner breaks with ClassCastException on UnboundedSource
>>
>> 3. Direct runner has a relatively big performance regression when
>> dealing with UnboundedSources. in particular the impact on query 7 of
>> Nexmark is considerable.
>>
>> Just with the small SMOKE suite in my machine I get:
>>
>> ========================================
>> Beam 2.2.0 Beam 2.3.0
>> Query Runtime(sec) Runtime(sec)
>> ========================================
>> 0000 6.4 10.6
>> 0001 5.1 10.2
>> 0002 3.0 5.8
>> 0003 3.8 6.2
>> 0004 0.9 1.4
>> 0005 5.8 11.4
>> 0006 0.8 1.4
>> 0007 193.8 1249.1
>> 0008 3.9 6.9
>> 0009 0.9 1.3
>> 0010 6.4 8.2
>> 0011 5.0 9.4
>> 0012 4.7 9.1
>>
>> This can be reproduced by running this command:
>>
>> mvn exec:java -Dexec.mainClass=org.apache.beam.sdk.nexmark.Main
>> -Pdirect-runner -Dexec.args="--runner=DirectRunner --suite=SMOKE
>> --streaming=true --manageResources=false --monitorJobs=true
>> --enforceEncodability=true --enforceImmutability=true" -pl
>> 'sdks/java/nexmark'
>>
>> I think 2 and 3 deserve to be fixed or at least evaluated as important
>> enough to cancel the vote. And if possible I would love to cherry-pick
>> the Nexmark fixes for a future RC.
>>
>> On Thu, Feb 1, 2018 at 7:20 AM, Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
>>> Hi all,
>>>
>>> just a quick reminder about the vote process:
>>>
>>> 1. Any vote can be changed during the vote period. A -1 vote has to be argued
>>> (especially if there's not change to do in the project codebase).
>>> 2. For convenience to the release manager, please inform if your vote is binding
>>> or non-binding (the vote from PMC members are binding)
>>> 3. It's not possible to "veto" a release: if we have at least 3 binding votes,
>>> the vote can pass.
>>> 4. Please, keep only vote in the thread. If you have some tests in progress,
>>> please use another thread. It would be great if the thread only contains
>>> concrete votes.
>>> 5. The vote duration can be extended on request.
>>>
>>> So, I'm extending this vote to 72 more hours to give us time to review
>>> especially the dataflow worker images test and the Flink TextIO potential issue.
>>>
>>> Thanks !
>>> Regards
>>> JB
>>>
>>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>>>> Hi everyone,
>>>>
>>>> Please review and vote on the release candidate #1 for the version 2.3.0, as
>>>> follows:
>>>>
>>>> [ ] +1, Approve the release
>>>> [ ] -1, Do not approve the release (please provide specific comments)
>>>>
>>>>
>>>> The complete staging area is available for your review, which includes:
>>>> * JIRA release notes [1],
>>>> * the official Apache source release to be deployed to dist.apache.org [2],
>>>> which is signed with the key with fingerprint C8282E76 [3],
>>>> * all artifacts to be deployed to the Maven Central Repository [4],
>>>> * source code tag "v2.3.0-RC1" [5],
>>>> * website pull request listing the release and publishing the API reference
>>>> manual [6].
>>>> * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
>>>> * Python artifacts are deployed along with the source release to the
>>>> dist.apache.org [2].
>>>>
>>>> The vote will be open for at least 72 hours. It is adopted by majority approval,
>>>> with at least 3 PMC affirmative votes.
>>>>
>>>> Thanks,
>>>> JB
>>>>
>>>> [1]
>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>>> [4] https://repository.apache.org/content/repositories/orgapachebeam-1026/
>>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>>>> [6] https://github.com/apache/beam-site/pull/381
>>>>
>>>
>>> --
>>> Jean-Baptiste Onofré
>>> jbonofre@apache.org
>>> http://blog.nanthrax.net
>>> Talend - http://www.talend.com
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
Re: [VOTE] Release 2.3.0, release candidate #1
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
So, are you casting -1 vote ?
I guess so.
Regards
JB
On 02/01/2018 03:53 PM, Ismaël Mejía wrote:
> Hi,
>
> I started to test the release with Nexmark and found three issues
> (from minor to more important):
>
> 1. Small issues to run Nexmark with the release (BEAM-3531 fixed,
> BEAM-3592 in PR):
>
> BEAM-3531 Nexmark failed with NPE with DEFAULT suite
> BEAM-3592 Spark-runner profile is broken on Nexmark after move to Spark 2.x
>
> 2. Flink is broken with Unbounded sources, big enough to deserve a new
> vote. Pred:
> BEAM-3589 Flink runner breaks with ClassCastException on UnboundedSource
>
> 3. Direct runner has a relatively big performance regression when
> dealing with UnboundedSources. in particular the impact on query 7 of
> Nexmark is considerable.
>
> Just with the small SMOKE suite in my machine I get:
>
> ========================================
> Beam 2.2.0 Beam 2.3.0
> Query Runtime(sec) Runtime(sec)
> ========================================
> 0000 6.4 10.6
> 0001 5.1 10.2
> 0002 3.0 5.8
> 0003 3.8 6.2
> 0004 0.9 1.4
> 0005 5.8 11.4
> 0006 0.8 1.4
> 0007 193.8 1249.1
> 0008 3.9 6.9
> 0009 0.9 1.3
> 0010 6.4 8.2
> 0011 5.0 9.4
> 0012 4.7 9.1
>
> This can be reproduced by running this command:
>
> mvn exec:java -Dexec.mainClass=org.apache.beam.sdk.nexmark.Main
> -Pdirect-runner -Dexec.args="--runner=DirectRunner --suite=SMOKE
> --streaming=true --manageResources=false --monitorJobs=true
> --enforceEncodability=true --enforceImmutability=true" -pl
> 'sdks/java/nexmark'
>
> I think 2 and 3 deserve to be fixed or at least evaluated as important
> enough to cancel the vote. And if possible I would love to cherry-pick
> the Nexmark fixes for a future RC.
>
> On Thu, Feb 1, 2018 at 7:20 AM, Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
>> Hi all,
>>
>> just a quick reminder about the vote process:
>>
>> 1. Any vote can be changed during the vote period. A -1 vote has to be argued
>> (especially if there's not change to do in the project codebase).
>> 2. For convenience to the release manager, please inform if your vote is binding
>> or non-binding (the vote from PMC members are binding)
>> 3. It's not possible to "veto" a release: if we have at least 3 binding votes,
>> the vote can pass.
>> 4. Please, keep only vote in the thread. If you have some tests in progress,
>> please use another thread. It would be great if the thread only contains
>> concrete votes.
>> 5. The vote duration can be extended on request.
>>
>> So, I'm extending this vote to 72 more hours to give us time to review
>> especially the dataflow worker images test and the Flink TextIO potential issue.
>>
>> Thanks !
>> Regards
>> JB
>>
>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>>> Hi everyone,
>>>
>>> Please review and vote on the release candidate #1 for the version 2.3.0, as
>>> follows:
>>>
>>> [ ] +1, Approve the release
>>> [ ] -1, Do not approve the release (please provide specific comments)
>>>
>>>
>>> The complete staging area is available for your review, which includes:
>>> * JIRA release notes [1],
>>> * the official Apache source release to be deployed to dist.apache.org [2],
>>> which is signed with the key with fingerprint C8282E76 [3],
>>> * all artifacts to be deployed to the Maven Central Repository [4],
>>> * source code tag "v2.3.0-RC1" [5],
>>> * website pull request listing the release and publishing the API reference
>>> manual [6].
>>> * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
>>> * Python artifacts are deployed along with the source release to the
>>> dist.apache.org [2].
>>>
>>> The vote will be open for at least 72 hours. It is adopted by majority approval,
>>> with at least 3 PMC affirmative votes.
>>>
>>> Thanks,
>>> JB
>>>
>>> [1]
>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>> [4] https://repository.apache.org/content/repositories/orgapachebeam-1026/
>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>>> [6] https://github.com/apache/beam-site/pull/381
>>>
>>
>> --
>> Jean-Baptiste Onofré
>> jbonofre@apache.org
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: [VOTE] Release 2.3.0, release candidate #1
Posted by Ismaël Mejía <ie...@gmail.com>.
Hi,
I started to test the release with Nexmark and found three issues
(from minor to more important):
1. Small issues to run Nexmark with the release (BEAM-3531 fixed,
BEAM-3592 in PR):
BEAM-3531 Nexmark failed with NPE with DEFAULT suite
BEAM-3592 Spark-runner profile is broken on Nexmark after move to Spark 2.x
2. Flink is broken with Unbounded sources, big enough to deserve a new
vote. Pred:
BEAM-3589 Flink runner breaks with ClassCastException on UnboundedSource
3. Direct runner has a relatively big performance regression when
dealing with UnboundedSources. in particular the impact on query 7 of
Nexmark is considerable.
Just with the small SMOKE suite in my machine I get:
========================================
Beam 2.2.0 Beam 2.3.0
Query Runtime(sec) Runtime(sec)
========================================
0000 6.4 10.6
0001 5.1 10.2
0002 3.0 5.8
0003 3.8 6.2
0004 0.9 1.4
0005 5.8 11.4
0006 0.8 1.4
0007 193.8 1249.1
0008 3.9 6.9
0009 0.9 1.3
0010 6.4 8.2
0011 5.0 9.4
0012 4.7 9.1
This can be reproduced by running this command:
mvn exec:java -Dexec.mainClass=org.apache.beam.sdk.nexmark.Main
-Pdirect-runner -Dexec.args="--runner=DirectRunner --suite=SMOKE
--streaming=true --manageResources=false --monitorJobs=true
--enforceEncodability=true --enforceImmutability=true" -pl
'sdks/java/nexmark'
I think 2 and 3 deserve to be fixed or at least evaluated as important
enough to cancel the vote. And if possible I would love to cherry-pick
the Nexmark fixes for a future RC.
On Thu, Feb 1, 2018 at 7:20 AM, Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
> Hi all,
>
> just a quick reminder about the vote process:
>
> 1. Any vote can be changed during the vote period. A -1 vote has to be argued
> (especially if there's not change to do in the project codebase).
> 2. For convenience to the release manager, please inform if your vote is binding
> or non-binding (the vote from PMC members are binding)
> 3. It's not possible to "veto" a release: if we have at least 3 binding votes,
> the vote can pass.
> 4. Please, keep only vote in the thread. If you have some tests in progress,
> please use another thread. It would be great if the thread only contains
> concrete votes.
> 5. The vote duration can be extended on request.
>
> So, I'm extending this vote to 72 more hours to give us time to review
> especially the dataflow worker images test and the Flink TextIO potential issue.
>
> Thanks !
> Regards
> JB
>
> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>> Hi everyone,
>>
>> Please review and vote on the release candidate #1 for the version 2.3.0, as
>> follows:
>>
>> [ ] +1, Approve the release
>> [ ] -1, Do not approve the release (please provide specific comments)
>>
>>
>> The complete staging area is available for your review, which includes:
>> * JIRA release notes [1],
>> * the official Apache source release to be deployed to dist.apache.org [2],
>> which is signed with the key with fingerprint C8282E76 [3],
>> * all artifacts to be deployed to the Maven Central Repository [4],
>> * source code tag "v2.3.0-RC1" [5],
>> * website pull request listing the release and publishing the API reference
>> manual [6].
>> * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
>> * Python artifacts are deployed along with the source release to the
>> dist.apache.org [2].
>>
>> The vote will be open for at least 72 hours. It is adopted by majority approval,
>> with at least 3 PMC affirmative votes.
>>
>> Thanks,
>> JB
>>
>> [1]
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>> [4] https://repository.apache.org/content/repositories/orgapachebeam-1026/
>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>> [6] https://github.com/apache/beam-site/pull/381
>>
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
Re: [VOTE] Release 2.3.0, release candidate #1
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi all,
just a quick reminder about the vote process:
1. Any vote can be changed during the vote period. A -1 vote has to be argued
(especially if there's not change to do in the project codebase).
2. For convenience to the release manager, please inform if your vote is binding
or non-binding (the vote from PMC members are binding)
3. It's not possible to "veto" a release: if we have at least 3 binding votes,
the vote can pass.
4. Please, keep only vote in the thread. If you have some tests in progress,
please use another thread. It would be great if the thread only contains
concrete votes.
5. The vote duration can be extended on request.
So, I'm extending this vote to 72 more hours to give us time to review
especially the dataflow worker images test and the Flink TextIO potential issue.
Thanks !
Regards
JB
On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
> Hi everyone,
>
> Please review and vote on the release candidate #1 for the version 2.3.0, as
> follows:
>
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
>
> The complete staging area is available for your review, which includes:
> * JIRA release notes [1],
> * the official Apache source release to be deployed to dist.apache.org [2],
> which is signed with the key with fingerprint C8282E76 [3],
> * all artifacts to be deployed to the Maven Central Repository [4],
> * source code tag "v2.3.0-RC1" [5],
> * website pull request listing the release and publishing the API reference
> manual [6].
> * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
> * Python artifacts are deployed along with the source release to the
> dist.apache.org [2].
>
> The vote will be open for at least 72 hours. It is adopted by majority approval,
> with at least 3 PMC affirmative votes.
>
> Thanks,
> JB
>
> [1]
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> [4] https://repository.apache.org/content/repositories/orgapachebeam-1026/
> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
> [6] https://github.com/apache/beam-site/pull/381
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com