You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Jean-Baptiste Onofré <jb...@nanthrax.net> on 2018/02/01 15:16:56 UTC
[CANCEL][VOTE] Release 2.3.0, release candidate #1
Hi guys,
Especially due to BEAM-3587 & BEAM-3186 regressions, I cancel RC1.
We will cherry-pick fixes on release-2.3.0 branch.
I'm updating Jira right now. When the fixes will be cherry-picked, I will submit
a RC2 to vote.
Thanks !
Regards
JB
On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
> Hi everyone,
>
> Please review and vote on the release candidate #1 for the version 2.3.0, as
> follows:
>
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
>
> The complete staging area is available for your review, which includes:
> * JIRA release notes [1],
> * the official Apache source release to be deployed to dist.apache.org [2],
> which is signed with the key with fingerprint C8282E76 [3],
> * all artifacts to be deployed to the Maven Central Repository [4],
> * source code tag "v2.3.0-RC1" [5],
> * website pull request listing the release and publishing the API reference
> manual [6].
> * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
> * Python artifacts are deployed along with the source release to the
> dist.apache.org [2].
>
> The vote will be open for at least 72 hours. It is adopted by majority approval,
> with at least 3 PMC affirmative votes.
>
> Thanks,
> JB
>
> [1]
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> [4] https://repository.apache.org/content/repositories/orgapachebeam-1026/
> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
> [6] https://github.com/apache/beam-site/pull/381
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Thanks to Kenn, I'm able to move forward on RC2.
I will start the RC2 release process in the coming hour.
Regards
JB
On 02/06/2018 11:17 PM, Kenneth Knowles wrote:
> I've cherry-picked JB's commit to https://github.com/apache/beam/pull/4621
>
> On master it seems that HBase tests are failing reliably, but actually mostly
> isolated to this PR. Hopefully the changes on the release branch will go green.
>
> On Tue, Feb 6, 2018 at 2:12 PM, Kenneth Knowles <klk@google.com
> <ma...@google.com>> wrote:
>
> Changes are cherrypicked to the release-2.3.0 branch, or applied directly there.
>
> On Tue, Feb 6, 2018 at 2:08 PM, Eugene Kirpichov <kirpichov@google.com
> <ma...@google.com>> wrote:
>
> Possibly stupid question: are new RCs created from master, or by
> cherrypicks of desired changes on top of previous RCs?
> (it affects whether or not my recently merged change on "master" will be
> in 2.3.0; I'd like it to be :) )
>
> On Tue, Feb 6, 2018 at 1:49 PM Lukasz Cwik <lcwik@google.com
> <ma...@google.com>> wrote:
>
> +1 for rollback as well.
>
> On Tue, Feb 6, 2018 at 12:37 PM, Reuven Lax <relax@google.com
> <ma...@google.com>> wrote:
>
> +1 for the rollback.
>
> On Tue, Feb 6, 2018 at 9:24 AM, Kenneth Knowles <klk@google.com
> <ma...@google.com>> wrote:
>
> +1 for option 1. It is almost certainly repeated proto
> parsing. That is fixed for ParDo via generalized caching but
> not for other transforms. Flink also has a proto round trip,
> but the design might avoid the problem anyhow. We should
> investigate before releasing, or just preemptively roll it
> back too. The proto round trips are helpful for maturing
> portability and for preventing hacks from returning but
> otherwise don't matter so much yet.
>
> Kenn
>
> On Tue, Feb 6, 2018 at 8:54 AM, Jean-Baptiste Onofré
> <jb@nanthrax.net <ma...@nanthrax.net>> wrote:
>
> Hi all,
>
> As you might have seen, I found the cause of performance
> degradation on the
> direct runner (BEAM-3617).
>
> We have basically three options for RC2:
>
> 1. We revert the change for 2.3.0. I think it's the
> fastest and more secure way.
> I created a PR for that
> (https://github.com/apache/beam/pull/4609
> <https://github.com/apache/beam/pull/4609>). It gives us
> time to investigate and provide a better fix in Runner
> API for 2.4.0.
>
> 2. If the fix is easy and with limited impact in Runner
> API, we do that.
>
> 3. We leave as it is, adding a note that direct runner
> has worse performance in
> 2.3.0 compared to 2.2.0.
>
> In any case, I would like to cut RC2 later tonight or
> tomorrow morning (my time).
>
> Thoughts ?
>
> Regards
> JB
>
> On 02/06/2018 09:42 AM, Jean-Baptiste Onofré wrote:
> > Hi Reuven,
> >
> > it's what I'm suspecting. git bisect should give us
> more information (still in
> > progress, 3 more steps to complete ;)). I keep you posted.
> >
> > Regards
> > JB
> >
> > On 02/06/2018 08:36 AM, Reuven Lax wrote:
> >> Could this be related to any of the portability changes?
> >>
> >> On Mon, Feb 5, 2018 at 7:51 AM, Jean-Baptiste Onofré
> <jb@nanthrax.net <ma...@nanthrax.net>
> >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
> >>
> >> Created:
> >>
> >> https://issues.apache.org/jira/browse/BEAM-3617
> <https://issues.apache.org/jira/browse/BEAM-3617>
> >> <https://issues.apache.org/jira/browse/BEAM-3617
> <https://issues.apache.org/jira/browse/BEAM-3617>>
> >>
> >> Regards
> >> JB
> >>
> >> On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
> >> > What is the Jira for direct runner perf?
> >> >
> >> > On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste
> Onofré <jb@nanthrax.net <ma...@nanthrax.net>
> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>
> >> > <mailto:jb@nanthrax.net
> <ma...@nanthrax.net> <mailto:jb@nanthrax.net
> <ma...@nanthrax.net>>>> wrote:
> >> >
> >> > Thanks !
> >> >
> >> > I cherry-pick on release-2.3.0 branch.
> >> >
> >> > I'm on the direct runner perf test in the
> mean time.
> >> >
> >> > Thanks again !
> >> >
> >> > Regards
> >> > JB
> >> >
> >> > On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
> >> > > I merged fixes for:
> >> > >
> - https://issues.apache.org/jira/browse/BEAM-3186
> <https://issues.apache.org/jira/browse/BEAM-3186>
> >> <https://issues.apache.org/jira/browse/BEAM-3186
> <https://issues.apache.org/jira/browse/BEAM-3186>>
> >> >
> <https://issues.apache.org/jira/browse/BEAM-3186
> <https://issues.apache.org/jira/browse/BEAM-3186>
> >> <https://issues.apache.org/jira/browse/BEAM-3186
> <https://issues.apache.org/jira/browse/BEAM-3186>>>
> >> > >
> - https://issues.apache.org/jira/browse/BEAM-3589
> <https://issues.apache.org/jira/browse/BEAM-3589>
> >> <https://issues.apache.org/jira/browse/BEAM-3589
> <https://issues.apache.org/jira/browse/BEAM-3589>>
> >> >
> <https://issues.apache.org/jira/browse/BEAM-3589
> <https://issues.apache.org/jira/browse/BEAM-3589>
> >> <https://issues.apache.org/jira/browse/BEAM-3589
> <https://issues.apache.org/jira/browse/BEAM-3589>>>
> >> > >
> >> > > @JB I didn't yet merge them on the 2.3.0
> branch, though, but I can or you can go
> >> > > ahead.
> >> > >
> >> > >> On 5. Feb 2018, at 06:30, Jean-Baptiste
> Onofré <jb@nanthrax.net <ma...@nanthrax.net>
> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>
> <mailto:jb@nanthrax.net <ma...@nanthrax.net>
> >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>>
> >> > >> <mailto:jb@nanthrax.net
> <ma...@nanthrax.net> <mailto:jb@nanthrax.net
> <ma...@nanthrax.net>>
> >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>
> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>>>> wrote:
> >> > >>
> >> > >> Hi guys,
> >> > >>
> >> > >> Quick update on the RC2 preparation:
> >> > >>
> >> > >> * BEAM-3587 (TextIO with Flink) seems
> related to a custom build with
> >> > Gradle (not
> >> > >> using artifacts created by Maven).
> Anyway, I will take a look today.
> >> > >> * BEAM-3186 has a PR. Aljoscha will do
> the review pretty soon.
> >> > >> * I'm also taking a look on the direct
> runner performance change.
> >> > >>
> >> > >> The goal is to cut RC2 tomorrow or the
> day after.
> >> > >>
> >> > >> Regards
> >> > >> JB
> >> > >>
> >> > >> On 02/01/2018 04:16 PM, Jean-Baptiste
> Onofré wrote:
> >> > >>> Hi guys,
> >> > >>>
> >> > >>> Especially due to BEAM-3587 & BEAM-3186
> regressions, I cancel RC1.
> >> > >>>
> >> > >>> We will cherry-pick fixes on
> release-2.3.0 branch.
> >> > >>>
> >> > >>> I'm updating Jira right now. When the
> fixes will be cherry-picked, I
> >> > will submit
> >> > >>> a RC2 to vote.
> >> > >>>
> >> > >>> Thanks !
> >> > >>> Regards
> >> > >>> JB
> >> > >>>
> >> > >>> On 01/30/2018 09:04 AM, Jean-Baptiste
> Onofré wrote:
> >> > >>>> Hi everyone,
> >> > >>>>
> >> > >>>> Please review and vote on the release
> candidate #1 for the version
> >> > 2.3.0, as
> >> > >>>> follows:
> >> > >>>>
> >> > >>>> [ ] +1, Approve the release
> >> > >>>> [ ] -1, Do not approve the release
> (please provide specific comments)
> >> > >>>>
> >> > >>>>
> >> > >>>> The complete staging area is available
> for your review, which
> >> includes:
> >> > >>>> * JIRA release notes [1],
> >> > >>>> * the official Apache source release
> to be deployed to
> >> dist.apache.org <http://dist.apache.org>
> <http://dist.apache.org>
> >> > <http://dist.apache.org>
> >> > >>>> <http://dist.apache.org> [2],
> >> > >>>> which is signed with the key with
> fingerprint C8282E76 [3],
> >> > >>>> * all artifacts to be deployed to the
> Maven Central Repository [4],
> >> > >>>> * source code tag "v2.3.0-RC1" [5],
> >> > >>>> * website pull request listing the
> release and publishing the API
> >> reference
> >> > >>>> manual [6].
> >> > >>>> * Java artifacts were built with Maven
> 3.3.9 and Oracle JDK
> >> 1.8.0_111.
> >> > >>>> * Python artifacts are deployed along
> with the source release to the
> >> > >>>> dist.apache.org
> <http://dist.apache.org> <http://dist.apache.org>
> <http://dist.apache.org>
> >> <http://dist.apache.org> [2].
> >> > >>>>
> >> > >>>> The vote will be open for at least 72
> hours. It is adopted by majority approval,
> >> > >>>> with at least 3 PMC affirmative votes.
> >> > >>>>
> >> > >>>> Thanks,
> >> > >>>> JB
> >> > >>>>
> >> > >>>> [1]
> >> > >>>>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>
> >>
> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>>
> >> >
> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>
> >>
> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>>>
> >> > >>>> [2]
> https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
> >>
> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>
> >> >
> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
> >>
> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>>
> >> > >>>> [3]
> https://dist.apache.org/repos/dist/release/beam/KEYS
> <https://dist.apache.org/repos/dist/release/beam/KEYS>
> >>
> <https://dist.apache.org/repos/dist/release/beam/KEYS
> <https://dist.apache.org/repos/dist/release/beam/KEYS>>
> >> >
> <https://dist.apache.org/repos/dist/release/beam/KEYS
> <https://dist.apache.org/repos/dist/release/beam/KEYS>
> >>
> <https://dist.apache.org/repos/dist/release/beam/KEYS
> <https://dist.apache.org/repos/dist/release/beam/KEYS>>>
> >> > >>>> [4]
> https://repository.apache.org/content/repositories/orgapachebeam-1026/
> <https://repository.apache.org/content/repositories/orgapachebeam-1026/>
> >>
> <https://repository.apache.org/content/repositories/orgapachebeam-1026/
> <https://repository.apache.org/content/repositories/orgapachebeam-1026/>>
> >> >
> <https://repository.apache.org/content/repositories/orgapachebeam-1026/
> <https://repository.apache.org/content/repositories/orgapachebeam-1026/>
> >>
> <https://repository.apache.org/content/repositories/orgapachebeam-1026/
> <https://repository.apache.org/content/repositories/orgapachebeam-1026/>>>
> >> > >>>> [5]
> https://github.com/apache/beam/tree/v2.3.0-RC1
> <https://github.com/apache/beam/tree/v2.3.0-RC1>
> >> <https://github.com/apache/beam/tree/v2.3.0-RC1
> <https://github.com/apache/beam/tree/v2.3.0-RC1>>
> >> >
> <https://github.com/apache/beam/tree/v2.3.0-RC1
> <https://github.com/apache/beam/tree/v2.3.0-RC1>
> >> <https://github.com/apache/beam/tree/v2.3.0-RC1
> <https://github.com/apache/beam/tree/v2.3.0-RC1>>>
> >> > >>>> [6]
> https://github.com/apache/beam-site/pull/381
> <https://github.com/apache/beam-site/pull/381>
> >> <https://github.com/apache/beam-site/pull/381
> <https://github.com/apache/beam-site/pull/381>>
> >> >
> <https://github.com/apache/beam-site/pull/381
> <https://github.com/apache/beam-site/pull/381>
> >> <https://github.com/apache/beam-site/pull/381
> <https://github.com/apache/beam-site/pull/381>>>
> >> > >>>>
> >> > >>>
> >> > >>
> >> > >> --
> >> > >> Jean-Baptiste Onofré
> >> > >> jbonofre@apache.org
> <ma...@apache.org> <mailto:jbonofre@apache.org
> <ma...@apache.org>>
> <mailto:jbonofre@apache.org <ma...@apache.org>
> >> <mailto:jbonofre@apache.org
> <ma...@apache.org>>>
> >> > <mailto:jbonofre@apache.org
> <ma...@apache.org> <mailto:jbonofre@apache.org
> <ma...@apache.org>>
> >> <mailto:jbonofre@apache.org
> <ma...@apache.org> <mailto:jbonofre@apache.org
> <ma...@apache.org>>>>
> >> > >> http://blog.nanthrax.net
> >> > >> Talend - http://www.talend.com
> >> > >
> >> >
> >> > --
> >> > Jean-Baptiste Onofré
> >> > jbonofre@apache.org
> <ma...@apache.org> <mailto:jbonofre@apache.org
> <ma...@apache.org>>
> >> <mailto:jbonofre@apache.org
> <ma...@apache.org> <mailto:jbonofre@apache.org
> <ma...@apache.org>>>
> >> > http://blog.nanthrax.net
> >> > Talend - http://www.talend.com
> >> >
> >> >
> >>
> >> --
> >> Jean-Baptiste Onofré
> >> jbonofre@apache.org <ma...@apache.org>
> <mailto:jbonofre@apache.org <ma...@apache.org>>
> >> http://blog.nanthrax.net
> >> Talend - http://www.talend.com
> >>
> >>
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org <ma...@apache.org>
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
>
>
>
>
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Kenneth Knowles <kl...@google.com>.
I've cherry-picked JB's commit to https://github.com/apache/beam/pull/4621
On master it seems that HBase tests are failing reliably, but actually
mostly isolated to this PR. Hopefully the changes on the release branch
will go green.
On Tue, Feb 6, 2018 at 2:12 PM, Kenneth Knowles <kl...@google.com> wrote:
> Changes are cherrypicked to the release-2.3.0 branch, or applied directly
> there.
>
> On Tue, Feb 6, 2018 at 2:08 PM, Eugene Kirpichov <ki...@google.com>
> wrote:
>
>> Possibly stupid question: are new RCs created from master, or by
>> cherrypicks of desired changes on top of previous RCs?
>> (it affects whether or not my recently merged change on "master" will be
>> in 2.3.0; I'd like it to be :) )
>>
>> On Tue, Feb 6, 2018 at 1:49 PM Lukasz Cwik <lc...@google.com> wrote:
>>
>>> +1 for rollback as well.
>>>
>>> On Tue, Feb 6, 2018 at 12:37 PM, Reuven Lax <re...@google.com> wrote:
>>>
>>>> +1 for the rollback.
>>>>
>>>> On Tue, Feb 6, 2018 at 9:24 AM, Kenneth Knowles <kl...@google.com> wrote:
>>>>
>>>>> +1 for option 1. It is almost certainly repeated proto parsing. That
>>>>> is fixed for ParDo via generalized caching but not for other transforms.
>>>>> Flink also has a proto round trip, but the design might avoid the problem
>>>>> anyhow. We should investigate before releasing, or just preemptively roll
>>>>> it back too. The proto round trips are helpful for maturing portability and
>>>>> for preventing hacks from returning but otherwise don't matter so much yet.
>>>>>
>>>>> Kenn
>>>>>
>>>>> On Tue, Feb 6, 2018 at 8:54 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
>>>>> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> As you might have seen, I found the cause of performance degradation
>>>>>> on the
>>>>>> direct runner (BEAM-3617).
>>>>>>
>>>>>> We have basically three options for RC2:
>>>>>>
>>>>>> 1. We revert the change for 2.3.0. I think it's the fastest and more
>>>>>> secure way.
>>>>>> I created a PR for that (https://github.com/apache/beam/pull/4609).
>>>>>> It gives us
>>>>>> time to investigate and provide a better fix in Runner API for 2.4.0.
>>>>>>
>>>>>> 2. If the fix is easy and with limited impact in Runner API, we do
>>>>>> that.
>>>>>>
>>>>>> 3. We leave as it is, adding a note that direct runner has worse
>>>>>> performance in
>>>>>> 2.3.0 compared to 2.2.0.
>>>>>>
>>>>>> In any case, I would like to cut RC2 later tonight or tomorrow
>>>>>> morning (my time).
>>>>>>
>>>>>> Thoughts ?
>>>>>>
>>>>>> Regards
>>>>>> JB
>>>>>>
>>>>>> On 02/06/2018 09:42 AM, Jean-Baptiste Onofré wrote:
>>>>>> > Hi Reuven,
>>>>>> >
>>>>>> > it's what I'm suspecting. git bisect should give us more
>>>>>> information (still in
>>>>>> > progress, 3 more steps to complete ;)). I keep you posted.
>>>>>> >
>>>>>> > Regards
>>>>>> > JB
>>>>>> >
>>>>>> > On 02/06/2018 08:36 AM, Reuven Lax wrote:
>>>>>> >> Could this be related to any of the portability changes?
>>>>>> >>
>>>>>> >> On Mon, Feb 5, 2018 at 7:51 AM, Jean-Baptiste Onofré <
>>>>>> jb@nanthrax.net
>>>>>> >> <ma...@nanthrax.net>> wrote:
>>>>>> >>
>>>>>> >> Created:
>>>>>> >>
>>>>>> >> https://issues.apache.org/jira/browse/BEAM-3617
>>>>>> >> <https://issues.apache.org/jira/browse/BEAM-3617>
>>>>>> >>
>>>>>> >> Regards
>>>>>> >> JB
>>>>>> >>
>>>>>> >> On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
>>>>>> >> > What is the Jira for direct runner perf?
>>>>>> >> >
>>>>>> >> > On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré <
>>>>>> jb@nanthrax.net <ma...@nanthrax.net>
>>>>>> >> > <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
>>>>>> >> >
>>>>>> >> > Thanks !
>>>>>> >> >
>>>>>> >> > I cherry-pick on release-2.3.0 branch.
>>>>>> >> >
>>>>>> >> > I'm on the direct runner perf test in the mean time.
>>>>>> >> >
>>>>>> >> > Thanks again !
>>>>>> >> >
>>>>>> >> > Regards
>>>>>> >> > JB
>>>>>> >> >
>>>>>> >> > On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
>>>>>> >> > > I merged fixes for:
>>>>>> >> > > - https://issues.apache.org/jira/browse/BEAM-3186
>>>>>> >> <https://issues.apache.org/jira/browse/BEAM-3186>
>>>>>> >> > <https://issues.apache.org/jira/browse/BEAM-3186
>>>>>> >> <https://issues.apache.org/jira/browse/BEAM-3186>>
>>>>>> >> > > - https://issues.apache.org/jira/browse/BEAM-3589
>>>>>> >> <https://issues.apache.org/jira/browse/BEAM-3589>
>>>>>> >> > <https://issues.apache.org/jira/browse/BEAM-3589
>>>>>> >> <https://issues.apache.org/jira/browse/BEAM-3589>>
>>>>>> >> > >
>>>>>> >> > > @JB I didn't yet merge them on the 2.3.0 branch,
>>>>>> though, but I can or you can go
>>>>>> >> > > ahead.
>>>>>> >> > >
>>>>>> >> > >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <
>>>>>> jb@nanthrax.net <ma...@nanthrax.net> <mailto:jb@nanthrax.net
>>>>>> >> <ma...@nanthrax.net>>
>>>>>> >> > >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>
>>>>>> >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>>> wrote:
>>>>>> >> > >>
>>>>>> >> > >> Hi guys,
>>>>>> >> > >>
>>>>>> >> > >> Quick update on the RC2 preparation:
>>>>>> >> > >>
>>>>>> >> > >> * BEAM-3587 (TextIO with Flink) seems related to a
>>>>>> custom build with
>>>>>> >> > Gradle (not
>>>>>> >> > >> using artifacts created by Maven). Anyway, I will
>>>>>> take a look today.
>>>>>> >> > >> * BEAM-3186 has a PR. Aljoscha will do the review
>>>>>> pretty soon.
>>>>>> >> > >> * I'm also taking a look on the direct runner
>>>>>> performance change.
>>>>>> >> > >>
>>>>>> >> > >> The goal is to cut RC2 tomorrow or the day after.
>>>>>> >> > >>
>>>>>> >> > >> Regards
>>>>>> >> > >> JB
>>>>>> >> > >>
>>>>>> >> > >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
>>>>>> >> > >>> Hi guys,
>>>>>> >> > >>>
>>>>>> >> > >>> Especially due to BEAM-3587 & BEAM-3186 regressions,
>>>>>> I cancel RC1.
>>>>>> >> > >>>
>>>>>> >> > >>> We will cherry-pick fixes on release-2.3.0 branch.
>>>>>> >> > >>>
>>>>>> >> > >>> I'm updating Jira right now. When the fixes will be
>>>>>> cherry-picked, I
>>>>>> >> > will submit
>>>>>> >> > >>> a RC2 to vote.
>>>>>> >> > >>>
>>>>>> >> > >>> Thanks !
>>>>>> >> > >>> Regards
>>>>>> >> > >>> JB
>>>>>> >> > >>>
>>>>>> >> > >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>>>>>> >> > >>>> Hi everyone,
>>>>>> >> > >>>>
>>>>>> >> > >>>> Please review and vote on the release candidate #1
>>>>>> for the version
>>>>>> >> > 2.3.0, as
>>>>>> >> > >>>> follows:
>>>>>> >> > >>>>
>>>>>> >> > >>>> [ ] +1, Approve the release
>>>>>> >> > >>>> [ ] -1, Do not approve the release (please provide
>>>>>> specific comments)
>>>>>> >> > >>>>
>>>>>> >> > >>>>
>>>>>> >> > >>>> The complete staging area is available for your
>>>>>> review, which
>>>>>> >> includes:
>>>>>> >> > >>>> * JIRA release notes [1],
>>>>>> >> > >>>> * the official Apache source release to be deployed
>>>>>> to
>>>>>> >> dist.apache.org <http://dist.apache.org>
>>>>>> >> > <http://dist.apache.org>
>>>>>> >> > >>>> <http://dist.apache.org> [2],
>>>>>> >> > >>>> which is signed with the key with fingerprint
>>>>>> C8282E76 [3],
>>>>>> >> > >>>> * all artifacts to be deployed to the Maven Central
>>>>>> Repository [4],
>>>>>> >> > >>>> * source code tag "v2.3.0-RC1" [5],
>>>>>> >> > >>>> * website pull request listing the release and
>>>>>> publishing the API
>>>>>> >> reference
>>>>>> >> > >>>> manual [6].
>>>>>> >> > >>>> * Java artifacts were built with Maven 3.3.9 and
>>>>>> Oracle JDK
>>>>>> >> 1.8.0_111.
>>>>>> >> > >>>> * Python artifacts are deployed along with the
>>>>>> source release to the
>>>>>> >> > >>>> dist.apache.org <http://dist.apache.org> <
>>>>>> http://dist.apache.org>
>>>>>> >> <http://dist.apache.org> [2].
>>>>>> >> > >>>>
>>>>>> >> > >>>> The vote will be open for at least 72 hours. It is
>>>>>> adopted by majority approval,
>>>>>> >> > >>>> with at least 3 PMC affirmative votes.
>>>>>> >> > >>>>
>>>>>> >> > >>>> Thanks,
>>>>>> >> > >>>> JB
>>>>>> >> > >>>>
>>>>>> >> > >>>> [1]
>>>>>> >> > >>>> https://issues.apache.org/jira
>>>>>> /secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>>>>>> >> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?pro
>>>>>> jectId=12319527&version=12341608>
>>>>>> >> > <https://issues.apache.org/ji
>>>>>> ra/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>>>>>> >> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?pro
>>>>>> jectId=12319527&version=12341608>>
>>>>>> >> > >>>> [2] https://dist.apache.org/repos/
>>>>>> dist/dev/beam/2.3.0/
>>>>>> >> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
>>>>>> >> > <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>>>>>> >> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>
>>>>>> >> > >>>> [3] https://dist.apache.org/repos/
>>>>>> dist/release/beam/KEYS
>>>>>> >> <https://dist.apache.org/repos/dist/release/beam/KEYS>
>>>>>> >> > <https://dist.apache.org/repos/dist/release/beam/KEYS
>>>>>> >> <https://dist.apache.org/repos/dist/release/beam/KEYS>>
>>>>>> >> > >>>> [4] https://repository.apache.org/
>>>>>> content/repositories/orgapachebeam-1026/
>>>>>> >> <https://repository.apache.org/content/repositories/orgapac
>>>>>> hebeam-1026/>
>>>>>> >> > <https://repository.apache.or
>>>>>> g/content/repositories/orgapachebeam-1026/
>>>>>> >> <https://repository.apache.org/content/repositories/orgapac
>>>>>> hebeam-1026/>>
>>>>>> >> > >>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>>>>>> >> <https://github.com/apache/beam/tree/v2.3.0-RC1>
>>>>>> >> > <https://github.com/apache/beam/tree/v2.3.0-RC1
>>>>>> >> <https://github.com/apache/beam/tree/v2.3.0-RC1>>
>>>>>> >> > >>>> [6] https://github.com/apache/beam-site/pull/381
>>>>>> >> <https://github.com/apache/beam-site/pull/381>
>>>>>> >> > <https://github.com/apache/beam-site/pull/381
>>>>>> >> <https://github.com/apache/beam-site/pull/381>>
>>>>>> >> > >>>>
>>>>>> >> > >>>
>>>>>> >> > >>
>>>>>> >> > >> --
>>>>>> >> > >> Jean-Baptiste Onofré
>>>>>> >> > >> jbonofre@apache.org <ma...@apache.org>
>>>>>> <mailto:jbonofre@apache.org
>>>>>> >> <ma...@apache.org>>
>>>>>> >> > <mailto:jbonofre@apache.org <ma...@apache.org>
>>>>>> >> <mailto:jbonofre@apache.org <ma...@apache.org>>>
>>>>>> >> > >> http://blog.nanthrax.net
>>>>>> >> > >> Talend - http://www.talend.com
>>>>>> >> > >
>>>>>> >> >
>>>>>> >> > --
>>>>>> >> > Jean-Baptiste Onofré
>>>>>> >> > jbonofre@apache.org <ma...@apache.org>
>>>>>> >> <mailto:jbonofre@apache.org <ma...@apache.org>>
>>>>>> >> > http://blog.nanthrax.net
>>>>>> >> > Talend - http://www.talend.com
>>>>>> >> >
>>>>>> >> >
>>>>>> >>
>>>>>> >> --
>>>>>> >> Jean-Baptiste Onofré
>>>>>> >> jbonofre@apache.org <ma...@apache.org>
>>>>>> >> http://blog.nanthrax.net
>>>>>> >> Talend - http://www.talend.com
>>>>>> >>
>>>>>> >>
>>>>>> >
>>>>>>
>>>>>> --
>>>>>> Jean-Baptiste Onofré
>>>>>> jbonofre@apache.org
>>>>>> http://blog.nanthrax.net
>>>>>> Talend - http://www.talend.com
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Kenneth Knowles <kl...@google.com>.
Changes are cherrypicked to the release-2.3.0 branch, or applied directly
there.
On Tue, Feb 6, 2018 at 2:08 PM, Eugene Kirpichov <ki...@google.com>
wrote:
> Possibly stupid question: are new RCs created from master, or by
> cherrypicks of desired changes on top of previous RCs?
> (it affects whether or not my recently merged change on "master" will be
> in 2.3.0; I'd like it to be :) )
>
> On Tue, Feb 6, 2018 at 1:49 PM Lukasz Cwik <lc...@google.com> wrote:
>
>> +1 for rollback as well.
>>
>> On Tue, Feb 6, 2018 at 12:37 PM, Reuven Lax <re...@google.com> wrote:
>>
>>> +1 for the rollback.
>>>
>>> On Tue, Feb 6, 2018 at 9:24 AM, Kenneth Knowles <kl...@google.com> wrote:
>>>
>>>> +1 for option 1. It is almost certainly repeated proto parsing. That is
>>>> fixed for ParDo via generalized caching but not for other transforms. Flink
>>>> also has a proto round trip, but the design might avoid the problem anyhow.
>>>> We should investigate before releasing, or just preemptively roll it back
>>>> too. The proto round trips are helpful for maturing portability and for
>>>> preventing hacks from returning but otherwise don't matter so much yet.
>>>>
>>>> Kenn
>>>>
>>>> On Tue, Feb 6, 2018 at 8:54 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
>>>> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> As you might have seen, I found the cause of performance degradation
>>>>> on the
>>>>> direct runner (BEAM-3617).
>>>>>
>>>>> We have basically three options for RC2:
>>>>>
>>>>> 1. We revert the change for 2.3.0. I think it's the fastest and more
>>>>> secure way.
>>>>> I created a PR for that (https://github.com/apache/beam/pull/4609).
>>>>> It gives us
>>>>> time to investigate and provide a better fix in Runner API for 2.4.0.
>>>>>
>>>>> 2. If the fix is easy and with limited impact in Runner API, we do
>>>>> that.
>>>>>
>>>>> 3. We leave as it is, adding a note that direct runner has worse
>>>>> performance in
>>>>> 2.3.0 compared to 2.2.0.
>>>>>
>>>>> In any case, I would like to cut RC2 later tonight or tomorrow morning
>>>>> (my time).
>>>>>
>>>>> Thoughts ?
>>>>>
>>>>> Regards
>>>>> JB
>>>>>
>>>>> On 02/06/2018 09:42 AM, Jean-Baptiste Onofré wrote:
>>>>> > Hi Reuven,
>>>>> >
>>>>> > it's what I'm suspecting. git bisect should give us more information
>>>>> (still in
>>>>> > progress, 3 more steps to complete ;)). I keep you posted.
>>>>> >
>>>>> > Regards
>>>>> > JB
>>>>> >
>>>>> > On 02/06/2018 08:36 AM, Reuven Lax wrote:
>>>>> >> Could this be related to any of the portability changes?
>>>>> >>
>>>>> >> On Mon, Feb 5, 2018 at 7:51 AM, Jean-Baptiste Onofré <
>>>>> jb@nanthrax.net
>>>>> >> <ma...@nanthrax.net>> wrote:
>>>>> >>
>>>>> >> Created:
>>>>> >>
>>>>> >> https://issues.apache.org/jira/browse/BEAM-3617
>>>>> >> <https://issues.apache.org/jira/browse/BEAM-3617>
>>>>> >>
>>>>> >> Regards
>>>>> >> JB
>>>>> >>
>>>>> >> On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
>>>>> >> > What is the Jira for direct runner perf?
>>>>> >> >
>>>>> >> > On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré <
>>>>> jb@nanthrax.net <ma...@nanthrax.net>
>>>>> >> > <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
>>>>> >> >
>>>>> >> > Thanks !
>>>>> >> >
>>>>> >> > I cherry-pick on release-2.3.0 branch.
>>>>> >> >
>>>>> >> > I'm on the direct runner perf test in the mean time.
>>>>> >> >
>>>>> >> > Thanks again !
>>>>> >> >
>>>>> >> > Regards
>>>>> >> > JB
>>>>> >> >
>>>>> >> > On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
>>>>> >> > > I merged fixes for:
>>>>> >> > > - https://issues.apache.org/jira/browse/BEAM-3186
>>>>> >> <https://issues.apache.org/jira/browse/BEAM-3186>
>>>>> >> > <https://issues.apache.org/jira/browse/BEAM-3186
>>>>> >> <https://issues.apache.org/jira/browse/BEAM-3186>>
>>>>> >> > > - https://issues.apache.org/jira/browse/BEAM-3589
>>>>> >> <https://issues.apache.org/jira/browse/BEAM-3589>
>>>>> >> > <https://issues.apache.org/jira/browse/BEAM-3589
>>>>> >> <https://issues.apache.org/jira/browse/BEAM-3589>>
>>>>> >> > >
>>>>> >> > > @JB I didn't yet merge them on the 2.3.0 branch,
>>>>> though, but I can or you can go
>>>>> >> > > ahead.
>>>>> >> > >
>>>>> >> > >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <
>>>>> jb@nanthrax.net <ma...@nanthrax.net> <mailto:jb@nanthrax.net
>>>>> >> <ma...@nanthrax.net>>
>>>>> >> > >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>
>>>>> >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>>> wrote:
>>>>> >> > >>
>>>>> >> > >> Hi guys,
>>>>> >> > >>
>>>>> >> > >> Quick update on the RC2 preparation:
>>>>> >> > >>
>>>>> >> > >> * BEAM-3587 (TextIO with Flink) seems related to a
>>>>> custom build with
>>>>> >> > Gradle (not
>>>>> >> > >> using artifacts created by Maven). Anyway, I will take
>>>>> a look today.
>>>>> >> > >> * BEAM-3186 has a PR. Aljoscha will do the review
>>>>> pretty soon.
>>>>> >> > >> * I'm also taking a look on the direct runner
>>>>> performance change.
>>>>> >> > >>
>>>>> >> > >> The goal is to cut RC2 tomorrow or the day after.
>>>>> >> > >>
>>>>> >> > >> Regards
>>>>> >> > >> JB
>>>>> >> > >>
>>>>> >> > >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
>>>>> >> > >>> Hi guys,
>>>>> >> > >>>
>>>>> >> > >>> Especially due to BEAM-3587 & BEAM-3186 regressions,
>>>>> I cancel RC1.
>>>>> >> > >>>
>>>>> >> > >>> We will cherry-pick fixes on release-2.3.0 branch.
>>>>> >> > >>>
>>>>> >> > >>> I'm updating Jira right now. When the fixes will be
>>>>> cherry-picked, I
>>>>> >> > will submit
>>>>> >> > >>> a RC2 to vote.
>>>>> >> > >>>
>>>>> >> > >>> Thanks !
>>>>> >> > >>> Regards
>>>>> >> > >>> JB
>>>>> >> > >>>
>>>>> >> > >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>>>>> >> > >>>> Hi everyone,
>>>>> >> > >>>>
>>>>> >> > >>>> Please review and vote on the release candidate #1
>>>>> for the version
>>>>> >> > 2.3.0, as
>>>>> >> > >>>> follows:
>>>>> >> > >>>>
>>>>> >> > >>>> [ ] +1, Approve the release
>>>>> >> > >>>> [ ] -1, Do not approve the release (please provide
>>>>> specific comments)
>>>>> >> > >>>>
>>>>> >> > >>>>
>>>>> >> > >>>> The complete staging area is available for your
>>>>> review, which
>>>>> >> includes:
>>>>> >> > >>>> * JIRA release notes [1],
>>>>> >> > >>>> * the official Apache source release to be deployed
>>>>> to
>>>>> >> dist.apache.org <http://dist.apache.org>
>>>>> >> > <http://dist.apache.org>
>>>>> >> > >>>> <http://dist.apache.org> [2],
>>>>> >> > >>>> which is signed with the key with fingerprint
>>>>> C8282E76 [3],
>>>>> >> > >>>> * all artifacts to be deployed to the Maven Central
>>>>> Repository [4],
>>>>> >> > >>>> * source code tag "v2.3.0-RC1" [5],
>>>>> >> > >>>> * website pull request listing the release and
>>>>> publishing the API
>>>>> >> reference
>>>>> >> > >>>> manual [6].
>>>>> >> > >>>> * Java artifacts were built with Maven 3.3.9 and
>>>>> Oracle JDK
>>>>> >> 1.8.0_111.
>>>>> >> > >>>> * Python artifacts are deployed along with the
>>>>> source release to the
>>>>> >> > >>>> dist.apache.org <http://dist.apache.org> <
>>>>> http://dist.apache.org>
>>>>> >> <http://dist.apache.org> [2].
>>>>> >> > >>>>
>>>>> >> > >>>> The vote will be open for at least 72 hours. It is
>>>>> adopted by majority approval,
>>>>> >> > >>>> with at least 3 PMC affirmative votes.
>>>>> >> > >>>>
>>>>> >> > >>>> Thanks,
>>>>> >> > >>>> JB
>>>>> >> > >>>>
>>>>> >> > >>>> [1]
>>>>> >> > >>>> https://issues.apache.org/
>>>>> jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>>>>> >> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?
>>>>> projectId=12319527&version=12341608>
>>>>> >> > <https://issues.apache.org/jira/secure/ReleaseNote.jspa?
>>>>> projectId=12319527&version=12341608
>>>>> >> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?
>>>>> projectId=12319527&version=12341608>>
>>>>> >> > >>>> [2] https://dist.apache.org/repos/
>>>>> dist/dev/beam/2.3.0/
>>>>> >> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
>>>>> >> > <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>>>>> >> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>
>>>>> >> > >>>> [3] https://dist.apache.org/repos/
>>>>> dist/release/beam/KEYS
>>>>> >> <https://dist.apache.org/repos/dist/release/beam/KEYS>
>>>>> >> > <https://dist.apache.org/repos/dist/release/beam/KEYS
>>>>> >> <https://dist.apache.org/repos/dist/release/beam/KEYS>>
>>>>> >> > >>>> [4] https://repository.apache.org/
>>>>> content/repositories/orgapachebeam-1026/
>>>>> >> <https://repository.apache.org/content/repositories/
>>>>> orgapachebeam-1026/>
>>>>> >> > <https://repository.apache.org/content/repositories/
>>>>> orgapachebeam-1026/
>>>>> >> <https://repository.apache.org/content/repositories/
>>>>> orgapachebeam-1026/>>
>>>>> >> > >>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>>>>> >> <https://github.com/apache/beam/tree/v2.3.0-RC1>
>>>>> >> > <https://github.com/apache/beam/tree/v2.3.0-RC1
>>>>> >> <https://github.com/apache/beam/tree/v2.3.0-RC1>>
>>>>> >> > >>>> [6] https://github.com/apache/beam-site/pull/381
>>>>> >> <https://github.com/apache/beam-site/pull/381>
>>>>> >> > <https://github.com/apache/beam-site/pull/381
>>>>> >> <https://github.com/apache/beam-site/pull/381>>
>>>>> >> > >>>>
>>>>> >> > >>>
>>>>> >> > >>
>>>>> >> > >> --
>>>>> >> > >> Jean-Baptiste Onofré
>>>>> >> > >> jbonofre@apache.org <ma...@apache.org>
>>>>> <mailto:jbonofre@apache.org
>>>>> >> <ma...@apache.org>>
>>>>> >> > <mailto:jbonofre@apache.org <ma...@apache.org>
>>>>> >> <mailto:jbonofre@apache.org <ma...@apache.org>>>
>>>>> >> > >> http://blog.nanthrax.net
>>>>> >> > >> Talend - http://www.talend.com
>>>>> >> > >
>>>>> >> >
>>>>> >> > --
>>>>> >> > Jean-Baptiste Onofré
>>>>> >> > jbonofre@apache.org <ma...@apache.org>
>>>>> >> <mailto:jbonofre@apache.org <ma...@apache.org>>
>>>>> >> > http://blog.nanthrax.net
>>>>> >> > Talend - http://www.talend.com
>>>>> >> >
>>>>> >> >
>>>>> >>
>>>>> >> --
>>>>> >> Jean-Baptiste Onofré
>>>>> >> jbonofre@apache.org <ma...@apache.org>
>>>>> >> http://blog.nanthrax.net
>>>>> >> Talend - http://www.talend.com
>>>>> >>
>>>>> >>
>>>>> >
>>>>>
>>>>> --
>>>>> Jean-Baptiste Onofré
>>>>> jbonofre@apache.org
>>>>> http://blog.nanthrax.net
>>>>> Talend - http://www.talend.com
>>>>>
>>>>
>>>>
>>>
>>
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Eugene Kirpichov <ki...@google.com>.
Possibly stupid question: are new RCs created from master, or by
cherrypicks of desired changes on top of previous RCs?
(it affects whether or not my recently merged change on "master" will be in
2.3.0; I'd like it to be :) )
On Tue, Feb 6, 2018 at 1:49 PM Lukasz Cwik <lc...@google.com> wrote:
> +1 for rollback as well.
>
> On Tue, Feb 6, 2018 at 12:37 PM, Reuven Lax <re...@google.com> wrote:
>
>> +1 for the rollback.
>>
>> On Tue, Feb 6, 2018 at 9:24 AM, Kenneth Knowles <kl...@google.com> wrote:
>>
>>> +1 for option 1. It is almost certainly repeated proto parsing. That is
>>> fixed for ParDo via generalized caching but not for other transforms. Flink
>>> also has a proto round trip, but the design might avoid the problem anyhow.
>>> We should investigate before releasing, or just preemptively roll it back
>>> too. The proto round trips are helpful for maturing portability and for
>>> preventing hacks from returning but otherwise don't matter so much yet.
>>>
>>> Kenn
>>>
>>> On Tue, Feb 6, 2018 at 8:54 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
>>> wrote:
>>>
>>>> Hi all,
>>>>
>>>> As you might have seen, I found the cause of performance degradation on
>>>> the
>>>> direct runner (BEAM-3617).
>>>>
>>>> We have basically three options for RC2:
>>>>
>>>> 1. We revert the change for 2.3.0. I think it's the fastest and more
>>>> secure way.
>>>> I created a PR for that (https://github.com/apache/beam/pull/4609). It
>>>> gives us
>>>> time to investigate and provide a better fix in Runner API for 2.4.0.
>>>>
>>>> 2. If the fix is easy and with limited impact in Runner API, we do that.
>>>>
>>>> 3. We leave as it is, adding a note that direct runner has worse
>>>> performance in
>>>> 2.3.0 compared to 2.2.0.
>>>>
>>>> In any case, I would like to cut RC2 later tonight or tomorrow morning
>>>> (my time).
>>>>
>>>> Thoughts ?
>>>>
>>>> Regards
>>>> JB
>>>>
>>>> On 02/06/2018 09:42 AM, Jean-Baptiste Onofré wrote:
>>>> > Hi Reuven,
>>>> >
>>>> > it's what I'm suspecting. git bisect should give us more information
>>>> (still in
>>>> > progress, 3 more steps to complete ;)). I keep you posted.
>>>> >
>>>> > Regards
>>>> > JB
>>>> >
>>>> > On 02/06/2018 08:36 AM, Reuven Lax wrote:
>>>> >> Could this be related to any of the portability changes?
>>>> >>
>>>> >> On Mon, Feb 5, 2018 at 7:51 AM, Jean-Baptiste Onofré <
>>>> jb@nanthrax.net
>>>> >> <ma...@nanthrax.net>> wrote:
>>>> >>
>>>> >> Created:
>>>> >>
>>>> >> https://issues.apache.org/jira/browse/BEAM-3617
>>>> >> <https://issues.apache.org/jira/browse/BEAM-3617>
>>>> >>
>>>> >> Regards
>>>> >> JB
>>>> >>
>>>> >> On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
>>>> >> > What is the Jira for direct runner perf?
>>>> >> >
>>>> >> > On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré <
>>>> jb@nanthrax.net <ma...@nanthrax.net>
>>>> >> > <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
>>>> >> >
>>>> >> > Thanks !
>>>> >> >
>>>> >> > I cherry-pick on release-2.3.0 branch.
>>>> >> >
>>>> >> > I'm on the direct runner perf test in the mean time.
>>>> >> >
>>>> >> > Thanks again !
>>>> >> >
>>>> >> > Regards
>>>> >> > JB
>>>> >> >
>>>> >> > On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
>>>> >> > > I merged fixes for:
>>>> >> > > - https://issues.apache.org/jira/browse/BEAM-3186
>>>> >> <https://issues.apache.org/jira/browse/BEAM-3186>
>>>> >> > <https://issues.apache.org/jira/browse/BEAM-3186
>>>> >> <https://issues.apache.org/jira/browse/BEAM-3186>>
>>>> >> > > - https://issues.apache.org/jira/browse/BEAM-3589
>>>> >> <https://issues.apache.org/jira/browse/BEAM-3589>
>>>> >> > <https://issues.apache.org/jira/browse/BEAM-3589
>>>> >> <https://issues.apache.org/jira/browse/BEAM-3589>>
>>>> >> > >
>>>> >> > > @JB I didn't yet merge them on the 2.3.0 branch, though,
>>>> but I can or you can go
>>>> >> > > ahead.
>>>> >> > >
>>>> >> > >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <
>>>> jb@nanthrax.net <ma...@nanthrax.net> <mailto:jb@nanthrax.net
>>>> >> <ma...@nanthrax.net>>
>>>> >> > >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>
>>>> >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>>> wrote:
>>>> >> > >>
>>>> >> > >> Hi guys,
>>>> >> > >>
>>>> >> > >> Quick update on the RC2 preparation:
>>>> >> > >>
>>>> >> > >> * BEAM-3587 (TextIO with Flink) seems related to a
>>>> custom build with
>>>> >> > Gradle (not
>>>> >> > >> using artifacts created by Maven). Anyway, I will take
>>>> a look today.
>>>> >> > >> * BEAM-3186 has a PR. Aljoscha will do the review
>>>> pretty soon.
>>>> >> > >> * I'm also taking a look on the direct runner
>>>> performance change.
>>>> >> > >>
>>>> >> > >> The goal is to cut RC2 tomorrow or the day after.
>>>> >> > >>
>>>> >> > >> Regards
>>>> >> > >> JB
>>>> >> > >>
>>>> >> > >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
>>>> >> > >>> Hi guys,
>>>> >> > >>>
>>>> >> > >>> Especially due to BEAM-3587 & BEAM-3186 regressions, I
>>>> cancel RC1.
>>>> >> > >>>
>>>> >> > >>> We will cherry-pick fixes on release-2.3.0 branch.
>>>> >> > >>>
>>>> >> > >>> I'm updating Jira right now. When the fixes will be
>>>> cherry-picked, I
>>>> >> > will submit
>>>> >> > >>> a RC2 to vote.
>>>> >> > >>>
>>>> >> > >>> Thanks !
>>>> >> > >>> Regards
>>>> >> > >>> JB
>>>> >> > >>>
>>>> >> > >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>>>> >> > >>>> Hi everyone,
>>>> >> > >>>>
>>>> >> > >>>> Please review and vote on the release candidate #1
>>>> for the version
>>>> >> > 2.3.0, as
>>>> >> > >>>> follows:
>>>> >> > >>>>
>>>> >> > >>>> [ ] +1, Approve the release
>>>> >> > >>>> [ ] -1, Do not approve the release (please provide
>>>> specific comments)
>>>> >> > >>>>
>>>> >> > >>>>
>>>> >> > >>>> The complete staging area is available for your
>>>> review, which
>>>> >> includes:
>>>> >> > >>>> * JIRA release notes [1],
>>>> >> > >>>> * the official Apache source release to be deployed to
>>>> >> dist.apache.org <http://dist.apache.org>
>>>> >> > <http://dist.apache.org>
>>>> >> > >>>> <http://dist.apache.org> [2],
>>>> >> > >>>> which is signed with the key with fingerprint
>>>> C8282E76 [3],
>>>> >> > >>>> * all artifacts to be deployed to the Maven Central
>>>> Repository [4],
>>>> >> > >>>> * source code tag "v2.3.0-RC1" [5],
>>>> >> > >>>> * website pull request listing the release and
>>>> publishing the API
>>>> >> reference
>>>> >> > >>>> manual [6].
>>>> >> > >>>> * Java artifacts were built with Maven 3.3.9 and
>>>> Oracle JDK
>>>> >> 1.8.0_111.
>>>> >> > >>>> * Python artifacts are deployed along with the source
>>>> release to the
>>>> >> > >>>> dist.apache.org <http://dist.apache.org> <
>>>> http://dist.apache.org>
>>>> >> <http://dist.apache.org> [2].
>>>> >> > >>>>
>>>> >> > >>>> The vote will be open for at least 72 hours. It is
>>>> adopted by majority approval,
>>>> >> > >>>> with at least 3 PMC affirmative votes.
>>>> >> > >>>>
>>>> >> > >>>> Thanks,
>>>> >> > >>>> JB
>>>> >> > >>>>
>>>> >> > >>>> [1]
>>>> >> > >>>>
>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>>>> >> <
>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>>>> >
>>>> >> > <
>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>>>> >> <
>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>>>> >>
>>>> >> > >>>> [2]
>>>> https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>>>> >> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
>>>> >> > <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>>>> >> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>
>>>> >> > >>>> [3]
>>>> https://dist.apache.org/repos/dist/release/beam/KEYS
>>>> >> <https://dist.apache.org/repos/dist/release/beam/KEYS>
>>>> >> > <https://dist.apache.org/repos/dist/release/beam/KEYS
>>>> >> <https://dist.apache.org/repos/dist/release/beam/KEYS>>
>>>> >> > >>>> [4]
>>>> https://repository.apache.org/content/repositories/orgapachebeam-1026/
>>>> >> <
>>>> https://repository.apache.org/content/repositories/orgapachebeam-1026/>
>>>> >> > <
>>>> https://repository.apache.org/content/repositories/orgapachebeam-1026/
>>>> >> <
>>>> https://repository.apache.org/content/repositories/orgapachebeam-1026/
>>>> >>
>>>> >> > >>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>>>> >> <https://github.com/apache/beam/tree/v2.3.0-RC1>
>>>> >> > <https://github.com/apache/beam/tree/v2.3.0-RC1
>>>> >> <https://github.com/apache/beam/tree/v2.3.0-RC1>>
>>>> >> > >>>> [6] https://github.com/apache/beam-site/pull/381
>>>> >> <https://github.com/apache/beam-site/pull/381>
>>>> >> > <https://github.com/apache/beam-site/pull/381
>>>> >> <https://github.com/apache/beam-site/pull/381>>
>>>> >> > >>>>
>>>> >> > >>>
>>>> >> > >>
>>>> >> > >> --
>>>> >> > >> Jean-Baptiste Onofré
>>>> >> > >> jbonofre@apache.org <ma...@apache.org>
>>>> <mailto:jbonofre@apache.org
>>>> >> <ma...@apache.org>>
>>>> >> > <mailto:jbonofre@apache.org <ma...@apache.org>
>>>> >> <mailto:jbonofre@apache.org <ma...@apache.org>>>
>>>> >> > >> http://blog.nanthrax.net
>>>> >> > >> Talend - http://www.talend.com
>>>> >> > >
>>>> >> >
>>>> >> > --
>>>> >> > Jean-Baptiste Onofré
>>>> >> > jbonofre@apache.org <ma...@apache.org>
>>>> >> <mailto:jbonofre@apache.org <ma...@apache.org>>
>>>> >> > http://blog.nanthrax.net
>>>> >> > Talend - http://www.talend.com
>>>> >> >
>>>> >> >
>>>> >>
>>>> >> --
>>>> >> Jean-Baptiste Onofré
>>>> >> jbonofre@apache.org <ma...@apache.org>
>>>> >> http://blog.nanthrax.net
>>>> >> Talend - http://www.talend.com
>>>> >>
>>>> >>
>>>> >
>>>>
>>>> --
>>>> Jean-Baptiste Onofré
>>>> jbonofre@apache.org
>>>> http://blog.nanthrax.net
>>>> Talend - http://www.talend.com
>>>>
>>>
>>>
>>
>
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Lukasz Cwik <lc...@google.com>.
+1 for rollback as well.
On Tue, Feb 6, 2018 at 12:37 PM, Reuven Lax <re...@google.com> wrote:
> +1 for the rollback.
>
> On Tue, Feb 6, 2018 at 9:24 AM, Kenneth Knowles <kl...@google.com> wrote:
>
>> +1 for option 1. It is almost certainly repeated proto parsing. That is
>> fixed for ParDo via generalized caching but not for other transforms. Flink
>> also has a proto round trip, but the design might avoid the problem anyhow.
>> We should investigate before releasing, or just preemptively roll it back
>> too. The proto round trips are helpful for maturing portability and for
>> preventing hacks from returning but otherwise don't matter so much yet.
>>
>> Kenn
>>
>> On Tue, Feb 6, 2018 at 8:54 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
>> wrote:
>>
>>> Hi all,
>>>
>>> As you might have seen, I found the cause of performance degradation on
>>> the
>>> direct runner (BEAM-3617).
>>>
>>> We have basically three options for RC2:
>>>
>>> 1. We revert the change for 2.3.0. I think it's the fastest and more
>>> secure way.
>>> I created a PR for that (https://github.com/apache/beam/pull/4609). It
>>> gives us
>>> time to investigate and provide a better fix in Runner API for 2.4.0.
>>>
>>> 2. If the fix is easy and with limited impact in Runner API, we do that.
>>>
>>> 3. We leave as it is, adding a note that direct runner has worse
>>> performance in
>>> 2.3.0 compared to 2.2.0.
>>>
>>> In any case, I would like to cut RC2 later tonight or tomorrow morning
>>> (my time).
>>>
>>> Thoughts ?
>>>
>>> Regards
>>> JB
>>>
>>> On 02/06/2018 09:42 AM, Jean-Baptiste Onofré wrote:
>>> > Hi Reuven,
>>> >
>>> > it's what I'm suspecting. git bisect should give us more information
>>> (still in
>>> > progress, 3 more steps to complete ;)). I keep you posted.
>>> >
>>> > Regards
>>> > JB
>>> >
>>> > On 02/06/2018 08:36 AM, Reuven Lax wrote:
>>> >> Could this be related to any of the portability changes?
>>> >>
>>> >> On Mon, Feb 5, 2018 at 7:51 AM, Jean-Baptiste Onofré <jb@nanthrax.net
>>> >> <ma...@nanthrax.net>> wrote:
>>> >>
>>> >> Created:
>>> >>
>>> >> https://issues.apache.org/jira/browse/BEAM-3617
>>> >> <https://issues.apache.org/jira/browse/BEAM-3617>
>>> >>
>>> >> Regards
>>> >> JB
>>> >>
>>> >> On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
>>> >> > What is the Jira for direct runner perf?
>>> >> >
>>> >> > On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré <
>>> jb@nanthrax.net <ma...@nanthrax.net>
>>> >> > <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
>>> >> >
>>> >> > Thanks !
>>> >> >
>>> >> > I cherry-pick on release-2.3.0 branch.
>>> >> >
>>> >> > I'm on the direct runner perf test in the mean time.
>>> >> >
>>> >> > Thanks again !
>>> >> >
>>> >> > Regards
>>> >> > JB
>>> >> >
>>> >> > On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
>>> >> > > I merged fixes for:
>>> >> > > - https://issues.apache.org/jira/browse/BEAM-3186
>>> >> <https://issues.apache.org/jira/browse/BEAM-3186>
>>> >> > <https://issues.apache.org/jira/browse/BEAM-3186
>>> >> <https://issues.apache.org/jira/browse/BEAM-3186>>
>>> >> > > - https://issues.apache.org/jira/browse/BEAM-3589
>>> >> <https://issues.apache.org/jira/browse/BEAM-3589>
>>> >> > <https://issues.apache.org/jira/browse/BEAM-3589
>>> >> <https://issues.apache.org/jira/browse/BEAM-3589>>
>>> >> > >
>>> >> > > @JB I didn't yet merge them on the 2.3.0 branch, though,
>>> but I can or you can go
>>> >> > > ahead.
>>> >> > >
>>> >> > >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <
>>> jb@nanthrax.net <ma...@nanthrax.net> <mailto:jb@nanthrax.net
>>> >> <ma...@nanthrax.net>>
>>> >> > >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>
>>> >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>>> wrote:
>>> >> > >>
>>> >> > >> Hi guys,
>>> >> > >>
>>> >> > >> Quick update on the RC2 preparation:
>>> >> > >>
>>> >> > >> * BEAM-3587 (TextIO with Flink) seems related to a
>>> custom build with
>>> >> > Gradle (not
>>> >> > >> using artifacts created by Maven). Anyway, I will take a
>>> look today.
>>> >> > >> * BEAM-3186 has a PR. Aljoscha will do the review pretty
>>> soon.
>>> >> > >> * I'm also taking a look on the direct runner
>>> performance change.
>>> >> > >>
>>> >> > >> The goal is to cut RC2 tomorrow or the day after.
>>> >> > >>
>>> >> > >> Regards
>>> >> > >> JB
>>> >> > >>
>>> >> > >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
>>> >> > >>> Hi guys,
>>> >> > >>>
>>> >> > >>> Especially due to BEAM-3587 & BEAM-3186 regressions, I
>>> cancel RC1.
>>> >> > >>>
>>> >> > >>> We will cherry-pick fixes on release-2.3.0 branch.
>>> >> > >>>
>>> >> > >>> I'm updating Jira right now. When the fixes will be
>>> cherry-picked, I
>>> >> > will submit
>>> >> > >>> a RC2 to vote.
>>> >> > >>>
>>> >> > >>> Thanks !
>>> >> > >>> Regards
>>> >> > >>> JB
>>> >> > >>>
>>> >> > >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>>> >> > >>>> Hi everyone,
>>> >> > >>>>
>>> >> > >>>> Please review and vote on the release candidate #1 for
>>> the version
>>> >> > 2.3.0, as
>>> >> > >>>> follows:
>>> >> > >>>>
>>> >> > >>>> [ ] +1, Approve the release
>>> >> > >>>> [ ] -1, Do not approve the release (please provide
>>> specific comments)
>>> >> > >>>>
>>> >> > >>>>
>>> >> > >>>> The complete staging area is available for your
>>> review, which
>>> >> includes:
>>> >> > >>>> * JIRA release notes [1],
>>> >> > >>>> * the official Apache source release to be deployed to
>>> >> dist.apache.org <http://dist.apache.org>
>>> >> > <http://dist.apache.org>
>>> >> > >>>> <http://dist.apache.org> [2],
>>> >> > >>>> which is signed with the key with fingerprint C8282E76
>>> [3],
>>> >> > >>>> * all artifacts to be deployed to the Maven Central
>>> Repository [4],
>>> >> > >>>> * source code tag "v2.3.0-RC1" [5],
>>> >> > >>>> * website pull request listing the release and
>>> publishing the API
>>> >> reference
>>> >> > >>>> manual [6].
>>> >> > >>>> * Java artifacts were built with Maven 3.3.9 and
>>> Oracle JDK
>>> >> 1.8.0_111.
>>> >> > >>>> * Python artifacts are deployed along with the source
>>> release to the
>>> >> > >>>> dist.apache.org <http://dist.apache.org> <
>>> http://dist.apache.org>
>>> >> <http://dist.apache.org> [2].
>>> >> > >>>>
>>> >> > >>>> The vote will be open for at least 72 hours. It is
>>> adopted by majority approval,
>>> >> > >>>> with at least 3 PMC affirmative votes.
>>> >> > >>>>
>>> >> > >>>> Thanks,
>>> >> > >>>> JB
>>> >> > >>>>
>>> >> > >>>> [1]
>>> >> > >>>> https://issues.apache.org/jira
>>> /secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>>> >> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?pro
>>> jectId=12319527&version=12341608>
>>> >> > <https://issues.apache.org/jira/secure/ReleaseNote.jspa?pro
>>> jectId=12319527&version=12341608
>>> >> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?pro
>>> jectId=12319527&version=12341608>>
>>> >> > >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>>> >> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
>>> >> > <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>>> >> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>
>>> >> > >>>> [3] https://dist.apache.org/repos/
>>> dist/release/beam/KEYS
>>> >> <https://dist.apache.org/repos/dist/release/beam/KEYS>
>>> >> > <https://dist.apache.org/repos/dist/release/beam/KEYS
>>> >> <https://dist.apache.org/repos/dist/release/beam/KEYS>>
>>> >> > >>>> [4] https://repository.apache.org/
>>> content/repositories/orgapachebeam-1026/
>>> >> <https://repository.apache.org/content/repositories/orgapac
>>> hebeam-1026/>
>>> >> > <https://repository.apache.org/content/repositories/orgapac
>>> hebeam-1026/
>>> >> <https://repository.apache.org/content/repositories/orgapac
>>> hebeam-1026/>>
>>> >> > >>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>>> >> <https://github.com/apache/beam/tree/v2.3.0-RC1>
>>> >> > <https://github.com/apache/beam/tree/v2.3.0-RC1
>>> >> <https://github.com/apache/beam/tree/v2.3.0-RC1>>
>>> >> > >>>> [6] https://github.com/apache/beam-site/pull/381
>>> >> <https://github.com/apache/beam-site/pull/381>
>>> >> > <https://github.com/apache/beam-site/pull/381
>>> >> <https://github.com/apache/beam-site/pull/381>>
>>> >> > >>>>
>>> >> > >>>
>>> >> > >>
>>> >> > >> --
>>> >> > >> Jean-Baptiste Onofré
>>> >> > >> jbonofre@apache.org <ma...@apache.org>
>>> <mailto:jbonofre@apache.org
>>> >> <ma...@apache.org>>
>>> >> > <mailto:jbonofre@apache.org <ma...@apache.org>
>>> >> <mailto:jbonofre@apache.org <ma...@apache.org>>>
>>> >> > >> http://blog.nanthrax.net
>>> >> > >> Talend - http://www.talend.com
>>> >> > >
>>> >> >
>>> >> > --
>>> >> > Jean-Baptiste Onofré
>>> >> > jbonofre@apache.org <ma...@apache.org>
>>> >> <mailto:jbonofre@apache.org <ma...@apache.org>>
>>> >> > http://blog.nanthrax.net
>>> >> > Talend - http://www.talend.com
>>> >> >
>>> >> >
>>> >>
>>> >> --
>>> >> Jean-Baptiste Onofré
>>> >> jbonofre@apache.org <ma...@apache.org>
>>> >> http://blog.nanthrax.net
>>> >> Talend - http://www.talend.com
>>> >>
>>> >>
>>> >
>>>
>>> --
>>> Jean-Baptiste Onofré
>>> jbonofre@apache.org
>>> http://blog.nanthrax.net
>>> Talend - http://www.talend.com
>>>
>>
>>
>
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Reuven Lax <re...@google.com>.
+1 for the rollback.
On Tue, Feb 6, 2018 at 9:24 AM, Kenneth Knowles <kl...@google.com> wrote:
> +1 for option 1. It is almost certainly repeated proto parsing. That is
> fixed for ParDo via generalized caching but not for other transforms. Flink
> also has a proto round trip, but the design might avoid the problem anyhow.
> We should investigate before releasing, or just preemptively roll it back
> too. The proto round trips are helpful for maturing portability and for
> preventing hacks from returning but otherwise don't matter so much yet.
>
> Kenn
>
> On Tue, Feb 6, 2018 at 8:54 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
>
>> Hi all,
>>
>> As you might have seen, I found the cause of performance degradation on
>> the
>> direct runner (BEAM-3617).
>>
>> We have basically three options for RC2:
>>
>> 1. We revert the change for 2.3.0. I think it's the fastest and more
>> secure way.
>> I created a PR for that (https://github.com/apache/beam/pull/4609). It
>> gives us
>> time to investigate and provide a better fix in Runner API for 2.4.0.
>>
>> 2. If the fix is easy and with limited impact in Runner API, we do that.
>>
>> 3. We leave as it is, adding a note that direct runner has worse
>> performance in
>> 2.3.0 compared to 2.2.0.
>>
>> In any case, I would like to cut RC2 later tonight or tomorrow morning
>> (my time).
>>
>> Thoughts ?
>>
>> Regards
>> JB
>>
>> On 02/06/2018 09:42 AM, Jean-Baptiste Onofré wrote:
>> > Hi Reuven,
>> >
>> > it's what I'm suspecting. git bisect should give us more information
>> (still in
>> > progress, 3 more steps to complete ;)). I keep you posted.
>> >
>> > Regards
>> > JB
>> >
>> > On 02/06/2018 08:36 AM, Reuven Lax wrote:
>> >> Could this be related to any of the portability changes?
>> >>
>> >> On Mon, Feb 5, 2018 at 7:51 AM, Jean-Baptiste Onofré <jb@nanthrax.net
>> >> <ma...@nanthrax.net>> wrote:
>> >>
>> >> Created:
>> >>
>> >> https://issues.apache.org/jira/browse/BEAM-3617
>> >> <https://issues.apache.org/jira/browse/BEAM-3617>
>> >>
>> >> Regards
>> >> JB
>> >>
>> >> On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
>> >> > What is the Jira for direct runner perf?
>> >> >
>> >> > On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré <
>> jb@nanthrax.net <ma...@nanthrax.net>
>> >> > <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
>> >> >
>> >> > Thanks !
>> >> >
>> >> > I cherry-pick on release-2.3.0 branch.
>> >> >
>> >> > I'm on the direct runner perf test in the mean time.
>> >> >
>> >> > Thanks again !
>> >> >
>> >> > Regards
>> >> > JB
>> >> >
>> >> > On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
>> >> > > I merged fixes for:
>> >> > > - https://issues.apache.org/jira/browse/BEAM-3186
>> >> <https://issues.apache.org/jira/browse/BEAM-3186>
>> >> > <https://issues.apache.org/jira/browse/BEAM-3186
>> >> <https://issues.apache.org/jira/browse/BEAM-3186>>
>> >> > > - https://issues.apache.org/jira/browse/BEAM-3589
>> >> <https://issues.apache.org/jira/browse/BEAM-3589>
>> >> > <https://issues.apache.org/jira/browse/BEAM-3589
>> >> <https://issues.apache.org/jira/browse/BEAM-3589>>
>> >> > >
>> >> > > @JB I didn't yet merge them on the 2.3.0 branch, though,
>> but I can or you can go
>> >> > > ahead.
>> >> > >
>> >> > >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <
>> jb@nanthrax.net <ma...@nanthrax.net> <mailto:jb@nanthrax.net
>> >> <ma...@nanthrax.net>>
>> >> > >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>
>> >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>>> wrote:
>> >> > >>
>> >> > >> Hi guys,
>> >> > >>
>> >> > >> Quick update on the RC2 preparation:
>> >> > >>
>> >> > >> * BEAM-3587 (TextIO with Flink) seems related to a custom
>> build with
>> >> > Gradle (not
>> >> > >> using artifacts created by Maven). Anyway, I will take a
>> look today.
>> >> > >> * BEAM-3186 has a PR. Aljoscha will do the review pretty
>> soon.
>> >> > >> * I'm also taking a look on the direct runner performance
>> change.
>> >> > >>
>> >> > >> The goal is to cut RC2 tomorrow or the day after.
>> >> > >>
>> >> > >> Regards
>> >> > >> JB
>> >> > >>
>> >> > >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
>> >> > >>> Hi guys,
>> >> > >>>
>> >> > >>> Especially due to BEAM-3587 & BEAM-3186 regressions, I
>> cancel RC1.
>> >> > >>>
>> >> > >>> We will cherry-pick fixes on release-2.3.0 branch.
>> >> > >>>
>> >> > >>> I'm updating Jira right now. When the fixes will be
>> cherry-picked, I
>> >> > will submit
>> >> > >>> a RC2 to vote.
>> >> > >>>
>> >> > >>> Thanks !
>> >> > >>> Regards
>> >> > >>> JB
>> >> > >>>
>> >> > >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>> >> > >>>> Hi everyone,
>> >> > >>>>
>> >> > >>>> Please review and vote on the release candidate #1 for
>> the version
>> >> > 2.3.0, as
>> >> > >>>> follows:
>> >> > >>>>
>> >> > >>>> [ ] +1, Approve the release
>> >> > >>>> [ ] -1, Do not approve the release (please provide
>> specific comments)
>> >> > >>>>
>> >> > >>>>
>> >> > >>>> The complete staging area is available for your review,
>> which
>> >> includes:
>> >> > >>>> * JIRA release notes [1],
>> >> > >>>> * the official Apache source release to be deployed to
>> >> dist.apache.org <http://dist.apache.org>
>> >> > <http://dist.apache.org>
>> >> > >>>> <http://dist.apache.org> [2],
>> >> > >>>> which is signed with the key with fingerprint C8282E76
>> [3],
>> >> > >>>> * all artifacts to be deployed to the Maven Central
>> Repository [4],
>> >> > >>>> * source code tag "v2.3.0-RC1" [5],
>> >> > >>>> * website pull request listing the release and
>> publishing the API
>> >> reference
>> >> > >>>> manual [6].
>> >> > >>>> * Java artifacts were built with Maven 3.3.9 and Oracle
>> JDK
>> >> 1.8.0_111.
>> >> > >>>> * Python artifacts are deployed along with the source
>> release to the
>> >> > >>>> dist.apache.org <http://dist.apache.org> <
>> http://dist.apache.org>
>> >> <http://dist.apache.org> [2].
>> >> > >>>>
>> >> > >>>> The vote will be open for at least 72 hours. It is
>> adopted by majority approval,
>> >> > >>>> with at least 3 PMC affirmative votes.
>> >> > >>>>
>> >> > >>>> Thanks,
>> >> > >>>> JB
>> >> > >>>>
>> >> > >>>> [1]
>> >> > >>>> https://issues.apache.org/jira
>> /secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>> >> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?pro
>> jectId=12319527&version=12341608>
>> >> > <https://issues.apache.org/jira/secure/ReleaseNote.jspa?pro
>> jectId=12319527&version=12341608
>> >> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?pro
>> jectId=12319527&version=12341608>>
>> >> > >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>> >> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
>> >> > <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>> >> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>
>> >> > >>>> [3] https://dist.apache.org/repos/
>> dist/release/beam/KEYS
>> >> <https://dist.apache.org/repos/dist/release/beam/KEYS>
>> >> > <https://dist.apache.org/repos/dist/release/beam/KEYS
>> >> <https://dist.apache.org/repos/dist/release/beam/KEYS>>
>> >> > >>>> [4] https://repository.apache.org/
>> content/repositories/orgapachebeam-1026/
>> >> <https://repository.apache.org/content/repositories/orgapac
>> hebeam-1026/>
>> >> > <https://repository.apache.org/content/repositories/orgapac
>> hebeam-1026/
>> >> <https://repository.apache.org/content/repositories/orgapac
>> hebeam-1026/>>
>> >> > >>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>> >> <https://github.com/apache/beam/tree/v2.3.0-RC1>
>> >> > <https://github.com/apache/beam/tree/v2.3.0-RC1
>> >> <https://github.com/apache/beam/tree/v2.3.0-RC1>>
>> >> > >>>> [6] https://github.com/apache/beam-site/pull/381
>> >> <https://github.com/apache/beam-site/pull/381>
>> >> > <https://github.com/apache/beam-site/pull/381
>> >> <https://github.com/apache/beam-site/pull/381>>
>> >> > >>>>
>> >> > >>>
>> >> > >>
>> >> > >> --
>> >> > >> Jean-Baptiste Onofré
>> >> > >> jbonofre@apache.org <ma...@apache.org> <mailto:
>> jbonofre@apache.org
>> >> <ma...@apache.org>>
>> >> > <mailto:jbonofre@apache.org <ma...@apache.org>
>> >> <mailto:jbonofre@apache.org <ma...@apache.org>>>
>> >> > >> http://blog.nanthrax.net
>> >> > >> Talend - http://www.talend.com
>> >> > >
>> >> >
>> >> > --
>> >> > Jean-Baptiste Onofré
>> >> > jbonofre@apache.org <ma...@apache.org>
>> >> <mailto:jbonofre@apache.org <ma...@apache.org>>
>> >> > http://blog.nanthrax.net
>> >> > Talend - http://www.talend.com
>> >> >
>> >> >
>> >>
>> >> --
>> >> Jean-Baptiste Onofré
>> >> jbonofre@apache.org <ma...@apache.org>
>> >> http://blog.nanthrax.net
>> >> Talend - http://www.talend.com
>> >>
>> >>
>> >
>>
>> --
>> Jean-Baptiste Onofré
>> jbonofre@apache.org
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>
>
>
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Kenneth Knowles <kl...@google.com>.
+1 for option 1. It is almost certainly repeated proto parsing. That is
fixed for ParDo via generalized caching but not for other transforms. Flink
also has a proto round trip, but the design might avoid the problem anyhow.
We should investigate before releasing, or just preemptively roll it back
too. The proto round trips are helpful for maturing portability and for
preventing hacks from returning but otherwise don't matter so much yet.
Kenn
On Tue, Feb 6, 2018 at 8:54 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> Hi all,
>
> As you might have seen, I found the cause of performance degradation on the
> direct runner (BEAM-3617).
>
> We have basically three options for RC2:
>
> 1. We revert the change for 2.3.0. I think it's the fastest and more
> secure way.
> I created a PR for that (https://github.com/apache/beam/pull/4609). It
> gives us
> time to investigate and provide a better fix in Runner API for 2.4.0.
>
> 2. If the fix is easy and with limited impact in Runner API, we do that.
>
> 3. We leave as it is, adding a note that direct runner has worse
> performance in
> 2.3.0 compared to 2.2.0.
>
> In any case, I would like to cut RC2 later tonight or tomorrow morning (my
> time).
>
> Thoughts ?
>
> Regards
> JB
>
> On 02/06/2018 09:42 AM, Jean-Baptiste Onofré wrote:
> > Hi Reuven,
> >
> > it's what I'm suspecting. git bisect should give us more information
> (still in
> > progress, 3 more steps to complete ;)). I keep you posted.
> >
> > Regards
> > JB
> >
> > On 02/06/2018 08:36 AM, Reuven Lax wrote:
> >> Could this be related to any of the portability changes?
> >>
> >> On Mon, Feb 5, 2018 at 7:51 AM, Jean-Baptiste Onofré <jb@nanthrax.net
> >> <ma...@nanthrax.net>> wrote:
> >>
> >> Created:
> >>
> >> https://issues.apache.org/jira/browse/BEAM-3617
> >> <https://issues.apache.org/jira/browse/BEAM-3617>
> >>
> >> Regards
> >> JB
> >>
> >> On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
> >> > What is the Jira for direct runner perf?
> >> >
> >> > On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré <
> jb@nanthrax.net <ma...@nanthrax.net>
> >> > <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
> >> >
> >> > Thanks !
> >> >
> >> > I cherry-pick on release-2.3.0 branch.
> >> >
> >> > I'm on the direct runner perf test in the mean time.
> >> >
> >> > Thanks again !
> >> >
> >> > Regards
> >> > JB
> >> >
> >> > On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
> >> > > I merged fixes for:
> >> > > - https://issues.apache.org/jira/browse/BEAM-3186
> >> <https://issues.apache.org/jira/browse/BEAM-3186>
> >> > <https://issues.apache.org/jira/browse/BEAM-3186
> >> <https://issues.apache.org/jira/browse/BEAM-3186>>
> >> > > - https://issues.apache.org/jira/browse/BEAM-3589
> >> <https://issues.apache.org/jira/browse/BEAM-3589>
> >> > <https://issues.apache.org/jira/browse/BEAM-3589
> >> <https://issues.apache.org/jira/browse/BEAM-3589>>
> >> > >
> >> > > @JB I didn't yet merge them on the 2.3.0 branch, though,
> but I can or you can go
> >> > > ahead.
> >> > >
> >> > >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <
> jb@nanthrax.net <ma...@nanthrax.net> <mailto:jb@nanthrax.net
> >> <ma...@nanthrax.net>>
> >> > >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>
> >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>>> wrote:
> >> > >>
> >> > >> Hi guys,
> >> > >>
> >> > >> Quick update on the RC2 preparation:
> >> > >>
> >> > >> * BEAM-3587 (TextIO with Flink) seems related to a custom
> build with
> >> > Gradle (not
> >> > >> using artifacts created by Maven). Anyway, I will take a
> look today.
> >> > >> * BEAM-3186 has a PR. Aljoscha will do the review pretty
> soon.
> >> > >> * I'm also taking a look on the direct runner performance
> change.
> >> > >>
> >> > >> The goal is to cut RC2 tomorrow or the day after.
> >> > >>
> >> > >> Regards
> >> > >> JB
> >> > >>
> >> > >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
> >> > >>> Hi guys,
> >> > >>>
> >> > >>> Especially due to BEAM-3587 & BEAM-3186 regressions, I
> cancel RC1.
> >> > >>>
> >> > >>> We will cherry-pick fixes on release-2.3.0 branch.
> >> > >>>
> >> > >>> I'm updating Jira right now. When the fixes will be
> cherry-picked, I
> >> > will submit
> >> > >>> a RC2 to vote.
> >> > >>>
> >> > >>> Thanks !
> >> > >>> Regards
> >> > >>> JB
> >> > >>>
> >> > >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
> >> > >>>> Hi everyone,
> >> > >>>>
> >> > >>>> Please review and vote on the release candidate #1 for
> the version
> >> > 2.3.0, as
> >> > >>>> follows:
> >> > >>>>
> >> > >>>> [ ] +1, Approve the release
> >> > >>>> [ ] -1, Do not approve the release (please provide
> specific comments)
> >> > >>>>
> >> > >>>>
> >> > >>>> The complete staging area is available for your review,
> which
> >> includes:
> >> > >>>> * JIRA release notes [1],
> >> > >>>> * the official Apache source release to be deployed to
> >> dist.apache.org <http://dist.apache.org>
> >> > <http://dist.apache.org>
> >> > >>>> <http://dist.apache.org> [2],
> >> > >>>> which is signed with the key with fingerprint C8282E76
> [3],
> >> > >>>> * all artifacts to be deployed to the Maven Central
> Repository [4],
> >> > >>>> * source code tag "v2.3.0-RC1" [5],
> >> > >>>> * website pull request listing the release and
> publishing the API
> >> reference
> >> > >>>> manual [6].
> >> > >>>> * Java artifacts were built with Maven 3.3.9 and Oracle
> JDK
> >> 1.8.0_111.
> >> > >>>> * Python artifacts are deployed along with the source
> release to the
> >> > >>>> dist.apache.org <http://dist.apache.org> <
> http://dist.apache.org>
> >> <http://dist.apache.org> [2].
> >> > >>>>
> >> > >>>> The vote will be open for at least 72 hours. It is
> adopted by majority approval,
> >> > >>>> with at least 3 PMC affirmative votes.
> >> > >>>>
> >> > >>>> Thanks,
> >> > >>>> JB
> >> > >>>>
> >> > >>>> [1]
> >> > >>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12319527&version=12341608
> >> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12319527&version=12341608>
> >> > <https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12319527&version=12341608
> >> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12319527&version=12341608>>
> >> > >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> >> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
> >> > <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> >> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>
> >> > >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> >> <https://dist.apache.org/repos/dist/release/beam/KEYS>
> >> > <https://dist.apache.org/repos/dist/release/beam/KEYS
> >> <https://dist.apache.org/repos/dist/release/beam/KEYS>>
> >> > >>>> [4] https://repository.apache.org/content/repositories/
> orgapachebeam-1026/
> >> <https://repository.apache.org/content/repositories/
> orgapachebeam-1026/>
> >> > <https://repository.apache.org/content/repositories/
> orgapachebeam-1026/
> >> <https://repository.apache.org/content/repositories/
> orgapachebeam-1026/>>
> >> > >>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
> >> <https://github.com/apache/beam/tree/v2.3.0-RC1>
> >> > <https://github.com/apache/beam/tree/v2.3.0-RC1
> >> <https://github.com/apache/beam/tree/v2.3.0-RC1>>
> >> > >>>> [6] https://github.com/apache/beam-site/pull/381
> >> <https://github.com/apache/beam-site/pull/381>
> >> > <https://github.com/apache/beam-site/pull/381
> >> <https://github.com/apache/beam-site/pull/381>>
> >> > >>>>
> >> > >>>
> >> > >>
> >> > >> --
> >> > >> Jean-Baptiste Onofré
> >> > >> jbonofre@apache.org <ma...@apache.org> <mailto:
> jbonofre@apache.org
> >> <ma...@apache.org>>
> >> > <mailto:jbonofre@apache.org <ma...@apache.org>
> >> <mailto:jbonofre@apache.org <ma...@apache.org>>>
> >> > >> http://blog.nanthrax.net
> >> > >> Talend - http://www.talend.com
> >> > >
> >> >
> >> > --
> >> > Jean-Baptiste Onofré
> >> > jbonofre@apache.org <ma...@apache.org>
> >> <mailto:jbonofre@apache.org <ma...@apache.org>>
> >> > http://blog.nanthrax.net
> >> > Talend - http://www.talend.com
> >> >
> >> >
> >>
> >> --
> >> Jean-Baptiste Onofré
> >> jbonofre@apache.org <ma...@apache.org>
> >> http://blog.nanthrax.net
> >> Talend - http://www.talend.com
> >>
> >>
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi all,
As you might have seen, I found the cause of performance degradation on the
direct runner (BEAM-3617).
We have basically three options for RC2:
1. We revert the change for 2.3.0. I think it's the fastest and more secure way.
I created a PR for that (https://github.com/apache/beam/pull/4609). It gives us
time to investigate and provide a better fix in Runner API for 2.4.0.
2. If the fix is easy and with limited impact in Runner API, we do that.
3. We leave as it is, adding a note that direct runner has worse performance in
2.3.0 compared to 2.2.0.
In any case, I would like to cut RC2 later tonight or tomorrow morning (my time).
Thoughts ?
Regards
JB
On 02/06/2018 09:42 AM, Jean-Baptiste Onofré wrote:
> Hi Reuven,
>
> it's what I'm suspecting. git bisect should give us more information (still in
> progress, 3 more steps to complete ;)). I keep you posted.
>
> Regards
> JB
>
> On 02/06/2018 08:36 AM, Reuven Lax wrote:
>> Could this be related to any of the portability changes?
>>
>> On Mon, Feb 5, 2018 at 7:51 AM, Jean-Baptiste Onofré <jb@nanthrax.net
>> <ma...@nanthrax.net>> wrote:
>>
>> Created:
>>
>> https://issues.apache.org/jira/browse/BEAM-3617
>> <https://issues.apache.org/jira/browse/BEAM-3617>
>>
>> Regards
>> JB
>>
>> On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
>> > What is the Jira for direct runner perf?
>> >
>> > On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré <jb@nanthrax.net <ma...@nanthrax.net>
>> > <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
>> >
>> > Thanks !
>> >
>> > I cherry-pick on release-2.3.0 branch.
>> >
>> > I'm on the direct runner perf test in the mean time.
>> >
>> > Thanks again !
>> >
>> > Regards
>> > JB
>> >
>> > On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
>> > > I merged fixes for:
>> > > - https://issues.apache.org/jira/browse/BEAM-3186
>> <https://issues.apache.org/jira/browse/BEAM-3186>
>> > <https://issues.apache.org/jira/browse/BEAM-3186
>> <https://issues.apache.org/jira/browse/BEAM-3186>>
>> > > - https://issues.apache.org/jira/browse/BEAM-3589
>> <https://issues.apache.org/jira/browse/BEAM-3589>
>> > <https://issues.apache.org/jira/browse/BEAM-3589
>> <https://issues.apache.org/jira/browse/BEAM-3589>>
>> > >
>> > > @JB I didn't yet merge them on the 2.3.0 branch, though, but I can or you can go
>> > > ahead.
>> > >
>> > >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <jb@nanthrax.net <ma...@nanthrax.net> <mailto:jb@nanthrax.net
>> <ma...@nanthrax.net>>
>> > >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>
>> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>>> wrote:
>> > >>
>> > >> Hi guys,
>> > >>
>> > >> Quick update on the RC2 preparation:
>> > >>
>> > >> * BEAM-3587 (TextIO with Flink) seems related to a custom build with
>> > Gradle (not
>> > >> using artifacts created by Maven). Anyway, I will take a look today.
>> > >> * BEAM-3186 has a PR. Aljoscha will do the review pretty soon.
>> > >> * I'm also taking a look on the direct runner performance change.
>> > >>
>> > >> The goal is to cut RC2 tomorrow or the day after.
>> > >>
>> > >> Regards
>> > >> JB
>> > >>
>> > >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
>> > >>> Hi guys,
>> > >>>
>> > >>> Especially due to BEAM-3587 & BEAM-3186 regressions, I cancel RC1.
>> > >>>
>> > >>> We will cherry-pick fixes on release-2.3.0 branch.
>> > >>>
>> > >>> I'm updating Jira right now. When the fixes will be cherry-picked, I
>> > will submit
>> > >>> a RC2 to vote.
>> > >>>
>> > >>> Thanks !
>> > >>> Regards
>> > >>> JB
>> > >>>
>> > >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>> > >>>> Hi everyone,
>> > >>>>
>> > >>>> Please review and vote on the release candidate #1 for the version
>> > 2.3.0, as
>> > >>>> follows:
>> > >>>>
>> > >>>> [ ] +1, Approve the release
>> > >>>> [ ] -1, Do not approve the release (please provide specific comments)
>> > >>>>
>> > >>>>
>> > >>>> The complete staging area is available for your review, which
>> includes:
>> > >>>> * JIRA release notes [1],
>> > >>>> * the official Apache source release to be deployed to
>> dist.apache.org <http://dist.apache.org>
>> > <http://dist.apache.org>
>> > >>>> <http://dist.apache.org> [2],
>> > >>>> which is signed with the key with fingerprint C8282E76 [3],
>> > >>>> * all artifacts to be deployed to the Maven Central Repository [4],
>> > >>>> * source code tag "v2.3.0-RC1" [5],
>> > >>>> * website pull request listing the release and publishing the API
>> reference
>> > >>>> manual [6].
>> > >>>> * Java artifacts were built with Maven 3.3.9 and Oracle JDK
>> 1.8.0_111.
>> > >>>> * Python artifacts are deployed along with the source release to the
>> > >>>> dist.apache.org <http://dist.apache.org> <http://dist.apache.org>
>> <http://dist.apache.org> [2].
>> > >>>>
>> > >>>> The vote will be open for at least 72 hours. It is adopted by majority approval,
>> > >>>> with at least 3 PMC affirmative votes.
>> > >>>>
>> > >>>> Thanks,
>> > >>>> JB
>> > >>>>
>> > >>>> [1]
>> > >>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>
>> > <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>>
>> > >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
>> > <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>
>> > >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>> <https://dist.apache.org/repos/dist/release/beam/KEYS>
>> > <https://dist.apache.org/repos/dist/release/beam/KEYS
>> <https://dist.apache.org/repos/dist/release/beam/KEYS>>
>> > >>>> [4] https://repository.apache.org/content/repositories/orgapachebeam-1026/
>> <https://repository.apache.org/content/repositories/orgapachebeam-1026/>
>> > <https://repository.apache.org/content/repositories/orgapachebeam-1026/
>> <https://repository.apache.org/content/repositories/orgapachebeam-1026/>>
>> > >>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>> <https://github.com/apache/beam/tree/v2.3.0-RC1>
>> > <https://github.com/apache/beam/tree/v2.3.0-RC1
>> <https://github.com/apache/beam/tree/v2.3.0-RC1>>
>> > >>>> [6] https://github.com/apache/beam-site/pull/381
>> <https://github.com/apache/beam-site/pull/381>
>> > <https://github.com/apache/beam-site/pull/381
>> <https://github.com/apache/beam-site/pull/381>>
>> > >>>>
>> > >>>
>> > >>
>> > >> --
>> > >> Jean-Baptiste Onofré
>> > >> jbonofre@apache.org <ma...@apache.org> <mailto:jbonofre@apache.org
>> <ma...@apache.org>>
>> > <mailto:jbonofre@apache.org <ma...@apache.org>
>> <mailto:jbonofre@apache.org <ma...@apache.org>>>
>> > >> http://blog.nanthrax.net
>> > >> Talend - http://www.talend.com
>> > >
>> >
>> > --
>> > Jean-Baptiste Onofré
>> > jbonofre@apache.org <ma...@apache.org>
>> <mailto:jbonofre@apache.org <ma...@apache.org>>
>> > http://blog.nanthrax.net
>> > Talend - http://www.talend.com
>> >
>> >
>>
>> --
>> Jean-Baptiste Onofré
>> jbonofre@apache.org <ma...@apache.org>
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>
>>
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi Reuven,
it's what I'm suspecting. git bisect should give us more information (still in
progress, 3 more steps to complete ;)). I keep you posted.
Regards
JB
On 02/06/2018 08:36 AM, Reuven Lax wrote:
> Could this be related to any of the portability changes?
>
> On Mon, Feb 5, 2018 at 7:51 AM, Jean-Baptiste Onofré <jb@nanthrax.net
> <ma...@nanthrax.net>> wrote:
>
> Created:
>
> https://issues.apache.org/jira/browse/BEAM-3617
> <https://issues.apache.org/jira/browse/BEAM-3617>
>
> Regards
> JB
>
> On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
> > What is the Jira for direct runner perf?
> >
> > On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré <jb@nanthrax.net <ma...@nanthrax.net>
> > <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
> >
> > Thanks !
> >
> > I cherry-pick on release-2.3.0 branch.
> >
> > I'm on the direct runner perf test in the mean time.
> >
> > Thanks again !
> >
> > Regards
> > JB
> >
> > On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
> > > I merged fixes for:
> > > - https://issues.apache.org/jira/browse/BEAM-3186
> <https://issues.apache.org/jira/browse/BEAM-3186>
> > <https://issues.apache.org/jira/browse/BEAM-3186
> <https://issues.apache.org/jira/browse/BEAM-3186>>
> > > - https://issues.apache.org/jira/browse/BEAM-3589
> <https://issues.apache.org/jira/browse/BEAM-3589>
> > <https://issues.apache.org/jira/browse/BEAM-3589
> <https://issues.apache.org/jira/browse/BEAM-3589>>
> > >
> > > @JB I didn't yet merge them on the 2.3.0 branch, though, but I can or you can go
> > > ahead.
> > >
> > >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <jb@nanthrax.net <ma...@nanthrax.net> <mailto:jb@nanthrax.net
> <ma...@nanthrax.net>>
> > >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>
> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>>> wrote:
> > >>
> > >> Hi guys,
> > >>
> > >> Quick update on the RC2 preparation:
> > >>
> > >> * BEAM-3587 (TextIO with Flink) seems related to a custom build with
> > Gradle (not
> > >> using artifacts created by Maven). Anyway, I will take a look today.
> > >> * BEAM-3186 has a PR. Aljoscha will do the review pretty soon.
> > >> * I'm also taking a look on the direct runner performance change.
> > >>
> > >> The goal is to cut RC2 tomorrow or the day after.
> > >>
> > >> Regards
> > >> JB
> > >>
> > >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
> > >>> Hi guys,
> > >>>
> > >>> Especially due to BEAM-3587 & BEAM-3186 regressions, I cancel RC1.
> > >>>
> > >>> We will cherry-pick fixes on release-2.3.0 branch.
> > >>>
> > >>> I'm updating Jira right now. When the fixes will be cherry-picked, I
> > will submit
> > >>> a RC2 to vote.
> > >>>
> > >>> Thanks !
> > >>> Regards
> > >>> JB
> > >>>
> > >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
> > >>>> Hi everyone,
> > >>>>
> > >>>> Please review and vote on the release candidate #1 for the version
> > 2.3.0, as
> > >>>> follows:
> > >>>>
> > >>>> [ ] +1, Approve the release
> > >>>> [ ] -1, Do not approve the release (please provide specific comments)
> > >>>>
> > >>>>
> > >>>> The complete staging area is available for your review, which
> includes:
> > >>>> * JIRA release notes [1],
> > >>>> * the official Apache source release to be deployed to
> dist.apache.org <http://dist.apache.org>
> > <http://dist.apache.org>
> > >>>> <http://dist.apache.org> [2],
> > >>>> which is signed with the key with fingerprint C8282E76 [3],
> > >>>> * all artifacts to be deployed to the Maven Central Repository [4],
> > >>>> * source code tag "v2.3.0-RC1" [5],
> > >>>> * website pull request listing the release and publishing the API
> reference
> > >>>> manual [6].
> > >>>> * Java artifacts were built with Maven 3.3.9 and Oracle JDK
> 1.8.0_111.
> > >>>> * Python artifacts are deployed along with the source release to the
> > >>>> dist.apache.org <http://dist.apache.org> <http://dist.apache.org>
> <http://dist.apache.org> [2].
> > >>>>
> > >>>> The vote will be open for at least 72 hours. It is adopted by majority approval,
> > >>>> with at least 3 PMC affirmative votes.
> > >>>>
> > >>>> Thanks,
> > >>>> JB
> > >>>>
> > >>>> [1]
> > >>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>
> > <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>>
> > >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
> > <https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>
> > >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> <https://dist.apache.org/repos/dist/release/beam/KEYS>
> > <https://dist.apache.org/repos/dist/release/beam/KEYS
> <https://dist.apache.org/repos/dist/release/beam/KEYS>>
> > >>>> [4] https://repository.apache.org/content/repositories/orgapachebeam-1026/
> <https://repository.apache.org/content/repositories/orgapachebeam-1026/>
> > <https://repository.apache.org/content/repositories/orgapachebeam-1026/
> <https://repository.apache.org/content/repositories/orgapachebeam-1026/>>
> > >>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
> <https://github.com/apache/beam/tree/v2.3.0-RC1>
> > <https://github.com/apache/beam/tree/v2.3.0-RC1
> <https://github.com/apache/beam/tree/v2.3.0-RC1>>
> > >>>> [6] https://github.com/apache/beam-site/pull/381
> <https://github.com/apache/beam-site/pull/381>
> > <https://github.com/apache/beam-site/pull/381
> <https://github.com/apache/beam-site/pull/381>>
> > >>>>
> > >>>
> > >>
> > >> --
> > >> Jean-Baptiste Onofré
> > >> jbonofre@apache.org <ma...@apache.org> <mailto:jbonofre@apache.org
> <ma...@apache.org>>
> > <mailto:jbonofre@apache.org <ma...@apache.org>
> <mailto:jbonofre@apache.org <ma...@apache.org>>>
> > >> http://blog.nanthrax.net
> > >> Talend - http://www.talend.com
> > >
> >
> > --
> > Jean-Baptiste Onofré
> > jbonofre@apache.org <ma...@apache.org>
> <mailto:jbonofre@apache.org <ma...@apache.org>>
> > http://blog.nanthrax.net
> > Talend - http://www.talend.com
> >
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org <ma...@apache.org>
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Reuven Lax <re...@google.com>.
Could this be related to any of the portability changes?
On Mon, Feb 5, 2018 at 7:51 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> Created:
>
> https://issues.apache.org/jira/browse/BEAM-3617
>
> Regards
> JB
>
> On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
> > What is the Jira for direct runner perf?
> >
> > On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré <jb@nanthrax.net
> > <ma...@nanthrax.net>> wrote:
> >
> > Thanks !
> >
> > I cherry-pick on release-2.3.0 branch.
> >
> > I'm on the direct runner perf test in the mean time.
> >
> > Thanks again !
> >
> > Regards
> > JB
> >
> > On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
> > > I merged fixes for:
> > > - https://issues.apache.org/jira/browse/BEAM-3186
> > <https://issues.apache.org/jira/browse/BEAM-3186>
> > > - https://issues.apache.org/jira/browse/BEAM-3589
> > <https://issues.apache.org/jira/browse/BEAM-3589>
> > >
> > > @JB I didn't yet merge them on the 2.3.0 branch, though, but I can
> or you can go
> > > ahead.
> > >
> > >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <jb@nanthrax.net
> <ma...@nanthrax.net>
> > >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
> > >>
> > >> Hi guys,
> > >>
> > >> Quick update on the RC2 preparation:
> > >>
> > >> * BEAM-3587 (TextIO with Flink) seems related to a custom build
> with
> > Gradle (not
> > >> using artifacts created by Maven). Anyway, I will take a look
> today.
> > >> * BEAM-3186 has a PR. Aljoscha will do the review pretty soon.
> > >> * I'm also taking a look on the direct runner performance change.
> > >>
> > >> The goal is to cut RC2 tomorrow or the day after.
> > >>
> > >> Regards
> > >> JB
> > >>
> > >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
> > >>> Hi guys,
> > >>>
> > >>> Especially due to BEAM-3587 & BEAM-3186 regressions, I cancel
> RC1.
> > >>>
> > >>> We will cherry-pick fixes on release-2.3.0 branch.
> > >>>
> > >>> I'm updating Jira right now. When the fixes will be
> cherry-picked, I
> > will submit
> > >>> a RC2 to vote.
> > >>>
> > >>> Thanks !
> > >>> Regards
> > >>> JB
> > >>>
> > >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
> > >>>> Hi everyone,
> > >>>>
> > >>>> Please review and vote on the release candidate #1 for the
> version
> > 2.3.0, as
> > >>>> follows:
> > >>>>
> > >>>> [ ] +1, Approve the release
> > >>>> [ ] -1, Do not approve the release (please provide specific
> comments)
> > >>>>
> > >>>>
> > >>>> The complete staging area is available for your review, which
> includes:
> > >>>> * JIRA release notes [1],
> > >>>> * the official Apache source release to be deployed to
> dist.apache.org
> > <http://dist.apache.org>
> > >>>> <http://dist.apache.org> [2],
> > >>>> which is signed with the key with fingerprint C8282E76 [3],
> > >>>> * all artifacts to be deployed to the Maven Central Repository
> [4],
> > >>>> * source code tag "v2.3.0-RC1" [5],
> > >>>> * website pull request listing the release and publishing the
> API reference
> > >>>> manual [6].
> > >>>> * Java artifacts were built with Maven 3.3.9 and Oracle JDK
> 1.8.0_111.
> > >>>> * Python artifacts are deployed along with the source release
> to the
> > >>>> dist.apache.org <http://dist.apache.org> <
> http://dist.apache.org> [2].
> > >>>>
> > >>>> The vote will be open for at least 72 hours. It is adopted by
> majority approval,
> > >>>> with at least 3 PMC affirmative votes.
> > >>>>
> > >>>> Thanks,
> > >>>> JB
> > >>>>
> > >>>> [1]
> > >>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12319527&version=12341608
> > <https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12319527&version=12341608>
> > >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> > <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
> > >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> > <https://dist.apache.org/repos/dist/release/beam/KEYS>
> > >>>> [4] https://repository.apache.org/content/repositories/
> orgapachebeam-1026/
> > <https://repository.apache.org/content/repositories/
> orgapachebeam-1026/>
> > >>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
> > <https://github.com/apache/beam/tree/v2.3.0-RC1>
> > >>>> [6] https://github.com/apache/beam-site/pull/381
> > <https://github.com/apache/beam-site/pull/381>
> > >>>>
> > >>>
> > >>
> > >> --
> > >> Jean-Baptiste Onofré
> > >> jbonofre@apache.org <ma...@apache.org>
> > <mailto:jbonofre@apache.org <ma...@apache.org>>
> > >> http://blog.nanthrax.net
> > >> Talend - http://www.talend.com
> > >
> >
> > --
> > Jean-Baptiste Onofré
> > jbonofre@apache.org <ma...@apache.org>
> > http://blog.nanthrax.net
> > Talend - http://www.talend.com
> >
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi,
quick update about RC2: BEAM-3617 is the only Jira pending for the release.
I'm doing a git bisect to identify the commit who caused the performance
degradation.
Depending of the result, if it's a easy fix than we will try to do it for RC2,
else I will start the RC2 as now.
I will keep you posted.
Regards
JB
On 02/05/2018 04:51 PM, Jean-Baptiste Onofré wrote:
> Created:
>
> https://issues.apache.org/jira/browse/BEAM-3617
>
> Regards
> JB
>
> On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
>> What is the Jira for direct runner perf?
>>
>> On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré <jb@nanthrax.net
>> <ma...@nanthrax.net>> wrote:
>>
>> Thanks !
>>
>> I cherry-pick on release-2.3.0 branch.
>>
>> I'm on the direct runner perf test in the mean time.
>>
>> Thanks again !
>>
>> Regards
>> JB
>>
>> On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
>> > I merged fixes for:
>> > - https://issues.apache.org/jira/browse/BEAM-3186
>> <https://issues.apache.org/jira/browse/BEAM-3186>
>> > - https://issues.apache.org/jira/browse/BEAM-3589
>> <https://issues.apache.org/jira/browse/BEAM-3589>
>> >
>> > @JB I didn't yet merge them on the 2.3.0 branch, though, but I can or you can go
>> > ahead.
>> >
>> >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <jb@nanthrax.net <ma...@nanthrax.net>
>> >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
>> >>
>> >> Hi guys,
>> >>
>> >> Quick update on the RC2 preparation:
>> >>
>> >> * BEAM-3587 (TextIO with Flink) seems related to a custom build with
>> Gradle (not
>> >> using artifacts created by Maven). Anyway, I will take a look today.
>> >> * BEAM-3186 has a PR. Aljoscha will do the review pretty soon.
>> >> * I'm also taking a look on the direct runner performance change.
>> >>
>> >> The goal is to cut RC2 tomorrow or the day after.
>> >>
>> >> Regards
>> >> JB
>> >>
>> >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
>> >>> Hi guys,
>> >>>
>> >>> Especially due to BEAM-3587 & BEAM-3186 regressions, I cancel RC1.
>> >>>
>> >>> We will cherry-pick fixes on release-2.3.0 branch.
>> >>>
>> >>> I'm updating Jira right now. When the fixes will be cherry-picked, I
>> will submit
>> >>> a RC2 to vote.
>> >>>
>> >>> Thanks !
>> >>> Regards
>> >>> JB
>> >>>
>> >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>> >>>> Hi everyone,
>> >>>>
>> >>>> Please review and vote on the release candidate #1 for the version
>> 2.3.0, as
>> >>>> follows:
>> >>>>
>> >>>> [ ] +1, Approve the release
>> >>>> [ ] -1, Do not approve the release (please provide specific comments)
>> >>>>
>> >>>>
>> >>>> The complete staging area is available for your review, which includes:
>> >>>> * JIRA release notes [1],
>> >>>> * the official Apache source release to be deployed to dist.apache.org
>> <http://dist.apache.org>
>> >>>> <http://dist.apache.org> [2],
>> >>>> which is signed with the key with fingerprint C8282E76 [3],
>> >>>> * all artifacts to be deployed to the Maven Central Repository [4],
>> >>>> * source code tag "v2.3.0-RC1" [5],
>> >>>> * website pull request listing the release and publishing the API reference
>> >>>> manual [6].
>> >>>> * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
>> >>>> * Python artifacts are deployed along with the source release to the
>> >>>> dist.apache.org <http://dist.apache.org> <http://dist.apache.org> [2].
>> >>>>
>> >>>> The vote will be open for at least 72 hours. It is adopted by majority approval,
>> >>>> with at least 3 PMC affirmative votes.
>> >>>>
>> >>>> Thanks,
>> >>>> JB
>> >>>>
>> >>>> [1]
>> >>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>
>> >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
>> >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>> <https://dist.apache.org/repos/dist/release/beam/KEYS>
>> >>>> [4] https://repository.apache.org/content/repositories/orgapachebeam-1026/
>> <https://repository.apache.org/content/repositories/orgapachebeam-1026/>
>> >>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>> <https://github.com/apache/beam/tree/v2.3.0-RC1>
>> >>>> [6] https://github.com/apache/beam-site/pull/381
>> <https://github.com/apache/beam-site/pull/381>
>> >>>>
>> >>>
>> >>
>> >> --
>> >> Jean-Baptiste Onofré
>> >> jbonofre@apache.org <ma...@apache.org>
>> <mailto:jbonofre@apache.org <ma...@apache.org>>
>> >> http://blog.nanthrax.net
>> >> Talend - http://www.talend.com
>> >
>>
>> --
>> Jean-Baptiste Onofré
>> jbonofre@apache.org <ma...@apache.org>
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>
>>
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Created:
https://issues.apache.org/jira/browse/BEAM-3617
Regards
JB
On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
> What is the Jira for direct runner perf?
>
> On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré <jb@nanthrax.net
> <ma...@nanthrax.net>> wrote:
>
> Thanks !
>
> I cherry-pick on release-2.3.0 branch.
>
> I'm on the direct runner perf test in the mean time.
>
> Thanks again !
>
> Regards
> JB
>
> On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
> > I merged fixes for:
> > - https://issues.apache.org/jira/browse/BEAM-3186
> <https://issues.apache.org/jira/browse/BEAM-3186>
> > - https://issues.apache.org/jira/browse/BEAM-3589
> <https://issues.apache.org/jira/browse/BEAM-3589>
> >
> > @JB I didn't yet merge them on the 2.3.0 branch, though, but I can or you can go
> > ahead.
> >
> >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <jb@nanthrax.net <ma...@nanthrax.net>
> >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
> >>
> >> Hi guys,
> >>
> >> Quick update on the RC2 preparation:
> >>
> >> * BEAM-3587 (TextIO with Flink) seems related to a custom build with
> Gradle (not
> >> using artifacts created by Maven). Anyway, I will take a look today.
> >> * BEAM-3186 has a PR. Aljoscha will do the review pretty soon.
> >> * I'm also taking a look on the direct runner performance change.
> >>
> >> The goal is to cut RC2 tomorrow or the day after.
> >>
> >> Regards
> >> JB
> >>
> >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
> >>> Hi guys,
> >>>
> >>> Especially due to BEAM-3587 & BEAM-3186 regressions, I cancel RC1.
> >>>
> >>> We will cherry-pick fixes on release-2.3.0 branch.
> >>>
> >>> I'm updating Jira right now. When the fixes will be cherry-picked, I
> will submit
> >>> a RC2 to vote.
> >>>
> >>> Thanks !
> >>> Regards
> >>> JB
> >>>
> >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
> >>>> Hi everyone,
> >>>>
> >>>> Please review and vote on the release candidate #1 for the version
> 2.3.0, as
> >>>> follows:
> >>>>
> >>>> [ ] +1, Approve the release
> >>>> [ ] -1, Do not approve the release (please provide specific comments)
> >>>>
> >>>>
> >>>> The complete staging area is available for your review, which includes:
> >>>> * JIRA release notes [1],
> >>>> * the official Apache source release to be deployed to dist.apache.org
> <http://dist.apache.org>
> >>>> <http://dist.apache.org> [2],
> >>>> which is signed with the key with fingerprint C8282E76 [3],
> >>>> * all artifacts to be deployed to the Maven Central Repository [4],
> >>>> * source code tag "v2.3.0-RC1" [5],
> >>>> * website pull request listing the release and publishing the API reference
> >>>> manual [6].
> >>>> * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
> >>>> * Python artifacts are deployed along with the source release to the
> >>>> dist.apache.org <http://dist.apache.org> <http://dist.apache.org> [2].
> >>>>
> >>>> The vote will be open for at least 72 hours. It is adopted by majority approval,
> >>>> with at least 3 PMC affirmative votes.
> >>>>
> >>>> Thanks,
> >>>> JB
> >>>>
> >>>> [1]
> >>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>
> >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
> >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> <https://dist.apache.org/repos/dist/release/beam/KEYS>
> >>>> [4] https://repository.apache.org/content/repositories/orgapachebeam-1026/
> <https://repository.apache.org/content/repositories/orgapachebeam-1026/>
> >>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
> <https://github.com/apache/beam/tree/v2.3.0-RC1>
> >>>> [6] https://github.com/apache/beam-site/pull/381
> <https://github.com/apache/beam-site/pull/381>
> >>>>
> >>>
> >>
> >> --
> >> Jean-Baptiste Onofré
> >> jbonofre@apache.org <ma...@apache.org>
> <mailto:jbonofre@apache.org <ma...@apache.org>>
> >> http://blog.nanthrax.net
> >> Talend - http://www.talend.com
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org <ma...@apache.org>
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi Kenn,
my bad, I didn't create one yet (I was busy on the TextIO with flink runner, now
identify \o/ ;)).
Let me create it right now.
Thanks !
Regards
JB
On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
> What is the Jira for direct runner perf?
>
> On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré <jb@nanthrax.net
> <ma...@nanthrax.net>> wrote:
>
> Thanks !
>
> I cherry-pick on release-2.3.0 branch.
>
> I'm on the direct runner perf test in the mean time.
>
> Thanks again !
>
> Regards
> JB
>
> On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
> > I merged fixes for:
> > - https://issues.apache.org/jira/browse/BEAM-3186
> <https://issues.apache.org/jira/browse/BEAM-3186>
> > - https://issues.apache.org/jira/browse/BEAM-3589
> <https://issues.apache.org/jira/browse/BEAM-3589>
> >
> > @JB I didn't yet merge them on the 2.3.0 branch, though, but I can or you can go
> > ahead.
> >
> >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <jb@nanthrax.net <ma...@nanthrax.net>
> >> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
> >>
> >> Hi guys,
> >>
> >> Quick update on the RC2 preparation:
> >>
> >> * BEAM-3587 (TextIO with Flink) seems related to a custom build with
> Gradle (not
> >> using artifacts created by Maven). Anyway, I will take a look today.
> >> * BEAM-3186 has a PR. Aljoscha will do the review pretty soon.
> >> * I'm also taking a look on the direct runner performance change.
> >>
> >> The goal is to cut RC2 tomorrow or the day after.
> >>
> >> Regards
> >> JB
> >>
> >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
> >>> Hi guys,
> >>>
> >>> Especially due to BEAM-3587 & BEAM-3186 regressions, I cancel RC1.
> >>>
> >>> We will cherry-pick fixes on release-2.3.0 branch.
> >>>
> >>> I'm updating Jira right now. When the fixes will be cherry-picked, I
> will submit
> >>> a RC2 to vote.
> >>>
> >>> Thanks !
> >>> Regards
> >>> JB
> >>>
> >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
> >>>> Hi everyone,
> >>>>
> >>>> Please review and vote on the release candidate #1 for the version
> 2.3.0, as
> >>>> follows:
> >>>>
> >>>> [ ] +1, Approve the release
> >>>> [ ] -1, Do not approve the release (please provide specific comments)
> >>>>
> >>>>
> >>>> The complete staging area is available for your review, which includes:
> >>>> * JIRA release notes [1],
> >>>> * the official Apache source release to be deployed to dist.apache.org
> <http://dist.apache.org>
> >>>> <http://dist.apache.org> [2],
> >>>> which is signed with the key with fingerprint C8282E76 [3],
> >>>> * all artifacts to be deployed to the Maven Central Repository [4],
> >>>> * source code tag "v2.3.0-RC1" [5],
> >>>> * website pull request listing the release and publishing the API reference
> >>>> manual [6].
> >>>> * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
> >>>> * Python artifacts are deployed along with the source release to the
> >>>> dist.apache.org <http://dist.apache.org> <http://dist.apache.org> [2].
> >>>>
> >>>> The vote will be open for at least 72 hours. It is adopted by majority approval,
> >>>> with at least 3 PMC affirmative votes.
> >>>>
> >>>> Thanks,
> >>>> JB
> >>>>
> >>>> [1]
> >>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>
> >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>
> >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> <https://dist.apache.org/repos/dist/release/beam/KEYS>
> >>>> [4] https://repository.apache.org/content/repositories/orgapachebeam-1026/
> <https://repository.apache.org/content/repositories/orgapachebeam-1026/>
> >>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
> <https://github.com/apache/beam/tree/v2.3.0-RC1>
> >>>> [6] https://github.com/apache/beam-site/pull/381
> <https://github.com/apache/beam-site/pull/381>
> >>>>
> >>>
> >>
> >> --
> >> Jean-Baptiste Onofré
> >> jbonofre@apache.org <ma...@apache.org>
> <mailto:jbonofre@apache.org <ma...@apache.org>>
> >> http://blog.nanthrax.net
> >> Talend - http://www.talend.com
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org <ma...@apache.org>
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Kenneth Knowles <kl...@google.com>.
What is the Jira for direct runner perf?
On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> Thanks !
>
> I cherry-pick on release-2.3.0 branch.
>
> I'm on the direct runner perf test in the mean time.
>
> Thanks again !
>
> Regards
> JB
>
> On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
> > I merged fixes for:
> > - https://issues.apache.org/jira/browse/BEAM-3186
> > - https://issues.apache.org/jira/browse/BEAM-3589
> >
> > @JB I didn't yet merge them on the 2.3.0 branch, though, but I can or
> you can go
> > ahead.
> >
> >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <jb@nanthrax.net
> >> <ma...@nanthrax.net>> wrote:
> >>
> >> Hi guys,
> >>
> >> Quick update on the RC2 preparation:
> >>
> >> * BEAM-3587 (TextIO with Flink) seems related to a custom build with
> Gradle (not
> >> using artifacts created by Maven). Anyway, I will take a look today.
> >> * BEAM-3186 has a PR. Aljoscha will do the review pretty soon.
> >> * I'm also taking a look on the direct runner performance change.
> >>
> >> The goal is to cut RC2 tomorrow or the day after.
> >>
> >> Regards
> >> JB
> >>
> >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
> >>> Hi guys,
> >>>
> >>> Especially due to BEAM-3587 & BEAM-3186 regressions, I cancel RC1.
> >>>
> >>> We will cherry-pick fixes on release-2.3.0 branch.
> >>>
> >>> I'm updating Jira right now. When the fixes will be cherry-picked, I
> will submit
> >>> a RC2 to vote.
> >>>
> >>> Thanks !
> >>> Regards
> >>> JB
> >>>
> >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
> >>>> Hi everyone,
> >>>>
> >>>> Please review and vote on the release candidate #1 for the version
> 2.3.0, as
> >>>> follows:
> >>>>
> >>>> [ ] +1, Approve the release
> >>>> [ ] -1, Do not approve the release (please provide specific comments)
> >>>>
> >>>>
> >>>> The complete staging area is available for your review, which
> includes:
> >>>> * JIRA release notes [1],
> >>>> * the official Apache source release to be deployed to
> dist.apache.org
> >>>> <http://dist.apache.org> [2],
> >>>> which is signed with the key with fingerprint C8282E76 [3],
> >>>> * all artifacts to be deployed to the Maven Central Repository [4],
> >>>> * source code tag "v2.3.0-RC1" [5],
> >>>> * website pull request listing the release and publishing the API
> reference
> >>>> manual [6].
> >>>> * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
> >>>> * Python artifacts are deployed along with the source release to the
> >>>> dist.apache.org <http://dist.apache.org> [2].
> >>>>
> >>>> The vote will be open for at least 72 hours. It is adopted by
> majority approval,
> >>>> with at least 3 PMC affirmative votes.
> >>>>
> >>>> Thanks,
> >>>> JB
> >>>>
> >>>> [1]
> >>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12319527&version=12341608
> >>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> >>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> >>>> [4] https://repository.apache.org/content/repositories/
> orgapachebeam-1026/
> >>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
> >>>> [6] https://github.com/apache/beam-site/pull/381
> >>>>
> >>>
> >>
> >> --
> >> Jean-Baptiste Onofré
> >> jbonofre@apache.org <ma...@apache.org>
> >> http://blog.nanthrax.net
> >> Talend - http://www.talend.com
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Thanks !
I cherry-pick on release-2.3.0 branch.
I'm on the direct runner perf test in the mean time.
Thanks again !
Regards
JB
On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
> I merged fixes for:
> - https://issues.apache.org/jira/browse/BEAM-3186
> - https://issues.apache.org/jira/browse/BEAM-3589
>
> @JB I didn't yet merge them on the 2.3.0 branch, though, but I can or you can go
> ahead.
>
>> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <jb@nanthrax.net
>> <ma...@nanthrax.net>> wrote:
>>
>> Hi guys,
>>
>> Quick update on the RC2 preparation:
>>
>> * BEAM-3587 (TextIO with Flink) seems related to a custom build with Gradle (not
>> using artifacts created by Maven). Anyway, I will take a look today.
>> * BEAM-3186 has a PR. Aljoscha will do the review pretty soon.
>> * I'm also taking a look on the direct runner performance change.
>>
>> The goal is to cut RC2 tomorrow or the day after.
>>
>> Regards
>> JB
>>
>> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
>>> Hi guys,
>>>
>>> Especially due to BEAM-3587 & BEAM-3186 regressions, I cancel RC1.
>>>
>>> We will cherry-pick fixes on release-2.3.0 branch.
>>>
>>> I'm updating Jira right now. When the fixes will be cherry-picked, I will submit
>>> a RC2 to vote.
>>>
>>> Thanks !
>>> Regards
>>> JB
>>>
>>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>>>> Hi everyone,
>>>>
>>>> Please review and vote on the release candidate #1 for the version 2.3.0, as
>>>> follows:
>>>>
>>>> [ ] +1, Approve the release
>>>> [ ] -1, Do not approve the release (please provide specific comments)
>>>>
>>>>
>>>> The complete staging area is available for your review, which includes:
>>>> * JIRA release notes [1],
>>>> * the official Apache source release to be deployed to dist.apache.org
>>>> <http://dist.apache.org> [2],
>>>> which is signed with the key with fingerprint C8282E76 [3],
>>>> * all artifacts to be deployed to the Maven Central Repository [4],
>>>> * source code tag "v2.3.0-RC1" [5],
>>>> * website pull request listing the release and publishing the API reference
>>>> manual [6].
>>>> * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
>>>> * Python artifacts are deployed along with the source release to the
>>>> dist.apache.org <http://dist.apache.org> [2].
>>>>
>>>> The vote will be open for at least 72 hours. It is adopted by majority approval,
>>>> with at least 3 PMC affirmative votes.
>>>>
>>>> Thanks,
>>>> JB
>>>>
>>>> [1]
>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>>> [4] https://repository.apache.org/content/repositories/orgapachebeam-1026/
>>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>>>> [6] https://github.com/apache/beam-site/pull/381
>>>>
>>>
>>
>> --
>> Jean-Baptiste Onofré
>> jbonofre@apache.org <ma...@apache.org>
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Aljoscha Krettek <al...@apache.org>.
I merged fixes for:
- https://issues.apache.org/jira/browse/BEAM-3186 <https://issues.apache.org/jira/browse/BEAM-3186>
- https://issues.apache.org/jira/browse/BEAM-3589 <https://issues.apache.org/jira/browse/BEAM-3589>
@JB I didn't yet merge them on the 2.3.0 branch, though, but I can or you can go ahead.
> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
>
> Hi guys,
>
> Quick update on the RC2 preparation:
>
> * BEAM-3587 (TextIO with Flink) seems related to a custom build with Gradle (not
> using artifacts created by Maven). Anyway, I will take a look today.
> * BEAM-3186 has a PR. Aljoscha will do the review pretty soon.
> * I'm also taking a look on the direct runner performance change.
>
> The goal is to cut RC2 tomorrow or the day after.
>
> Regards
> JB
>
> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
>> Hi guys,
>>
>> Especially due to BEAM-3587 & BEAM-3186 regressions, I cancel RC1.
>>
>> We will cherry-pick fixes on release-2.3.0 branch.
>>
>> I'm updating Jira right now. When the fixes will be cherry-picked, I will submit
>> a RC2 to vote.
>>
>> Thanks !
>> Regards
>> JB
>>
>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>>> Hi everyone,
>>>
>>> Please review and vote on the release candidate #1 for the version 2.3.0, as
>>> follows:
>>>
>>> [ ] +1, Approve the release
>>> [ ] -1, Do not approve the release (please provide specific comments)
>>>
>>>
>>> The complete staging area is available for your review, which includes:
>>> * JIRA release notes [1],
>>> * the official Apache source release to be deployed to dist.apache.org [2],
>>> which is signed with the key with fingerprint C8282E76 [3],
>>> * all artifacts to be deployed to the Maven Central Repository [4],
>>> * source code tag "v2.3.0-RC1" [5],
>>> * website pull request listing the release and publishing the API reference
>>> manual [6].
>>> * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
>>> * Python artifacts are deployed along with the source release to the
>>> dist.apache.org [2].
>>>
>>> The vote will be open for at least 72 hours. It is adopted by majority approval,
>>> with at least 3 PMC affirmative votes.
>>>
>>> Thanks,
>>> JB
>>>
>>> [1]
>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>> [4] https://repository.apache.org/content/repositories/orgapachebeam-1026/
>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>>> [6] https://github.com/apache/beam-site/pull/381
>>>
>>
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi guys,
Quick update on the RC2 preparation:
* BEAM-3587 (TextIO with Flink) seems related to a custom build with Gradle (not
using artifacts created by Maven). Anyway, I will take a look today.
* BEAM-3186 has a PR. Aljoscha will do the review pretty soon.
* I'm also taking a look on the direct runner performance change.
The goal is to cut RC2 tomorrow or the day after.
Regards
JB
On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
> Hi guys,
>
> Especially due to BEAM-3587 & BEAM-3186 regressions, I cancel RC1.
>
> We will cherry-pick fixes on release-2.3.0 branch.
>
> I'm updating Jira right now. When the fixes will be cherry-picked, I will submit
> a RC2 to vote.
>
> Thanks !
> Regards
> JB
>
> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>> Hi everyone,
>>
>> Please review and vote on the release candidate #1 for the version 2.3.0, as
>> follows:
>>
>> [ ] +1, Approve the release
>> [ ] -1, Do not approve the release (please provide specific comments)
>>
>>
>> The complete staging area is available for your review, which includes:
>> * JIRA release notes [1],
>> * the official Apache source release to be deployed to dist.apache.org [2],
>> which is signed with the key with fingerprint C8282E76 [3],
>> * all artifacts to be deployed to the Maven Central Repository [4],
>> * source code tag "v2.3.0-RC1" [5],
>> * website pull request listing the release and publishing the API reference
>> manual [6].
>> * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
>> * Python artifacts are deployed along with the source release to the
>> dist.apache.org [2].
>>
>> The vote will be open for at least 72 hours. It is adopted by majority approval,
>> with at least 3 PMC affirmative votes.
>>
>> Thanks,
>> JB
>>
>> [1]
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>> [4] https://repository.apache.org/content/repositories/orgapachebeam-1026/
>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>> [6] https://github.com/apache/beam-site/pull/381
>>
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com