You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Alan Myrvold <am...@google.com> on 2018/02/26 21:54:47 UTC
Pain points in the 2.3.0 release / improvements to 2.4.0 release process?
Is there a list of pain points during the 2.3.0 release and improvements
that can be made to the 2.4.0 release process?
Re: Pain points in the 2.3.0 release / improvements to 2.4.0 release process?
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
I think it would be a "good to have" point (maybe not a blocker).
I will propose a release guide PR about this tomorrow.
Regards
JB
Le 1 mars 2018 à 09:09, à 09:09, Lukasz Cwik <lc...@google.com> a écrit:
>Should we change release policy so that the validation tests are
>required
>to pass on a release candidate as a requirement for the release to be
>approved?
>
>If so, we should update the release guide saying so.
>
>On Wed, Feb 28, 2018 at 7:00 PM, Jean-Baptiste Onofré <jb...@nanthrax.net>
>wrote:
>
>> Cool, thanks !
>>
>> Maybe at least a note to run nexmark on every RC to compare the
>results
>> with
>> previous releases is interesting.
>>
>> Regards
>> JB
>>
>> On 03/01/2018 02:02 AM, Alan Myrvold wrote:
>> > Thanks for the feedback. Yifan and I have automated the Java
>> quickstarts for
>> > apex, direct, dataflow, flink local and spark [1] and automated
>python
>> release
>> > validation [2]. Yifan is working on fixing the Java mobile
>archetype and
>> > validating the mobile example. The spark quickstart automation came
>in
>> late, but
>> > identified the [BEAM-3668] issue in RC2.
>> >
>> > The Java quickstarts are now running daily with the snapshot
>release [3]
>> and are
>> > parameterized to make it easy to run with the next RC candidate.
>> >
>> > Hope these help make the next release smoother. Open to ideas for
>other
>> areas to
>> > automate.
>> >
>> > [1]
>https://github.com/apache/beam/tree/master/release/src/main/groovy
>> > [2]
>> > https://github.com/apache/beam/blob/master/release/src/
>> main/groovy/run_release_candidate_python_validation.sh
>> > [3] https://builds.apache.org/view/A-D/view/Beam/job/beam_
>> PostRelease_NightlySnapshot/
>> >
>> >
>> >
>> > On Wed, Feb 28, 2018 at 9:54 AM Lukasz Cwik <lcwik@google.com
>> > <ma...@google.com>> wrote:
>> >
>> > Validating the release by following quickstarts being a manual
>> process I
>> > believe is still the largest pain point:
>> > * We missed that the archetypes were missing the mobile gaming
>> examples.
>> > * The tcnative dependency conflict that we needed to cut RC2
>for.
>> >
>> > Overall much smoother then the prior release but still a good
>amount
>> of
>> > manual steps involved.
>> >
>> > On Tue, Feb 27, 2018 at 9:16 AM, Reuven Lax <relax@google.com
>> > <ma...@google.com>> wrote:
>> >
>> > Thanks for fixing the manual cleanup issue! This is
>something we
>> kept
>> > punting on in previous releases.
>> >
>> >
>> > On Mon, Feb 26, 2018 at 7:14 PM Jean-Baptiste Onofré <
>> jb@nanthrax.net
>> > <ma...@nanthrax.net>> wrote:
>> >
>> > Hi Alan,
>> >
>> > Honestly, it was an easy and smooth release, similar
>to
>> other
>> > Apache project
>> > release.
>> >
>> > Maybe the points where we could a little bit improve
>are:
>> >
>> > 1. The website update was pretty long as @asfgit merge
>> didn't work
>> > (due to a out
>> > of sync on the github mirror). I think the website
>publish
>> PR can be
>> > automatized.
>> > 2. Upload to pylib required manual action (like
>renaming the
>> files)
>> >
>> > Thanks to the change I did on the assembly, the
>artifacts
>> don't
>> > require any
>> > manual cleanup (whereas it was the case in previous
>release).
>> >
>> > Regards
>> > JB
>> >
>> > On 02/26/2018 10:54 PM, Alan Myrvold wrote:
>> > > Is there a list of pain points during the 2.3.0
>release and
>> > improvements that
>> > > can be made to the 2.4.0 release process?
>> > >
>> >
>> > --
>> > Jean-Baptiste Onofré
>> > jbonofre@apache.org <ma...@apache.org>
>> > http://blog.nanthrax.net
>> > Talend - http://www.talend.com
>> >
>> >
>>
>> --
>> Jean-Baptiste Onofré
>> jbonofre@apache.org
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>
Re: Pain points in the 2.3.0 release / improvements to 2.4.0 release process?
Posted by Lukasz Cwik <lc...@google.com>.
Should we change release policy so that the validation tests are required
to pass on a release candidate as a requirement for the release to be
approved?
If so, we should update the release guide saying so.
On Wed, Feb 28, 2018 at 7:00 PM, Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> Cool, thanks !
>
> Maybe at least a note to run nexmark on every RC to compare the results
> with
> previous releases is interesting.
>
> Regards
> JB
>
> On 03/01/2018 02:02 AM, Alan Myrvold wrote:
> > Thanks for the feedback. Yifan and I have automated the Java
> quickstarts for
> > apex, direct, dataflow, flink local and spark [1] and automated python
> release
> > validation [2]. Yifan is working on fixing the Java mobile archetype and
> > validating the mobile example. The spark quickstart automation came in
> late, but
> > identified the [BEAM-3668] issue in RC2.
> >
> > The Java quickstarts are now running daily with the snapshot release [3]
> and are
> > parameterized to make it easy to run with the next RC candidate.
> >
> > Hope these help make the next release smoother. Open to ideas for other
> areas to
> > automate.
> >
> > [1] https://github.com/apache/beam/tree/master/release/src/main/groovy
> > [2]
> > https://github.com/apache/beam/blob/master/release/src/
> main/groovy/run_release_candidate_python_validation.sh
> > [3] https://builds.apache.org/view/A-D/view/Beam/job/beam_
> PostRelease_NightlySnapshot/
> >
> >
> >
> > On Wed, Feb 28, 2018 at 9:54 AM Lukasz Cwik <lcwik@google.com
> > <ma...@google.com>> wrote:
> >
> > Validating the release by following quickstarts being a manual
> process I
> > believe is still the largest pain point:
> > * We missed that the archetypes were missing the mobile gaming
> examples.
> > * The tcnative dependency conflict that we needed to cut RC2 for.
> >
> > Overall much smoother then the prior release but still a good amount
> of
> > manual steps involved.
> >
> > On Tue, Feb 27, 2018 at 9:16 AM, Reuven Lax <relax@google.com
> > <ma...@google.com>> wrote:
> >
> > Thanks for fixing the manual cleanup issue! This is something we
> kept
> > punting on in previous releases.
> >
> >
> > On Mon, Feb 26, 2018 at 7:14 PM Jean-Baptiste Onofré <
> jb@nanthrax.net
> > <ma...@nanthrax.net>> wrote:
> >
> > Hi Alan,
> >
> > Honestly, it was an easy and smooth release, similar to
> other
> > Apache project
> > release.
> >
> > Maybe the points where we could a little bit improve are:
> >
> > 1. The website update was pretty long as @asfgit merge
> didn't work
> > (due to a out
> > of sync on the github mirror). I think the website publish
> PR can be
> > automatized.
> > 2. Upload to pylib required manual action (like renaming the
> files)
> >
> > Thanks to the change I did on the assembly, the artifacts
> don't
> > require any
> > manual cleanup (whereas it was the case in previous release).
> >
> > Regards
> > JB
> >
> > On 02/26/2018 10:54 PM, Alan Myrvold wrote:
> > > Is there a list of pain points during the 2.3.0 release and
> > improvements that
> > > can be made to the 2.4.0 release process?
> > >
> >
> > --
> > Jean-Baptiste Onofré
> > jbonofre@apache.org <ma...@apache.org>
> > http://blog.nanthrax.net
> > Talend - http://www.talend.com
> >
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
Re: Pain points in the 2.3.0 release / improvements to 2.4.0 release
process?
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Cool, thanks !
Maybe at least a note to run nexmark on every RC to compare the results with
previous releases is interesting.
Regards
JB
On 03/01/2018 02:02 AM, Alan Myrvold wrote:
> Thanks for the feedback. Yifan and I have automated the Java quickstarts for
> apex, direct, dataflow, flink local and spark [1] and automated python release
> validation [2]. Yifan is working on fixing the Java mobile archetype and
> validating the mobile example. The spark quickstart automation came in late, but
> identified the [BEAM-3668] issue in RC2.
>
> The Java quickstarts are now running daily with the snapshot release [3] and are
> parameterized to make it easy to run with the next RC candidate.
>
> Hope these help make the next release smoother. Open to ideas for other areas to
> automate.
>
> [1] https://github.com/apache/beam/tree/master/release/src/main/groovy
> [2]
> https://github.com/apache/beam/blob/master/release/src/main/groovy/run_release_candidate_python_validation.sh
> [3] https://builds.apache.org/view/A-D/view/Beam/job/beam_PostRelease_NightlySnapshot/
>
>
>
> On Wed, Feb 28, 2018 at 9:54 AM Lukasz Cwik <lcwik@google.com
> <ma...@google.com>> wrote:
>
> Validating the release by following quickstarts being a manual process I
> believe is still the largest pain point:
> * We missed that the archetypes were missing the mobile gaming examples.
> * The tcnative dependency conflict that we needed to cut RC2 for.
>
> Overall much smoother then the prior release but still a good amount of
> manual steps involved.
>
> On Tue, Feb 27, 2018 at 9:16 AM, Reuven Lax <relax@google.com
> <ma...@google.com>> wrote:
>
> Thanks for fixing the manual cleanup issue! This is something we kept
> punting on in previous releases.
>
>
> On Mon, Feb 26, 2018 at 7:14 PM Jean-Baptiste Onofré <jb@nanthrax.net
> <ma...@nanthrax.net>> wrote:
>
> Hi Alan,
>
> Honestly, it was an easy and smooth release, similar to other
> Apache project
> release.
>
> Maybe the points where we could a little bit improve are:
>
> 1. The website update was pretty long as @asfgit merge didn't work
> (due to a out
> of sync on the github mirror). I think the website publish PR can be
> automatized.
> 2. Upload to pylib required manual action (like renaming the files)
>
> Thanks to the change I did on the assembly, the artifacts don't
> require any
> manual cleanup (whereas it was the case in previous release).
>
> Regards
> JB
>
> On 02/26/2018 10:54 PM, Alan Myrvold wrote:
> > Is there a list of pain points during the 2.3.0 release and
> improvements that
> > can be made to the 2.4.0 release process?
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org <ma...@apache.org>
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: Pain points in the 2.3.0 release / improvements to 2.4.0 release process?
Posted by Alan Myrvold <am...@google.com>.
Thanks for the feedback. Yifan and I have automated the Java quickstarts
for apex, direct, dataflow, flink local and spark [1] and automated python
release validation [2]. Yifan is working on fixing the Java mobile
archetype and validating the mobile example. The spark quickstart
automation came in late, but identified the [BEAM-3668] issue in RC2.
The Java quickstarts are now running daily with the snapshot release [3]
and are parameterized to make it easy to run with the next RC candidate.
Hope these help make the next release smoother. Open to ideas for other
areas to automate.
[1] https://github.com/apache/beam/tree/master/release/src/main/groovy
[2]
https://github.com/apache/beam/blob/master/release/src/main/groovy/run_release_candidate_python_validation.sh
[3]
https://builds.apache.org/view/A-D/view/Beam/job/beam_PostRelease_NightlySnapshot/
On Wed, Feb 28, 2018 at 9:54 AM Lukasz Cwik <lc...@google.com> wrote:
> Validating the release by following quickstarts being a manual process I
> believe is still the largest pain point:
> * We missed that the archetypes were missing the mobile gaming examples.
> * The tcnative dependency conflict that we needed to cut RC2 for.
>
> Overall much smoother then the prior release but still a good amount of
> manual steps involved.
>
> On Tue, Feb 27, 2018 at 9:16 AM, Reuven Lax <re...@google.com> wrote:
>
>> Thanks for fixing the manual cleanup issue! This is something we kept
>> punting on in previous releases.
>>
>>
>> On Mon, Feb 26, 2018 at 7:14 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
>> wrote:
>>
>>> Hi Alan,
>>>
>>> Honestly, it was an easy and smooth release, similar to other Apache
>>> project
>>> release.
>>>
>>> Maybe the points where we could a little bit improve are:
>>>
>>> 1. The website update was pretty long as @asfgit merge didn't work (due
>>> to a out
>>> of sync on the github mirror). I think the website publish PR can be
>>> automatized.
>>> 2. Upload to pylib required manual action (like renaming the files)
>>>
>>> Thanks to the change I did on the assembly, the artifacts don't require
>>> any
>>> manual cleanup (whereas it was the case in previous release).
>>>
>>> Regards
>>> JB
>>>
>>> On 02/26/2018 10:54 PM, Alan Myrvold wrote:
>>> > Is there a list of pain points during the 2.3.0 release and
>>> improvements that
>>> > can be made to the 2.4.0 release process?
>>> >
>>>
>>> --
>>> Jean-Baptiste Onofré
>>> jbonofre@apache.org
>>> http://blog.nanthrax.net
>>> Talend - http://www.talend.com
>>>
>>
>
Re: Pain points in the 2.3.0 release / improvements to 2.4.0 release process?
Posted by Lukasz Cwik <lc...@google.com>.
Validating the release by following quickstarts being a manual process I
believe is still the largest pain point:
* We missed that the archetypes were missing the mobile gaming examples.
* The tcnative dependency conflict that we needed to cut RC2 for.
Overall much smoother then the prior release but still a good amount of
manual steps involved.
On Tue, Feb 27, 2018 at 9:16 AM, Reuven Lax <re...@google.com> wrote:
> Thanks for fixing the manual cleanup issue! This is something we kept
> punting on in previous releases.
>
>
> On Mon, Feb 26, 2018 at 7:14 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
>
>> Hi Alan,
>>
>> Honestly, it was an easy and smooth release, similar to other Apache
>> project
>> release.
>>
>> Maybe the points where we could a little bit improve are:
>>
>> 1. The website update was pretty long as @asfgit merge didn't work (due
>> to a out
>> of sync on the github mirror). I think the website publish PR can be
>> automatized.
>> 2. Upload to pylib required manual action (like renaming the files)
>>
>> Thanks to the change I did on the assembly, the artifacts don't require
>> any
>> manual cleanup (whereas it was the case in previous release).
>>
>> Regards
>> JB
>>
>> On 02/26/2018 10:54 PM, Alan Myrvold wrote:
>> > Is there a list of pain points during the 2.3.0 release and
>> improvements that
>> > can be made to the 2.4.0 release process?
>> >
>>
>> --
>> Jean-Baptiste Onofré
>> jbonofre@apache.org
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>
>
Re: Pain points in the 2.3.0 release / improvements to 2.4.0 release process?
Posted by Reuven Lax <re...@google.com>.
Thanks for fixing the manual cleanup issue! This is something we kept
punting on in previous releases.
On Mon, Feb 26, 2018 at 7:14 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> Hi Alan,
>
> Honestly, it was an easy and smooth release, similar to other Apache
> project
> release.
>
> Maybe the points where we could a little bit improve are:
>
> 1. The website update was pretty long as @asfgit merge didn't work (due to
> a out
> of sync on the github mirror). I think the website publish PR can be
> automatized.
> 2. Upload to pylib required manual action (like renaming the files)
>
> Thanks to the change I did on the assembly, the artifacts don't require any
> manual cleanup (whereas it was the case in previous release).
>
> Regards
> JB
>
> On 02/26/2018 10:54 PM, Alan Myrvold wrote:
> > Is there a list of pain points during the 2.3.0 release and improvements
> that
> > can be made to the 2.4.0 release process?
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
Re: Pain points in the 2.3.0 release / improvements to 2.4.0 release
process?
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi Alan,
Honestly, it was an easy and smooth release, similar to other Apache project
release.
Maybe the points where we could a little bit improve are:
1. The website update was pretty long as @asfgit merge didn't work (due to a out
of sync on the github mirror). I think the website publish PR can be automatized.
2. Upload to pylib required manual action (like renaming the files)
Thanks to the change I did on the assembly, the artifacts don't require any
manual cleanup (whereas it was the case in previous release).
Regards
JB
On 02/26/2018 10:54 PM, Alan Myrvold wrote:
> Is there a list of pain points during the 2.3.0 release and improvements that
> can be made to the 2.4.0 release process?
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com