Process for getting the first stable release out

[Davor Bonaci <da...@apache.org>]

We've been working on the first stable release for a while now -- we've
been tracking blocking issues [1], resolved many of them, organized a
hackathon [2], and improved testing coverage in various areas.

I think we are nearly done -- there are a few more blocking issues left in
JIRA, and probably a few more that we'll discover as we go along. That
said, I think it is time to discuss the process for getting this release
finalized.

I'd like to propose the following (tweaked) process for this special
release:

* Create a release branch, and start building release candidates *now*
This would accelerate branch creation compared to the normal process, but
would separate the first stable release from other development on the
master branch. This yields to stability and avoids unnecessary churn.

* Community-driven acceptance criteria
This would be an added step where everyone can recommend necessary
conditions for acceptance of the release candidate. Those conditions would
be *double-checked* manually, in addition to having all automated tests
pass. This could include important scenarios that we want to be really sure
about, e.g., WordCount example works on a given runner, TextIO can read
from HDFS, and we'd jointly validate those scenarios.

* Vote
As usual.

Time-wise, I think it would be really awesome to make this release coincide
with the ApacheCon conference that starts in about 2 weeks (on May 16th). I
think this would be a great goal -- and I'll do my part to make this
possible.

Finally, we haven't formally closed on the version designation (see
separate thread).

Thoughts? If there are no objections, I'll start this process soon, and we
can adjust as we go.

Thanks!

Davor

[1] https://issues.apache.org/jira/issues/?jql=project%20%
3D%20BEAM%20AND%20fixVersion%20%3D%20%22First%20stable%20release%22%20AND%
20resolution%20%3D%20Unresolved%20ORDER%20BY%20due%20ASC%2C%20priority%
20DESC%2C%20created%20ASC
[2]
https://docs.google.com/document/d/1UKC2R_9FkSdMVTz2nt2sIW18KoLbIu6w0aj9bwSSPiw/

Re: Process for getting the first stable release out

[Davor Bonaci <da...@apache.org>]

Indeed -- the release would come with backward compatibility guarantee
within the given major version (modulo known exceptions: @Internal,
@Experimental, util package, etc.), all according to the standard semantic
versioning [1]. Also, see pending BEAM-1345 [2] that calls for audit and
sprinkling of annotations on parts that are not ready to be frozen. Anybody
should feel free to send PRs to keep things subject-to-change, if
appropriate.

Going forward, we'll have to add add automated testing for compatibility --
tracked by BEAM-1131 [3], and I believe Jason is on it.

[1] http://semver.org/
[2] https://issues.apache.org/jira/browse/BEAM-1345
[3] https://issues.apache.org/jira/browse/BEAM-1131

On Thu, May 4, 2017 at 12:53 PM, Thomas Weise <th...@gmail.com>
wrote:

> Since the release would come with backward compatibility promise (?),
> creation of the release branch would also affect what happens on master
> from that point onwards. It would be good to provide some guidance on that?
>
> --
> sent from mobile
> On May 4, 2017 12:08 PM, "Davor Bonaci" <da...@apache.org> wrote:
>
> > We've been working on the first stable release for a while now -- we've
> > been tracking blocking issues [1], resolved many of them, organized a
> > hackathon [2], and improved testing coverage in various areas.
> >
> > I think we are nearly done -- there are a few more blocking issues left
> in
> > JIRA, and probably a few more that we'll discover as we go along. That
> > said, I think it is time to discuss the process for getting this release
> > finalized.
> >
> > I'd like to propose the following (tweaked) process for this special
> > release:
> >
> > * Create a release branch, and start building release candidates *now*
> > This would accelerate branch creation compared to the normal process, but
> > would separate the first stable release from other development on the
> > master branch. This yields to stability and avoids unnecessary churn.
> >
> > * Community-driven acceptance criteria
> > This would be an added step where everyone can recommend necessary
> > conditions for acceptance of the release candidate. Those conditions
> would
> > be *double-checked* manually, in addition to having all automated tests
> > pass. This could include important scenarios that we want to be really
> sure
> > about, e.g., WordCount example works on a given runner, TextIO can read
> > from HDFS, and we'd jointly validate those scenarios.
> >
> > * Vote
> > As usual.
> >
> > Time-wise, I think it would be really awesome to make this release
> coincide
> > with the ApacheCon conference that starts in about 2 weeks (on May
> 16th). I
> > think this would be a great goal -- and I'll do my part to make this
> > possible.
> >
> > Finally, we haven't formally closed on the version designation (see
> > separate thread).
> >
> > Thoughts? If there are no objections, I'll start this process soon, and
> we
> > can adjust as we go.
> >
> > Thanks!
> >
> > Davor
> >
> > [1] https://issues.apache.org/jira/issues/?jql=project%20%
> > 3D%20BEAM%20AND%20fixVersion%20%3D%20%22First%20stable%
> 20release%22%20AND%
> > 20resolution%20%3D%20Unresolved%20ORDER%20BY%20due%20ASC%2C%20priority%
> > 20DESC%2C%20created%20ASC
> > [2]
> > https://docs.google.com/document/d/1UKC2R_9FkSdMVTz2nt2sIW18KoLbIu6w0aj9
> > bwSSPiw/
> >
>

Re: Process for getting the first stable release out

[Thomas Weise <th...@gmail.com>]

Since the release would come with backward compatibility promise (?),
creation of the release branch would also affect what happens on master
from that point onwards. It would be good to provide some guidance on that?

--
sent from mobile
On May 4, 2017 12:08 PM, "Davor Bonaci" <da...@apache.org> wrote:

> We've been working on the first stable release for a while now -- we've
> been tracking blocking issues [1], resolved many of them, organized a
> hackathon [2], and improved testing coverage in various areas.
>
> I think we are nearly done -- there are a few more blocking issues left in
> JIRA, and probably a few more that we'll discover as we go along. That
> said, I think it is time to discuss the process for getting this release
> finalized.
>
> I'd like to propose the following (tweaked) process for this special
> release:
>
> * Create a release branch, and start building release candidates *now*
> This would accelerate branch creation compared to the normal process, but
> would separate the first stable release from other development on the
> master branch. This yields to stability and avoids unnecessary churn.
>
> * Community-driven acceptance criteria
> This would be an added step where everyone can recommend necessary
> conditions for acceptance of the release candidate. Those conditions would
> be *double-checked* manually, in addition to having all automated tests
> pass. This could include important scenarios that we want to be really sure
> about, e.g., WordCount example works on a given runner, TextIO can read
> from HDFS, and we'd jointly validate those scenarios.
>
> * Vote
> As usual.
>
> Time-wise, I think it would be really awesome to make this release coincide
> with the ApacheCon conference that starts in about 2 weeks (on May 16th). I
> think this would be a great goal -- and I'll do my part to make this
> possible.
>
> Finally, we haven't formally closed on the version designation (see
> separate thread).
>
> Thoughts? If there are no objections, I'll start this process soon, and we
> can adjust as we go.
>
> Thanks!
>
> Davor
>
> [1] https://issues.apache.org/jira/issues/?jql=project%20%
> 3D%20BEAM%20AND%20fixVersion%20%3D%20%22First%20stable%20release%22%20AND%
> 20resolution%20%3D%20Unresolved%20ORDER%20BY%20due%20ASC%2C%20priority%
> 20DESC%2C%20created%20ASC
> [2]
> https://docs.google.com/document/d/1UKC2R_9FkSdMVTz2nt2sIW18KoLbIu6w0aj9
> bwSSPiw/
>

Re: Process for getting the first stable release out

[Kenneth Knowles <kl...@google.com.INVALID>]

On Thu, May 4, 2017 at 12:07 PM, Davor Bonaci <da...@apache.org> wrote:

> I'd like to propose the following (tweaked) process for this special
> release:
>
> * Create a release branch, and start building release candidates *now*
> This would accelerate branch creation compared to the normal process, but
> would separate the first stable release from other development on the
> master branch. This yields to stability and avoids unnecessary churn.
>

+1 to cutting a release branch now.

This sounds compatible with the release process [1] to me, actually. This
thread seems like the dev@ thread where we "decide to release" and I agree
that we should decide to release. Certainly `master` is not ready nor is
the web site - there are ~29 issues as I write this though many are not
really significant code changes. But we should never wait until `master` is
"ready".

We know what we want to get done, and there are no radical changes, so I
think that makes this the right time to branch. We can easily cherry pick
fixes for our burndown list to ensure we don't introduce additional
blockers.

Some of the burndown list are of the form "investigate if this suspected
bug still repros" and a release candidate is the perfect thing to use for
that.

[1] https://beam.apache.org/contribute/release-guide/#decide-to-release