Re: Looking to a Hadoop 3 release

[Andrew Wang <an...@cloudera.com>]

A heads up that I think we're getting close on the blockers for the first
alpha. Looking at my list, I see two I'd like to get in still: YARN-5270
and HADOOP-13316. Will cut a branch and roll the release once those go in;
my test builds have looked good thus far.

My original plan was to do alphas and then beta in Aug/Sep, but given how
the create-release and L&N changes delayed us by a few months, it also
pushes out the beta timeframe. Given that Nov/Dec is often a quiet period
of development, I think a realistic new beta date is sometime early next
year (Jan/Feb). FYI.

Thanks,
Andrew

On Thu, May 12, 2016 at 5:20 PM, Karthik Kambatla <ka...@cloudera.com>
wrote:

> I am with Vinod on avoiding merging mostly_complete_branches to trunk since
> we are not shipping any release off it. If 3.x releases going off of trunk
> is going to help with this, I am fine with that approach. We should still
> make sure to keep trunk-incompat small and not include large features.
>
> On Sat, Apr 23, 2016 at 6:53 PM, Chris Douglas <cd...@apache.org>
> wrote:
>
> > If we're not starting branch-3/trunk, what would distinguish it from
> > trunk/trunk-incompat? Is it the same mechanism with different labels?
> >
> > That may be a reasonable strategy when we create branch-3, as a
> > release branch for beta. Releasing 3.x from trunk will help us figure
> > out which incompatibilities can be called out in an upgrade guide
> > (e.g., "new feature X is incompatible with uncommon configuration Y")
> > and which require code changes (e.g., "data loss upgrading a cluster
> > with feature X"). Given how long trunk has been unreleased, we need
> > more data from deployments to triage. How to manage transitions
> > between major versions will always be case-by-case; consensus on how
> > we'll address generic incompatible changes is not saving any work.
> >
> > Once created, removing functionality from branch-3 (leaving it in
> > trunk) _because_ nobody volunteers cycles to address urgent
> > compatibility issues is fair. It's also more workable than asking that
> > features be committed to a branch that we have no plan to release,
> > even as alpha. -C
> >
> > On Fri, Apr 22, 2016 at 6:50 PM, Vinod Kumar Vavilapalli
> > <vi...@apache.org> wrote:
> > > Tx for your replies, Andrew.
> > >
> > >>> For exit criteria, how about we time box it? My plan was to do
> monthly
> > >> alphas through the summer, leading up to beta in late August / early
> > Sep.
> > >> At that point we freeze and stabilize for GA in Nov/Dec.
> > >
> > >
> > > Time-boxing is a reasonable exit-criterion.
> > >
> > >
> > >> In this case, does trunk-incompat essentially become the new trunk? Or
> > are
> > >> we treating trunk-incompat as a feature branch, which periodically
> > merges
> > >> changes from trunk?
> > >
> > >
> > > It’s the later. Essentially
> > >  - trunk-incompat = trunk + only incompatible changes, periodically
> kept
> > up-to-date to trunk
> > >  - trunk is always ready to ship
> > >  - and no compatible code gets left behind
> > >
> > > The reason for my proposal like this is to address the tension between
> > “there is lot of compatible code in trunk that we are not shipping” and
> > “don’t ship trunk, it has incompatibilities”. With this, we will not have
> > (compatible) code not getting shipped to users.
> > >
> > > Obviously, we can forget about all of my proposal completely if
> everyone
> > puts in all compatible code into branch-2 / branch-3 or whatever the main
> > releasable branch is. This didn’t work in practice, have seen this not
> > happening prominently during 0.21, and now 3.x.
> > >
> > > There is another related issue - "my feature is nearly ready, so I’ll
> > just merge it into trunk as we don’t release that anyways, but not the
> > current releasable branch - I’m lazy to fix the last few stability
> related
> > issues”. With this, we will (should) get more disciplined, take feature
> > stability on a branch seriously and merge a feature branch only when it
> is
> > truly ready!
> > >
> > >> For 3.x, my strawman was to release off trunk for the alphas, then
> > branch a
> > >> branch-3 for the beta and onwards.
> > >
> > >
> > > Repeating above, I’m proposing continuing to make GA 3.x releases also
> > off of trunk! This way only incompatible changes don’t get shipped to
> users
> > - by design! Eventually, trunk-incompat will be latest 3.x GA + enough
> > incompatible code to warrant a 4.x, 5.x etc.
> > >
> > > +Vinod
> >
>