BIgtop 0.3.0 (was: [VOTE] Bigtop 0.3.0: target release date, supported platforms, bill of materials)

[Roman Shaposhnik <rv...@apache.org>]

Pulling this into a separate thread:

First of all,  I wanted to reiterate that the proposal was
from a stand point of me volunteering for a RM role for
Bigtop 0.3.0. It was intended to reflect my personal confidence
and the consensus that we've built in other thread (obviously
given -1, the consensus wasn't really there :-().

> 1/ Latest openSUSE and Fedora are respectively 12.1 and 16

You're absolutely correct. I specifically didn't say "latest" versions,
but stuck with the current ones. Is there any reason you feel
so strongly about bumping the versions of these OSes?

> 2/ We should move trunk to Apache Hadoop 0.23 before January. By then a
> few projects should have fixed their compatibility issue with Apache
> Hadoop enabling us to push for a release with more than just Hadoop 0.23.

I disagree strongly. By doing that we would be making any future release
of Bigtop hostage of all the components that we have to support on top of
.23. I will expected quite a few of them not to have official releases
compatible
with .23 till at least mid 2012. Does it mean that we don't do release of Bigtop
between now and (worst case scenario) Mid 2012?

>From my point of view -- that would be mostly unfortunate.

> 3/ I thought we would base releases based on a ~ quaterly schedule and
> not features.
> Regarding 2/, we should probably do a last release before moving to
> Hadoop 0.23.X (probably X =1 or 2) and use that branch for future
> versions of Hadoop 0.20.20X as stable branch.

That is, essentially, why I started the vote. I strongly believe that having
a release in mid-end Jan 2012 with so much new stuff in it will be
very beneficial to the Bigtop community. We will be putting long anticipated
versions of components (such as HBase 0.92 and Zookeeper 3.4) into
the hands of the users and that's what Bigtop is meant to do.

Yes, I would like to stick to the purely quarterly release schedule myself,
but having a 20.20X based release end of Feb is going to feel odd at best.

So the choice is a simple one -- have the next release (I don't care what
version we give to it as long as it is monotonically increasing from 0.2.0)
of Bigtop 2 months from now (and have it based on 0.20.200X) or have
a release of Bigtop (worst case scenario) in 6 months.

Thanks,
Roman.

Re: BIgtop 0.3.0 (was: [VOTE] Bigtop 0.3.0: target release date, supported platforms, bill of materials)

[Andrew Bayer <an...@gmail.com>]

I firmly believe we should not be releasing anything 0.23-related until a
critical mass of the components are compatible with 0.23.x - there's no
reason we can't get everything ready ahead of time in terms of packaging,
the already-ongoing work on patching components for compatibility, etc, so
that as soon as the stack is ready, we can release.

In fact, I'd personally tend towards waiting to release 0.3.0 until 0.23 is
ready - I don't see a real reason to release off trunk until there's
something significant to release, and 0.23.x and friends would be
significant. If we want to release incremental improvements on top of
0.2.0, do that as 0.2.1, .2, etc.

Mind you, I also firmly believe that we need to get 0.3.0 (based on 0.23.x)
out sooner than later - obviously, we're stuck waiting for components to be
ready, but we should push the components as we have been, we should
consider the possibility of temporarily dropping components from 0.3.0 if
they're being really tardy on compatibility (i.e., if they still don't have
a compatible release coming up by late January, say), and we should be
targeting a release of an 0.23.x-compatible stack, as 0.3.0, by mid
February. 0.2.x is for stability based on the 0.20.20x line, 0.3.x is for
potentially more bleeding-edge stuff based on the 0.23.x line.

Which branch this all is done on is irrelevant to me, really.

A.

On Tue, Nov 22, 2011 at 3:49 PM, Roman Shaposhnik <rv...@apache.org> wrote:

> On Tue, Nov 22, 2011 at 3:37 PM, Bruno Mahé <bm...@apache.org> wrote:
> >>> 1/ Latest openSUSE and Fedora are respectively 12.1 and 16
> >> You're absolutely correct. I specifically didn't say "latest" versions,
> >> but stuck with the current ones. Is there any reason you feel
> >> so strongly about bumping the versions of these OSes?
> >
> > I do all my work on latest Fedora/Mageia. So I will not test Fedora 15
> > at all. So you get support for Fedora 16 for free.
>
> Good point. As long as you're willing to support the neccessary
> infrastructure on bigtop Jenkins -- I have no objections to bumping
> Fedora version.
>
> > Hence my suggestion to have a stable branch for releases based on the
> > 0.20.20X generation and having trunk based on the next gen Hadoop.
>
> We had this discussion a couple of month ago and you, yourself, made
> a point that trunk should be reserved to changes which are stable.
> E.g. putting versions of unreleased Apache projects in trunk is a no-no.
>
> Any reason you're reversing your position now?
>
> > The stable branch being there for people wishing to use a stable
> > distribution and the coming releases of Bigtop to be used as a solid
> > base for helping projects improving their compatibility with Hadoop
> > 0.23.
>
> Sure and last time we discussed it, we had a consensus that it is called
> trunk.
>
> >This will also have the added benefit on helping putting Hadoop
> > 0.23 in more people's hands and improve Bigtop's Hadoop 23 support.
>
> I don't follow. There's absolutely no difference checking out trunk
> or checking out a branch and building the stack.  And it is THE only
> wait to get Bigtop built for .23. Or you can pull packages from our
> Jenkins job -- either way, I don't see how you can make it *easier*
> to get .23. Care to elaborate?
>
> > I wouldn't mind at all having a release of Bigtop with just Hadoop 0.23
> > + HBase + other compatible projects, and reactivating these projects one
> > by one as they get compatible with Hadoop 0.23.
>
> I would strongly -1 that decision. And I'm most certainly would not my name
> to be associated with such a release. Dropping components willy-nilly
> is the biggest threat to Bigtop's credibility as an Apache Bigtdata
> distribution.
>
> > So we don't have to wait mid 2012 to have a release of Bigtop based on
> > Hadoop 0.23.
>
> From my stand point -- we absolutely *have* to. Otherwise it will not be
> Bigtop we're talking about.
>
> > I am rather in favour of "release early and often" and therefore the
> > later option :)
>
> Now I'm totally confused -- with that motto, why do you NOT want to
> release 0.3.0?
>
> Thanks,
> Roman.
>

Re: BIgtop 0.3.0 (was: [VOTE] Bigtop 0.3.0: target release date, supported platforms, bill of materials)

[Bruno Mahé <bm...@apache.org>]

On 11/22/2011 03:49 PM, Roman Shaposhnik wrote:
> On Tue, Nov 22, 2011 at 3:37 PM, Bruno Mahé <bm...@apache.org> wrote:
>>>> 1/ Latest openSUSE and Fedora are respectively 12.1 and 16
>>> You're absolutely correct. I specifically didn't say "latest" versions,
>>> but stuck with the current ones. Is there any reason you feel
>>> so strongly about bumping the versions of these OSes?
>> I do all my work on latest Fedora/Mageia. So I will not test Fedora 15
>> at all. So you get support for Fedora 16 for free.
> Good point. As long as you're willing to support the neccessary
> infrastructure on bigtop Jenkins -- I have no objections to bumping
> Fedora version.

I can handle the Fedora side of things.

>> Hence my suggestion to have a stable branch for releases based on the
>> 0.20.20X generation and having trunk based on the next gen Hadoop.
> We had this discussion a couple of month ago and you, yourself, made
> a point that trunk should be reserved to changes which are stable.
> E.g. putting versions of unreleased Apache projects in trunk is a no-no.
>
> Any reason you're reversing your position now?
>

How am I reversing my position?
We are talking about released versions of Apache Hadoop 0.23.X which
will get increasingly more stable by then.


>> The stable branch being there for people wishing to use a stable
>> distribution and the coming releases of Bigtop to be used as a solid
>> base for helping projects improving their compatibility with Hadoop
>> 0.23.
> Sure and last time we discussed it, we had a consensus that it is called
> trunk.

0.23 is one of its kind :)

>> This will also have the added benefit on helping putting Hadoop
>> 0.23 in more people's hands and improve Bigtop's Hadoop 23 support.
> I don't follow. There's absolutely no difference checking out trunk
> or checking out a branch and building the stack.  And it is THE only
> wait to get Bigtop built for .23. Or you can pull packages from our
> Jenkins job -- either way, I don't see how you can make it *easier*
> to get .23. Care to elaborate?

We cannot call this a release. Whatever we put there, it can be as
awesome as we wish, it is pointless if it is not released.

>> I wouldn't mind at all having a release of Bigtop with just Hadoop 0.23
>> + HBase + other compatible projects, and reactivating these projects one
>> by one as they get compatible with Hadoop 0.23.
> I would strongly -1 that decision. And I'm most certainly would not my name
> to be associated with such a release. Dropping components willy-nilly
> is the biggest threat to Bigtop's credibility as an Apache Bigtdata
> distribution.

Hence the stable branch with all the components enabled.
But you called a vote and we will see what the community wants. I am not
the only member of this community :)

>> So we don't have to wait mid 2012 to have a release of Bigtop based on
>> Hadoop 0.23.
> From my stand point -- we absolutely *have* to. Otherwise it will not be
> Bigtop we're talking about.
>
>> I am rather in favour of "release early and often" and therefore the
>> later option :)
> Now I'm totally confused -- with that motto, why do you NOT want to
> release 0.3.0?

Who said I don't want to release 0.3.0?
I just disagree with the conditions of the release, not releasing in
itself :)

> Thanks,
> Roman.

Re: BIgtop 0.3.0 (was: [VOTE] Bigtop 0.3.0: target release date, supported platforms, bill of materials)

[Roman Shaposhnik <rv...@apache.org>]

On Tue, Nov 22, 2011 at 3:37 PM, Bruno Mahé <bm...@apache.org> wrote:
>>> 1/ Latest openSUSE and Fedora are respectively 12.1 and 16
>> You're absolutely correct. I specifically didn't say "latest" versions,
>> but stuck with the current ones. Is there any reason you feel
>> so strongly about bumping the versions of these OSes?
>
> I do all my work on latest Fedora/Mageia. So I will not test Fedora 15
> at all. So you get support for Fedora 16 for free.

Good point. As long as you're willing to support the neccessary
infrastructure on bigtop Jenkins -- I have no objections to bumping
Fedora version.

> Hence my suggestion to have a stable branch for releases based on the
> 0.20.20X generation and having trunk based on the next gen Hadoop.

We had this discussion a couple of month ago and you, yourself, made
a point that trunk should be reserved to changes which are stable.
E.g. putting versions of unreleased Apache projects in trunk is a no-no.

Any reason you're reversing your position now?

> The stable branch being there for people wishing to use a stable
> distribution and the coming releases of Bigtop to be used as a solid
> base for helping projects improving their compatibility with Hadoop
> 0.23.

Sure and last time we discussed it, we had a consensus that it is called
trunk.

>This will also have the added benefit on helping putting Hadoop
> 0.23 in more people's hands and improve Bigtop's Hadoop 23 support.

I don't follow. There's absolutely no difference checking out trunk
or checking out a branch and building the stack.  And it is THE only
wait to get Bigtop built for .23. Or you can pull packages from our
Jenkins job -- either way, I don't see how you can make it *easier*
to get .23. Care to elaborate?

> I wouldn't mind at all having a release of Bigtop with just Hadoop 0.23
> + HBase + other compatible projects, and reactivating these projects one
> by one as they get compatible with Hadoop 0.23.

I would strongly -1 that decision. And I'm most certainly would not my name
to be associated with such a release. Dropping components willy-nilly
is the biggest threat to Bigtop's credibility as an Apache Bigtdata
distribution.

> So we don't have to wait mid 2012 to have a release of Bigtop based on
> Hadoop 0.23.

>From my stand point -- we absolutely *have* to. Otherwise it will not be
Bigtop we're talking about.

> I am rather in favour of "release early and often" and therefore the
> later option :)

Now I'm totally confused -- with that motto, why do you NOT want to
release 0.3.0?

Thanks,
Roman.

Re: BIgtop 0.3.0 (was: [VOTE] Bigtop 0.3.0: target release date, supported platforms, bill of materials)

[Bruno Mahé <bm...@apache.org>]

See comments inline.

On 11/22/2011 03:11 PM, Roman Shaposhnik wrote:
> Pulling this into a separate thread:
>
> First of all,  I wanted to reiterate that the proposal was
> from a stand point of me volunteering for a RM role for
> Bigtop 0.3.0. It was intended to reflect my personal confidence
> and the consensus that we've built in other thread (obviously
> given -1, the consensus wasn't really there :-().
>
>> 1/ Latest openSUSE and Fedora are respectively 12.1 and 16
> You're absolutely correct. I specifically didn't say "latest" versions,
> but stuck with the current ones. Is there any reason you feel
> so strongly about bumping the versions of these OSes?

I do all my work on latest Fedora/Mageia. So I will not test Fedora 15
at all. So you get support for Fedora 16 for free.

>> 2/ We should move trunk to Apache Hadoop 0.23 before January. By then a
>> few projects should have fixed their compatibility issue with Apache
>> Hadoop enabling us to push for a release with more than just Hadoop 0.23.
> I disagree strongly. By doing that we would be making any future release
> of Bigtop hostage of all the components that we have to support on top of
> .23. I will expected quite a few of them not to have official releases
> compatible
> with .23 till at least mid 2012. Does it mean that we don't do release of Bigtop
> between now and (worst case scenario) Mid 2012?
>
> From my point of view -- that would be mostly unfortunate.
Hence my suggestion to have a stable branch for releases based on the
0.20.20X generation and having trunk based on the next gen Hadoop.
The stable branch being there for people wishing to use a stable
distribution and the coming releases of Bigtop to be used as a solid
base for helping projects improving their compatibility with Hadoop
0.23. This will also have the added benefit on helping putting Hadoop
0.23 in more people's hands and improve Bigtop's Hadoop 23 support.

I wouldn't mind at all having a release of Bigtop with just Hadoop 0.23
+ HBase + other compatible projects, and reactivating these projects one
by one as they get compatible with Hadoop 0.23.
So we don't have to wait mid 2012 to have a release of Bigtop based on
Hadoop 0.23.
But either way, we cannot wait for +10 projects to all sync up to get
Bigtop moving.

>> 3/ I thought we would base releases based on a ~ quaterly schedule and
>> not features.
>> Regarding 2/, we should probably do a last release before moving to
>> Hadoop 0.23.X (probably X =1 or 2) and use that branch for future
>> versions of Hadoop 0.20.20X as stable branch.
> That is, essentially, why I started the vote. I strongly believe that having
> a release in mid-end Jan 2012 with so much new stuff in it will be
> very beneficial to the Bigtop community. We will be putting long anticipated
> versions of components (such as HBase 0.92 and Zookeeper 3.4) into
> the hands of the users and that's what Bigtop is meant to do.
>
> Yes, I would like to stick to the purely quarterly release schedule myself,
> but having a 20.20X based release end of Feb is going to feel odd at best.
>
> So the choice is a simple one -- have the next release (I don't care what
> version we give to it as long as it is monotonically increasing from 0.2.0)
> of Bigtop 2 months from now (and have it based on 0.20.200X) or have
> a release of Bigtop (worst case scenario) in 6 months.

See my reply above.
I disagree with the choice you are presenting. This is about having
trunk as stable hadoop 0.20.20X and a unreleased hadoop 0.23 branch no
one will use, versus a branch for stable hadoop 0.20.20X and trunk for
future releases of Hadoop 0.23.
I am rather in favour of "release early and often" and therefore the
later option :)


> Thanks,
> Roman.