You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Andrew Purtell <ap...@apache.org> on 2011/02/26 21:34:17 UTC
putting a border around 0.92 release
Stack and I were chatting on IRC about settling with should get into 0.92 before pulling the trigger on the release.
Stack thinks we need online region schema editing. I agree because per-table coprocessor loading is configured via table attributes. We'd also need some kind of notification of schema update to trigger various actions in the regionserver. (For CPs, (re)load.)
I'd also really like to see some form of secondary indexing. This is an important feature for HBase to have. All of our in house devs ask for this sooner or later in one form or another. Other projects have options in this arena, while HBase used to in core, but no longer. We have three people starting on this ASAP. I'd like to at least do co-design with the community. We should aim for 'simple and effective'.
There are 14 blockers: https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+HBASE+AND+fixVersion+%3D+%220.92.0%22+AND+resolution+%3D+Unresolved+AND+priority+%3D+Blocker
Additionally, 22 marked as critical: https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+HBASE+AND+fixVersion+%3D+%220.92.0%22+AND+resolution+%3D+Unresolved+AND+priority+%3D+Critical
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back.
- Piet Hein (via Tom White)
Re: putting a border around 0.92 release
Posted by Stack <st...@duboce.net>.
Yeah, time-based. May 1st for 0.92 branch! +1 on big changes out on
branch keeping trunk relatively releasable at all times. I like your
list J-D. 0.92 should run on hadoop-0.20 through hadoop trunk at time
of our 0.92 branch. If all are good w/ that, lets do a pass over 0.92
issues so its reflective of what we think we might do in the 0.92
timeframe (will help Lars too).
St.Ack
Re: putting a border around 0.92 release
Posted by Ryan Rawson <ry...@gmail.com>.
I'd generally vote for a time-based release. The "big feature"
releases while are good for attracting new users with new features,
present a problem in that it can really delay releases for a long
time. More releases are better! If a feature takes more than 3
months then it's too big to implement in one go.
On Mon, Feb 28, 2011 at 2:00 PM, Todd Lipcon <to...@cloudera.com> wrote:
> On Sat, Feb 26, 2011 at 2:24 PM, Jean-Daniel Cryans <jd...@apache.org> wrote:
>> Woah those are huge tasks!
>>
>> Also to consider:
>>
>> - integration with hadoop 0.22, should we support it and should we
>> also support 0.20 at the same time? 0.22 was branched but IIRC it
>> still has quite a few blockers.
>> - removing heartbeats, this is in the pipeline from Stack and IMO
>> will have ripple effects on online schema editing.
>> - HBASE-2856, pretty critical.
>> - replication-related issues like multi-slave (which I'm working on),
>> and ideally multi-master. I'd like to add better management tools too.
>>
>> And lastly we need to plan when we want to branch 0.92... should we
>> target late May in order to be ready for the Hadoop Summit in June?
>> For once it would be nice to offer more than an alpha release :)
>
> In my view, we can do one or the other: either it's a feature-based
> release, in which case we "release it when it's done", or it's a
> time-based release, in which case we release at some decided-upon time
> with whatever's done.
>
> I personally prefer time-based releases, though we need to make sure
> if we decide to do this that any large destabilizing (or half
> complete) features are guarded either by config flags or are developed
> in a branch. Thus trunk stays relatively releasable at all times and
> we can be pretty confident we'll hit the decided-upon timeline.
>
> Looking back at the 0.90 release, we got caught in a bind because we
> were trying to do both feature-based (new master) and time-based (end
> of 2010).
>
> So, my vote is either:
> plan a: hybrid model - 0.91.X becomes a time-based release series
> where we drop trunk once every month or two, and 0.92.0 is gated on
> features
> or:
> plan b: strict time-based: we release 0.92.0 around summit, and lock
> down the branch at least a month or so ahead of time for bugfix only.
>
> Thoughts?
>
> -Todd
>
>
>>
>> On Sat, Feb 26, 2011 at 12:34 PM, Andrew Purtell <ap...@apache.org> wrote:
>>> Stack and I were chatting on IRC about settling with should get into 0.92 before pulling the trigger on the release.
>>>
>>> Stack thinks we need online region schema editing. I agree because per-table coprocessor loading is configured via table attributes. We'd also need some kind of notification of schema update to trigger various actions in the regionserver. (For CPs, (re)load.)
>>>
>>> I'd also really like to see some form of secondary indexing. This is an important feature for HBase to have. All of our in house devs ask for this sooner or later in one form or another. Other projects have options in this arena, while HBase used to in core, but no longer. We have three people starting on this ASAP. I'd like to at least do co-design with the community. We should aim for 'simple and effective'.
>>>
>>> There are 14 blockers: https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+HBASE+AND+fixVersion+%3D+%220.92.0%22+AND+resolution+%3D+Unresolved+AND+priority+%3D+Blocker
>>>
>>> Additionally, 22 marked as critical: https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+HBASE+AND+fixVersion+%3D+%220.92.0%22+AND+resolution+%3D+Unresolved+AND+priority+%3D+Critical
>>>
>>> Best regards,
>>>
>>> - Andy
>>>
>>> Problems worthy of attack prove their worth by hitting back.
>>> - Piet Hein (via Tom White)
>>>
>>>
>>>
>>>
>>
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>
Re: putting a border around 0.92 release
Posted by Stack <st...@duboce.net>.
I like how you put it Gary and go with your reasoning that for our
development velocity is not (yet) sufficient for linux kernel river of
releases (w/ Todd fishing out the odd one to stabilize on). I agree
even/odd for now and timeboxing is the way to go, especially after
finishing the Linus interview where he's feature/smeature, they come
and go; the real question is are they ready, w/ users and support.
I said May 1st cut-off above. After reading the article, I'm thinking
April 15th if not April 1st (or wait to settle at the Hackathon).
St.Ack
On Wed, Mar 2, 2011 at 3:46 PM, Gary Helmling <gh...@gmail.com> wrote:
> I think it's very relevant that the linux kernel moved from one approach to
> the other as it reached a certain scale of developer participation and
> broader ecosystem support. I think what really enables the "new linux"
> approach is the fact that there are multiple layers of stabilization between
> the kernel.org releases and what the average end user actually runs (a
> stable vintage maintainer, distributor kernel teams, distributions with
> their own stable and more frequent releases). This is really well
> articulated in the Linus interview sent around (thanks for the link Todd!).
>
> I think it's great that Cloudera is starting to play this role for HBase.
> And I expect we'll continue to see the ecosystem expand. But many people
> run the Apache releases and I would hate to add any confusion to which
> release people should download and run. (We already have Hadoop
> contributing enough confusion there).
>
> IMO, we're also not yet at the scale of developer participation where we
> have lots of free cycles to volunteer to maintain a specific "stable"
> release. I certainly feel too busy for it, and the rest of you all seem
> even busier than me!
>
> I do believe we should timebox our releases to have a predictable schedule
> and punt features if they're not ready. Maybe that alone produces enough
> friction against the pain of upgrading large-scale, long running clusters
> that "stable vintages" will start to seem worthwhile, but I'm not totally
> convinced yet. Let's get some iterative release rhythm going first.
>
> To this end, I think full read/write git support from ASF infra would help a
> lot. It would close the loop on more easily handling feature-branch
> development. Github goes a long way, but that last bit of: apply patch to
> svn, then remerge back to git is sometimes annoying. Anyone know of any
> more progress on this: https://issues.apache.org/jira/browse/INFRA-3165 ?
>
> So my vote is to do odd-even stable for now. But let's get to the point
> where we're iterating frequently enough and have grown enough hands that the
> "stable vintages" becomes a better alternative and we can drop the odd
> releases completely.
>
> Can we set a goal of targeting a 0.92 release date at the hackathon (or at
> least RC target date)?
>
> --gh
>
>
> On Wed, Mar 2, 2011 at 2:50 PM, Todd Lipcon <to...@cloudera.com> wrote:
>
>> On Wed, Mar 2, 2011 at 2:44 PM, Andrew Purtell <ap...@yahoo.com> wrote:
>> > My vote is odd-even stable.
>> >
>> > I don't like the idea of vintages. We have this situation internally, a
>> release based on 0.20.6 that I won't be able to kill fast enough now that we
>> have something based on 0.90.0. If I needed to support the 0.20 "vintage"
>> I'd have someone working full time on making 0.90.0 API and RPC compatible
>> with 0.20.6 and upgradeable via rolling restart. A total nightmare.
>>
>> Of course vintages get killed off after a certain time. Also, to a
>> certain extent this becomes the job of distributors. For example,
>> Cloudera will be supporting 0.90.1 (or something that's 100%
>> compatible with it) for at least the next year, regardless of how
>> whiz-bang 0.92 or future releases will be.
>>
>> -Todd
>>
>> >
>> > --- On Wed, 3/2/11, Todd Lipcon <to...@cloudera.com> wrote:
>> >
>> >> From: Todd Lipcon <to...@cloudera.com>
>> >> Subject: Re: putting a border around 0.92 release
>> >> To: dev@hbase.apache.org
>> >> Cc: "Jean-Daniel Cryans" <jd...@apache.org>
>> >> Date: Wednesday, March 2, 2011, 2:39 PM
>> >> On Tue, Mar 1, 2011 at 9:34 AM,
>> >> Jean-Daniel Cryans <jd...@apache.org>
>> >> wrote:
>> >> >
>> >> > That's a pretty good interview, thanks for sharing!
>> >> Soooo is our idea
>> >> > of using odd numbered release as DRs bad? Should we
>> >> just release more
>> >> > often like every 3 months? I vote Stack as our Linus!
>> >>
>> >> I think there are basically two ways of going about this,
>> >> and it
>> >> depends on a philosophical choice:
>> >>
>> >> The old Linux way (odd-even stable):
>> >> - users knew that 2.3.x and 2.5.x were development series
>> >> and wouldn't
>> >> be stable.
>> >> - distros and almost every user ran off even-numbered
>> >> releases
>> >> - odd-number releases might be fairly broken
>> >> - there's some pre-release decision that the release will
>> >> be an
>> >> even-numbered "stable"/"good" release as opposed to a
>> >> odd-numbered
>> >> dev-quality release.
>> >>
>> >> This is basically what we did for 0.89.x. I thought it
>> >> worked fairly well.
>> >>
>> >>
>> >> The new Linux way ("good vintages and long-lived
>> >> branches"):
>> >> - new releases are done time-based and there's no guarantee
>> >> of any
>> >> particular stability level
>> >> - users rely on some documentation somewhere to know what a
>> >> "good vintage" is
>> >> - either distributors or some section of the community
>> >> steps up to
>> >> designate some good vintage to be a long-lived branch,
>> >> where critical
>> >> bugfixes will get backported. eg 2.6.18 was a good vintage,
>> >> 2.6.32 is
>> >> another.
>> >> - the "good vintages" aren't decided at release-time, but
>> >> rather
>> >> retroactively - ie we just release on the train model, and
>> >> at some
>> >> point someone takes one of those trains that "seems pretty
>> >> good" and
>> >> starts stabilizing it up.
>> >>
>> >> The interesting thing about the "good vintages" model is
>> >> that you
>> >> don't follow the usual policy of a bugfix having to be
>> >> backported to
>> >> all releases in between trunk and the vintage. Let's
>> >> pretend we had
>> >> 0.90, 0.91, 0.92, 0.93, and 0.94, where 0.90 was the "good
>> >> vintage"
>> >> release. If we commit a fix to trunk in today's Apache
>> >> model, we'd
>> >> also have to commit to ALL of those releases back through
>> >> 0.90, which
>> >> is a giant pain.
>> >>
>> >> The other interesting (and more philosophical) bit is that
>> >> the "good
>> >> vintage" model serves to separate the developers from the
>> >> users, in
>> >> that the users are expected to be several releases back on
>> >> a stable
>> >> branch. This can be both good and bad - bad because it
>> >> takes a while
>> >> for a new feature to trickle to a user, good because
>> >> changes go
>> >> through a few layers of vetting for bugs before they hit
>> >> users.
>> >>
>> >> Any thoughts?
>> >>
>> >> -Todd
>> >> --
>> >> Todd Lipcon
>> >> Software Engineer, Cloudera
>> >>
>> >
>> >
>> >
>> >
>>
>>
>>
>> --
>> Todd Lipcon
>> Software Engineer, Cloudera
>>
>
Re: putting a border around 0.92 release
Posted by Gary Helmling <gh...@gmail.com>.
I think it's very relevant that the linux kernel moved from one approach to
the other as it reached a certain scale of developer participation and
broader ecosystem support. I think what really enables the "new linux"
approach is the fact that there are multiple layers of stabilization between
the kernel.org releases and what the average end user actually runs (a
stable vintage maintainer, distributor kernel teams, distributions with
their own stable and more frequent releases). This is really well
articulated in the Linus interview sent around (thanks for the link Todd!).
I think it's great that Cloudera is starting to play this role for HBase.
And I expect we'll continue to see the ecosystem expand. But many people
run the Apache releases and I would hate to add any confusion to which
release people should download and run. (We already have Hadoop
contributing enough confusion there).
IMO, we're also not yet at the scale of developer participation where we
have lots of free cycles to volunteer to maintain a specific "stable"
release. I certainly feel too busy for it, and the rest of you all seem
even busier than me!
I do believe we should timebox our releases to have a predictable schedule
and punt features if they're not ready. Maybe that alone produces enough
friction against the pain of upgrading large-scale, long running clusters
that "stable vintages" will start to seem worthwhile, but I'm not totally
convinced yet. Let's get some iterative release rhythm going first.
To this end, I think full read/write git support from ASF infra would help a
lot. It would close the loop on more easily handling feature-branch
development. Github goes a long way, but that last bit of: apply patch to
svn, then remerge back to git is sometimes annoying. Anyone know of any
more progress on this: https://issues.apache.org/jira/browse/INFRA-3165 ?
So my vote is to do odd-even stable for now. But let's get to the point
where we're iterating frequently enough and have grown enough hands that the
"stable vintages" becomes a better alternative and we can drop the odd
releases completely.
Can we set a goal of targeting a 0.92 release date at the hackathon (or at
least RC target date)?
--gh
On Wed, Mar 2, 2011 at 2:50 PM, Todd Lipcon <to...@cloudera.com> wrote:
> On Wed, Mar 2, 2011 at 2:44 PM, Andrew Purtell <ap...@yahoo.com> wrote:
> > My vote is odd-even stable.
> >
> > I don't like the idea of vintages. We have this situation internally, a
> release based on 0.20.6 that I won't be able to kill fast enough now that we
> have something based on 0.90.0. If I needed to support the 0.20 "vintage"
> I'd have someone working full time on making 0.90.0 API and RPC compatible
> with 0.20.6 and upgradeable via rolling restart. A total nightmare.
>
> Of course vintages get killed off after a certain time. Also, to a
> certain extent this becomes the job of distributors. For example,
> Cloudera will be supporting 0.90.1 (or something that's 100%
> compatible with it) for at least the next year, regardless of how
> whiz-bang 0.92 or future releases will be.
>
> -Todd
>
> >
> > --- On Wed, 3/2/11, Todd Lipcon <to...@cloudera.com> wrote:
> >
> >> From: Todd Lipcon <to...@cloudera.com>
> >> Subject: Re: putting a border around 0.92 release
> >> To: dev@hbase.apache.org
> >> Cc: "Jean-Daniel Cryans" <jd...@apache.org>
> >> Date: Wednesday, March 2, 2011, 2:39 PM
> >> On Tue, Mar 1, 2011 at 9:34 AM,
> >> Jean-Daniel Cryans <jd...@apache.org>
> >> wrote:
> >> >
> >> > That's a pretty good interview, thanks for sharing!
> >> Soooo is our idea
> >> > of using odd numbered release as DRs bad? Should we
> >> just release more
> >> > often like every 3 months? I vote Stack as our Linus!
> >>
> >> I think there are basically two ways of going about this,
> >> and it
> >> depends on a philosophical choice:
> >>
> >> The old Linux way (odd-even stable):
> >> - users knew that 2.3.x and 2.5.x were development series
> >> and wouldn't
> >> be stable.
> >> - distros and almost every user ran off even-numbered
> >> releases
> >> - odd-number releases might be fairly broken
> >> - there's some pre-release decision that the release will
> >> be an
> >> even-numbered "stable"/"good" release as opposed to a
> >> odd-numbered
> >> dev-quality release.
> >>
> >> This is basically what we did for 0.89.x. I thought it
> >> worked fairly well.
> >>
> >>
> >> The new Linux way ("good vintages and long-lived
> >> branches"):
> >> - new releases are done time-based and there's no guarantee
> >> of any
> >> particular stability level
> >> - users rely on some documentation somewhere to know what a
> >> "good vintage" is
> >> - either distributors or some section of the community
> >> steps up to
> >> designate some good vintage to be a long-lived branch,
> >> where critical
> >> bugfixes will get backported. eg 2.6.18 was a good vintage,
> >> 2.6.32 is
> >> another.
> >> - the "good vintages" aren't decided at release-time, but
> >> rather
> >> retroactively - ie we just release on the train model, and
> >> at some
> >> point someone takes one of those trains that "seems pretty
> >> good" and
> >> starts stabilizing it up.
> >>
> >> The interesting thing about the "good vintages" model is
> >> that you
> >> don't follow the usual policy of a bugfix having to be
> >> backported to
> >> all releases in between trunk and the vintage. Let's
> >> pretend we had
> >> 0.90, 0.91, 0.92, 0.93, and 0.94, where 0.90 was the "good
> >> vintage"
> >> release. If we commit a fix to trunk in today's Apache
> >> model, we'd
> >> also have to commit to ALL of those releases back through
> >> 0.90, which
> >> is a giant pain.
> >>
> >> The other interesting (and more philosophical) bit is that
> >> the "good
> >> vintage" model serves to separate the developers from the
> >> users, in
> >> that the users are expected to be several releases back on
> >> a stable
> >> branch. This can be both good and bad - bad because it
> >> takes a while
> >> for a new feature to trickle to a user, good because
> >> changes go
> >> through a few layers of vetting for bugs before they hit
> >> users.
> >>
> >> Any thoughts?
> >>
> >> -Todd
> >> --
> >> Todd Lipcon
> >> Software Engineer, Cloudera
> >>
> >
> >
> >
> >
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>
Re: putting a border around 0.92 release
Posted by Todd Lipcon <to...@cloudera.com>.
On Wed, Mar 2, 2011 at 2:44 PM, Andrew Purtell <ap...@yahoo.com> wrote:
> My vote is odd-even stable.
>
> I don't like the idea of vintages. We have this situation internally, a release based on 0.20.6 that I won't be able to kill fast enough now that we have something based on 0.90.0. If I needed to support the 0.20 "vintage" I'd have someone working full time on making 0.90.0 API and RPC compatible with 0.20.6 and upgradeable via rolling restart. A total nightmare.
Of course vintages get killed off after a certain time. Also, to a
certain extent this becomes the job of distributors. For example,
Cloudera will be supporting 0.90.1 (or something that's 100%
compatible with it) for at least the next year, regardless of how
whiz-bang 0.92 or future releases will be.
-Todd
>
> --- On Wed, 3/2/11, Todd Lipcon <to...@cloudera.com> wrote:
>
>> From: Todd Lipcon <to...@cloudera.com>
>> Subject: Re: putting a border around 0.92 release
>> To: dev@hbase.apache.org
>> Cc: "Jean-Daniel Cryans" <jd...@apache.org>
>> Date: Wednesday, March 2, 2011, 2:39 PM
>> On Tue, Mar 1, 2011 at 9:34 AM,
>> Jean-Daniel Cryans <jd...@apache.org>
>> wrote:
>> >
>> > That's a pretty good interview, thanks for sharing!
>> Soooo is our idea
>> > of using odd numbered release as DRs bad? Should we
>> just release more
>> > often like every 3 months? I vote Stack as our Linus!
>>
>> I think there are basically two ways of going about this,
>> and it
>> depends on a philosophical choice:
>>
>> The old Linux way (odd-even stable):
>> - users knew that 2.3.x and 2.5.x were development series
>> and wouldn't
>> be stable.
>> - distros and almost every user ran off even-numbered
>> releases
>> - odd-number releases might be fairly broken
>> - there's some pre-release decision that the release will
>> be an
>> even-numbered "stable"/"good" release as opposed to a
>> odd-numbered
>> dev-quality release.
>>
>> This is basically what we did for 0.89.x. I thought it
>> worked fairly well.
>>
>>
>> The new Linux way ("good vintages and long-lived
>> branches"):
>> - new releases are done time-based and there's no guarantee
>> of any
>> particular stability level
>> - users rely on some documentation somewhere to know what a
>> "good vintage" is
>> - either distributors or some section of the community
>> steps up to
>> designate some good vintage to be a long-lived branch,
>> where critical
>> bugfixes will get backported. eg 2.6.18 was a good vintage,
>> 2.6.32 is
>> another.
>> - the "good vintages" aren't decided at release-time, but
>> rather
>> retroactively - ie we just release on the train model, and
>> at some
>> point someone takes one of those trains that "seems pretty
>> good" and
>> starts stabilizing it up.
>>
>> The interesting thing about the "good vintages" model is
>> that you
>> don't follow the usual policy of a bugfix having to be
>> backported to
>> all releases in between trunk and the vintage. Let's
>> pretend we had
>> 0.90, 0.91, 0.92, 0.93, and 0.94, where 0.90 was the "good
>> vintage"
>> release. If we commit a fix to trunk in today's Apache
>> model, we'd
>> also have to commit to ALL of those releases back through
>> 0.90, which
>> is a giant pain.
>>
>> The other interesting (and more philosophical) bit is that
>> the "good
>> vintage" model serves to separate the developers from the
>> users, in
>> that the users are expected to be several releases back on
>> a stable
>> branch. This can be both good and bad - bad because it
>> takes a while
>> for a new feature to trickle to a user, good because
>> changes go
>> through a few layers of vetting for bugs before they hit
>> users.
>>
>> Any thoughts?
>>
>> -Todd
>> --
>> Todd Lipcon
>> Software Engineer, Cloudera
>>
>
>
>
>
--
Todd Lipcon
Software Engineer, Cloudera
Re: putting a border around 0.92 release
Posted by Andrew Purtell <ap...@yahoo.com>.
My vote is odd-even stable.
I don't like the idea of vintages. We have this situation internally, a release based on 0.20.6 that I won't be able to kill fast enough now that we have something based on 0.90.0. If I needed to support the 0.20 "vintage" I'd have someone working full time on making 0.90.0 API and RPC compatible with 0.20.6 and upgradeable via rolling restart. A total nightmare.
- Andy
Problems worthy of attack prove their worth by hitting back.
- Piet Hein (via Tom White)
--- On Wed, 3/2/11, Todd Lipcon <to...@cloudera.com> wrote:
> From: Todd Lipcon <to...@cloudera.com>
> Subject: Re: putting a border around 0.92 release
> To: dev@hbase.apache.org
> Cc: "Jean-Daniel Cryans" <jd...@apache.org>
> Date: Wednesday, March 2, 2011, 2:39 PM
> On Tue, Mar 1, 2011 at 9:34 AM,
> Jean-Daniel Cryans <jd...@apache.org>
> wrote:
> >
> > That's a pretty good interview, thanks for sharing!
> Soooo is our idea
> > of using odd numbered release as DRs bad? Should we
> just release more
> > often like every 3 months? I vote Stack as our Linus!
>
> I think there are basically two ways of going about this,
> and it
> depends on a philosophical choice:
>
> The old Linux way (odd-even stable):
> - users knew that 2.3.x and 2.5.x were development series
> and wouldn't
> be stable.
> - distros and almost every user ran off even-numbered
> releases
> - odd-number releases might be fairly broken
> - there's some pre-release decision that the release will
> be an
> even-numbered "stable"/"good" release as opposed to a
> odd-numbered
> dev-quality release.
>
> This is basically what we did for 0.89.x. I thought it
> worked fairly well.
>
>
> The new Linux way ("good vintages and long-lived
> branches"):
> - new releases are done time-based and there's no guarantee
> of any
> particular stability level
> - users rely on some documentation somewhere to know what a
> "good vintage" is
> - either distributors or some section of the community
> steps up to
> designate some good vintage to be a long-lived branch,
> where critical
> bugfixes will get backported. eg 2.6.18 was a good vintage,
> 2.6.32 is
> another.
> - the "good vintages" aren't decided at release-time, but
> rather
> retroactively - ie we just release on the train model, and
> at some
> point someone takes one of those trains that "seems pretty
> good" and
> starts stabilizing it up.
>
> The interesting thing about the "good vintages" model is
> that you
> don't follow the usual policy of a bugfix having to be
> backported to
> all releases in between trunk and the vintage. Let's
> pretend we had
> 0.90, 0.91, 0.92, 0.93, and 0.94, where 0.90 was the "good
> vintage"
> release. If we commit a fix to trunk in today's Apache
> model, we'd
> also have to commit to ALL of those releases back through
> 0.90, which
> is a giant pain.
>
> The other interesting (and more philosophical) bit is that
> the "good
> vintage" model serves to separate the developers from the
> users, in
> that the users are expected to be several releases back on
> a stable
> branch. This can be both good and bad - bad because it
> takes a while
> for a new feature to trickle to a user, good because
> changes go
> through a few layers of vetting for bugs before they hit
> users.
>
> Any thoughts?
>
> -Todd
> --
> Todd Lipcon
> Software Engineer, Cloudera
>
Re: putting a border around 0.92 release
Posted by Todd Lipcon <to...@cloudera.com>.
On Tue, Mar 1, 2011 at 9:34 AM, Jean-Daniel Cryans <jd...@apache.org> wrote:
>
> That's a pretty good interview, thanks for sharing! Soooo is our idea
> of using odd numbered release as DRs bad? Should we just release more
> often like every 3 months? I vote Stack as our Linus!
I think there are basically two ways of going about this, and it
depends on a philosophical choice:
The old Linux way (odd-even stable):
- users knew that 2.3.x and 2.5.x were development series and wouldn't
be stable.
- distros and almost every user ran off even-numbered releases
- odd-number releases might be fairly broken
- there's some pre-release decision that the release will be an
even-numbered "stable"/"good" release as opposed to a odd-numbered
dev-quality release.
This is basically what we did for 0.89.x. I thought it worked fairly well.
The new Linux way ("good vintages and long-lived branches"):
- new releases are done time-based and there's no guarantee of any
particular stability level
- users rely on some documentation somewhere to know what a "good vintage" is
- either distributors or some section of the community steps up to
designate some good vintage to be a long-lived branch, where critical
bugfixes will get backported. eg 2.6.18 was a good vintage, 2.6.32 is
another.
- the "good vintages" aren't decided at release-time, but rather
retroactively - ie we just release on the train model, and at some
point someone takes one of those trains that "seems pretty good" and
starts stabilizing it up.
The interesting thing about the "good vintages" model is that you
don't follow the usual policy of a bugfix having to be backported to
all releases in between trunk and the vintage. Let's pretend we had
0.90, 0.91, 0.92, 0.93, and 0.94, where 0.90 was the "good vintage"
release. If we commit a fix to trunk in today's Apache model, we'd
also have to commit to ALL of those releases back through 0.90, which
is a giant pain.
The other interesting (and more philosophical) bit is that the "good
vintage" model serves to separate the developers from the users, in
that the users are expected to be several releases back on a stable
branch. This can be both good and bad - bad because it takes a while
for a new feature to trickle to a user, good because changes go
through a few layers of vetting for bugs before they hit users.
Any thoughts?
-Todd
--
Todd Lipcon
Software Engineer, Cloudera
Re: putting a border around 0.92 release
Posted by Jean-Daniel Cryans <jd...@apache.org>.
I knew I should I have used <rant> </rant> around my reply!
>
> Was the second 0.89 a failure? If I recall correctly, various people
> ran it, found bugs, and reported them. We then fixed the bugs for the
> third 0.89 release. Doing the broken one got us all testing a similar
> set of bits (including some more adventurous users) and we used that
> feedback to fix things up. 0.90.0 had a few bugs, but in general I'd
> say 0.90.1 is a pretty nice release.
I was not saying it was a failure, my point was more like "branching
for some features might be good, but it's not a general solution". The
master rewrite was a pretty huge change that affected everything,
maybe it's just a special case. In any case, DR 2 wasn't really usable
without a lot of fixes to the master and in DR 3 we rolled back most
of that stuff. At the same time, we were badly in need for a release
so having "prod ready" DRs kind of became necessary. Maybe DR 3 should
have been called 0.90 instead?
>
> If anything I wish we'd done one more 0.89 release after the new
> master got merged :) Even if it's broken, it's a valuable thing to
> learn from.
>
I agree, we were a long time without any sort of release.
> I mentioned this interview to a few folks last week -- I thought it
> was pretty interesting:
> http://www.itwire.com/opinion-and-analysis/open-sauce/44975-linus-torvalds-looking-back-looking-forward?tmpl=component&print=1&layout=default&page=
That's a pretty good interview, thanks for sharing! Soooo is our idea
of using odd numbered release as DRs bad? Should we just release more
often like every 3 months? I vote Stack as our Linus!
J-D
Re: putting a border around 0.92 release
Posted by Andrew Purtell <ap...@apache.org>.
> From: Todd Lipcon <to...@cloudera.com>
> Was the second 0.89 a failure? If I recall correctly, various people
> ran it, found bugs, and reported them. We then fixed the bugs for the
> third 0.89 release. Doing the broken one got us all testing a similar
> set of bits (including some more adventurous users) and we used that
I do not feel that putting out a broken DR is a failure by definition.
There's no excuse for releasing known toxic waste but a DR that has issues may have issues not found until users try out real applications. That is something very valuable about open source, that a subset of users exist interested in doing this.
- Andy
Re: putting a border around 0.92 release
Posted by Todd Lipcon <to...@cloudera.com>.
On Mon, Feb 28, 2011 at 2:18 PM, Jean-Daniel Cryans <jd...@apache.org> wrote:
> I think we tried a couple of times to release according to a schedule
> in the past but always failed, be it 0.2, 0.20 or 0.90, and always
> ended with something like an alpha or preview release at agreed-upon
> time.
>
> On on hand, releasing more often would give users access to features
> earlier (those already committed). On the other hand, when you are
> developing one feature and it takes a lot of time, you'd like to delay
> a release to have it included (kudos to the TM team who agreed on not
> putting CP in 0.90 when we thought it was only a few weeks from
> release and it ended up taking a few months to stabilize).
Yep, but I think this has been successful. The CP API as I understand
it has gone through a couple incompatible changes while living in
trunk as different people have started experimenting with it. Keeping
it away from the general userbase has allowed the API to coalesce into
something full-featured while not having to support a bunch of
backward-compatibility issues.
While we always have the temptation to throw out new features to
users, we have to keep in mind the support burden as well :)
>
> I'm +1 on time-based releases, but keeping huge patches in separate
> branches doesn't seem such a good idea in retrospect. We tried to do
> it for the master in 0.90 and it was just impossible to review/merge
> back... I don't really have a better solution in mind tho. Piecemeal
> patches kind of work sometimes, but let's all remind ourselves of the
> second 0.89...
Was the second 0.89 a failure? If I recall correctly, various people
ran it, found bugs, and reported them. We then fixed the bugs for the
third 0.89 release. Doing the broken one got us all testing a similar
set of bits (including some more adventurous users) and we used that
feedback to fix things up. 0.90.0 had a few bugs, but in general I'd
say 0.90.1 is a pretty nice release.
If anything I wish we'd done one more 0.89 release after the new
master got merged :) Even if it's broken, it's a valuable thing to
learn from.
I mentioned this interview to a few folks last week -- I thought it
was pretty interesting:
http://www.itwire.com/opinion-and-analysis/open-sauce/44975-linus-torvalds-looking-back-looking-forward?tmpl=component&print=1&layout=default&page=
-Todd
>
> On Mon, Feb 28, 2011 at 2:00 PM, Todd Lipcon <to...@cloudera.com> wrote:
>> In my view, we can do one or the other: either it's a feature-based
>> release, in which case we "release it when it's done", or it's a
>> time-based release, in which case we release at some decided-upon time
>> with whatever's done.
>>
>> I personally prefer time-based releases, though we need to make sure
>> if we decide to do this that any large destabilizing (or half
>> complete) features are guarded either by config flags or are developed
>> in a branch. Thus trunk stays relatively releasable at all times and
>> we can be pretty confident we'll hit the decided-upon timeline.
>>
>> Looking back at the 0.90 release, we got caught in a bind because we
>> were trying to do both feature-based (new master) and time-based (end
>> of 2010).
>>
>> So, my vote is either:
>> plan a: hybrid model - 0.91.X becomes a time-based release series
>> where we drop trunk once every month or two, and 0.92.0 is gated on
>> features
>> or:
>> plan b: strict time-based: we release 0.92.0 around summit, and lock
>> down the branch at least a month or so ahead of time for bugfix only.
>>
>> Thoughts?
>>
>> -Todd
>>
>>
>
--
Todd Lipcon
Software Engineer, Cloudera
Re: putting a border around 0.92 release
Posted by Jean-Daniel Cryans <jd...@apache.org>.
I think we tried a couple of times to release according to a schedule
in the past but always failed, be it 0.2, 0.20 or 0.90, and always
ended with something like an alpha or preview release at agreed-upon
time.
On on hand, releasing more often would give users access to features
earlier (those already committed). On the other hand, when you are
developing one feature and it takes a lot of time, you'd like to delay
a release to have it included (kudos to the TM team who agreed on not
putting CP in 0.90 when we thought it was only a few weeks from
release and it ended up taking a few months to stabilize).
I'm +1 on time-based releases, but keeping huge patches in separate
branches doesn't seem such a good idea in retrospect. We tried to do
it for the master in 0.90 and it was just impossible to review/merge
back... I don't really have a better solution in mind tho. Piecemeal
patches kind of work sometimes, but let's all remind ourselves of the
second 0.89...
Let's do our best effort?
J-D
On Mon, Feb 28, 2011 at 2:00 PM, Todd Lipcon <to...@cloudera.com> wrote:
> In my view, we can do one or the other: either it's a feature-based
> release, in which case we "release it when it's done", or it's a
> time-based release, in which case we release at some decided-upon time
> with whatever's done.
>
> I personally prefer time-based releases, though we need to make sure
> if we decide to do this that any large destabilizing (or half
> complete) features are guarded either by config flags or are developed
> in a branch. Thus trunk stays relatively releasable at all times and
> we can be pretty confident we'll hit the decided-upon timeline.
>
> Looking back at the 0.90 release, we got caught in a bind because we
> were trying to do both feature-based (new master) and time-based (end
> of 2010).
>
> So, my vote is either:
> plan a: hybrid model - 0.91.X becomes a time-based release series
> where we drop trunk once every month or two, and 0.92.0 is gated on
> features
> or:
> plan b: strict time-based: we release 0.92.0 around summit, and lock
> down the branch at least a month or so ahead of time for bugfix only.
>
> Thoughts?
>
> -Todd
>
>
Re: putting a border around 0.92 release
Posted by Andrew Purtell <ap...@apache.org>.
> From: Todd Lipcon <to...@cloudera.com>
[...]
> So, my vote is either:
> plan a: hybrid model - 0.91.X becomes a time-based release series
> where we drop trunk once every month or two, and 0.92.0 is gated on
> features
> or:
> plan b: strict time-based: we release 0.92.0 around summit, and lock
> down the branch at least a month or so ahead of time for bugfix only.
>
I vote for plan A.
- Andy
Re: putting a border around 0.92 release
Posted by Andrew Purtell <ap...@apache.org>.
> From: Todd Lipcon <to...@cloudera.com>
[...]
> So, my vote is either:
> plan a: hybrid model - 0.91.X becomes a time-based release series
> where we drop trunk once every month or two, and 0.92.0 is gated on
> features
> or:
> plan b: strict time-based: we release 0.92.0 around summit, and lock
> down the branch at least a month or so ahead of time for bugfix only.
>
I vote for plan A.
- Andy
Re: putting a border around 0.92 release
Posted by Todd Lipcon <to...@cloudera.com>.
On Sat, Feb 26, 2011 at 2:24 PM, Jean-Daniel Cryans <jd...@apache.org> wrote:
> Woah those are huge tasks!
>
> Also to consider:
>
> - integration with hadoop 0.22, should we support it and should we
> also support 0.20 at the same time? 0.22 was branched but IIRC it
> still has quite a few blockers.
> - removing heartbeats, this is in the pipeline from Stack and IMO
> will have ripple effects on online schema editing.
> - HBASE-2856, pretty critical.
> - replication-related issues like multi-slave (which I'm working on),
> and ideally multi-master. I'd like to add better management tools too.
>
> And lastly we need to plan when we want to branch 0.92... should we
> target late May in order to be ready for the Hadoop Summit in June?
> For once it would be nice to offer more than an alpha release :)
In my view, we can do one or the other: either it's a feature-based
release, in which case we "release it when it's done", or it's a
time-based release, in which case we release at some decided-upon time
with whatever's done.
I personally prefer time-based releases, though we need to make sure
if we decide to do this that any large destabilizing (or half
complete) features are guarded either by config flags or are developed
in a branch. Thus trunk stays relatively releasable at all times and
we can be pretty confident we'll hit the decided-upon timeline.
Looking back at the 0.90 release, we got caught in a bind because we
were trying to do both feature-based (new master) and time-based (end
of 2010).
So, my vote is either:
plan a: hybrid model - 0.91.X becomes a time-based release series
where we drop trunk once every month or two, and 0.92.0 is gated on
features
or:
plan b: strict time-based: we release 0.92.0 around summit, and lock
down the branch at least a month or so ahead of time for bugfix only.
Thoughts?
-Todd
>
> On Sat, Feb 26, 2011 at 12:34 PM, Andrew Purtell <ap...@apache.org> wrote:
>> Stack and I were chatting on IRC about settling with should get into 0.92 before pulling the trigger on the release.
>>
>> Stack thinks we need online region schema editing. I agree because per-table coprocessor loading is configured via table attributes. We'd also need some kind of notification of schema update to trigger various actions in the regionserver. (For CPs, (re)load.)
>>
>> I'd also really like to see some form of secondary indexing. This is an important feature for HBase to have. All of our in house devs ask for this sooner or later in one form or another. Other projects have options in this arena, while HBase used to in core, but no longer. We have three people starting on this ASAP. I'd like to at least do co-design with the community. We should aim for 'simple and effective'.
>>
>> There are 14 blockers: https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+HBASE+AND+fixVersion+%3D+%220.92.0%22+AND+resolution+%3D+Unresolved+AND+priority+%3D+Blocker
>>
>> Additionally, 22 marked as critical: https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+HBASE+AND+fixVersion+%3D+%220.92.0%22+AND+resolution+%3D+Unresolved+AND+priority+%3D+Critical
>>
>> Best regards,
>>
>> - Andy
>>
>> Problems worthy of attack prove their worth by hitting back.
>> - Piet Hein (via Tom White)
>>
>>
>>
>>
>
--
Todd Lipcon
Software Engineer, Cloudera
Re: putting a border around 0.92 release
Posted by Todd Lipcon <to...@cloudera.com>.
On Sat, Feb 26, 2011 at 2:24 PM, Jean-Daniel Cryans <jd...@apache.org> wrote:
> Woah those are huge tasks!
>
> Also to consider:
>
> - integration with hadoop 0.22, should we support it and should we
> also support 0.20 at the same time? 0.22 was branched but IIRC it
> still has quite a few blockers.
> - removing heartbeats, this is in the pipeline from Stack and IMO
> will have ripple effects on online schema editing.
> - HBASE-2856, pretty critical.
> - replication-related issues like multi-slave (which I'm working on),
> and ideally multi-master. I'd like to add better management tools too.
>
> And lastly we need to plan when we want to branch 0.92... should we
> target late May in order to be ready for the Hadoop Summit in June?
> For once it would be nice to offer more than an alpha release :)
In my view, we can do one or the other: either it's a feature-based
release, in which case we "release it when it's done", or it's a
time-based release, in which case we release at some decided-upon time
with whatever's done.
I personally prefer time-based releases, though we need to make sure
if we decide to do this that any large destabilizing (or half
complete) features are guarded either by config flags or are developed
in a branch. Thus trunk stays relatively releasable at all times and
we can be pretty confident we'll hit the decided-upon timeline.
Looking back at the 0.90 release, we got caught in a bind because we
were trying to do both feature-based (new master) and time-based (end
of 2010).
So, my vote is either:
plan a: hybrid model - 0.91.X becomes a time-based release series
where we drop trunk once every month or two, and 0.92.0 is gated on
features
or:
plan b: strict time-based: we release 0.92.0 around summit, and lock
down the branch at least a month or so ahead of time for bugfix only.
Thoughts?
-Todd
>
> On Sat, Feb 26, 2011 at 12:34 PM, Andrew Purtell <ap...@apache.org> wrote:
>> Stack and I were chatting on IRC about settling with should get into 0.92 before pulling the trigger on the release.
>>
>> Stack thinks we need online region schema editing. I agree because per-table coprocessor loading is configured via table attributes. We'd also need some kind of notification of schema update to trigger various actions in the regionserver. (For CPs, (re)load.)
>>
>> I'd also really like to see some form of secondary indexing. This is an important feature for HBase to have. All of our in house devs ask for this sooner or later in one form or another. Other projects have options in this arena, while HBase used to in core, but no longer. We have three people starting on this ASAP. I'd like to at least do co-design with the community. We should aim for 'simple and effective'.
>>
>> There are 14 blockers: https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+HBASE+AND+fixVersion+%3D+%220.92.0%22+AND+resolution+%3D+Unresolved+AND+priority+%3D+Blocker
>>
>> Additionally, 22 marked as critical: https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+HBASE+AND+fixVersion+%3D+%220.92.0%22+AND+resolution+%3D+Unresolved+AND+priority+%3D+Critical
>>
>> Best regards,
>>
>> - Andy
>>
>> Problems worthy of attack prove their worth by hitting back.
>> - Piet Hein (via Tom White)
>>
>>
>>
>>
>
--
Todd Lipcon
Software Engineer, Cloudera
Re: putting a border around 0.92 release
Posted by Lars George <la...@gmail.com>.
+1 on 0.91 and have something for the summit. Also great to box the feature set now as I will base the book on 0.92 - anything else would futile.
On Feb 26, 2011, at 23:24, Jean-Daniel Cryans <jd...@apache.org> wrote:
> Woah those are huge tasks!
>
> Also to consider:
>
> - integration with hadoop 0.22, should we support it and should we
> also support 0.20 at the same time? 0.22 was branched but IIRC it
> still has quite a few blockers.
> - removing heartbeats, this is in the pipeline from Stack and IMO
> will have ripple effects on online schema editing.
> - HBASE-2856, pretty critical.
> - replication-related issues like multi-slave (which I'm working on),
> and ideally multi-master. I'd like to add better management tools too.
>
> And lastly we need to plan when we want to branch 0.92... should we
> target late May in order to be ready for the Hadoop Summit in June?
> For once it would be nice to offer more than an alpha release :)
>
> Oh and we need a 0.91 pretty soon, putting coprocessors out there would be nice.
>
> Exciting!
>
> J-D
>
> On Sat, Feb 26, 2011 at 12:34 PM, Andrew Purtell <ap...@apache.org> wrote:
>> Stack and I were chatting on IRC about settling with should get into 0.92 before pulling the trigger on the release.
>>
>> Stack thinks we need online region schema editing. I agree because per-table coprocessor loading is configured via table attributes. We'd also need some kind of notification of schema update to trigger various actions in the regionserver. (For CPs, (re)load.)
>>
>> I'd also really like to see some form of secondary indexing. This is an important feature for HBase to have. All of our in house devs ask for this sooner or later in one form or another. Other projects have options in this arena, while HBase used to in core, but no longer. We have three people starting on this ASAP. I'd like to at least do co-design with the community. We should aim for 'simple and effective'.
>>
>> There are 14 blockers: https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+HBASE+AND+fixVersion+%3D+%220.92.0%22+AND+resolution+%3D+Unresolved+AND+priority+%3D+Blocker
>>
>> Additionally, 22 marked as critical: https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+HBASE+AND+fixVersion+%3D+%220.92.0%22+AND+resolution+%3D+Unresolved+AND+priority+%3D+Critical
>>
>> Best regards,
>>
>> - Andy
>>
>> Problems worthy of attack prove their worth by hitting back.
>> - Piet Hein (via Tom White)
>>
>>
>>
>>
Re: putting a border around 0.92 release
Posted by Lars George <la...@gmail.com>.
+1 on 0.91 and have something for the summit. Also great to box the feature set now as I will base the book on 0.92 - anything else would futile.
On Feb 26, 2011, at 23:24, Jean-Daniel Cryans <jd...@apache.org> wrote:
> Woah those are huge tasks!
>
> Also to consider:
>
> - integration with hadoop 0.22, should we support it and should we
> also support 0.20 at the same time? 0.22 was branched but IIRC it
> still has quite a few blockers.
> - removing heartbeats, this is in the pipeline from Stack and IMO
> will have ripple effects on online schema editing.
> - HBASE-2856, pretty critical.
> - replication-related issues like multi-slave (which I'm working on),
> and ideally multi-master. I'd like to add better management tools too.
>
> And lastly we need to plan when we want to branch 0.92... should we
> target late May in order to be ready for the Hadoop Summit in June?
> For once it would be nice to offer more than an alpha release :)
>
> Oh and we need a 0.91 pretty soon, putting coprocessors out there would be nice.
>
> Exciting!
>
> J-D
>
> On Sat, Feb 26, 2011 at 12:34 PM, Andrew Purtell <ap...@apache.org> wrote:
>> Stack and I were chatting on IRC about settling with should get into 0.92 before pulling the trigger on the release.
>>
>> Stack thinks we need online region schema editing. I agree because per-table coprocessor loading is configured via table attributes. We'd also need some kind of notification of schema update to trigger various actions in the regionserver. (For CPs, (re)load.)
>>
>> I'd also really like to see some form of secondary indexing. This is an important feature for HBase to have. All of our in house devs ask for this sooner or later in one form or another. Other projects have options in this arena, while HBase used to in core, but no longer. We have three people starting on this ASAP. I'd like to at least do co-design with the community. We should aim for 'simple and effective'.
>>
>> There are 14 blockers: https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+HBASE+AND+fixVersion+%3D+%220.92.0%22+AND+resolution+%3D+Unresolved+AND+priority+%3D+Blocker
>>
>> Additionally, 22 marked as critical: https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+HBASE+AND+fixVersion+%3D+%220.92.0%22+AND+resolution+%3D+Unresolved+AND+priority+%3D+Critical
>>
>> Best regards,
>>
>> - Andy
>>
>> Problems worthy of attack prove their worth by hitting back.
>> - Piet Hein (via Tom White)
>>
>>
>>
>>
Re: putting a border around 0.92 release
Posted by Ted Yu <yu...@gmail.com>.
>> should we also support 0.20 at the same time
Definitely.
On Sat, Feb 26, 2011 at 2:24 PM, Jean-Daniel Cryans <jd...@apache.org>wrote:
> Woah those are huge tasks!
>
> Also to consider:
>
> - integration with hadoop 0.22, should we support it and should we
> also support 0.20 at the same time? 0.22 was branched but IIRC it
> still has quite a few blockers.
> - removing heartbeats, this is in the pipeline from Stack and IMO
> will have ripple effects on online schema editing.
> - HBASE-2856, pretty critical.
> - replication-related issues like multi-slave (which I'm working on),
> and ideally multi-master. I'd like to add better management tools too.
>
> And lastly we need to plan when we want to branch 0.92... should we
> target late May in order to be ready for the Hadoop Summit in June?
> For once it would be nice to offer more than an alpha release :)
>
> Oh and we need a 0.91 pretty soon, putting coprocessors out there would be
> nice.
>
> Exciting!
>
> J-D
>
> On Sat, Feb 26, 2011 at 12:34 PM, Andrew Purtell <ap...@apache.org>
> wrote:
> > Stack and I were chatting on IRC about settling with should get into 0.92
> before pulling the trigger on the release.
> >
> > Stack thinks we need online region schema editing. I agree because
> per-table coprocessor loading is configured via table attributes. We'd also
> need some kind of notification of schema update to trigger various actions
> in the regionserver. (For CPs, (re)load.)
> >
> > I'd also really like to see some form of secondary indexing. This is an
> important feature for HBase to have. All of our in house devs ask for this
> sooner or later in one form or another. Other projects have options in this
> arena, while HBase used to in core, but no longer. We have three people
> starting on this ASAP. I'd like to at least do co-design with the community.
> We should aim for 'simple and effective'.
> >
> > There are 14 blockers:
> https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+HBASE+AND+fixVersion+%3D+%220.92.0%22+AND+resolution+%3D+Unresolved+AND+priority+%3D+Blocker
> >
> > Additionally, 22 marked as critical:
> https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+HBASE+AND+fixVersion+%3D+%220.92.0%22+AND+resolution+%3D+Unresolved+AND+priority+%3D+Critical
> >
> > Best regards,
> >
> > - Andy
> >
> > Problems worthy of attack prove their worth by hitting back.
> > - Piet Hein (via Tom White)
> >
> >
> >
> >
>
Re: putting a border around 0.92 release
Posted by Jean-Daniel Cryans <jd...@apache.org>.
Woah those are huge tasks!
Also to consider:
- integration with hadoop 0.22, should we support it and should we
also support 0.20 at the same time? 0.22 was branched but IIRC it
still has quite a few blockers.
- removing heartbeats, this is in the pipeline from Stack and IMO
will have ripple effects on online schema editing.
- HBASE-2856, pretty critical.
- replication-related issues like multi-slave (which I'm working on),
and ideally multi-master. I'd like to add better management tools too.
And lastly we need to plan when we want to branch 0.92... should we
target late May in order to be ready for the Hadoop Summit in June?
For once it would be nice to offer more than an alpha release :)
Oh and we need a 0.91 pretty soon, putting coprocessors out there would be nice.
Exciting!
J-D
On Sat, Feb 26, 2011 at 12:34 PM, Andrew Purtell <ap...@apache.org> wrote:
> Stack and I were chatting on IRC about settling with should get into 0.92 before pulling the trigger on the release.
>
> Stack thinks we need online region schema editing. I agree because per-table coprocessor loading is configured via table attributes. We'd also need some kind of notification of schema update to trigger various actions in the regionserver. (For CPs, (re)load.)
>
> I'd also really like to see some form of secondary indexing. This is an important feature for HBase to have. All of our in house devs ask for this sooner or later in one form or another. Other projects have options in this arena, while HBase used to in core, but no longer. We have three people starting on this ASAP. I'd like to at least do co-design with the community. We should aim for 'simple and effective'.
>
> There are 14 blockers: https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+HBASE+AND+fixVersion+%3D+%220.92.0%22+AND+resolution+%3D+Unresolved+AND+priority+%3D+Blocker
>
> Additionally, 22 marked as critical: https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+HBASE+AND+fixVersion+%3D+%220.92.0%22+AND+resolution+%3D+Unresolved+AND+priority+%3D+Critical
>
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back.
> - Piet Hein (via Tom White)
>
>
>
>
Re: putting a border around 0.92 release
Posted by Jean-Daniel Cryans <jd...@apache.org>.
Woah those are huge tasks!
Also to consider:
- integration with hadoop 0.22, should we support it and should we
also support 0.20 at the same time? 0.22 was branched but IIRC it
still has quite a few blockers.
- removing heartbeats, this is in the pipeline from Stack and IMO
will have ripple effects on online schema editing.
- HBASE-2856, pretty critical.
- replication-related issues like multi-slave (which I'm working on),
and ideally multi-master. I'd like to add better management tools too.
And lastly we need to plan when we want to branch 0.92... should we
target late May in order to be ready for the Hadoop Summit in June?
For once it would be nice to offer more than an alpha release :)
Oh and we need a 0.91 pretty soon, putting coprocessors out there would be nice.
Exciting!
J-D
On Sat, Feb 26, 2011 at 12:34 PM, Andrew Purtell <ap...@apache.org> wrote:
> Stack and I were chatting on IRC about settling with should get into 0.92 before pulling the trigger on the release.
>
> Stack thinks we need online region schema editing. I agree because per-table coprocessor loading is configured via table attributes. We'd also need some kind of notification of schema update to trigger various actions in the regionserver. (For CPs, (re)load.)
>
> I'd also really like to see some form of secondary indexing. This is an important feature for HBase to have. All of our in house devs ask for this sooner or later in one form or another. Other projects have options in this arena, while HBase used to in core, but no longer. We have three people starting on this ASAP. I'd like to at least do co-design with the community. We should aim for 'simple and effective'.
>
> There are 14 blockers: https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+HBASE+AND+fixVersion+%3D+%220.92.0%22+AND+resolution+%3D+Unresolved+AND+priority+%3D+Blocker
>
> Additionally, 22 marked as critical: https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+HBASE+AND+fixVersion+%3D+%220.92.0%22+AND+resolution+%3D+Unresolved+AND+priority+%3D+Critical
>
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back.
> - Piet Hein (via Tom White)
>
>
>
>
Re: putting a border around 0.92 release
Posted by Jean-Daniel Cryans <jd...@apache.org>.
I'm in favor of releasing major versions more often so that the
changes be less disruptive, the sooner the better.
We'll never know for sure, but I wonder if releasing more often
compared to waiting for those features would put those same features
in the hands of the users sooner or later. Like would releasing 0.92
without online schema edits and others and then releasing 0.94 not too
far enough would coincide with a 0.92 with all those features in
another timeline.
J-D
On Mon, May 2, 2011 at 10:48 AM, Stack <st...@duboce.net> wrote:
> Whats our thinking on 0.92.0 now May 1st has passed? Interestingly,
> the numbers of blockers and criticals are the same as Andrew reports
> below in the threads' first message (we won some since his email was
> written but then some new ones showed up since). Online merge and
> schema edits have not made it in nor has a secondary index
> implementation (distributed log splitting has though).
>
> There seemed to be a strong sense in the below thread that we should
> go for mostly time-based releases anyways so we should try stabilizing
> TRUNK so we can cut a 0.92.0 soon? When? This week? Looks like
> it'll be two weeks at least before we nail the blockers and criticals
> as is. Branch May 14th or so?
>
> St.Ack
>
>
>
> On Sat, Feb 26, 2011 at 12:34 PM, Andrew Purtell <ap...@apache.org> wrote:
>> Stack and I were chatting on IRC about settling with should get into 0.92 before pulling the trigger on the release.
>>
>> Stack thinks we need online region schema editing. I agree because per-table coprocessor loading is configured via table attributes. We'd also need some kind of notification of schema update to trigger various actions in the regionserver. (For CPs, (re)load.)
>>
>> I'd also really like to see some form of secondary indexing. This is an important feature for HBase to have. All of our in house devs ask for this sooner or later in one form or another. Other projects have options in this arena, while HBase used to in core, but no longer. We have three people starting on this ASAP. I'd like to at least do co-design with the community. We should aim for 'simple and effective'.
>>
>> There are 14 blockers: https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+HBASE+AND+fixVersion+%3D+%220.92.0%22+AND+resolution+%3D+Unresolved+AND+priority+%3D+Blocker
>>
>> Additionally, 22 marked as critical: https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+HBASE+AND+fixVersion+%3D+%220.92.0%22+AND+resolution+%3D+Unresolved+AND+priority+%3D+Critical
>>
>> Best regards,
>>
>> - Andy
>>
>> Problems worthy of attack prove their worth by hitting back.
>> - Piet Hein (via Tom White)
>>
>>
>>
>>
>
Re: putting a border around 0.92 release
Posted by Andrew Purtell <ap...@apache.org>.
I agree.
We did get code back from an internal dev group for the secondary indexing implementation but I don't think we are satisfied with its current state.
Also I have been swamped, therefore remiss, in online schema edit.
Shouldn't hold up a release on our account. Next one.
- Andy
> From: Stack <st...@duboce.net>
> Subject: Re: putting a border around 0.92 release
> To: user@hbase.apache.org, apurtell@apache.org
> Cc: dev@hbase.apache.org
> Date: Monday, May 2, 2011, 10:48 AM
> Whats our thinking on 0.92.0 now May 1st has passed? Interestingly,
> the numbers of blockers and criticals are the same as Andrew reports
> below in the threads' first message (we won some since his email was
> written but then some new ones showed up since). Online merge and
> schema edits have not made it in nor has a secondary index
> implementation (distributed log splitting has though).
Re: putting a border around 0.92 release
Posted by Andrew Purtell <ap...@apache.org>.
I agree.
We did get code back from an internal dev group for the secondary indexing implementation but I don't think we are satisfied with its current state.
Also I have been swamped, therefore remiss, in online schema edit.
Shouldn't hold up a release on our account. Next one.
- Andy
> From: Stack <st...@duboce.net>
> Subject: Re: putting a border around 0.92 release
> To: user@hbase.apache.org, apurtell@apache.org
> Cc: dev@hbase.apache.org
> Date: Monday, May 2, 2011, 10:48 AM
> Whats our thinking on 0.92.0 now May 1st has passed? Interestingly,
> the numbers of blockers and criticals are the same as Andrew reports
> below in the threads' first message (we won some since his email was
> written but then some new ones showed up since). Online merge and
> schema edits have not made it in nor has a secondary index
> implementation (distributed log splitting has though).
Re: putting a border around 0.92 release
Posted by Stack <st...@duboce.net>.
Whats our thinking on 0.92.0 now May 1st has passed? Interestingly,
the numbers of blockers and criticals are the same as Andrew reports
below in the threads' first message (we won some since his email was
written but then some new ones showed up since). Online merge and
schema edits have not made it in nor has a secondary index
implementation (distributed log splitting has though).
There seemed to be a strong sense in the below thread that we should
go for mostly time-based releases anyways so we should try stabilizing
TRUNK so we can cut a 0.92.0 soon? When? This week? Looks like
it'll be two weeks at least before we nail the blockers and criticals
as is. Branch May 14th or so?
St.Ack
On Sat, Feb 26, 2011 at 12:34 PM, Andrew Purtell <ap...@apache.org> wrote:
> Stack and I were chatting on IRC about settling with should get into 0.92 before pulling the trigger on the release.
>
> Stack thinks we need online region schema editing. I agree because per-table coprocessor loading is configured via table attributes. We'd also need some kind of notification of schema update to trigger various actions in the regionserver. (For CPs, (re)load.)
>
> I'd also really like to see some form of secondary indexing. This is an important feature for HBase to have. All of our in house devs ask for this sooner or later in one form or another. Other projects have options in this arena, while HBase used to in core, but no longer. We have three people starting on this ASAP. I'd like to at least do co-design with the community. We should aim for 'simple and effective'.
>
> There are 14 blockers: https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+HBASE+AND+fixVersion+%3D+%220.92.0%22+AND+resolution+%3D+Unresolved+AND+priority+%3D+Blocker
>
> Additionally, 22 marked as critical: https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+HBASE+AND+fixVersion+%3D+%220.92.0%22+AND+resolution+%3D+Unresolved+AND+priority+%3D+Critical
>
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back.
> - Piet Hein (via Tom White)
>
>
>
>
Re: putting a border around 0.92 release
Posted by Stack <st...@duboce.net>.
Whats our thinking on 0.92.0 now May 1st has passed? Interestingly,
the numbers of blockers and criticals are the same as Andrew reports
below in the threads' first message (we won some since his email was
written but then some new ones showed up since). Online merge and
schema edits have not made it in nor has a secondary index
implementation (distributed log splitting has though).
There seemed to be a strong sense in the below thread that we should
go for mostly time-based releases anyways so we should try stabilizing
TRUNK so we can cut a 0.92.0 soon? When? This week? Looks like
it'll be two weeks at least before we nail the blockers and criticals
as is. Branch May 14th or so?
St.Ack
On Sat, Feb 26, 2011 at 12:34 PM, Andrew Purtell <ap...@apache.org> wrote:
> Stack and I were chatting on IRC about settling with should get into 0.92 before pulling the trigger on the release.
>
> Stack thinks we need online region schema editing. I agree because per-table coprocessor loading is configured via table attributes. We'd also need some kind of notification of schema update to trigger various actions in the regionserver. (For CPs, (re)load.)
>
> I'd also really like to see some form of secondary indexing. This is an important feature for HBase to have. All of our in house devs ask for this sooner or later in one form or another. Other projects have options in this arena, while HBase used to in core, but no longer. We have three people starting on this ASAP. I'd like to at least do co-design with the community. We should aim for 'simple and effective'.
>
> There are 14 blockers: https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+HBASE+AND+fixVersion+%3D+%220.92.0%22+AND+resolution+%3D+Unresolved+AND+priority+%3D+Blocker
>
> Additionally, 22 marked as critical: https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+HBASE+AND+fixVersion+%3D+%220.92.0%22+AND+resolution+%3D+Unresolved+AND+priority+%3D+Critical
>
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back.
> - Piet Hein (via Tom White)
>
>
>
>