HBase releases...

[lars hofhansl <lh...@yahoo.com>]

HBase 0.90 was released Jan 2011. By the time HBase 0.92 is released it will probably be close to
a year between stable releases.

Should we try to have more frequent, smaller releases? Maybe 3-4 a year.
For example I would almost say that the performance enhancements from the Facebook guys would
warrant a new (performance) release "shortly" after 0.92.

That would hopefully reduce the time and effort needed to stabilize the releases, at the expense of having to
maintain two or even three branches in addition to trunk that people would still be actively using.

Thoughts?

-- Lars

RE: HBase releases...

[Jonathan Gray <jg...@fb.com>]

Vladimir, if you take a look at the dev and issues lists, you'll see that lots of FB people have been extremely active, especially in the past few months.

Nearly all of our changes have been brought into hbase-92 or trunk.  Those that have not made it are being worked on (or are blocked by something like an HDFS incompatibility).  All of our current development work is being done on JIRA.

But having the contributions and features, of course, does not mean it has the same level of stability.  I'm spending time working on stabilization of 92/trunk and hoping to get that into a state for production as soon as we can.  This is something that you can help on.  Please test and report bugs.

JG

> -----Original Message-----
> From: Vladimir Rodionov [mailto:vrodionov@carrieriq.com]
> Sent: Friday, October 14, 2011 10:42 AM
> To: dev@hbase.apache.org
> Subject: RE: HBase releases...
> 
> Jonathan, to make your community happier could you (your company)
> contribute all Facebook's internal patches to 0.92?
>  I apologize to everyone who think that I was rude in my previous message.
> 
> Best regards,
> Vladimir Rodionov
> Principal Platform Engineer
> Carrier IQ, www.carrieriq.com
> e-mail: vrodionov@carrieriq.com
> 
> ________________________________________
> From: Jonathan Gray [jgray@fb.com]
> Sent: Thursday, October 13, 2011 7:12 PM
> To: dev@hbase.apache.org
> Subject: RE: HBase releases...
> 
> Vladimir, I appreciate the contributions of your company and welcome your
> personal input around the roadmap and future of HBase.
> 
> However, tying your perception of the community's overemphasis on
> features to the fact that you had three RS go down today is a bit of a reach.
> About a million different things could be the cause.  And to me, your tone
> comes off as rude and entitled.  This is an open source project.
> 
> HBase is driven by the priorities of the companies and developers
> contributing to it.  There's no single controlling entity and contributions come
> from all over.
> 
> If you want to talk about stability, a number of applications at Facebook are
> running on an 0.89 based release, because those applications care about
> stability above all else.  It's an inherently more stable version because it
> hasn't undergone significant code change for more than a year and every
> individual commit went through a rigorous review and testing process.  It's
> not inherently more stable because of superior architecture, it's more stable
> because the codebase itself is so stable and has undergone so much testing.
> 
> But of course, there is truth to what you say!
> 
> We all agree we need to continue to get better at stability.  There has been
> some pretty significant amounts of code change in the past year that were
> destabilizing but the road forward seems clear and my sense is that most of
> our ideas around stability-impacting architecture changes deal with
> decreasing complexity (like removing root) and increasing availability.
> Performance improvements will always be a constant.  Coprocessors make
> extending features into HBase much simpler and less invasive.
> 
> I'm looking forward to getting 92 out the door soon and 93 or 94 shortly
> thereafter.  (wouldn't that be some shit?  Release a 93 dev soon after 92
> release?!)
> 
> JG
> 
> > -----Original Message-----
> > From: Vladimir Rodionov [mailto:vrodionov@carrieriq.com]
> > Sent: Thursday, October 13, 2011 5:15 PM
> > To: dev@hbase.apache.org
> > Subject: RE: HBase releases...
> >
> >
> > Just today 3 our region servers went down for no obvious reason and
> > M/R jobs started failing after that (we are on 0.90.4 + some Ted's
> > patch) You have a already  lot of features , could you please focus on just
> two of them :
> > performance and stability of a core functionality.
> >
> > We need Put, Get and Scan.
> >
> > Best regards,
> > Vladimir Rodionov
> > Principal Platform Engineer
> > Carrier IQ, www.carrieriq.com
> > e-mail: vrodionov@carrieriq.com
> >
> > ________________________________________
> > From: Jonathan Gray [jgray@fb.com]
> > Sent: Thursday, October 13, 2011 4:35 PM
> > To: dev@hbase.apache.org
> > Subject: RE: HBase releases...
> >
> > +1 on all of this below.
> >
> > I'm all for frequent releases.  Big features move to branches and the
> > author is required to keep it up against whatever the current trunk is.
> >
> > And by forcing ourselves to keep features/improvements into trunk
> > rather than into the currently active branch, we will incentivize
> > ourselves to push forward new releases to get the goodies ;)
> >
> > JG
> >
> > > On Tue, Oct 11, 2011 at 10:22 PM, lars hofhansl
> > > <lh...@yahoo.com>
> > > wrote:
> > > > Should we try to have more frequent, smaller releases? Maybe 3-4 a
> > year.
> > >
> > > I'd be in favor of this.
> > >
> > > Our 0.92 has gotten way too fat and a little difficult to land
> > > because its busting at seams with good stuff.  It was let run too long.
> > >
> > > > For example I would almost say that the performance enhancements
> > > > from the Facebook guys would warrant a new (performance) release
> > "shortly"
> > > after 0.92.
> > > >
> > >
> > > Sounds good to me too.
> > >
> > > St.Ack

Re: HBase releases...

[Steven Noels <st...@outerthought.org>]

On Fri, Oct 14, 2011 at 7:42 PM, Vladimir Rodionov
<vr...@carrieriq.com>wrote:

Jonathan, to make your community happier could you (your company)
>  contribute all Facebook's internal patches to 0.92?
>

Vladimir, it's not Jon's community here. It's *ours*, yours included. Stop
the polarization.


>  I apologize to everyone who think that I was rude in my previous message.
>

Yes, you were. HBase needs more fulltime devs, but nobody can reasonably
expect these to be appear out of thin air just because "Facebook" uses
HBase.

Steven.
-- 
Steven Noels
http://outerthought.org/
Scalable Smart Data
Makers of Lily

RE: HBase releases...

[Vladimir Rodionov <vr...@carrieriq.com>]

Jonathan, to make your community happier could you (your company)  contribute all Facebook's internal patches to 0.92?
 I apologize to everyone who think that I was rude in my previous message. 

Best regards,
Vladimir Rodionov
Principal Platform Engineer
Carrier IQ, www.carrieriq.com
e-mail: vrodionov@carrieriq.com

________________________________________
From: Jonathan Gray [jgray@fb.com]
Sent: Thursday, October 13, 2011 7:12 PM
To: dev@hbase.apache.org
Subject: RE: HBase releases...

Vladimir, I appreciate the contributions of your company and welcome your personal input around the roadmap and future of HBase.

However, tying your perception of the community's overemphasis on features to the fact that you had three RS go down today is a bit of a reach.  About a million different things could be the cause.  And to me, your tone comes off as rude and entitled.  This is an open source project.

HBase is driven by the priorities of the companies and developers contributing to it.  There's no single controlling entity and contributions come from all over.

If you want to talk about stability, a number of applications at Facebook are running on an 0.89 based release, because those applications care about stability above all else.  It's an inherently more stable version because it hasn't undergone significant code change for more than a year and every individual commit went through a rigorous review and testing process.  It's not inherently more stable because of superior architecture, it's more stable because the codebase itself is so stable and has undergone so much testing.

But of course, there is truth to what you say!

We all agree we need to continue to get better at stability.  There has been some pretty significant amounts of code change in the past year that were destabilizing but the road forward seems clear and my sense is that most of our ideas around stability-impacting architecture changes deal with decreasing complexity (like removing root) and increasing availability.  Performance improvements will always be a constant.  Coprocessors make extending features into HBase much simpler and less invasive.

I'm looking forward to getting 92 out the door soon and 93 or 94 shortly thereafter.  (wouldn't that be some shit?  Release a 93 dev soon after 92 release?!)

JG

> -----Original Message-----
> From: Vladimir Rodionov [mailto:vrodionov@carrieriq.com]
> Sent: Thursday, October 13, 2011 5:15 PM
> To: dev@hbase.apache.org
> Subject: RE: HBase releases...
>
>
> Just today 3 our region servers went down for no obvious reason and M/R
> jobs started failing after that (we are on 0.90.4 + some Ted's patch) You have
> a already  lot of features , could you please focus on just two of them :
> performance and stability of a core functionality.
>
> We need Put, Get and Scan.
>
> Best regards,
> Vladimir Rodionov
> Principal Platform Engineer
> Carrier IQ, www.carrieriq.com
> e-mail: vrodionov@carrieriq.com
>
> ________________________________________
> From: Jonathan Gray [jgray@fb.com]
> Sent: Thursday, October 13, 2011 4:35 PM
> To: dev@hbase.apache.org
> Subject: RE: HBase releases...
>
> +1 on all of this below.
>
> I'm all for frequent releases.  Big features move to branches and the author is
> required to keep it up against whatever the current trunk is.
>
> And by forcing ourselves to keep features/improvements into trunk rather
> than into the currently active branch, we will incentivize ourselves to push
> forward new releases to get the goodies ;)
>
> JG
>
> > On Tue, Oct 11, 2011 at 10:22 PM, lars hofhansl <lh...@yahoo.com>
> > wrote:
> > > Should we try to have more frequent, smaller releases? Maybe 3-4 a
> year.
> >
> > I'd be in favor of this.
> >
> > Our 0.92 has gotten way too fat and a little difficult to land because
> > its busting at seams with good stuff.  It was let run too long.
> >
> > > For example I would almost say that the performance enhancements
> > > from the Facebook guys would warrant a new (performance) release
> "shortly"
> > after 0.92.
> > >
> >
> > Sounds good to me too.
> >
> > St.Ack

RE: HBase releases...

[Jonathan Gray <jg...@fb.com>]

Vladimir, I appreciate the contributions of your company and welcome your personal input around the roadmap and future of HBase.

However, tying your perception of the community's overemphasis on features to the fact that you had three RS go down today is a bit of a reach.  About a million different things could be the cause.  And to me, your tone comes off as rude and entitled.  This is an open source project.

HBase is driven by the priorities of the companies and developers contributing to it.  There's no single controlling entity and contributions come from all over.

If you want to talk about stability, a number of applications at Facebook are running on an 0.89 based release, because those applications care about stability above all else.  It's an inherently more stable version because it hasn't undergone significant code change for more than a year and every individual commit went through a rigorous review and testing process.  It's not inherently more stable because of superior architecture, it's more stable because the codebase itself is so stable and has undergone so much testing.

But of course, there is truth to what you say!

We all agree we need to continue to get better at stability.  There has been some pretty significant amounts of code change in the past year that were destabilizing but the road forward seems clear and my sense is that most of our ideas around stability-impacting architecture changes deal with decreasing complexity (like removing root) and increasing availability.  Performance improvements will always be a constant.  Coprocessors make extending features into HBase much simpler and less invasive.

I'm looking forward to getting 92 out the door soon and 93 or 94 shortly thereafter.  (wouldn't that be some shit?  Release a 93 dev soon after 92 release?!)

JG

> -----Original Message-----
> From: Vladimir Rodionov [mailto:vrodionov@carrieriq.com]
> Sent: Thursday, October 13, 2011 5:15 PM
> To: dev@hbase.apache.org
> Subject: RE: HBase releases...
> 
> 
> Just today 3 our region servers went down for no obvious reason and M/R
> jobs started failing after that (we are on 0.90.4 + some Ted's patch) You have
> a already  lot of features , could you please focus on just two of them :
> performance and stability of a core functionality.
> 
> We need Put, Get and Scan.
> 
> Best regards,
> Vladimir Rodionov
> Principal Platform Engineer
> Carrier IQ, www.carrieriq.com
> e-mail: vrodionov@carrieriq.com
> 
> ________________________________________
> From: Jonathan Gray [jgray@fb.com]
> Sent: Thursday, October 13, 2011 4:35 PM
> To: dev@hbase.apache.org
> Subject: RE: HBase releases...
> 
> +1 on all of this below.
> 
> I'm all for frequent releases.  Big features move to branches and the author is
> required to keep it up against whatever the current trunk is.
> 
> And by forcing ourselves to keep features/improvements into trunk rather
> than into the currently active branch, we will incentivize ourselves to push
> forward new releases to get the goodies ;)
> 
> JG
> 
> > On Tue, Oct 11, 2011 at 10:22 PM, lars hofhansl <lh...@yahoo.com>
> > wrote:
> > > Should we try to have more frequent, smaller releases? Maybe 3-4 a
> year.
> >
> > I'd be in favor of this.
> >
> > Our 0.92 has gotten way too fat and a little difficult to land because
> > its busting at seams with good stuff.  It was let run too long.
> >
> > > For example I would almost say that the performance enhancements
> > > from the Facebook guys would warrant a new (performance) release
> "shortly"
> > after 0.92.
> > >
> >
> > Sounds good to me too.
> >
> > St.Ack

Re: HBase releases...

[Stack <st...@duboce.net>]

On Fri, Oct 14, 2011 at 12:15 AM, Vladimir Rodionov
<vr...@carrieriq.com> wrote:
>
> Just today 3 our region servers went down for no obvious reason and M/R jobs started failing after that (we are on 0.90.4 + some Ted's patch)
> You have a already  lot of features , could you please focus on just two of them : performance and stability of a core functionality.
>

Why did they go down?  What do the logs say?
St.Ack

RE: HBase releases...

[Vladimir Rodionov <vr...@carrieriq.com>]

Just today 3 our region servers went down for no obvious reason and M/R jobs started failing after that (we are on 0.90.4 + some Ted's patch)
You have a already  lot of features , could you please focus on just two of them : performance and stability of a core functionality.

We need Put, Get and Scan.

Best regards,
Vladimir Rodionov
Principal Platform Engineer
Carrier IQ, www.carrieriq.com
e-mail: vrodionov@carrieriq.com

________________________________________
From: Jonathan Gray [jgray@fb.com]
Sent: Thursday, October 13, 2011 4:35 PM
To: dev@hbase.apache.org
Subject: RE: HBase releases...

+1 on all of this below.

I'm all for frequent releases.  Big features move to branches and the author is required to keep it up against whatever the current trunk is.

And by forcing ourselves to keep features/improvements into trunk rather than into the currently active branch, we will incentivize ourselves to push forward new releases to get the goodies ;)

JG

> On Tue, Oct 11, 2011 at 10:22 PM, lars hofhansl <lh...@yahoo.com>
> wrote:
> > Should we try to have more frequent, smaller releases? Maybe 3-4 a year.
>
> I'd be in favor of this.
>
> Our 0.92 has gotten way too fat and a little difficult to land because its busting
> at seams with good stuff.  It was let run too long.
>
> > For example I would almost say that the performance enhancements from
> > the Facebook guys would warrant a new (performance) release "shortly"
> after 0.92.
> >
>
> Sounds good to me too.
>
> St.Ack

RE: HBase releases...

[Jonathan Gray <jg...@fb.com>]

+1 on all of this below.

I'm all for frequent releases.  Big features move to branches and the author is required to keep it up against whatever the current trunk is.

And by forcing ourselves to keep features/improvements into trunk rather than into the currently active branch, we will incentivize ourselves to push forward new releases to get the goodies ;)

JG

> On Tue, Oct 11, 2011 at 10:22 PM, lars hofhansl <lh...@yahoo.com>
> wrote:
> > Should we try to have more frequent, smaller releases? Maybe 3-4 a year.
> 
> I'd be in favor of this.
> 
> Our 0.92 has gotten way too fat and a little difficult to land because its busting
> at seams with good stuff.  It was let run too long.
> 
> > For example I would almost say that the performance enhancements from
> > the Facebook guys would warrant a new (performance) release "shortly"
> after 0.92.
> >
> 
> Sounds good to me too.
> 
> St.Ack

Re: HBase releases...

[Stack <st...@duboce.net>]

On Tue, Oct 11, 2011 at 10:22 PM, lars hofhansl <lh...@yahoo.com> wrote:
> Should we try to have more frequent, smaller releases? Maybe 3-4 a year.

I'd be in favor of this.

Our 0.92 has gotten way too fat and a little difficult to land because
its busting at seams with good stuff.  It was let run too long.

> For example I would almost say that the performance enhancements from the Facebook guys would
> warrant a new (performance) release "shortly" after 0.92.
>

Sounds good to me too.

St.Ack

Re: HBase releases...

["M. C. Srivas" <mc...@gmail.com>]

ok, my $.02

HBase has plenty of features that already make it worthwhile to use, so
stability is lot more important than features (at this stage of HBase's
life).

Once the core is super-solid, it is easier to add features and yet produce
stable releases quickly.

Items which are really not "features", but improvements to the core itself
(eg, prefix compression, or hfile v2), have to be rolled out cautiously with
lots of thorough testing.


On Wed, Oct 12, 2011 at 1:48 PM, lars hofhansl <lh...@yahoo.com> wrote:

> Debian moves at glacial speeds, though :)
>
>
> At the speed that HBase is still going, porting to stable even after a few
> weeks,
> might be a monumental task.
> On the other hand it would make for stable releases.
>
>
>
> ----- Original Message -----
> From: Li Pi <li...@idle.li>
> To: dev@hbase.apache.org
> Cc:
> Sent: Wednesday, October 12, 2011 12:17 PM
> Subject: Re: HBase releases...
>
> We could do a Debian style release system with stable and feature branches.
> Once the feature branches get stabilized - they can be ported to stable.
> On Oct 12, 2011 10:56 AM, "Stack" <st...@duboce.net> wrote:
>
> > On Tue, Oct 11, 2011 at 10:42 PM, Jesse Yates <je...@gmail.com>
> > wrote:
> > > If we can release more frequently, with truly stable releases, then
> more
> > > people will be more likely to run clusters with codebases that are
> closer
> > to
> > > trunk. Therefore they will have more benefits like bug fixes and
> > performance
> > > increases without the worry that they are running unstable/buggy code.
> > > However, there is a big 'if' here - if we can make sure that the builds
> > that
> > > go out frequently are really rock solid.
> > >
> >
> > We can't afford to go backwards when it comes to perceived stability.
> > Its a separate discussion -- belongs more in the testing thread that
> > has been running of late -- but I'd like to talk about how we can
> > ensure stability goes up as time goes by even as we cross major
> > release versions.
> >
> > St.Ack
> >
>
>

Re: HBase releases...

[lars hofhansl <lh...@yahoo.com>]

Debian moves at glacial speeds, though :)


At the speed that HBase is still going, porting to stable even after a few weeks,
might be a monumental task.
On the other hand it would make for stable releases.



----- Original Message -----
From: Li Pi <li...@idle.li>
To: dev@hbase.apache.org
Cc: 
Sent: Wednesday, October 12, 2011 12:17 PM
Subject: Re: HBase releases...

We could do a Debian style release system with stable and feature branches.
Once the feature branches get stabilized - they can be ported to stable.
On Oct 12, 2011 10:56 AM, "Stack" <st...@duboce.net> wrote:

> On Tue, Oct 11, 2011 at 10:42 PM, Jesse Yates <je...@gmail.com>
> wrote:
> > If we can release more frequently, with truly stable releases, then more
> > people will be more likely to run clusters with codebases that are closer
> to
> > trunk. Therefore they will have more benefits like bug fixes and
> performance
> > increases without the worry that they are running unstable/buggy code.
> > However, there is a big 'if' here - if we can make sure that the builds
> that
> > go out frequently are really rock solid.
> >
>
> We can't afford to go backwards when it comes to perceived stability.
> Its a separate discussion -- belongs more in the testing thread that
> has been running of late -- but I'd like to talk about how we can
> ensure stability goes up as time goes by even as we cross major
> release versions.
>
> St.Ack
>

Re: HBase releases...

[Li Pi <li...@idle.li>]

We could do a Debian style release system with stable and feature branches.
Once the feature branches get stabilized - they can be ported to stable.
On Oct 12, 2011 10:56 AM, "Stack" <st...@duboce.net> wrote:

> On Tue, Oct 11, 2011 at 10:42 PM, Jesse Yates <je...@gmail.com>
> wrote:
> > If we can release more frequently, with truly stable releases, then more
> > people will be more likely to run clusters with codebases that are closer
> to
> > trunk. Therefore they will have more benefits like bug fixes and
> performance
> > increases without the worry that they are running unstable/buggy code.
> > However, there is a big 'if' here - if we can make sure that the builds
> that
> > go out frequently are really rock solid.
> >
>
> We can't afford to go backwards when it comes to perceived stability.
> Its a separate discussion -- belongs more in the testing thread that
> has been running of late -- but I'd like to talk about how we can
> ensure stability goes up as time goes by even as we cross major
> release versions.
>
> St.Ack
>

Re: HBase releases...

[Stack <st...@duboce.net>]

On Tue, Oct 11, 2011 at 10:42 PM, Jesse Yates <je...@gmail.com> wrote:
> If we can release more frequently, with truly stable releases, then more
> people will be more likely to run clusters with codebases that are closer to
> trunk. Therefore they will have more benefits like bug fixes and performance
> increases without the worry that they are running unstable/buggy code.
> However, there is a big 'if' here - if we can make sure that the builds that
> go out frequently are really rock solid.
>

We can't afford to go backwards when it comes to perceived stability.
Its a separate discussion -- belongs more in the testing thread that
has been running of late -- but I'd like to talk about how we can
ensure stability goes up as time goes by even as we cross major
release versions.

St.Ack

Re: HBase releases...

[Andrew Purtell <ap...@apache.org>]

> What I am thinking to nudge the 'feels right' point a bit sooner. :)


Concur.


> For example, what if we had branched right after coprocessors went in. Would
> that been useful?

From my point of view, coprocessors and security were two sides of the same coin, we just implemented in phases to reap benefits of one independent of the other... and then coprocessors took on a life of their own.

So I'd say 0.92 should have coprocessors AND security. But this is because of the prospect of waiting a long time for another release after 0.92, if 0.92 were to only have coprocessors.


At this point HBASE-3025 and the RPC changes are basically there, so perhaps they can go in and 0.92 can go out the door after one more round of pounding upon by all.


Best regards,


  - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)


----- Original Message -----
> From: lars hofhansl <lh...@yahoo.com>
> To: "dev@hbase.apache.org" <de...@hbase.apache.org>
> Cc: 
> Sent: Wednesday, October 12, 2011 2:08 PM
> Subject: Re: HBase releases...
> 
>T hanks Jesse,
> 
> I think the 'it feels right' approach is a corner stone of open source 
> software.
> (Although the Linux kernel is pretty much time based now, but that's a
> different story).
> 
> So, I am not big fan of time based releases. What I am thinking to nudge the
> 'feels right' point a bit sooner. :)
> 
> Software engineers (me included) tend to want to cram as many features as 
> possible
> into a release and sometimes it is good to make this tendency conscious and to
> counter this a bit to avoid feature bloat. (not saying that is a problem in 
> HBase, just a general statement here).
> 
> As you point out there is a balance to find between few releases with lots of
> features for which users have to wait 12 or more months and releases that 
> don't add much value. We definitely want to avoid the latter.
> 
> And not all releases are the same size and we *have* to be able to integrate big
> features (such as coprocessors).
> Your formula below is good idea as it is used as a guiding post and we still
> follow the 'feels right' approach.
> 
> For example, what if we had branched right after coprocessors went in. Would
> that been useful?
> 
> -- Lars
> 
> 
> ----- Original Message -----
> From: Jesse Yates <je...@gmail.com>
> To: dev@hbase.apache.org
> Cc: 
> Sent: Tuesday, October 11, 2011 10:42 PM
> Subject: Re: HBase releases...
> 
> I see a couple other dimensions as well, and mostly they revolve around the
> user community.
> 
> If we can release more frequently, with truly stable releases, then more
> people will be more likely to run clusters with codebases that are closer to
> trunk. Therefore they will have more benefits like bug fixes and performance
> increases without the worry that they are running unstable/buggy code.
> However, there is a big 'if' here - if we can make sure that the builds 
> that
> go out frequently are really rock solid.
> 
> I think in the past we have had a good track record with putting out stable
> releases, especially given the amount of testing that people in dev are
> doing on real, big clusters (thanks everyone!).
> 
> This then presents the problem of maintaining a _ton_ of branches compounded
> by the difficulty of adding in a sweeping feature (coprocessor-style). That
> was a huge pain to integrate (awesome job getting it in - super excited to
> see .92 go out with it).
> 
> Lars, are you proposing that we stick to more of a time based schedule
> rather than a 'it feels right' mechanism? I worry that we can get caught 
> in
> between making some really good changes and then having essentially a
> half-baked release come out. That will hurt credibility with end users if we
> say yeah, you could you release "x", but you might as well wait till 
> "x+1"
> cause some really good stuff is coming in then. Then why did we take the
> time to release it in the first place?
> 
> As a middle ground, maybe we can look at the number of major and minor
> updates since the last branch and drop a cut a new release when it exceeds
> some threshold. For the first couple of iterations, this would be a flexible
> limit until we find what works and makes sense to maintain. Maybe this also
> means developing something like "x minor changes = 1 major change, and we
> release every Y major change" kind of formula. After that, we can use a
> community voting process to bump the limits for exceptional cases.
> 
> This ensures that we don't do pointless releases, but instead put out
> versions and still minimizes the pain involved in maintaining multiple
> branches.
> 
> What do you think?
> 
> -Jesse Yates
> 
> On Tue, Oct 11, 2011 at 10:22 PM, lars hofhansl <lh...@yahoo.com> 
> wrote:
> 
>>  HBase 0.90 was released Jan 2011. By the time HBase 0.92 is released it
>>  will probably be close to
>>  a year between stable releases.
>> 
>>  Should we try to have more frequent, smaller releases? Maybe 3-4 a year.
>>  For example I would almost say that the performance enhancements from the
>>  Facebook guys would
>>  warrant a new (performance) release "shortly" after 0.92.
>> 
>>  That would hopefully reduce the time and effort needed to stabilize the
>>  releases, at the expense of having to
>>  maintain two or even three branches in addition to trunk that people would
>>  still be actively using.
>> 
>>  Thoughts?
>> 
>>  -- Lars
>> 
>

Re: HBase releases...

[lars hofhansl <lh...@yahoo.com>]

Thanks Jesse,

I think the 'it feels right' approach is a corner stone of open source software.
(Although the Linux kernel is pretty much time based now, but that's a
different story).

So, I am not big fan of time based releases. What I am thinking to nudge the
'feels right' point a bit sooner. :)


Software engineers (me included) tend to want to cram as many features as possible
into a release and sometimes it is good to make this tendency conscious and to
counter this a bit to avoid feature bloat. (not saying that is a problem in HBase, just a 

general statement here).

As you point out there is a balance to find between few releases with lots of
features for which users have to wait 12 or more months and releases that don't
add much value. We definitely want to avoid the latter.


And not all releases are the same size and we *have* to be able to integrate big
features (such as coprocessors).
Your formula below is good idea as it is used as a guiding post and we still
follow the 'feels right' approach.

For example, what if we had branched right after coprocessors went in. Would
that been useful?

-- Lars


----- Original Message -----
From: Jesse Yates <je...@gmail.com>
To: dev@hbase.apache.org
Cc: 
Sent: Tuesday, October 11, 2011 10:42 PM
Subject: Re: HBase releases...

I see a couple other dimensions as well, and mostly they revolve around the
user community.

If we can release more frequently, with truly stable releases, then more
people will be more likely to run clusters with codebases that are closer to
trunk. Therefore they will have more benefits like bug fixes and performance
increases without the worry that they are running unstable/buggy code.
However, there is a big 'if' here - if we can make sure that the builds that
go out frequently are really rock solid.

I think in the past we have had a good track record with putting out stable
releases, especially given the amount of testing that people in dev are
doing on real, big clusters (thanks everyone!).

This then presents the problem of maintaining a _ton_ of branches compounded
by the difficulty of adding in a sweeping feature (coprocessor-style). That
was a huge pain to integrate (awesome job getting it in - super excited to
see .92 go out with it).

Lars, are you proposing that we stick to more of a time based schedule
rather than a 'it feels right' mechanism? I worry that we can get caught in
between making some really good changes and then having essentially a
half-baked release come out. That will hurt credibility with end users if we
say yeah, you could you release "x", but you might as well wait till "x+1"
cause some really good stuff is coming in then. Then why did we take the
time to release it in the first place?

As a middle ground, maybe we can look at the number of major and minor
updates since the last branch and drop a cut a new release when it exceeds
some threshold. For the first couple of iterations, this would be a flexible
limit until we find what works and makes sense to maintain. Maybe this also
means developing something like "x minor changes = 1 major change, and we
release every Y major change" kind of formula. After that, we can use a
community voting process to bump the limits for exceptional cases.

This ensures that we don't do pointless releases, but instead put out
versions and still minimizes the pain involved in maintaining multiple
branches.

What do you think?

-Jesse Yates

On Tue, Oct 11, 2011 at 10:22 PM, lars hofhansl <lh...@yahoo.com> wrote:

> HBase 0.90 was released Jan 2011. By the time HBase 0.92 is released it
> will probably be close to
> a year between stable releases.
>
> Should we try to have more frequent, smaller releases? Maybe 3-4 a year.
> For example I would almost say that the performance enhancements from the
> Facebook guys would
> warrant a new (performance) release "shortly" after 0.92.
>
> That would hopefully reduce the time and effort needed to stabilize the
> releases, at the expense of having to
> maintain two or even three branches in addition to trunk that people would
> still be actively using.
>
> Thoughts?
>
> -- Lars
>

Re: HBase releases...

[Jesse Yates <je...@gmail.com>]

@Todd
Doing the linux release style works well if we have enough people who are
willing and able to maintain a bunch of separate branches. As it stands the
HBase community is pretty small compared to Linux, and I don't know how many
people would be willing to maintain a release - almost a full time job on
its own, leaving little time for writing code.

Maybe before we do some of the 'early' branches we get someone from the
community to volunteer to maintain it?

I'm also worried that could lead to a LOT of fracturing in releases (hence
the million and a half flavors of linux), which I feel is not all that good
for the community (though maybe good for lining people's
**cough**RedHat**cough** pockets).

@Lars

>So, I am not big fan of time based releases. What I am thinking to nudge
the
'>feels right' point a bit sooner. :)

That is kind of what I was getting at. I wanted to formalize the 'feels
right' process, so we don't start doing massive code drops.

>Software engineers (me included) tend to want to cram as many features as
possible
>into a release and sometimes it is good to make this tendency conscious and
to
>counter this a bit to avoid feature bloat. (not saying that is a problem in
HBase, just a
>general statement here).

I'm all for adding good stuff (and I can be as much a culprit as anyone ;)

>For example, what if we had branched right after coprocessors went in.
Would
>that been useful?

That is definitely fair that we have done some solid stuff since. But CPs
are the exception, not the rule. Hence, the provision that we can vote to
keep going.

Basically, I'm also lobbying for a way to hit a 'feels right' sooner, and I
think metrics (rather than emotion) is the way to go.

-Jesse Yates

On Tue, Oct 11, 2011 at 11:19 PM, lars hofhansl <lh...@yahoo.com> wrote:

> I just wrote in parallel response to Jesse that I am not a big fan of
> time based releases :)
>
> The way you describe it it makes sense, though.
>
> o we'd branch at given intervals (once a week/month/quarter/whatever)
> o some releases will turn out to be good ones, users could standardize on
> those
>
> o we would only provide updates in a form of a new release, or maybe a dev
> and stable release
>
>
> Might be hard to get bigger features in that way; although Linux
> manages that with merge windows.
>
>
> Seems like this'd be a big shift. Hence my softer approach to just change
> a bit what 'it feels right' means.
>
> -- Lars
>
>
> ----- Original Message -----
> From: Todd Lipcon <to...@cloudera.com>
> To: dev@hbase.apache.org
> Cc:
> Sent: Tuesday, October 11, 2011 11:02 PM
> Subject: Re: HBase releases...
>
> I was discussing this with a few other contributors a while back. In
> my mind there are two ways to do it:
>
> a) feature based releases that we test the crap out of and decide are
> stable. We can't really do this more than once a year, I don't think.
> b) time based releases where we make few guarantees beyond "it appears
> to work". We assume that once a year or so, major users and
> distributors latch on to one of these that feels good, and start
> maintaining it as a stable/maintenance series.
>
> B is the Linux model. No one in their right mind runs the latest
> kernel.org in production - but certain vintages of kernel gather
> enough folks around them, pick up a stable maintainer, etc, and after
> a while become pretty rock solid. For example, 2.6.18 and 2.6.32 seem
> to be popular kernel vintages.
>
> I'm personally of the mind that B is the more reasonable model. Keeps
> a release train ticking all the time, allows experimental users to get
> new features fast, and avoids the burden of feeling that any release
> we do has to be perfect. It's also often hard to say what "vintage"
> will be good until it's been out in the wild for a little while - for
> example, the 0.89.20100621 dev release turned out to be pretty decent
> despite being branded as a dev release. The later releases in that
> series had some more major problems.
>
> The risk with the "B" model is that everyone might end up on different
> versions, making it very hard to support the user community, etc. This
> might be partially ameliorated by some clear descriptions on the
> download/release pages which delineate which are stable/recommended
> releases and which are just time-based.
>
> -Todd
>
>
> On Tue, Oct 11, 2011 at 10:42 PM, Jesse Yates <je...@gmail.com>
> wrote:
> > I see a couple other dimensions as well, and mostly they revolve around
> the
> > user community.
> >
> > If we can release more frequently, with truly stable releases, then more
> > people will be more likely to run clusters with codebases that are closer
> to
> > trunk. Therefore they will have more benefits like bug fixes and
> performance
> > increases without the worry that they are running unstable/buggy code.
> > However, there is a big 'if' here - if we can make sure that the builds
> that
> > go out frequently are really rock solid.
> >
> > I think in the past we have had a good track record with putting out
> stable
> > releases, especially given the amount of testing that people in dev are
> > doing on real, big clusters (thanks everyone!).
> >
> > This then presents the problem of maintaining a _ton_ of branches
> compounded
> > by the difficulty of adding in a sweeping feature (coprocessor-style).
> That
> > was a huge pain to integrate (awesome job getting it in - super excited
> to
> > see .92 go out with it).
> >
> > Lars, are you proposing that we stick to more of a time based schedule
> > rather than a 'it feels right' mechanism? I worry that we can get caught
> in
> > between making some really good changes and then having essentially a
> > half-baked release come out. That will hurt credibility with end users if
> we
> > say yeah, you could you release "x", but you might as well wait till
> "x+1"
> > cause some really good stuff is coming in then. Then why did we take the
> > time to release it in the first place?
> >
> > As a middle ground, maybe we can look at the number of major and minor
> > updates since the last branch and drop a cut a new release when it
> exceeds
> > some threshold. For the first couple of iterations, this would be a
> flexible
> > limit until we find what works and makes sense to maintain. Maybe this
> also
> > means developing something like "x minor changes = 1 major change, and we
> > release every Y major change" kind of formula. After that, we can use a
> > community voting process to bump the limits for exceptional cases.
> >
> > This ensures that we don't do pointless releases, but instead put out
> > versions and still minimizes the pain involved in maintaining multiple
> > branches.
> >
> > What do you think?
> >
> > -Jesse Yates
> >
> > On Tue, Oct 11, 2011 at 10:22 PM, lars hofhansl <lh...@yahoo.com>
> wrote:
> >
> >> HBase 0.90 was released Jan 2011. By the time HBase 0.92 is released it
> >> will probably be close to
> >> a year between stable releases.
> >>
> >> Should we try to have more frequent, smaller releases? Maybe 3-4 a year.
> >> For example I would almost say that the performance enhancements from
> the
> >> Facebook guys would
> >> warrant a new (performance) release "shortly" after 0.92.
> >>
> >> That would hopefully reduce the time and effort needed to stabilize the
> >> releases, at the expense of having to
> >> maintain two or even three branches in addition to trunk that people
> would
> >> still be actively using.
> >>
> >> Thoughts?
> >>
> >> -- Lars
> >>
> >
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>
>

Re: HBase releases...

[lars hofhansl <lh...@yahoo.com>]

I just wrote in parallel response to Jesse that I am not a big fan of
time based releases :)

The way you describe it it makes sense, though.

o we'd branch at given intervals (once a week/month/quarter/whatever)
o some releases will turn out to be good ones, users could standardize on those

o we would only provide updates in a form of a new release, or maybe a dev and stable release


Might be hard to get bigger features in that way; although Linux
manages that with merge windows.


Seems like this'd be a big shift. Hence my softer approach to just change
a bit what 'it feels right' means.

-- Lars


----- Original Message -----
From: Todd Lipcon <to...@cloudera.com>
To: dev@hbase.apache.org
Cc: 
Sent: Tuesday, October 11, 2011 11:02 PM
Subject: Re: HBase releases...

I was discussing this with a few other contributors a while back. In
my mind there are two ways to do it:

a) feature based releases that we test the crap out of and decide are
stable. We can't really do this more than once a year, I don't think.
b) time based releases where we make few guarantees beyond "it appears
to work". We assume that once a year or so, major users and
distributors latch on to one of these that feels good, and start
maintaining it as a stable/maintenance series.

B is the Linux model. No one in their right mind runs the latest
kernel.org in production - but certain vintages of kernel gather
enough folks around them, pick up a stable maintainer, etc, and after
a while become pretty rock solid. For example, 2.6.18 and 2.6.32 seem
to be popular kernel vintages.

I'm personally of the mind that B is the more reasonable model. Keeps
a release train ticking all the time, allows experimental users to get
new features fast, and avoids the burden of feeling that any release
we do has to be perfect. It's also often hard to say what "vintage"
will be good until it's been out in the wild for a little while - for
example, the 0.89.20100621 dev release turned out to be pretty decent
despite being branded as a dev release. The later releases in that
series had some more major problems.

The risk with the "B" model is that everyone might end up on different
versions, making it very hard to support the user community, etc. This
might be partially ameliorated by some clear descriptions on the
download/release pages which delineate which are stable/recommended
releases and which are just time-based.

-Todd


On Tue, Oct 11, 2011 at 10:42 PM, Jesse Yates <je...@gmail.com> wrote:
> I see a couple other dimensions as well, and mostly they revolve around the
> user community.
>
> If we can release more frequently, with truly stable releases, then more
> people will be more likely to run clusters with codebases that are closer to
> trunk. Therefore they will have more benefits like bug fixes and performance
> increases without the worry that they are running unstable/buggy code.
> However, there is a big 'if' here - if we can make sure that the builds that
> go out frequently are really rock solid.
>
> I think in the past we have had a good track record with putting out stable
> releases, especially given the amount of testing that people in dev are
> doing on real, big clusters (thanks everyone!).
>
> This then presents the problem of maintaining a _ton_ of branches compounded
> by the difficulty of adding in a sweeping feature (coprocessor-style). That
> was a huge pain to integrate (awesome job getting it in - super excited to
> see .92 go out with it).
>
> Lars, are you proposing that we stick to more of a time based schedule
> rather than a 'it feels right' mechanism? I worry that we can get caught in
> between making some really good changes and then having essentially a
> half-baked release come out. That will hurt credibility with end users if we
> say yeah, you could you release "x", but you might as well wait till "x+1"
> cause some really good stuff is coming in then. Then why did we take the
> time to release it in the first place?
>
> As a middle ground, maybe we can look at the number of major and minor
> updates since the last branch and drop a cut a new release when it exceeds
> some threshold. For the first couple of iterations, this would be a flexible
> limit until we find what works and makes sense to maintain. Maybe this also
> means developing something like "x minor changes = 1 major change, and we
> release every Y major change" kind of formula. After that, we can use a
> community voting process to bump the limits for exceptional cases.
>
> This ensures that we don't do pointless releases, but instead put out
> versions and still minimizes the pain involved in maintaining multiple
> branches.
>
> What do you think?
>
> -Jesse Yates
>
> On Tue, Oct 11, 2011 at 10:22 PM, lars hofhansl <lh...@yahoo.com> wrote:
>
>> HBase 0.90 was released Jan 2011. By the time HBase 0.92 is released it
>> will probably be close to
>> a year between stable releases.
>>
>> Should we try to have more frequent, smaller releases? Maybe 3-4 a year.
>> For example I would almost say that the performance enhancements from the
>> Facebook guys would
>> warrant a new (performance) release "shortly" after 0.92.
>>
>> That would hopefully reduce the time and effort needed to stabilize the
>> releases, at the expense of having to
>> maintain two or even three branches in addition to trunk that people would
>> still be actively using.
>>
>> Thoughts?
>>
>> -- Lars
>>
>



-- 
Todd Lipcon
Software Engineer, Cloudera

Re: HBase releases...

[Todd Lipcon <to...@cloudera.com>]

I was discussing this with a few other contributors a while back. In
my mind there are two ways to do it:

a) feature based releases that we test the crap out of and decide are
stable. We can't really do this more than once a year, I don't think.
b) time based releases where we make few guarantees beyond "it appears
to work". We assume that once a year or so, major users and
distributors latch on to one of these that feels good, and start
maintaining it as a stable/maintenance series.

B is the Linux model. No one in their right mind runs the latest
kernel.org in production - but certain vintages of kernel gather
enough folks around them, pick up a stable maintainer, etc, and after
a while become pretty rock solid. For example, 2.6.18 and 2.6.32 seem
to be popular kernel vintages.

I'm personally of the mind that B is the more reasonable model. Keeps
a release train ticking all the time, allows experimental users to get
new features fast, and avoids the burden of feeling that any release
we do has to be perfect. It's also often hard to say what "vintage"
will be good until it's been out in the wild for a little while - for
example, the 0.89.20100621 dev release turned out to be pretty decent
despite being branded as a dev release. The later releases in that
series had some more major problems.

The risk with the "B" model is that everyone might end up on different
versions, making it very hard to support the user community, etc. This
might be partially ameliorated by some clear descriptions on the
download/release pages which delineate which are stable/recommended
releases and which are just time-based.

-Todd


On Tue, Oct 11, 2011 at 10:42 PM, Jesse Yates <je...@gmail.com> wrote:
> I see a couple other dimensions as well, and mostly they revolve around the
> user community.
>
> If we can release more frequently, with truly stable releases, then more
> people will be more likely to run clusters with codebases that are closer to
> trunk. Therefore they will have more benefits like bug fixes and performance
> increases without the worry that they are running unstable/buggy code.
> However, there is a big 'if' here - if we can make sure that the builds that
> go out frequently are really rock solid.
>
> I think in the past we have had a good track record with putting out stable
> releases, especially given the amount of testing that people in dev are
> doing on real, big clusters (thanks everyone!).
>
> This then presents the problem of maintaining a _ton_ of branches compounded
> by the difficulty of adding in a sweeping feature (coprocessor-style). That
> was a huge pain to integrate (awesome job getting it in - super excited to
> see .92 go out with it).
>
> Lars, are you proposing that we stick to more of a time based schedule
> rather than a 'it feels right' mechanism? I worry that we can get caught in
> between making some really good changes and then having essentially a
> half-baked release come out. That will hurt credibility with end users if we
> say yeah, you could you release "x", but you might as well wait till "x+1"
> cause some really good stuff is coming in then. Then why did we take the
> time to release it in the first place?
>
> As a middle ground, maybe we can look at the number of major and minor
> updates since the last branch and drop a cut a new release when it exceeds
> some threshold. For the first couple of iterations, this would be a flexible
> limit until we find what works and makes sense to maintain. Maybe this also
> means developing something like "x minor changes = 1 major change, and we
> release every Y major change" kind of formula. After that, we can use a
> community voting process to bump the limits for exceptional cases.
>
> This ensures that we don't do pointless releases, but instead put out
> versions and still minimizes the pain involved in maintaining multiple
> branches.
>
> What do you think?
>
> -Jesse Yates
>
> On Tue, Oct 11, 2011 at 10:22 PM, lars hofhansl <lh...@yahoo.com> wrote:
>
>> HBase 0.90 was released Jan 2011. By the time HBase 0.92 is released it
>> will probably be close to
>> a year between stable releases.
>>
>> Should we try to have more frequent, smaller releases? Maybe 3-4 a year.
>> For example I would almost say that the performance enhancements from the
>> Facebook guys would
>> warrant a new (performance) release "shortly" after 0.92.
>>
>> That would hopefully reduce the time and effort needed to stabilize the
>> releases, at the expense of having to
>> maintain two or even three branches in addition to trunk that people would
>> still be actively using.
>>
>> Thoughts?
>>
>> -- Lars
>>
>



-- 
Todd Lipcon
Software Engineer, Cloudera

Re: HBase releases...

[Jesse Yates <je...@gmail.com>]

I see a couple other dimensions as well, and mostly they revolve around the
user community.

If we can release more frequently, with truly stable releases, then more
people will be more likely to run clusters with codebases that are closer to
trunk. Therefore they will have more benefits like bug fixes and performance
increases without the worry that they are running unstable/buggy code.
However, there is a big 'if' here - if we can make sure that the builds that
go out frequently are really rock solid.

I think in the past we have had a good track record with putting out stable
releases, especially given the amount of testing that people in dev are
doing on real, big clusters (thanks everyone!).

This then presents the problem of maintaining a _ton_ of branches compounded
by the difficulty of adding in a sweeping feature (coprocessor-style). That
was a huge pain to integrate (awesome job getting it in - super excited to
see .92 go out with it).

Lars, are you proposing that we stick to more of a time based schedule
rather than a 'it feels right' mechanism? I worry that we can get caught in
between making some really good changes and then having essentially a
half-baked release come out. That will hurt credibility with end users if we
say yeah, you could you release "x", but you might as well wait till "x+1"
cause some really good stuff is coming in then. Then why did we take the
time to release it in the first place?

As a middle ground, maybe we can look at the number of major and minor
updates since the last branch and drop a cut a new release when it exceeds
some threshold. For the first couple of iterations, this would be a flexible
limit until we find what works and makes sense to maintain. Maybe this also
means developing something like "x minor changes = 1 major change, and we
release every Y major change" kind of formula. After that, we can use a
community voting process to bump the limits for exceptional cases.

This ensures that we don't do pointless releases, but instead put out
versions and still minimizes the pain involved in maintaining multiple
branches.

What do you think?

-Jesse Yates

On Tue, Oct 11, 2011 at 10:22 PM, lars hofhansl <lh...@yahoo.com> wrote:

> HBase 0.90 was released Jan 2011. By the time HBase 0.92 is released it
> will probably be close to
> a year between stable releases.
>
> Should we try to have more frequent, smaller releases? Maybe 3-4 a year.
> For example I would almost say that the performance enhancements from the
> Facebook guys would
> warrant a new (performance) release "shortly" after 0.92.
>
> That would hopefully reduce the time and effort needed to stabilize the
> releases, at the expense of having to
> maintain two or even three branches in addition to trunk that people would
> still be actively using.
>
> Thoughts?
>
> -- Lars
>