Maintenance releases

[Gary Dusbabek <gd...@gmail.com>]

I've been uncomfortable with the amount of features I perceive are
going into our maintenance releases for a while now.  I thought it
would stop after we committed ourselves to having a more predictable
major release schedule.  But getting 0.7.1 out feels like it's taken a
lot more effort than it should have.  I wonder if part of the problem
is that we've been committing destabilizing features into it?  IMO,
maintenance releases (0.7.1, 0.7.2, etc.) should only contain bug
fixes and *carefully* vetted features.

I've scanned down the list of 0.7.1 changes in CHANGES.txt and about
half of them are features that I think could have stayed in trunk.  I
think we did this a lot with the early maintenance releases of 0.6 as
well, probably in an effort to get features out *now* instead of
waiting for an 0.7 that was not happening soon enough.  We've decided
to pick up the pace of our major release schedule (sticking to four
months).  I think maintaining this pace will be difficult if we
continue to commit as many features into the minor releases as we have
been.

I'm willing to concede that I may have an abnormally conservative
opinion about this.  But I wanted to voice my concern in hopes we can
improve the quality and delivery of our maintenance releases.

Gary.

Re: Maintenance releases

[Johan Oskarsson <jo...@oskarsson.nu>]

+1. 

Cassandra has matured a lot lately and more users are relying heavily on it in production. For those users, including us, stability and predictability becomes very important.
Not including new and potentially unstable features in maintenance releases is an easy way to decrease risk at a low cost.

/Johan

On 11 feb 2011, at 08.35, Gary Dusbabek wrote:

> I've been uncomfortable with the amount of features I perceive are
> going into our maintenance releases for a while now.  I thought it
> would stop after we committed ourselves to having a more predictable
> major release schedule.  But getting 0.7.1 out feels like it's taken a
> lot more effort than it should have.  I wonder if part of the problem
> is that we've been committing destabilizing features into it?  IMO,
> maintenance releases (0.7.1, 0.7.2, etc.) should only contain bug
> fixes and *carefully* vetted features.
> 
> I've scanned down the list of 0.7.1 changes in CHANGES.txt and about
> half of them are features that I think could have stayed in trunk.  I
> think we did this a lot with the early maintenance releases of 0.6 as
> well, probably in an effort to get features out *now* instead of
> waiting for an 0.7 that was not happening soon enough.  We've decided
> to pick up the pace of our major release schedule (sticking to four
> months).  I think maintaining this pace will be difficult if we
> continue to commit as many features into the minor releases as we have
> been.
> 
> I'm willing to concede that I may have an abnormally conservative
> opinion about this.  But I wanted to voice my concern in hopes we can
> improve the quality and delivery of our maintenance releases.
> 
> Gary.

Re: Maintenance releases

[Ryan King <ry...@twitter.com>]

On Fri, Feb 11, 2011 at 8:35 AM, Gary Dusbabek <gd...@gmail.com> wrote:
> I've been uncomfortable with the amount of features I perceive are
> going into our maintenance releases for a while now.  I thought it
> would stop after we committed ourselves to having a more predictable
> major release schedule.  But getting 0.7.1 out feels like it's taken a
> lot more effort than it should have.  I wonder if part of the problem
> is that we've been committing destabilizing features into it?  IMO,
> maintenance releases (0.7.1, 0.7.2, etc.) should only contain bug
> fixes and *carefully* vetted features.
>
> I've scanned down the list of 0.7.1 changes in CHANGES.txt and about
> half of them are features that I think could have stayed in trunk.  I
> think we did this a lot with the early maintenance releases of 0.6 as
> well, probably in an effort to get features out *now* instead of
> waiting for an 0.7 that was not happening soon enough.  We've decided
> to pick up the pace of our major release schedule (sticking to four
> months).  I think maintaining this pace will be difficult if we
> continue to commit as many features into the minor releases as we have
> been.
>
> I'm willing to concede that I may have an abnormally conservative
> opinion about this.  But I wanted to voice my concern in hopes we can
> improve the quality and delivery of our maintenance releases.

I agree with you. We've tried both approaches and I believe that its
clear that releasing features in maintenance releases leads to more
pain and unpredictability.

-ryan

Re: Maintenance releases

[Jonathan Ellis <jb...@gmail.com>]

Qualified +1 from me -- I went back and checked the 3 prior 0.7.1
votes, and all of them were canceled because of regressions from the
#1905/#1959/#2058 series, which was a bug fix ("make dynamic snitch
actually work") not a new feature.  It turned out to be more work to
get all the corner cases worked out than anyone thought, but dynamic
snitch is critical for systems under heavy load so even with the
benefit of hindsight I would say it was worth doing in 0.7.

On Fri, Feb 11, 2011 at 10:35 AM, Gary Dusbabek <gd...@gmail.com> wrote:
> I've been uncomfortable with the amount of features I perceive are
> going into our maintenance releases for a while now.  I thought it
> would stop after we committed ourselves to having a more predictable
> major release schedule.  But getting 0.7.1 out feels like it's taken a
> lot more effort than it should have.  I wonder if part of the problem
> is that we've been committing destabilizing features into it?  IMO,
> maintenance releases (0.7.1, 0.7.2, etc.) should only contain bug
> fixes and *carefully* vetted features.
>
> I've scanned down the list of 0.7.1 changes in CHANGES.txt and about
> half of them are features that I think could have stayed in trunk.  I
> think we did this a lot with the early maintenance releases of 0.6 as
> well, probably in an effort to get features out *now* instead of
> waiting for an 0.7 that was not happening soon enough.  We've decided
> to pick up the pace of our major release schedule (sticking to four
> months).  I think maintaining this pace will be difficult if we
> continue to commit as many features into the minor releases as we have
> been.
>
> I'm willing to concede that I may have an abnormally conservative
> opinion about this.  But I wanted to voice my concern in hopes we can
> improve the quality and delivery of our maintenance releases.
>
> Gary.
>



-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com

Re: Maintenance releases

[Robert Coli <rc...@digg.com>]

On Fri, Feb 11, 2011 at 8:35 AM, Gary Dusbabek <gd...@gmail.com> wrote:
> I've been uncomfortable with the amount of features I perceive are
> going into our maintenance releases for a while now.  [...]  IMO,
> maintenance releases (0.7.1, 0.7.2, etc.) should only contain bug
> fixes and *carefully* vetted features.
> [...]
> I'm willing to concede that I may have an abnormally conservative
> opinion about this.  But I wanted to voice my concern in hopes we can
> improve the quality and delivery of our maintenance releases.

It should surprise almost no-one that I am +1 on the above. :)

I'd like to also mention a potential semantic challenge regarding
"bug-fix" releases. Once a "bug-fix" is over a certain scope, or
involves a new technique to solve the old problem, it may qualify as a
"refactor" instead of a "bug-fix." I believe that we would benefit by
consciously attempting to avoid "refactors" while doing "bug-fix"
release. I understand that the above is semantic squish, but I think
the concepts can be a useful part of this discussion.

Peter Schuller also said :
> For example, from the point of view of the user, I think that
> things like CASSANDRA-1992 should preferably result in an almost
> immediate bugfix-only release with instructions and impact information
> for users.

+1 this very much, from an ops/DBA perspective. If, for example,
upgrading to a version can BREAK YOUR STORED DATA PERMANENTLY UNDER
NORMAL OPERATING CONDITIONS, that version should either be immediately
superceded by a paper-bag release containing only the relevant fix, or
a GIANT BLINKING RED WARNING should be posted everywhere indicating
its known-unsafeness.

=Rob

Re: Maintenance releases

[Jonathan Ellis <jb...@gmail.com>]

On Fri, Feb 11, 2011 at 11:07 AM, Peter Schuller
<pe...@infidyne.com> wrote:
> For example, from the point of view of the user, I think that
> things like CASSANDRA-1992 should preferably result in an almost
> immediate bugfix-only release with instructions and impact information
> for users.

+1

-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com

Re: Maintenance releases

[Peter Schuller <pe...@infidyne.com>]

> I'm willing to concede that I may have an abnormally conservative
> opinion about this.  But I wanted to voice my concern in hopes we can
> improve the quality and delivery of our maintenance releases.

(speaking now from the perspective of a consumer, disregarding the
implications on development)

I don't think you're being conservative. In particular in light of the
recent 1.0 discussion and whether or not to signal that Cassandra is
"ready" and no longer needing committers on staff etc; having solid
releases is probably an important part of instilling confidence in the
project and decreasing the risk associated with adopting a new
release. For example, from the point of view of the user, I think that
things like CASSANDRA-1992 should preferably result in an almost
immediate bugfix-only release with instructions and impact information
for users.

Being only a very minor contributor I don't have a full grasp of the
implications on the agility of development that a change would have,
but certainly I think that if a goal is to have users feel more
confident (and in fact be safer) in using a Cassandre release without
careful monitoring of the mailing lists and JIRA, adjusting the
release engineering a bit seems like a high-priority change towards
that goal.

-- 
/ Peter Schuller

Re: Maintenance releases

[Jonathan Shook <js...@gmail.com>]

As a user, this sounds like great news. To see the consensus on this
issue is reassuring.

For me, release stability and planning are more important that new
features. I would rather wait longer for the features if it means I'm
getting a solid release. It would be great if there were some clearing
of the air with respect to release discipline going forward.  Granted,
there was a time when everybody expected there to be hard and fast
changes, as Cassandra was relatively new on the landscape. I think we
are past that expectation now, or at least to the knee of the curve.


On Fri, Feb 11, 2011 at 1:36 PM, Jeremy Hanna
<je...@gmail.com> wrote:
> strong unbinding +1 :)
>
> I think that there were several lessons learned in the 0.6.x line about walking that line.  Wrt regression testing, hopefully the distributed tests (thanks Stu and Kelvin and others!) will act as a core for something like that.  I would imagine that heavy loads can be utilized in there as well.
>
> On Feb 11, 2011, at 12:19 PM, Jake Luciani wrote:
>
>> +1
>>
>> I'm also concerned with our lack of regression testing.  A lot of this is
>> done by individual committers firing up EC2 clusters and running basic
>> sanity checks and workloads.  Most of the bugs we are finding pop up under
>> heavy load.
>>
>> It would be great if the community could identify and contribute use cases
>> that could be bundled into a regression test suite.
>>
>>
>>
>> On Fri, Feb 11, 2011 at 11:35 AM, Gary Dusbabek <gd...@gmail.com> wrote:
>>
>>> I've been uncomfortable with the amount of features I perceive are
>>> going into our maintenance releases for a while now.  I thought it
>>> would stop after we committed ourselves to having a more predictable
>>> major release schedule.  But getting 0.7.1 out feels like it's taken a
>>> lot more effort than it should have.  I wonder if part of the problem
>>> is that we've been committing destabilizing features into it?  IMO,
>>> maintenance releases (0.7.1, 0.7.2, etc.) should only contain bug
>>> fixes and *carefully* vetted features.
>>>
>>> I've scanned down the list of 0.7.1 changes in CHANGES.txt and about
>>> half of them are features that I think could have stayed in trunk.  I
>>> think we did this a lot with the early maintenance releases of 0.6 as
>>> well, probably in an effort to get features out *now* instead of
>>> waiting for an 0.7 that was not happening soon enough.  We've decided
>>> to pick up the pace of our major release schedule (sticking to four
>>> months).  I think maintaining this pace will be difficult if we
>>> continue to commit as many features into the minor releases as we have
>>> been.
>>>
>>> I'm willing to concede that I may have an abnormally conservative
>>> opinion about this.  But I wanted to voice my concern in hopes we can
>>> improve the quality and delivery of our maintenance releases.
>>>
>>> Gary.
>>>
>
>

Re: Maintenance releases

[Jeremy Hanna <je...@gmail.com>]

strong unbinding +1 :)

I think that there were several lessons learned in the 0.6.x line about walking that line.  Wrt regression testing, hopefully the distributed tests (thanks Stu and Kelvin and others!) will act as a core for something like that.  I would imagine that heavy loads can be utilized in there as well.

On Feb 11, 2011, at 12:19 PM, Jake Luciani wrote:

> +1
> 
> I'm also concerned with our lack of regression testing.  A lot of this is
> done by individual committers firing up EC2 clusters and running basic
> sanity checks and workloads.  Most of the bugs we are finding pop up under
> heavy load.
> 
> It would be great if the community could identify and contribute use cases
> that could be bundled into a regression test suite.
> 
> 
> 
> On Fri, Feb 11, 2011 at 11:35 AM, Gary Dusbabek <gd...@gmail.com> wrote:
> 
>> I've been uncomfortable with the amount of features I perceive are
>> going into our maintenance releases for a while now.  I thought it
>> would stop after we committed ourselves to having a more predictable
>> major release schedule.  But getting 0.7.1 out feels like it's taken a
>> lot more effort than it should have.  I wonder if part of the problem
>> is that we've been committing destabilizing features into it?  IMO,
>> maintenance releases (0.7.1, 0.7.2, etc.) should only contain bug
>> fixes and *carefully* vetted features.
>> 
>> I've scanned down the list of 0.7.1 changes in CHANGES.txt and about
>> half of them are features that I think could have stayed in trunk.  I
>> think we did this a lot with the early maintenance releases of 0.6 as
>> well, probably in an effort to get features out *now* instead of
>> waiting for an 0.7 that was not happening soon enough.  We've decided
>> to pick up the pace of our major release schedule (sticking to four
>> months).  I think maintaining this pace will be difficult if we
>> continue to commit as many features into the minor releases as we have
>> been.
>> 
>> I'm willing to concede that I may have an abnormally conservative
>> opinion about this.  But I wanted to voice my concern in hopes we can
>> improve the quality and delivery of our maintenance releases.
>> 
>> Gary.
>> 

Re: Maintenance releases

[Jake Luciani <ja...@gmail.com>]

+1

I'm also concerned with our lack of regression testing.  A lot of this is
done by individual committers firing up EC2 clusters and running basic
sanity checks and workloads.  Most of the bugs we are finding pop up under
heavy load.

It would be great if the community could identify and contribute use cases
that could be bundled into a regression test suite.



On Fri, Feb 11, 2011 at 11:35 AM, Gary Dusbabek <gd...@gmail.com> wrote:

> I've been uncomfortable with the amount of features I perceive are
> going into our maintenance releases for a while now.  I thought it
> would stop after we committed ourselves to having a more predictable
> major release schedule.  But getting 0.7.1 out feels like it's taken a
> lot more effort than it should have.  I wonder if part of the problem
> is that we've been committing destabilizing features into it?  IMO,
> maintenance releases (0.7.1, 0.7.2, etc.) should only contain bug
> fixes and *carefully* vetted features.
>
> I've scanned down the list of 0.7.1 changes in CHANGES.txt and about
> half of them are features that I think could have stayed in trunk.  I
> think we did this a lot with the early maintenance releases of 0.6 as
> well, probably in an effort to get features out *now* instead of
> waiting for an 0.7 that was not happening soon enough.  We've decided
> to pick up the pace of our major release schedule (sticking to four
> months).  I think maintaining this pace will be difficult if we
> continue to commit as many features into the minor releases as we have
> been.
>
> I'm willing to concede that I may have an abnormally conservative
> opinion about this.  But I wanted to voice my concern in hopes we can
> improve the quality and delivery of our maintenance releases.
>
> Gary.
>