Beyond 1.6

[Josh Elser <jo...@gmail.com>]

All,

I'd like to start the discussion about where we go after we release 
1.6.0. We have a lot of great ideas that people have outlined that 
together, I think, would make a really compelling version of Accumulo.

Besides a set of new features/changes, we've also talked about changing 
how we version the software in such a way that would better align with 
how we want to support it and the  guarantees we want to provide. Some 
of these will require additional discussion, but I believe there are a 
few things we could knock out ahead of time that will help in these 
later discussions (especially those regarding API, data and wire 
compatitbility).

The easy one: is everyone happy with calling the next "major" version 
after 1.6.0, "2.0.0"? I believe that reclaiming that extra digit in our 
current version string (the "1" in 1.6.0) will help alleviate many of 
the problems that we've been facing in where code is "allowed" to go.

Going a little deeper, I think we can also agree to some *general* 
guidelines about what these numbers mean, following what semantic 
versioning generally outlines based on what we've been trying to follow 
to date. The major (most-significant) number refers to releases in which 
very impactful changes were made to the codebase. The minor (middle) 
number refers to changes/improvements which are lesser in nature, but do 
not represent a major shift in how to use or administer Accumulo. The 
bugix (least-significant) number should *only* contain the least 
impactful change which address some breakage in the code.

  - an important point that needs clarification is how we draw the line 
between which non-breaking changes can be put in a minor release and 
which must be introduced in a major release.

Bugfix versions should not make any compatibility changes at all - fix 
the bug as it stands. Minor versions can introduce additions to the 
public API, storage or wire implementations, but they should be done in 
a backwards compatible way. Major versions can remove deprecated public 
API calls. Data compatibility can be met with some upgrade process 
instead of full compatibility with the previous versions. Wire 
compatibility does not need to be guaranteed across major versions if 
this is not possible.

Then, we can target major releases ~yearly, minor releases every few 
months, and bugfix releases in order of weeks based on severity. I would 
also suggest that we reduce our testing burden for bugfixes to be more 
focused on verifying that the bugs have been identified in tests and 
verified as fixed.

This got a lot longer than I wanted, so I'm sorry for that. Please feel 
free to suggest pruning to this list so we can have a discussion that 
nets some actual results.

- Josh

Re: Beyond 1.6

[Bill Havanki <bh...@clouderagovt.com>]

Thanks for setting out the discussion, Josh. I agree with you on all of
these issues. My thoughts:

* +1 on "2.0.0". We absolutely could use the extra "room" for versioning.

* The closer we stick to semantic versioning [1] (or some other well-known
versioning spec ... are there any?), the better. a) Versioning is an
encoded message to our users on what changed, and using a common language
is best for that (with limited tailoring as needed). b) Lots of other
projects' experiences are baked into such a standard, like a design
pattern, and we can use that.

* I think we'll naturally get more frequent releases by adopting a more
flexible versioning scheme, adjusting CM practices to match, and improving
and refining the test framework. So, release timing won't be a big deal -
we'll have the power to pick the cadence we like. Therefore, I'd see
release timing as something to define later on in this process.

I suggest the first step is to define the Accumulo versioning scheme,
starting at 2.0.0. The definition can be driven by all the compatibility
concerns, so those need to be enumerated and analyzed for how they evolve
and how that should reflect in versioning.

Bill H

[1] http://semver.org/


On Thu, Apr 24, 2014 at 11:45 AM, Josh Elser <jo...@gmail.com> wrote:

> All,
>
> I'd like to start the discussion about where we go after we release 1.6.0.
> We have a lot of great ideas that people have outlined that together, I
> think, would make a really compelling version of Accumulo.
>
> Besides a set of new features/changes, we've also talked about changing
> how we version the software in such a way that would better align with how
> we want to support it and the  guarantees we want to provide. Some of these
> will require additional discussion, but I believe there are a few things we
> could knock out ahead of time that will help in these later discussions
> (especially those regarding API, data and wire compatitbility).
>
> The easy one: is everyone happy with calling the next "major" version
> after 1.6.0, "2.0.0"? I believe that reclaiming that extra digit in our
> current version string (the "1" in 1.6.0) will help alleviate many of the
> problems that we've been facing in where code is "allowed" to go.
>
> Going a little deeper, I think we can also agree to some *general*
> guidelines about what these numbers mean, following what semantic
> versioning generally outlines based on what we've been trying to follow to
> date. The major (most-significant) number refers to releases in which very
> impactful changes were made to the codebase. The minor (middle) number
> refers to changes/improvements which are lesser in nature, but do not
> represent a major shift in how to use or administer Accumulo. The bugix
> (least-significant) number should *only* contain the least impactful change
> which address some breakage in the code.
>
>  - an important point that needs clarification is how we draw the line
> between which non-breaking changes can be put in a minor release and which
> must be introduced in a major release.
>
> Bugfix versions should not make any compatibility changes at all - fix the
> bug as it stands. Minor versions can introduce additions to the public API,
> storage or wire implementations, but they should be done in a backwards
> compatible way. Major versions can remove deprecated public API calls. Data
> compatibility can be met with some upgrade process instead of full
> compatibility with the previous versions. Wire compatibility does not need
> to be guaranteed across major versions if this is not possible.
>
> Then, we can target major releases ~yearly, minor releases every few
> months, and bugfix releases in order of weeks based on severity. I would
> also suggest that we reduce our testing burden for bugfixes to be more
> focused on verifying that the bugs have been identified in tests and
> verified as fixed.
>
> This got a lot longer than I wanted, so I'm sorry for that. Please feel
> free to suggest pruning to this list so we can have a discussion that nets
> some actual results.
>
> - Josh
>



-- 
// Bill Havanki
// Solutions Architect, Cloudera Govt Solutions
// 443.686.9283

Re: Beyond 1.6

[Christopher <ct...@apache.org>]

On Thu, Apr 24, 2014 at 11:45 AM, Josh Elser <jo...@gmail.com> wrote:
> All,
>
> I'd like to start the discussion about where we go after we release 1.6.0.
> We have a lot of great ideas that people have outlined that together, I
> think, would make a really compelling version of Accumulo.
>
> Besides a set of new features/changes, we've also talked about changing how
> we version the software in such a way that would better align with how we
> want to support it and the  guarantees we want to provide. Some of these
> will require additional discussion, but I believe there are a few things we
> could knock out ahead of time that will help in these later discussions
> (especially those regarding API, data and wire compatitbility).
>
> The easy one: is everyone happy with calling the next "major" version after
> 1.6.0, "2.0.0"? I believe that reclaiming that extra digit in our current
> version string (the "1" in 1.6.0) will help alleviate many of the problems
> that we've been facing in where code is "allowed" to go.

+1 for making the next major version 2.0.0, regardless of anything
else that comes out of this discussion. More specifically, I'm +1 with
targeting fixes for a 2.0.0 release which drops Hadoop 1 support, has
a new client API, has a minimum JDK of 1.7, at least. I'm also okay
with planning a 1.7.0 release with minor improvements over 1.6.0 (such
as ACCUMULO-1691), which are wire-incompatible with 1.6.0, but
represent a smaller changeset than 2.0.0, and a larger chanset than
1.6.0 bugfixes.

> Going a little deeper, I think we can also agree to some *general*
> guidelines about what these numbers mean, following what semantic versioning
> generally outlines based on what we've been trying to follow to date. The
> major (most-significant) number refers to releases in which very impactful
> changes were made to the codebase. The minor (middle) number refers to
> changes/improvements which are lesser in nature, but do not represent a
> major shift in how to use or administer Accumulo. The bugix
> (least-significant) number should *only* contain the least impactful change
> which address some breakage in the code.

+1 for these general guidelines.

>
>  - an important point that needs clarification is how we draw the line
> between which non-breaking changes can be put in a minor release and which
> must be introduced in a major release.

I think what would largely drive this decision is the impact on
testing/release that the non-breaking feature would have.

> Bugfix versions should not make any compatibility changes at all - fix the
> bug as it stands. Minor versions can introduce additions to the public API,
> storage or wire implementations, but they should be done in a backwards
> compatible way. Major versions can remove deprecated public API calls. Data
> compatibility can be met with some upgrade process instead of full
> compatibility with the previous versions. Wire compatibility does not need
> to be guaranteed across major versions if this is not possible.

+1

> Then, we can target major releases ~yearly, minor releases every few months,
> and bugfix releases in order of weeks based on severity. I would also
> suggest that we reduce our testing burden for bugfixes to be more focused on
> verifying that the bugs have been identified in tests and verified as fixed.

+1 for reduced testing burden for bugfixes, and for these general cycles.

>
> This got a lot longer than I wanted, so I'm sorry for that. Please feel free
> to suggest pruning to this list so we can have a discussion that nets some
> actual results.
>
> - Josh