You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Andrew Purtell <ap...@apache.org> on 2015/07/16 17:58:51 UTC
0.98 patch acceptance criteria discussion
Hi devs,
I'd like to call your attention to an interesting and important discussion
taking place on the tail of HBASE-12596. It starts from here:
https://issues.apache.org/jira/browse/HBASE-12596?focusedCommentId=14628295&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14628295
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: 0.98 patch acceptance criteria discussion
Posted by Andrew Purtell <an...@gmail.com>.
> So "newer" has some extra meaning. In practice it seems to mean the a
> release is "newer" if it's from a minor branch that was branched at a
> later date. That's a hard thing for a user to know. How do we meet
> the user's expectation with that definition? We need to make it
> visible. Give users a way to understand what releases are "newer"
> than others.
For that we could possibly adopt something suitable or build a plugin for doc and site generation that extracts release tags from git during build and renders them into a timeline.
> On Jul 17, 2015, at 8:26 AM, Dave Latham <la...@davelink.net> wrote:
>
> My thoughts on the patches and versions issue. Apologies as this
> overlaps some of what others said, since I got the notes in order.
>
> All of this stems from trying to make life easier for our users and
> meet their expectations. Here's what I think are the relevant user
> expectations:
> - Upgrading to a new patch release should not break my app.
> - Therefore patch releases only include bug fixes to maintain
> stability (semver).
> - Upgrading to a new minor release should be able to happen without
> downtime to my system.
> - Therefore minor releases should be rolling upgradeable and
> preserve most forms of compatibility (semver - see the table in the
> HBase book for details).
> - Upgrading to a "newer" release should not lose any features or bug
> fixes that I already had.
> - Therefore ???
>
> This last point seems to be the question that started this discussion.
> Is that expectation reasonable and if so, what policies do we need to
> adopt regarding commits, branches, and releases to meet it?
>
> I think it's reasonable depending on what "newer" means. If we say
> that a version with a greater major number is always newer than a
> version with a lesser major number (and likewise for minors) then we
> essentially have to put all releases in strict sequence. We've never
> had that expectation. For example, if someone upgraded from 0.94.27
> to 0.96.0 or from 0.98.14 to 1.0.0 they would lose some features and
> many bug fixes - hopefully not a surprise. Similarly for minor
> upgrades, it should not be a surprise if someone upgrades (someday)
> from 1.0.22 to 1.1.0 that they may lose many bug fixes.
>
> So "newer" has some extra meaning. In practice it seems to mean the a
> release is "newer" if it's from a minor branch that was branched at a
> later date. That's a hard thing for a user to know. How do we meet
> the user's expectation with that definition? We need to make it
> visible. Give users a way to understand what releases are "newer"
> than others. Tell users they should only do upgrades to "newer"
> releases (should those be the only upgrades supported?). Sean
> suggested each release indicate the minimum minor release of the next
> major which seems helpful. Showing a release date history or minor
> branch history somewhere may help (does it already exist?). We could
> keep some sort of graph showing the relationships.
>
> What does this mean for commits and branches? Here's the approach
> that I think is about what I've observed so far:
> - Commit incompatible changes to master only
> - Commit compatible features to master first, then major, non-EOL
> branches in reverse order (master, branch-2, branch-1, 0.98, 0.94) as
> far back as accepted by RMs. In other words, *don't commit a feature
> to a major branch unless master and all greater major, non-EOL
> branches already have it.*
> - Commit bug fixes to master first, then major, non-EOL branches in
> reverse order as accepted by RMs, then minor, non-EOL branches in
> reverse "newer" order as accepted by RMs. In other words *don't
> commit a bug fix to a minor branch unless master, the related major
> branch, and all greater, "newer", minor, non-EOL branches already have
> it.* I.e. if 2.2 is "newer" than 1.3, be sure that 2.2 has the fix
> before 1.3 gets it.
>
> There will likely be some exceptions made by RM judgement and RMs will
> have to deal with the notions of whether "newer" lines block older
> ones from getting commits or older ones force "newer" ones. But we
> should keep in mind that whenever we make those exceptions we may
> break user expectations.
>
> Another thing that this means is that if minor releases from a new
> major version come at a significantly slower pace than minor releases
> from a previous major version that users could be stuck on a previous
> major release, wanting to upgrade to a newer major release but waiting
> for a "newer" minor release on that major line to happen. We should
> try to avoid that situation by not waiting too long for "newer" minor
> releases on greater major releases after updating older major
> releases.
>
> Dave
>
> On Fri, Jul 17, 2015 at 7:31 AM, Sean Busbey <bu...@cloudera.com> wrote:
>>> I think the extra statement we have to make is that only the latest minor
>> version of the next major branch
>>> is guaranteed have all the improvements of the previous major branch.Or
>> phrased in other words:
>>> Improvements that are not bug fixes will only go into the x.y.0 minor
>> version, but not (by default anyway,
>>> the RM should use good judgment) into any existing minor version (and
>> thus not in a patch version > 0)
>>
>> I think that's a fine statement.
>>
>> I don't know if it's worth doing, but we could say in each minor release's
>> notes what the minimum minor release in the next major version is needed to
>> get a superset of functionality. This would only really impact Andrew right
>> now since all the 1.y.0 versions would be "2.0".
>>
>>
>>> On Fri, Jul 17, 2015 at 8:47 AM, lars hofhansl <la...@apache.org> wrote:
>>>
>>> Thanks Andy.
>>> I think the gist of the discussion boils down to this:We generally have
>>> two goals: (1) follow semver from 1.0.0 onward and (2) avoid losing
>>> features/improvements when upgrading from an older version to a newer one.
>>> Turns out these two are conflicting unless we follow certain additional
>>> policies.
>>> The issue at hand was a performance improvement that we added to 0.98,
>>> 1.3.0, and 2.0.0, but not 1.0.x, 1.1.x, and 1.2.x (x >= 1 in all cases)So
>>> when somebody would upgrade from 0.98 to (say) 1.1.7 (if/when that's out)
>>> that improvement would "silently" be lost.
>>> I think the extra statement we have to make is that only the latest minor
>>> version of the next major branch is guaranteed have all the improvements of
>>> the previous major branch.Or phrased in other words: Improvements that are
>>> not bug fixes will only go into the x.y.0 minor version, but not (by
>>> default anyway, the RM should use good judgment) into any existing minor
>>> version (and thus not in a patch version > 0)
>>>
>>> If that's OK with everybody we can just state that and move on (and I'll
>>> shut up :) ).
>>> -- Lars
>>>
>>> From: Andrew Purtell <ap...@apache.org>
>>> To: "dev@hbase.apache.org" <de...@hbase.apache.org>
>>> Sent: Thursday, July 16, 2015 8:58 AM
>>> Subject: 0.98 patch acceptance criteria discussion
>>>
>>> Hi devs,
>>>
>>> I'd like to call your attention to an interesting and important discussion
>>> taking place on the tail of HBASE-12596. It starts from here:
>>>
>>> https://issues.apache.org/jira/browse/HBASE-12596?focusedCommentId=14628295&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14628295
>>>
>>> --
>>> Best regards,
>>>
>>> - Andy
>>>
>>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>>> (via Tom White)
>>
>>
>>
>> --
>> Sean
Re: 0.98 patch acceptance criteria discussion
Posted by Dave Latham <la...@davelink.net>.
My thoughts on the patches and versions issue. Apologies as this
overlaps some of what others said, since I got the notes in order.
All of this stems from trying to make life easier for our users and
meet their expectations. Here's what I think are the relevant user
expectations:
- Upgrading to a new patch release should not break my app.
- Therefore patch releases only include bug fixes to maintain
stability (semver).
- Upgrading to a new minor release should be able to happen without
downtime to my system.
- Therefore minor releases should be rolling upgradeable and
preserve most forms of compatibility (semver - see the table in the
HBase book for details).
- Upgrading to a "newer" release should not lose any features or bug
fixes that I already had.
- Therefore ???
This last point seems to be the question that started this discussion.
Is that expectation reasonable and if so, what policies do we need to
adopt regarding commits, branches, and releases to meet it?
I think it's reasonable depending on what "newer" means. If we say
that a version with a greater major number is always newer than a
version with a lesser major number (and likewise for minors) then we
essentially have to put all releases in strict sequence. We've never
had that expectation. For example, if someone upgraded from 0.94.27
to 0.96.0 or from 0.98.14 to 1.0.0 they would lose some features and
many bug fixes - hopefully not a surprise. Similarly for minor
upgrades, it should not be a surprise if someone upgrades (someday)
from 1.0.22 to 1.1.0 that they may lose many bug fixes.
So "newer" has some extra meaning. In practice it seems to mean the a
release is "newer" if it's from a minor branch that was branched at a
later date. That's a hard thing for a user to know. How do we meet
the user's expectation with that definition? We need to make it
visible. Give users a way to understand what releases are "newer"
than others. Tell users they should only do upgrades to "newer"
releases (should those be the only upgrades supported?). Sean
suggested each release indicate the minimum minor release of the next
major which seems helpful. Showing a release date history or minor
branch history somewhere may help (does it already exist?). We could
keep some sort of graph showing the relationships.
What does this mean for commits and branches? Here's the approach
that I think is about what I've observed so far:
- Commit incompatible changes to master only
- Commit compatible features to master first, then major, non-EOL
branches in reverse order (master, branch-2, branch-1, 0.98, 0.94) as
far back as accepted by RMs. In other words, *don't commit a feature
to a major branch unless master and all greater major, non-EOL
branches already have it.*
- Commit bug fixes to master first, then major, non-EOL branches in
reverse order as accepted by RMs, then minor, non-EOL branches in
reverse "newer" order as accepted by RMs. In other words *don't
commit a bug fix to a minor branch unless master, the related major
branch, and all greater, "newer", minor, non-EOL branches already have
it.* I.e. if 2.2 is "newer" than 1.3, be sure that 2.2 has the fix
before 1.3 gets it.
There will likely be some exceptions made by RM judgement and RMs will
have to deal with the notions of whether "newer" lines block older
ones from getting commits or older ones force "newer" ones. But we
should keep in mind that whenever we make those exceptions we may
break user expectations.
Another thing that this means is that if minor releases from a new
major version come at a significantly slower pace than minor releases
from a previous major version that users could be stuck on a previous
major release, wanting to upgrade to a newer major release but waiting
for a "newer" minor release on that major line to happen. We should
try to avoid that situation by not waiting too long for "newer" minor
releases on greater major releases after updating older major
releases.
Dave
On Fri, Jul 17, 2015 at 7:31 AM, Sean Busbey <bu...@cloudera.com> wrote:
>> I think the extra statement we have to make is that only the latest minor
> version of the next major branch
>> is guaranteed have all the improvements of the previous major branch.Or
> phrased in other words:
>> Improvements that are not bug fixes will only go into the x.y.0 minor
> version, but not (by default anyway,
>> the RM should use good judgment) into any existing minor version (and
> thus not in a patch version > 0)
>
> I think that's a fine statement.
>
> I don't know if it's worth doing, but we could say in each minor release's
> notes what the minimum minor release in the next major version is needed to
> get a superset of functionality. This would only really impact Andrew right
> now since all the 1.y.0 versions would be "2.0".
>
>
> On Fri, Jul 17, 2015 at 8:47 AM, lars hofhansl <la...@apache.org> wrote:
>
>> Thanks Andy.
>> I think the gist of the discussion boils down to this:We generally have
>> two goals: (1) follow semver from 1.0.0 onward and (2) avoid losing
>> features/improvements when upgrading from an older version to a newer one.
>> Turns out these two are conflicting unless we follow certain additional
>> policies.
>> The issue at hand was a performance improvement that we added to 0.98,
>> 1.3.0, and 2.0.0, but not 1.0.x, 1.1.x, and 1.2.x (x >= 1 in all cases)So
>> when somebody would upgrade from 0.98 to (say) 1.1.7 (if/when that's out)
>> that improvement would "silently" be lost.
>> I think the extra statement we have to make is that only the latest minor
>> version of the next major branch is guaranteed have all the improvements of
>> the previous major branch.Or phrased in other words: Improvements that are
>> not bug fixes will only go into the x.y.0 minor version, but not (by
>> default anyway, the RM should use good judgment) into any existing minor
>> version (and thus not in a patch version > 0)
>>
>> If that's OK with everybody we can just state that and move on (and I'll
>> shut up :) ).
>> -- Lars
>>
>> From: Andrew Purtell <ap...@apache.org>
>> To: "dev@hbase.apache.org" <de...@hbase.apache.org>
>> Sent: Thursday, July 16, 2015 8:58 AM
>> Subject: 0.98 patch acceptance criteria discussion
>>
>> Hi devs,
>>
>> I'd like to call your attention to an interesting and important discussion
>> taking place on the tail of HBASE-12596. It starts from here:
>>
>> https://issues.apache.org/jira/browse/HBASE-12596?focusedCommentId=14628295&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14628295
>>
>> --
>> Best regards,
>>
>> - Andy
>>
>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> (via Tom White)
>>
>>
>>
>>
>
>
>
> --
> Sean
Re: 0.98 patch acceptance criteria discussion
Posted by Sean Busbey <bu...@cloudera.com>.
> I think the extra statement we have to make is that only the latest minor
version of the next major branch
> is guaranteed have all the improvements of the previous major branch.Or
phrased in other words:
> Improvements that are not bug fixes will only go into the x.y.0 minor
version, but not (by default anyway,
> the RM should use good judgment) into any existing minor version (and
thus not in a patch version > 0)
I think that's a fine statement.
I don't know if it's worth doing, but we could say in each minor release's
notes what the minimum minor release in the next major version is needed to
get a superset of functionality. This would only really impact Andrew right
now since all the 1.y.0 versions would be "2.0".
On Fri, Jul 17, 2015 at 8:47 AM, lars hofhansl <la...@apache.org> wrote:
> Thanks Andy.
> I think the gist of the discussion boils down to this:We generally have
> two goals: (1) follow semver from 1.0.0 onward and (2) avoid losing
> features/improvements when upgrading from an older version to a newer one.
> Turns out these two are conflicting unless we follow certain additional
> policies.
> The issue at hand was a performance improvement that we added to 0.98,
> 1.3.0, and 2.0.0, but not 1.0.x, 1.1.x, and 1.2.x (x >= 1 in all cases)So
> when somebody would upgrade from 0.98 to (say) 1.1.7 (if/when that's out)
> that improvement would "silently" be lost.
> I think the extra statement we have to make is that only the latest minor
> version of the next major branch is guaranteed have all the improvements of
> the previous major branch.Or phrased in other words: Improvements that are
> not bug fixes will only go into the x.y.0 minor version, but not (by
> default anyway, the RM should use good judgment) into any existing minor
> version (and thus not in a patch version > 0)
>
> If that's OK with everybody we can just state that and move on (and I'll
> shut up :) ).
> -- Lars
>
> From: Andrew Purtell <ap...@apache.org>
> To: "dev@hbase.apache.org" <de...@hbase.apache.org>
> Sent: Thursday, July 16, 2015 8:58 AM
> Subject: 0.98 patch acceptance criteria discussion
>
> Hi devs,
>
> I'd like to call your attention to an interesting and important discussion
> taking place on the tail of HBASE-12596. It starts from here:
>
> https://issues.apache.org/jira/browse/HBASE-12596?focusedCommentId=14628295&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14628295
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
>
>
>
--
Sean
Re: 0.98 patch acceptance criteria discussion
Posted by lars hofhansl <la...@apache.org>.
Thanks Andy.
I think the gist of the discussion boils down to this:We generally have two goals: (1) follow semver from 1.0.0 onward and (2) avoid losing features/improvements when upgrading from an older version to a newer one.
Turns out these two are conflicting unless we follow certain additional policies.
The issue at hand was a performance improvement that we added to 0.98, 1.3.0, and 2.0.0, but not 1.0.x, 1.1.x, and 1.2.x (x >= 1 in all cases)So when somebody would upgrade from 0.98 to (say) 1.1.7 (if/when that's out) that improvement would "silently" be lost.
I think the extra statement we have to make is that only the latest minor version of the next major branch is guaranteed have all the improvements of the previous major branch.Or phrased in other words: Improvements that are not bug fixes will only go into the x.y.0 minor version, but not (by default anyway, the RM should use good judgment) into any existing minor version (and thus not in a patch version > 0)
If that's OK with everybody we can just state that and move on (and I'll shut up :) ).
-- Lars
From: Andrew Purtell <ap...@apache.org>
To: "dev@hbase.apache.org" <de...@hbase.apache.org>
Sent: Thursday, July 16, 2015 8:58 AM
Subject: 0.98 patch acceptance criteria discussion
Hi devs,
I'd like to call your attention to an interesting and important discussion
taking place on the tail of HBASE-12596. It starts from here:
https://issues.apache.org/jira/browse/HBASE-12596?focusedCommentId=14628295&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14628295
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)