You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cassandra.apache.org by Joshua McKenzie <jm...@apache.org> on 2021/09/01 17:28:03 UTC
Re: Defining which code changes target which release types
So to attempt to distill a set of heuristics from the discussion:
Patch release: Goal: maintain and improve stability
- Non-disruptive, non-API changing bugs are fine w/out consensus
- Bugs that change defaults or interfaces require consensus
- Improvements of any kind require consensus and must be small, in safe
areas, non default changing
Minor release: Goal: stable introduction of new features
- New features, always with feature flag (added; happy to drop if
controversial)
- No API breaking changes; additive API changes w/sane defaults acceptable
Major release: Goal: provide avenue to make breaking, non-backwards
compatible evolutionary changes
- API breaking changes
- Deprecating functionality
- Changing default behaviors
- Destructive / removing changes (params, features, etc)
- Major, well-tested and well documented component refactors (added; happy
to drop if controversial)
With the above structure, we may go from Major to patches to Major without
a Minor, or go from Minor to patches for awhile to Minors if we don't have
destructive changes.
Is that consistent w/what you're saying above Mick? If so - what do others
think?
~Josh
On Thu, Aug 26, 2021 at 3:37 PM Mick Semb Wever <mc...@apache.org> wrote:
> Throwing my two cents out…
>
> I'm keen for us to better adhere to SemVer rules (I've said this before).
> Specifically defining the difference between a Major and a Minor version,
> and defining the difference between a Minor and a Patch version (which in
> turn addresses your questions Josh). But this is rather tricky for us
> because it is a big and complex codebase and technology, one which focuses
> on availability at all times. The easiest way so far for me to think about
> this is through the eyes of the user, and the benefits we can give to them.
>
> Some examples of benefits that come to mind…
>
> * Major versions can be used to demarcate upgrade paths, e.g. you cannot
> jump over a major when upgrading.
> * A clearer expectation of when deprecated features and APIs will be
> removed, e.g. removal of deprecated features and API can only happen in a
> Major version.
> * Building on the previous, give the user some structure they can follow
> (helping to build intuition) of what might be in NEWS.txt and CHANGES.txt.
> * Safety in upgrades if we are stricter about can go into a Patch version.
>
> Then it is easy to think about the knock-on benefits from this to the dev
> community. Clarity about what versions may contain, and what degree of
> testing is involved in each upgrade, will encourage users to upgrade more
> often, which helps us. Worth noting that this is something we are already
> now asking of users, with annual releases only being supported for 3 years.
>
> Where it does get tricky is, we really cannot define this by jira issue
> types. Improvements and Bugs can have API changes, or incur significant
> behavioural changes (8099 comes to mind, which was an improvement).
> Features can also be clean additions onto existing SPIs, marked as
> alpha|beta and applied safely on any past version where that SPI exists.
> And Improvements are quite often operational fixes (a quick scan of
> CHNAGES.txt for any patch release shows this). This will be easier to
> formulate if we better define project components and work off them. Stuff
> like tools and client-side stuff, packaging, tests, and SPI
> implementations, can be dealt with quite differently. Such a component
> list will also help us ensure we get adequate reviews on patches and what
> they touch. I've half-baked started this here:
>
> https://cwiki.apache.org/confluence/display/CASSANDRA/%28draft%29+Project+Roles+and+Responsibilities
> ; but am struggling to find time to progress it, any help would be
> appreciated. (We have the detailed component list already in jira that we
> can re-used here as well.) I do doubt this will be but a helping guide,
> where the guidelines are ultimately defined on user-centric criteria.
>
> In lieu of such guidelines in place, I am in favour of the following for
> restricting what can go into a Patch version:
>
> - Only bugs that are non-disruptive and non-API breaking can go in without
> any discussion.
> - Other bugs (those that impact existing behaviour/interfaces) require a
> consensus.
> - Minor improvements, particularly in safe parts of the code, or are in
> essence an operational fix, that make sense to include require a consensus.
>
>
> Consensus in all of the above is according to our Code Contributions
> voting as described under our Project Governance, and it can be part of the
> discussion on the ticket, but also raised to the ML if doubts exist. This
> is a very light enforcement of the rules, allowing common sense to prevail,
> while letting the encouragement of discussion help organically define our
> guidelines over time.
>
> As to what makes trunk bump up to a Major version (if it hasn't already
> during the current release cycle), I'm for starting with some basics like
> "this patch breaks upgrades from the previous major" and "this is a
> significant underlying change to a subsystem", and again encouraging
> discussion as we figure it out.
>
> Finally, to the topic of "stable trunk" and feature flags. Feature flags
> should absolutely be used more, and new features should come out as
> disabled by default (as we have been doing more of). There is also some
> commonality here with SPI implementations in how they can be accepted into
> the code. On the topic of stable trunk: this is a new journey for us and
> it was tried (in a way) before with tick-tock without much luck. I do
> believe we can get it right this time, but I am scared of applying it to
> anything but trunk for the next year until we can validate its success. It
> is only to imagine if tick-tock had been applied to all release branches,
> help. Stable Trunk will no doubt need some fine-tuning as we go. So, I
> would like to leave our release branches alone as much as possible.
> Furthermore, my understanding of a stable trunk is that its success will be
> validated by how few patch versions we release, and how quiet our release
> branches end up being. This will have the benefit that our (very limited)
> reviewing capacity can focus more on trunk. And in turn, the more eyes
> that focus on trunk the greater success of a stable trunk.
>
>
>
> On Thu, 26 Aug 2021 at 19:37, Joshua McKenzie <jm...@apache.org>
> wrote:
>
> > The discussion came up on
> > https://issues.apache.org/jira/browse/CASSANDRA-16873 concerning where
> > we land different diff types and it was clear the conversation should
> > come to the ML and the outcome be codified for the future.
> >
> > Some useful pre-reading on the Release Lifecycle on our wiki:
> > https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle
> >
> > For context if it's not clear in the wiki article, the three release
> > types we have currently:
> > 1. Patch: N.M.x
> > 2. Minor: N.x
> > 3. Major: N
> >
> > Currently the wiki articulates the following under General Availability:
> > * Incremental fixes for post-GA issues identified will be included in
> > patchlevel releases.
> > * A new branch is created for the release with the new major version,
> > limiting any new feature addition to the new release branch, with new
> > feature development will continue to happen only on trunk.
> >
> > The question that came up discussing C-16873:
> > What qualifies as an "Incremental fix" for purposes of inclusion in a
> > patch release?
> > * Do all bugs qualify? Only some?
> > * What if a bugfix breaks an API? Where does that land?
> > * Do small improvements that are not API mutating qualify?
> > * If improvements, who decides? Lazy consensus w/2 committers?
> >
> > I'm personally pretty neutral; there's a cost-benefit to this and no
> > clear optimal answer to me at face value.
> >
> > Lastly, I also advocate for a clear definition of expectations on
> > these releases in the wiki, something like:
> >
> > Major: API changes, feature deprecation, protocol changes
> > Minor: New features, changes in default behavior
> > Patches: Whatever we decide in this thread
> >
> > What do we think?
> >
> > ~Josh
> >
>
Re: Defining which code changes target which release types
Posted by Brandon Williams <dr...@gmail.com>.
This looks good to me, though consensus requirements were dropped
after being noted for patch releases, was that intentional?
On Wed, Sep 1, 2021 at 12:28 PM Joshua McKenzie <jm...@apache.org> wrote:
>
> So to attempt to distill a set of heuristics from the discussion:
>
> Patch release: Goal: maintain and improve stability
> - Non-disruptive, non-API changing bugs are fine w/out consensus
> - Bugs that change defaults or interfaces require consensus
> - Improvements of any kind require consensus and must be small, in safe
> areas, non default changing
>
> Minor release: Goal: stable introduction of new features
> - New features, always with feature flag (added; happy to drop if
> controversial)
> - No API breaking changes; additive API changes w/sane defaults acceptable
>
> Major release: Goal: provide avenue to make breaking, non-backwards
> compatible evolutionary changes
> - API breaking changes
> - Deprecating functionality
> - Changing default behaviors
> - Destructive / removing changes (params, features, etc)
> - Major, well-tested and well documented component refactors (added; happy
> to drop if controversial)
>
> With the above structure, we may go from Major to patches to Major without
> a Minor, or go from Minor to patches for awhile to Minors if we don't have
> destructive changes.
>
> Is that consistent w/what you're saying above Mick? If so - what do others
> think?
>
> ~Josh
>
> On Thu, Aug 26, 2021 at 3:37 PM Mick Semb Wever <mc...@apache.org> wrote:
>
> > Throwing my two cents out…
> >
> > I'm keen for us to better adhere to SemVer rules (I've said this before).
> > Specifically defining the difference between a Major and a Minor version,
> > and defining the difference between a Minor and a Patch version (which in
> > turn addresses your questions Josh). But this is rather tricky for us
> > because it is a big and complex codebase and technology, one which focuses
> > on availability at all times. The easiest way so far for me to think about
> > this is through the eyes of the user, and the benefits we can give to them.
> >
> > Some examples of benefits that come to mind…
> >
> > * Major versions can be used to demarcate upgrade paths, e.g. you cannot
> > jump over a major when upgrading.
> > * A clearer expectation of when deprecated features and APIs will be
> > removed, e.g. removal of deprecated features and API can only happen in a
> > Major version.
> > * Building on the previous, give the user some structure they can follow
> > (helping to build intuition) of what might be in NEWS.txt and CHANGES.txt.
> > * Safety in upgrades if we are stricter about can go into a Patch version.
> >
> > Then it is easy to think about the knock-on benefits from this to the dev
> > community. Clarity about what versions may contain, and what degree of
> > testing is involved in each upgrade, will encourage users to upgrade more
> > often, which helps us. Worth noting that this is something we are already
> > now asking of users, with annual releases only being supported for 3 years.
> >
> > Where it does get tricky is, we really cannot define this by jira issue
> > types. Improvements and Bugs can have API changes, or incur significant
> > behavioural changes (8099 comes to mind, which was an improvement).
> > Features can also be clean additions onto existing SPIs, marked as
> > alpha|beta and applied safely on any past version where that SPI exists.
> > And Improvements are quite often operational fixes (a quick scan of
> > CHNAGES.txt for any patch release shows this). This will be easier to
> > formulate if we better define project components and work off them. Stuff
> > like tools and client-side stuff, packaging, tests, and SPI
> > implementations, can be dealt with quite differently. Such a component
> > list will also help us ensure we get adequate reviews on patches and what
> > they touch. I've half-baked started this here:
> >
> > https://cwiki.apache.org/confluence/display/CASSANDRA/%28draft%29+Project+Roles+and+Responsibilities
> > ; but am struggling to find time to progress it, any help would be
> > appreciated. (We have the detailed component list already in jira that we
> > can re-used here as well.) I do doubt this will be but a helping guide,
> > where the guidelines are ultimately defined on user-centric criteria.
> >
> > In lieu of such guidelines in place, I am in favour of the following for
> > restricting what can go into a Patch version:
> >
> > - Only bugs that are non-disruptive and non-API breaking can go in without
> > any discussion.
> > - Other bugs (those that impact existing behaviour/interfaces) require a
> > consensus.
> > - Minor improvements, particularly in safe parts of the code, or are in
> > essence an operational fix, that make sense to include require a consensus.
> >
> >
> > Consensus in all of the above is according to our Code Contributions
> > voting as described under our Project Governance, and it can be part of the
> > discussion on the ticket, but also raised to the ML if doubts exist. This
> > is a very light enforcement of the rules, allowing common sense to prevail,
> > while letting the encouragement of discussion help organically define our
> > guidelines over time.
> >
> > As to what makes trunk bump up to a Major version (if it hasn't already
> > during the current release cycle), I'm for starting with some basics like
> > "this patch breaks upgrades from the previous major" and "this is a
> > significant underlying change to a subsystem", and again encouraging
> > discussion as we figure it out.
> >
> > Finally, to the topic of "stable trunk" and feature flags. Feature flags
> > should absolutely be used more, and new features should come out as
> > disabled by default (as we have been doing more of). There is also some
> > commonality here with SPI implementations in how they can be accepted into
> > the code. On the topic of stable trunk: this is a new journey for us and
> > it was tried (in a way) before with tick-tock without much luck. I do
> > believe we can get it right this time, but I am scared of applying it to
> > anything but trunk for the next year until we can validate its success. It
> > is only to imagine if tick-tock had been applied to all release branches,
> > help. Stable Trunk will no doubt need some fine-tuning as we go. So, I
> > would like to leave our release branches alone as much as possible.
> > Furthermore, my understanding of a stable trunk is that its success will be
> > validated by how few patch versions we release, and how quiet our release
> > branches end up being. This will have the benefit that our (very limited)
> > reviewing capacity can focus more on trunk. And in turn, the more eyes
> > that focus on trunk the greater success of a stable trunk.
> >
> >
> >
> > On Thu, 26 Aug 2021 at 19:37, Joshua McKenzie <jm...@apache.org>
> > wrote:
> >
> > > The discussion came up on
> > > https://issues.apache.org/jira/browse/CASSANDRA-16873 concerning where
> > > we land different diff types and it was clear the conversation should
> > > come to the ML and the outcome be codified for the future.
> > >
> > > Some useful pre-reading on the Release Lifecycle on our wiki:
> > > https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle
> > >
> > > For context if it's not clear in the wiki article, the three release
> > > types we have currently:
> > > 1. Patch: N.M.x
> > > 2. Minor: N.x
> > > 3. Major: N
> > >
> > > Currently the wiki articulates the following under General Availability:
> > > * Incremental fixes for post-GA issues identified will be included in
> > > patchlevel releases.
> > > * A new branch is created for the release with the new major version,
> > > limiting any new feature addition to the new release branch, with new
> > > feature development will continue to happen only on trunk.
> > >
> > > The question that came up discussing C-16873:
> > > What qualifies as an "Incremental fix" for purposes of inclusion in a
> > > patch release?
> > > * Do all bugs qualify? Only some?
> > > * What if a bugfix breaks an API? Where does that land?
> > > * Do small improvements that are not API mutating qualify?
> > > * If improvements, who decides? Lazy consensus w/2 committers?
> > >
> > > I'm personally pretty neutral; there's a cost-benefit to this and no
> > > clear optimal answer to me at face value.
> > >
> > > Lastly, I also advocate for a clear definition of expectations on
> > > these releases in the wiki, something like:
> > >
> > > Major: API changes, feature deprecation, protocol changes
> > > Minor: New features, changes in default behavior
> > > Patches: Whatever we decide in this thread
> > >
> > > What do we think?
> > >
> > > ~Josh
> > >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org
Re: Defining which code changes target which release types
Posted by Joshua McKenzie <jm...@apache.org>.
>
> Where I think there becomes a grey area is on refactoring
ISTM we have two types of refactors:
1) Improving logic, tracking, state machines, etc (behavior, invasive)
2) Should be opaque to end user with zero change to behavior (hygiene, code
organization)
Feature flag on #1 as it's kind of a new feature / new way of operating,
deferred to major on #2 as we have a complex code-base where refactors
often have side-effects that should be more thoroughly vetted like a major?
Take as a loose set of ideas; potentially bad ones.
~Josh
On Mon, Sep 13, 2021 at 10:28 AM Ekaterina Dimitrova <e....@gmail.com>
wrote:
> I don’t think we can or we should cover every particular case but this is a
> good baseline/guideline and we should encourage people to hit the mailing
> list when there is uncertainty.
> My understanding is that this document will support also the initially
> mentioned one where I saw something that probably partially addresses
> David’s concerns but it is said as something tbd as far as I understand:
> https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle
>
> “Compatibility between major versions - Content we have so far based on the
> feedback - Developer community will try not to make any backwards
> incompatible changes as much as possible, except in extreme cases like to
> ensure correctness. Introducing a backward incompatibility change needs dev
> community approval via voting [voting open for 48 hours].”
>
> On Fri, 10 Sep 2021 at 14:29, David Capwell <dc...@apple.com.invalid>
> wrote:
>
> > > I believe that always having a feature flag for every new feature might
> > be
> > > too complicated in practice for different reasons.
> > > Some new features might be low impact like new nodetool commands or new
> > > virtual tables and adding flags for those might simply be extra
> > > complication for the developers and users.
> > > For some other features it might be simply too hard to hide them behind
> > > feature flags.
> > >
> > > Feature flag basically means "experimental" so it would be good when a
> > > feature flag is introduced to also have a clear plan on when and how
> the
> > > flag will be removed.
> > >
> > > I would personally limit the feature flag to significant new features.
> As
> > > those types of features now require a CEP, we could make the feature
> fag
> > > discussion part of the CEP discussion.
> > >
> > > What do you think?
> >
> > Personally I run with the idea we should default to “you need a feature
> > flag” and special case places which do not need; if we start with
> > “significant new feature” every feature will be argued that it isn’t
> > “significant enough” or that offering one would be “too complex”. I
> would
> > argue tables/nodetool act more like a feature flag so these examples
> > shouldn’t cause us to weaken the notation of a feature flag, as they do
> not
> > impact you unless you opt-into them…. which is what a feature flag does.
> >
> > > For some other features it might be simply too hard to hide them behind
> > > feature flags.
> >
> > In my experience these types of features get a feature flag after the
> fact
> > or warned to operators/users not to use them… While working on
> > CASSANDRA-16850 it was really annoying to support flags as I need to keep
> > track of state both at the coordinator and the replica to support this,
> and
> > at each check’s level (we also do not have a notion of a query context or
> > what actor is doing the action… which makes this even more painful to
> do);
> > this drastically increased my testing scope. This was still important to
> > do as after it is deployed it could cause a negative impact to operators
> or
> > users, so being able to act without code changes is important.
> >
> > Where I think there becomes a grey area is on refactoring… for example I
> > have put in a lot of work refactoring repair coordination and I plan to
> do
> > a lot more… do I support falling back to old logic or old behavior? In
> > CASSANDRA-16909 I document a lot of places which are buggy and have shown
> > to cause production issues… is the “fix” actually a “new feature” (fun
> > example that happens on prod from time to time… we drop the merkle tree
> and
> > hang forever… we could make this recoverable but is that a feature or a
> bug
> > fix)? Should this go into a prior release?
> >
> >
> > > On Sep 10, 2021, at 9:25 AM, Joshua McKenzie <jm...@apache.org>
> > wrote:
> > >
> > > I put together a gdoc documenting what was in this thread - should be
> > open
> > > to comment for everyone:
> > >
> >
> https://docs.google.com/document/d/1LhCNcbuhtqTkv_aKx1TQUgWEcq022fsAZs1C_oOxEJw/edit
> > >
> > > I'll let this thread sit to early next week and assuming no major
> > concerns
> > > we'll get that into either the wiki or the site or both.
> > >
> > > Thanks everyone for the feedback!
> > >
> > > ~Josh
> > >
> > > On Thu, Sep 2, 2021 at 9:57 AM Joshua McKenzie <jm...@apache.org>
> > wrote:
> > >
> > >> Feature flag basically means "experimental"
> > >>
> > >> I'm thinking of feature flags more as giving the power to operators to
> > >> decide what they do and don't allow users of the database access to.
> > Even
> > >> if a feature is very stable and non-experimental, it can have negative
> > >> effects on other use-cases on a shared cluster, be incompatible with
> the
> > >> underlying execution environment, be outside compliance policies of an
> > >> organization, require greater expertise to use correctly, etc.
> > >>
> > >> That said, I 100% agree w/you on the "limit it to significant new
> > >> features". I don't think feature flagging nodetool commands makes a
> lot
> > of
> > >> sense. :)
> > >>
> > >> Adding it to the CEP template as something to yes/no on would be a
> > simple
> > >> clarification for this. +1
> > >>
> > >> ~Josh
> > >>
> > >>
> > >> On Thu, Sep 2, 2021 at 3:14 AM Benjamin Lerer <bl...@apache.org>
> > wrote:
> > >>
> > >>>>
> > >>>> - New features, always with feature flag (added; happy to drop if
> > >>>> controversial)
> > >>>
> > >>>
> > >>> I believe that always having a feature flag for every new feature
> > might be
> > >>> too complicated in practice for different reasons.
> > >>> Some new features might be low impact like new nodetool commands or
> new
> > >>> virtual tables and adding flags for those might simply be extra
> > >>> complication for the developers and users.
> > >>> For some other features it might be simply too hard to hide them
> behind
> > >>> feature flags.
> > >>>
> > >>> Feature flag basically means "experimental" so it would be good when
> a
> > >>> feature flag is introduced to also have a clear plan on when and how
> > the
> > >>> flag will be removed.
> > >>>
> > >>> I would personally limit the feature flag to significant new
> features.
> > As
> > >>> those types of features now require a CEP, we could make the feature
> > fag
> > >>> discussion part of the CEP discussion.
> > >>>
> > >>> What do you think?
> > >>>
> > >>>
> > >>>
> > >>> Le jeu. 2 sept. 2021 à 08:41, Mick Semb Wever <mc...@apache.org> a
> > écrit :
> > >>>
> > >>>>>
> > >>>>>
> > >>>>> There's certainly a lot of complexity in a lot of the systems here,
> > no
> > >>>>> denying that, so maybe we treat the topic of API changes as "here's
> > >>> loose
> > >>>>> guidelines (destructive vs. additive w/sane defaults, etc) but plan
> > to
> > >>>> take
> > >>>>> it case-by-case" and be a bit more prescriptive on the "where do
> bug
> > >>>> fixes
> > >>>>> vs. improvements vs. new features go and why"?
> > >>>>>
> > >>>>
> > >>>>
> > >>>> Agree.
> > >>>>
> > >>>
> > >>
> >
> >
>
Re: Defining which code changes target which release types
Posted by Ekaterina Dimitrova <e....@gmail.com>.
I don’t think we can or we should cover every particular case but this is a
good baseline/guideline and we should encourage people to hit the mailing
list when there is uncertainty.
My understanding is that this document will support also the initially
mentioned one where I saw something that probably partially addresses
David’s concerns but it is said as something tbd as far as I understand:
https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle
“Compatibility between major versions - Content we have so far based on the
feedback - Developer community will try not to make any backwards
incompatible changes as much as possible, except in extreme cases like to
ensure correctness. Introducing a backward incompatibility change needs dev
community approval via voting [voting open for 48 hours].”
On Fri, 10 Sep 2021 at 14:29, David Capwell <dc...@apple.com.invalid>
wrote:
> > I believe that always having a feature flag for every new feature might
> be
> > too complicated in practice for different reasons.
> > Some new features might be low impact like new nodetool commands or new
> > virtual tables and adding flags for those might simply be extra
> > complication for the developers and users.
> > For some other features it might be simply too hard to hide them behind
> > feature flags.
> >
> > Feature flag basically means "experimental" so it would be good when a
> > feature flag is introduced to also have a clear plan on when and how the
> > flag will be removed.
> >
> > I would personally limit the feature flag to significant new features. As
> > those types of features now require a CEP, we could make the feature fag
> > discussion part of the CEP discussion.
> >
> > What do you think?
>
> Personally I run with the idea we should default to “you need a feature
> flag” and special case places which do not need; if we start with
> “significant new feature” every feature will be argued that it isn’t
> “significant enough” or that offering one would be “too complex”. I would
> argue tables/nodetool act more like a feature flag so these examples
> shouldn’t cause us to weaken the notation of a feature flag, as they do not
> impact you unless you opt-into them…. which is what a feature flag does.
>
> > For some other features it might be simply too hard to hide them behind
> > feature flags.
>
> In my experience these types of features get a feature flag after the fact
> or warned to operators/users not to use them… While working on
> CASSANDRA-16850 it was really annoying to support flags as I need to keep
> track of state both at the coordinator and the replica to support this, and
> at each check’s level (we also do not have a notion of a query context or
> what actor is doing the action… which makes this even more painful to do);
> this drastically increased my testing scope. This was still important to
> do as after it is deployed it could cause a negative impact to operators or
> users, so being able to act without code changes is important.
>
> Where I think there becomes a grey area is on refactoring… for example I
> have put in a lot of work refactoring repair coordination and I plan to do
> a lot more… do I support falling back to old logic or old behavior? In
> CASSANDRA-16909 I document a lot of places which are buggy and have shown
> to cause production issues… is the “fix” actually a “new feature” (fun
> example that happens on prod from time to time… we drop the merkle tree and
> hang forever… we could make this recoverable but is that a feature or a bug
> fix)? Should this go into a prior release?
>
>
> > On Sep 10, 2021, at 9:25 AM, Joshua McKenzie <jm...@apache.org>
> wrote:
> >
> > I put together a gdoc documenting what was in this thread - should be
> open
> > to comment for everyone:
> >
> https://docs.google.com/document/d/1LhCNcbuhtqTkv_aKx1TQUgWEcq022fsAZs1C_oOxEJw/edit
> >
> > I'll let this thread sit to early next week and assuming no major
> concerns
> > we'll get that into either the wiki or the site or both.
> >
> > Thanks everyone for the feedback!
> >
> > ~Josh
> >
> > On Thu, Sep 2, 2021 at 9:57 AM Joshua McKenzie <jm...@apache.org>
> wrote:
> >
> >> Feature flag basically means "experimental"
> >>
> >> I'm thinking of feature flags more as giving the power to operators to
> >> decide what they do and don't allow users of the database access to.
> Even
> >> if a feature is very stable and non-experimental, it can have negative
> >> effects on other use-cases on a shared cluster, be incompatible with the
> >> underlying execution environment, be outside compliance policies of an
> >> organization, require greater expertise to use correctly, etc.
> >>
> >> That said, I 100% agree w/you on the "limit it to significant new
> >> features". I don't think feature flagging nodetool commands makes a lot
> of
> >> sense. :)
> >>
> >> Adding it to the CEP template as something to yes/no on would be a
> simple
> >> clarification for this. +1
> >>
> >> ~Josh
> >>
> >>
> >> On Thu, Sep 2, 2021 at 3:14 AM Benjamin Lerer <bl...@apache.org>
> wrote:
> >>
> >>>>
> >>>> - New features, always with feature flag (added; happy to drop if
> >>>> controversial)
> >>>
> >>>
> >>> I believe that always having a feature flag for every new feature
> might be
> >>> too complicated in practice for different reasons.
> >>> Some new features might be low impact like new nodetool commands or new
> >>> virtual tables and adding flags for those might simply be extra
> >>> complication for the developers and users.
> >>> For some other features it might be simply too hard to hide them behind
> >>> feature flags.
> >>>
> >>> Feature flag basically means "experimental" so it would be good when a
> >>> feature flag is introduced to also have a clear plan on when and how
> the
> >>> flag will be removed.
> >>>
> >>> I would personally limit the feature flag to significant new features.
> As
> >>> those types of features now require a CEP, we could make the feature
> fag
> >>> discussion part of the CEP discussion.
> >>>
> >>> What do you think?
> >>>
> >>>
> >>>
> >>> Le jeu. 2 sept. 2021 à 08:41, Mick Semb Wever <mc...@apache.org> a
> écrit :
> >>>
> >>>>>
> >>>>>
> >>>>> There's certainly a lot of complexity in a lot of the systems here,
> no
> >>>>> denying that, so maybe we treat the topic of API changes as "here's
> >>> loose
> >>>>> guidelines (destructive vs. additive w/sane defaults, etc) but plan
> to
> >>>> take
> >>>>> it case-by-case" and be a bit more prescriptive on the "where do bug
> >>>> fixes
> >>>>> vs. improvements vs. new features go and why"?
> >>>>>
> >>>>
> >>>>
> >>>> Agree.
> >>>>
> >>>
> >>
>
>
Re: Defining which code changes target which release types
Posted by David Capwell <dc...@apple.com.INVALID>.
> I believe that always having a feature flag for every new feature might be
> too complicated in practice for different reasons.
> Some new features might be low impact like new nodetool commands or new
> virtual tables and adding flags for those might simply be extra
> complication for the developers and users.
> For some other features it might be simply too hard to hide them behind
> feature flags.
>
> Feature flag basically means "experimental" so it would be good when a
> feature flag is introduced to also have a clear plan on when and how the
> flag will be removed.
>
> I would personally limit the feature flag to significant new features. As
> those types of features now require a CEP, we could make the feature fag
> discussion part of the CEP discussion.
>
> What do you think?
Personally I run with the idea we should default to “you need a feature flag” and special case places which do not need; if we start with “significant new feature” every feature will be argued that it isn’t “significant enough” or that offering one would be “too complex”. I would argue tables/nodetool act more like a feature flag so these examples shouldn’t cause us to weaken the notation of a feature flag, as they do not impact you unless you opt-into them…. which is what a feature flag does.
> For some other features it might be simply too hard to hide them behind
> feature flags.
In my experience these types of features get a feature flag after the fact or warned to operators/users not to use them… While working on CASSANDRA-16850 it was really annoying to support flags as I need to keep track of state both at the coordinator and the replica to support this, and at each check’s level (we also do not have a notion of a query context or what actor is doing the action… which makes this even more painful to do); this drastically increased my testing scope. This was still important to do as after it is deployed it could cause a negative impact to operators or users, so being able to act without code changes is important.
Where I think there becomes a grey area is on refactoring… for example I have put in a lot of work refactoring repair coordination and I plan to do a lot more… do I support falling back to old logic or old behavior? In CASSANDRA-16909 I document a lot of places which are buggy and have shown to cause production issues… is the “fix” actually a “new feature” (fun example that happens on prod from time to time… we drop the merkle tree and hang forever… we could make this recoverable but is that a feature or a bug fix)? Should this go into a prior release?
> On Sep 10, 2021, at 9:25 AM, Joshua McKenzie <jm...@apache.org> wrote:
>
> I put together a gdoc documenting what was in this thread - should be open
> to comment for everyone:
> https://docs.google.com/document/d/1LhCNcbuhtqTkv_aKx1TQUgWEcq022fsAZs1C_oOxEJw/edit
>
> I'll let this thread sit to early next week and assuming no major concerns
> we'll get that into either the wiki or the site or both.
>
> Thanks everyone for the feedback!
>
> ~Josh
>
> On Thu, Sep 2, 2021 at 9:57 AM Joshua McKenzie <jm...@apache.org> wrote:
>
>> Feature flag basically means "experimental"
>>
>> I'm thinking of feature flags more as giving the power to operators to
>> decide what they do and don't allow users of the database access to. Even
>> if a feature is very stable and non-experimental, it can have negative
>> effects on other use-cases on a shared cluster, be incompatible with the
>> underlying execution environment, be outside compliance policies of an
>> organization, require greater expertise to use correctly, etc.
>>
>> That said, I 100% agree w/you on the "limit it to significant new
>> features". I don't think feature flagging nodetool commands makes a lot of
>> sense. :)
>>
>> Adding it to the CEP template as something to yes/no on would be a simple
>> clarification for this. +1
>>
>> ~Josh
>>
>>
>> On Thu, Sep 2, 2021 at 3:14 AM Benjamin Lerer <bl...@apache.org> wrote:
>>
>>>>
>>>> - New features, always with feature flag (added; happy to drop if
>>>> controversial)
>>>
>>>
>>> I believe that always having a feature flag for every new feature might be
>>> too complicated in practice for different reasons.
>>> Some new features might be low impact like new nodetool commands or new
>>> virtual tables and adding flags for those might simply be extra
>>> complication for the developers and users.
>>> For some other features it might be simply too hard to hide them behind
>>> feature flags.
>>>
>>> Feature flag basically means "experimental" so it would be good when a
>>> feature flag is introduced to also have a clear plan on when and how the
>>> flag will be removed.
>>>
>>> I would personally limit the feature flag to significant new features. As
>>> those types of features now require a CEP, we could make the feature fag
>>> discussion part of the CEP discussion.
>>>
>>> What do you think?
>>>
>>>
>>>
>>> Le jeu. 2 sept. 2021 à 08:41, Mick Semb Wever <mc...@apache.org> a écrit :
>>>
>>>>>
>>>>>
>>>>> There's certainly a lot of complexity in a lot of the systems here, no
>>>>> denying that, so maybe we treat the topic of API changes as "here's
>>> loose
>>>>> guidelines (destructive vs. additive w/sane defaults, etc) but plan to
>>>> take
>>>>> it case-by-case" and be a bit more prescriptive on the "where do bug
>>>> fixes
>>>>> vs. improvements vs. new features go and why"?
>>>>>
>>>>
>>>>
>>>> Agree.
>>>>
>>>
>>
Re: Defining which code changes target which release types
Posted by Joshua McKenzie <jm...@apache.org>.
I put together a gdoc documenting what was in this thread - should be open
to comment for everyone:
https://docs.google.com/document/d/1LhCNcbuhtqTkv_aKx1TQUgWEcq022fsAZs1C_oOxEJw/edit
I'll let this thread sit to early next week and assuming no major concerns
we'll get that into either the wiki or the site or both.
Thanks everyone for the feedback!
~Josh
On Thu, Sep 2, 2021 at 9:57 AM Joshua McKenzie <jm...@apache.org> wrote:
> Feature flag basically means "experimental"
>
> I'm thinking of feature flags more as giving the power to operators to
> decide what they do and don't allow users of the database access to. Even
> if a feature is very stable and non-experimental, it can have negative
> effects on other use-cases on a shared cluster, be incompatible with the
> underlying execution environment, be outside compliance policies of an
> organization, require greater expertise to use correctly, etc.
>
> That said, I 100% agree w/you on the "limit it to significant new
> features". I don't think feature flagging nodetool commands makes a lot of
> sense. :)
>
> Adding it to the CEP template as something to yes/no on would be a simple
> clarification for this. +1
>
> ~Josh
>
>
> On Thu, Sep 2, 2021 at 3:14 AM Benjamin Lerer <bl...@apache.org> wrote:
>
>> >
>> > - New features, always with feature flag (added; happy to drop if
>> > controversial)
>>
>>
>> I believe that always having a feature flag for every new feature might be
>> too complicated in practice for different reasons.
>> Some new features might be low impact like new nodetool commands or new
>> virtual tables and adding flags for those might simply be extra
>> complication for the developers and users.
>> For some other features it might be simply too hard to hide them behind
>> feature flags.
>>
>> Feature flag basically means "experimental" so it would be good when a
>> feature flag is introduced to also have a clear plan on when and how the
>> flag will be removed.
>>
>> I would personally limit the feature flag to significant new features. As
>> those types of features now require a CEP, we could make the feature fag
>> discussion part of the CEP discussion.
>>
>> What do you think?
>>
>>
>>
>> Le jeu. 2 sept. 2021 à 08:41, Mick Semb Wever <mc...@apache.org> a écrit :
>>
>> > >
>> > >
>> > > There's certainly a lot of complexity in a lot of the systems here, no
>> > > denying that, so maybe we treat the topic of API changes as "here's
>> loose
>> > > guidelines (destructive vs. additive w/sane defaults, etc) but plan to
>> > take
>> > > it case-by-case" and be a bit more prescriptive on the "where do bug
>> > fixes
>> > > vs. improvements vs. new features go and why"?
>> > >
>> >
>> >
>> > Agree.
>> >
>>
>
Re: Defining which code changes target which release types
Posted by Joshua McKenzie <jm...@apache.org>.
>
> Feature flag basically means "experimental"
I'm thinking of feature flags more as giving the power to operators to
decide what they do and don't allow users of the database access to. Even
if a feature is very stable and non-experimental, it can have negative
effects on other use-cases on a shared cluster, be incompatible with the
underlying execution environment, be outside compliance policies of an
organization, require greater expertise to use correctly, etc.
That said, I 100% agree w/you on the "limit it to significant new
features". I don't think feature flagging nodetool commands makes a lot of
sense. :)
Adding it to the CEP template as something to yes/no on would be a simple
clarification for this. +1
~Josh
On Thu, Sep 2, 2021 at 3:14 AM Benjamin Lerer <bl...@apache.org> wrote:
> >
> > - New features, always with feature flag (added; happy to drop if
> > controversial)
>
>
> I believe that always having a feature flag for every new feature might be
> too complicated in practice for different reasons.
> Some new features might be low impact like new nodetool commands or new
> virtual tables and adding flags for those might simply be extra
> complication for the developers and users.
> For some other features it might be simply too hard to hide them behind
> feature flags.
>
> Feature flag basically means "experimental" so it would be good when a
> feature flag is introduced to also have a clear plan on when and how the
> flag will be removed.
>
> I would personally limit the feature flag to significant new features. As
> those types of features now require a CEP, we could make the feature fag
> discussion part of the CEP discussion.
>
> What do you think?
>
>
>
> Le jeu. 2 sept. 2021 à 08:41, Mick Semb Wever <mc...@apache.org> a écrit :
>
> > >
> > >
> > > There's certainly a lot of complexity in a lot of the systems here, no
> > > denying that, so maybe we treat the topic of API changes as "here's
> loose
> > > guidelines (destructive vs. additive w/sane defaults, etc) but plan to
> > take
> > > it case-by-case" and be a bit more prescriptive on the "where do bug
> > fixes
> > > vs. improvements vs. new features go and why"?
> > >
> >
> >
> > Agree.
> >
>
Re: Defining which code changes target which release types
Posted by Benjamin Lerer <bl...@apache.org>.
>
> - New features, always with feature flag (added; happy to drop if
> controversial)
I believe that always having a feature flag for every new feature might be
too complicated in practice for different reasons.
Some new features might be low impact like new nodetool commands or new
virtual tables and adding flags for those might simply be extra
complication for the developers and users.
For some other features it might be simply too hard to hide them behind
feature flags.
Feature flag basically means "experimental" so it would be good when a
feature flag is introduced to also have a clear plan on when and how the
flag will be removed.
I would personally limit the feature flag to significant new features. As
those types of features now require a CEP, we could make the feature fag
discussion part of the CEP discussion.
What do you think?
Le jeu. 2 sept. 2021 à 08:41, Mick Semb Wever <mc...@apache.org> a écrit :
> >
> >
> > There's certainly a lot of complexity in a lot of the systems here, no
> > denying that, so maybe we treat the topic of API changes as "here's loose
> > guidelines (destructive vs. additive w/sane defaults, etc) but plan to
> take
> > it case-by-case" and be a bit more prescriptive on the "where do bug
> fixes
> > vs. improvements vs. new features go and why"?
> >
>
>
> Agree.
>
Re: Defining which code changes target which release types
Posted by Mick Semb Wever <mc...@apache.org>.
>
>
> There's certainly a lot of complexity in a lot of the systems here, no
> denying that, so maybe we treat the topic of API changes as "here's loose
> guidelines (destructive vs. additive w/sane defaults, etc) but plan to take
> it case-by-case" and be a bit more prescriptive on the "where do bug fixes
> vs. improvements vs. new features go and why"?
>
Agree.
Re: Defining which code changes target which release types
Posted by Joshua McKenzie <jm...@apache.org>.
>
> I would move "Deprecating functionality" to be under Minor version.
Good call. I'll get this into a wiki article once we hash it out on thread
here.
consensus requirements were dropped after being noted for patch releases,
> was that intentional?
Hm. With how our project works we should probably drop the verbiage
entirely since everything requires at least a lazy consensus? Definitely
not intended as a delta.
I think it's easy for us to
> look at it solely from a code and accuracy perspective, but that it would
> be more valuable to take a user-centric (WIIFM) approach and to be as
> pragmatic as possible
Agree with the general sentiment; there's a balance we need to strike here
between highly clear elucidation of rules and heuristics and leaving people
"wiggle room" to navigate outside those guidelines. We have quite a few
active committers and even more community members now (and hope to keep
growing that!), and for a newcomer on a project with a question of "what
fixversion should I put change X in", an answer of "it depends; check with
a committer" is something we should do our best to avoid and instead
empower folks as best we can.
There's certainly a lot of complexity in a lot of the systems here, no
denying that, so maybe we treat the topic of API changes as "here's loose
guidelines (destructive vs. additive w/sane defaults, etc) but plan to take
it case-by-case" and be a bit more prescriptive on the "where do bug fixes
vs. improvements vs. new features go and why"?
wdyt?
~Josh
On Wed, Sep 1, 2021 at 5:04 PM Mick Semb Wever <mc...@apache.org> wrote:
> > So to attempt to distill a set of heuristics from the discussion:
> >
> > Patch release: Goal: maintain and improve stability
> > - Non-disruptive, non-API changing bugs are fine w/out consensus
> > - Bugs that change defaults or interfaces require consensus
> > - Improvements of any kind require consensus and must be small, in safe
> > areas, non default changing
> >
> > Minor release: Goal: stable introduction of new features
> > - New features, always with feature flag (added; happy to drop if
> > controversial)
> > - No API breaking changes; additive API changes w/sane defaults
> acceptable
> >
> > Major release: Goal: provide avenue to make breaking, non-backwards
> > compatible evolutionary changes
> > - API breaking changes
> > - Deprecating functionality
> > - Changing default behaviors
> > - Destructive / removing changes (params, features, etc)
> > - Major, well-tested and well documented component refactors (added;
> happy
> > to drop if controversial)
> >
> > With the above structure, we may go from Major to patches to Major
> without
> > a Minor, or go from Minor to patches for awhile to Minors if we don't
> have
> > destructive changes.
> >
> > Is that consistent w/what you're saying above Mick? If so - what do
> others
> > think?
> >
>
>
> Yes, thanks Josh.
>
> I do think there's a lot of ambiguity and discussion to be had around what
> constitutes a Major change. And, to stress it, I think it's easy for us to
> look at it solely from a code and accuracy perspective, but that it would
> be more valuable to take a user-centric (WIIFM) approach and to be as
> pragmatic as possible. e.g. let's take the discussions whether certain
> interfaces/APIs are user-facing enough to signal a major change to all
> users (index, triggers, etc, come to mind), bearing in mind our mix-version
> upgrade ability deals with a lot (and the planned work raised to deal more
> easily with rolling back a started upgrade), us having a NEWS.txt, and that
> small API stuff is often changing in minor versions anyway.
>
> I would move "Deprecating functionality" to be under Minor version.
> Something could be deprecated in 4.1, and then removed in 5.0.
>
Re: Defining which code changes target which release types
Posted by Mick Semb Wever <mc...@apache.org>.
> So to attempt to distill a set of heuristics from the discussion:
>
> Patch release: Goal: maintain and improve stability
> - Non-disruptive, non-API changing bugs are fine w/out consensus
> - Bugs that change defaults or interfaces require consensus
> - Improvements of any kind require consensus and must be small, in safe
> areas, non default changing
>
> Minor release: Goal: stable introduction of new features
> - New features, always with feature flag (added; happy to drop if
> controversial)
> - No API breaking changes; additive API changes w/sane defaults acceptable
>
> Major release: Goal: provide avenue to make breaking, non-backwards
> compatible evolutionary changes
> - API breaking changes
> - Deprecating functionality
> - Changing default behaviors
> - Destructive / removing changes (params, features, etc)
> - Major, well-tested and well documented component refactors (added; happy
> to drop if controversial)
>
> With the above structure, we may go from Major to patches to Major without
> a Minor, or go from Minor to patches for awhile to Minors if we don't have
> destructive changes.
>
> Is that consistent w/what you're saying above Mick? If so - what do others
> think?
>
Yes, thanks Josh.
I do think there's a lot of ambiguity and discussion to be had around what
constitutes a Major change. And, to stress it, I think it's easy for us to
look at it solely from a code and accuracy perspective, but that it would
be more valuable to take a user-centric (WIIFM) approach and to be as
pragmatic as possible. e.g. let's take the discussions whether certain
interfaces/APIs are user-facing enough to signal a major change to all
users (index, triggers, etc, come to mind), bearing in mind our mix-version
upgrade ability deals with a lot (and the planned work raised to deal more
easily with rolling back a started upgrade), us having a NEWS.txt, and that
small API stuff is often changing in minor versions anyway.
I would move "Deprecating functionality" to be under Minor version.
Something could be deprecated in 4.1, and then removed in 5.0.