You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jclouds.apache.org by Andrew Phillips <ap...@qrmedia.com> on 2014/07/01 02:43:18 UTC
Re: major release cadence
> 1. Better use of semver
+1. Minors should be X.Y.0 and X.Y.Z should indeed be used only for
patch versions.
> 2. Deprecation of code and removal of deprecated code will only
> happen on major releases (e.g. MAJOR.0)
Whilst I can see how this might be easier for us to formulate as a
rule, this would mean that it might take *two* major releases before
we can effectively get rid of something, if I understand the proposal
correctly? E.g. if we've just released 2.0.0 (i.e. master is at
3.0.0-SNAPSHOT) and we want to deprecate something, the deprecation
has to wait until 3.0.0 and the removal happens in 4.0.0.
If we're still planning on releasing majors roughly every 6-9 months,
this would mean a waiting period of over a year. That seems a little
long to me? Obviously, if we allow deprecation in a minor version, we
shouldn't deprecate it in the minor version right before a major
release and then immediately rip out the code. But I think we can
avoid that, and can also formulate that in the guide if necessary.
> A concern about the jclouds-labs section [3]. By having that in
> there, I guarantee you users will extrapolate and apply that to
> *all* labs repos whether we like it or not.
Fair point. I can see two immediate alternatives: renaming the other
labs repos to remove the word "labs" (perhaps too painful), or going
through and adding @Beta to all providers and APIs in labs (more
accurate because then we just have to define what @Beta means).
> The old methods, classes etc. can be removed from master in the next
> major release
So that would mean they stay around on master too until the next major
release is cut (i.e. the last round of commits would be a bunch of
removals of deprecated stuff). Is there any specific reason not to
remove them from master - which to me represents "the bleeding edge",
i.e. the current state of the project - immediately? The reason I'd
prefer this, if possible, is that we can at least try to treat master
as "free of cruft that we are about to remove".
To your other question:
"jclouds aims to not promote a method or class out of beta - or to
discard it - by the major release ''after'' next (i.e. A.D.0 if the
current release branch is A.B.x). < I'm confused by this. I'm not sure
what the intent is here. Could this be replaced by..."
The intent was to somehow clarify to users that we don't intend to
declare APIs or providers as @Beta and then leave them in that state
forever. I was wondering if it would make sense to try to codify for
ourselves, and our users, that @Beta indeed indicates a limited trial
period with a clear "go/no-go" decision after a specific amount of time.
We can certainly take that out if we feel it doesn't make sense.
Also made a couple of very minor changes [1], just corrections, no
"content changes", I hope.
Regards
ap
[1]
https://wiki.apache.org/jclouds/DeprecationAndBetaPolicy?action=diff&rev2=4&rev1=3
Re: major release cadence
Posted by Andrew Gaul <ga...@apache.org>.
On Thu, Jul 03, 2014 at 01:14:14AM +0200, Andrew Phillips wrote:
> It's been insightful for me to see two quite different opinions on
> what we should be aiming for. I wonder whether we can get some data
> on how/if our current rate of change is causing problems for
> customers, to help guide this further?
Let's look at this another way: jclouds has a huge compatibility surface
area just from its REST API against its many providers. Maintaining
this is a large but valuable task which we have all contributed to over
the years. I look at adding new providers, improving existing ones, and
broadening the portable interfaces as our core challenges in the past
and in the future. Complicating this with non-core tasks such as strong
backwards compatibility guarantees, supporting many Java versions, and
even Clojure support make our core tasks more difficult, especially when
these tasks have dubious user value. Further we face challenges with a
byzantine repository layout, racy unit tests, broken integration tests,
and a backlog of pull requests. We should *streamline* our processes to
allow us to deliver on our core tasks, not *burden* our development with
nice-to-haves.
--
Andrew Gaul
http://gaul.org/
Re: major release cadence
Posted by Andrew Phillips <ap...@qrmedia.com>.
> Most of our users won't know our deprecation policy further than
> these annotations so doing more seems pointless.
I agree that annotations are likely the primary communication channel
for deprecations. I don't think the intention of the proposal is to
change that, just to make some notes about what we as a development
community think is a good balance between giving users reasonable
predictability in terms of preparing for breaking changes, and us
enough flexibility to fix broken stuff.
It's been insightful for me to see two quite different opinions on
what we should be aiming for. I wonder whether we can get some data on
how/if our current rate of change is causing problems for customers,
to help guide this further?
> jclouds can go this way but we should expect that every branch from
> master will increment the major version and just define minor
> releases away.
I guess that's the idea, yes. But in any case I don't think we need to
decide anything on that front until 2.0. Whether we say "the next
major version is 3.0", or whether we call it 2.0.0 and say "the next
major version is 2.1", seems like a bridge we can cross when we get to
it ;-)
ap
Re: major release cadence
Posted by Andrew Gaul <ga...@apache.org>.
On Thu, Jul 03, 2014 at 11:24:43PM +0000, Everett Toews wrote:
> So only 6 weeks to deal with a breaking change?
In the idealized case of someone upgrading from the last 1.7.x release
to the first 1.8.x release, they would have only six weeks notice. Or
they stay on their last 1.7.x release for however long they like. Note
that breaking changes are not introduced within 1.7.x; applications
continue to work from 1.7.1 to 1.7.2 to 1.7.3. Note that we actually
have a hole in our deprecation story today as we allow deprecations to
1.7.x branch without requiring a release of it.
> If the motivation for making breaking changes in 6 week cycles is because of this one thing then let’s tear this one off like a bandaid and get it done for 2.0. It should be dealt with so we can move forward in a stable and predictable way.
I believe you misunderstood my example, I hope the above clarifies it.
Note that I am describing the *existing* system, not some hypothetical
one.
> I disagree with this completely. Semver is well known and well understood throughout the software industry. Even the C-suite knows that a major release can contain breaking changes and impact their business. Heck, even the mainstream populace knows the accessories for version 1 or something won’t necessarily work with version 2.
Being semver compatible is just one of several versioning choices. Many
software, including older and existing jclouds releases, use different
schemes. Personally I have little interest in the versioning scheme but
believe we use of semver should have some justification for breaking
with the existing scheme. I do want to repeat my previous sentence and
add emphasis:
Whether we define and *communicate* that X.0.0 allows breakage or X.Y.0
matters to a vanishingly small number of users...
We did not communicate this well previously and might have been doing
more work than our users (and some developers?) expected. Perhaps we
should draw attention to this compatibility in our release notes for
1.7.x+1 releases?
> We’re not just moving numbers around. We’re communicating valuable information to our users that makes jclouds a predictable platform for them to build their applications upon.
When you look through the Maven dependencies of jclouds or some local
application do you know which ones are semver and which are not? You
might expect that upgrading from 2.x to 3.x introduces breakages, but
not really with x.7 to x.8 and personally I run integration tests on any
remotely interesting dependency before upgrading. I have been bitten
too many times with even micro version changes, and suspect others have
as well.
> We can’t expect that every branch from master will increment the major version. Then what’s the purpose of a major release cadence?
Sorry that this repeats from my earlier mail, but I still cannot
reconcile how we increment the major number against when we branch.
Perhaps you could expand on some of your examples?
--
Andrew Gaul
http://gaul.org/
Re: major release cadence
Posted by Everett Toews <ev...@RACKSPACE.COM>.
On Jul 2, 2014, at 5:58 PM, Andrew Gaul <ga...@apache.org> wrote:
> The current system seems fine: giving users compatibility across micro
> versions, e.g., 1.7.x -> 1.7.x + 1, and giving users a heads up at least
> one micro release via @Deprecated annotations.
So only 6 weeks to deal with a breaking change?
> Most of our users won't
> know our deprecation policy further than these annotations so doing more
> seems pointless.
Agreed. All deprecation annotations should include what the deprecated thing has been replaced with and when it will be removed.
> Further some of the interfaces that jclouds provides
> today are incorrect, e.g., Payload implements InputSupplier.getInput but
> opts out the repeatable guarantee via isRepeatable then provides
> confusing semantics to getInput and release. Fixing these while
> providing best-effort compatibility is a large challenge; doing so with
> strong compatibility might not be possible other than the strawman of
> percolating new Payload2 interfaces throughout jclouds.
If the motivation for making breaking changes in 6 week cycles is because of this one thing then let’s tear this one off like a bandaid and get it done for 2.0. It should be dealt with so we can move forward in a stable and predictable way.
> Semver is an academic distinction. Whether we define and communicate
> that X.0.0 allows breakage or X.Y.0 matters to a vanishingly small
> number of users: their application will have breakage when upgrading in
> either case. jclouds can go this way but we should expect that every
> branch from master will increment the major version and just define
> minor releases away. Although then we are basically back to where we
> started just with numbers moved around!
I disagree with this completely. Semver is well known and well understood throughout the software industry. Even the C-suite knows that a major release can contain breaking changes and impact their business. Heck, even the mainstream populace knows the accessories for version 1 or something won’t necessarily work with version 2.
We’re not just moving numbers around. We’re communicating valuable information to our users that makes jclouds a predictable platform for them to build their applications upon.
We can’t expect that every branch from master will increment the major version. Then what’s the purpose of a major release cadence?
Regards,
Everett
Re: major release cadence
Posted by Andrew Gaul <ga...@apache.org>.
On Wed, Jul 02, 2014 at 11:28:33PM +0200, Andrew Phillips wrote:
> >Strong promises of compatibility will discourage many (breaking)
> >improvements, big and small. I worry about this a lot given the
> >existing quirky jclouds API and the large about of suboptimal API use I
> >observe in Stack Overflow questions and jclouds-user mails.
>
> Thanks a lot for this, Gaul - I think this is exactly the kind of
> discussion (i.e. which kind of a tradeoff are we looking for) we
> want to be having here!
>
> As far as I'm concerned, everything suggested so far is exactly that
> - a suggestion - so if we feel it's not the right tradeoff to be
> making at this time, let's try to come up with something else.
>
> What would you suggestion be?
The current system seems fine: giving users compatibility across micro
versions, e.g., 1.7.x -> 1.7.x + 1, and giving users a heads up at least
one micro release via @Deprecated annotations. Most of our users won't
know our deprecation policy further than these annotations so doing more
seems pointless. Further some of the interfaces that jclouds provides
today are incorrect, e.g., Payload implements InputSupplier.getInput but
opts out the repeatable guarantee via isRepeatable then provides
confusing semantics to getInput and release. Fixing these while
providing best-effort compatibility is a large challenge; doing so with
strong compatibility might not be possible other than the strawman of
percolating new Payload2 interfaces throughout jclouds.
> ap
>
> PS: Also agree that the first step towards semver is "moving a
> number around". But even if that doesn't affect any other
> development practice, it still seems like a step in the right
> direction..? ;-)
Semver is an academic distinction. Whether we define and communicate
that X.0.0 allows breakage or X.Y.0 matters to a vanishingly small
number of users: their application will have breakage when upgrading in
either case. jclouds can go this way but we should expect that every
branch from master will increment the major version and just define
minor releases away. Although then we are basically back to where we
started just with numbers moved around!
--
Andrew Gaul
http://gaul.org/
Re: major release cadence
Posted by Andrew Phillips <ap...@qrmedia.com>.
> Basically I’m looking for a guaranteed *minimum* amount of time for
> users to be able to react to breaking changes. Considering the long
> release cycles in place at many organizations, I think 6 months is
> the absolute minimum.
Fully agree that 6 months as a minimum makes sense. Perhaps we can
make *that* the guideline? A deprecated piece of code will be removed
in a major release that is at least 6 months out from the moment of
deprecation?
> I’m okay not including such an indication. I think it’s okay to take
> things out of Beta in minor releases if we feel they’re ready. It’s
> an addition and not a change so no danger of breaking anyone.
Works for me, too. That would then be something like:
"Beta methods and classes can be promoted to non-beta status in major
or minor releases. Removing beta functionality will only happen in
major releases (e.g. MAJOR.0)"
?
ap
Re: major release cadence
Posted by Andrew Phillips <ap...@qrmedia.com>.
> Strong promises of compatibility will discourage many (breaking)
> improvements, big and small. I worry about this a lot given the
> existing quirky jclouds API and the large about of suboptimal API use I
> observe in Stack Overflow questions and jclouds-user mails.
Thanks a lot for this, Gaul - I think this is exactly the kind of
discussion (i.e. which kind of a tradeoff are we looking for) we want
to be having here!
As far as I'm concerned, everything suggested so far is exactly that -
a suggestion - so if we feel it's not the right tradeoff to be making
at this time, let's try to come up with something else.
What would you suggestion be?
ap
PS: Also agree that the first step towards semver is "moving a number
around". But even if that doesn't affect any other development
practice, it still seems like a step in the right direction..? ;-)
RE: major release cadence
Posted by Martin Smith <ma...@RACKSPACE.COM>.
Hi folks,
I agree that changing from 1.major.minor to major.minor would be confusing for users. I don't really have a horse in this race, but if I were trying to make a move from 1.major.minor to major.minor, I'd do it like the Java versions, so that users might have a chance at being familiar with it. For example,
old scheme -> new scheme
1.7.0 -> 7.0.0
1.8.0 -> 8.0.0
etc
So if the project *must* renumber closer to semver (dropping the leading "1." in front of every version), then at least users will be familiar with it that are familiar with Java version numbers (though unlike Oracle, I'd eventually try to drop the 1.x designations).
I don't really have strong feelings about the actual change. Just something I was thinking about while reading the thread. Hope it's a useful observation!
- Martin Smith
martin@mbs3.org
________________________________________
From: Andrew Gaul [gaul@apache.org]
Sent: Monday, August 25, 2014 11:00 AM
To: dev@jclouds.apache.org
Subject: Re: major release cadence
On Sun, Aug 24, 2014 at 02:28:18PM +0200, Andrew Phillips wrote:
> >Given the lack of positive feedback to semantic versioning
>
> Sorry, a bit behind in catching up with emails. To me, the rough
> timing of releases and decision on breaking changes are indeed the
> main two points, which seem fine to me.
Prior to this thread we introduced breaking changes in major releases,
as defined by 1.7 -> 1.8, and released minor releases, as defined by
1.7.0 -> 1.7.1, every 6-8 weeks. Earlier in this thread we agreed to
major releases every 6 months, again defined as 1.7 -> 1.8.
> I'm also fine with semantic versioning, but agree that we can move
> forward on the former two points and leave this for later if
> necessary. I don't immediately see any obvious areas where the
> transition cost would be that big.
>
> Could you elaborate on the challenges you see there, Andrew G?
If we were starting a new project, I would not care as much for which
versioning scheme we chose. However, we have an existing practice which
developers and users already understand and I see benefits to continuing
for consistency, outweighing the purported and modest benefits of other
schemes. Specifically, users expect today that X.Y+1.0 upgrades can
introduce breaking changes and delay upgrading to these releases.
Changing this requires messaging to our users, similar to other
transitions with source control, bug tracking, and mailing lists.
Further other projects have changed versioning schemes, such as Firefox
and Java, which caused user confusion without discernible benefit.
--
Andrew Gaul
http://gaul.org/
Re: major release cadence
Posted by Andrew Gaul <ga...@apache.org>.
On Sun, Aug 24, 2014 at 02:28:18PM +0200, Andrew Phillips wrote:
> >Given the lack of positive feedback to semantic versioning
>
> Sorry, a bit behind in catching up with emails. To me, the rough
> timing of releases and decision on breaking changes are indeed the
> main two points, which seem fine to me.
Prior to this thread we introduced breaking changes in major releases,
as defined by 1.7 -> 1.8, and released minor releases, as defined by
1.7.0 -> 1.7.1, every 6-8 weeks. Earlier in this thread we agreed to
major releases every 6 months, again defined as 1.7 -> 1.8.
> I'm also fine with semantic versioning, but agree that we can move
> forward on the former two points and leave this for later if
> necessary. I don't immediately see any obvious areas where the
> transition cost would be that big.
>
> Could you elaborate on the challenges you see there, Andrew G?
If we were starting a new project, I would not care as much for which
versioning scheme we chose. However, we have an existing practice which
developers and users already understand and I see benefits to continuing
for consistency, outweighing the purported and modest benefits of other
schemes. Specifically, users expect today that X.Y+1.0 upgrades can
introduce breaking changes and delay upgrading to these releases.
Changing this requires messaging to our users, similar to other
transitions with source control, bug tracking, and mailing lists.
Further other projects have changed versioning schemes, such as Firefox
and Java, which caused user confusion without discernible benefit.
--
Andrew Gaul
http://gaul.org/
Re: major release cadence
Posted by Andrew Phillips <ap...@qrmedia.com>.
> Given the lack of positive feedback to semantic versioning
Sorry, a bit behind in catching up with emails. To me, the rough
timing of releases and decision on breaking changes are indeed the
main two points, which seem fine to me.
I'm also fine with semantic versioning, but agree that we can move
forward on the former two points and leave this for later if
necessary. I don't immediately see any obvious areas where the
transition cost would be that big.
Could you elaborate on the challenges you see there, Andrew G?
Regards
ap
Re: major release cadence
Posted by Andrew Phillips <ap...@qrmedia.com>.
> Please familiarize yourself with existing jclouds versioning scheme,
> which I will summarize.
Thanks for putting this together, Gaul! Maybe it's just me, but I get
a feeling we're actually having two discussions at once here, and it
might make sense to split them:
1. We can make life easier for our users (and new committers!) by
better documenting our deprecation policy and release cadence.
2. Does our current deprecation policy and versioning scheme make
sense, and do we want to change them?
From what I can see, the former discussion is not really contentious.
I'm pretty sure users will not automatically *look* at the deprecation
policy, and I think good Javadocs are much more critical here. But
it's something we can refer to in release notes and point to for new
committers, and so I think it would be useful.
Publishing a release cadence also seems like a no-brainer: users have
repeatedly asked for this, and I think minor releases at 6-week
intervals seemed acceptable.
The second topic is more contentious. My take is roughly that the
debate is essentially "benefit in terms of predictability of sticking
with the current scheme" vs. "benefit of moving to a different scheme
that is probably more widely adopted elsewhere". Or, looked at a
slightly different way, "predictability for existing users vs.
predictability for new users".
I don't recall seeing any direct user requests or questions in this
area, so I personally find it harder to come down on either side of
this debate at this point.
My suggestion would be:
1) We document the release cadence and deprecation policy as it is
*right now*. We have to decide on a place for that, but the Wiki seems
as good a place as any. We already have [1] and would then need to add
a Release Cadence/Roadmap page. For the version numbers in the
Roadmap, we would stick with the current scheme or simply say "MAJOR"
and "MINOR" with something like "(provisionally 2.1.0)" or so next to
the versions.
2) We continue the discussion about *modifying* the deprecation policy
and versioning schemes as needed:
a) removing deprecated code in MAJOR+2 vs. MAJOR+1 as currently
b) moving from MAJOR = X.Y.0, MINOR = X.Y.Z to MAJOR = X.0.0, MINOR =
X.Y.0, PATCH = X.Y.Z
Regards
ap
[1] https://wiki.apache.org/jclouds/DeprecationAndBetaPolicy
Re: major release cadence
Posted by Andrew Gaul <ga...@apache.org>.
On Sun, Aug 24, 2014 at 12:22:42PM +0000, Everett Toews wrote:
> You’ve completely lost me at this point. You’ve even introduced new jargon. What’s a micro release?
>
> And I have no idea what "major" means to you. To the majority of people in the software industry a major release means incrementing the first number in the version. It seems you’ve overloaded the term major as its already commonly used in semver. But semver aside, what does “major” mean to you in this context?
>
> Can you please provide an example of the version numbers for the next 6 months?
Please familiarize yourself with existing jclouds versioning scheme,
which I will summarize. We have released jclouds major versions
branched from master as 1.6, 1.7, and 1.8. We have also released
jclouds minor versions from release branches as 1.7.1, 1.7.2, and 1.7.3.
Major versions introduce breaking changes while minor versions do not.
We have not incremented the leading version number in almost four
years[1][2] but despite this we refer to 1.7 as a major release and
1.7.1 as a minor release. Whatever you believe about the "majority of
software engineers" does not apply to our practices and no one has
expressed confusion on this matter, other than that which you have
introduced in this thread.
To plan future releases, look back at the existing releases. I have
summarized all the ASF releases with a few pre-ASF releases for context.
Consider both the major and minor releases, including cadence:
1.6.0/ 27-Apr-2013
1.6.1-incubating/ 20-Jun-2013 7 weeks
1.6.2-incubating/ 27-Aug-2013 9 weeks
1.6.3/ 30-Nov-2013 12 weeks
1.7.0/ 23-Dec-2013 4 weeks
1.7.1/ 11-Feb-2014 7 weeks
1.7.2/ 15-Apr-2014 8 weeks
1.7.3/ 30-May-2014 6 weeks
1.8.0/ 05-Aug-2014 9 weeks
Also consider only the major releases, including cadence:
1.5.0/ 17-Sep-2012
1.6.0/ 27-Apr-2013 7 months
1.7.0/ 23-Dec-2013 6 months
1.8.0/ 05-Aug-2014 8 months
Following the existing 6 weeks minor release cadence and including the 6
month major release cadence we agreed to earlier in this thread, we can
plan the following major and minor releases:
1.8.1/ 15-Sep-2014 6 weeks
1.8.2/ 01-Nov-2014 6 weeks
1.8.3/ 15-Dec-2014 6 weeks
2.0.0/ 01-Feb-2014 6 weeks
Note that we only deviated in increasing the leading version number in
2.0 to address your suggestion for the new Java 7 dependency. Had we
not followed this suggestion, we would have released jclouds 1.9
instead, avoiding your confusion about "major" version numbers entirely.
[1] http://repo1.maven.org/maven2/org/jclouds/jclouds-project/
[2] http://repo1.maven.org/maven2/org/apache/jclouds/jclouds-project/
--
Andrew Gaul
http://gaul.org/
Re: major release cadence
Posted by Everett Toews <ev...@RACKSPACE.COM>.
On Aug 23, 2014, at 4:08 PM, Andrew Gaul <ga...@apache.org> wrote:
> You explicitly used semvar-style version numbers. Is this an implicit
> part of your proposal?
Nope.
> We already agreed to a six-month major release
> cadence and already do not break compatibility between minor (micro)
> releases. What exactly do you propose other than using these new-style
> version numbers?
You’ve completely lost me at this point. You’ve even introduced new jargon. What’s a micro release?
And I have no idea what "major" means to you. To the majority of people in the software industry a major release means incrementing the first number in the version. It seems you’ve overloaded the term major as its already commonly used in semver. But semver aside, what does “major” mean to you in this context?
Can you please provide an example of the version numbers for the next 6 months?
Everett
Re: major release cadence
Posted by Andrew Gaul <ga...@apache.org>.
On Sat, Aug 23, 2014 at 08:41:53PM +0000, Everett Toews wrote:
> On Aug 23, 2014, at 1:23 PM, Andrew Gaul <ga...@apache.org> wrote:
> > Given the lack of positive feedback to semantic versioning perhaps we
> > should table your proposal until you can convince other developers of
> > its benefits? I do not exactly oppose this idea, but I see its
> > advantages as modest and smaller then the transition costs.
>
> Note that neither of the points below having anything to do with semver whatsoever.
>
> In fact, the first point is the original purpose of this thread that you started, choosing a major release cadence. The second point was quoted directly from you, regarding minor point upgrades.
>
> This is the common ground we agree on and I thought we could move forward on these 2 points alone for the sake of progress. Do you have any arguments for or against the 2 points below?
You explicitly used semvar-style version numbers. Is this an implicit
part of your proposal? We already agreed to a six-month major release
cadence and already do not break compatibility between minor (micro)
releases. What exactly do you propose other than using these new-style
version numbers?
--
Andrew Gaul
http://gaul.org/
Re: major release cadence
Posted by Everett Toews <ev...@RACKSPACE.COM>.
On Aug 23, 2014, at 1:23 PM, Andrew Gaul <ga...@apache.org> wrote:
> Given the lack of positive feedback to semantic versioning perhaps we
> should table your proposal until you can convince other developers of
> its benefits? I do not exactly oppose this idea, but I see its
> advantages as modest and smaller then the transition costs.
Note that neither of the points below having anything to do with semver whatsoever.
In fact, the first point is the original purpose of this thread that you started, choosing a major release cadence. The second point was quoted directly from you, regarding minor point upgrades.
This is the common ground we agree on and I thought we could move forward on these 2 points alone for the sake of progress. Do you have any arguments for or against the 2 points below?
> On Tue, Aug 19, 2014 at 10:36:58PM +0000, Everett Toews wrote:
>> Let’s see what we can agree on and move forward on those points.
>>
>> 1. 6 month major release cadence.
>>
>> That would mean the next releases are roughly:
>>
>> * 2.0.0 mid-Sept
>> * 2.1.0 early-Nov
>> * 2.2.0 late-Dec
>> * 2.3.0 mid-Feb
>> * 3.0.0 early-Mar
>>
>> 2. Minor point upgrades do not introduce breaking changes. e.g. "users upgrading from an earlier 3.x should be able to use a later
>> 3.x release without source modification.” (pasted from Gaul’s previous email below).
>>
>> Can everyone agree on those two points?
>>
>> Regards,
>> Everett
Re: major release cadence
Posted by Andrew Gaul <ga...@apache.org>.
Given the lack of positive feedback to semantic versioning perhaps we
should table your proposal until you can convince other developers of
its benefits? I do not exactly oppose this idea, but I see its
advantages as modest and smaller then the transition costs.
On Tue, Aug 19, 2014 at 10:36:58PM +0000, Everett Toews wrote:
> Let’s see what we can agree on and move forward on those points.
>
> 1. 6 month major release cadence.
>
> That would mean the next releases are roughly:
>
> * 2.0.0 mid-Sept
> * 2.1.0 early-Nov
> * 2.2.0 late-Dec
> * 2.3.0 mid-Feb
> * 3.0.0 early-Mar
>
> 2. Minor point upgrades do not introduce breaking changes. e.g. "users upgrading from an earlier 3.x should be able to use a later
> 3.x release without source modification.” (pasted from Gaul’s previous email below).
>
> Can everyone agree on those two points?
>
> Regards,
> Everett
>
>
>
> On Jul 11, 2014, at 11:45 PM, Everett Toews <ev...@rackspace.com> wrote:
>
> > On Jul 7, 2014, at 6:53 PM, Andrew Gaul <ga...@apache.org> wrote:
> >
> >> Sorry, I still do not understand and cannot reconcile these paragraphs
> >> against earlier mails. Do we branch from master every six months or
> >> not?
> >
> > Yes, we branch every 6 months.
> >
> >> Do we increase the major version every six months?
> >
> > Yes, we increment the major version every 6 months.
> >
> >> From your
> >> example, it appears that we create a 3.x branch from master and continue
> >> releasing 3.x from there.
> >
> > I think I see the confusion here. Trying to use text to communicate this was a bad idea. Maybe this diagram clears it up. It’s really not that much different than what we do now except for the major (instead of minor) release cadence.
> >
> > https://docs.google.com/drawings/d/1TDcsLyYPhWpcYEoNo8ohH9AMlpXJB3ywp1CzYCY9C38/edit?usp=sharing
> >
> >> This runs counter to my previous
> >> understanding that master would be 3.x until we branched for the first
> >> breaking change.
> >>
> >> Also I want to expand your example to explain how this will hurt
> >> development on master, specifically diking out bad code. Imagine you
> >> want to removing something like InputStreamMap after 3.0 releases:
> >>
> >> 3.0
> >> 3.1 - deprecate older API in master
> >> 3.x
> >> 4.0 - master branches to 4.x, commit removal to master
> >> 4.x
> >> 5.x - removal ships to users
> >>
> >> In this example, we delay committing the removal to master for almost
> >> six months (from 3.1 to 4.0) and delay communicating preferred
> >> mechanisms on the release branch for the same interval. Changes are not
> >> isolated and earlier changes block subsequent work, e.g., removing
> >> InputStreamMap enabled some ByteSource changes. Not being able to push
> >> these changes to master will make development much harder. Use of extra
> >> development branches partially addresses this, although does not publish
> >> these changes to all developers.
> >
> > I’m aware of this scenario and I would argue that it can be mitigated by better planning. If the deprecation and been well planned, it would have landed just prior to 3.0 and then the removal ships to users in 4.0. I realize that historically we haven’t been very good at planning but I don’t think that’s a good reason to not try to get better at this stuff.
> >
> >>> Not having a predictable deprecation policy guarantees that users will be more likely to stay marooned on old versions because they’ll have no idea at what point things will break on them.
> >>
> >> Perhaps we did not communicate this clearly, but jclouds has had a
> >> predictable deprecation policy for at least the last three years. Users
> >> can upgrade from an earlier 1.7.x release to a later 1.7 release and
> >> breaking changes were only introduced in 1.8.0. Sometimes a given 1.7.x
> >> release contained new deprecation annotations but otherwise applications
> >> continue to work. Despite this, users choose not to upgrade to the
> >> latest micro releases.
> >
> > When you follow something reasonably close to semver you don’t have to worry about this kind of communication because everybody already knows it. Every time I encounter a new project I automatically assume semver unless it’s otherwise stated because there’s really no alternative assumption to make. I’d argue that this is what most developers do.
> >
> >>> I think making potentially breaking changes every 6 months is actually very progressive in the world of Java. I think it’s a good trade off between providing a stable and predictable platform for users to build the businesses upon vs improving that platform in backwards incompatible ways.
> >>
> >> I believe that providing flexibility to evolve jclouds will benefit our
> >> future users as well as existing ones. Shackling ourselves to
> >> restrictive deprecation policies hurts a fair number of people who
> >> prefer to use a more rational cloud abstraction.
> >
> > Just as throwing it open to making deprecations at any point hurts a fair number of people who prefer to use a more stable and predictable cloud abstraction.
> >
> >> I believe some
> >> positive changes simply will not happen with the proposed system.
> >>
> >> In the interests of not talking past each other, I want to summarize
> >> some points in this thread as I understand them.
> >
> > Thanks for this. Agreed that we need to focus the discussion a bit more. So I’m only going to reply to this email to keep us on one email thread.
> >
> >> First, we all agree
> >> that users upgrading from an earlier 3.x should be able to use a later
> >> 3.x release without source modification.
> >
> > Yes. Agreed.
> >
> >> We disagree whether users
> >> should experience new deprecation warnings when upgrading.
> >
> > Yes. We disagree. My main point of contention here is that there needs to be a minimum amount of time for users to react to changes.
> >
> > You feel that 6 weeks is acceptable, such as in the case of 3.3 to 4.0.
> >
> > I feel that 6 weeks is far too short and have been arguing for 6 months.
> >
> >> Second, we
> >> disagree whether users should experience breaking changes when upgrading
> >> from 3.x to 4.x.
> >
> > I’m not so sure we disagree here. Because users are upgrading across a major release boundary there could be breaking changes.
> >
> > But breaking changes should only be in 4.0 and *not* in 4.1, 4.2, or 4.3.
> >
> >> We agree that users can experience breakages from 2.x
> >> to 4.x, with some intermediate deprecations.
> >
> > Yes. We agree (with my same caveat about breaking changes above).
> >
> >> Lastly we disagree on
> >> whether master can introduce breaking changes at any point, or only
> >> after some branching a major release from master.
> >
> > Yes. We disagree. This goes back to my main point of contention.
> >
> > Again, to me it really seems like the BlobStore is your main concern for being able to break things more quickly in order to rapidly improve the existing code. The ComputeService and other APIs don’t seem to be in such a state in general.
> >
> > What if you could work on the BlobStore and deprecate at any point between 2.0 and 3.0?
> >
> > If the BlobStore API had stabilized by 3.0 then perhaps it could move to the more conservative deprecation proposal. Thoughts?
> >
> > Regards,
> > Everett
> >
> >
--
Andrew Gaul
http://gaul.org/
Re: major release cadence
Posted by Everett Toews <ev...@RACKSPACE.COM>.
Let’s see what we can agree on and move forward on those points.
1. 6 month major release cadence.
That would mean the next releases are roughly:
* 2.0.0 mid-Sept
* 2.1.0 early-Nov
* 2.2.0 late-Dec
* 2.3.0 mid-Feb
* 3.0.0 early-Mar
2. Minor point upgrades do not introduce breaking changes. e.g. "users upgrading from an earlier 3.x should be able to use a later
3.x release without source modification.” (pasted from Gaul’s previous email below).
Can everyone agree on those two points?
Regards,
Everett
On Jul 11, 2014, at 11:45 PM, Everett Toews <ev...@rackspace.com> wrote:
> On Jul 7, 2014, at 6:53 PM, Andrew Gaul <ga...@apache.org> wrote:
>
>> Sorry, I still do not understand and cannot reconcile these paragraphs
>> against earlier mails. Do we branch from master every six months or
>> not?
>
> Yes, we branch every 6 months.
>
>> Do we increase the major version every six months?
>
> Yes, we increment the major version every 6 months.
>
>> From your
>> example, it appears that we create a 3.x branch from master and continue
>> releasing 3.x from there.
>
> I think I see the confusion here. Trying to use text to communicate this was a bad idea. Maybe this diagram clears it up. It’s really not that much different than what we do now except for the major (instead of minor) release cadence.
>
> https://docs.google.com/drawings/d/1TDcsLyYPhWpcYEoNo8ohH9AMlpXJB3ywp1CzYCY9C38/edit?usp=sharing
>
>> This runs counter to my previous
>> understanding that master would be 3.x until we branched for the first
>> breaking change.
>>
>> Also I want to expand your example to explain how this will hurt
>> development on master, specifically diking out bad code. Imagine you
>> want to removing something like InputStreamMap after 3.0 releases:
>>
>> 3.0
>> 3.1 - deprecate older API in master
>> 3.x
>> 4.0 - master branches to 4.x, commit removal to master
>> 4.x
>> 5.x - removal ships to users
>>
>> In this example, we delay committing the removal to master for almost
>> six months (from 3.1 to 4.0) and delay communicating preferred
>> mechanisms on the release branch for the same interval. Changes are not
>> isolated and earlier changes block subsequent work, e.g., removing
>> InputStreamMap enabled some ByteSource changes. Not being able to push
>> these changes to master will make development much harder. Use of extra
>> development branches partially addresses this, although does not publish
>> these changes to all developers.
>
> I’m aware of this scenario and I would argue that it can be mitigated by better planning. If the deprecation and been well planned, it would have landed just prior to 3.0 and then the removal ships to users in 4.0. I realize that historically we haven’t been very good at planning but I don’t think that’s a good reason to not try to get better at this stuff.
>
>>> Not having a predictable deprecation policy guarantees that users will be more likely to stay marooned on old versions because they’ll have no idea at what point things will break on them.
>>
>> Perhaps we did not communicate this clearly, but jclouds has had a
>> predictable deprecation policy for at least the last three years. Users
>> can upgrade from an earlier 1.7.x release to a later 1.7 release and
>> breaking changes were only introduced in 1.8.0. Sometimes a given 1.7.x
>> release contained new deprecation annotations but otherwise applications
>> continue to work. Despite this, users choose not to upgrade to the
>> latest micro releases.
>
> When you follow something reasonably close to semver you don’t have to worry about this kind of communication because everybody already knows it. Every time I encounter a new project I automatically assume semver unless it’s otherwise stated because there’s really no alternative assumption to make. I’d argue that this is what most developers do.
>
>>> I think making potentially breaking changes every 6 months is actually very progressive in the world of Java. I think it’s a good trade off between providing a stable and predictable platform for users to build the businesses upon vs improving that platform in backwards incompatible ways.
>>
>> I believe that providing flexibility to evolve jclouds will benefit our
>> future users as well as existing ones. Shackling ourselves to
>> restrictive deprecation policies hurts a fair number of people who
>> prefer to use a more rational cloud abstraction.
>
> Just as throwing it open to making deprecations at any point hurts a fair number of people who prefer to use a more stable and predictable cloud abstraction.
>
>> I believe some
>> positive changes simply will not happen with the proposed system.
>>
>> In the interests of not talking past each other, I want to summarize
>> some points in this thread as I understand them.
>
> Thanks for this. Agreed that we need to focus the discussion a bit more. So I’m only going to reply to this email to keep us on one email thread.
>
>> First, we all agree
>> that users upgrading from an earlier 3.x should be able to use a later
>> 3.x release without source modification.
>
> Yes. Agreed.
>
>> We disagree whether users
>> should experience new deprecation warnings when upgrading.
>
> Yes. We disagree. My main point of contention here is that there needs to be a minimum amount of time for users to react to changes.
>
> You feel that 6 weeks is acceptable, such as in the case of 3.3 to 4.0.
>
> I feel that 6 weeks is far too short and have been arguing for 6 months.
>
>> Second, we
>> disagree whether users should experience breaking changes when upgrading
>> from 3.x to 4.x.
>
> I’m not so sure we disagree here. Because users are upgrading across a major release boundary there could be breaking changes.
>
> But breaking changes should only be in 4.0 and *not* in 4.1, 4.2, or 4.3.
>
>> We agree that users can experience breakages from 2.x
>> to 4.x, with some intermediate deprecations.
>
> Yes. We agree (with my same caveat about breaking changes above).
>
>> Lastly we disagree on
>> whether master can introduce breaking changes at any point, or only
>> after some branching a major release from master.
>
> Yes. We disagree. This goes back to my main point of contention.
>
> Again, to me it really seems like the BlobStore is your main concern for being able to break things more quickly in order to rapidly improve the existing code. The ComputeService and other APIs don’t seem to be in such a state in general.
>
> What if you could work on the BlobStore and deprecate at any point between 2.0 and 3.0?
>
> If the BlobStore API had stabilized by 3.0 then perhaps it could move to the more conservative deprecation proposal. Thoughts?
>
> Regards,
> Everett
>
>
Re: major release cadence
Posted by Everett Toews <ev...@RACKSPACE.COM>.
On Jul 7, 2014, at 6:53 PM, Andrew Gaul <ga...@apache.org> wrote:
> Sorry, I still do not understand and cannot reconcile these paragraphs
> against earlier mails. Do we branch from master every six months or
> not?
Yes, we branch every 6 months.
> Do we increase the major version every six months?
Yes, we increment the major version every 6 months.
> From your
> example, it appears that we create a 3.x branch from master and continue
> releasing 3.x from there.
I think I see the confusion here. Trying to use text to communicate this was a bad idea. Maybe this diagram clears it up. It’s really not that much different than what we do now except for the major (instead of minor) release cadence.
https://docs.google.com/drawings/d/1TDcsLyYPhWpcYEoNo8ohH9AMlpXJB3ywp1CzYCY9C38/edit?usp=sharing
> This runs counter to my previous
> understanding that master would be 3.x until we branched for the first
> breaking change.
>
> Also I want to expand your example to explain how this will hurt
> development on master, specifically diking out bad code. Imagine you
> want to removing something like InputStreamMap after 3.0 releases:
>
> 3.0
> 3.1 - deprecate older API in master
> 3.x
> 4.0 - master branches to 4.x, commit removal to master
> 4.x
> 5.x - removal ships to users
>
> In this example, we delay committing the removal to master for almost
> six months (from 3.1 to 4.0) and delay communicating preferred
> mechanisms on the release branch for the same interval. Changes are not
> isolated and earlier changes block subsequent work, e.g., removing
> InputStreamMap enabled some ByteSource changes. Not being able to push
> these changes to master will make development much harder. Use of extra
> development branches partially addresses this, although does not publish
> these changes to all developers.
I’m aware of this scenario and I would argue that it can be mitigated by better planning. If the deprecation and been well planned, it would have landed just prior to 3.0 and then the removal ships to users in 4.0. I realize that historically we haven’t been very good at planning but I don’t think that’s a good reason to not try to get better at this stuff.
>> Not having a predictable deprecation policy guarantees that users will be more likely to stay marooned on old versions because they’ll have no idea at what point things will break on them.
>
> Perhaps we did not communicate this clearly, but jclouds has had a
> predictable deprecation policy for at least the last three years. Users
> can upgrade from an earlier 1.7.x release to a later 1.7 release and
> breaking changes were only introduced in 1.8.0. Sometimes a given 1.7.x
> release contained new deprecation annotations but otherwise applications
> continue to work. Despite this, users choose not to upgrade to the
> latest micro releases.
When you follow something reasonably close to semver you don’t have to worry about this kind of communication because everybody already knows it. Every time I encounter a new project I automatically assume semver unless it’s otherwise stated because there’s really no alternative assumption to make. I’d argue that this is what most developers do.
>> I think making potentially breaking changes every 6 months is actually very progressive in the world of Java. I think it’s a good trade off between providing a stable and predictable platform for users to build the businesses upon vs improving that platform in backwards incompatible ways.
>
> I believe that providing flexibility to evolve jclouds will benefit our
> future users as well as existing ones. Shackling ourselves to
> restrictive deprecation policies hurts a fair number of people who
> prefer to use a more rational cloud abstraction.
Just as throwing it open to making deprecations at any point hurts a fair number of people who prefer to use a more stable and predictable cloud abstraction.
> I believe some
> positive changes simply will not happen with the proposed system.
>
> In the interests of not talking past each other, I want to summarize
> some points in this thread as I understand them.
Thanks for this. Agreed that we need to focus the discussion a bit more. So I’m only going to reply to this email to keep us on one email thread.
> First, we all agree
> that users upgrading from an earlier 3.x should be able to use a later
> 3.x release without source modification.
Yes. Agreed.
> We disagree whether users
> should experience new deprecation warnings when upgrading.
Yes. We disagree. My main point of contention here is that there needs to be a minimum amount of time for users to react to changes.
You feel that 6 weeks is acceptable, such as in the case of 3.3 to 4.0.
I feel that 6 weeks is far too short and have been arguing for 6 months.
> Second, we
> disagree whether users should experience breaking changes when upgrading
> from 3.x to 4.x.
I’m not so sure we disagree here. Because users are upgrading across a major release boundary there could be breaking changes.
But breaking changes should only be in 4.0 and *not* in 4.1, 4.2, or 4.3.
> We agree that users can experience breakages from 2.x
> to 4.x, with some intermediate deprecations.
Yes. We agree (with my same caveat about breaking changes above).
> Lastly we disagree on
> whether master can introduce breaking changes at any point, or only
> after some branching a major release from master.
Yes. We disagree. This goes back to my main point of contention.
Again, to me it really seems like the BlobStore is your main concern for being able to break things more quickly in order to rapidly improve the existing code. The ComputeService and other APIs don’t seem to be in such a state in general.
What if you could work on the BlobStore and deprecate at any point between 2.0 and 3.0?
If the BlobStore API had stabilized by 3.0 then perhaps it could move to the more conservative deprecation proposal. Thoughts?
Regards,
Everett
Re: major release cadence
Posted by Andrew Phillips <ap...@qrmedia.com>.
Thank you for picking up and summarizing this thread, Andrew G!
> Perhaps we did not communicate this clearly, but jclouds has had a
> predictable deprecation policy for at least the last three years. Users
> can upgrade from an earlier 1.7.x release to a later 1.7 release and
> breaking changes were only introduced in 1.8.0. Sometimes a given 1.7.x
> release contained new deprecation annotations but otherwise applications
> continue to work. Despite this, users choose not to upgrade to the
> latest micro releases.
@Everett: do you have some information you could share about
challenges/complaints/other background that lead to the current
suggestion?
Just from a personal perspective, that would help me try to figure out
how big the relative challenges are that we're looking to balance here.
Regards
ap
Re: major release cadence
Posted by Andrew Gaul <ga...@apache.org>.
[I elided some of the discussion to focus on the core issues of how
development proceeds and our deprecation policy. Other issues has merit
but distract from these major points.]
On Thu, Jul 03, 2014 at 11:13:38PM +0000, Everett Toews wrote:
> On an example 6 month major release cadence
>
> 3.0
> 3.1 - 6 weeks
> 3.2 - 6 weeks
> 3.3 - 6 weeks
> 4.0
>
> Pre 3.0 you have class Foo you want to replace with class Bar.
>
> Anytime prior to 3.0 you can make commit (a) to master that adds class Bar and deprecates Foo which includes the annotation to use Bar and the fact that Foo will be removed in 4.0. Commit (a) should not be backported.
>
> Anytime after 3.0 you can make a commit (b) to master that deletes class Foo. Commit (b) should not be backported.
>
> That’s it. To me that’s simple and predictable. It always gives users 6 months to deal with breaking changes and backwards incompatible removals only happen in major versions.
Sorry, I still do not understand and cannot reconcile these paragraphs
against earlier mails. Do we branch from master every six months or
not? Do we increase the major version every six months? From your
example, it appears that we create a 3.x branch from master and continue
releasing 3.x from there. This runs counter to my previous
understanding that master would be 3.x until we branched for the first
breaking change.
Also I want to expand your example to explain how this will hurt
development on master, specifically diking out bad code. Imagine you
want to removing something like InputStreamMap after 3.0 releases:
3.0
3.1 - deprecate older API in master
3.x
4.0 - master branches to 4.x, commit removal to master
4.x
5.x - removal ships to users
In this example, we delay committing the removal to master for almost
six months (from 3.1 to 4.0) and delay communicating preferred
mechanisms on the release branch for the same interval. Changes are not
isolated and earlier changes block subsequent work, e.g., removing
InputStreamMap enabled some ByteSource changes. Not being able to push
these changes to master will make development much harder. Use of extra
development branches partially addresses this, although does not publish
these changes to all developers.
> I don’t think we should optimize around a particularly tangled example. There can always be exceptions to the policy or guidelines or whatever it is we’re talking about here though.
BlobStore is rotten with such examples and untangling them has proved
difficult. How should we deal with clearContainer after removing
AsyncBlobStore? How to tame the zoo of Payloads? How to support
cancellation of long-lived putBlob and getBlob? BlobStore is the oldest
abstraction and has sufficient technical baggage that adding many kinds
of features has proved difficult.
> Are you working under the idealized assumption that users are tracking jclouds code very closely and are consuming breaking changes rapidly? I can guarantee you this is not the case. So many old versions are still in use.
Using older versions is not any kind of sin and, while I appreciate
users reporting issues only against the current release branch, they
should continue to use older jclouds versions. Even your proposed
deprecation policy mostly benefits the eager upgraders; perhaps you are
working under the idealized assumption that people upgrade at least
every six months? A lot of people continue to use 1.6.2, 1.5.0, an even
1.0, and we can do little to help them move forward.
> Not having a predictable deprecation policy guarantees that users will be more likely to stay marooned on old versions because they’ll have no idea at what point things will break on them.
Perhaps we did not communicate this clearly, but jclouds has had a
predictable deprecation policy for at least the last three years. Users
can upgrade from an earlier 1.7.x release to a later 1.7 release and
breaking changes were only introduced in 1.8.0. Sometimes a given 1.7.x
release contained new deprecation annotations but otherwise applications
continue to work. Despite this, users choose not to upgrade to the
latest micro releases.
> I think making potentially breaking changes every 6 months is actually very progressive in the world of Java. I think it’s a good trade off between providing a stable and predictable platform for users to build the businesses upon vs improving that platform in backwards incompatible ways.
I believe that providing flexibility to evolve jclouds will benefit our
future users as well as existing ones. Shackling ourselves to
restrictive deprecation policies hurts a fair number of people who
prefer to use a more rational cloud abstraction. I believe some
positive changes simply will not happen with the proposed system.
In the interests of not talking past each other, I want to summarize
some points in this thread as I understand them. First, we all agree
that users upgrading from an earlier 3.x should be able to use a later
3.x release without source modification. We disagree whether users
should experience new deprecation warnings when upgrading. Second, we
disagree whether users should experience breaking changes when upgrading
from 3.x to 4.x. We agree that users can experience breakages from 2.x
to 4.x, with some intermediate deprecations. Lastly we disagree on
whether master can introduce breaking changes at any point, or only
after some branching a major release from master.
--
Andrew Gaul
http://gaul.org/
Re: major release cadence
Posted by Everett Toews <ev...@RACKSPACE.COM>.
On Jul 2, 2014, at 4:14 PM, Andrew Gaul <ga...@apache.org> wrote:
> Giving a date-based notice as Guava does for some recent breakages,
> e.g., InputSupplier scheduled for removal in December 2015, makes some
> amount of sense given jclouds historical lack of major release
> predictability. However, I generally doubt that we can provide strong
> guarantees about breakages while addressing the large backlog of jclouds
> technical debt and relying on Guava deprecated and beta APIs.
Because we’ll have both a major and minor release cadence the release dates will be predictable. Of course things will drift a bit because of holidays or voted down RCs but it will still be predictable. Saying something is going to be removed in version X is akin to saying something will be removed on date X.
> Further I do not understand the proposed workflow for changing or
> removing an API. Let's assume we are at 1.7.3 as today and will release
> 1.8.0 in a month, and new major versions every six months. At which
> point in the development cycle can I commit a breaking change to master:
>
> 1) at any point before 1.8.0 release
> 2) immediately after 1.8.0 release
> 3) at any time during 1.8.x
> 4) when someone changes the project version to 2.0.0
>
> Previously we allowed 1-4 but the proposed scheme only allows for 4?
> When do we decide to bump the major version number and which fraction of
> the release cycle allows breaking changes?
On an example 6 month major release cadence
3.0
3.1 - 6 weeks
3.2 - 6 weeks
3.3 - 6 weeks
4.0
Pre 3.0 you have class Foo you want to replace with class Bar.
Anytime prior to 3.0 you can make commit (a) to master that adds class Bar and deprecates Foo which includes the annotation to use Bar and the fact that Foo will be removed in 4.0. Commit (a) should not be backported.
Anytime after 3.0 you can make a commit (b) to master that deletes class Foo. Commit (b) should not be backported.
That’s it. To me that’s simple and predictable. It always gives users 6 months to deal with breaking changes and backwards incompatible removals only happen in major versions.
Yes we could open it up to say you can deprecate and remove things outside of major release boundaries as long as you give users 6 months notice (e.g. 3.2 to 4.2 as above) but that seems complicated and counter-intuitive to users.
> Also consider a more complicated example, I tried to accommodate the
> most number callers when moving from InputSupplier to ByteSource but
> strongly suspect that I caused breakages for several users and I still
> have not completed this work. How would we address this under the
> proposed scheme, given that Payload *implements* the deprecated
> InputSupplier interface, other than introducing a new Payload2 class.
I don’t think we should optimize around a particularly tangled example. There can always be exceptions to the policy or guidelines or whatever it is we’re talking about here though.
> This discussion discourages me since it discusses some idealized
> breakage policy and does not connect to the reality of where jclouds
> exists. Going forward this semantic versioning discipline and ongoing
> API changes will ensure that we bump the major release number
> (MAJOR.y.z) every time instead of the minor one (x.MINOR.z)
I’m not sure where this comes from. The major and minor release cadence would remain the major and minor release cadence.
We wouldn’t bump the major release arbitrarily because we felt like making breaking changes. We would bump it in line with the major release cadence ensuring our users have at minimum 6 months to deal with breaking changes. There’s no point in having a cadence if we don’t follow it.
> but this
> does not really do anything for users except for moving a number around.
> Strong promises of compatibility will discourage many (breaking)
> improvements, big and small. I worry about this a lot given the
> existing quirky jclouds API and the large about of suboptimal API use I
> observe in Stack Overflow questions and jclouds-user mails.
Are you working under the idealized assumption that users are tracking jclouds code very closely and are consuming breaking changes rapidly? I can guarantee you this is not the case. So many old versions are still in use.
Not having a predictable deprecation policy guarantees that users will be more likely to stay marooned on old versions because they’ll have no idea at what point things will break on them.
I think making potentially breaking changes every 6 months is actually very progressive in the world of Java. I think it’s a good trade off between providing a stable and predictable platform for users to build the businesses upon vs improving that platform in backwards incompatible ways.
Regards,
Everett
Re: major release cadence
Posted by Andrew Gaul <ga...@apache.org>.
On Wed, Jul 02, 2014 at 07:22:09PM +0000, Everett Toews wrote:
> Basically I’m looking for a guaranteed *minimum* amount of time for users to be able to react to breaking changes. Considering the long release cycles in place at many organizations, I think 6 months is the absolute minimum.
>
> If we had a 6 month major release cadence, announcing upcoming breaking changes on major release (X.0 only) would be necessary.
>
> If we had a 9 month major release cadence, announcing upcoming breaking changes on major release or the two releases after (X.0, X.1, and X.2 only) would be possible.
Giving a date-based notice as Guava does for some recent breakages,
e.g., InputSupplier scheduled for removal in December 2015, makes some
amount of sense given jclouds historical lack of major release
predictability. However, I generally doubt that we can provide strong
guarantees about breakages while addressing the large backlog of jclouds
technical debt and relying on Guava deprecated and beta APIs.
Further I do not understand the proposed workflow for changing or
removing an API. Let's assume we are at 1.7.3 as today and will release
1.8.0 in a month, and new major versions every six months. At which
point in the development cycle can I commit a breaking change to master:
1) at any point before 1.8.0 release
2) immediately after 1.8.0 release
3) at any time during 1.8.x
4) when someone changes the project version to 2.0.0
Previously we allowed 1-4 but the proposed scheme only allows for 4?
When do we decide to bump the major version number and which fraction of
the release cycle allows breaking changes?
Also consider a more complicated example, I tried to accommodate the
most number callers when moving from InputSupplier to ByteSource but
strongly suspect that I caused breakages for several users and I still
have not completed this work. How would we address this under the
proposed scheme, given that Payload *implements* the deprecated
InputSupplier interface, other than introducing a new Payload2 class.
> > Curious to hear what others feel about that?
>
> Me too. Perhaps it’s lazy consensus at work. :)
This discussion discourages me since it discusses some idealized
breakage policy and does not connect to the reality of where jclouds
exists. Going forward this semantic versioning discipline and ongoing
API changes will ensure that we bump the major release number
(MAJOR.y.z) every time instead of the minor one (x.MINOR.z) but this
does not really do anything for users except for moving a number around.
Strong promises of compatibility will discourage many (breaking)
improvements, big and small. I worry about this a lot given the
existing quirky jclouds API and the large about of suboptimal API use I
observe in Stack Overflow questions and jclouds-user mails.
--
Andrew Gaul
http://gaul.org/
Re: major release cadence
Posted by Everett Toews <ev...@RACKSPACE.COM>.
On Jul 2, 2014, at 12:45 PM, Andrew Phillips <ap...@qrmedia.com> wrote:
>> Some projects never break client code. I think that’s one extreme that won’t serve us or our users well. To ease any pain associated with breaking code, we do need to make it very predictable. And making breaking changes at major release boundaries only achieves that goal.
>
> I think we're definitely agreed that we shouldn't make breaking changes except in major releases. My comment - apologies if it wasn't stated clearly - was that I'm hesitant to restrict ourselves to also *announcing* upcoming breaking changes in major releases.
>
> I agree that we certainly should avoid deprecating something in 2.9 and breaking it in 3.0, but deprecating it in 2.1 and breaking it in 3.0 (with requisite warnings, questions on user@) etc. seems reasonable to me in general.
Basically I’m looking for a guaranteed *minimum* amount of time for users to be able to react to breaking changes. Considering the long release cycles in place at many organizations, I think 6 months is the absolute minimum.
If we had a 6 month major release cadence, announcing upcoming breaking changes on major release (X.0 only) would be necessary.
If we had a 9 month major release cadence, announcing upcoming breaking changes on major release or the two releases after (X.0, X.1, and X.2 only) would be possible.
> Curious to hear what others feel about that?
Me too. Perhaps it’s lazy consensus at work. :)
>> I see what you mean but wouldn’t that make backporting very difficult if not impossible in some situations?
>
> Could you explain? I would imagine that if we wanted to remove a piece of functionality on master (and, of course, we don't *have* to do this - I'm just not sure we want to ban it), we would:
>
> 1) Push a commit that deprecates the functionality we want to remove and presumably adds an alternative
> 2) Backport that commit to the current release branch
> 3) If desired, push a subsequent commit to master to remove the deprecated functionality
Hmmmm…maybe this would be okay. Any new code added after (3) that used the alternative would still be able to be backported. I can’t think of a counter-example...
>> I understand now. What did you think of this alternative wording?
>>
>> "Beta methods and classes will be promoted to non-beta status or removed only in major releases (e.g. MAJOR.0)"
>
> That sounds fine to me. It doesn't give any indication of *how many* major releases you might have to wait, though. I'm perfectly OK of not including such an indication if we feel it doesn't make sense, by the way.
I’m okay not including such an indication. I think it’s okay to take things out of Beta in minor releases if we feel they’re ready. It’s an addition and not a change so no danger of breaking anyone.
Thanks for the good discussion!
Everett
Re: major release cadence
Posted by Andrew Phillips <ap...@qrmedia.com>.
> Some projects never break client code. I think that’s one extreme
> that won’t serve us or our users well. To ease any pain associated
> with breaking code, we do need to make it very predictable. And
> making breaking changes at major release boundaries only achieves
> that goal.
I think we're definitely agreed that we shouldn't make breaking
changes except in major releases. My comment - apologies if it wasn't
stated clearly - was that I'm hesitant to restrict ourselves to also
*announcing* upcoming breaking changes in major releases.
I agree that we certainly should avoid deprecating something in 2.9
and breaking it in 3.0, but deprecating it in 2.1 and breaking it in
3.0 (with requisite warnings, questions on user@) etc. seems
reasonable to me in general.
Curious to hear what others feel about that?
> Frankly, I don’t know what to do here. Remove the section or keep
> it. If you want to keep this section, could you please start it out
> with “This applies to jclouds-lab and jclouds-labs only”?
>
> I really don’t want us to get too distracted by this subject.
Indeed. Modified [1].
> I see what you mean but wouldn’t that make backporting very
> difficult if not impossible in some situations?
Could you explain? I would imagine that if we wanted to remove a piece
of functionality on master (and, of course, we don't *have* to do this
- I'm just not sure we want to ban it), we would:
1) Push a commit that deprecates the functionality we want to remove
and presumably adds an alternative
2) Backport that commit to the current release branch
3) If desired, push a subsequent commit to master to remove the
deprecated functionality
> I understand now. What did you think of this alternative wording?
>
> "Beta methods and classes will be promoted to non-beta status or
> removed only in major releases (e.g. MAJOR.0)"
That sounds fine to me. It doesn't give any indication of *how many*
major releases you might have to wait, though. I'm perfectly OK of not
including such an indication if we feel it doesn't make sense, by the
way.
Thanks for the responses!
ap
[1]
https://wiki.apache.org/jclouds/DeprecationAndBetaPolicy?action=diff&rev2=6&rev1=4
Re: major release cadence
Posted by Everett Toews <ev...@RACKSPACE.COM>.
On Jun 30, 2014, at 7:43 PM, Andrew Phillips <ap...@qrmedia.com> wrote:
>> 2. Deprecation of code and removal of deprecated code will only happen on major releases (e.g. MAJOR.0)
>
> Whilst I can see how this might be easier for us to formulate as a rule, this would mean that it might take *two* major releases before we can effectively get rid of something, if I understand the proposal correctly? E.g. if we've just released 2.0.0 (i.e. master is at 3.0.0-SNAPSHOT) and we want to deprecate something, the deprecation has to wait until 3.0.0 and the removal happens in 4.0.0.
>
> If we're still planning on releasing majors roughly every 6-9 months, this would mean a waiting period of over a year. That seems a little long to me? Obviously, if we allow deprecation in a minor version, we shouldn't deprecate it in the minor version right before a major release and then immediately rip out the code. But I think we can avoid that, and can also formulate that in the guide if necessary.
You’ve understood my proposal correctly. Breaking client code (aka deprecation) is something that should be properly planned. If it hasn’t been properly planned then you could easily get into the situation you’ve described and that’s okay in my opinion. If it hasn’t been properly planned then it probably should wait longer.
Some projects never break client code. I think that’s one extreme that won’t serve us or our users well. To ease any pain associated with breaking code, we do need to make it very predictable. And making breaking changes at major release boundaries only achieves that goal.
>> A concern about the jclouds-labs section [3]. By having that in there, I guarantee you users will extrapolate and apply that to *all* labs repos whether we like it or not.
>
> Fair point. I can see two immediate alternatives: renaming the other labs repos to remove the word "labs" (perhaps too painful), or going through and adding @Beta to all providers and APIs in labs (more accurate because then we just have to define what @Beta means).
The renaming/moving repos is a whole other discussion. ;)
Frankly, I don’t know what to do here. Remove the section or keep it. If you want to keep this section, could you please start it out with “This applies to jclouds-lab and jclouds-labs only”?
I really don’t want us to get too distracted by this subject.
>> The old methods, classes etc. can be removed from master in the next major release
>
> So that would mean they stay around on master too until the next major release is cut (i.e. the last round of commits would be a bunch of removals of deprecated stuff). Is there any specific reason not to remove them from master - which to me represents "the bleeding edge", i.e. the current state of the project - immediately? The reason I'd prefer this, if possible, is that we can at least try to treat master as "free of cruft that we are about to remove”.
I see what you mean but wouldn’t that make backporting very difficult if not impossible in some situations?
> To your other question:
>
> "jclouds aims to not promote a method or class out of beta - or to discard it - by the major release ''after'' next (i.e. A.D.0 if the current release branch is A.B.x). < I'm confused by this. I'm not sure what the intent is here. Could this be replaced by..."
>
> The intent was to somehow clarify to users that we don't intend to declare APIs or providers as @Beta and then leave them in that state forever. I was wondering if it would make sense to try to codify for ourselves, and our users, that @Beta indeed indicates a limited trial period with a clear "go/no-go" decision after a specific amount of time.
I understand now. What did you think of this alternative wording?
"Beta methods and classes will be promoted to non-beta status or removed only in major releases (e.g. MAJOR.0)"
Thanks,
Everett