You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Andrew Purtell <ap...@apache.org> on 2015/04/22 18:29:55 UTC
Clarifying interface evolution freedom in patch releases (was: Re:
[VOTE] Third release candidate for HBase 1.0.1 (RC2))
[Subject changed]
On Tue, Apr 21, 2015 at 8:47 PM, Josh Elser <jo...@gmail.com> wrote:
> I was a little surprised when I noticed method additions to
> InterfaceAudience.Public annotated classes. This means that a user could
> write code against 1.0.1 that would not work against 1.0.0 which seems
> undesirable for a bugfix release. I read over the book section on
> compatibility and didn't see this addressed, so I thought I'd ask.
Let's clarify this. It's not the first time this question has been asked.
To get things moving:
I propose the following addition to the "Client API compatibility" section
of Section 11.1:
+ APIs available in a patch version will be available in all later
+ patch versions. However, new APIs may be added which will not be
+ available in earlier patch versions.
I propose the following change to the "Client Binary compatibility" section
of Section 11.1:
- Old client code can run unchanged (no recompilation needed) against new
jars.
+ Client code written to APIs available in a given patch release
+ can run unchanged (no recompilation needed) against the new
+ jars of later patch versions.
What do you think?
If these changes are (mostly) ok, then this clarifies in one direction.
If these changes are not acceptable, I will propose edits that clarify
toward the opposite meaning.
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Ted Yu <yu...@gmail.com>.
I think these changes are Okay.
On Wed, Apr 22, 2015 at 9:29 AM, Andrew Purtell <ap...@apache.org> wrote:
> [Subject changed]
>
> On Tue, Apr 21, 2015 at 8:47 PM, Josh Elser <jo...@gmail.com> wrote:
>
> > I was a little surprised when I noticed method additions to
> > InterfaceAudience.Public annotated classes. This means that a user could
> > write code against 1.0.1 that would not work against 1.0.0 which seems
> > undesirable for a bugfix release. I read over the book section on
> > compatibility and didn't see this addressed, so I thought I'd ask.
>
>
> Let's clarify this. It's not the first time this question has been asked.
>
> To get things moving:
>
> I propose the following addition to the "Client API compatibility" section
> of Section 11.1:
>
> + APIs available in a patch version will be available in all later
> + patch versions. However, new APIs may be added which will not be
> + available in earlier patch versions.
>
> I propose the following change to the "Client Binary compatibility" section
> of Section 11.1:
>
> - Old client code can run unchanged (no recompilation needed) against new
> jars.
> + Client code written to APIs available in a given patch release
> + can run unchanged (no recompilation needed) against the new
> + jars of later patch versions.
>
>
> What do you think?
>
> If these changes are (mostly) ok, then this clarifies in one direction.
>
> If these changes are not acceptable, I will propose edits that clarify
> toward the opposite meaning.
>
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Andrew Purtell <ap...@apache.org>.
> We also do it, so that users see less of "This was in 0.98, but how come
it is not in 1.0.x". I think we will get substantially better over time.
It will. Eventually we will EOL 0.98
In the meantime we could clarify how 0.98 relates to later releases now
that releases 1.0.0 and up are being more strict about compatibility. There
will always be a 1.x that provides a straight upgrade path from 0.98, or if
not that is an error in 0.98 maintenance that would need to be caught and
corrected (hopefully before release). The part of it which may not be
obvious is which 1.x is the straight path upgrade from 0.98 could change
over time.
As 0.98 RM I'm also ok with being more strict about backporting features at
this point. I think many (most?) are moving on from 0.98 and we should be
encouraging new users to start with 1.0. However, let's use HBASE-13477 as
an example. Where we have people running 0.98 in production and cannot
upgrade right now beyond a later 0.98 point version, not having access to
highly useful metrics simply due to self-imposed policy isn't operator
friendly. Sure, we could say "since you want it, upgrade :-)" but that's
unsatisfying if upgrading just isn't possible. If we want to narrow down
when and why we'd put something in 0.98 that could only go into the latest
1.x, we can have that conversation.
On Thu, Apr 23, 2015 at 2:58 PM, Enis Söztutar <en...@gmail.com> wrote:
> On Thu, Apr 23, 2015 at 1:31 PM, Sean Busbey <bu...@cloudera.com> wrote:
>
> > Why don't we just focus development after a minor release on the next
> minor
> > release instead of the next patch release?
> >
> > We could limit backports to the patch releases to critical bugs, which
> > would cut down on how often someone has to deal with the pain of making
> > sure we don't add to public APIs. It also reduces the risk someone going
> > through an upgrade has, since there are fewer changes.
> >
>
> Big +1. I think we will get there, but devs/committers are used to
> backporting simple
> new features across all branches. Not to nit pick or anything, an example
> is
> HBASE-13168 which I failed to catch. Ideally it should not have been in
> 1.0.1. We also
> do it, so that users see less of "This was in 0.98, but how come it is not
> in 1.0.x". I think
> we will get substantially better over time.
>
>
> >
> > If someone fixes a bug and doesn't want to do the work of making sure it
> > doesn't add methods in a patch release, they just don't backport to that
> > version and make a follow on e.g. "backport to 1.0.z" ticket.
> >
> >
> >
> > On Thu, Apr 23, 2015 at 1:50 PM, Enis Söztutar <en...@gmail.com>
> wrote:
> >
> > > +1 to the proposal.
> > >
> > > The problem is that we have a very big API surface especially with the
> > > coprocessors included in the report. Even simple bug fixes can
> introduce
> > > protected or public methods to base classes, which makes patch releases
> > > very hard to maintain. I would not want to spend the effort to spend
> tons
> > > of time trying to make a patch not introduce new methods in order to
> > > backport. That effort can be spent elsewhere IMO.
> > >
> > > Looking at the report
> > > https://people.apache.org/~enis/1.0.0_1.0.1RC2_compat_report.html,
> > nothing
> > > strikes me as "new functionality". Going from current 1.0.0 to 1.0.1RC2
> > > should actually be as you would expect from upgrading a patch release.
> > >
> > > Yes, adding new API in patch releases will make downgrading harder,
> but I
> > > think that is an acceptable tradeoff. We can document that if your
> > > application compiles (meaning that you are not using new API) with
> 1.0.0,
> > > then you can swap your jars in a binary compat manner.
> > >
> > > Enis
> > >
> > > On Thu, Apr 23, 2015 at 10:03 AM, Andrew Purtell <ap...@apache.org>
> > > wrote:
> > >
> > > > Anyone disagree with the point of view put forward by Josh and Sean?
> > > >
> > > >
> > > > On Wed, Apr 22, 2015 at 10:35 AM, Josh Elser <jo...@gmail.com>
> > > wrote:
> > > >
> > > > > Andy -- I understood your intent, but thanks for clarifying. (as
> well
> > > as
> > > > > taking the time to break this discussion out in the first place). I
> > > agree
> > > > > with your assessment.
> > > > >
> > > > > re: Sean's comments, if it wasn't clear by me asking in the first
> > > place,
> > > > I
> > > > > also think sticking as close as possible to semver's rules is the
> > best
> > > > > approach, although I'm getting the impression that there have been
> > some
> > > > > previous reservations to doing so (especially by your comment about
> > > > > backporting features if there is demand is).
> > > > >
> > > > > I've found adhering to the bug-fix release restrictions can be a
> very
> > > > > painful and time-consuming task, so this is something to get a
> > > > > representative sampling of those who do the work to make sure
> > everyone
> > > is
> > > > > on board.
> > > > >
> > > > >
> > > > > Sean Busbey wrote:
> > > > >
> > > > >> I'd much rather we stick with the definitions used in Semantic
> > > > Versioning.
> > > > >> Our use is already confusing enough given our matrix of
> > > compatibilities
> > > > >> that don't get "major version for breaking" protections.
> > > > >>
> > > > >> We've previously discussed how we'll do additional minor releases
> > when
> > > > >> there's sufficient interest in the new features present there.
> > What's
> > > > >> building that demand if any backwards compatible change can go
> back
> > > > into a
> > > > >> patch release?
> > > > >>
> > > > >> Would we have an easier time restraining ourselves if we had a
> > regular
> > > > >> schedule planned around new minor versions?
> > > > >>
> > > > >>
> > > > >> On Wed, Apr 22, 2015 at 12:03 PM, Josh Elser<josh.elser@gmail.com
> >
> > > > >> wrote:
> > > > >>
> > > > >> While I can understand the desire to want to add things, I do
> think
> > > it
> > > > >>> makes things harder for users to reliably write code against
> > versions
> > > > of
> > > > >>> HBase which (by their view) should be completely compatible with
> > one
> > > > >>> another.
> > > > >>>
> > > > >>> Take this extremely hypothetical situation: I'm new to HBase and
> > > start
> > > > >>> writing some code against HBase 1.0.1 which was just deployed at
> my
> > > > >>> $job. I
> > > > >>> don't _know_ what APIs are new, I just know what exists and treat
> > > that
> > > > as
> > > > >>> acceptable for me to be using. Meanwhile in production, some
> other
> > > > people
> > > > >>> find a bug with HBase 1.0.1 and roll back to 1.0.0 which they had
> > > been
> > > > >>> previously using. My reaction would be "of course my code should
> > work
> > > > >>> with
> > > > >>> HBase 1.0.0, I only used the public API" when in fact this is not
> > > true.
> > > > >>>
> > > > >>> Personally, I think it's a little bold to say semver is even in
> use
> > > if
> > > > >>> this principal isn't being followed as it doesn't follow at all
> > with
> > > my
> > > > >>> understanding on the guarantees defined by semver for bug-fix
> > > releases.
> > > > >>>
> > > > >>> That being said, if the intent *is* to allow ourselves to make
> > these
> > > > >>> sorts
> > > > >>> of changes, I just think some sort of disclaimer should be
> present:
> > > > >>>
> > > > >>> - HBase uses Semantic Versioning for its release versioning
> > > > >>> + HBase uses Semantic Versioning for its release versioning with
> a
> > > > caveat
> > > > >>> that methods and members might be added in newer bug-fix releases
> > > that
> > > > >>> were
> > > > >>> not present in the previous bug-fix release.
> > > > >>>
> > > > >>>
> > > > >>> Andrew Purtell wrote:
> > > > >>>
> > > > >>> [Subject changed]
> > > > >>>>
> > > > >>>> On Tue, Apr 21, 2015 at 8:47 PM, Josh Elser<
> josh.elser@gmail.com>
> > > > >>>> wrote:
> > > > >>>>
> > > > >>>> I was a little surprised when I noticed method additions to
> > > > >>>>
> > > > >>>>> InterfaceAudience.Public annotated classes. This means that a
> > user
> > > > >>>>> could
> > > > >>>>> write code against 1.0.1 that would not work against 1.0.0
> which
> > > > seems
> > > > >>>>> undesirable for a bugfix release. I read over the book section
> on
> > > > >>>>> compatibility and didn't see this addressed, so I thought I'd
> > ask.
> > > > >>>>>
> > > > >>>>>
> > > > >>>> Let's clarify this. It's not the first time this question has
> been
> > > > >>>> asked.
> > > > >>>>
> > > > >>>> To get things moving:
> > > > >>>>
> > > > >>>> I propose the following addition to the "Client API
> compatibility"
> > > > >>>> section
> > > > >>>> of Section 11.1:
> > > > >>>>
> > > > >>>> + APIs available in a patch version will be available in all
> later
> > > > >>>> + patch versions. However, new APIs may be added which will not
> be
> > > > >>>> + available in earlier patch versions.
> > > > >>>>
> > > > >>>> I propose the following change to the "Client Binary
> > compatibility"
> > > > >>>> section
> > > > >>>> of Section 11.1:
> > > > >>>>
> > > > >>>> - Old client code can run unchanged (no recompilation needed)
> > > against
> > > > >>>> new
> > > > >>>> jars.
> > > > >>>> + Client code written to APIs available in a given patch release
> > > > >>>> + can run unchanged (no recompilation needed) against the new
> > > > >>>> + jars of later patch versions.
> > > > >>>>
> > > > >>>>
> > > > >>>> What do you think?
> > > > >>>>
> > > > >>>> If these changes are (mostly) ok, then this clarifies in one
> > > > direction.
> > > > >>>>
> > > > >>>> If these changes are not acceptable, I will propose edits that
> > > clarify
> > > > >>>> toward the opposite meaning.
> > > > >>>>
> > > > >>>>
> > > > >>>>
> > > > >>>>
> > > > >>
> > > > >>
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > >
> > > > - Andy
> > > >
> > > > Problems worthy of attack prove their worth by hitting back. - Piet
> > Hein
> > > > (via Tom White)
> > > >
> > >
> >
> >
> >
> > --
> > Sean
> >
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Enis Söztutar <en...@gmail.com>.
On Thu, Apr 23, 2015 at 1:31 PM, Sean Busbey <bu...@cloudera.com> wrote:
> Why don't we just focus development after a minor release on the next minor
> release instead of the next patch release?
>
> We could limit backports to the patch releases to critical bugs, which
> would cut down on how often someone has to deal with the pain of making
> sure we don't add to public APIs. It also reduces the risk someone going
> through an upgrade has, since there are fewer changes.
>
Big +1. I think we will get there, but devs/committers are used to
backporting simple
new features across all branches. Not to nit pick or anything, an example
is
HBASE-13168 which I failed to catch. Ideally it should not have been in
1.0.1. We also
do it, so that users see less of "This was in 0.98, but how come it is not
in 1.0.x". I think
we will get substantially better over time.
>
> If someone fixes a bug and doesn't want to do the work of making sure it
> doesn't add methods in a patch release, they just don't backport to that
> version and make a follow on e.g. "backport to 1.0.z" ticket.
>
>
>
> On Thu, Apr 23, 2015 at 1:50 PM, Enis Söztutar <en...@gmail.com> wrote:
>
> > +1 to the proposal.
> >
> > The problem is that we have a very big API surface especially with the
> > coprocessors included in the report. Even simple bug fixes can introduce
> > protected or public methods to base classes, which makes patch releases
> > very hard to maintain. I would not want to spend the effort to spend tons
> > of time trying to make a patch not introduce new methods in order to
> > backport. That effort can be spent elsewhere IMO.
> >
> > Looking at the report
> > https://people.apache.org/~enis/1.0.0_1.0.1RC2_compat_report.html,
> nothing
> > strikes me as "new functionality". Going from current 1.0.0 to 1.0.1RC2
> > should actually be as you would expect from upgrading a patch release.
> >
> > Yes, adding new API in patch releases will make downgrading harder, but I
> > think that is an acceptable tradeoff. We can document that if your
> > application compiles (meaning that you are not using new API) with 1.0.0,
> > then you can swap your jars in a binary compat manner.
> >
> > Enis
> >
> > On Thu, Apr 23, 2015 at 10:03 AM, Andrew Purtell <ap...@apache.org>
> > wrote:
> >
> > > Anyone disagree with the point of view put forward by Josh and Sean?
> > >
> > >
> > > On Wed, Apr 22, 2015 at 10:35 AM, Josh Elser <jo...@gmail.com>
> > wrote:
> > >
> > > > Andy -- I understood your intent, but thanks for clarifying. (as well
> > as
> > > > taking the time to break this discussion out in the first place). I
> > agree
> > > > with your assessment.
> > > >
> > > > re: Sean's comments, if it wasn't clear by me asking in the first
> > place,
> > > I
> > > > also think sticking as close as possible to semver's rules is the
> best
> > > > approach, although I'm getting the impression that there have been
> some
> > > > previous reservations to doing so (especially by your comment about
> > > > backporting features if there is demand is).
> > > >
> > > > I've found adhering to the bug-fix release restrictions can be a very
> > > > painful and time-consuming task, so this is something to get a
> > > > representative sampling of those who do the work to make sure
> everyone
> > is
> > > > on board.
> > > >
> > > >
> > > > Sean Busbey wrote:
> > > >
> > > >> I'd much rather we stick with the definitions used in Semantic
> > > Versioning.
> > > >> Our use is already confusing enough given our matrix of
> > compatibilities
> > > >> that don't get "major version for breaking" protections.
> > > >>
> > > >> We've previously discussed how we'll do additional minor releases
> when
> > > >> there's sufficient interest in the new features present there.
> What's
> > > >> building that demand if any backwards compatible change can go back
> > > into a
> > > >> patch release?
> > > >>
> > > >> Would we have an easier time restraining ourselves if we had a
> regular
> > > >> schedule planned around new minor versions?
> > > >>
> > > >>
> > > >> On Wed, Apr 22, 2015 at 12:03 PM, Josh Elser<jo...@gmail.com>
> > > >> wrote:
> > > >>
> > > >> While I can understand the desire to want to add things, I do think
> > it
> > > >>> makes things harder for users to reliably write code against
> versions
> > > of
> > > >>> HBase which (by their view) should be completely compatible with
> one
> > > >>> another.
> > > >>>
> > > >>> Take this extremely hypothetical situation: I'm new to HBase and
> > start
> > > >>> writing some code against HBase 1.0.1 which was just deployed at my
> > > >>> $job. I
> > > >>> don't _know_ what APIs are new, I just know what exists and treat
> > that
> > > as
> > > >>> acceptable for me to be using. Meanwhile in production, some other
> > > people
> > > >>> find a bug with HBase 1.0.1 and roll back to 1.0.0 which they had
> > been
> > > >>> previously using. My reaction would be "of course my code should
> work
> > > >>> with
> > > >>> HBase 1.0.0, I only used the public API" when in fact this is not
> > true.
> > > >>>
> > > >>> Personally, I think it's a little bold to say semver is even in use
> > if
> > > >>> this principal isn't being followed as it doesn't follow at all
> with
> > my
> > > >>> understanding on the guarantees defined by semver for bug-fix
> > releases.
> > > >>>
> > > >>> That being said, if the intent *is* to allow ourselves to make
> these
> > > >>> sorts
> > > >>> of changes, I just think some sort of disclaimer should be present:
> > > >>>
> > > >>> - HBase uses Semantic Versioning for its release versioning
> > > >>> + HBase uses Semantic Versioning for its release versioning with a
> > > caveat
> > > >>> that methods and members might be added in newer bug-fix releases
> > that
> > > >>> were
> > > >>> not present in the previous bug-fix release.
> > > >>>
> > > >>>
> > > >>> Andrew Purtell wrote:
> > > >>>
> > > >>> [Subject changed]
> > > >>>>
> > > >>>> On Tue, Apr 21, 2015 at 8:47 PM, Josh Elser<jo...@gmail.com>
> > > >>>> wrote:
> > > >>>>
> > > >>>> I was a little surprised when I noticed method additions to
> > > >>>>
> > > >>>>> InterfaceAudience.Public annotated classes. This means that a
> user
> > > >>>>> could
> > > >>>>> write code against 1.0.1 that would not work against 1.0.0 which
> > > seems
> > > >>>>> undesirable for a bugfix release. I read over the book section on
> > > >>>>> compatibility and didn't see this addressed, so I thought I'd
> ask.
> > > >>>>>
> > > >>>>>
> > > >>>> Let's clarify this. It's not the first time this question has been
> > > >>>> asked.
> > > >>>>
> > > >>>> To get things moving:
> > > >>>>
> > > >>>> I propose the following addition to the "Client API compatibility"
> > > >>>> section
> > > >>>> of Section 11.1:
> > > >>>>
> > > >>>> + APIs available in a patch version will be available in all later
> > > >>>> + patch versions. However, new APIs may be added which will not be
> > > >>>> + available in earlier patch versions.
> > > >>>>
> > > >>>> I propose the following change to the "Client Binary
> compatibility"
> > > >>>> section
> > > >>>> of Section 11.1:
> > > >>>>
> > > >>>> - Old client code can run unchanged (no recompilation needed)
> > against
> > > >>>> new
> > > >>>> jars.
> > > >>>> + Client code written to APIs available in a given patch release
> > > >>>> + can run unchanged (no recompilation needed) against the new
> > > >>>> + jars of later patch versions.
> > > >>>>
> > > >>>>
> > > >>>> What do you think?
> > > >>>>
> > > >>>> If these changes are (mostly) ok, then this clarifies in one
> > > direction.
> > > >>>>
> > > >>>> If these changes are not acceptable, I will propose edits that
> > clarify
> > > >>>> toward the opposite meaning.
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>
> > > >>
> > >
> > >
> > > --
> > > Best regards,
> > >
> > > - Andy
> > >
> > > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > > (via Tom White)
> > >
> >
>
>
>
> --
> Sean
>
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Sean Busbey <bu...@cloudera.com>.
On Thu, Apr 23, 2015 at 5:08 PM, Andrew Purtell <ap...@apache.org> wrote:
> So "we like semver", not "we use semver"?
>
> Just want to make sure we're all on the same page (or find out if not).
>
>
+1 from me.
I'd prefer we not make the statements about binary compat for downgrading
because it's complicated and basically boils down to "we don't make
promises about downgrading."
--
Sean
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Nick Dimiduk <nd...@gmail.com>.
In case folks aren't following over on HBASE-13339, I ran our fancy compat
checking script with small changes [0] against the candidate Hadoop
versions. You can investigate the resulting report [1] in more detail, but
it looks like the big-ticket binary compatibility changes are related to
classes and methods marked Evolving or Unstable. Outstanding issues
(HADOOP-11708, HDFS-8270) aside, I think we should be able to make this
update for a minor release. Our compatibility guide has a 'Y' in the
"Dependency compatibility" row under "Minor", and this should be satisfied
by the above report.
[0]: http://people.apache.org/~ndimiduk/check_compat_hadoop.patch
[1]:
http://people.apache.org/~ndimiduk/release-2.5.1_release-2.6.0_compat_report.html
On Mon, Apr 27, 2015 at 9:53 AM, Nick Dimiduk <nd...@gmail.com> wrote:
> Seems I misunderstood the conclusion of this discussion and pushed
> prematurely HBASE-13339 (and perhaps HBASE-13149). On that ticket, Lars
> states
>
> > I don't recall that we agreed on this. Should be in trunk (2.0) no
> doubt. There was a longer discussion about 1.1 and I seem to recall
> > that we said we do not want to force a major Hadoop with a minor HBase
> upgrade and that instead we'd document in the book how
> > build with Hadoop 2.6.0.
>
> My understanding of this discussion was that we have relaxed out statement
> regarding adherence to semantic versioning guidelines. Thus a minor version
> bump of the Hadoop dependency was acceptable, particularly considering the
> bug found on 2.5. Furthermore, following Hadoop on it's Jackson version was
> the prudent decision for allowing our tools to work using either common
> invocation patterns.
>
> I got this wrong? Can someone summarize the decision here so that I can
> move forward with an RC?
>
> Thanks,
> Nick
>
> On Sun, Apr 26, 2015 at 11:33 AM, Nick Dimiduk <nd...@gmail.com> wrote:
>
>> I spoke with a couple of our HDFS folks on Friday, they sound confident
>> that 2.7.1 will be considered production-stable. I think we can try bumping
>> master to 2.7.1 when it's out.
>>
>> On Fri, Apr 24, 2015 at 10:03 AM, Sean Busbey <bu...@cloudera.com>
>> wrote:
>>
>>> On Fri, Apr 24, 2015 at 11:53 AM, Elliott Clark <ec...@apache.org>
>>> wrote:
>>>
>>> > Is 2.7.1 going to have that not for production tag or was that tag just
>>> > because of the issues found late in the cycle that are now fixed?
>>> >
>>> >
>>> From reading from hadoop lists, I'm not sure. Including the
>>> non-production
>>> tag seemed to be a compromise position to get 2.7.0 out the door. The
>>> cycle
>>> covered so much ground that I'm not sure when it got decided. The
>>> discussion also got them onto figuring out what their general policy on
>>> production / non-production will be. When they come to a decision it'll
>>> be
>>> good to revisit and see what it means for us.
>>>
>>> At least some of the Hadoopers are in agreement that 2.7.1 should be
>>> labeled production ready. No one was opposed to the matter, but there
>>> were
>>> some noticeable gaps in assenting voices[1].
>>>
>>> It's supposed to be soon, so we'll see I guess?
>>>
>>> [1]:
>>>
>>> http://markmail.org/thread/zwzze6cqqgwq4rmw#query:+page:1+mid:u6pihxyy36pzz52e+state:results
>>>
>>> --
>>> Sean
>>>
>>
>>
>
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Nick Dimiduk <nd...@gmail.com>.
Seems I misunderstood the conclusion of this discussion and pushed
prematurely HBASE-13339 (and perhaps HBASE-13149). On that ticket, Lars
states
> I don't recall that we agreed on this. Should be in trunk (2.0) no doubt.
There was a longer discussion about 1.1 and I seem to recall
> that we said we do not want to force a major Hadoop with a minor HBase
upgrade and that instead we'd document in the book how
> build with Hadoop 2.6.0.
My understanding of this discussion was that we have relaxed out statement
regarding adherence to semantic versioning guidelines. Thus a minor version
bump of the Hadoop dependency was acceptable, particularly considering the
bug found on 2.5. Furthermore, following Hadoop on it's Jackson version was
the prudent decision for allowing our tools to work using either common
invocation patterns.
I got this wrong? Can someone summarize the decision here so that I can
move forward with an RC?
Thanks,
Nick
On Sun, Apr 26, 2015 at 11:33 AM, Nick Dimiduk <nd...@gmail.com> wrote:
> I spoke with a couple of our HDFS folks on Friday, they sound confident
> that 2.7.1 will be considered production-stable. I think we can try bumping
> master to 2.7.1 when it's out.
>
> On Fri, Apr 24, 2015 at 10:03 AM, Sean Busbey <bu...@cloudera.com> wrote:
>
>> On Fri, Apr 24, 2015 at 11:53 AM, Elliott Clark <ec...@apache.org>
>> wrote:
>>
>> > Is 2.7.1 going to have that not for production tag or was that tag just
>> > because of the issues found late in the cycle that are now fixed?
>> >
>> >
>> From reading from hadoop lists, I'm not sure. Including the non-production
>> tag seemed to be a compromise position to get 2.7.0 out the door. The
>> cycle
>> covered so much ground that I'm not sure when it got decided. The
>> discussion also got them onto figuring out what their general policy on
>> production / non-production will be. When they come to a decision it'll be
>> good to revisit and see what it means for us.
>>
>> At least some of the Hadoopers are in agreement that 2.7.1 should be
>> labeled production ready. No one was opposed to the matter, but there were
>> some noticeable gaps in assenting voices[1].
>>
>> It's supposed to be soon, so we'll see I guess?
>>
>> [1]:
>>
>> http://markmail.org/thread/zwzze6cqqgwq4rmw#query:+page:1+mid:u6pihxyy36pzz52e+state:results
>>
>> --
>> Sean
>>
>
>
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Nick Dimiduk <nd...@gmail.com>.
I spoke with a couple of our HDFS folks on Friday, they sound confident
that 2.7.1 will be considered production-stable. I think we can try bumping
master to 2.7.1 when it's out.
On Fri, Apr 24, 2015 at 10:03 AM, Sean Busbey <bu...@cloudera.com> wrote:
> On Fri, Apr 24, 2015 at 11:53 AM, Elliott Clark <ec...@apache.org> wrote:
>
> > Is 2.7.1 going to have that not for production tag or was that tag just
> > because of the issues found late in the cycle that are now fixed?
> >
> >
> From reading from hadoop lists, I'm not sure. Including the non-production
> tag seemed to be a compromise position to get 2.7.0 out the door. The cycle
> covered so much ground that I'm not sure when it got decided. The
> discussion also got them onto figuring out what their general policy on
> production / non-production will be. When they come to a decision it'll be
> good to revisit and see what it means for us.
>
> At least some of the Hadoopers are in agreement that 2.7.1 should be
> labeled production ready. No one was opposed to the matter, but there were
> some noticeable gaps in assenting voices[1].
>
> It's supposed to be soon, so we'll see I guess?
>
> [1]:
>
> http://markmail.org/thread/zwzze6cqqgwq4rmw#query:+page:1+mid:u6pihxyy36pzz52e+state:results
>
> --
> Sean
>
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Sean Busbey <bu...@cloudera.com>.
On Fri, Apr 24, 2015 at 11:53 AM, Elliott Clark <ec...@apache.org> wrote:
> Is 2.7.1 going to have that not for production tag or was that tag just
> because of the issues found late in the cycle that are now fixed?
>
>
>From reading from hadoop lists, I'm not sure. Including the non-production
tag seemed to be a compromise position to get 2.7.0 out the door. The cycle
covered so much ground that I'm not sure when it got decided. The
discussion also got them onto figuring out what their general policy on
production / non-production will be. When they come to a decision it'll be
good to revisit and see what it means for us.
At least some of the Hadoopers are in agreement that 2.7.1 should be
labeled production ready. No one was opposed to the matter, but there were
some noticeable gaps in assenting voices[1].
It's supposed to be soon, so we'll see I guess?
[1]:
http://markmail.org/thread/zwzze6cqqgwq4rmw#query:+page:1+mid:u6pihxyy36pzz52e+state:results
--
Sean
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Elliott Clark <ec...@apache.org>.
Is 2.7.1 going to have that not for production tag or was that tag just
because of the issues found late in the cycle that are now fixed?
On Fri, Apr 24, 2015 at 9:50 AM, Sean Busbey <bu...@cloudera.com> wrote:
> On Thu, Apr 23, 2015 at 7:34 PM, Nick Dimiduk <nd...@gmail.com> wrote:
>
> > On Thu, Apr 23, 2015 at 4:13 PM, Stack <st...@duboce.net> wrote:
> >
> > > Does this 'admission' help with the which-hadoop thread too?
> > >
> >
> > Right -- "working toward semver".
> >
> > Are we now at liberty to bump to 2.6 or 2.7 even for branch-1.1? I still
> > have Karthik's offer to roll a 2.5.3 with the HDFS issue resolved.
> >
> > What about the jackson issue with 2.5 YARN runtime?
> >
> > Thanks Sean and Josh for being our community conscience on these issues.
> >
> >
> I think it would still be a good idea to encourage Hadoop to put out 2.5.3
> and 2.6.1 with their respective HDFS issues resolved. It'll be good for our
> downstream users who may not be able to upgrade to a new "major" Hadoop
> version yet.
>
> I don't think we should bump to 2.7.0, given the "not for production"
> disclaimer in the release notes. To me, it would seem equivalent to if
> Phoenix had updated to use HBase 0.99.
>
> On the Jackson thing, are folks opposed to compiling with the current older
> version and then packaging the newer version? That would make sure we don't
> start using 1.9 features in a way that would prevent downstream users from
> downgrading to 1.8, which combined with a release note would address my
> concerns.
>
>
> --
> Sean
>
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Sean Busbey <bu...@cloudera.com>.
On Thu, Apr 23, 2015 at 7:34 PM, Nick Dimiduk <nd...@gmail.com> wrote:
> On Thu, Apr 23, 2015 at 4:13 PM, Stack <st...@duboce.net> wrote:
>
> > Does this 'admission' help with the which-hadoop thread too?
> >
>
> Right -- "working toward semver".
>
> Are we now at liberty to bump to 2.6 or 2.7 even for branch-1.1? I still
> have Karthik's offer to roll a 2.5.3 with the HDFS issue resolved.
>
> What about the jackson issue with 2.5 YARN runtime?
>
> Thanks Sean and Josh for being our community conscience on these issues.
>
>
I think it would still be a good idea to encourage Hadoop to put out 2.5.3
and 2.6.1 with their respective HDFS issues resolved. It'll be good for our
downstream users who may not be able to upgrade to a new "major" Hadoop
version yet.
I don't think we should bump to 2.7.0, given the "not for production"
disclaimer in the release notes. To me, it would seem equivalent to if
Phoenix had updated to use HBase 0.99.
On the Jackson thing, are folks opposed to compiling with the current older
version and then packaging the newer version? That would make sure we don't
start using 1.9 features in a way that would prevent downstream users from
downgrading to 1.8, which combined with a release note would address my
concerns.
--
Sean
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Jerry He <je...@gmail.com>.
A second thought on this. I wonder how it is going to work. The test suite
probably will run with old Jackson version with the compile. We will ship
what we don't test with?
Whatever we do seem to imply similarly small risk ...
On Apr 25, 2015 8:02 PM, "Nick Dimiduk" <nd...@gmail.com> wrote:
> I'm game for giving it a shot if we can build some confidence it won't
> cause other issues. However given we're moving to Hadoop 2.6 I think we've
> already relaxed the constraints and this won't buy us much. More trouble
> than it's worth?
>
> On Saturday, April 25, 2015, Jerry He <je...@gmail.com> wrote:
>
> > This is an interesting idea from Sean.
> >
> > > On the Jackson thing, are folks opposed to compiling with the current
> > older
> > > version and then packaging the newer version? That would make sure we
> > don't
> > > start using 1.9 features in a way that would prevent downstream users
> > from
> > > downgrading to 1.8, which combined with a release note would address my
> > > concerns.
> >
> > I wonder if people have objections or concerns on this. Maybe cause
> > confusion down the road, or in the pom or assembly?
> >
> > Looks like we are moving in the direction of relaxing a little, and
> upgrade
> > Jackson in HBase 1.1.
> >
> > Jerry
> >
>
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Nick Dimiduk <nd...@gmail.com>.
I'm game for giving it a shot if we can build some confidence it won't
cause other issues. However given we're moving to Hadoop 2.6 I think we've
already relaxed the constraints and this won't buy us much. More trouble
than it's worth?
On Saturday, April 25, 2015, Jerry He <je...@gmail.com> wrote:
> This is an interesting idea from Sean.
>
> > On the Jackson thing, are folks opposed to compiling with the current
> older
> > version and then packaging the newer version? That would make sure we
> don't
> > start using 1.9 features in a way that would prevent downstream users
> from
> > downgrading to 1.8, which combined with a release note would address my
> > concerns.
>
> I wonder if people have objections or concerns on this. Maybe cause
> confusion down the road, or in the pom or assembly?
>
> Looks like we are moving in the direction of relaxing a little, and upgrade
> Jackson in HBase 1.1.
>
> Jerry
>
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Jerry He <je...@gmail.com>.
This is an interesting idea from Sean.
> On the Jackson thing, are folks opposed to compiling with the current
older
> version and then packaging the newer version? That would make sure we
don't
> start using 1.9 features in a way that would prevent downstream users from
> downgrading to 1.8, which combined with a release note would address my
> concerns.
I wonder if people have objections or concerns on this. Maybe cause
confusion down the road, or in the pom or assembly?
Looks like we are moving in the direction of relaxing a little, and upgrade
Jackson in HBase 1.1.
Jerry
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Nick Dimiduk <nd...@gmail.com>.
On Thu, Apr 23, 2015 at 6:39 PM, Enis Söztutar <en...@gmail.com> wrote:
> I would be in favor of going 2.6.0 and jackson upgrade in 1.1.
>
I am also +1 bumping to Hadoop 2.6.x and Jackson 1.9.x in HBase 1.1.0. This
seems like the most practical solution to the problems outlined here. Let
me explore the API surface area differences between Hadoop 2.5, 2.6 and see
what might crop up for users.
-n
On Thu, Apr 23, 2015 at 5:34 PM, Nick Dimiduk <nd...@gmail.com> wrote:
>
> > On Thu, Apr 23, 2015 at 4:13 PM, Stack <st...@duboce.net> wrote:
> >
> > > Does this 'admission' help with the which-hadoop thread too?
> > >
> >
> > Right -- "working toward semver".
> >
> > Are we now at liberty to bump to 2.6 or 2.7 even for branch-1.1? I still
> > have Karthik's offer to roll a 2.5.3 with the HDFS issue resolved.
> >
> > What about the jackson issue with 2.5 YARN runtime?
> >
> > Thanks Sean and Josh for being our community conscience on these issues.
> >
> > > Just want to make sure we're all on the same page (or find out if not).
> > > >
> > > > On Thu, Apr 23, 2015 at 2:59 PM, Enis Söztutar <en...@apache.org>
> > wrote:
> > > >
> > > > > Then let's get Andrew's proposal committed:
> > > > >
> > > > > + APIs available in a patch version will be available in all later
> > > > > + patch versions. However, new APIs may be added which will not be
> > > > > + available in earlier patch versions.
> > > > >
> > > > > I propose the following change to the "Client Binary compatibility"
> > > > section
> > > > > of Section 11.1:
> > > > >
> > > > > - Old client code can run unchanged (no recompilation needed)
> against
> > > new
> > > > > jars.
> > > > > + Client code written to APIs available in a given patch release
> > > > > + can run unchanged (no recompilation needed) against the new
> > > > > + jars of later patch versions.
> > > > >
> > > > > We can even claim that if you are not using new APIs, we are binary
> > > > compat
> > > > > for downgrade. But you have to make sure that your code compiles
> with
> > > > that
> > > > > downgraded version.
> > > > >
> > > > > Enis
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Apr 23, 2015 at 2:04 PM, Sean Busbey <bu...@cloudera.com>
> > > > wrote:
> > > > >
> > > > > > I'm fine with us adding methods to public APIs in patch releases
> so
> > > > long
> > > > > as
> > > > > > we stop claiming to follow semver. We can say we take the
> > principles
> > > of
> > > > > > semver as inspiration. This would reflect the current state of
> the
> > > > world
> > > > > > WRT 1.0.1 and would still give us a reason keep the narrower
> > > definition
> > > > > of
> > > > > > "new features" in minor versions.
> > > > > >
> > > > > >
> > > > > > No longer claiming semver would also have the added benefit of
> > making
> > > > it
> > > > > > for me to easier to explain our compatibility promises to people.
> > > Right
> > > > > now
> > > > > > I have to explain the difference between the things that get
> proper
> > > > > semver
> > > > > > treatment (e.g. public api, wire compat) and which things are
> > > > downgraded
> > > > > to
> > > > > > breaking-on-minor (e.g. LimitedPrivate, binary compatibility).
> > > Starting
> > > > > > with the context of "we like semver" will be easier than "we use
> > > > semver".
> > > > > >
> > > > > >
> > > > > > Like Josh, my main concern is that we accurately advertise what
> > we're
> > > > > > doing. There are few things I've found more frustrating than
> being
> > an
> > > > end
> > > > > > user of a project that claims to follow semver without
> > understanding
> > > > the
> > > > > > burden that carries (and subsequently not meeting it).
> > > > > >
> > > > > > On Thu, Apr 23, 2015 at 3:48 PM, Stack <st...@duboce.net> wrote:
> > > > > >
> > > > > > > I agree w/ the Enis characterization (so we need the callout on
> > > > semvar)
> > > > > > but
> > > > > > > think we should practice what Seans says (patch is bug fixes
> > only).
> > > > > > > St.Ack
> > > > > > >
> > > > > > > On Thu, Apr 23, 2015 at 1:31 PM, Sean Busbey <
> > busbey@cloudera.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > > Why don't we just focus development after a minor release on
> > the
> > > > next
> > > > > > > minor
> > > > > > > > release instead of the next patch release?
> > > > > > > >
> > > > > > > > We could limit backports to the patch releases to critical
> > bugs,
> > > > > which
> > > > > > > > would cut down on how often someone has to deal with the pain
> > of
> > > > > making
> > > > > > > > sure we don't add to public APIs. It also reduces the risk
> > > someone
> > > > > > going
> > > > > > > > through an upgrade has, since there are fewer changes.
> > > > > > > >
> > > > > > > > If someone fixes a bug and doesn't want to do the work of
> > making
> > > > sure
> > > > > > it
> > > > > > > > doesn't add methods in a patch release, they just don't
> > backport
> > > to
> > > > > > that
> > > > > > > > version and make a follow on e.g. "backport to 1.0.z" ticket.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Thu, Apr 23, 2015 at 1:50 PM, Enis Söztutar <
> > > enis.soz@gmail.com
> > > > >
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > > +1 to the proposal.
> > > > > > > > >
> > > > > > > > > The problem is that we have a very big API surface
> especially
> > > > with
> > > > > > the
> > > > > > > > > coprocessors included in the report. Even simple bug fixes
> > can
> > > > > > > introduce
> > > > > > > > > protected or public methods to base classes, which makes
> > patch
> > > > > > releases
> > > > > > > > > very hard to maintain. I would not want to spend the effort
> > to
> > > > > spend
> > > > > > > tons
> > > > > > > > > of time trying to make a patch not introduce new methods in
> > > order
> > > > > to
> > > > > > > > > backport. That effort can be spent elsewhere IMO.
> > > > > > > > >
> > > > > > > > > Looking at the report
> > > > > > > > >
> > > > https://people.apache.org/~enis/1.0.0_1.0.1RC2_compat_report.html,
> > > > > > > > nothing
> > > > > > > > > strikes me as "new functionality". Going from current 1.0.0
> > to
> > > > > > 1.0.1RC2
> > > > > > > > > should actually be as you would expect from upgrading a
> patch
> > > > > > release.
> > > > > > > > >
> > > > > > > > > Yes, adding new API in patch releases will make downgrading
> > > > harder,
> > > > > > > but I
> > > > > > > > > think that is an acceptable tradeoff. We can document that
> if
> > > > your
> > > > > > > > > application compiles (meaning that you are not using new
> API)
> > > > with
> > > > > > > 1.0.0,
> > > > > > > > > then you can swap your jars in a binary compat manner.
> > > > > > > > >
> > > > > > > > > Enis
> > > > > > > > >
> > > > > > > > > On Thu, Apr 23, 2015 at 10:03 AM, Andrew Purtell <
> > > > > > apurtell@apache.org>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Anyone disagree with the point of view put forward by
> Josh
> > > and
> > > > > > Sean?
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Wed, Apr 22, 2015 at 10:35 AM, Josh Elser <
> > > > > josh.elser@gmail.com
> > > > > > >
> > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Andy -- I understood your intent, but thanks for
> > > clarifying.
> > > > > (as
> > > > > > > well
> > > > > > > > > as
> > > > > > > > > > > taking the time to break this discussion out in the
> first
> > > > > > place). I
> > > > > > > > > agree
> > > > > > > > > > > with your assessment.
> > > > > > > > > > >
> > > > > > > > > > > re: Sean's comments, if it wasn't clear by me asking in
> > the
> > > > > first
> > > > > > > > > place,
> > > > > > > > > > I
> > > > > > > > > > > also think sticking as close as possible to semver's
> > rules
> > > is
> > > > > the
> > > > > > > > best
> > > > > > > > > > > approach, although I'm getting the impression that
> there
> > > have
> > > > > > been
> > > > > > > > some
> > > > > > > > > > > previous reservations to doing so (especially by your
> > > comment
> > > > > > about
> > > > > > > > > > > backporting features if there is demand is).
> > > > > > > > > > >
> > > > > > > > > > > I've found adhering to the bug-fix release restrictions
> > can
> > > > be
> > > > > a
> > > > > > > very
> > > > > > > > > > > painful and time-consuming task, so this is something
> to
> > > get
> > > > a
> > > > > > > > > > > representative sampling of those who do the work to
> make
> > > sure
> > > > > > > > everyone
> > > > > > > > > is
> > > > > > > > > > > on board.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Sean Busbey wrote:
> > > > > > > > > > >
> > > > > > > > > > >> I'd much rather we stick with the definitions used in
> > > > Semantic
> > > > > > > > > > Versioning.
> > > > > > > > > > >> Our use is already confusing enough given our matrix
> of
> > > > > > > > > compatibilities
> > > > > > > > > > >> that don't get "major version for breaking"
> protections.
> > > > > > > > > > >>
> > > > > > > > > > >> We've previously discussed how we'll do additional
> minor
> > > > > > releases
> > > > > > > > when
> > > > > > > > > > >> there's sufficient interest in the new features
> present
> > > > there.
> > > > > > > > What's
> > > > > > > > > > >> building that demand if any backwards compatible
> change
> > > can
> > > > go
> > > > > > > back
> > > > > > > > > > into a
> > > > > > > > > > >> patch release?
> > > > > > > > > > >>
> > > > > > > > > > >> Would we have an easier time restraining ourselves if
> we
> > > > had a
> > > > > > > > regular
> > > > > > > > > > >> schedule planned around new minor versions?
> > > > > > > > > > >>
> > > > > > > > > > >>
> > > > > > > > > > >> On Wed, Apr 22, 2015 at 12:03 PM, Josh Elser<
> > > > > > josh.elser@gmail.com
> > > > > > > >
> > > > > > > > > > >> wrote:
> > > > > > > > > > >>
> > > > > > > > > > >> While I can understand the desire to want to add
> > things,
> > > I
> > > > do
> > > > > > > think
> > > > > > > > > it
> > > > > > > > > > >>> makes things harder for users to reliably write code
> > > > against
> > > > > > > > versions
> > > > > > > > > > of
> > > > > > > > > > >>> HBase which (by their view) should be completely
> > > compatible
> > > > > > with
> > > > > > > > one
> > > > > > > > > > >>> another.
> > > > > > > > > > >>>
> > > > > > > > > > >>> Take this extremely hypothetical situation: I'm new
> to
> > > > HBase
> > > > > > and
> > > > > > > > > start
> > > > > > > > > > >>> writing some code against HBase 1.0.1 which was just
> > > > deployed
> > > > > > at
> > > > > > > my
> > > > > > > > > > >>> $job. I
> > > > > > > > > > >>> don't _know_ what APIs are new, I just know what
> exists
> > > and
> > > > > > treat
> > > > > > > > > that
> > > > > > > > > > as
> > > > > > > > > > >>> acceptable for me to be using. Meanwhile in
> production,
> > > > some
> > > > > > > other
> > > > > > > > > > people
> > > > > > > > > > >>> find a bug with HBase 1.0.1 and roll back to 1.0.0
> > which
> > > > they
> > > > > > had
> > > > > > > > > been
> > > > > > > > > > >>> previously using. My reaction would be "of course my
> > code
> > > > > > should
> > > > > > > > work
> > > > > > > > > > >>> with
> > > > > > > > > > >>> HBase 1.0.0, I only used the public API" when in fact
> > > this
> > > > is
> > > > > > not
> > > > > > > > > true.
> > > > > > > > > > >>>
> > > > > > > > > > >>> Personally, I think it's a little bold to say semver
> is
> > > > even
> > > > > in
> > > > > > > use
> > > > > > > > > if
> > > > > > > > > > >>> this principal isn't being followed as it doesn't
> > follow
> > > at
> > > > > all
> > > > > > > > with
> > > > > > > > > my
> > > > > > > > > > >>> understanding on the guarantees defined by semver for
> > > > bug-fix
> > > > > > > > > releases.
> > > > > > > > > > >>>
> > > > > > > > > > >>> That being said, if the intent *is* to allow
> ourselves
> > to
> > > > > make
> > > > > > > > these
> > > > > > > > > > >>> sorts
> > > > > > > > > > >>> of changes, I just think some sort of disclaimer
> should
> > > be
> > > > > > > present:
> > > > > > > > > > >>>
> > > > > > > > > > >>> - HBase uses Semantic Versioning for its release
> > > versioning
> > > > > > > > > > >>> + HBase uses Semantic Versioning for its release
> > > versioning
> > > > > > with
> > > > > > > a
> > > > > > > > > > caveat
> > > > > > > > > > >>> that methods and members might be added in newer
> > bug-fix
> > > > > > releases
> > > > > > > > > that
> > > > > > > > > > >>> were
> > > > > > > > > > >>> not present in the previous bug-fix release.
> > > > > > > > > > >>>
> > > > > > > > > > >>>
> > > > > > > > > > >>> Andrew Purtell wrote:
> > > > > > > > > > >>>
> > > > > > > > > > >>> [Subject changed]
> > > > > > > > > > >>>>
> > > > > > > > > > >>>> On Tue, Apr 21, 2015 at 8:47 PM, Josh Elser<
> > > > > > > josh.elser@gmail.com>
> > > > > > > > > > >>>> wrote:
> > > > > > > > > > >>>>
> > > > > > > > > > >>>> I was a little surprised when I noticed method
> > > additions
> > > > > to
> > > > > > > > > > >>>>
> > > > > > > > > > >>>>> InterfaceAudience.Public annotated classes. This
> > means
> > > > > that a
> > > > > > > > user
> > > > > > > > > > >>>>> could
> > > > > > > > > > >>>>> write code against 1.0.1 that would not work
> against
> > > > 1.0.0
> > > > > > > which
> > > > > > > > > > seems
> > > > > > > > > > >>>>> undesirable for a bugfix release. I read over the
> > book
> > > > > > section
> > > > > > > on
> > > > > > > > > > >>>>> compatibility and didn't see this addressed, so I
> > > thought
> > > > > I'd
> > > > > > > > ask.
> > > > > > > > > > >>>>>
> > > > > > > > > > >>>>>
> > > > > > > > > > >>>> Let's clarify this. It's not the first time this
> > > question
> > > > > has
> > > > > > > been
> > > > > > > > > > >>>> asked.
> > > > > > > > > > >>>>
> > > > > > > > > > >>>> To get things moving:
> > > > > > > > > > >>>>
> > > > > > > > > > >>>> I propose the following addition to the "Client API
> > > > > > > compatibility"
> > > > > > > > > > >>>> section
> > > > > > > > > > >>>> of Section 11.1:
> > > > > > > > > > >>>>
> > > > > > > > > > >>>> + APIs available in a patch version will be
> available
> > in
> > > > all
> > > > > > > later
> > > > > > > > > > >>>> + patch versions. However, new APIs may be added
> which
> > > > will
> > > > > > not
> > > > > > > be
> > > > > > > > > > >>>> + available in earlier patch versions.
> > > > > > > > > > >>>>
> > > > > > > > > > >>>> I propose the following change to the "Client Binary
> > > > > > > > compatibility"
> > > > > > > > > > >>>> section
> > > > > > > > > > >>>> of Section 11.1:
> > > > > > > > > > >>>>
> > > > > > > > > > >>>> - Old client code can run unchanged (no
> recompilation
> > > > > needed)
> > > > > > > > > against
> > > > > > > > > > >>>> new
> > > > > > > > > > >>>> jars.
> > > > > > > > > > >>>> + Client code written to APIs available in a given
> > patch
> > > > > > release
> > > > > > > > > > >>>> + can run unchanged (no recompilation needed)
> against
> > > the
> > > > > new
> > > > > > > > > > >>>> + jars of later patch versions.
> > > > > > > > > > >>>>
> > > > > > > > > > >>>>
> > > > > > > > > > >>>> What do you think?
> > > > > > > > > > >>>>
> > > > > > > > > > >>>> If these changes are (mostly) ok, then this
> clarifies
> > in
> > > > one
> > > > > > > > > > direction.
> > > > > > > > > > >>>>
> > > > > > > > > > >>>> If these changes are not acceptable, I will propose
> > > edits
> > > > > that
> > > > > > > > > clarify
> > > > > > > > > > >>>> toward the opposite meaning.
> > > > > > > > > > >>>>
> > > > >
> > > >
> > >
> >
>
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Stack <st...@duboce.net>.
On Fri, Apr 24, 2015 at 10:25 AM, Sean Busbey <bu...@cloudera.com> wrote:
> On Fri, Apr 24, 2015 at 12:21 PM, Stack <st...@duboce.net> wrote:
>
> > On Fri, Apr 24, 2015 at 9:21 AM, Josh Elser <jo...@gmail.com>
> wrote:
> >
> > > Cool, it seems that we have consensus. Let me try to give a patch since
> > > you all were nice enough to entertain me with a good discussion on the
> > > matter!
> > >
> > > https://issues.apache.org/jira/browse/HBASE-13554
> > >
> > >
> > >
> > Thanks for filing the issue.
> >
> > Do we want to say SemVer in 2.0+?
> >
> > St.Ack
> >
> >
> Let's see how we do on establishing release cadence on 1+ and maintaining
> our existing compatibility promises before we set goals on how strict we'll
> be on versions in 2+. :)
>
>
>
Fair enough.
St.Ack
> --
> Sean
>
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Sean Busbey <bu...@cloudera.com>.
On Fri, Apr 24, 2015 at 12:21 PM, Stack <st...@duboce.net> wrote:
> On Fri, Apr 24, 2015 at 9:21 AM, Josh Elser <jo...@gmail.com> wrote:
>
> > Cool, it seems that we have consensus. Let me try to give a patch since
> > you all were nice enough to entertain me with a good discussion on the
> > matter!
> >
> > https://issues.apache.org/jira/browse/HBASE-13554
> >
> >
> >
> Thanks for filing the issue.
>
> Do we want to say SemVer in 2.0+?
>
> St.Ack
>
>
Let's see how we do on establishing release cadence on 1+ and maintaining
our existing compatibility promises before we set goals on how strict we'll
be on versions in 2+. :)
--
Sean
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Stack <st...@duboce.net>.
On Fri, Apr 24, 2015 at 9:21 AM, Josh Elser <jo...@gmail.com> wrote:
> Cool, it seems that we have consensus. Let me try to give a patch since
> you all were nice enough to entertain me with a good discussion on the
> matter!
>
> https://issues.apache.org/jira/browse/HBASE-13554
>
>
>
Thanks for filing the issue.
Do we want to say SemVer in 2.0+?
St.Ack
> Enis Söztutar wrote:
>
>> +1 to "working toward semver". Is there any documentation that we would
>> like to clarify in the book given this enlightenment?
>>
>> I would be in favor of going 2.6.0 and jackson upgrade in 1.1.
>>
>> Enis
>>
>> On Thu, Apr 23, 2015 at 5:34 PM, Nick Dimiduk<nd...@gmail.com> wrote:
>>
>> On Thu, Apr 23, 2015 at 4:13 PM, Stack<st...@duboce.net> wrote:
>>>
>>> Does this 'admission' help with the which-hadoop thread too?
>>>>
>>>> Right -- "working toward semver".
>>>
>>> Are we now at liberty to bump to 2.6 or 2.7 even for branch-1.1? I still
>>> have Karthik's offer to roll a 2.5.3 with the HDFS issue resolved.
>>>
>>> What about the jackson issue with 2.5 YARN runtime?
>>>
>>> Thanks Sean and Josh for being our community conscience on these issues.
>>>
>>> Just want to make sure we're all on the same page (or find out if not).
>>>>
>>>>> On Thu, Apr 23, 2015 at 2:59 PM, Enis Söztutar<en...@apache.org>
>>>>>
>>>> wrote:
>>>
>>>> Then let's get Andrew's proposal committed:
>>>>>>
>>>>>> + APIs available in a patch version will be available in all later
>>>>>> + patch versions. However, new APIs may be added which will not be
>>>>>> + available in earlier patch versions.
>>>>>>
>>>>>> I propose the following change to the "Client Binary compatibility"
>>>>>>
>>>>> section
>>>>>
>>>>>> of Section 11.1:
>>>>>>
>>>>>> - Old client code can run unchanged (no recompilation needed) against
>>>>>>
>>>>> new
>>>>
>>>>> jars.
>>>>>> + Client code written to APIs available in a given patch release
>>>>>> + can run unchanged (no recompilation needed) against the new
>>>>>> + jars of later patch versions.
>>>>>>
>>>>>> We can even claim that if you are not using new APIs, we are binary
>>>>>>
>>>>> compat
>>>>>
>>>>>> for downgrade. But you have to make sure that your code compiles with
>>>>>>
>>>>> that
>>>>>
>>>>>> downgraded version.
>>>>>>
>>>>>> Enis
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Apr 23, 2015 at 2:04 PM, Sean Busbey<bu...@cloudera.com>
>>>>>>
>>>>> wrote:
>>>>>
>>>>>> I'm fine with us adding methods to public APIs in patch releases so
>>>>>>>
>>>>>> long
>>>>>
>>>>>> as
>>>>>>
>>>>>>> we stop claiming to follow semver. We can say we take the
>>>>>>>
>>>>>> principles
>>>
>>>> of
>>>>
>>>>> semver as inspiration. This would reflect the current state of the
>>>>>>>
>>>>>> world
>>>>>
>>>>>> WRT 1.0.1 and would still give us a reason keep the narrower
>>>>>>>
>>>>>> definition
>>>>
>>>>> of
>>>>>>
>>>>>>> "new features" in minor versions.
>>>>>>>
>>>>>>>
>>>>>>> No longer claiming semver would also have the added benefit of
>>>>>>>
>>>>>> making
>>>
>>>> it
>>>>>
>>>>>> for me to easier to explain our compatibility promises to people.
>>>>>>>
>>>>>> Right
>>>>
>>>>> now
>>>>>>
>>>>>>> I have to explain the difference between the things that get proper
>>>>>>>
>>>>>> semver
>>>>>>
>>>>>>> treatment (e.g. public api, wire compat) and which things are
>>>>>>>
>>>>>> downgraded
>>>>>
>>>>>> to
>>>>>>
>>>>>>> breaking-on-minor (e.g. LimitedPrivate, binary compatibility).
>>>>>>>
>>>>>> Starting
>>>>
>>>>> with the context of "we like semver" will be easier than "we use
>>>>>>>
>>>>>> semver".
>>>>>
>>>>>>
>>>>>>> Like Josh, my main concern is that we accurately advertise what
>>>>>>>
>>>>>> we're
>>>
>>>> doing. There are few things I've found more frustrating than being
>>>>>>>
>>>>>> an
>>>
>>>> end
>>>>>
>>>>>> user of a project that claims to follow semver without
>>>>>>>
>>>>>> understanding
>>>
>>>> the
>>>>>
>>>>>> burden that carries (and subsequently not meeting it).
>>>>>>>
>>>>>>> On Thu, Apr 23, 2015 at 3:48 PM, Stack<st...@duboce.net> wrote:
>>>>>>>
>>>>>>> I agree w/ the Enis characterization (so we need the callout on
>>>>>>>>
>>>>>>> semvar)
>>>>>
>>>>>> but
>>>>>>>
>>>>>>>> think we should practice what Seans says (patch is bug fixes
>>>>>>>>
>>>>>>> only).
>>>
>>>> St.Ack
>>>>>>>>
>>>>>>>> On Thu, Apr 23, 2015 at 1:31 PM, Sean Busbey<
>>>>>>>>
>>>>>>> busbey@cloudera.com>
>>>
>>>> wrote:
>>>>>>>
>>>>>>>> Why don't we just focus development after a minor release on
>>>>>>>>>
>>>>>>>> the
>>>
>>>> next
>>>>>
>>>>>> minor
>>>>>>>>
>>>>>>>>> release instead of the next patch release?
>>>>>>>>>
>>>>>>>>> We could limit backports to the patch releases to critical
>>>>>>>>>
>>>>>>>> bugs,
>>>
>>>> which
>>>>>>
>>>>>>> would cut down on how often someone has to deal with the pain
>>>>>>>>>
>>>>>>>> of
>>>
>>>> making
>>>>>>
>>>>>>> sure we don't add to public APIs. It also reduces the risk
>>>>>>>>>
>>>>>>>> someone
>>>>
>>>>> going
>>>>>>>
>>>>>>>> through an upgrade has, since there are fewer changes.
>>>>>>>>>
>>>>>>>>> If someone fixes a bug and doesn't want to do the work of
>>>>>>>>>
>>>>>>>> making
>>>
>>>> sure
>>>>>
>>>>>> it
>>>>>>>
>>>>>>>> doesn't add methods in a patch release, they just don't
>>>>>>>>>
>>>>>>>> backport
>>>
>>>> to
>>>>
>>>>> that
>>>>>>>
>>>>>>>> version and make a follow on e.g. "backport to 1.0.z" ticket.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Apr 23, 2015 at 1:50 PM, Enis Söztutar<
>>>>>>>>>
>>>>>>>> enis.soz@gmail.com
>>>>
>>>>> wrote:
>>>>>>>>
>>>>>>>>> +1 to the proposal.
>>>>>>>>>>
>>>>>>>>>> The problem is that we have a very big API surface especially
>>>>>>>>>>
>>>>>>>>> with
>>>>>
>>>>>> the
>>>>>>>
>>>>>>>> coprocessors included in the report. Even simple bug fixes
>>>>>>>>>>
>>>>>>>>> can
>>>
>>>> introduce
>>>>>>>>
>>>>>>>>> protected or public methods to base classes, which makes
>>>>>>>>>>
>>>>>>>>> patch
>>>
>>>> releases
>>>>>>>
>>>>>>>> very hard to maintain. I would not want to spend the effort
>>>>>>>>>>
>>>>>>>>> to
>>>
>>>> spend
>>>>>>
>>>>>>> tons
>>>>>>>>
>>>>>>>>> of time trying to make a patch not introduce new methods in
>>>>>>>>>>
>>>>>>>>> order
>>>>
>>>>> to
>>>>>>
>>>>>>> backport. That effort can be spent elsewhere IMO.
>>>>>>>>>>
>>>>>>>>>> Looking at the report
>>>>>>>>>>
>>>>>>>>>>
>>>>> https://people.apache.org/~enis/1.0.0_1.0.1RC2_compat_report.html,
>>>>>
>>>>>> nothing
>>>>>>>>>
>>>>>>>>>> strikes me as "new functionality". Going from current 1.0.0
>>>>>>>>>>
>>>>>>>>> to
>>>
>>>> 1.0.1RC2
>>>>>>>
>>>>>>>> should actually be as you would expect from upgrading a patch
>>>>>>>>>>
>>>>>>>>> release.
>>>>>>>
>>>>>>>> Yes, adding new API in patch releases will make downgrading
>>>>>>>>>>
>>>>>>>>> harder,
>>>>>
>>>>>> but I
>>>>>>>>
>>>>>>>>> think that is an acceptable tradeoff. We can document that if
>>>>>>>>>>
>>>>>>>>> your
>>>>>
>>>>>> application compiles (meaning that you are not using new API)
>>>>>>>>>>
>>>>>>>>> with
>>>>>
>>>>>> 1.0.0,
>>>>>>>>
>>>>>>>>> then you can swap your jars in a binary compat manner.
>>>>>>>>>>
>>>>>>>>>> Enis
>>>>>>>>>>
>>>>>>>>>> On Thu, Apr 23, 2015 at 10:03 AM, Andrew Purtell<
>>>>>>>>>>
>>>>>>>>> apurtell@apache.org>
>>>>>>>
>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Anyone disagree with the point of view put forward by Josh
>>>>>>>>>>>
>>>>>>>>>> and
>>>>
>>>>> Sean?
>>>>>>>
>>>>>>>>
>>>>>>>>>>> On Wed, Apr 22, 2015 at 10:35 AM, Josh Elser<
>>>>>>>>>>>
>>>>>>>>>> josh.elser@gmail.com
>>>>>>
>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Andy -- I understood your intent, but thanks for
>>>>>>>>>>>>
>>>>>>>>>>> clarifying.
>>>>
>>>>> (as
>>>>>>
>>>>>>> well
>>>>>>>>
>>>>>>>>> as
>>>>>>>>>>
>>>>>>>>>>> taking the time to break this discussion out in the first
>>>>>>>>>>>>
>>>>>>>>>>> place). I
>>>>>>>
>>>>>>>> agree
>>>>>>>>>>
>>>>>>>>>>> with your assessment.
>>>>>>>>>>>>
>>>>>>>>>>>> re: Sean's comments, if it wasn't clear by me asking in
>>>>>>>>>>>>
>>>>>>>>>>> the
>>>
>>>> first
>>>>>>
>>>>>>> place,
>>>>>>>>>>
>>>>>>>>>>> I
>>>>>>>>>>>
>>>>>>>>>>>> also think sticking as close as possible to semver's
>>>>>>>>>>>>
>>>>>>>>>>> rules
>>>
>>>> is
>>>>
>>>>> the
>>>>>>
>>>>>>> best
>>>>>>>>>
>>>>>>>>>> approach, although I'm getting the impression that there
>>>>>>>>>>>>
>>>>>>>>>>> have
>>>>
>>>>> been
>>>>>>>
>>>>>>>> some
>>>>>>>>>
>>>>>>>>>> previous reservations to doing so (especially by your
>>>>>>>>>>>>
>>>>>>>>>>> comment
>>>>
>>>>> about
>>>>>>>
>>>>>>>> backporting features if there is demand is).
>>>>>>>>>>>>
>>>>>>>>>>>> I've found adhering to the bug-fix release restrictions
>>>>>>>>>>>>
>>>>>>>>>>> can
>>>
>>>> be
>>>>>
>>>>>> a
>>>>>>
>>>>>>> very
>>>>>>>>
>>>>>>>>> painful and time-consuming task, so this is something to
>>>>>>>>>>>>
>>>>>>>>>>> get
>>>>
>>>>> a
>>>>>
>>>>>> representative sampling of those who do the work to make
>>>>>>>>>>>>
>>>>>>>>>>> sure
>>>>
>>>>> everyone
>>>>>>>>>
>>>>>>>>>> is
>>>>>>>>>>
>>>>>>>>>>> on board.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Sean Busbey wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> I'd much rather we stick with the definitions used in
>>>>>>>>>>>>>
>>>>>>>>>>>> Semantic
>>>>>
>>>>>> Versioning.
>>>>>>>>>>>
>>>>>>>>>>>> Our use is already confusing enough given our matrix of
>>>>>>>>>>>>>
>>>>>>>>>>>> compatibilities
>>>>>>>>>>
>>>>>>>>>>> that don't get "major version for breaking" protections.
>>>>>>>>>>>>>
>>>>>>>>>>>>> We've previously discussed how we'll do additional minor
>>>>>>>>>>>>>
>>>>>>>>>>>> releases
>>>>>>>
>>>>>>>> when
>>>>>>>>>
>>>>>>>>>> there's sufficient interest in the new features present
>>>>>>>>>>>>>
>>>>>>>>>>>> there.
>>>>>
>>>>>> What's
>>>>>>>>>
>>>>>>>>>> building that demand if any backwards compatible change
>>>>>>>>>>>>>
>>>>>>>>>>>> can
>>>>
>>>>> go
>>>>>
>>>>>> back
>>>>>>>>
>>>>>>>>> into a
>>>>>>>>>>>
>>>>>>>>>>>> patch release?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Would we have an easier time restraining ourselves if we
>>>>>>>>>>>>>
>>>>>>>>>>>> had a
>>>>>
>>>>>> regular
>>>>>>>>>
>>>>>>>>>> schedule planned around new minor versions?
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Apr 22, 2015 at 12:03 PM, Josh Elser<
>>>>>>>>>>>>>
>>>>>>>>>>>> josh.elser@gmail.com
>>>>>>>
>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> While I can understand the desire to want to add
>>>>>>>>>>>>>
>>>>>>>>>>>> things,
>>>
>>>> I
>>>>
>>>>> do
>>>>>
>>>>>> think
>>>>>>>>
>>>>>>>>> it
>>>>>>>>>>
>>>>>>>>>>> makes things harder for users to reliably write code
>>>>>>>>>>>>>>
>>>>>>>>>>>>> against
>>>>>
>>>>>> versions
>>>>>>>>>
>>>>>>>>>> of
>>>>>>>>>>>
>>>>>>>>>>>> HBase which (by their view) should be completely
>>>>>>>>>>>>>>
>>>>>>>>>>>>> compatible
>>>>
>>>>> with
>>>>>>>
>>>>>>>> one
>>>>>>>>>
>>>>>>>>>> another.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Take this extremely hypothetical situation: I'm new to
>>>>>>>>>>>>>>
>>>>>>>>>>>>> HBase
>>>>>
>>>>>> and
>>>>>>>
>>>>>>>> start
>>>>>>>>>>
>>>>>>>>>>> writing some code against HBase 1.0.1 which was just
>>>>>>>>>>>>>>
>>>>>>>>>>>>> deployed
>>>>>
>>>>>> at
>>>>>>>
>>>>>>>> my
>>>>>>>>
>>>>>>>>> $job. I
>>>>>>>>>>>>>> don't _know_ what APIs are new, I just know what exists
>>>>>>>>>>>>>>
>>>>>>>>>>>>> and
>>>>
>>>>> treat
>>>>>>>
>>>>>>>> that
>>>>>>>>>>
>>>>>>>>>>> as
>>>>>>>>>>>
>>>>>>>>>>>> acceptable for me to be using. Meanwhile in production,
>>>>>>>>>>>>>>
>>>>>>>>>>>>> some
>>>>>
>>>>>> other
>>>>>>>>
>>>>>>>>> people
>>>>>>>>>>>
>>>>>>>>>>>> find a bug with HBase 1.0.1 and roll back to 1.0.0
>>>>>>>>>>>>>>
>>>>>>>>>>>>> which
>>>
>>>> they
>>>>>
>>>>>> had
>>>>>>>
>>>>>>>> been
>>>>>>>>>>
>>>>>>>>>>> previously using. My reaction would be "of course my
>>>>>>>>>>>>>>
>>>>>>>>>>>>> code
>>>
>>>> should
>>>>>>>
>>>>>>>> work
>>>>>>>>>
>>>>>>>>>> with
>>>>>>>>>>>>>> HBase 1.0.0, I only used the public API" when in fact
>>>>>>>>>>>>>>
>>>>>>>>>>>>> this
>>>>
>>>>> is
>>>>>
>>>>>> not
>>>>>>>
>>>>>>>> true.
>>>>>>>>>>
>>>>>>>>>>> Personally, I think it's a little bold to say semver is
>>>>>>>>>>>>>>
>>>>>>>>>>>>> even
>>>>>
>>>>>> in
>>>>>>
>>>>>>> use
>>>>>>>>
>>>>>>>>> if
>>>>>>>>>>
>>>>>>>>>>> this principal isn't being followed as it doesn't
>>>>>>>>>>>>>>
>>>>>>>>>>>>> follow
>>>
>>>> at
>>>>
>>>>> all
>>>>>>
>>>>>>> with
>>>>>>>>>
>>>>>>>>>> my
>>>>>>>>>>
>>>>>>>>>>> understanding on the guarantees defined by semver for
>>>>>>>>>>>>>>
>>>>>>>>>>>>> bug-fix
>>>>>
>>>>>> releases.
>>>>>>>>>>
>>>>>>>>>>> That being said, if the intent *is* to allow ourselves
>>>>>>>>>>>>>>
>>>>>>>>>>>>> to
>>>
>>>> make
>>>>>>
>>>>>>> these
>>>>>>>>>
>>>>>>>>>> sorts
>>>>>>>>>>>>>> of changes, I just think some sort of disclaimer should
>>>>>>>>>>>>>>
>>>>>>>>>>>>> be
>>>>
>>>>> present:
>>>>>>>>
>>>>>>>>> - HBase uses Semantic Versioning for its release
>>>>>>>>>>>>>>
>>>>>>>>>>>>> versioning
>>>>
>>>>> + HBase uses Semantic Versioning for its release
>>>>>>>>>>>>>>
>>>>>>>>>>>>> versioning
>>>>
>>>>> with
>>>>>>>
>>>>>>>> a
>>>>>>>>
>>>>>>>>> caveat
>>>>>>>>>>>
>>>>>>>>>>>> that methods and members might be added in newer
>>>>>>>>>>>>>>
>>>>>>>>>>>>> bug-fix
>>>
>>>> releases
>>>>>>>
>>>>>>>> that
>>>>>>>>>>
>>>>>>>>>>> were
>>>>>>>>>>>>>> not present in the previous bug-fix release.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Andrew Purtell wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [Subject changed]
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, Apr 21, 2015 at 8:47 PM, Josh Elser<
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> josh.elser@gmail.com>
>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I was a little surprised when I noticed method
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> additions
>>>>
>>>>> to
>>>>>>
>>>>>>> InterfaceAudience.Public annotated classes. This
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> means
>>>
>>>> that a
>>>>>>
>>>>>>> user
>>>>>>>>>
>>>>>>>>>> could
>>>>>>>>>>>>>>>> write code against 1.0.1 that would not work against
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 1.0.0
>>>>>
>>>>>> which
>>>>>>>>
>>>>>>>>> seems
>>>>>>>>>>>
>>>>>>>>>>>> undesirable for a bugfix release. I read over the
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> book
>>>
>>>> section
>>>>>>>
>>>>>>>> on
>>>>>>>>
>>>>>>>>> compatibility and didn't see this addressed, so I
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> thought
>>>>
>>>>> I'd
>>>>>>
>>>>>>> ask.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>>>>>> Let's clarify this. It's not the first time this
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> question
>>>>
>>>>> has
>>>>>>
>>>>>>> been
>>>>>>>>
>>>>>>>>> asked.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> To get things moving:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I propose the following addition to the "Client API
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> compatibility"
>>>>>>>>
>>>>>>>>> section
>>>>>>>>>>>>>>> of Section 11.1:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> + APIs available in a patch version will be available
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> in
>>>
>>>> all
>>>>>
>>>>>> later
>>>>>>>>
>>>>>>>>> + patch versions. However, new APIs may be added which
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> will
>>>>>
>>>>>> not
>>>>>>>
>>>>>>>> be
>>>>>>>>
>>>>>>>>> + available in earlier patch versions.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I propose the following change to the "Client Binary
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> compatibility"
>>>>>>>>>
>>>>>>>>>> section
>>>>>>>>>>>>>>> of Section 11.1:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - Old client code can run unchanged (no recompilation
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> needed)
>>>>>>
>>>>>>> against
>>>>>>>>>>
>>>>>>>>>>> new
>>>>>>>>>>>>>>> jars.
>>>>>>>>>>>>>>> + Client code written to APIs available in a given
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> patch
>>>
>>>> release
>>>>>>>
>>>>>>>> + can run unchanged (no recompilation needed) against
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> the
>>>>
>>>>> new
>>>>>>
>>>>>>> + jars of later patch versions.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> If these changes are (mostly) ok, then this clarifies
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> in
>>>
>>>> one
>>>>>
>>>>>> direction.
>>>>>>>>>>>
>>>>>>>>>>>> If these changes are not acceptable, I will propose
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> edits
>>>>
>>>>> that
>>>>>>
>>>>>>> clarify
>>>>>>>>>>
>>>>>>>>>>> toward the opposite meaning.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Josh Elser <jo...@gmail.com>.
Cool, it seems that we have consensus. Let me try to give a patch since
you all were nice enough to entertain me with a good discussion on the
matter!
https://issues.apache.org/jira/browse/HBASE-13554
Enis Söztutar wrote:
> +1 to "working toward semver". Is there any documentation that we would
> like to clarify in the book given this enlightenment?
>
> I would be in favor of going 2.6.0 and jackson upgrade in 1.1.
>
> Enis
>
> On Thu, Apr 23, 2015 at 5:34 PM, Nick Dimiduk<nd...@gmail.com> wrote:
>
>> On Thu, Apr 23, 2015 at 4:13 PM, Stack<st...@duboce.net> wrote:
>>
>>> Does this 'admission' help with the which-hadoop thread too?
>>>
>> Right -- "working toward semver".
>>
>> Are we now at liberty to bump to 2.6 or 2.7 even for branch-1.1? I still
>> have Karthik's offer to roll a 2.5.3 with the HDFS issue resolved.
>>
>> What about the jackson issue with 2.5 YARN runtime?
>>
>> Thanks Sean and Josh for being our community conscience on these issues.
>>
>>> Just want to make sure we're all on the same page (or find out if not).
>>>> On Thu, Apr 23, 2015 at 2:59 PM, Enis Söztutar<en...@apache.org>
>> wrote:
>>>>> Then let's get Andrew's proposal committed:
>>>>>
>>>>> + APIs available in a patch version will be available in all later
>>>>> + patch versions. However, new APIs may be added which will not be
>>>>> + available in earlier patch versions.
>>>>>
>>>>> I propose the following change to the "Client Binary compatibility"
>>>> section
>>>>> of Section 11.1:
>>>>>
>>>>> - Old client code can run unchanged (no recompilation needed) against
>>> new
>>>>> jars.
>>>>> + Client code written to APIs available in a given patch release
>>>>> + can run unchanged (no recompilation needed) against the new
>>>>> + jars of later patch versions.
>>>>>
>>>>> We can even claim that if you are not using new APIs, we are binary
>>>> compat
>>>>> for downgrade. But you have to make sure that your code compiles with
>>>> that
>>>>> downgraded version.
>>>>>
>>>>> Enis
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Apr 23, 2015 at 2:04 PM, Sean Busbey<bu...@cloudera.com>
>>>> wrote:
>>>>>> I'm fine with us adding methods to public APIs in patch releases so
>>>> long
>>>>> as
>>>>>> we stop claiming to follow semver. We can say we take the
>> principles
>>> of
>>>>>> semver as inspiration. This would reflect the current state of the
>>>> world
>>>>>> WRT 1.0.1 and would still give us a reason keep the narrower
>>> definition
>>>>> of
>>>>>> "new features" in minor versions.
>>>>>>
>>>>>>
>>>>>> No longer claiming semver would also have the added benefit of
>> making
>>>> it
>>>>>> for me to easier to explain our compatibility promises to people.
>>> Right
>>>>> now
>>>>>> I have to explain the difference between the things that get proper
>>>>> semver
>>>>>> treatment (e.g. public api, wire compat) and which things are
>>>> downgraded
>>>>> to
>>>>>> breaking-on-minor (e.g. LimitedPrivate, binary compatibility).
>>> Starting
>>>>>> with the context of "we like semver" will be easier than "we use
>>>> semver".
>>>>>>
>>>>>> Like Josh, my main concern is that we accurately advertise what
>> we're
>>>>>> doing. There are few things I've found more frustrating than being
>> an
>>>> end
>>>>>> user of a project that claims to follow semver without
>> understanding
>>>> the
>>>>>> burden that carries (and subsequently not meeting it).
>>>>>>
>>>>>> On Thu, Apr 23, 2015 at 3:48 PM, Stack<st...@duboce.net> wrote:
>>>>>>
>>>>>>> I agree w/ the Enis characterization (so we need the callout on
>>>> semvar)
>>>>>> but
>>>>>>> think we should practice what Seans says (patch is bug fixes
>> only).
>>>>>>> St.Ack
>>>>>>>
>>>>>>> On Thu, Apr 23, 2015 at 1:31 PM, Sean Busbey<
>> busbey@cloudera.com>
>>>>>> wrote:
>>>>>>>> Why don't we just focus development after a minor release on
>> the
>>>> next
>>>>>>> minor
>>>>>>>> release instead of the next patch release?
>>>>>>>>
>>>>>>>> We could limit backports to the patch releases to critical
>> bugs,
>>>>> which
>>>>>>>> would cut down on how often someone has to deal with the pain
>> of
>>>>> making
>>>>>>>> sure we don't add to public APIs. It also reduces the risk
>>> someone
>>>>>> going
>>>>>>>> through an upgrade has, since there are fewer changes.
>>>>>>>>
>>>>>>>> If someone fixes a bug and doesn't want to do the work of
>> making
>>>> sure
>>>>>> it
>>>>>>>> doesn't add methods in a patch release, they just don't
>> backport
>>> to
>>>>>> that
>>>>>>>> version and make a follow on e.g. "backport to 1.0.z" ticket.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Apr 23, 2015 at 1:50 PM, Enis Söztutar<
>>> enis.soz@gmail.com
>>>>>>> wrote:
>>>>>>>>> +1 to the proposal.
>>>>>>>>>
>>>>>>>>> The problem is that we have a very big API surface especially
>>>> with
>>>>>> the
>>>>>>>>> coprocessors included in the report. Even simple bug fixes
>> can
>>>>>>> introduce
>>>>>>>>> protected or public methods to base classes, which makes
>> patch
>>>>>> releases
>>>>>>>>> very hard to maintain. I would not want to spend the effort
>> to
>>>>> spend
>>>>>>> tons
>>>>>>>>> of time trying to make a patch not introduce new methods in
>>> order
>>>>> to
>>>>>>>>> backport. That effort can be spent elsewhere IMO.
>>>>>>>>>
>>>>>>>>> Looking at the report
>>>>>>>>>
>>>> https://people.apache.org/~enis/1.0.0_1.0.1RC2_compat_report.html,
>>>>>>>> nothing
>>>>>>>>> strikes me as "new functionality". Going from current 1.0.0
>> to
>>>>>> 1.0.1RC2
>>>>>>>>> should actually be as you would expect from upgrading a patch
>>>>>> release.
>>>>>>>>> Yes, adding new API in patch releases will make downgrading
>>>> harder,
>>>>>>> but I
>>>>>>>>> think that is an acceptable tradeoff. We can document that if
>>>> your
>>>>>>>>> application compiles (meaning that you are not using new API)
>>>> with
>>>>>>> 1.0.0,
>>>>>>>>> then you can swap your jars in a binary compat manner.
>>>>>>>>>
>>>>>>>>> Enis
>>>>>>>>>
>>>>>>>>> On Thu, Apr 23, 2015 at 10:03 AM, Andrew Purtell<
>>>>>> apurtell@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Anyone disagree with the point of view put forward by Josh
>>> and
>>>>>> Sean?
>>>>>>>>>>
>>>>>>>>>> On Wed, Apr 22, 2015 at 10:35 AM, Josh Elser<
>>>>> josh.elser@gmail.com
>>>>>>>>> wrote:
>>>>>>>>>>> Andy -- I understood your intent, but thanks for
>>> clarifying.
>>>>> (as
>>>>>>> well
>>>>>>>>> as
>>>>>>>>>>> taking the time to break this discussion out in the first
>>>>>> place). I
>>>>>>>>> agree
>>>>>>>>>>> with your assessment.
>>>>>>>>>>>
>>>>>>>>>>> re: Sean's comments, if it wasn't clear by me asking in
>> the
>>>>> first
>>>>>>>>> place,
>>>>>>>>>> I
>>>>>>>>>>> also think sticking as close as possible to semver's
>> rules
>>> is
>>>>> the
>>>>>>>> best
>>>>>>>>>>> approach, although I'm getting the impression that there
>>> have
>>>>>> been
>>>>>>>> some
>>>>>>>>>>> previous reservations to doing so (especially by your
>>> comment
>>>>>> about
>>>>>>>>>>> backporting features if there is demand is).
>>>>>>>>>>>
>>>>>>>>>>> I've found adhering to the bug-fix release restrictions
>> can
>>>> be
>>>>> a
>>>>>>> very
>>>>>>>>>>> painful and time-consuming task, so this is something to
>>> get
>>>> a
>>>>>>>>>>> representative sampling of those who do the work to make
>>> sure
>>>>>>>> everyone
>>>>>>>>> is
>>>>>>>>>>> on board.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Sean Busbey wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I'd much rather we stick with the definitions used in
>>>> Semantic
>>>>>>>>>> Versioning.
>>>>>>>>>>>> Our use is already confusing enough given our matrix of
>>>>>>>>> compatibilities
>>>>>>>>>>>> that don't get "major version for breaking" protections.
>>>>>>>>>>>>
>>>>>>>>>>>> We've previously discussed how we'll do additional minor
>>>>>> releases
>>>>>>>> when
>>>>>>>>>>>> there's sufficient interest in the new features present
>>>> there.
>>>>>>>> What's
>>>>>>>>>>>> building that demand if any backwards compatible change
>>> can
>>>> go
>>>>>>> back
>>>>>>>>>> into a
>>>>>>>>>>>> patch release?
>>>>>>>>>>>>
>>>>>>>>>>>> Would we have an easier time restraining ourselves if we
>>>> had a
>>>>>>>> regular
>>>>>>>>>>>> schedule planned around new minor versions?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Apr 22, 2015 at 12:03 PM, Josh Elser<
>>>>>> josh.elser@gmail.com
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> While I can understand the desire to want to add
>> things,
>>> I
>>>> do
>>>>>>> think
>>>>>>>>> it
>>>>>>>>>>>>> makes things harder for users to reliably write code
>>>> against
>>>>>>>> versions
>>>>>>>>>> of
>>>>>>>>>>>>> HBase which (by their view) should be completely
>>> compatible
>>>>>> with
>>>>>>>> one
>>>>>>>>>>>>> another.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Take this extremely hypothetical situation: I'm new to
>>>> HBase
>>>>>> and
>>>>>>>>> start
>>>>>>>>>>>>> writing some code against HBase 1.0.1 which was just
>>>> deployed
>>>>>> at
>>>>>>> my
>>>>>>>>>>>>> $job. I
>>>>>>>>>>>>> don't _know_ what APIs are new, I just know what exists
>>> and
>>>>>> treat
>>>>>>>>> that
>>>>>>>>>> as
>>>>>>>>>>>>> acceptable for me to be using. Meanwhile in production,
>>>> some
>>>>>>> other
>>>>>>>>>> people
>>>>>>>>>>>>> find a bug with HBase 1.0.1 and roll back to 1.0.0
>> which
>>>> they
>>>>>> had
>>>>>>>>> been
>>>>>>>>>>>>> previously using. My reaction would be "of course my
>> code
>>>>>> should
>>>>>>>> work
>>>>>>>>>>>>> with
>>>>>>>>>>>>> HBase 1.0.0, I only used the public API" when in fact
>>> this
>>>> is
>>>>>> not
>>>>>>>>> true.
>>>>>>>>>>>>> Personally, I think it's a little bold to say semver is
>>>> even
>>>>> in
>>>>>>> use
>>>>>>>>> if
>>>>>>>>>>>>> this principal isn't being followed as it doesn't
>> follow
>>> at
>>>>> all
>>>>>>>> with
>>>>>>>>> my
>>>>>>>>>>>>> understanding on the guarantees defined by semver for
>>>> bug-fix
>>>>>>>>> releases.
>>>>>>>>>>>>> That being said, if the intent *is* to allow ourselves
>> to
>>>>> make
>>>>>>>> these
>>>>>>>>>>>>> sorts
>>>>>>>>>>>>> of changes, I just think some sort of disclaimer should
>>> be
>>>>>>> present:
>>>>>>>>>>>>> - HBase uses Semantic Versioning for its release
>>> versioning
>>>>>>>>>>>>> + HBase uses Semantic Versioning for its release
>>> versioning
>>>>>> with
>>>>>>> a
>>>>>>>>>> caveat
>>>>>>>>>>>>> that methods and members might be added in newer
>> bug-fix
>>>>>> releases
>>>>>>>>> that
>>>>>>>>>>>>> were
>>>>>>>>>>>>> not present in the previous bug-fix release.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Andrew Purtell wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> [Subject changed]
>>>>>>>>>>>>>> On Tue, Apr 21, 2015 at 8:47 PM, Josh Elser<
>>>>>>> josh.elser@gmail.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I was a little surprised when I noticed method
>>> additions
>>>>> to
>>>>>>>>>>>>>>> InterfaceAudience.Public annotated classes. This
>> means
>>>>> that a
>>>>>>>> user
>>>>>>>>>>>>>>> could
>>>>>>>>>>>>>>> write code against 1.0.1 that would not work against
>>>> 1.0.0
>>>>>>> which
>>>>>>>>>> seems
>>>>>>>>>>>>>>> undesirable for a bugfix release. I read over the
>> book
>>>>>> section
>>>>>>> on
>>>>>>>>>>>>>>> compatibility and didn't see this addressed, so I
>>> thought
>>>>> I'd
>>>>>>>> ask.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Let's clarify this. It's not the first time this
>>> question
>>>>> has
>>>>>>> been
>>>>>>>>>>>>>> asked.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> To get things moving:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I propose the following addition to the "Client API
>>>>>>> compatibility"
>>>>>>>>>>>>>> section
>>>>>>>>>>>>>> of Section 11.1:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> + APIs available in a patch version will be available
>> in
>>>> all
>>>>>>> later
>>>>>>>>>>>>>> + patch versions. However, new APIs may be added which
>>>> will
>>>>>> not
>>>>>>> be
>>>>>>>>>>>>>> + available in earlier patch versions.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I propose the following change to the "Client Binary
>>>>>>>> compatibility"
>>>>>>>>>>>>>> section
>>>>>>>>>>>>>> of Section 11.1:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - Old client code can run unchanged (no recompilation
>>>>> needed)
>>>>>>>>> against
>>>>>>>>>>>>>> new
>>>>>>>>>>>>>> jars.
>>>>>>>>>>>>>> + Client code written to APIs available in a given
>> patch
>>>>>> release
>>>>>>>>>>>>>> + can run unchanged (no recompilation needed) against
>>> the
>>>>> new
>>>>>>>>>>>>>> + jars of later patch versions.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> If these changes are (mostly) ok, then this clarifies
>> in
>>>> one
>>>>>>>>>> direction.
>>>>>>>>>>>>>> If these changes are not acceptable, I will propose
>>> edits
>>>>> that
>>>>>>>>> clarify
>>>>>>>>>>>>>> toward the opposite meaning.
>>>>>>>>>>>>>>
>
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Enis Söztutar <en...@gmail.com>.
+1 to "working toward semver". Is there any documentation that we would
like to clarify in the book given this enlightenment?
I would be in favor of going 2.6.0 and jackson upgrade in 1.1.
Enis
On Thu, Apr 23, 2015 at 5:34 PM, Nick Dimiduk <nd...@gmail.com> wrote:
> On Thu, Apr 23, 2015 at 4:13 PM, Stack <st...@duboce.net> wrote:
>
> > Does this 'admission' help with the which-hadoop thread too?
> >
>
> Right -- "working toward semver".
>
> Are we now at liberty to bump to 2.6 or 2.7 even for branch-1.1? I still
> have Karthik's offer to roll a 2.5.3 with the HDFS issue resolved.
>
> What about the jackson issue with 2.5 YARN runtime?
>
> Thanks Sean and Josh for being our community conscience on these issues.
>
> > Just want to make sure we're all on the same page (or find out if not).
> > >
> > > On Thu, Apr 23, 2015 at 2:59 PM, Enis Söztutar <en...@apache.org>
> wrote:
> > >
> > > > Then let's get Andrew's proposal committed:
> > > >
> > > > + APIs available in a patch version will be available in all later
> > > > + patch versions. However, new APIs may be added which will not be
> > > > + available in earlier patch versions.
> > > >
> > > > I propose the following change to the "Client Binary compatibility"
> > > section
> > > > of Section 11.1:
> > > >
> > > > - Old client code can run unchanged (no recompilation needed) against
> > new
> > > > jars.
> > > > + Client code written to APIs available in a given patch release
> > > > + can run unchanged (no recompilation needed) against the new
> > > > + jars of later patch versions.
> > > >
> > > > We can even claim that if you are not using new APIs, we are binary
> > > compat
> > > > for downgrade. But you have to make sure that your code compiles with
> > > that
> > > > downgraded version.
> > > >
> > > > Enis
> > > >
> > > >
> > > >
> > > > On Thu, Apr 23, 2015 at 2:04 PM, Sean Busbey <bu...@cloudera.com>
> > > wrote:
> > > >
> > > > > I'm fine with us adding methods to public APIs in patch releases so
> > > long
> > > > as
> > > > > we stop claiming to follow semver. We can say we take the
> principles
> > of
> > > > > semver as inspiration. This would reflect the current state of the
> > > world
> > > > > WRT 1.0.1 and would still give us a reason keep the narrower
> > definition
> > > > of
> > > > > "new features" in minor versions.
> > > > >
> > > > >
> > > > > No longer claiming semver would also have the added benefit of
> making
> > > it
> > > > > for me to easier to explain our compatibility promises to people.
> > Right
> > > > now
> > > > > I have to explain the difference between the things that get proper
> > > > semver
> > > > > treatment (e.g. public api, wire compat) and which things are
> > > downgraded
> > > > to
> > > > > breaking-on-minor (e.g. LimitedPrivate, binary compatibility).
> > Starting
> > > > > with the context of "we like semver" will be easier than "we use
> > > semver".
> > > > >
> > > > >
> > > > > Like Josh, my main concern is that we accurately advertise what
> we're
> > > > > doing. There are few things I've found more frustrating than being
> an
> > > end
> > > > > user of a project that claims to follow semver without
> understanding
> > > the
> > > > > burden that carries (and subsequently not meeting it).
> > > > >
> > > > > On Thu, Apr 23, 2015 at 3:48 PM, Stack <st...@duboce.net> wrote:
> > > > >
> > > > > > I agree w/ the Enis characterization (so we need the callout on
> > > semvar)
> > > > > but
> > > > > > think we should practice what Seans says (patch is bug fixes
> only).
> > > > > > St.Ack
> > > > > >
> > > > > > On Thu, Apr 23, 2015 at 1:31 PM, Sean Busbey <
> busbey@cloudera.com>
> > > > > wrote:
> > > > > >
> > > > > > > Why don't we just focus development after a minor release on
> the
> > > next
> > > > > > minor
> > > > > > > release instead of the next patch release?
> > > > > > >
> > > > > > > We could limit backports to the patch releases to critical
> bugs,
> > > > which
> > > > > > > would cut down on how often someone has to deal with the pain
> of
> > > > making
> > > > > > > sure we don't add to public APIs. It also reduces the risk
> > someone
> > > > > going
> > > > > > > through an upgrade has, since there are fewer changes.
> > > > > > >
> > > > > > > If someone fixes a bug and doesn't want to do the work of
> making
> > > sure
> > > > > it
> > > > > > > doesn't add methods in a patch release, they just don't
> backport
> > to
> > > > > that
> > > > > > > version and make a follow on e.g. "backport to 1.0.z" ticket.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Thu, Apr 23, 2015 at 1:50 PM, Enis Söztutar <
> > enis.soz@gmail.com
> > > >
> > > > > > wrote:
> > > > > > >
> > > > > > > > +1 to the proposal.
> > > > > > > >
> > > > > > > > The problem is that we have a very big API surface especially
> > > with
> > > > > the
> > > > > > > > coprocessors included in the report. Even simple bug fixes
> can
> > > > > > introduce
> > > > > > > > protected or public methods to base classes, which makes
> patch
> > > > > releases
> > > > > > > > very hard to maintain. I would not want to spend the effort
> to
> > > > spend
> > > > > > tons
> > > > > > > > of time trying to make a patch not introduce new methods in
> > order
> > > > to
> > > > > > > > backport. That effort can be spent elsewhere IMO.
> > > > > > > >
> > > > > > > > Looking at the report
> > > > > > > >
> > > https://people.apache.org/~enis/1.0.0_1.0.1RC2_compat_report.html,
> > > > > > > nothing
> > > > > > > > strikes me as "new functionality". Going from current 1.0.0
> to
> > > > > 1.0.1RC2
> > > > > > > > should actually be as you would expect from upgrading a patch
> > > > > release.
> > > > > > > >
> > > > > > > > Yes, adding new API in patch releases will make downgrading
> > > harder,
> > > > > > but I
> > > > > > > > think that is an acceptable tradeoff. We can document that if
> > > your
> > > > > > > > application compiles (meaning that you are not using new API)
> > > with
> > > > > > 1.0.0,
> > > > > > > > then you can swap your jars in a binary compat manner.
> > > > > > > >
> > > > > > > > Enis
> > > > > > > >
> > > > > > > > On Thu, Apr 23, 2015 at 10:03 AM, Andrew Purtell <
> > > > > apurtell@apache.org>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Anyone disagree with the point of view put forward by Josh
> > and
> > > > > Sean?
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Wed, Apr 22, 2015 at 10:35 AM, Josh Elser <
> > > > josh.elser@gmail.com
> > > > > >
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Andy -- I understood your intent, but thanks for
> > clarifying.
> > > > (as
> > > > > > well
> > > > > > > > as
> > > > > > > > > > taking the time to break this discussion out in the first
> > > > > place). I
> > > > > > > > agree
> > > > > > > > > > with your assessment.
> > > > > > > > > >
> > > > > > > > > > re: Sean's comments, if it wasn't clear by me asking in
> the
> > > > first
> > > > > > > > place,
> > > > > > > > > I
> > > > > > > > > > also think sticking as close as possible to semver's
> rules
> > is
> > > > the
> > > > > > > best
> > > > > > > > > > approach, although I'm getting the impression that there
> > have
> > > > > been
> > > > > > > some
> > > > > > > > > > previous reservations to doing so (especially by your
> > comment
> > > > > about
> > > > > > > > > > backporting features if there is demand is).
> > > > > > > > > >
> > > > > > > > > > I've found adhering to the bug-fix release restrictions
> can
> > > be
> > > > a
> > > > > > very
> > > > > > > > > > painful and time-consuming task, so this is something to
> > get
> > > a
> > > > > > > > > > representative sampling of those who do the work to make
> > sure
> > > > > > > everyone
> > > > > > > > is
> > > > > > > > > > on board.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Sean Busbey wrote:
> > > > > > > > > >
> > > > > > > > > >> I'd much rather we stick with the definitions used in
> > > Semantic
> > > > > > > > > Versioning.
> > > > > > > > > >> Our use is already confusing enough given our matrix of
> > > > > > > > compatibilities
> > > > > > > > > >> that don't get "major version for breaking" protections.
> > > > > > > > > >>
> > > > > > > > > >> We've previously discussed how we'll do additional minor
> > > > > releases
> > > > > > > when
> > > > > > > > > >> there's sufficient interest in the new features present
> > > there.
> > > > > > > What's
> > > > > > > > > >> building that demand if any backwards compatible change
> > can
> > > go
> > > > > > back
> > > > > > > > > into a
> > > > > > > > > >> patch release?
> > > > > > > > > >>
> > > > > > > > > >> Would we have an easier time restraining ourselves if we
> > > had a
> > > > > > > regular
> > > > > > > > > >> schedule planned around new minor versions?
> > > > > > > > > >>
> > > > > > > > > >>
> > > > > > > > > >> On Wed, Apr 22, 2015 at 12:03 PM, Josh Elser<
> > > > > josh.elser@gmail.com
> > > > > > >
> > > > > > > > > >> wrote:
> > > > > > > > > >>
> > > > > > > > > >> While I can understand the desire to want to add
> things,
> > I
> > > do
> > > > > > think
> > > > > > > > it
> > > > > > > > > >>> makes things harder for users to reliably write code
> > > against
> > > > > > > versions
> > > > > > > > > of
> > > > > > > > > >>> HBase which (by their view) should be completely
> > compatible
> > > > > with
> > > > > > > one
> > > > > > > > > >>> another.
> > > > > > > > > >>>
> > > > > > > > > >>> Take this extremely hypothetical situation: I'm new to
> > > HBase
> > > > > and
> > > > > > > > start
> > > > > > > > > >>> writing some code against HBase 1.0.1 which was just
> > > deployed
> > > > > at
> > > > > > my
> > > > > > > > > >>> $job. I
> > > > > > > > > >>> don't _know_ what APIs are new, I just know what exists
> > and
> > > > > treat
> > > > > > > > that
> > > > > > > > > as
> > > > > > > > > >>> acceptable for me to be using. Meanwhile in production,
> > > some
> > > > > > other
> > > > > > > > > people
> > > > > > > > > >>> find a bug with HBase 1.0.1 and roll back to 1.0.0
> which
> > > they
> > > > > had
> > > > > > > > been
> > > > > > > > > >>> previously using. My reaction would be "of course my
> code
> > > > > should
> > > > > > > work
> > > > > > > > > >>> with
> > > > > > > > > >>> HBase 1.0.0, I only used the public API" when in fact
> > this
> > > is
> > > > > not
> > > > > > > > true.
> > > > > > > > > >>>
> > > > > > > > > >>> Personally, I think it's a little bold to say semver is
> > > even
> > > > in
> > > > > > use
> > > > > > > > if
> > > > > > > > > >>> this principal isn't being followed as it doesn't
> follow
> > at
> > > > all
> > > > > > > with
> > > > > > > > my
> > > > > > > > > >>> understanding on the guarantees defined by semver for
> > > bug-fix
> > > > > > > > releases.
> > > > > > > > > >>>
> > > > > > > > > >>> That being said, if the intent *is* to allow ourselves
> to
> > > > make
> > > > > > > these
> > > > > > > > > >>> sorts
> > > > > > > > > >>> of changes, I just think some sort of disclaimer should
> > be
> > > > > > present:
> > > > > > > > > >>>
> > > > > > > > > >>> - HBase uses Semantic Versioning for its release
> > versioning
> > > > > > > > > >>> + HBase uses Semantic Versioning for its release
> > versioning
> > > > > with
> > > > > > a
> > > > > > > > > caveat
> > > > > > > > > >>> that methods and members might be added in newer
> bug-fix
> > > > > releases
> > > > > > > > that
> > > > > > > > > >>> were
> > > > > > > > > >>> not present in the previous bug-fix release.
> > > > > > > > > >>>
> > > > > > > > > >>>
> > > > > > > > > >>> Andrew Purtell wrote:
> > > > > > > > > >>>
> > > > > > > > > >>> [Subject changed]
> > > > > > > > > >>>>
> > > > > > > > > >>>> On Tue, Apr 21, 2015 at 8:47 PM, Josh Elser<
> > > > > > josh.elser@gmail.com>
> > > > > > > > > >>>> wrote:
> > > > > > > > > >>>>
> > > > > > > > > >>>> I was a little surprised when I noticed method
> > additions
> > > > to
> > > > > > > > > >>>>
> > > > > > > > > >>>>> InterfaceAudience.Public annotated classes. This
> means
> > > > that a
> > > > > > > user
> > > > > > > > > >>>>> could
> > > > > > > > > >>>>> write code against 1.0.1 that would not work against
> > > 1.0.0
> > > > > > which
> > > > > > > > > seems
> > > > > > > > > >>>>> undesirable for a bugfix release. I read over the
> book
> > > > > section
> > > > > > on
> > > > > > > > > >>>>> compatibility and didn't see this addressed, so I
> > thought
> > > > I'd
> > > > > > > ask.
> > > > > > > > > >>>>>
> > > > > > > > > >>>>>
> > > > > > > > > >>>> Let's clarify this. It's not the first time this
> > question
> > > > has
> > > > > > been
> > > > > > > > > >>>> asked.
> > > > > > > > > >>>>
> > > > > > > > > >>>> To get things moving:
> > > > > > > > > >>>>
> > > > > > > > > >>>> I propose the following addition to the "Client API
> > > > > > compatibility"
> > > > > > > > > >>>> section
> > > > > > > > > >>>> of Section 11.1:
> > > > > > > > > >>>>
> > > > > > > > > >>>> + APIs available in a patch version will be available
> in
> > > all
> > > > > > later
> > > > > > > > > >>>> + patch versions. However, new APIs may be added which
> > > will
> > > > > not
> > > > > > be
> > > > > > > > > >>>> + available in earlier patch versions.
> > > > > > > > > >>>>
> > > > > > > > > >>>> I propose the following change to the "Client Binary
> > > > > > > compatibility"
> > > > > > > > > >>>> section
> > > > > > > > > >>>> of Section 11.1:
> > > > > > > > > >>>>
> > > > > > > > > >>>> - Old client code can run unchanged (no recompilation
> > > > needed)
> > > > > > > > against
> > > > > > > > > >>>> new
> > > > > > > > > >>>> jars.
> > > > > > > > > >>>> + Client code written to APIs available in a given
> patch
> > > > > release
> > > > > > > > > >>>> + can run unchanged (no recompilation needed) against
> > the
> > > > new
> > > > > > > > > >>>> + jars of later patch versions.
> > > > > > > > > >>>>
> > > > > > > > > >>>>
> > > > > > > > > >>>> What do you think?
> > > > > > > > > >>>>
> > > > > > > > > >>>> If these changes are (mostly) ok, then this clarifies
> in
> > > one
> > > > > > > > > direction.
> > > > > > > > > >>>>
> > > > > > > > > >>>> If these changes are not acceptable, I will propose
> > edits
> > > > that
> > > > > > > > clarify
> > > > > > > > > >>>> toward the opposite meaning.
> > > > > > > > > >>>>
> > > >
> > >
> >
>
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Nick Dimiduk <nd...@gmail.com>.
On Thu, Apr 23, 2015 at 4:13 PM, Stack <st...@duboce.net> wrote:
> Does this 'admission' help with the which-hadoop thread too?
>
Right -- "working toward semver".
Are we now at liberty to bump to 2.6 or 2.7 even for branch-1.1? I still
have Karthik's offer to roll a 2.5.3 with the HDFS issue resolved.
What about the jackson issue with 2.5 YARN runtime?
Thanks Sean and Josh for being our community conscience on these issues.
> Just want to make sure we're all on the same page (or find out if not).
> >
> > On Thu, Apr 23, 2015 at 2:59 PM, Enis Söztutar <en...@apache.org> wrote:
> >
> > > Then let's get Andrew's proposal committed:
> > >
> > > + APIs available in a patch version will be available in all later
> > > + patch versions. However, new APIs may be added which will not be
> > > + available in earlier patch versions.
> > >
> > > I propose the following change to the "Client Binary compatibility"
> > section
> > > of Section 11.1:
> > >
> > > - Old client code can run unchanged (no recompilation needed) against
> new
> > > jars.
> > > + Client code written to APIs available in a given patch release
> > > + can run unchanged (no recompilation needed) against the new
> > > + jars of later patch versions.
> > >
> > > We can even claim that if you are not using new APIs, we are binary
> > compat
> > > for downgrade. But you have to make sure that your code compiles with
> > that
> > > downgraded version.
> > >
> > > Enis
> > >
> > >
> > >
> > > On Thu, Apr 23, 2015 at 2:04 PM, Sean Busbey <bu...@cloudera.com>
> > wrote:
> > >
> > > > I'm fine with us adding methods to public APIs in patch releases so
> > long
> > > as
> > > > we stop claiming to follow semver. We can say we take the principles
> of
> > > > semver as inspiration. This would reflect the current state of the
> > world
> > > > WRT 1.0.1 and would still give us a reason keep the narrower
> definition
> > > of
> > > > "new features" in minor versions.
> > > >
> > > >
> > > > No longer claiming semver would also have the added benefit of making
> > it
> > > > for me to easier to explain our compatibility promises to people.
> Right
> > > now
> > > > I have to explain the difference between the things that get proper
> > > semver
> > > > treatment (e.g. public api, wire compat) and which things are
> > downgraded
> > > to
> > > > breaking-on-minor (e.g. LimitedPrivate, binary compatibility).
> Starting
> > > > with the context of "we like semver" will be easier than "we use
> > semver".
> > > >
> > > >
> > > > Like Josh, my main concern is that we accurately advertise what we're
> > > > doing. There are few things I've found more frustrating than being an
> > end
> > > > user of a project that claims to follow semver without understanding
> > the
> > > > burden that carries (and subsequently not meeting it).
> > > >
> > > > On Thu, Apr 23, 2015 at 3:48 PM, Stack <st...@duboce.net> wrote:
> > > >
> > > > > I agree w/ the Enis characterization (so we need the callout on
> > semvar)
> > > > but
> > > > > think we should practice what Seans says (patch is bug fixes only).
> > > > > St.Ack
> > > > >
> > > > > On Thu, Apr 23, 2015 at 1:31 PM, Sean Busbey <bu...@cloudera.com>
> > > > wrote:
> > > > >
> > > > > > Why don't we just focus development after a minor release on the
> > next
> > > > > minor
> > > > > > release instead of the next patch release?
> > > > > >
> > > > > > We could limit backports to the patch releases to critical bugs,
> > > which
> > > > > > would cut down on how often someone has to deal with the pain of
> > > making
> > > > > > sure we don't add to public APIs. It also reduces the risk
> someone
> > > > going
> > > > > > through an upgrade has, since there are fewer changes.
> > > > > >
> > > > > > If someone fixes a bug and doesn't want to do the work of making
> > sure
> > > > it
> > > > > > doesn't add methods in a patch release, they just don't backport
> to
> > > > that
> > > > > > version and make a follow on e.g. "backport to 1.0.z" ticket.
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Thu, Apr 23, 2015 at 1:50 PM, Enis Söztutar <
> enis.soz@gmail.com
> > >
> > > > > wrote:
> > > > > >
> > > > > > > +1 to the proposal.
> > > > > > >
> > > > > > > The problem is that we have a very big API surface especially
> > with
> > > > the
> > > > > > > coprocessors included in the report. Even simple bug fixes can
> > > > > introduce
> > > > > > > protected or public methods to base classes, which makes patch
> > > > releases
> > > > > > > very hard to maintain. I would not want to spend the effort to
> > > spend
> > > > > tons
> > > > > > > of time trying to make a patch not introduce new methods in
> order
> > > to
> > > > > > > backport. That effort can be spent elsewhere IMO.
> > > > > > >
> > > > > > > Looking at the report
> > > > > > >
> > https://people.apache.org/~enis/1.0.0_1.0.1RC2_compat_report.html,
> > > > > > nothing
> > > > > > > strikes me as "new functionality". Going from current 1.0.0 to
> > > > 1.0.1RC2
> > > > > > > should actually be as you would expect from upgrading a patch
> > > > release.
> > > > > > >
> > > > > > > Yes, adding new API in patch releases will make downgrading
> > harder,
> > > > > but I
> > > > > > > think that is an acceptable tradeoff. We can document that if
> > your
> > > > > > > application compiles (meaning that you are not using new API)
> > with
> > > > > 1.0.0,
> > > > > > > then you can swap your jars in a binary compat manner.
> > > > > > >
> > > > > > > Enis
> > > > > > >
> > > > > > > On Thu, Apr 23, 2015 at 10:03 AM, Andrew Purtell <
> > > > apurtell@apache.org>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Anyone disagree with the point of view put forward by Josh
> and
> > > > Sean?
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Apr 22, 2015 at 10:35 AM, Josh Elser <
> > > josh.elser@gmail.com
> > > > >
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Andy -- I understood your intent, but thanks for
> clarifying.
> > > (as
> > > > > well
> > > > > > > as
> > > > > > > > > taking the time to break this discussion out in the first
> > > > place). I
> > > > > > > agree
> > > > > > > > > with your assessment.
> > > > > > > > >
> > > > > > > > > re: Sean's comments, if it wasn't clear by me asking in the
> > > first
> > > > > > > place,
> > > > > > > > I
> > > > > > > > > also think sticking as close as possible to semver's rules
> is
> > > the
> > > > > > best
> > > > > > > > > approach, although I'm getting the impression that there
> have
> > > > been
> > > > > > some
> > > > > > > > > previous reservations to doing so (especially by your
> comment
> > > > about
> > > > > > > > > backporting features if there is demand is).
> > > > > > > > >
> > > > > > > > > I've found adhering to the bug-fix release restrictions can
> > be
> > > a
> > > > > very
> > > > > > > > > painful and time-consuming task, so this is something to
> get
> > a
> > > > > > > > > representative sampling of those who do the work to make
> sure
> > > > > > everyone
> > > > > > > is
> > > > > > > > > on board.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Sean Busbey wrote:
> > > > > > > > >
> > > > > > > > >> I'd much rather we stick with the definitions used in
> > Semantic
> > > > > > > > Versioning.
> > > > > > > > >> Our use is already confusing enough given our matrix of
> > > > > > > compatibilities
> > > > > > > > >> that don't get "major version for breaking" protections.
> > > > > > > > >>
> > > > > > > > >> We've previously discussed how we'll do additional minor
> > > > releases
> > > > > > when
> > > > > > > > >> there's sufficient interest in the new features present
> > there.
> > > > > > What's
> > > > > > > > >> building that demand if any backwards compatible change
> can
> > go
> > > > > back
> > > > > > > > into a
> > > > > > > > >> patch release?
> > > > > > > > >>
> > > > > > > > >> Would we have an easier time restraining ourselves if we
> > had a
> > > > > > regular
> > > > > > > > >> schedule planned around new minor versions?
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >> On Wed, Apr 22, 2015 at 12:03 PM, Josh Elser<
> > > > josh.elser@gmail.com
> > > > > >
> > > > > > > > >> wrote:
> > > > > > > > >>
> > > > > > > > >> While I can understand the desire to want to add things,
> I
> > do
> > > > > think
> > > > > > > it
> > > > > > > > >>> makes things harder for users to reliably write code
> > against
> > > > > > versions
> > > > > > > > of
> > > > > > > > >>> HBase which (by their view) should be completely
> compatible
> > > > with
> > > > > > one
> > > > > > > > >>> another.
> > > > > > > > >>>
> > > > > > > > >>> Take this extremely hypothetical situation: I'm new to
> > HBase
> > > > and
> > > > > > > start
> > > > > > > > >>> writing some code against HBase 1.0.1 which was just
> > deployed
> > > > at
> > > > > my
> > > > > > > > >>> $job. I
> > > > > > > > >>> don't _know_ what APIs are new, I just know what exists
> and
> > > > treat
> > > > > > > that
> > > > > > > > as
> > > > > > > > >>> acceptable for me to be using. Meanwhile in production,
> > some
> > > > > other
> > > > > > > > people
> > > > > > > > >>> find a bug with HBase 1.0.1 and roll back to 1.0.0 which
> > they
> > > > had
> > > > > > > been
> > > > > > > > >>> previously using. My reaction would be "of course my code
> > > > should
> > > > > > work
> > > > > > > > >>> with
> > > > > > > > >>> HBase 1.0.0, I only used the public API" when in fact
> this
> > is
> > > > not
> > > > > > > true.
> > > > > > > > >>>
> > > > > > > > >>> Personally, I think it's a little bold to say semver is
> > even
> > > in
> > > > > use
> > > > > > > if
> > > > > > > > >>> this principal isn't being followed as it doesn't follow
> at
> > > all
> > > > > > with
> > > > > > > my
> > > > > > > > >>> understanding on the guarantees defined by semver for
> > bug-fix
> > > > > > > releases.
> > > > > > > > >>>
> > > > > > > > >>> That being said, if the intent *is* to allow ourselves to
> > > make
> > > > > > these
> > > > > > > > >>> sorts
> > > > > > > > >>> of changes, I just think some sort of disclaimer should
> be
> > > > > present:
> > > > > > > > >>>
> > > > > > > > >>> - HBase uses Semantic Versioning for its release
> versioning
> > > > > > > > >>> + HBase uses Semantic Versioning for its release
> versioning
> > > > with
> > > > > a
> > > > > > > > caveat
> > > > > > > > >>> that methods and members might be added in newer bug-fix
> > > > releases
> > > > > > > that
> > > > > > > > >>> were
> > > > > > > > >>> not present in the previous bug-fix release.
> > > > > > > > >>>
> > > > > > > > >>>
> > > > > > > > >>> Andrew Purtell wrote:
> > > > > > > > >>>
> > > > > > > > >>> [Subject changed]
> > > > > > > > >>>>
> > > > > > > > >>>> On Tue, Apr 21, 2015 at 8:47 PM, Josh Elser<
> > > > > josh.elser@gmail.com>
> > > > > > > > >>>> wrote:
> > > > > > > > >>>>
> > > > > > > > >>>> I was a little surprised when I noticed method
> additions
> > > to
> > > > > > > > >>>>
> > > > > > > > >>>>> InterfaceAudience.Public annotated classes. This means
> > > that a
> > > > > > user
> > > > > > > > >>>>> could
> > > > > > > > >>>>> write code against 1.0.1 that would not work against
> > 1.0.0
> > > > > which
> > > > > > > > seems
> > > > > > > > >>>>> undesirable for a bugfix release. I read over the book
> > > > section
> > > > > on
> > > > > > > > >>>>> compatibility and didn't see this addressed, so I
> thought
> > > I'd
> > > > > > ask.
> > > > > > > > >>>>>
> > > > > > > > >>>>>
> > > > > > > > >>>> Let's clarify this. It's not the first time this
> question
> > > has
> > > > > been
> > > > > > > > >>>> asked.
> > > > > > > > >>>>
> > > > > > > > >>>> To get things moving:
> > > > > > > > >>>>
> > > > > > > > >>>> I propose the following addition to the "Client API
> > > > > compatibility"
> > > > > > > > >>>> section
> > > > > > > > >>>> of Section 11.1:
> > > > > > > > >>>>
> > > > > > > > >>>> + APIs available in a patch version will be available in
> > all
> > > > > later
> > > > > > > > >>>> + patch versions. However, new APIs may be added which
> > will
> > > > not
> > > > > be
> > > > > > > > >>>> + available in earlier patch versions.
> > > > > > > > >>>>
> > > > > > > > >>>> I propose the following change to the "Client Binary
> > > > > > compatibility"
> > > > > > > > >>>> section
> > > > > > > > >>>> of Section 11.1:
> > > > > > > > >>>>
> > > > > > > > >>>> - Old client code can run unchanged (no recompilation
> > > needed)
> > > > > > > against
> > > > > > > > >>>> new
> > > > > > > > >>>> jars.
> > > > > > > > >>>> + Client code written to APIs available in a given patch
> > > > release
> > > > > > > > >>>> + can run unchanged (no recompilation needed) against
> the
> > > new
> > > > > > > > >>>> + jars of later patch versions.
> > > > > > > > >>>>
> > > > > > > > >>>>
> > > > > > > > >>>> What do you think?
> > > > > > > > >>>>
> > > > > > > > >>>> If these changes are (mostly) ok, then this clarifies in
> > one
> > > > > > > > direction.
> > > > > > > > >>>>
> > > > > > > > >>>> If these changes are not acceptable, I will propose
> edits
> > > that
> > > > > > > clarify
> > > > > > > > >>>> toward the opposite meaning.
> > > > > > > > >>>>
> > >
> >
>
Re: Clarifying interface evolution freedom in patch releases (was: Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Ted Yu <yu...@gmail.com>.
+1 as well.
> On Apr 23, 2015, at 4:36 PM, Josh Elser <jo...@gmail.com> wrote:
>
> Stack wrote:
>>> On Thu, Apr 23, 2015 at 3:08 PM, Andrew Purtell<ap...@apache.org> wrote:
>>>
>>> > So "we like semver", not "we use semver"?
>>> >
>>> >
>> and Sean's
>>
>>> > No longer claiming semver would also have the added benefit of making it for
>> me to easier to explain our compatibility promises to people.
>>
>> Yeah, 'we like semvar'/'we are almost semvar' + and are actively working
>> toward 'semvar' would give us leeway that we seem to need as we learn what
>> semvar implies.
>>
>> Does this 'admission' help with the which-hadoop thread too?
>>
>> St.Ack
>
> +1
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Josh Elser <jo...@gmail.com>.
Stack wrote:
> On Thu, Apr 23, 2015 at 3:08 PM, Andrew Purtell<ap...@apache.org> wrote:
>
>> > So "we like semver", not "we use semver"?
>> >
>> >
> and Sean's
>
>> > No longer claiming semver would also have the added benefit of making it for
> me to easier to explain our compatibility promises to people.
>
> Yeah, 'we like semvar'/'we are almost semvar' + and are actively working
> toward 'semvar' would give us leeway that we seem to need as we learn what
> semvar implies.
>
> Does this 'admission' help with the which-hadoop thread too?
>
> St.Ack
>
+1
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Stack <st...@duboce.net>.
On Thu, Apr 23, 2015 at 3:08 PM, Andrew Purtell <ap...@apache.org> wrote:
> So "we like semver", not "we use semver"?
>
>
and Sean's
> No longer claiming semver would also have the added benefit of making it for
me to easier to explain our compatibility promises to people.
Yeah, 'we like semvar'/'we are almost semvar' + and are actively working
toward 'semvar' would give us leeway that we seem to need as we learn what
semvar implies.
Does this 'admission' help with the which-hadoop thread too?
St.Ack
> Just want to make sure we're all on the same page (or find out if not).
>
> On Thu, Apr 23, 2015 at 2:59 PM, Enis Söztutar <en...@apache.org> wrote:
>
> > Then let's get Andrew's proposal committed:
> >
> > + APIs available in a patch version will be available in all later
> > + patch versions. However, new APIs may be added which will not be
> > + available in earlier patch versions.
> >
> > I propose the following change to the "Client Binary compatibility"
> section
> > of Section 11.1:
> >
> > - Old client code can run unchanged (no recompilation needed) against new
> > jars.
> > + Client code written to APIs available in a given patch release
> > + can run unchanged (no recompilation needed) against the new
> > + jars of later patch versions.
> >
> > We can even claim that if you are not using new APIs, we are binary
> compat
> > for downgrade. But you have to make sure that your code compiles with
> that
> > downgraded version.
> >
> > Enis
> >
> >
> >
> > On Thu, Apr 23, 2015 at 2:04 PM, Sean Busbey <bu...@cloudera.com>
> wrote:
> >
> > > I'm fine with us adding methods to public APIs in patch releases so
> long
> > as
> > > we stop claiming to follow semver. We can say we take the principles of
> > > semver as inspiration. This would reflect the current state of the
> world
> > > WRT 1.0.1 and would still give us a reason keep the narrower definition
> > of
> > > "new features" in minor versions.
> > >
> > >
> > > No longer claiming semver would also have the added benefit of making
> it
> > > for me to easier to explain our compatibility promises to people. Right
> > now
> > > I have to explain the difference between the things that get proper
> > semver
> > > treatment (e.g. public api, wire compat) and which things are
> downgraded
> > to
> > > breaking-on-minor (e.g. LimitedPrivate, binary compatibility). Starting
> > > with the context of "we like semver" will be easier than "we use
> semver".
> > >
> > >
> > > Like Josh, my main concern is that we accurately advertise what we're
> > > doing. There are few things I've found more frustrating than being an
> end
> > > user of a project that claims to follow semver without understanding
> the
> > > burden that carries (and subsequently not meeting it).
> > >
> > > On Thu, Apr 23, 2015 at 3:48 PM, Stack <st...@duboce.net> wrote:
> > >
> > > > I agree w/ the Enis characterization (so we need the callout on
> semvar)
> > > but
> > > > think we should practice what Seans says (patch is bug fixes only).
> > > > St.Ack
> > > >
> > > > On Thu, Apr 23, 2015 at 1:31 PM, Sean Busbey <bu...@cloudera.com>
> > > wrote:
> > > >
> > > > > Why don't we just focus development after a minor release on the
> next
> > > > minor
> > > > > release instead of the next patch release?
> > > > >
> > > > > We could limit backports to the patch releases to critical bugs,
> > which
> > > > > would cut down on how often someone has to deal with the pain of
> > making
> > > > > sure we don't add to public APIs. It also reduces the risk someone
> > > going
> > > > > through an upgrade has, since there are fewer changes.
> > > > >
> > > > > If someone fixes a bug and doesn't want to do the work of making
> sure
> > > it
> > > > > doesn't add methods in a patch release, they just don't backport to
> > > that
> > > > > version and make a follow on e.g. "backport to 1.0.z" ticket.
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Apr 23, 2015 at 1:50 PM, Enis Söztutar <enis.soz@gmail.com
> >
> > > > wrote:
> > > > >
> > > > > > +1 to the proposal.
> > > > > >
> > > > > > The problem is that we have a very big API surface especially
> with
> > > the
> > > > > > coprocessors included in the report. Even simple bug fixes can
> > > > introduce
> > > > > > protected or public methods to base classes, which makes patch
> > > releases
> > > > > > very hard to maintain. I would not want to spend the effort to
> > spend
> > > > tons
> > > > > > of time trying to make a patch not introduce new methods in order
> > to
> > > > > > backport. That effort can be spent elsewhere IMO.
> > > > > >
> > > > > > Looking at the report
> > > > > >
> https://people.apache.org/~enis/1.0.0_1.0.1RC2_compat_report.html,
> > > > > nothing
> > > > > > strikes me as "new functionality". Going from current 1.0.0 to
> > > 1.0.1RC2
> > > > > > should actually be as you would expect from upgrading a patch
> > > release.
> > > > > >
> > > > > > Yes, adding new API in patch releases will make downgrading
> harder,
> > > > but I
> > > > > > think that is an acceptable tradeoff. We can document that if
> your
> > > > > > application compiles (meaning that you are not using new API)
> with
> > > > 1.0.0,
> > > > > > then you can swap your jars in a binary compat manner.
> > > > > >
> > > > > > Enis
> > > > > >
> > > > > > On Thu, Apr 23, 2015 at 10:03 AM, Andrew Purtell <
> > > apurtell@apache.org>
> > > > > > wrote:
> > > > > >
> > > > > > > Anyone disagree with the point of view put forward by Josh and
> > > Sean?
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Apr 22, 2015 at 10:35 AM, Josh Elser <
> > josh.elser@gmail.com
> > > >
> > > > > > wrote:
> > > > > > >
> > > > > > > > Andy -- I understood your intent, but thanks for clarifying.
> > (as
> > > > well
> > > > > > as
> > > > > > > > taking the time to break this discussion out in the first
> > > place). I
> > > > > > agree
> > > > > > > > with your assessment.
> > > > > > > >
> > > > > > > > re: Sean's comments, if it wasn't clear by me asking in the
> > first
> > > > > > place,
> > > > > > > I
> > > > > > > > also think sticking as close as possible to semver's rules is
> > the
> > > > > best
> > > > > > > > approach, although I'm getting the impression that there have
> > > been
> > > > > some
> > > > > > > > previous reservations to doing so (especially by your comment
> > > about
> > > > > > > > backporting features if there is demand is).
> > > > > > > >
> > > > > > > > I've found adhering to the bug-fix release restrictions can
> be
> > a
> > > > very
> > > > > > > > painful and time-consuming task, so this is something to get
> a
> > > > > > > > representative sampling of those who do the work to make sure
> > > > > everyone
> > > > > > is
> > > > > > > > on board.
> > > > > > > >
> > > > > > > >
> > > > > > > > Sean Busbey wrote:
> > > > > > > >
> > > > > > > >> I'd much rather we stick with the definitions used in
> Semantic
> > > > > > > Versioning.
> > > > > > > >> Our use is already confusing enough given our matrix of
> > > > > > compatibilities
> > > > > > > >> that don't get "major version for breaking" protections.
> > > > > > > >>
> > > > > > > >> We've previously discussed how we'll do additional minor
> > > releases
> > > > > when
> > > > > > > >> there's sufficient interest in the new features present
> there.
> > > > > What's
> > > > > > > >> building that demand if any backwards compatible change can
> go
> > > > back
> > > > > > > into a
> > > > > > > >> patch release?
> > > > > > > >>
> > > > > > > >> Would we have an easier time restraining ourselves if we
> had a
> > > > > regular
> > > > > > > >> schedule planned around new minor versions?
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> On Wed, Apr 22, 2015 at 12:03 PM, Josh Elser<
> > > josh.elser@gmail.com
> > > > >
> > > > > > > >> wrote:
> > > > > > > >>
> > > > > > > >> While I can understand the desire to want to add things, I
> do
> > > > think
> > > > > > it
> > > > > > > >>> makes things harder for users to reliably write code
> against
> > > > > versions
> > > > > > > of
> > > > > > > >>> HBase which (by their view) should be completely compatible
> > > with
> > > > > one
> > > > > > > >>> another.
> > > > > > > >>>
> > > > > > > >>> Take this extremely hypothetical situation: I'm new to
> HBase
> > > and
> > > > > > start
> > > > > > > >>> writing some code against HBase 1.0.1 which was just
> deployed
> > > at
> > > > my
> > > > > > > >>> $job. I
> > > > > > > >>> don't _know_ what APIs are new, I just know what exists and
> > > treat
> > > > > > that
> > > > > > > as
> > > > > > > >>> acceptable for me to be using. Meanwhile in production,
> some
> > > > other
> > > > > > > people
> > > > > > > >>> find a bug with HBase 1.0.1 and roll back to 1.0.0 which
> they
> > > had
> > > > > > been
> > > > > > > >>> previously using. My reaction would be "of course my code
> > > should
> > > > > work
> > > > > > > >>> with
> > > > > > > >>> HBase 1.0.0, I only used the public API" when in fact this
> is
> > > not
> > > > > > true.
> > > > > > > >>>
> > > > > > > >>> Personally, I think it's a little bold to say semver is
> even
> > in
> > > > use
> > > > > > if
> > > > > > > >>> this principal isn't being followed as it doesn't follow at
> > all
> > > > > with
> > > > > > my
> > > > > > > >>> understanding on the guarantees defined by semver for
> bug-fix
> > > > > > releases.
> > > > > > > >>>
> > > > > > > >>> That being said, if the intent *is* to allow ourselves to
> > make
> > > > > these
> > > > > > > >>> sorts
> > > > > > > >>> of changes, I just think some sort of disclaimer should be
> > > > present:
> > > > > > > >>>
> > > > > > > >>> - HBase uses Semantic Versioning for its release versioning
> > > > > > > >>> + HBase uses Semantic Versioning for its release versioning
> > > with
> > > > a
> > > > > > > caveat
> > > > > > > >>> that methods and members might be added in newer bug-fix
> > > releases
> > > > > > that
> > > > > > > >>> were
> > > > > > > >>> not present in the previous bug-fix release.
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>> Andrew Purtell wrote:
> > > > > > > >>>
> > > > > > > >>> [Subject changed]
> > > > > > > >>>>
> > > > > > > >>>> On Tue, Apr 21, 2015 at 8:47 PM, Josh Elser<
> > > > josh.elser@gmail.com>
> > > > > > > >>>> wrote:
> > > > > > > >>>>
> > > > > > > >>>> I was a little surprised when I noticed method additions
> > to
> > > > > > > >>>>
> > > > > > > >>>>> InterfaceAudience.Public annotated classes. This means
> > that a
> > > > > user
> > > > > > > >>>>> could
> > > > > > > >>>>> write code against 1.0.1 that would not work against
> 1.0.0
> > > > which
> > > > > > > seems
> > > > > > > >>>>> undesirable for a bugfix release. I read over the book
> > > section
> > > > on
> > > > > > > >>>>> compatibility and didn't see this addressed, so I thought
> > I'd
> > > > > ask.
> > > > > > > >>>>>
> > > > > > > >>>>>
> > > > > > > >>>> Let's clarify this. It's not the first time this question
> > has
> > > > been
> > > > > > > >>>> asked.
> > > > > > > >>>>
> > > > > > > >>>> To get things moving:
> > > > > > > >>>>
> > > > > > > >>>> I propose the following addition to the "Client API
> > > > compatibility"
> > > > > > > >>>> section
> > > > > > > >>>> of Section 11.1:
> > > > > > > >>>>
> > > > > > > >>>> + APIs available in a patch version will be available in
> all
> > > > later
> > > > > > > >>>> + patch versions. However, new APIs may be added which
> will
> > > not
> > > > be
> > > > > > > >>>> + available in earlier patch versions.
> > > > > > > >>>>
> > > > > > > >>>> I propose the following change to the "Client Binary
> > > > > compatibility"
> > > > > > > >>>> section
> > > > > > > >>>> of Section 11.1:
> > > > > > > >>>>
> > > > > > > >>>> - Old client code can run unchanged (no recompilation
> > needed)
> > > > > > against
> > > > > > > >>>> new
> > > > > > > >>>> jars.
> > > > > > > >>>> + Client code written to APIs available in a given patch
> > > release
> > > > > > > >>>> + can run unchanged (no recompilation needed) against the
> > new
> > > > > > > >>>> + jars of later patch versions.
> > > > > > > >>>>
> > > > > > > >>>>
> > > > > > > >>>> What do you think?
> > > > > > > >>>>
> > > > > > > >>>> If these changes are (mostly) ok, then this clarifies in
> one
> > > > > > > direction.
> > > > > > > >>>>
> > > > > > > >>>> If these changes are not acceptable, I will propose edits
> > that
> > > > > > clarify
> > > > > > > >>>> toward the opposite meaning.
> > > > > > > >>>>
> >
>
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Andrew Purtell <ap...@apache.org>.
So "we like semver", not "we use semver"?
Just want to make sure we're all on the same page (or find out if not).
On Thu, Apr 23, 2015 at 2:59 PM, Enis Söztutar <en...@apache.org> wrote:
> Then let's get Andrew's proposal committed:
>
> + APIs available in a patch version will be available in all later
> + patch versions. However, new APIs may be added which will not be
> + available in earlier patch versions.
>
> I propose the following change to the "Client Binary compatibility" section
> of Section 11.1:
>
> - Old client code can run unchanged (no recompilation needed) against new
> jars.
> + Client code written to APIs available in a given patch release
> + can run unchanged (no recompilation needed) against the new
> + jars of later patch versions.
>
> We can even claim that if you are not using new APIs, we are binary compat
> for downgrade. But you have to make sure that your code compiles with that
> downgraded version.
>
> Enis
>
>
>
> On Thu, Apr 23, 2015 at 2:04 PM, Sean Busbey <bu...@cloudera.com> wrote:
>
> > I'm fine with us adding methods to public APIs in patch releases so long
> as
> > we stop claiming to follow semver. We can say we take the principles of
> > semver as inspiration. This would reflect the current state of the world
> > WRT 1.0.1 and would still give us a reason keep the narrower definition
> of
> > "new features" in minor versions.
> >
> >
> > No longer claiming semver would also have the added benefit of making it
> > for me to easier to explain our compatibility promises to people. Right
> now
> > I have to explain the difference between the things that get proper
> semver
> > treatment (e.g. public api, wire compat) and which things are downgraded
> to
> > breaking-on-minor (e.g. LimitedPrivate, binary compatibility). Starting
> > with the context of "we like semver" will be easier than "we use semver".
> >
> >
> > Like Josh, my main concern is that we accurately advertise what we're
> > doing. There are few things I've found more frustrating than being an end
> > user of a project that claims to follow semver without understanding the
> > burden that carries (and subsequently not meeting it).
> >
> > On Thu, Apr 23, 2015 at 3:48 PM, Stack <st...@duboce.net> wrote:
> >
> > > I agree w/ the Enis characterization (so we need the callout on semvar)
> > but
> > > think we should practice what Seans says (patch is bug fixes only).
> > > St.Ack
> > >
> > > On Thu, Apr 23, 2015 at 1:31 PM, Sean Busbey <bu...@cloudera.com>
> > wrote:
> > >
> > > > Why don't we just focus development after a minor release on the next
> > > minor
> > > > release instead of the next patch release?
> > > >
> > > > We could limit backports to the patch releases to critical bugs,
> which
> > > > would cut down on how often someone has to deal with the pain of
> making
> > > > sure we don't add to public APIs. It also reduces the risk someone
> > going
> > > > through an upgrade has, since there are fewer changes.
> > > >
> > > > If someone fixes a bug and doesn't want to do the work of making sure
> > it
> > > > doesn't add methods in a patch release, they just don't backport to
> > that
> > > > version and make a follow on e.g. "backport to 1.0.z" ticket.
> > > >
> > > >
> > > >
> > > > On Thu, Apr 23, 2015 at 1:50 PM, Enis Söztutar <en...@gmail.com>
> > > wrote:
> > > >
> > > > > +1 to the proposal.
> > > > >
> > > > > The problem is that we have a very big API surface especially with
> > the
> > > > > coprocessors included in the report. Even simple bug fixes can
> > > introduce
> > > > > protected or public methods to base classes, which makes patch
> > releases
> > > > > very hard to maintain. I would not want to spend the effort to
> spend
> > > tons
> > > > > of time trying to make a patch not introduce new methods in order
> to
> > > > > backport. That effort can be spent elsewhere IMO.
> > > > >
> > > > > Looking at the report
> > > > > https://people.apache.org/~enis/1.0.0_1.0.1RC2_compat_report.html,
> > > > nothing
> > > > > strikes me as "new functionality". Going from current 1.0.0 to
> > 1.0.1RC2
> > > > > should actually be as you would expect from upgrading a patch
> > release.
> > > > >
> > > > > Yes, adding new API in patch releases will make downgrading harder,
> > > but I
> > > > > think that is an acceptable tradeoff. We can document that if your
> > > > > application compiles (meaning that you are not using new API) with
> > > 1.0.0,
> > > > > then you can swap your jars in a binary compat manner.
> > > > >
> > > > > Enis
> > > > >
> > > > > On Thu, Apr 23, 2015 at 10:03 AM, Andrew Purtell <
> > apurtell@apache.org>
> > > > > wrote:
> > > > >
> > > > > > Anyone disagree with the point of view put forward by Josh and
> > Sean?
> > > > > >
> > > > > >
> > > > > > On Wed, Apr 22, 2015 at 10:35 AM, Josh Elser <
> josh.elser@gmail.com
> > >
> > > > > wrote:
> > > > > >
> > > > > > > Andy -- I understood your intent, but thanks for clarifying.
> (as
> > > well
> > > > > as
> > > > > > > taking the time to break this discussion out in the first
> > place). I
> > > > > agree
> > > > > > > with your assessment.
> > > > > > >
> > > > > > > re: Sean's comments, if it wasn't clear by me asking in the
> first
> > > > > place,
> > > > > > I
> > > > > > > also think sticking as close as possible to semver's rules is
> the
> > > > best
> > > > > > > approach, although I'm getting the impression that there have
> > been
> > > > some
> > > > > > > previous reservations to doing so (especially by your comment
> > about
> > > > > > > backporting features if there is demand is).
> > > > > > >
> > > > > > > I've found adhering to the bug-fix release restrictions can be
> a
> > > very
> > > > > > > painful and time-consuming task, so this is something to get a
> > > > > > > representative sampling of those who do the work to make sure
> > > > everyone
> > > > > is
> > > > > > > on board.
> > > > > > >
> > > > > > >
> > > > > > > Sean Busbey wrote:
> > > > > > >
> > > > > > >> I'd much rather we stick with the definitions used in Semantic
> > > > > > Versioning.
> > > > > > >> Our use is already confusing enough given our matrix of
> > > > > compatibilities
> > > > > > >> that don't get "major version for breaking" protections.
> > > > > > >>
> > > > > > >> We've previously discussed how we'll do additional minor
> > releases
> > > > when
> > > > > > >> there's sufficient interest in the new features present there.
> > > > What's
> > > > > > >> building that demand if any backwards compatible change can go
> > > back
> > > > > > into a
> > > > > > >> patch release?
> > > > > > >>
> > > > > > >> Would we have an easier time restraining ourselves if we had a
> > > > regular
> > > > > > >> schedule planned around new minor versions?
> > > > > > >>
> > > > > > >>
> > > > > > >> On Wed, Apr 22, 2015 at 12:03 PM, Josh Elser<
> > josh.elser@gmail.com
> > > >
> > > > > > >> wrote:
> > > > > > >>
> > > > > > >> While I can understand the desire to want to add things, I do
> > > think
> > > > > it
> > > > > > >>> makes things harder for users to reliably write code against
> > > > versions
> > > > > > of
> > > > > > >>> HBase which (by their view) should be completely compatible
> > with
> > > > one
> > > > > > >>> another.
> > > > > > >>>
> > > > > > >>> Take this extremely hypothetical situation: I'm new to HBase
> > and
> > > > > start
> > > > > > >>> writing some code against HBase 1.0.1 which was just deployed
> > at
> > > my
> > > > > > >>> $job. I
> > > > > > >>> don't _know_ what APIs are new, I just know what exists and
> > treat
> > > > > that
> > > > > > as
> > > > > > >>> acceptable for me to be using. Meanwhile in production, some
> > > other
> > > > > > people
> > > > > > >>> find a bug with HBase 1.0.1 and roll back to 1.0.0 which they
> > had
> > > > > been
> > > > > > >>> previously using. My reaction would be "of course my code
> > should
> > > > work
> > > > > > >>> with
> > > > > > >>> HBase 1.0.0, I only used the public API" when in fact this is
> > not
> > > > > true.
> > > > > > >>>
> > > > > > >>> Personally, I think it's a little bold to say semver is even
> in
> > > use
> > > > > if
> > > > > > >>> this principal isn't being followed as it doesn't follow at
> all
> > > > with
> > > > > my
> > > > > > >>> understanding on the guarantees defined by semver for bug-fix
> > > > > releases.
> > > > > > >>>
> > > > > > >>> That being said, if the intent *is* to allow ourselves to
> make
> > > > these
> > > > > > >>> sorts
> > > > > > >>> of changes, I just think some sort of disclaimer should be
> > > present:
> > > > > > >>>
> > > > > > >>> - HBase uses Semantic Versioning for its release versioning
> > > > > > >>> + HBase uses Semantic Versioning for its release versioning
> > with
> > > a
> > > > > > caveat
> > > > > > >>> that methods and members might be added in newer bug-fix
> > releases
> > > > > that
> > > > > > >>> were
> > > > > > >>> not present in the previous bug-fix release.
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> Andrew Purtell wrote:
> > > > > > >>>
> > > > > > >>> [Subject changed]
> > > > > > >>>>
> > > > > > >>>> On Tue, Apr 21, 2015 at 8:47 PM, Josh Elser<
> > > josh.elser@gmail.com>
> > > > > > >>>> wrote:
> > > > > > >>>>
> > > > > > >>>> I was a little surprised when I noticed method additions
> to
> > > > > > >>>>
> > > > > > >>>>> InterfaceAudience.Public annotated classes. This means
> that a
> > > > user
> > > > > > >>>>> could
> > > > > > >>>>> write code against 1.0.1 that would not work against 1.0.0
> > > which
> > > > > > seems
> > > > > > >>>>> undesirable for a bugfix release. I read over the book
> > section
> > > on
> > > > > > >>>>> compatibility and didn't see this addressed, so I thought
> I'd
> > > > ask.
> > > > > > >>>>>
> > > > > > >>>>>
> > > > > > >>>> Let's clarify this. It's not the first time this question
> has
> > > been
> > > > > > >>>> asked.
> > > > > > >>>>
> > > > > > >>>> To get things moving:
> > > > > > >>>>
> > > > > > >>>> I propose the following addition to the "Client API
> > > compatibility"
> > > > > > >>>> section
> > > > > > >>>> of Section 11.1:
> > > > > > >>>>
> > > > > > >>>> + APIs available in a patch version will be available in all
> > > later
> > > > > > >>>> + patch versions. However, new APIs may be added which will
> > not
> > > be
> > > > > > >>>> + available in earlier patch versions.
> > > > > > >>>>
> > > > > > >>>> I propose the following change to the "Client Binary
> > > > compatibility"
> > > > > > >>>> section
> > > > > > >>>> of Section 11.1:
> > > > > > >>>>
> > > > > > >>>> - Old client code can run unchanged (no recompilation
> needed)
> > > > > against
> > > > > > >>>> new
> > > > > > >>>> jars.
> > > > > > >>>> + Client code written to APIs available in a given patch
> > release
> > > > > > >>>> + can run unchanged (no recompilation needed) against the
> new
> > > > > > >>>> + jars of later patch versions.
> > > > > > >>>>
> > > > > > >>>>
> > > > > > >>>> What do you think?
> > > > > > >>>>
> > > > > > >>>> If these changes are (mostly) ok, then this clarifies in one
> > > > > > direction.
> > > > > > >>>>
> > > > > > >>>> If these changes are not acceptable, I will propose edits
> that
> > > > > clarify
> > > > > > >>>> toward the opposite meaning.
> > > > > > >>>>
>
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Enis Söztutar <en...@apache.org>.
Then let's get Andrew's proposal committed:
+ APIs available in a patch version will be available in all later
+ patch versions. However, new APIs may be added which will not be
+ available in earlier patch versions.
I propose the following change to the "Client Binary compatibility" section
of Section 11.1:
- Old client code can run unchanged (no recompilation needed) against new
jars.
+ Client code written to APIs available in a given patch release
+ can run unchanged (no recompilation needed) against the new
+ jars of later patch versions.
We can even claim that if you are not using new APIs, we are binary compat
for downgrade. But you have to make sure that your code compiles with that
downgraded version.
Enis
On Thu, Apr 23, 2015 at 2:04 PM, Sean Busbey <bu...@cloudera.com> wrote:
> I'm fine with us adding methods to public APIs in patch releases so long as
> we stop claiming to follow semver. We can say we take the principles of
> semver as inspiration. This would reflect the current state of the world
> WRT 1.0.1 and would still give us a reason keep the narrower definition of
> "new features" in minor versions.
>
>
> No longer claiming semver would also have the added benefit of making it
> for me to easier to explain our compatibility promises to people. Right now
> I have to explain the difference between the things that get proper semver
> treatment (e.g. public api, wire compat) and which things are downgraded to
> breaking-on-minor (e.g. LimitedPrivate, binary compatibility). Starting
> with the context of "we like semver" will be easier than "we use semver".
>
>
> Like Josh, my main concern is that we accurately advertise what we're
> doing. There are few things I've found more frustrating than being an end
> user of a project that claims to follow semver without understanding the
> burden that carries (and subsequently not meeting it).
>
> On Thu, Apr 23, 2015 at 3:48 PM, Stack <st...@duboce.net> wrote:
>
> > I agree w/ the Enis characterization (so we need the callout on semvar)
> but
> > think we should practice what Seans says (patch is bug fixes only).
> > St.Ack
> >
> > On Thu, Apr 23, 2015 at 1:31 PM, Sean Busbey <bu...@cloudera.com>
> wrote:
> >
> > > Why don't we just focus development after a minor release on the next
> > minor
> > > release instead of the next patch release?
> > >
> > > We could limit backports to the patch releases to critical bugs, which
> > > would cut down on how often someone has to deal with the pain of making
> > > sure we don't add to public APIs. It also reduces the risk someone
> going
> > > through an upgrade has, since there are fewer changes.
> > >
> > > If someone fixes a bug and doesn't want to do the work of making sure
> it
> > > doesn't add methods in a patch release, they just don't backport to
> that
> > > version and make a follow on e.g. "backport to 1.0.z" ticket.
> > >
> > >
> > >
> > > On Thu, Apr 23, 2015 at 1:50 PM, Enis Söztutar <en...@gmail.com>
> > wrote:
> > >
> > > > +1 to the proposal.
> > > >
> > > > The problem is that we have a very big API surface especially with
> the
> > > > coprocessors included in the report. Even simple bug fixes can
> > introduce
> > > > protected or public methods to base classes, which makes patch
> releases
> > > > very hard to maintain. I would not want to spend the effort to spend
> > tons
> > > > of time trying to make a patch not introduce new methods in order to
> > > > backport. That effort can be spent elsewhere IMO.
> > > >
> > > > Looking at the report
> > > > https://people.apache.org/~enis/1.0.0_1.0.1RC2_compat_report.html,
> > > nothing
> > > > strikes me as "new functionality". Going from current 1.0.0 to
> 1.0.1RC2
> > > > should actually be as you would expect from upgrading a patch
> release.
> > > >
> > > > Yes, adding new API in patch releases will make downgrading harder,
> > but I
> > > > think that is an acceptable tradeoff. We can document that if your
> > > > application compiles (meaning that you are not using new API) with
> > 1.0.0,
> > > > then you can swap your jars in a binary compat manner.
> > > >
> > > > Enis
> > > >
> > > > On Thu, Apr 23, 2015 at 10:03 AM, Andrew Purtell <
> apurtell@apache.org>
> > > > wrote:
> > > >
> > > > > Anyone disagree with the point of view put forward by Josh and
> Sean?
> > > > >
> > > > >
> > > > > On Wed, Apr 22, 2015 at 10:35 AM, Josh Elser <josh.elser@gmail.com
> >
> > > > wrote:
> > > > >
> > > > > > Andy -- I understood your intent, but thanks for clarifying. (as
> > well
> > > > as
> > > > > > taking the time to break this discussion out in the first
> place). I
> > > > agree
> > > > > > with your assessment.
> > > > > >
> > > > > > re: Sean's comments, if it wasn't clear by me asking in the first
> > > > place,
> > > > > I
> > > > > > also think sticking as close as possible to semver's rules is the
> > > best
> > > > > > approach, although I'm getting the impression that there have
> been
> > > some
> > > > > > previous reservations to doing so (especially by your comment
> about
> > > > > > backporting features if there is demand is).
> > > > > >
> > > > > > I've found adhering to the bug-fix release restrictions can be a
> > very
> > > > > > painful and time-consuming task, so this is something to get a
> > > > > > representative sampling of those who do the work to make sure
> > > everyone
> > > > is
> > > > > > on board.
> > > > > >
> > > > > >
> > > > > > Sean Busbey wrote:
> > > > > >
> > > > > >> I'd much rather we stick with the definitions used in Semantic
> > > > > Versioning.
> > > > > >> Our use is already confusing enough given our matrix of
> > > > compatibilities
> > > > > >> that don't get "major version for breaking" protections.
> > > > > >>
> > > > > >> We've previously discussed how we'll do additional minor
> releases
> > > when
> > > > > >> there's sufficient interest in the new features present there.
> > > What's
> > > > > >> building that demand if any backwards compatible change can go
> > back
> > > > > into a
> > > > > >> patch release?
> > > > > >>
> > > > > >> Would we have an easier time restraining ourselves if we had a
> > > regular
> > > > > >> schedule planned around new minor versions?
> > > > > >>
> > > > > >>
> > > > > >> On Wed, Apr 22, 2015 at 12:03 PM, Josh Elser<
> josh.elser@gmail.com
> > >
> > > > > >> wrote:
> > > > > >>
> > > > > >> While I can understand the desire to want to add things, I do
> > think
> > > > it
> > > > > >>> makes things harder for users to reliably write code against
> > > versions
> > > > > of
> > > > > >>> HBase which (by their view) should be completely compatible
> with
> > > one
> > > > > >>> another.
> > > > > >>>
> > > > > >>> Take this extremely hypothetical situation: I'm new to HBase
> and
> > > > start
> > > > > >>> writing some code against HBase 1.0.1 which was just deployed
> at
> > my
> > > > > >>> $job. I
> > > > > >>> don't _know_ what APIs are new, I just know what exists and
> treat
> > > > that
> > > > > as
> > > > > >>> acceptable for me to be using. Meanwhile in production, some
> > other
> > > > > people
> > > > > >>> find a bug with HBase 1.0.1 and roll back to 1.0.0 which they
> had
> > > > been
> > > > > >>> previously using. My reaction would be "of course my code
> should
> > > work
> > > > > >>> with
> > > > > >>> HBase 1.0.0, I only used the public API" when in fact this is
> not
> > > > true.
> > > > > >>>
> > > > > >>> Personally, I think it's a little bold to say semver is even in
> > use
> > > > if
> > > > > >>> this principal isn't being followed as it doesn't follow at all
> > > with
> > > > my
> > > > > >>> understanding on the guarantees defined by semver for bug-fix
> > > > releases.
> > > > > >>>
> > > > > >>> That being said, if the intent *is* to allow ourselves to make
> > > these
> > > > > >>> sorts
> > > > > >>> of changes, I just think some sort of disclaimer should be
> > present:
> > > > > >>>
> > > > > >>> - HBase uses Semantic Versioning for its release versioning
> > > > > >>> + HBase uses Semantic Versioning for its release versioning
> with
> > a
> > > > > caveat
> > > > > >>> that methods and members might be added in newer bug-fix
> releases
> > > > that
> > > > > >>> were
> > > > > >>> not present in the previous bug-fix release.
> > > > > >>>
> > > > > >>>
> > > > > >>> Andrew Purtell wrote:
> > > > > >>>
> > > > > >>> [Subject changed]
> > > > > >>>>
> > > > > >>>> On Tue, Apr 21, 2015 at 8:47 PM, Josh Elser<
> > josh.elser@gmail.com>
> > > > > >>>> wrote:
> > > > > >>>>
> > > > > >>>> I was a little surprised when I noticed method additions to
> > > > > >>>>
> > > > > >>>>> InterfaceAudience.Public annotated classes. This means that a
> > > user
> > > > > >>>>> could
> > > > > >>>>> write code against 1.0.1 that would not work against 1.0.0
> > which
> > > > > seems
> > > > > >>>>> undesirable for a bugfix release. I read over the book
> section
> > on
> > > > > >>>>> compatibility and didn't see this addressed, so I thought I'd
> > > ask.
> > > > > >>>>>
> > > > > >>>>>
> > > > > >>>> Let's clarify this. It's not the first time this question has
> > been
> > > > > >>>> asked.
> > > > > >>>>
> > > > > >>>> To get things moving:
> > > > > >>>>
> > > > > >>>> I propose the following addition to the "Client API
> > compatibility"
> > > > > >>>> section
> > > > > >>>> of Section 11.1:
> > > > > >>>>
> > > > > >>>> + APIs available in a patch version will be available in all
> > later
> > > > > >>>> + patch versions. However, new APIs may be added which will
> not
> > be
> > > > > >>>> + available in earlier patch versions.
> > > > > >>>>
> > > > > >>>> I propose the following change to the "Client Binary
> > > compatibility"
> > > > > >>>> section
> > > > > >>>> of Section 11.1:
> > > > > >>>>
> > > > > >>>> - Old client code can run unchanged (no recompilation needed)
> > > > against
> > > > > >>>> new
> > > > > >>>> jars.
> > > > > >>>> + Client code written to APIs available in a given patch
> release
> > > > > >>>> + can run unchanged (no recompilation needed) against the new
> > > > > >>>> + jars of later patch versions.
> > > > > >>>>
> > > > > >>>>
> > > > > >>>> What do you think?
> > > > > >>>>
> > > > > >>>> If these changes are (mostly) ok, then this clarifies in one
> > > > > direction.
> > > > > >>>>
> > > > > >>>> If these changes are not acceptable, I will propose edits that
> > > > clarify
> > > > > >>>> toward the opposite meaning.
> > > > > >>>>
> > > > > >>>>
> > > > > >>>>
> > > > > >>>>
> > > > > >>
> > > > > >>
> > > > >
> > > > >
> > > > > --
> > > > > Best regards,
> > > > >
> > > > > - Andy
> > > > >
> > > > > Problems worthy of attack prove their worth by hitting back. - Piet
> > > Hein
> > > > > (via Tom White)
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Sean
> > >
> >
>
>
>
> --
> Sean
>
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Sean Busbey <bu...@cloudera.com>.
I'm fine with us adding methods to public APIs in patch releases so long as
we stop claiming to follow semver. We can say we take the principles of
semver as inspiration. This would reflect the current state of the world
WRT 1.0.1 and would still give us a reason keep the narrower definition of
"new features" in minor versions.
No longer claiming semver would also have the added benefit of making it
for me to easier to explain our compatibility promises to people. Right now
I have to explain the difference between the things that get proper semver
treatment (e.g. public api, wire compat) and which things are downgraded to
breaking-on-minor (e.g. LimitedPrivate, binary compatibility). Starting
with the context of "we like semver" will be easier than "we use semver".
Like Josh, my main concern is that we accurately advertise what we're
doing. There are few things I've found more frustrating than being an end
user of a project that claims to follow semver without understanding the
burden that carries (and subsequently not meeting it).
On Thu, Apr 23, 2015 at 3:48 PM, Stack <st...@duboce.net> wrote:
> I agree w/ the Enis characterization (so we need the callout on semvar) but
> think we should practice what Seans says (patch is bug fixes only).
> St.Ack
>
> On Thu, Apr 23, 2015 at 1:31 PM, Sean Busbey <bu...@cloudera.com> wrote:
>
> > Why don't we just focus development after a minor release on the next
> minor
> > release instead of the next patch release?
> >
> > We could limit backports to the patch releases to critical bugs, which
> > would cut down on how often someone has to deal with the pain of making
> > sure we don't add to public APIs. It also reduces the risk someone going
> > through an upgrade has, since there are fewer changes.
> >
> > If someone fixes a bug and doesn't want to do the work of making sure it
> > doesn't add methods in a patch release, they just don't backport to that
> > version and make a follow on e.g. "backport to 1.0.z" ticket.
> >
> >
> >
> > On Thu, Apr 23, 2015 at 1:50 PM, Enis Söztutar <en...@gmail.com>
> wrote:
> >
> > > +1 to the proposal.
> > >
> > > The problem is that we have a very big API surface especially with the
> > > coprocessors included in the report. Even simple bug fixes can
> introduce
> > > protected or public methods to base classes, which makes patch releases
> > > very hard to maintain. I would not want to spend the effort to spend
> tons
> > > of time trying to make a patch not introduce new methods in order to
> > > backport. That effort can be spent elsewhere IMO.
> > >
> > > Looking at the report
> > > https://people.apache.org/~enis/1.0.0_1.0.1RC2_compat_report.html,
> > nothing
> > > strikes me as "new functionality". Going from current 1.0.0 to 1.0.1RC2
> > > should actually be as you would expect from upgrading a patch release.
> > >
> > > Yes, adding new API in patch releases will make downgrading harder,
> but I
> > > think that is an acceptable tradeoff. We can document that if your
> > > application compiles (meaning that you are not using new API) with
> 1.0.0,
> > > then you can swap your jars in a binary compat manner.
> > >
> > > Enis
> > >
> > > On Thu, Apr 23, 2015 at 10:03 AM, Andrew Purtell <ap...@apache.org>
> > > wrote:
> > >
> > > > Anyone disagree with the point of view put forward by Josh and Sean?
> > > >
> > > >
> > > > On Wed, Apr 22, 2015 at 10:35 AM, Josh Elser <jo...@gmail.com>
> > > wrote:
> > > >
> > > > > Andy -- I understood your intent, but thanks for clarifying. (as
> well
> > > as
> > > > > taking the time to break this discussion out in the first place). I
> > > agree
> > > > > with your assessment.
> > > > >
> > > > > re: Sean's comments, if it wasn't clear by me asking in the first
> > > place,
> > > > I
> > > > > also think sticking as close as possible to semver's rules is the
> > best
> > > > > approach, although I'm getting the impression that there have been
> > some
> > > > > previous reservations to doing so (especially by your comment about
> > > > > backporting features if there is demand is).
> > > > >
> > > > > I've found adhering to the bug-fix release restrictions can be a
> very
> > > > > painful and time-consuming task, so this is something to get a
> > > > > representative sampling of those who do the work to make sure
> > everyone
> > > is
> > > > > on board.
> > > > >
> > > > >
> > > > > Sean Busbey wrote:
> > > > >
> > > > >> I'd much rather we stick with the definitions used in Semantic
> > > > Versioning.
> > > > >> Our use is already confusing enough given our matrix of
> > > compatibilities
> > > > >> that don't get "major version for breaking" protections.
> > > > >>
> > > > >> We've previously discussed how we'll do additional minor releases
> > when
> > > > >> there's sufficient interest in the new features present there.
> > What's
> > > > >> building that demand if any backwards compatible change can go
> back
> > > > into a
> > > > >> patch release?
> > > > >>
> > > > >> Would we have an easier time restraining ourselves if we had a
> > regular
> > > > >> schedule planned around new minor versions?
> > > > >>
> > > > >>
> > > > >> On Wed, Apr 22, 2015 at 12:03 PM, Josh Elser<josh.elser@gmail.com
> >
> > > > >> wrote:
> > > > >>
> > > > >> While I can understand the desire to want to add things, I do
> think
> > > it
> > > > >>> makes things harder for users to reliably write code against
> > versions
> > > > of
> > > > >>> HBase which (by their view) should be completely compatible with
> > one
> > > > >>> another.
> > > > >>>
> > > > >>> Take this extremely hypothetical situation: I'm new to HBase and
> > > start
> > > > >>> writing some code against HBase 1.0.1 which was just deployed at
> my
> > > > >>> $job. I
> > > > >>> don't _know_ what APIs are new, I just know what exists and treat
> > > that
> > > > as
> > > > >>> acceptable for me to be using. Meanwhile in production, some
> other
> > > > people
> > > > >>> find a bug with HBase 1.0.1 and roll back to 1.0.0 which they had
> > > been
> > > > >>> previously using. My reaction would be "of course my code should
> > work
> > > > >>> with
> > > > >>> HBase 1.0.0, I only used the public API" when in fact this is not
> > > true.
> > > > >>>
> > > > >>> Personally, I think it's a little bold to say semver is even in
> use
> > > if
> > > > >>> this principal isn't being followed as it doesn't follow at all
> > with
> > > my
> > > > >>> understanding on the guarantees defined by semver for bug-fix
> > > releases.
> > > > >>>
> > > > >>> That being said, if the intent *is* to allow ourselves to make
> > these
> > > > >>> sorts
> > > > >>> of changes, I just think some sort of disclaimer should be
> present:
> > > > >>>
> > > > >>> - HBase uses Semantic Versioning for its release versioning
> > > > >>> + HBase uses Semantic Versioning for its release versioning with
> a
> > > > caveat
> > > > >>> that methods and members might be added in newer bug-fix releases
> > > that
> > > > >>> were
> > > > >>> not present in the previous bug-fix release.
> > > > >>>
> > > > >>>
> > > > >>> Andrew Purtell wrote:
> > > > >>>
> > > > >>> [Subject changed]
> > > > >>>>
> > > > >>>> On Tue, Apr 21, 2015 at 8:47 PM, Josh Elser<
> josh.elser@gmail.com>
> > > > >>>> wrote:
> > > > >>>>
> > > > >>>> I was a little surprised when I noticed method additions to
> > > > >>>>
> > > > >>>>> InterfaceAudience.Public annotated classes. This means that a
> > user
> > > > >>>>> could
> > > > >>>>> write code against 1.0.1 that would not work against 1.0.0
> which
> > > > seems
> > > > >>>>> undesirable for a bugfix release. I read over the book section
> on
> > > > >>>>> compatibility and didn't see this addressed, so I thought I'd
> > ask.
> > > > >>>>>
> > > > >>>>>
> > > > >>>> Let's clarify this. It's not the first time this question has
> been
> > > > >>>> asked.
> > > > >>>>
> > > > >>>> To get things moving:
> > > > >>>>
> > > > >>>> I propose the following addition to the "Client API
> compatibility"
> > > > >>>> section
> > > > >>>> of Section 11.1:
> > > > >>>>
> > > > >>>> + APIs available in a patch version will be available in all
> later
> > > > >>>> + patch versions. However, new APIs may be added which will not
> be
> > > > >>>> + available in earlier patch versions.
> > > > >>>>
> > > > >>>> I propose the following change to the "Client Binary
> > compatibility"
> > > > >>>> section
> > > > >>>> of Section 11.1:
> > > > >>>>
> > > > >>>> - Old client code can run unchanged (no recompilation needed)
> > > against
> > > > >>>> new
> > > > >>>> jars.
> > > > >>>> + Client code written to APIs available in a given patch release
> > > > >>>> + can run unchanged (no recompilation needed) against the new
> > > > >>>> + jars of later patch versions.
> > > > >>>>
> > > > >>>>
> > > > >>>> What do you think?
> > > > >>>>
> > > > >>>> If these changes are (mostly) ok, then this clarifies in one
> > > > direction.
> > > > >>>>
> > > > >>>> If these changes are not acceptable, I will propose edits that
> > > clarify
> > > > >>>> toward the opposite meaning.
> > > > >>>>
> > > > >>>>
> > > > >>>>
> > > > >>>>
> > > > >>
> > > > >>
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > >
> > > > - Andy
> > > >
> > > > Problems worthy of attack prove their worth by hitting back. - Piet
> > Hein
> > > > (via Tom White)
> > > >
> > >
> >
> >
> >
> > --
> > Sean
> >
>
--
Sean
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Stack <st...@duboce.net>.
I agree w/ the Enis characterization (so we need the callout on semvar) but
think we should practice what Seans says (patch is bug fixes only).
St.Ack
On Thu, Apr 23, 2015 at 1:31 PM, Sean Busbey <bu...@cloudera.com> wrote:
> Why don't we just focus development after a minor release on the next minor
> release instead of the next patch release?
>
> We could limit backports to the patch releases to critical bugs, which
> would cut down on how often someone has to deal with the pain of making
> sure we don't add to public APIs. It also reduces the risk someone going
> through an upgrade has, since there are fewer changes.
>
> If someone fixes a bug and doesn't want to do the work of making sure it
> doesn't add methods in a patch release, they just don't backport to that
> version and make a follow on e.g. "backport to 1.0.z" ticket.
>
>
>
> On Thu, Apr 23, 2015 at 1:50 PM, Enis Söztutar <en...@gmail.com> wrote:
>
> > +1 to the proposal.
> >
> > The problem is that we have a very big API surface especially with the
> > coprocessors included in the report. Even simple bug fixes can introduce
> > protected or public methods to base classes, which makes patch releases
> > very hard to maintain. I would not want to spend the effort to spend tons
> > of time trying to make a patch not introduce new methods in order to
> > backport. That effort can be spent elsewhere IMO.
> >
> > Looking at the report
> > https://people.apache.org/~enis/1.0.0_1.0.1RC2_compat_report.html,
> nothing
> > strikes me as "new functionality". Going from current 1.0.0 to 1.0.1RC2
> > should actually be as you would expect from upgrading a patch release.
> >
> > Yes, adding new API in patch releases will make downgrading harder, but I
> > think that is an acceptable tradeoff. We can document that if your
> > application compiles (meaning that you are not using new API) with 1.0.0,
> > then you can swap your jars in a binary compat manner.
> >
> > Enis
> >
> > On Thu, Apr 23, 2015 at 10:03 AM, Andrew Purtell <ap...@apache.org>
> > wrote:
> >
> > > Anyone disagree with the point of view put forward by Josh and Sean?
> > >
> > >
> > > On Wed, Apr 22, 2015 at 10:35 AM, Josh Elser <jo...@gmail.com>
> > wrote:
> > >
> > > > Andy -- I understood your intent, but thanks for clarifying. (as well
> > as
> > > > taking the time to break this discussion out in the first place). I
> > agree
> > > > with your assessment.
> > > >
> > > > re: Sean's comments, if it wasn't clear by me asking in the first
> > place,
> > > I
> > > > also think sticking as close as possible to semver's rules is the
> best
> > > > approach, although I'm getting the impression that there have been
> some
> > > > previous reservations to doing so (especially by your comment about
> > > > backporting features if there is demand is).
> > > >
> > > > I've found adhering to the bug-fix release restrictions can be a very
> > > > painful and time-consuming task, so this is something to get a
> > > > representative sampling of those who do the work to make sure
> everyone
> > is
> > > > on board.
> > > >
> > > >
> > > > Sean Busbey wrote:
> > > >
> > > >> I'd much rather we stick with the definitions used in Semantic
> > > Versioning.
> > > >> Our use is already confusing enough given our matrix of
> > compatibilities
> > > >> that don't get "major version for breaking" protections.
> > > >>
> > > >> We've previously discussed how we'll do additional minor releases
> when
> > > >> there's sufficient interest in the new features present there.
> What's
> > > >> building that demand if any backwards compatible change can go back
> > > into a
> > > >> patch release?
> > > >>
> > > >> Would we have an easier time restraining ourselves if we had a
> regular
> > > >> schedule planned around new minor versions?
> > > >>
> > > >>
> > > >> On Wed, Apr 22, 2015 at 12:03 PM, Josh Elser<jo...@gmail.com>
> > > >> wrote:
> > > >>
> > > >> While I can understand the desire to want to add things, I do think
> > it
> > > >>> makes things harder for users to reliably write code against
> versions
> > > of
> > > >>> HBase which (by their view) should be completely compatible with
> one
> > > >>> another.
> > > >>>
> > > >>> Take this extremely hypothetical situation: I'm new to HBase and
> > start
> > > >>> writing some code against HBase 1.0.1 which was just deployed at my
> > > >>> $job. I
> > > >>> don't _know_ what APIs are new, I just know what exists and treat
> > that
> > > as
> > > >>> acceptable for me to be using. Meanwhile in production, some other
> > > people
> > > >>> find a bug with HBase 1.0.1 and roll back to 1.0.0 which they had
> > been
> > > >>> previously using. My reaction would be "of course my code should
> work
> > > >>> with
> > > >>> HBase 1.0.0, I only used the public API" when in fact this is not
> > true.
> > > >>>
> > > >>> Personally, I think it's a little bold to say semver is even in use
> > if
> > > >>> this principal isn't being followed as it doesn't follow at all
> with
> > my
> > > >>> understanding on the guarantees defined by semver for bug-fix
> > releases.
> > > >>>
> > > >>> That being said, if the intent *is* to allow ourselves to make
> these
> > > >>> sorts
> > > >>> of changes, I just think some sort of disclaimer should be present:
> > > >>>
> > > >>> - HBase uses Semantic Versioning for its release versioning
> > > >>> + HBase uses Semantic Versioning for its release versioning with a
> > > caveat
> > > >>> that methods and members might be added in newer bug-fix releases
> > that
> > > >>> were
> > > >>> not present in the previous bug-fix release.
> > > >>>
> > > >>>
> > > >>> Andrew Purtell wrote:
> > > >>>
> > > >>> [Subject changed]
> > > >>>>
> > > >>>> On Tue, Apr 21, 2015 at 8:47 PM, Josh Elser<jo...@gmail.com>
> > > >>>> wrote:
> > > >>>>
> > > >>>> I was a little surprised when I noticed method additions to
> > > >>>>
> > > >>>>> InterfaceAudience.Public annotated classes. This means that a
> user
> > > >>>>> could
> > > >>>>> write code against 1.0.1 that would not work against 1.0.0 which
> > > seems
> > > >>>>> undesirable for a bugfix release. I read over the book section on
> > > >>>>> compatibility and didn't see this addressed, so I thought I'd
> ask.
> > > >>>>>
> > > >>>>>
> > > >>>> Let's clarify this. It's not the first time this question has been
> > > >>>> asked.
> > > >>>>
> > > >>>> To get things moving:
> > > >>>>
> > > >>>> I propose the following addition to the "Client API compatibility"
> > > >>>> section
> > > >>>> of Section 11.1:
> > > >>>>
> > > >>>> + APIs available in a patch version will be available in all later
> > > >>>> + patch versions. However, new APIs may be added which will not be
> > > >>>> + available in earlier patch versions.
> > > >>>>
> > > >>>> I propose the following change to the "Client Binary
> compatibility"
> > > >>>> section
> > > >>>> of Section 11.1:
> > > >>>>
> > > >>>> - Old client code can run unchanged (no recompilation needed)
> > against
> > > >>>> new
> > > >>>> jars.
> > > >>>> + Client code written to APIs available in a given patch release
> > > >>>> + can run unchanged (no recompilation needed) against the new
> > > >>>> + jars of later patch versions.
> > > >>>>
> > > >>>>
> > > >>>> What do you think?
> > > >>>>
> > > >>>> If these changes are (mostly) ok, then this clarifies in one
> > > direction.
> > > >>>>
> > > >>>> If these changes are not acceptable, I will propose edits that
> > clarify
> > > >>>> toward the opposite meaning.
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>
> > > >>
> > >
> > >
> > > --
> > > Best regards,
> > >
> > > - Andy
> > >
> > > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > > (via Tom White)
> > >
> >
>
>
>
> --
> Sean
>
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Sean Busbey <bu...@cloudera.com>.
Why don't we just focus development after a minor release on the next minor
release instead of the next patch release?
We could limit backports to the patch releases to critical bugs, which
would cut down on how often someone has to deal with the pain of making
sure we don't add to public APIs. It also reduces the risk someone going
through an upgrade has, since there are fewer changes.
If someone fixes a bug and doesn't want to do the work of making sure it
doesn't add methods in a patch release, they just don't backport to that
version and make a follow on e.g. "backport to 1.0.z" ticket.
On Thu, Apr 23, 2015 at 1:50 PM, Enis Söztutar <en...@gmail.com> wrote:
> +1 to the proposal.
>
> The problem is that we have a very big API surface especially with the
> coprocessors included in the report. Even simple bug fixes can introduce
> protected or public methods to base classes, which makes patch releases
> very hard to maintain. I would not want to spend the effort to spend tons
> of time trying to make a patch not introduce new methods in order to
> backport. That effort can be spent elsewhere IMO.
>
> Looking at the report
> https://people.apache.org/~enis/1.0.0_1.0.1RC2_compat_report.html, nothing
> strikes me as "new functionality". Going from current 1.0.0 to 1.0.1RC2
> should actually be as you would expect from upgrading a patch release.
>
> Yes, adding new API in patch releases will make downgrading harder, but I
> think that is an acceptable tradeoff. We can document that if your
> application compiles (meaning that you are not using new API) with 1.0.0,
> then you can swap your jars in a binary compat manner.
>
> Enis
>
> On Thu, Apr 23, 2015 at 10:03 AM, Andrew Purtell <ap...@apache.org>
> wrote:
>
> > Anyone disagree with the point of view put forward by Josh and Sean?
> >
> >
> > On Wed, Apr 22, 2015 at 10:35 AM, Josh Elser <jo...@gmail.com>
> wrote:
> >
> > > Andy -- I understood your intent, but thanks for clarifying. (as well
> as
> > > taking the time to break this discussion out in the first place). I
> agree
> > > with your assessment.
> > >
> > > re: Sean's comments, if it wasn't clear by me asking in the first
> place,
> > I
> > > also think sticking as close as possible to semver's rules is the best
> > > approach, although I'm getting the impression that there have been some
> > > previous reservations to doing so (especially by your comment about
> > > backporting features if there is demand is).
> > >
> > > I've found adhering to the bug-fix release restrictions can be a very
> > > painful and time-consuming task, so this is something to get a
> > > representative sampling of those who do the work to make sure everyone
> is
> > > on board.
> > >
> > >
> > > Sean Busbey wrote:
> > >
> > >> I'd much rather we stick with the definitions used in Semantic
> > Versioning.
> > >> Our use is already confusing enough given our matrix of
> compatibilities
> > >> that don't get "major version for breaking" protections.
> > >>
> > >> We've previously discussed how we'll do additional minor releases when
> > >> there's sufficient interest in the new features present there. What's
> > >> building that demand if any backwards compatible change can go back
> > into a
> > >> patch release?
> > >>
> > >> Would we have an easier time restraining ourselves if we had a regular
> > >> schedule planned around new minor versions?
> > >>
> > >>
> > >> On Wed, Apr 22, 2015 at 12:03 PM, Josh Elser<jo...@gmail.com>
> > >> wrote:
> > >>
> > >> While I can understand the desire to want to add things, I do think
> it
> > >>> makes things harder for users to reliably write code against versions
> > of
> > >>> HBase which (by their view) should be completely compatible with one
> > >>> another.
> > >>>
> > >>> Take this extremely hypothetical situation: I'm new to HBase and
> start
> > >>> writing some code against HBase 1.0.1 which was just deployed at my
> > >>> $job. I
> > >>> don't _know_ what APIs are new, I just know what exists and treat
> that
> > as
> > >>> acceptable for me to be using. Meanwhile in production, some other
> > people
> > >>> find a bug with HBase 1.0.1 and roll back to 1.0.0 which they had
> been
> > >>> previously using. My reaction would be "of course my code should work
> > >>> with
> > >>> HBase 1.0.0, I only used the public API" when in fact this is not
> true.
> > >>>
> > >>> Personally, I think it's a little bold to say semver is even in use
> if
> > >>> this principal isn't being followed as it doesn't follow at all with
> my
> > >>> understanding on the guarantees defined by semver for bug-fix
> releases.
> > >>>
> > >>> That being said, if the intent *is* to allow ourselves to make these
> > >>> sorts
> > >>> of changes, I just think some sort of disclaimer should be present:
> > >>>
> > >>> - HBase uses Semantic Versioning for its release versioning
> > >>> + HBase uses Semantic Versioning for its release versioning with a
> > caveat
> > >>> that methods and members might be added in newer bug-fix releases
> that
> > >>> were
> > >>> not present in the previous bug-fix release.
> > >>>
> > >>>
> > >>> Andrew Purtell wrote:
> > >>>
> > >>> [Subject changed]
> > >>>>
> > >>>> On Tue, Apr 21, 2015 at 8:47 PM, Josh Elser<jo...@gmail.com>
> > >>>> wrote:
> > >>>>
> > >>>> I was a little surprised when I noticed method additions to
> > >>>>
> > >>>>> InterfaceAudience.Public annotated classes. This means that a user
> > >>>>> could
> > >>>>> write code against 1.0.1 that would not work against 1.0.0 which
> > seems
> > >>>>> undesirable for a bugfix release. I read over the book section on
> > >>>>> compatibility and didn't see this addressed, so I thought I'd ask.
> > >>>>>
> > >>>>>
> > >>>> Let's clarify this. It's not the first time this question has been
> > >>>> asked.
> > >>>>
> > >>>> To get things moving:
> > >>>>
> > >>>> I propose the following addition to the "Client API compatibility"
> > >>>> section
> > >>>> of Section 11.1:
> > >>>>
> > >>>> + APIs available in a patch version will be available in all later
> > >>>> + patch versions. However, new APIs may be added which will not be
> > >>>> + available in earlier patch versions.
> > >>>>
> > >>>> I propose the following change to the "Client Binary compatibility"
> > >>>> section
> > >>>> of Section 11.1:
> > >>>>
> > >>>> - Old client code can run unchanged (no recompilation needed)
> against
> > >>>> new
> > >>>> jars.
> > >>>> + Client code written to APIs available in a given patch release
> > >>>> + can run unchanged (no recompilation needed) against the new
> > >>>> + jars of later patch versions.
> > >>>>
> > >>>>
> > >>>> What do you think?
> > >>>>
> > >>>> If these changes are (mostly) ok, then this clarifies in one
> > direction.
> > >>>>
> > >>>> If these changes are not acceptable, I will propose edits that
> clarify
> > >>>> toward the opposite meaning.
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>
> > >>
> >
> >
> > --
> > Best regards,
> >
> > - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
>
--
Sean
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Josh Elser <jo...@gmail.com>.
Enis Söztutar wrote:
> +1 to the proposal.
>
> The problem is that we have a very big API surface especially with the
> coprocessors included in the report. Even simple bug fixes can introduce
> protected or public methods to base classes, which makes patch releases
> very hard to maintain. I would not want to spend the effort to spend tons
> of time trying to make a patch not introduce new methods in order to
> backport. That effort can be spent elsewhere IMO.
Abso-friggin-lutely. Following bug-fix strictness is a _massive_ pain in
the rear (been dealing with this over in Accumulo-landia and it sucks).
IMO, the crux of what to consider here is finding the acceptable level
to which HBase Devs want to hold themselves and how much knowledge is
just expected of users.
Like you said, it is a lot of effort for what seems like ultimately very
little effect on users. It will take constant effort by everyone landing
patches in a bug-fix branch to not make the RM want to punch themself in
the face.
> Looking at the report
> https://people.apache.org/~enis/1.0.0_1.0.1RC2_compat_report.html, nothing
> strikes me as "new functionality". Going from current 1.0.0 to 1.0.1RC2
> should actually be as you would expect from upgrading a patch release.
>
> Yes, adding new API in patch releases will make downgrading harder, but I
> think that is an acceptable tradeoff. We can document that if your
> application compiles (meaning that you are not using new API) with 1.0.0,
> then you can swap your jars in a binary compat manner.
Like you pointed out in the other thread, though, choosing to follow
semver is for the user. Your proposed solution adds in extra complexity
on users that would otherwise be as simple as flipping the version on
their Maven/Ivy configuration. Like you say, they should still be able
to do things with the new jars against the old cluster, but it is a
sharp-corner users would need to be aware of that isn't semver as
advertised.
Really, I'm trying to play devil's advocate here. In the end, my only
concern would be appropriate advertisement of what decision is made.
> Enis
>
> On Thu, Apr 23, 2015 at 10:03 AM, Andrew Purtell<ap...@apache.org>
> wrote:
>
>> Anyone disagree with the point of view put forward by Josh and Sean?
>>
>>
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Enis Söztutar <en...@gmail.com>.
+1 to the proposal.
The problem is that we have a very big API surface especially with the
coprocessors included in the report. Even simple bug fixes can introduce
protected or public methods to base classes, which makes patch releases
very hard to maintain. I would not want to spend the effort to spend tons
of time trying to make a patch not introduce new methods in order to
backport. That effort can be spent elsewhere IMO.
Looking at the report
https://people.apache.org/~enis/1.0.0_1.0.1RC2_compat_report.html, nothing
strikes me as "new functionality". Going from current 1.0.0 to 1.0.1RC2
should actually be as you would expect from upgrading a patch release.
Yes, adding new API in patch releases will make downgrading harder, but I
think that is an acceptable tradeoff. We can document that if your
application compiles (meaning that you are not using new API) with 1.0.0,
then you can swap your jars in a binary compat manner.
Enis
On Thu, Apr 23, 2015 at 10:03 AM, Andrew Purtell <ap...@apache.org>
wrote:
> Anyone disagree with the point of view put forward by Josh and Sean?
>
>
> On Wed, Apr 22, 2015 at 10:35 AM, Josh Elser <jo...@gmail.com> wrote:
>
> > Andy -- I understood your intent, but thanks for clarifying. (as well as
> > taking the time to break this discussion out in the first place). I agree
> > with your assessment.
> >
> > re: Sean's comments, if it wasn't clear by me asking in the first place,
> I
> > also think sticking as close as possible to semver's rules is the best
> > approach, although I'm getting the impression that there have been some
> > previous reservations to doing so (especially by your comment about
> > backporting features if there is demand is).
> >
> > I've found adhering to the bug-fix release restrictions can be a very
> > painful and time-consuming task, so this is something to get a
> > representative sampling of those who do the work to make sure everyone is
> > on board.
> >
> >
> > Sean Busbey wrote:
> >
> >> I'd much rather we stick with the definitions used in Semantic
> Versioning.
> >> Our use is already confusing enough given our matrix of compatibilities
> >> that don't get "major version for breaking" protections.
> >>
> >> We've previously discussed how we'll do additional minor releases when
> >> there's sufficient interest in the new features present there. What's
> >> building that demand if any backwards compatible change can go back
> into a
> >> patch release?
> >>
> >> Would we have an easier time restraining ourselves if we had a regular
> >> schedule planned around new minor versions?
> >>
> >>
> >> On Wed, Apr 22, 2015 at 12:03 PM, Josh Elser<jo...@gmail.com>
> >> wrote:
> >>
> >> While I can understand the desire to want to add things, I do think it
> >>> makes things harder for users to reliably write code against versions
> of
> >>> HBase which (by their view) should be completely compatible with one
> >>> another.
> >>>
> >>> Take this extremely hypothetical situation: I'm new to HBase and start
> >>> writing some code against HBase 1.0.1 which was just deployed at my
> >>> $job. I
> >>> don't _know_ what APIs are new, I just know what exists and treat that
> as
> >>> acceptable for me to be using. Meanwhile in production, some other
> people
> >>> find a bug with HBase 1.0.1 and roll back to 1.0.0 which they had been
> >>> previously using. My reaction would be "of course my code should work
> >>> with
> >>> HBase 1.0.0, I only used the public API" when in fact this is not true.
> >>>
> >>> Personally, I think it's a little bold to say semver is even in use if
> >>> this principal isn't being followed as it doesn't follow at all with my
> >>> understanding on the guarantees defined by semver for bug-fix releases.
> >>>
> >>> That being said, if the intent *is* to allow ourselves to make these
> >>> sorts
> >>> of changes, I just think some sort of disclaimer should be present:
> >>>
> >>> - HBase uses Semantic Versioning for its release versioning
> >>> + HBase uses Semantic Versioning for its release versioning with a
> caveat
> >>> that methods and members might be added in newer bug-fix releases that
> >>> were
> >>> not present in the previous bug-fix release.
> >>>
> >>>
> >>> Andrew Purtell wrote:
> >>>
> >>> [Subject changed]
> >>>>
> >>>> On Tue, Apr 21, 2015 at 8:47 PM, Josh Elser<jo...@gmail.com>
> >>>> wrote:
> >>>>
> >>>> I was a little surprised when I noticed method additions to
> >>>>
> >>>>> InterfaceAudience.Public annotated classes. This means that a user
> >>>>> could
> >>>>> write code against 1.0.1 that would not work against 1.0.0 which
> seems
> >>>>> undesirable for a bugfix release. I read over the book section on
> >>>>> compatibility and didn't see this addressed, so I thought I'd ask.
> >>>>>
> >>>>>
> >>>> Let's clarify this. It's not the first time this question has been
> >>>> asked.
> >>>>
> >>>> To get things moving:
> >>>>
> >>>> I propose the following addition to the "Client API compatibility"
> >>>> section
> >>>> of Section 11.1:
> >>>>
> >>>> + APIs available in a patch version will be available in all later
> >>>> + patch versions. However, new APIs may be added which will not be
> >>>> + available in earlier patch versions.
> >>>>
> >>>> I propose the following change to the "Client Binary compatibility"
> >>>> section
> >>>> of Section 11.1:
> >>>>
> >>>> - Old client code can run unchanged (no recompilation needed) against
> >>>> new
> >>>> jars.
> >>>> + Client code written to APIs available in a given patch release
> >>>> + can run unchanged (no recompilation needed) against the new
> >>>> + jars of later patch versions.
> >>>>
> >>>>
> >>>> What do you think?
> >>>>
> >>>> If these changes are (mostly) ok, then this clarifies in one
> direction.
> >>>>
> >>>> If these changes are not acceptable, I will propose edits that clarify
> >>>> toward the opposite meaning.
> >>>>
> >>>>
> >>>>
> >>>>
> >>
> >>
>
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Andrew Purtell <ap...@apache.org>.
Anyone disagree with the point of view put forward by Josh and Sean?
On Wed, Apr 22, 2015 at 10:35 AM, Josh Elser <jo...@gmail.com> wrote:
> Andy -- I understood your intent, but thanks for clarifying. (as well as
> taking the time to break this discussion out in the first place). I agree
> with your assessment.
>
> re: Sean's comments, if it wasn't clear by me asking in the first place, I
> also think sticking as close as possible to semver's rules is the best
> approach, although I'm getting the impression that there have been some
> previous reservations to doing so (especially by your comment about
> backporting features if there is demand is).
>
> I've found adhering to the bug-fix release restrictions can be a very
> painful and time-consuming task, so this is something to get a
> representative sampling of those who do the work to make sure everyone is
> on board.
>
>
> Sean Busbey wrote:
>
>> I'd much rather we stick with the definitions used in Semantic Versioning.
>> Our use is already confusing enough given our matrix of compatibilities
>> that don't get "major version for breaking" protections.
>>
>> We've previously discussed how we'll do additional minor releases when
>> there's sufficient interest in the new features present there. What's
>> building that demand if any backwards compatible change can go back into a
>> patch release?
>>
>> Would we have an easier time restraining ourselves if we had a regular
>> schedule planned around new minor versions?
>>
>>
>> On Wed, Apr 22, 2015 at 12:03 PM, Josh Elser<jo...@gmail.com>
>> wrote:
>>
>> While I can understand the desire to want to add things, I do think it
>>> makes things harder for users to reliably write code against versions of
>>> HBase which (by their view) should be completely compatible with one
>>> another.
>>>
>>> Take this extremely hypothetical situation: I'm new to HBase and start
>>> writing some code against HBase 1.0.1 which was just deployed at my
>>> $job. I
>>> don't _know_ what APIs are new, I just know what exists and treat that as
>>> acceptable for me to be using. Meanwhile in production, some other people
>>> find a bug with HBase 1.0.1 and roll back to 1.0.0 which they had been
>>> previously using. My reaction would be "of course my code should work
>>> with
>>> HBase 1.0.0, I only used the public API" when in fact this is not true.
>>>
>>> Personally, I think it's a little bold to say semver is even in use if
>>> this principal isn't being followed as it doesn't follow at all with my
>>> understanding on the guarantees defined by semver for bug-fix releases.
>>>
>>> That being said, if the intent *is* to allow ourselves to make these
>>> sorts
>>> of changes, I just think some sort of disclaimer should be present:
>>>
>>> - HBase uses Semantic Versioning for its release versioning
>>> + HBase uses Semantic Versioning for its release versioning with a caveat
>>> that methods and members might be added in newer bug-fix releases that
>>> were
>>> not present in the previous bug-fix release.
>>>
>>>
>>> Andrew Purtell wrote:
>>>
>>> [Subject changed]
>>>>
>>>> On Tue, Apr 21, 2015 at 8:47 PM, Josh Elser<jo...@gmail.com>
>>>> wrote:
>>>>
>>>> I was a little surprised when I noticed method additions to
>>>>
>>>>> InterfaceAudience.Public annotated classes. This means that a user
>>>>> could
>>>>> write code against 1.0.1 that would not work against 1.0.0 which seems
>>>>> undesirable for a bugfix release. I read over the book section on
>>>>> compatibility and didn't see this addressed, so I thought I'd ask.
>>>>>
>>>>>
>>>> Let's clarify this. It's not the first time this question has been
>>>> asked.
>>>>
>>>> To get things moving:
>>>>
>>>> I propose the following addition to the "Client API compatibility"
>>>> section
>>>> of Section 11.1:
>>>>
>>>> + APIs available in a patch version will be available in all later
>>>> + patch versions. However, new APIs may be added which will not be
>>>> + available in earlier patch versions.
>>>>
>>>> I propose the following change to the "Client Binary compatibility"
>>>> section
>>>> of Section 11.1:
>>>>
>>>> - Old client code can run unchanged (no recompilation needed) against
>>>> new
>>>> jars.
>>>> + Client code written to APIs available in a given patch release
>>>> + can run unchanged (no recompilation needed) against the new
>>>> + jars of later patch versions.
>>>>
>>>>
>>>> What do you think?
>>>>
>>>> If these changes are (mostly) ok, then this clarifies in one direction.
>>>>
>>>> If these changes are not acceptable, I will propose edits that clarify
>>>> toward the opposite meaning.
>>>>
>>>>
>>>>
>>>>
>>
>>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Josh Elser <jo...@gmail.com>.
Andy -- I understood your intent, but thanks for clarifying. (as well as
taking the time to break this discussion out in the first place). I
agree with your assessment.
re: Sean's comments, if it wasn't clear by me asking in the first place,
I also think sticking as close as possible to semver's rules is the best
approach, although I'm getting the impression that there have been some
previous reservations to doing so (especially by your comment about
backporting features if there is demand is).
I've found adhering to the bug-fix release restrictions can be a very
painful and time-consuming task, so this is something to get a
representative sampling of those who do the work to make sure everyone
is on board.
Sean Busbey wrote:
> I'd much rather we stick with the definitions used in Semantic Versioning.
> Our use is already confusing enough given our matrix of compatibilities
> that don't get "major version for breaking" protections.
>
> We've previously discussed how we'll do additional minor releases when
> there's sufficient interest in the new features present there. What's
> building that demand if any backwards compatible change can go back into a
> patch release?
>
> Would we have an easier time restraining ourselves if we had a regular
> schedule planned around new minor versions?
>
>
> On Wed, Apr 22, 2015 at 12:03 PM, Josh Elser<jo...@gmail.com> wrote:
>
>> While I can understand the desire to want to add things, I do think it
>> makes things harder for users to reliably write code against versions of
>> HBase which (by their view) should be completely compatible with one
>> another.
>>
>> Take this extremely hypothetical situation: I'm new to HBase and start
>> writing some code against HBase 1.0.1 which was just deployed at my $job. I
>> don't _know_ what APIs are new, I just know what exists and treat that as
>> acceptable for me to be using. Meanwhile in production, some other people
>> find a bug with HBase 1.0.1 and roll back to 1.0.0 which they had been
>> previously using. My reaction would be "of course my code should work with
>> HBase 1.0.0, I only used the public API" when in fact this is not true.
>>
>> Personally, I think it's a little bold to say semver is even in use if
>> this principal isn't being followed as it doesn't follow at all with my
>> understanding on the guarantees defined by semver for bug-fix releases.
>>
>> That being said, if the intent *is* to allow ourselves to make these sorts
>> of changes, I just think some sort of disclaimer should be present:
>>
>> - HBase uses Semantic Versioning for its release versioning
>> + HBase uses Semantic Versioning for its release versioning with a caveat
>> that methods and members might be added in newer bug-fix releases that were
>> not present in the previous bug-fix release.
>>
>>
>> Andrew Purtell wrote:
>>
>>> [Subject changed]
>>>
>>> On Tue, Apr 21, 2015 at 8:47 PM, Josh Elser<jo...@gmail.com> wrote:
>>>
>>> I was a little surprised when I noticed method additions to
>>>> InterfaceAudience.Public annotated classes. This means that a user could
>>>> write code against 1.0.1 that would not work against 1.0.0 which seems
>>>> undesirable for a bugfix release. I read over the book section on
>>>> compatibility and didn't see this addressed, so I thought I'd ask.
>>>>
>>>
>>> Let's clarify this. It's not the first time this question has been asked.
>>>
>>> To get things moving:
>>>
>>> I propose the following addition to the "Client API compatibility" section
>>> of Section 11.1:
>>>
>>> + APIs available in a patch version will be available in all later
>>> + patch versions. However, new APIs may be added which will not be
>>> + available in earlier patch versions.
>>>
>>> I propose the following change to the "Client Binary compatibility"
>>> section
>>> of Section 11.1:
>>>
>>> - Old client code can run unchanged (no recompilation needed) against new
>>> jars.
>>> + Client code written to APIs available in a given patch release
>>> + can run unchanged (no recompilation needed) against the new
>>> + jars of later patch versions.
>>>
>>>
>>> What do you think?
>>>
>>> If these changes are (mostly) ok, then this clarifies in one direction.
>>>
>>> If these changes are not acceptable, I will propose edits that clarify
>>> toward the opposite meaning.
>>>
>>>
>>>
>
>
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Sean Busbey <bu...@cloudera.com>.
I'd much rather we stick with the definitions used in Semantic Versioning.
Our use is already confusing enough given our matrix of compatibilities
that don't get "major version for breaking" protections.
We've previously discussed how we'll do additional minor releases when
there's sufficient interest in the new features present there. What's
building that demand if any backwards compatible change can go back into a
patch release?
Would we have an easier time restraining ourselves if we had a regular
schedule planned around new minor versions?
On Wed, Apr 22, 2015 at 12:03 PM, Josh Elser <jo...@gmail.com> wrote:
> While I can understand the desire to want to add things, I do think it
> makes things harder for users to reliably write code against versions of
> HBase which (by their view) should be completely compatible with one
> another.
>
> Take this extremely hypothetical situation: I'm new to HBase and start
> writing some code against HBase 1.0.1 which was just deployed at my $job. I
> don't _know_ what APIs are new, I just know what exists and treat that as
> acceptable for me to be using. Meanwhile in production, some other people
> find a bug with HBase 1.0.1 and roll back to 1.0.0 which they had been
> previously using. My reaction would be "of course my code should work with
> HBase 1.0.0, I only used the public API" when in fact this is not true.
>
> Personally, I think it's a little bold to say semver is even in use if
> this principal isn't being followed as it doesn't follow at all with my
> understanding on the guarantees defined by semver for bug-fix releases.
>
> That being said, if the intent *is* to allow ourselves to make these sorts
> of changes, I just think some sort of disclaimer should be present:
>
> - HBase uses Semantic Versioning for its release versioning
> + HBase uses Semantic Versioning for its release versioning with a caveat
> that methods and members might be added in newer bug-fix releases that were
> not present in the previous bug-fix release.
>
>
> Andrew Purtell wrote:
>
>> [Subject changed]
>>
>> On Tue, Apr 21, 2015 at 8:47 PM, Josh Elser<jo...@gmail.com> wrote:
>>
>> I was a little surprised when I noticed method additions to
>>> InterfaceAudience.Public annotated classes. This means that a user could
>>> write code against 1.0.1 that would not work against 1.0.0 which seems
>>> undesirable for a bugfix release. I read over the book section on
>>> compatibility and didn't see this addressed, so I thought I'd ask.
>>>
>>
>>
>> Let's clarify this. It's not the first time this question has been asked.
>>
>> To get things moving:
>>
>> I propose the following addition to the "Client API compatibility" section
>> of Section 11.1:
>>
>> + APIs available in a patch version will be available in all later
>> + patch versions. However, new APIs may be added which will not be
>> + available in earlier patch versions.
>>
>> I propose the following change to the "Client Binary compatibility"
>> section
>> of Section 11.1:
>>
>> - Old client code can run unchanged (no recompilation needed) against new
>> jars.
>> + Client code written to APIs available in a given patch release
>> + can run unchanged (no recompilation needed) against the new
>> + jars of later patch versions.
>>
>>
>> What do you think?
>>
>> If these changes are (mostly) ok, then this clarifies in one direction.
>>
>> If these changes are not acceptable, I will propose edits that clarify
>> toward the opposite meaning.
>>
>>
>>
--
Sean
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Andrew Purtell <ap...@apache.org>.
Just to clarify something, I've proposed edits that clarify the de-facto
practice, since additional methods are turning up in patch releases, but am
not taking a position.
So far we don't have consensus.
> + HBase uses Semantic Versioning for its release versioning with a caveat
that methods and members might be added in newer bug-fix releases that were
not present in the previous bug-fix release.
This is a good point. We should make this change too, to be consistent, or
if we don't want to make this change, then we should:
- Reject my proposal (I will make a new one clarifying with the opposite
meaning)
- Sink the current 1.0.1 RC
- Remove any changes in branch-1.0 that added new methods to the API
surface before proposing a new 1.0.1 RC
On Wed, Apr 22, 2015 at 10:03 AM, Josh Elser <jo...@gmail.com> wrote:
> While I can understand the desire to want to add things, I do think it
> makes things harder for users to reliably write code against versions of
> HBase which (by their view) should be completely compatible with one
> another.
>
> Take this extremely hypothetical situation: I'm new to HBase and start
> writing some code against HBase 1.0.1 which was just deployed at my $job. I
> don't _know_ what APIs are new, I just know what exists and treat that as
> acceptable for me to be using. Meanwhile in production, some other people
> find a bug with HBase 1.0.1 and roll back to 1.0.0 which they had been
> previously using. My reaction would be "of course my code should work with
> HBase 1.0.0, I only used the public API" when in fact this is not true.
>
> Personally, I think it's a little bold to say semver is even in use if
> this principal isn't being followed as it doesn't follow at all with my
> understanding on the guarantees defined by semver for bug-fix releases.
>
> That being said, if the intent *is* to allow ourselves to make these sorts
> of changes, I just think some sort of disclaimer should be present:
>
> - HBase uses Semantic Versioning for its release versioning
> + HBase uses Semantic Versioning for its release versioning with a caveat
> that methods and members might be added in newer bug-fix releases that were
> not present in the previous bug-fix release.
>
>
> Andrew Purtell wrote:
>
>> [Subject changed]
>>
>> On Tue, Apr 21, 2015 at 8:47 PM, Josh Elser<jo...@gmail.com> wrote:
>>
>> I was a little surprised when I noticed method additions to
>>> InterfaceAudience.Public annotated classes. This means that a user could
>>> write code against 1.0.1 that would not work against 1.0.0 which seems
>>> undesirable for a bugfix release. I read over the book section on
>>> compatibility and didn't see this addressed, so I thought I'd ask.
>>>
>>
>>
>> Let's clarify this. It's not the first time this question has been asked.
>>
>> To get things moving:
>>
>> I propose the following addition to the "Client API compatibility" section
>> of Section 11.1:
>>
>> + APIs available in a patch version will be available in all later
>> + patch versions. However, new APIs may be added which will not be
>> + available in earlier patch versions.
>>
>> I propose the following change to the "Client Binary compatibility"
>> section
>> of Section 11.1:
>>
>> - Old client code can run unchanged (no recompilation needed) against new
>> jars.
>> + Client code written to APIs available in a given patch release
>> + can run unchanged (no recompilation needed) against the new
>> + jars of later patch versions.
>>
>>
>> What do you think?
>>
>> If these changes are (mostly) ok, then this clarifies in one direction.
>>
>> If these changes are not acceptable, I will propose edits that clarify
>> toward the opposite meaning.
>>
>>
>>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: Clarifying interface evolution freedom in patch releases (was:
Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Posted by Josh Elser <jo...@gmail.com>.
While I can understand the desire to want to add things, I do think it
makes things harder for users to reliably write code against versions of
HBase which (by their view) should be completely compatible with one
another.
Take this extremely hypothetical situation: I'm new to HBase and start
writing some code against HBase 1.0.1 which was just deployed at my
$job. I don't _know_ what APIs are new, I just know what exists and
treat that as acceptable for me to be using. Meanwhile in production,
some other people find a bug with HBase 1.0.1 and roll back to 1.0.0
which they had been previously using. My reaction would be "of course my
code should work with HBase 1.0.0, I only used the public API" when in
fact this is not true.
Personally, I think it's a little bold to say semver is even in use if
this principal isn't being followed as it doesn't follow at all with my
understanding on the guarantees defined by semver for bug-fix releases.
That being said, if the intent *is* to allow ourselves to make these
sorts of changes, I just think some sort of disclaimer should be present:
- HBase uses Semantic Versioning for its release versioning
+ HBase uses Semantic Versioning for its release versioning with a
caveat that methods and members might be added in newer bug-fix releases
that were not present in the previous bug-fix release.
Andrew Purtell wrote:
> [Subject changed]
>
> On Tue, Apr 21, 2015 at 8:47 PM, Josh Elser<jo...@gmail.com> wrote:
>
>> I was a little surprised when I noticed method additions to
>> InterfaceAudience.Public annotated classes. This means that a user could
>> write code against 1.0.1 that would not work against 1.0.0 which seems
>> undesirable for a bugfix release. I read over the book section on
>> compatibility and didn't see this addressed, so I thought I'd ask.
>
>
> Let's clarify this. It's not the first time this question has been asked.
>
> To get things moving:
>
> I propose the following addition to the "Client API compatibility" section
> of Section 11.1:
>
> + APIs available in a patch version will be available in all later
> + patch versions. However, new APIs may be added which will not be
> + available in earlier patch versions.
>
> I propose the following change to the "Client Binary compatibility" section
> of Section 11.1:
>
> - Old client code can run unchanged (no recompilation needed) against new
> jars.
> + Client code written to APIs available in a given patch release
> + can run unchanged (no recompilation needed) against the new
> + jars of later patch versions.
>
>
> What do you think?
>
> If these changes are (mostly) ok, then this clarifies in one direction.
>
> If these changes are not acceptable, I will propose edits that clarify
> toward the opposite meaning.
>
>