You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Josh Elser <jo...@gmail.com> on 2016/08/15 19:07:22 UTC
Source incompatibility between 1.2.[0,1] and 1.2.2 breaks Phoenix
Hi folks,
I just noticed a ticket over in Phoenix [1] in which some method
additions to the Table interface [2] breaks the source compatibility of
Phoenix with HBase 1.2.2.
My understanding of the current guidelines is that these are allowed as
they do not invalidate binary compatibility of clients using the API.
Personally, I am very hard-pressed to use the word "compatible" in a
sentence describing this change that isn't sarcastic :)
A couple of questions:
1) I find the InterfaceAudience annotations on this really strange. How
can we have a Public audience Interface (o.a.h.h.c.Table) with Private
methods? Is that just "how things are", or am I missing something? If
this is not something that's meant to be public, I would think these new
methods should be defined in a non-public interface.
2) Now that we've had quite a few releases in the "not-quite-SemVer"
compatibility guide, is there any interest in trying to make the
compatibility guarantees more stringent? I would like to see us get to
source-compatibility on patch releases, not just binary compatibility. I
am happy to try to help, but I know I don't have the time to devote to
catch everything.
3) What do people think about changing this in a 1.2.3 with a quick
turn-around?
Thanks!
- Josh
[1] https://issues.apache.org/jira/browse/PHOENIX-3116
[2] https://issues.apache.org/jira/browse/HBASE-15645
Re: Source incompatibility between 1.2.[0,1] and 1.2.2 breaks Phoenix
Posted by Josh Elser <jo...@gmail.com>.
Thanks for the great reply, Andrew!
Andrew Purtell wrote:
> \u200b
> I find the InterfaceAudience annotations on this really strange. How can
> we have a Public audience Interface (o.a.h.h.c.Table) with Private methods?
>
> I'm also not sure the Private annotations on the Table interface are that
> useful. Any change to an interface renders it source incompatible, and
> removal (or effective removal via signature change) of methods from an
> interface introduces a binary incompatibility. I think the Private
> annotations on methods of a Public interface imply we should refactor those
> methods to a non-public interface.
I agree. This is how I would've expected to see such an
non-public-facing method to have been added.
> Now that we've had quite a few releases in the "not-quite-SemVer"
> compatibility guide, is there any interest in trying to make the
> compatibility guarantees more stringent?
>
> I was looking at our compat guidelines (
> http://hbase.apache.org/book.html#hbase.versioning) and think we could make
> a few refinements.
>
> We make several allowances for public interface changes that are binary
> compatible but not source compatible in patch releases. I think we are only
> taking into consideration callers but should also consider implementors of
> public interfaces. Changing a public interface on a patch release renders
> it source incompatible. Can we avoid doing that on *patch* releases, and
> restrict this type of change to minors?
>
> Although we make allowances for public interface changes that are binary
> compatible we also say "A minor upgrade requires no application/client code
> modification." which could imply source compatibility even across minors,
> which I believe is not the intent.
>
> We could add a source compatibility dimension for patch releases.
I like the way this sounds. Would it make sense to try to work on
terminology to encapsulate this (after getting some more consensus that
this is desirable)?
> I would like to see us get to source-compatibility on patch releases, not
> just binary compatibility
>
> You mean source compatibility for Public annotated interfaces only, right?
>
> In that case evaluating compliance would be easy: RMs would run the API
> compat checker on a RC and if a patch release the number of binary and
> source compat issues should both be zero.
Yes, sorry, I could've been more specified. Source-compatibility on the
"public API" (defined presently by the Public audience annotation).
Re: Source incompatibility between 1.2.[0,1] and 1.2.2 breaks Phoenix
Posted by Dima Spivak <ds...@cloudera.com>.
I have HBASE-16158 open to run check_compatibility.sh as part of our Yetus
personality, I just got sidetracked by other work. Let me try to get
something up by week's end.
-Dima
On Mon, Aug 15, 2016 at 5:15 PM, Stack <st...@duboce.net> wrote:
> On Mon, Aug 15, 2016 at 12:21 PM, Andrew Purtell <ap...@apache.org>
> wrote:
>
> > ...
>
>
> > I was looking at our compat guidelines (
> > http://hbase.apache.org/book.html#hbase.versioning) and think we could
> > make
> > a few refinements.
> >
> > We make several allowances for public interface changes that are binary
> > compatible but not source compatible in patch releases. I think we are
> only
> > taking into consideration callers but should also consider implementors
> of
> > public interfaces. Changing a public interface on a patch release renders
> > it source incompatible. Can we avoid doing that on *patch* releases, and
> > restrict this type of change to minors?
> >
> > Although we make allowances for public interface changes that are binary
> > compatible we also say "A minor upgrade requires no application/client
> code
> > modification." which could imply source compatibility even across minors,
> > which I believe is not the intent.
> >
> > We could add a source compatibility dimension for patch releases.
> >
> >
>
> Makes sense Andrew.
>
>
>
> > >
> >
> > I would like to see us get to source-compatibility on patch releases, not
> > just binary compatibility
> >
> > You mean source compatibility for Public annotated interfaces only,
> right?
> >
> > In that case evaluating compliance would be easy: RMs would run the API
> > compat checker on a RC and if a patch release the number of binary and
> > source compat issues should both be zero.
> >
> >
> Can we have yetus report on compat?
>
> I'd be up for helping get out a 1.2.3 (and a fix in 1.1.6) to address this
> compat hiccup especially given I was party to the change. I +1'd and
> committed the patch thinking addition of methods ok not thinking of the
> Table implementors.
>
> St.Ack
>
>
>
>
> >
> > On Mon, Aug 15, 2016 at 12:07 PM, Josh Elser <jo...@gmail.com>
> wrote:
> >
> > > Hi folks,
> > >
> > > I just noticed a ticket over in Phoenix [1] in which some method
> > additions
> > > to the Table interface [2] breaks the source compatibility of Phoenix
> > with
> > > HBase 1.2.2.
> > >
> > > My understanding of the current guidelines is that these are allowed as
> > > they do not invalidate binary compatibility of clients using the API.
> > > Personally, I am very hard-pressed to use the word "compatible" in a
> > > sentence describing this change that isn't sarcastic :)
> > >
> > > A couple of questions:
> > >
> > > 1)
> > >
> > > I find the InterfaceAudience annotations on this really strange. How
> can
> > > we have a Public audience Interface (o.a.h.h.c.Table) with Private
> > methods?
> > > Is that just "how things are", or am I missing something? If this is
> not
> > > something that's meant to be public, I would think these new methods
> > should
> > > be defined in a non-public interface.
> > >
> > > 2)
> > >
> > > Now that we've had quite a few releases in the "not-quite-SemVer"
> > > compatibility guide, is there any interest in trying to make the
> > > compatibility guarantees more stringent?
> > >
> > > I would like to see us get to source-compatibility on patch releases,
> not
> > > just binary compatibility. I am happy to try to help, but I know I
> don't
> > > have the time to devote to catch everything.
> > >
> > > 3) What do people think about changing this in a 1.2.3 with a quick
> > > turn-around?
> > >
> > > Thanks!
> > >
> > > - Josh
> > >
> > > [1] https://issues.apache.org/jira/browse/PHOENIX-3116
> > > [2] https://issues.apache.org/jira/browse/HBASE-15645
> > >
> >
> >
> >
> > --
> > Best regards,
> >
> > - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
>
--
-Dima
Re: Source incompatibility between 1.2.[0,1] and 1.2.2 breaks Phoenix
Posted by Stack <st...@duboce.net>.
On Mon, Aug 15, 2016 at 12:21 PM, Andrew Purtell <ap...@apache.org>
wrote:
> ...
> I was looking at our compat guidelines (
> http://hbase.apache.org/book.html#hbase.versioning) and think we could
> make
> a few refinements.
>
> We make several allowances for public interface changes that are binary
> compatible but not source compatible in patch releases. I think we are only
> taking into consideration callers but should also consider implementors of
> public interfaces. Changing a public interface on a patch release renders
> it source incompatible. Can we avoid doing that on *patch* releases, and
> restrict this type of change to minors?
>
> Although we make allowances for public interface changes that are binary
> compatible we also say "A minor upgrade requires no application/client code
> modification." which could imply source compatibility even across minors,
> which I believe is not the intent.
>
> We could add a source compatibility dimension for patch releases.
>
>
Makes sense Andrew.
> >
>
> I would like to see us get to source-compatibility on patch releases, not
> just binary compatibility
>
> You mean source compatibility for Public annotated interfaces only, right?
>
> In that case evaluating compliance would be easy: RMs would run the API
> compat checker on a RC and if a patch release the number of binary and
> source compat issues should both be zero.
>
>
Can we have yetus report on compat?
I'd be up for helping get out a 1.2.3 (and a fix in 1.1.6) to address this
compat hiccup especially given I was party to the change. I +1'd and
committed the patch thinking addition of methods ok not thinking of the
Table implementors.
St.Ack
>
> On Mon, Aug 15, 2016 at 12:07 PM, Josh Elser <jo...@gmail.com> wrote:
>
> > Hi folks,
> >
> > I just noticed a ticket over in Phoenix [1] in which some method
> additions
> > to the Table interface [2] breaks the source compatibility of Phoenix
> with
> > HBase 1.2.2.
> >
> > My understanding of the current guidelines is that these are allowed as
> > they do not invalidate binary compatibility of clients using the API.
> > Personally, I am very hard-pressed to use the word "compatible" in a
> > sentence describing this change that isn't sarcastic :)
> >
> > A couple of questions:
> >
> > 1)
> >
> > I find the InterfaceAudience annotations on this really strange. How can
> > we have a Public audience Interface (o.a.h.h.c.Table) with Private
> methods?
> > Is that just "how things are", or am I missing something? If this is not
> > something that's meant to be public, I would think these new methods
> should
> > be defined in a non-public interface.
> >
> > 2)
> >
> > Now that we've had quite a few releases in the "not-quite-SemVer"
> > compatibility guide, is there any interest in trying to make the
> > compatibility guarantees more stringent?
> >
> > I would like to see us get to source-compatibility on patch releases, not
> > just binary compatibility. I am happy to try to help, but I know I don't
> > have the time to devote to catch everything.
> >
> > 3) What do people think about changing this in a 1.2.3 with a quick
> > turn-around?
> >
> > Thanks!
> >
> > - Josh
> >
> > [1] https://issues.apache.org/jira/browse/PHOENIX-3116
> > [2] https://issues.apache.org/jira/browse/HBASE-15645
> >
>
>
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
Re: Source incompatibility between 1.2.[0,1] and 1.2.2 breaks Phoenix
Posted by Andrew Purtell <ap...@apache.org>.
> 1) starting to have a distinction between places we expect folks to just
>
consume API vs extend it?
> I used to be in favor of this, but Andrew's concern on bikeshedding has
me
>
reconsidering. Simpler rules seem better both to reduce the complexity of
>
communicating downstream and what the RMs have to keep in their heads when
>
evaluating changes.
P
lease prefer simpler rules, and checks and analyses that can be pushed
into tooling like RAT, the compat API checker, Yetus, and make_rc.sh. The
RM process has a fairly high cognitive load: Apache release requirements
(like dependency analysis and NOTICE and LICENSE spot-checks and audits),
metadata assembly (eg CHANGES.txt), the manual steps of the staging and
release workflow, test hygiene, integration testing, etc.
On Tue, Aug 16, 2016 at 8:20 AM, Sean Busbey <bu...@apache.org> wrote:
> On Tue, Aug 16, 2016 at 6:40 AM, Stack <st...@duboce.net> wrote:
>
> > On Tue, Aug 16, 2016 at 1:07 AM, Phil Yang <ud...@gmail.com> wrote:
> >
> > > I notice that HBASE-15645 is made up of several commits and revert in
> > git,
> > > maybe it is more convenient to apply a new patch to remove the added
> > > methods.
> > >
> > > Created a new issue (HBASE-16420
> > > <https://issues.apache.org/jira/browse/HBASE-16420>) and waiting for
> the
> > > result of pre-commit build. The patch only fix the incompatibility of
> 1.1
> > > and 1.2. Do we need keep the compatibility between 1.x branches? If so
> > we
> > > need also remove new methods from branch-1.3 and branch-1. And seems
> some
> > > other issues also added new methods to non-Private interface on
> > > branch-1/1.3...
> > >
> > >
> >
> > Patch looks good Phil. Thanks for putting it up.
> >
> > On your question, adding API in a manner that breaks compile is allowed
> > going by our updated understanding.
> >
> >
> This should be "is not allowed" right?
>
> My reading of the consensus in the thread thus far is that adding methods
> to IA.Public interfaces won't be possible in the 1.y line of releases due
> to breaking source compatibility, but in the 2.y line we'll be able to use
> new Java 8 features to do so.
>
> Probably worth a different thread or two, but what do folks think about
>
> 1) starting to have a distinction between places we expect folks to just
> consume API vs extend it?
>
> I used to be in favor of this, but Andrew's concern on bikeshedding has me
> reconsidering. Simpler rules seem better both to reduce the complexity of
> communicating downstream and what the RMs have to keep in their heads when
> evaluating changes.
>
> 2) dropping our use of IS.Stable, IS.Evolving, etc.
>
> I've never found this distinction particularly useful since we aren't very
> consistent on it. The compat guide only specifies what it means for "server
> side APIs" without really defining what that means precisely, and we use it
> in client APIs as well. I think a reasonable reading of our current guide
> could take the IS.Evolving designation to mean that the breaking change to
> Table was okay, but reading the comments on PHOENIX-3116 that
> interpretation would clearly be surprising to at least one set of
> downstream folks. (Plus none of the discussions I saw around this change
> used this rationale, so its presence or lack thereof wouldn't have changed
> the conversation.)
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: Source incompatibility between 1.2.[0,1] and 1.2.2 breaks Phoenix
Posted by Stack <st...@duboce.net>.
On Fri, Aug 19, 2016 at 8:45 AM, Nick Dimiduk <nd...@gmail.com> wrote:
> My bandwidth for tracking this thread has been limited. Have we concluded
> here that HBASE-16420 is the only fix we need before the next round of RCs?
>
>
Yes.
There is also the tightening of our compatibility guarantees on patch
versions done in doc via HBASE-16422.
> I think we do, right? Isn't this exactly what the Bigtable Cloud folks
are doing? They would appear to have our blessing.
You have a point.
St.Ack
> On Tuesday, August 16, 2016, Josh Elser <jo...@gmail.com> wrote:
>
> > (top-post since I can't find a better place to respond to everyone who
> > chimed in here)
> >
> > Huge thanks, everyone! This was absolutely the best email thread (and
> JIRA
> > issue) I could've come back to after not keeping up with email for the
> day.
> >
> > Stack wrote:
> >
> >> On Tue, Aug 16, 2016 at 8:20 AM, Sean Busbey<bu...@apache.org> wrote:
> >>
> >> On Tue, Aug 16, 2016 at 6:40 AM, Stack<st...@duboce.net> wrote:
> >>>
> >>> On Tue, Aug 16, 2016 at 1:07 AM, Phil Yang<ud...@gmail.com> wrote:
> >>>>
> >>>> I notice that HBASE-15645 is made up of several commits and revert in
> >>>>>
> >>>> git,
> >>>>
> >>>>> maybe it is more convenient to apply a new patch to remove the added
> >>>>> methods.
> >>>>>
> >>>>> Created a new issue (HBASE-16420
> >>>>> <https://issues.apache.org/jira/browse/HBASE-16420>) and waiting for
> >>>>>
> >>>> the
> >>>
> >>>> result of pre-commit build. The patch only fix the incompatibility of
> >>>>>
> >>>> 1.1
> >>>
> >>>> and 1.2. Do we need keep the compatibility between 1.x branches? If
> so
> >>>>>
> >>>> we
> >>>>
> >>>>> need also remove new methods from branch-1.3 and branch-1. And seems
> >>>>>
> >>>> some
> >>>
> >>>> other issues also added new methods to non-Private interface on
> >>>>> branch-1/1.3...
> >>>>>
> >>>>>
> >>>>> Patch looks good Phil. Thanks for putting it up.
> >>>>
> >>>> On your question, adding API in a manner that breaks compile is
> allowed
> >>>> going by our updated understanding.
> >>>>
> >>>>
> >>>> This should be "is not allowed" right?
> >>>
> >>>
> >>> Thanks Sean. Yes (Doc says right thing in HBASE-16422 patch)
> >>
> >>
> >>
> >> My reading of the consensus in the thread thus far is that adding
> methods
> >>> to IA.Public interfaces won't be possible in the 1.y line of releases
> due
> >>> to breaking source compatibility,
> >>>
> >>
> >>
> >>
> >> HBASE-16422 makes exclusion for patch release only. It does not preclude
> >> our breaking on minor versions.
> >>
> >>
> >>
> >> 1) starting to have a distinction between places we expect folks to just
> >>> consume API vs extend it?
> >>>
> >>> I used to be in favor of this, but Andrew's concern on bikeshedding has
> >>> me
> >>> reconsidering. Simpler rules seem better both to reduce the complexity
> of
> >>> communicating downstream and what the RMs have to keep in their heads
> >>> when
> >>> evaluating changes.
> >>>
> >>>
> >>> This and the below would be better on another thread I'd say Sean.
> >>
> >> Lets keep this thread for handling this little compat episode.
> >>
> >> Thanks,
> >> St.Ack
> >>
> >>
> >>
> >> 2) dropping our use of IS.Stable, IS.Evolving, etc.
> >>>
> >>> I've never found this distinction particularly useful since we aren't
> >>> very
> >>> consistent on it. The compat guide only specifies what it means for
> >>> "server
> >>> side APIs" without really defining what that means precisely, and we
> use
> >>> it
> >>> in client APIs as well. I think a reasonable reading of our current
> guide
> >>> could take the IS.Evolving designation to mean that the breaking change
> >>> to
> >>> Table was okay, but reading the comments on PHOENIX-3116 that
> >>> interpretation would clearly be surprising to at least one set of
> >>> downstream folks. (Plus none of the discussions I saw around this
> change
> >>> used this rationale, so its presence or lack thereof wouldn't have
> >>> changed
> >>> the conversation.)
> >>>
> >>>
> >>
>
Re: Source incompatibility between 1.2.[0,1] and 1.2.2 breaks Phoenix
Posted by Nick Dimiduk <nd...@gmail.com>.
> Are we extending this to all interfaces? Do we support folks
implementing their own Connection? Admin?
I think we do, right? Isn't this exactly what the Bigtable Cloud folks are
doing? They would appear to have our blessing.
On Friday, August 19, 2016, Nick Dimiduk <nd...@gmail.com> wrote:
> My bandwidth for tracking this thread has been limited. Have we concluded
> here that HBASE-16420 is the only fix we need before the next round of
> RCs?
>
> On Tuesday, August 16, 2016, Josh Elser <josh.elser@gmail.com
> <javascript:_e(%7B%7D,'cvml','josh.elser@gmail.com');>> wrote:
>
>> (top-post since I can't find a better place to respond to everyone who
>> chimed in here)
>>
>> Huge thanks, everyone! This was absolutely the best email thread (and
>> JIRA issue) I could've come back to after not keeping up with email for the
>> day.
>>
>> Stack wrote:
>>
>>> On Tue, Aug 16, 2016 at 8:20 AM, Sean Busbey<bu...@apache.org> wrote:
>>>
>>> On Tue, Aug 16, 2016 at 6:40 AM, Stack<st...@duboce.net> wrote:
>>>>
>>>> On Tue, Aug 16, 2016 at 1:07 AM, Phil Yang<ud...@gmail.com> wrote:
>>>>>
>>>>> I notice that HBASE-15645 is made up of several commits and revert in
>>>>>>
>>>>> git,
>>>>>
>>>>>> maybe it is more convenient to apply a new patch to remove the added
>>>>>> methods.
>>>>>>
>>>>>> Created a new issue (HBASE-16420
>>>>>> <https://issues.apache.org/jira/browse/HBASE-16420>) and waiting for
>>>>>>
>>>>> the
>>>>
>>>>> result of pre-commit build. The patch only fix the incompatibility of
>>>>>>
>>>>> 1.1
>>>>
>>>>> and 1.2. Do we need keep the compatibility between 1.x branches? If so
>>>>>>
>>>>> we
>>>>>
>>>>>> need also remove new methods from branch-1.3 and branch-1. And seems
>>>>>>
>>>>> some
>>>>
>>>>> other issues also added new methods to non-Private interface on
>>>>>> branch-1/1.3...
>>>>>>
>>>>>>
>>>>>> Patch looks good Phil. Thanks for putting it up.
>>>>>
>>>>> On your question, adding API in a manner that breaks compile is allowed
>>>>> going by our updated understanding.
>>>>>
>>>>>
>>>>> This should be "is not allowed" right?
>>>>
>>>>
>>>> Thanks Sean. Yes (Doc says right thing in HBASE-16422 patch)
>>>
>>>
>>>
>>> My reading of the consensus in the thread thus far is that adding methods
>>>> to IA.Public interfaces won't be possible in the 1.y line of releases
>>>> due
>>>> to breaking source compatibility,
>>>>
>>>
>>>
>>>
>>> HBASE-16422 makes exclusion for patch release only. It does not preclude
>>> our breaking on minor versions.
>>>
>>>
>>>
>>> 1) starting to have a distinction between places we expect folks to just
>>>> consume API vs extend it?
>>>>
>>>> I used to be in favor of this, but Andrew's concern on bikeshedding has
>>>> me
>>>> reconsidering. Simpler rules seem better both to reduce the complexity
>>>> of
>>>> communicating downstream and what the RMs have to keep in their heads
>>>> when
>>>> evaluating changes.
>>>>
>>>>
>>>> This and the below would be better on another thread I'd say Sean.
>>>
>>> Lets keep this thread for handling this little compat episode.
>>>
>>> Thanks,
>>> St.Ack
>>>
>>>
>>>
>>> 2) dropping our use of IS.Stable, IS.Evolving, etc.
>>>>
>>>> I've never found this distinction particularly useful since we aren't
>>>> very
>>>> consistent on it. The compat guide only specifies what it means for
>>>> "server
>>>> side APIs" without really defining what that means precisely, and we
>>>> use it
>>>> in client APIs as well. I think a reasonable reading of our current
>>>> guide
>>>> could take the IS.Evolving designation to mean that the breaking change
>>>> to
>>>> Table was okay, but reading the comments on PHOENIX-3116 that
>>>> interpretation would clearly be surprising to at least one set of
>>>> downstream folks. (Plus none of the discussions I saw around this change
>>>> used this rationale, so its presence or lack thereof wouldn't have
>>>> changed
>>>> the conversation.)
>>>>
>>>>
>>>
Re: Source incompatibility between 1.2.[0,1] and 1.2.2 breaks Phoenix
Posted by Nick Dimiduk <nd...@gmail.com>.
My bandwidth for tracking this thread has been limited. Have we concluded
here that HBASE-16420 is the only fix we need before the next round of RCs?
On Tuesday, August 16, 2016, Josh Elser <jo...@gmail.com> wrote:
> (top-post since I can't find a better place to respond to everyone who
> chimed in here)
>
> Huge thanks, everyone! This was absolutely the best email thread (and JIRA
> issue) I could've come back to after not keeping up with email for the day.
>
> Stack wrote:
>
>> On Tue, Aug 16, 2016 at 8:20 AM, Sean Busbey<bu...@apache.org> wrote:
>>
>> On Tue, Aug 16, 2016 at 6:40 AM, Stack<st...@duboce.net> wrote:
>>>
>>> On Tue, Aug 16, 2016 at 1:07 AM, Phil Yang<ud...@gmail.com> wrote:
>>>>
>>>> I notice that HBASE-15645 is made up of several commits and revert in
>>>>>
>>>> git,
>>>>
>>>>> maybe it is more convenient to apply a new patch to remove the added
>>>>> methods.
>>>>>
>>>>> Created a new issue (HBASE-16420
>>>>> <https://issues.apache.org/jira/browse/HBASE-16420>) and waiting for
>>>>>
>>>> the
>>>
>>>> result of pre-commit build. The patch only fix the incompatibility of
>>>>>
>>>> 1.1
>>>
>>>> and 1.2. Do we need keep the compatibility between 1.x branches? If so
>>>>>
>>>> we
>>>>
>>>>> need also remove new methods from branch-1.3 and branch-1. And seems
>>>>>
>>>> some
>>>
>>>> other issues also added new methods to non-Private interface on
>>>>> branch-1/1.3...
>>>>>
>>>>>
>>>>> Patch looks good Phil. Thanks for putting it up.
>>>>
>>>> On your question, adding API in a manner that breaks compile is allowed
>>>> going by our updated understanding.
>>>>
>>>>
>>>> This should be "is not allowed" right?
>>>
>>>
>>> Thanks Sean. Yes (Doc says right thing in HBASE-16422 patch)
>>
>>
>>
>> My reading of the consensus in the thread thus far is that adding methods
>>> to IA.Public interfaces won't be possible in the 1.y line of releases due
>>> to breaking source compatibility,
>>>
>>
>>
>>
>> HBASE-16422 makes exclusion for patch release only. It does not preclude
>> our breaking on minor versions.
>>
>>
>>
>> 1) starting to have a distinction between places we expect folks to just
>>> consume API vs extend it?
>>>
>>> I used to be in favor of this, but Andrew's concern on bikeshedding has
>>> me
>>> reconsidering. Simpler rules seem better both to reduce the complexity of
>>> communicating downstream and what the RMs have to keep in their heads
>>> when
>>> evaluating changes.
>>>
>>>
>>> This and the below would be better on another thread I'd say Sean.
>>
>> Lets keep this thread for handling this little compat episode.
>>
>> Thanks,
>> St.Ack
>>
>>
>>
>> 2) dropping our use of IS.Stable, IS.Evolving, etc.
>>>
>>> I've never found this distinction particularly useful since we aren't
>>> very
>>> consistent on it. The compat guide only specifies what it means for
>>> "server
>>> side APIs" without really defining what that means precisely, and we use
>>> it
>>> in client APIs as well. I think a reasonable reading of our current guide
>>> could take the IS.Evolving designation to mean that the breaking change
>>> to
>>> Table was okay, but reading the comments on PHOENIX-3116 that
>>> interpretation would clearly be surprising to at least one set of
>>> downstream folks. (Plus none of the discussions I saw around this change
>>> used this rationale, so its presence or lack thereof wouldn't have
>>> changed
>>> the conversation.)
>>>
>>>
>>
Re: Source incompatibility between 1.2.[0,1] and 1.2.2 breaks Phoenix
Posted by Josh Elser <jo...@gmail.com>.
(top-post since I can't find a better place to respond to everyone who
chimed in here)
Huge thanks, everyone! This was absolutely the best email thread (and
JIRA issue) I could've come back to after not keeping up with email for
the day.
Stack wrote:
> On Tue, Aug 16, 2016 at 8:20 AM, Sean Busbey<bu...@apache.org> wrote:
>
>> On Tue, Aug 16, 2016 at 6:40 AM, Stack<st...@duboce.net> wrote:
>>
>>> On Tue, Aug 16, 2016 at 1:07 AM, Phil Yang<ud...@gmail.com> wrote:
>>>
>>>> I notice that HBASE-15645 is made up of several commits and revert in
>>> git,
>>>> maybe it is more convenient to apply a new patch to remove the added
>>>> methods.
>>>>
>>>> Created a new issue (HBASE-16420
>>>> <https://issues.apache.org/jira/browse/HBASE-16420>) and waiting for
>> the
>>>> result of pre-commit build. The patch only fix the incompatibility of
>> 1.1
>>>> and 1.2. Do we need keep the compatibility between 1.x branches? If so
>>> we
>>>> need also remove new methods from branch-1.3 and branch-1. And seems
>> some
>>>> other issues also added new methods to non-Private interface on
>>>> branch-1/1.3...
>>>>
>>>>
>>> Patch looks good Phil. Thanks for putting it up.
>>>
>>> On your question, adding API in a manner that breaks compile is allowed
>>> going by our updated understanding.
>>>
>>>
>> This should be "is not allowed" right?
>>
>>
> Thanks Sean. Yes (Doc says right thing in HBASE-16422 patch)
>
>
>
>> My reading of the consensus in the thread thus far is that adding methods
>> to IA.Public interfaces won't be possible in the 1.y line of releases due
>> to breaking source compatibility,
>
>
>
> HBASE-16422 makes exclusion for patch release only. It does not preclude
> our breaking on minor versions.
>
>
>
>> 1) starting to have a distinction between places we expect folks to just
>> consume API vs extend it?
>>
>> I used to be in favor of this, but Andrew's concern on bikeshedding has me
>> reconsidering. Simpler rules seem better both to reduce the complexity of
>> communicating downstream and what the RMs have to keep in their heads when
>> evaluating changes.
>>
>>
> This and the below would be better on another thread I'd say Sean.
>
> Lets keep this thread for handling this little compat episode.
>
> Thanks,
> St.Ack
>
>
>
>> 2) dropping our use of IS.Stable, IS.Evolving, etc.
>>
>> I've never found this distinction particularly useful since we aren't very
>> consistent on it. The compat guide only specifies what it means for "server
>> side APIs" without really defining what that means precisely, and we use it
>> in client APIs as well. I think a reasonable reading of our current guide
>> could take the IS.Evolving designation to mean that the breaking change to
>> Table was okay, but reading the comments on PHOENIX-3116 that
>> interpretation would clearly be surprising to at least one set of
>> downstream folks. (Plus none of the discussions I saw around this change
>> used this rationale, so its presence or lack thereof wouldn't have changed
>> the conversation.)
>>
>
Re: Source incompatibility between 1.2.[0,1] and 1.2.2 breaks Phoenix
Posted by Stack <st...@duboce.net>.
On Tue, Aug 16, 2016 at 8:20 AM, Sean Busbey <bu...@apache.org> wrote:
> On Tue, Aug 16, 2016 at 6:40 AM, Stack <st...@duboce.net> wrote:
>
> > On Tue, Aug 16, 2016 at 1:07 AM, Phil Yang <ud...@gmail.com> wrote:
> >
> > > I notice that HBASE-15645 is made up of several commits and revert in
> > git,
> > > maybe it is more convenient to apply a new patch to remove the added
> > > methods.
> > >
> > > Created a new issue (HBASE-16420
> > > <https://issues.apache.org/jira/browse/HBASE-16420>) and waiting for
> the
> > > result of pre-commit build. The patch only fix the incompatibility of
> 1.1
> > > and 1.2. Do we need keep the compatibility between 1.x branches? If so
> > we
> > > need also remove new methods from branch-1.3 and branch-1. And seems
> some
> > > other issues also added new methods to non-Private interface on
> > > branch-1/1.3...
> > >
> > >
> >
> > Patch looks good Phil. Thanks for putting it up.
> >
> > On your question, adding API in a manner that breaks compile is allowed
> > going by our updated understanding.
> >
> >
> This should be "is not allowed" right?
>
>
Thanks Sean. Yes (Doc says right thing in HBASE-16422 patch)
> My reading of the consensus in the thread thus far is that adding methods
> to IA.Public interfaces won't be possible in the 1.y line of releases due
> to breaking source compatibility,
HBASE-16422 makes exclusion for patch release only. It does not preclude
our breaking on minor versions.
>
> 1) starting to have a distinction between places we expect folks to just
> consume API vs extend it?
>
> I used to be in favor of this, but Andrew's concern on bikeshedding has me
> reconsidering. Simpler rules seem better both to reduce the complexity of
> communicating downstream and what the RMs have to keep in their heads when
> evaluating changes.
>
>
This and the below would be better on another thread I'd say Sean.
Lets keep this thread for handling this little compat episode.
Thanks,
St.Ack
> 2) dropping our use of IS.Stable, IS.Evolving, etc.
>
> I've never found this distinction particularly useful since we aren't very
> consistent on it. The compat guide only specifies what it means for "server
> side APIs" without really defining what that means precisely, and we use it
> in client APIs as well. I think a reasonable reading of our current guide
> could take the IS.Evolving designation to mean that the breaking change to
> Table was okay, but reading the comments on PHOENIX-3116 that
> interpretation would clearly be surprising to at least one set of
> downstream folks. (Plus none of the discussions I saw around this change
> used this rationale, so its presence or lack thereof wouldn't have changed
> the conversation.)
>
Re: Source incompatibility between 1.2.[0,1] and 1.2.2 breaks Phoenix
Posted by Sean Busbey <bu...@apache.org>.
On Tue, Aug 16, 2016 at 6:40 AM, Stack <st...@duboce.net> wrote:
> On Tue, Aug 16, 2016 at 1:07 AM, Phil Yang <ud...@gmail.com> wrote:
>
> > I notice that HBASE-15645 is made up of several commits and revert in
> git,
> > maybe it is more convenient to apply a new patch to remove the added
> > methods.
> >
> > Created a new issue (HBASE-16420
> > <https://issues.apache.org/jira/browse/HBASE-16420>) and waiting for the
> > result of pre-commit build. The patch only fix the incompatibility of 1.1
> > and 1.2. Do we need keep the compatibility between 1.x branches? If so
> we
> > need also remove new methods from branch-1.3 and branch-1. And seems some
> > other issues also added new methods to non-Private interface on
> > branch-1/1.3...
> >
> >
>
> Patch looks good Phil. Thanks for putting it up.
>
> On your question, adding API in a manner that breaks compile is allowed
> going by our updated understanding.
>
>
This should be "is not allowed" right?
My reading of the consensus in the thread thus far is that adding methods
to IA.Public interfaces won't be possible in the 1.y line of releases due
to breaking source compatibility, but in the 2.y line we'll be able to use
new Java 8 features to do so.
Probably worth a different thread or two, but what do folks think about
1) starting to have a distinction between places we expect folks to just
consume API vs extend it?
I used to be in favor of this, but Andrew's concern on bikeshedding has me
reconsidering. Simpler rules seem better both to reduce the complexity of
communicating downstream and what the RMs have to keep in their heads when
evaluating changes.
2) dropping our use of IS.Stable, IS.Evolving, etc.
I've never found this distinction particularly useful since we aren't very
consistent on it. The compat guide only specifies what it means for "server
side APIs" without really defining what that means precisely, and we use it
in client APIs as well. I think a reasonable reading of our current guide
could take the IS.Evolving designation to mean that the breaking change to
Table was okay, but reading the comments on PHOENIX-3116 that
interpretation would clearly be surprising to at least one set of
downstream folks. (Plus none of the discussions I saw around this change
used this rationale, so its presence or lack thereof wouldn't have changed
the conversation.)
Re: Source incompatibility between 1.2.[0,1] and 1.2.2 breaks Phoenix
Posted by Stack <st...@duboce.net>.
On Tue, Aug 16, 2016 at 1:07 AM, Phil Yang <ud...@gmail.com> wrote:
> I notice that HBASE-15645 is made up of several commits and revert in git,
> maybe it is more convenient to apply a new patch to remove the added
> methods.
>
> Created a new issue (HBASE-16420
> <https://issues.apache.org/jira/browse/HBASE-16420>) and waiting for the
> result of pre-commit build. The patch only fix the incompatibility of 1.1
> and 1.2. Do we need keep the compatibility between 1.x branches? If so we
> need also remove new methods from branch-1.3 and branch-1. And seems some
> other issues also added new methods to non-Private interface on
> branch-1/1.3...
>
>
Patch looks good Phil. Thanks for putting it up.
On your question, adding API in a manner that breaks compile is allowed
going by our updated understanding.
I pushed suggested tightenting text for what our story is across patch
versions here: HBASE-16422
Working w/ Sean to roll a hbase-1.2.3 RC with your reverts.
Thanks for prompt fix.
St.Ack
> Thanks,
> Phil
>
>
> 2016-08-16 15:09 GMT+08:00 Phil Yang <ud...@gmail.com>:
>
> > Sorry for breaking the compatibility of Table interface. As the author
> > of HBASE-15645, I can help making a new patch that only fixes the bug and
> > not adding any new methods to Table.
> >
>
Re: Source incompatibility between 1.2.[0,1] and 1.2.2 breaks Phoenix
Posted by Phil Yang <ud...@gmail.com>.
I notice that HBASE-15645 is made up of several commits and revert in git,
maybe it is more convenient to apply a new patch to remove the added
methods.
Created a new issue (HBASE-16420
<https://issues.apache.org/jira/browse/HBASE-16420>) and waiting for the
result of pre-commit build. The patch only fix the incompatibility of 1.1
and 1.2. Do we need keep the compatibility between 1.x branches? If so we
need also remove new methods from branch-1.3 and branch-1. And seems some
other issues also added new methods to non-Private interface on
branch-1/1.3...
Thanks,
Phil
2016-08-16 15:09 GMT+08:00 Phil Yang <ud...@gmail.com>:
> Sorry for breaking the compatibility of Table interface. As the author
> of HBASE-15645, I can help making a new patch that only fixes the bug and
> not adding any new methods to Table.
>
Re: Source incompatibility between 1.2.[0,1] and 1.2.2 breaks Phoenix
Posted by Phil Yang <ud...@gmail.com>.
Sorry for breaking the compatibility of Table interface. As the author
of HBASE-15645, I can help making a new patch that only fixes the bug and
not adding any new methods to Table.
Re: Source incompatibility between 1.2.[0,1] and 1.2.2 breaks Phoenix
Posted by Sean Busbey <bu...@apache.org>.
On Aug 16, 2016 00:24, "张铎" <pa...@gmail.com> wrote:
>
> Yes, with jdk8 we could add new methods in interface without breaking the
> sub classes.
>
> So let's revert HBASE-15645 from branch-1 and branch-1,x and add default
> implementations in master? Is yetus OK to run pre-commit with jdk8 only on
> master now?
>
> Thanks.
>
Yetus has a work around available to do this, but AFAIK we haven't
implemented it yet.
HBASE-15624
Re: Source incompatibility between 1.2.[0,1] and 1.2.2 breaks Phoenix
Posted by 张铎 <pa...@gmail.com>.
Yes, with jdk8 we could add new methods in interface without breaking the
sub classes.
So let's revert HBASE-15645 from branch-1 and branch-1,x and add default
implementations in master? Is yetus OK to run pre-commit with jdk8 only on
master now?
Thanks.
2016-08-16 13:12 GMT+08:00 Sean Busbey <bu...@apache.org>:
> On Tue, Aug 16, 2016 at 12:09 AM, Sean Busbey <bu...@apache.org> wrote:
>
> >
> > Moving from interfaces to abstract classes requires a deprecation cycle,
> > so to transition I think we'd need to introduce a new API. Anyone up for
> > this in 2.0?
> >
>
> I just remembered that HBase 2.0 is moving to jdk8+, which means we could
> use default methods when adding to existing interfaces, so there's
> substantially less draw for moving to abstract classes.
>
Re: Source incompatibility between 1.2.[0,1] and 1.2.2 breaks Phoenix
Posted by Sean Busbey <bu...@apache.org>.
On Tue, Aug 16, 2016 at 12:09 AM, Sean Busbey <bu...@apache.org> wrote:
>
> Moving from interfaces to abstract classes requires a deprecation cycle,
> so to transition I think we'd need to introduce a new API. Anyone up for
> this in 2.0?
>
I just remembered that HBase 2.0 is moving to jdk8+, which means we could
use default methods when adding to existing interfaces, so there's
substantially less draw for moving to abstract classes.
Re: Source incompatibility between 1.2.[0,1] and 1.2.2 breaks Phoenix
Posted by Stack <st...@duboce.net>.
On Mon, Aug 15, 2016 at 10:09 PM, Sean Busbey <bu...@apache.org> wrote:
> On Mon, Aug 15, 2016 at 10:55 PM, Andrew Purtell <andrew.purtell@gmail.com
> >
> wrote:
>
> > > Are we extending this to all interfaces? Do we support folks
> > implementing their own Connection? Admin?
> >
> > This will bury us in weeds and bikeshedding. We can make a blanket rule
> > about source compatibility appropriately scoped to minors and/or patches
> > without that drama. To wit:
> >
> > > So when I read the existing guides, what I see is that we're supposed
> to
> > be _source_ compatible on minor and patch releases, but binary compatible
> > only on patch
> >
> > We should take the opportunity to clarify the language of the
> > compatibility guide. And if the above is the policy then the change to
> > Table is not allowed.
> >
> >
> so "yes". :) Works for me.
>
>
I'm good on the blanket rule on source and binary fix to spec. going
forward and a 1.2.3 tout de suite with revert of HBASE-15645
<https://issues.apache.org/jira/browse/HBASE-15645>.
St.Ack
Re: Source incompatibility between 1.2.[0,1] and 1.2.2 breaks Phoenix
Posted by Sean Busbey <bu...@apache.org>.
On Mon, Aug 15, 2016 at 10:55 PM, Andrew Purtell <an...@gmail.com>
wrote:
> > Are we extending this to all interfaces? Do we support folks
> implementing their own Connection? Admin?
>
> This will bury us in weeds and bikeshedding. We can make a blanket rule
> about source compatibility appropriately scoped to minors and/or patches
> without that drama. To wit:
>
> > So when I read the existing guides, what I see is that we're supposed to
> be _source_ compatible on minor and patch releases, but binary compatible
> only on patch
>
> We should take the opportunity to clarify the language of the
> compatibility guide. And if the above is the policy then the change to
> Table is not allowed.
>
>
so "yes". :) Works for me.
Moving from interfaces to abstract classes requires a deprecation cycle, so
to transition I think we'd need to introduce a new API. Anyone up for this
in 2.0?
Re: Source incompatibility between 1.2.[0,1] and 1.2.2 breaks Phoenix
Posted by ramkrishna vasudevan <ra...@gmail.com>.
I agree with Sean here.
Interfaces with @Private annotations are going to be a problem. I think we
should strongly discourage this. Either allow abstract classes are don't
allow such things. We should have a note about this in the compatibility
guide.
Even regarding the Util classes and the respective static methods - though
we mark APIs private, if we change or remove those APIs still it will have
compatability issues? We are following a practice of deprecating any API
in such Util classes and removing them only in major APIs but the ones
marked as @Private may be source of problem.
Regards
Ram
On Tue, Aug 16, 2016 at 9:25 AM, Andrew Purtell <an...@gmail.com>
wrote:
> > Are we extending this to all interfaces? Do we support folks
> implementing their own Connection? Admin?
>
> This will bury us in weeds and bikeshedding. We can make a blanket rule
> about source compatibility appropriately scoped to minors and/or patches
> without that drama. To wit:
>
> > So when I read the existing guides, what I see is that we're supposed to
> be _source_ compatible on minor and patch releases, but binary compatible
> only on patch
>
> We should take the opportunity to clarify the language of the
> compatibility guide. And if the above is the policy then the change to
> Table is not allowed.
>
>
> > On Aug 15, 2016, at 8:26 PM, Sean Busbey <bu...@apache.org> wrote:
> >
> > Thanks for bringing this up Josh!
> >
> >
> >
> > On Mon, Aug 15, 2016 at 2:21 PM, Andrew Purtell <ap...@apache.org>
> wrote:
> >
> >>
> >> I find the InterfaceAudience annotations on this really strange. How can
> >> we have a Public audience Interface (o.a.h.h.c.Table) with Private
> methods?
> >>
> >> I'm also not sure the Private annotations on the Table interface are
> that
> >> useful. Any change to an interface renders it source incompatible, and
> >> removal (or effective removal via signature change) of methods from an
> >> interface introduces a binary incompatibility. I think the Private
> >> annotations on methods of a Public interface imply we should refactor
> those
> >> methods to a non-public interface.
> >>
> >
> >
> > Generally, we use more restricted audience annotations on members within
> a
> > given API to show where there are limits on our support that we can't
> > express in Java access modifiers. I think it's important for us to keep
> > this around so that we can avoid a proliferation of "FooUtil" and other
> > classes that exist just to make a Public/Private distinction. (I
> recognize
> > we have several of these but I was hoping we'd move in the direction of
> > fewer over time.)
> >
> > I agree that in the case of interfaces it's problematic, since we have no
> > way to distinguish between "should be consumed" and "can be extended".
> > Perhaps we should make sure we have abstract classes instead of
> interfaces?
> > We could also add an annotation to make the consumed/extended
> distinction;
> > I've seen it come up a few times in other users of audience annotations.
> >
> >
> >
> >>
> >> Now that we've had quite a few releases in the "not-quite-SemVer"
> >> compatibility guide, is there any interest in trying to make the
> >> compatibility guarantees more stringent?
> >>
> >> I was looking at our compat guidelines (
> >> http://hbase.apache.org/book.html#hbase.versioning) and think we could
> >> make
> >> a few refinements.
> >>
> >> We make several allowances for public interface changes that are binary
> >> compatible but not source compatible in patch releases. I think we are
> only
> >> taking into consideration callers but should also consider implementors
> of
> >> public interfaces. Changing a public interface on a patch release
> renders
> >> it source incompatible. Can we avoid doing that on *patch* releases, and
> >> restrict this type of change to minors?
> > Are we extending this to all interfaces? Do we support folks implementing
> > their own Connection? Admin?
> >
> >
> >> Although we make allowances for public interface changes that are binary
> >> compatible we also say "A minor upgrade requires no application/client
> code
> >> modification." which could imply source compatibility even across
> minors,
> >> which I believe is not the intent.
> >>
> >> We could add a source compatibility dimension for patch releases.
> >>
> >>
> >>
> >> I would like to see us get to source-compatibility on patch releases,
> not
> >> just binary compatibility
> >>
> >> You mean source compatibility for Public annotated interfaces only,
> right?
> >>
> >> In that case evaluating compliance would be easy: RMs would run the API
> >> compat checker on a RC and if a patch release the number of binary and
> >> source compat issues should both be zero.
> > So when I read the existing guides, what I see is that we're supposed to
> be
> > _source_ compatible on minor and patch releases, but binary compatible
> only
> > on patch releases. I think we discussed this a year or two ago, and IIRC
> > the reasoning was that since we maintain wire compatibility on minor and
> > patch releases, those who need to ignore a given set of binary
> incompatible
> > changes can just make use of the older library.
>
Re: Source incompatibility between 1.2.[0,1] and 1.2.2 breaks Phoenix
Posted by Andrew Purtell <an...@gmail.com>.
> Are we extending this to all interfaces? Do we support folks implementing their own Connection? Admin?
This will bury us in weeds and bikeshedding. We can make a blanket rule about source compatibility appropriately scoped to minors and/or patches without that drama. To wit:
> So when I read the existing guides, what I see is that we're supposed to be _source_ compatible on minor and patch releases, but binary compatible only on patch
We should take the opportunity to clarify the language of the compatibility guide. And if the above is the policy then the change to Table is not allowed.
> On Aug 15, 2016, at 8:26 PM, Sean Busbey <bu...@apache.org> wrote:
>
> Thanks for bringing this up Josh!
>
>
>
> On Mon, Aug 15, 2016 at 2:21 PM, Andrew Purtell <ap...@apache.org> wrote:
>
>>
>> I find the InterfaceAudience annotations on this really strange. How can
>> we have a Public audience Interface (o.a.h.h.c.Table) with Private methods?
>>
>> I'm also not sure the Private annotations on the Table interface are that
>> useful. Any change to an interface renders it source incompatible, and
>> removal (or effective removal via signature change) of methods from an
>> interface introduces a binary incompatibility. I think the Private
>> annotations on methods of a Public interface imply we should refactor those
>> methods to a non-public interface.
>>
>
>
> Generally, we use more restricted audience annotations on members within a
> given API to show where there are limits on our support that we can't
> express in Java access modifiers. I think it's important for us to keep
> this around so that we can avoid a proliferation of "FooUtil" and other
> classes that exist just to make a Public/Private distinction. (I recognize
> we have several of these but I was hoping we'd move in the direction of
> fewer over time.)
>
> I agree that in the case of interfaces it's problematic, since we have no
> way to distinguish between "should be consumed" and "can be extended".
> Perhaps we should make sure we have abstract classes instead of interfaces?
> We could also add an annotation to make the consumed/extended distinction;
> I've seen it come up a few times in other users of audience annotations.
>
>
>
>>
>> Now that we've had quite a few releases in the "not-quite-SemVer"
>> compatibility guide, is there any interest in trying to make the
>> compatibility guarantees more stringent?
>>
>> I was looking at our compat guidelines (
>> http://hbase.apache.org/book.html#hbase.versioning) and think we could
>> make
>> a few refinements.
>>
>> We make several allowances for public interface changes that are binary
>> compatible but not source compatible in patch releases. I think we are only
>> taking into consideration callers but should also consider implementors of
>> public interfaces. Changing a public interface on a patch release renders
>> it source incompatible. Can we avoid doing that on *patch* releases, and
>> restrict this type of change to minors?
> Are we extending this to all interfaces? Do we support folks implementing
> their own Connection? Admin?
>
>
>> Although we make allowances for public interface changes that are binary
>> compatible we also say "A minor upgrade requires no application/client code
>> modification." which could imply source compatibility even across minors,
>> which I believe is not the intent.
>>
>> We could add a source compatibility dimension for patch releases.
>>
>>
>>
>> I would like to see us get to source-compatibility on patch releases, not
>> just binary compatibility
>>
>> You mean source compatibility for Public annotated interfaces only, right?
>>
>> In that case evaluating compliance would be easy: RMs would run the API
>> compat checker on a RC and if a patch release the number of binary and
>> source compat issues should both be zero.
> So when I read the existing guides, what I see is that we're supposed to be
> _source_ compatible on minor and patch releases, but binary compatible only
> on patch releases. I think we discussed this a year or two ago, and IIRC
> the reasoning was that since we maintain wire compatibility on minor and
> patch releases, those who need to ignore a given set of binary incompatible
> changes can just make use of the older library.
Re: Source incompatibility between 1.2.[0,1] and 1.2.2 breaks Phoenix
Posted by Sean Busbey <bu...@apache.org>.
Thanks for bringing this up Josh!
On Mon, Aug 15, 2016 at 2:21 PM, Andrew Purtell <ap...@apache.org> wrote:
> >
>
> I find the InterfaceAudience annotations on this really strange. How can
> we have a Public audience Interface (o.a.h.h.c.Table) with Private methods?
>
> I'm also not sure the Private annotations on the Table interface are that
> useful. Any change to an interface renders it source incompatible, and
> removal (or effective removal via signature change) of methods from an
> interface introduces a binary incompatibility. I think the Private
> annotations on methods of a Public interface imply we should refactor those
> methods to a non-public interface.
>
> >
>
Generally, we use more restricted audience annotations on members within a
given API to show where there are limits on our support that we can't
express in Java access modifiers. I think it's important for us to keep
this around so that we can avoid a proliferation of "FooUtil" and other
classes that exist just to make a Public/Private distinction. (I recognize
we have several of these but I was hoping we'd move in the direction of
fewer over time.)
I agree that in the case of interfaces it's problematic, since we have no
way to distinguish between "should be consumed" and "can be extended".
Perhaps we should make sure we have abstract classes instead of interfaces?
We could also add an annotation to make the consumed/extended distinction;
I've seen it come up a few times in other users of audience annotations.
>
> Now that we've had quite a few releases in the "not-quite-SemVer"
> compatibility guide, is there any interest in trying to make the
> compatibility guarantees more stringent?
>
> I was looking at our compat guidelines (
> http://hbase.apache.org/book.html#hbase.versioning) and think we could
> make
> a few refinements.
>
> We make several allowances for public interface changes that are binary
> compatible but not source compatible in patch releases. I think we are only
> taking into consideration callers but should also consider implementors of
> public interfaces. Changing a public interface on a patch release renders
> it source incompatible. Can we avoid doing that on *patch* releases, and
> restrict this type of change to minors?
>
>
Are we extending this to all interfaces? Do we support folks implementing
their own Connection? Admin?
> Although we make allowances for public interface changes that are binary
> compatible we also say "A minor upgrade requires no application/client code
> modification." which could imply source compatibility even across minors,
> which I believe is not the intent.
>
> We could add a source compatibility dimension for patch releases.
>
>
>
> I would like to see us get to source-compatibility on patch releases, not
> just binary compatibility
>
> You mean source compatibility for Public annotated interfaces only, right?
>
> In that case evaluating compliance would be easy: RMs would run the API
> compat checker on a RC and if a patch release the number of binary and
> source compat issues should both be zero.
>
>
So when I read the existing guides, what I see is that we're supposed to be
_source_ compatible on minor and patch releases, but binary compatible only
on patch releases. I think we discussed this a year or two ago, and IIRC
the reasoning was that since we maintain wire compatibility on minor and
patch releases, those who need to ignore a given set of binary incompatible
changes can just make use of the older library.
Re: Source incompatibility between 1.2.[0,1] and 1.2.2 breaks Phoenix
Posted by Andrew Purtell <ap...@apache.org>.
Also, the change to Table (HBASE-15645) went out in both 1.1.5 and 1.2.2.
On Mon, Aug 15, 2016 at 12:21 PM, Andrew Purtell <ap...@apache.org>
wrote:
> >
>
> I find the InterfaceAudience annotations on this really strange. How can
> we have a Public audience Interface (o.a.h.h.c.Table) with Private methods?
>
>
> I'm also not sure the Private annotations on the Table interface are that
> useful. Any change to an interface renders it source incompatible, and
> removal (or effective removal via signature change) of methods from an
> interface introduces a binary incompatibility. I think the Private
> annotations on methods of a Public interface imply we should refactor those
> methods to a non-public interface.
>
> >
>
> Now that we've had quite a few releases in the "not-quite-SemVer"
> compatibility guide, is there any interest in trying to make the
> compatibility guarantees more stringent?
>
> I was looking at our compat guidelines (http://hbase.apache.org/book.
> html#hbase.versioning) and think we could make a few refinements.
>
> We make several allowances for public interface changes that are binary
> compatible but not source compatible in patch releases. I think we are only
> taking into consideration callers but should also consider implementors of
> public interfaces. Changing a public interface on a patch release renders
> it source incompatible. Can we avoid doing that on *patch* releases, and
> restrict this type of change to minors?
>
> Although we make allowances for public interface changes that are binary
> compatible we also say "A minor upgrade requires no application/client code
> modification." which could imply source compatibility even across minors,
> which I believe is not the intent.
>
> We could add a source compatibility dimension for patch releases.
>
> >
>
> I would like to see us get to source-compatibility on patch releases, not
> just binary compatibility
>
> You mean source compatibility for Public annotated interfaces only, right?
>
> In that case evaluating compliance would be easy: RMs would run the API
> compat checker on a RC and if a patch release the number of binary and
> source compat issues should both be zero.
>
>
> On Mon, Aug 15, 2016 at 12:07 PM, Josh Elser <jo...@gmail.com> wrote:
>
>> Hi folks,
>>
>> I just noticed a ticket over in Phoenix [1] in which some method
>> additions to the Table interface [2] breaks the source compatibility of
>> Phoenix with HBase 1.2.2.
>>
>> My understanding of the current guidelines is that these are allowed as
>> they do not invalidate binary compatibility of clients using the API.
>> Personally, I am very hard-pressed to use the word "compatible" in a
>> sentence describing this change that isn't sarcastic :)
>>
>> A couple of questions:
>>
>> 1)
>>
>> I find the InterfaceAudience annotations on this really strange. How can
>> we have a Public audience Interface (o.a.h.h.c.Table) with Private methods?
>> Is that just "how things are", or am I missing something? If this is not
>> something that's meant to be public, I would think these new methods should
>> be defined in a non-public interface.
>>
>> 2)
>>
>> Now that we've had quite a few releases in the "not-quite-SemVer"
>> compatibility guide, is there any interest in trying to make the
>> compatibility guarantees more stringent?
>>
>> I would like to see us get to source-compatibility on patch releases, not
>> just binary compatibility. I am happy to try to help, but I know I don't
>> have the time to devote to catch everything.
>>
>> 3) What do people think about changing this in a 1.2.3 with a quick
>> turn-around?
>>
>> Thanks!
>>
>> - Josh
>>
>> [1] https://issues.apache.org/jira/browse/PHOENIX-3116
>> [2] https://issues.apache.org/jira/browse/HBASE-15645
>>
>
>
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: Source incompatibility between 1.2.[0,1] and 1.2.2 breaks Phoenix
Posted by Andrew Purtell <ap...@apache.org>.
>
I find the InterfaceAudience annotations on this really strange. How can
we have a Public audience Interface (o.a.h.h.c.Table) with Private methods?
I'm also not sure the Private annotations on the Table interface are that
useful. Any change to an interface renders it source incompatible, and
removal (or effective removal via signature change) of methods from an
interface introduces a binary incompatibility. I think the Private
annotations on methods of a Public interface imply we should refactor those
methods to a non-public interface.
>
Now that we've had quite a few releases in the "not-quite-SemVer"
compatibility guide, is there any interest in trying to make the
compatibility guarantees more stringent?
I was looking at our compat guidelines (
http://hbase.apache.org/book.html#hbase.versioning) and think we could make
a few refinements.
We make several allowances for public interface changes that are binary
compatible but not source compatible in patch releases. I think we are only
taking into consideration callers but should also consider implementors of
public interfaces. Changing a public interface on a patch release renders
it source incompatible. Can we avoid doing that on *patch* releases, and
restrict this type of change to minors?
Although we make allowances for public interface changes that are binary
compatible we also say "A minor upgrade requires no application/client code
modification." which could imply source compatibility even across minors,
which I believe is not the intent.
We could add a source compatibility dimension for patch releases.
>
I would like to see us get to source-compatibility on patch releases, not
just binary compatibility
You mean source compatibility for Public annotated interfaces only, right?
In that case evaluating compliance would be easy: RMs would run the API
compat checker on a RC and if a patch release the number of binary and
source compat issues should both be zero.
On Mon, Aug 15, 2016 at 12:07 PM, Josh Elser <jo...@gmail.com> wrote:
> Hi folks,
>
> I just noticed a ticket over in Phoenix [1] in which some method additions
> to the Table interface [2] breaks the source compatibility of Phoenix with
> HBase 1.2.2.
>
> My understanding of the current guidelines is that these are allowed as
> they do not invalidate binary compatibility of clients using the API.
> Personally, I am very hard-pressed to use the word "compatible" in a
> sentence describing this change that isn't sarcastic :)
>
> A couple of questions:
>
> 1)
>
> I find the InterfaceAudience annotations on this really strange. How can
> we have a Public audience Interface (o.a.h.h.c.Table) with Private methods?
> Is that just "how things are", or am I missing something? If this is not
> something that's meant to be public, I would think these new methods should
> be defined in a non-public interface.
>
> 2)
>
> Now that we've had quite a few releases in the "not-quite-SemVer"
> compatibility guide, is there any interest in trying to make the
> compatibility guarantees more stringent?
>
> I would like to see us get to source-compatibility on patch releases, not
> just binary compatibility. I am happy to try to help, but I know I don't
> have the time to devote to catch everything.
>
> 3) What do people think about changing this in a 1.2.3 with a quick
> turn-around?
>
> Thanks!
>
> - Josh
>
> [1] https://issues.apache.org/jira/browse/PHOENIX-3116
> [2] https://issues.apache.org/jira/browse/HBASE-15645
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: Source incompatibility between 1.2.[0,1] and 1.2.2 breaks Phoenix
Posted by Sean Busbey <bu...@apache.org>.
Sorry I missed this other thing I wanted to comment on:
On Mon, Aug 15, 2016 at 2:07 PM, Josh Elser <jo...@gmail.com> wrote:
>
>
>
> 3) What do people think about changing this in a 1.2.3 with a quick
> turn-around?
>
>
I'm actually behind cadence for 1.2.3 by about a week, so I'm fine with
getting that release out whenever we have a consensus solution to this
issue.