Re: HBase - Semantic Versioning

[Nick Dimiduk <nd...@gmail.com>]

Where does the HBase Shell implementation fall in this matrix? Asking for a
friend [0] :)

[0]: https://issues.apache.org/jira/browse/HBASE-12833

On Fri, Dec 5, 2014 at 4:36 PM, Nick Dimiduk <nd...@gmail.com> wrote:

> Right. So either two sections or simply remove the specificity of
> "Server-Side".
>
> On Fri, Dec 5, 2014 at 4:19 PM, lars hofhansl <la...@apache.org> wrote:
>
>> I'd think so. So maybe the section for client API compatibility should
>> break down into the same subclasses.
>>       From: Nick Dimiduk <nd...@gmail.com>
>>  To: hbase-dev <de...@hbase.apache.org>; lars hofhansl <la...@apache.org>
>> Cc: Enis Söztutar <en...@gmail.com>; Sean Busbey <bu...@cloudera.com>
>>  Sent: Friday, December 5, 2014 2:32 PM
>>  Subject: Re: HBase - Semantic Versioning
>>
>> We have a section on Server-Side Limited API Compatibility but what about
>> similar limited compatibility on client side? We use those same annotation
>> {stable,evolving,unstable} in client-side classes. Same rules apply as
>> server-side?
>>
>> I got to thinking about this again because I recently came back to
>> thinking
>> about changes/improvements to the DataType API. It was added as Evolving,
>> but I'm thinking it should have been Unstable from the onset to give it
>> time to bake.
>>
>>
>>
>> On Fri, Dec 5, 2014 at 2:02 PM, lars hofhansl <la...@apache.org> wrote:
>>
>> > I think we've given this enough time, let's add it.Note that nothing is
>> > set in stone, and the semantic versioning doc is itself semantically
>> > versioned. :)We can always change/augment/clarify later.
>> > -- Lars
>> >
>> >      From: Enis Söztutar <en...@gmail.com>
>> >  To: "dev@hbase.apache.org" <de...@hbase.apache.org>; lars hofhansl <
>> > larsh@apache.org>
>> > Cc: Sean Busbey <bu...@cloudera.com>
>> >  Sent: Monday, November 24, 2014 5:00 PM
>> >  Subject: Re: HBase - Semantic Versioning
>> >
>> > I've created https://issues.apache.org/jira/browse/HBASE-12568 for
>> > putting this to the book, and making it official.
>> > Enis
>> >
>> >
>> > On Fri, Nov 21, 2014 at 11:15 PM, lars hofhansl <la...@apache.org>
>> wrote:
>> >
>> > Hey Sean,
>> > thanks for taking a look. All good points.Send me your gmail id offlist,
>> > I'll add you as editor (maybe in suggest mode). Might be easier that
>> way.
>> > If possible I would like to keep it to two pages total (otherwise nobody
>> > will ever read it :) )(I tried one page, but that was not possible)
>> >
>> > -- Lars
>> >      From: Sean Busbey <bu...@cloudera.com>
>> >  To: dev <de...@hbase.apache.org>
>> > Cc: lars hofhansl <la...@apache.org>
>> >  Sent: Friday, November 21, 2014 5:19 PM
>> >  Subject: Re: HBase - Semantic Versioning
>> >
>> > First, this is excellent work and will help greatly with planning around
>> > HBase deployments. I think the document is big enough, of wide enough
>> > interest, and important enough that we should keep it as a page in the
>> site
>> > outside of the ref guide. Maybe it could live near wherever we currently
>> > keep our bylaws?
>> > A very brief restatement of how major, minor, and patch map to version
>> > strings X.Y.Z would help make the compatibility matrix easier to follow
>> for
>> > those not familiar with semantic versioning. Some may go read the full
>> > reference, but most probably won't. Maybe "Summary" could be retitled
>> > "Version Changes" and the bullet points could be preceded with a couple
>> of
>> > sentences.
>> > I notice the document doesn't make any reference to our use of
>> > InterfaceAudience[1]. Can we update it to do so?
>> > Specifically, what is covered in "Client API", is it just stuff that's
>> > IA.Public in hbase-client or IA.Public anywhere? If the former, then
>> what
>> > covers the rest?
>> > I think the section on "server side limited API compatibility" is for
>> > things marked IA.LimitedPrivate, but it would be nice to have it spelled
>> > out.
>> > The document covers InterfaceStability markings but only for server
>> side.
>> > Our current ref guide[1] covers them only for things marked public. Can
>> we
>> > unify this one way or the other? I suspect the answer is that we ought
>> to
>> > have them for both.
>> > [1]: http://hbase.apache.org/book.html#d0e21466
>> >
>> >
>> > On Fri, Nov 21, 2014 at 6:15 PM, Stack <st...@duboce.net> wrote:
>> >
>> > How should we progress here?  Unless objection by tomorrow -- a week --
>> > then lets just adopt this doc?  I can squash it into our dev section in
>> > refguide.
>> > St.Ack
>> >
>> > On Sat, Nov 15, 2014 at 5:06 PM, lars hofhansl <la...@apache.org>
>> wrote:
>> >
>> > > As we approach the released of HBase 1.0.0, your PMC has been working
>> > hard
>> > > on documenting the exact compatibility guarantees we plan to support
>> for
>> > > HBase versions going forward.
>> > > You can find the current version of the document here:
>> > >
>> > >
>> >
>> https://docs.google.com/document/d/1p5pP7v2OuzSSaomK2S2v7sfKky1Hex6OqwsJO0sZTUY/edit?usp=sharing
>> > >
>> > > For convenience I am also including the entire text at the end of this
>> > > email (hopefully the formatting comes through).
>> > > Please have a look and let us know what you think.While we cannot
>> > possibly
>> > > include provisions for every corner case, this should be as
>> comprehensive
>> > > as possible.Note that the semantic versioning document is itself
>> > > semantically versioned :)
>> > > Thanks.
>> > > -- Lars
>> > > --------------------------------------
>> > >
>> > > HBase Semantic Versioning - 1.0.4As HBase approaches its 1.0.0
>> version we
>> > > should start using semantic versioning.In addition to the usual API
>> > > versioning considerations HBase has other compatibility dimensions
>> that
>> > we
>> > > need to consider.
>> > > Compatibility Dimensions
>> > >
>> > >    - Client-Server wire protocol compatibility
>> > >
>> > >    - Allows updating client and server out of sync.
>> > >    - We could only allow upgrading the server first. I.e. the server
>> > would
>> > > be backward compatible to an old client, that way new APIs are OK.
>> > >    - Example: A user should be able to use an old client to connect
>> to an
>> > > upgraded cluster.
>> > >
>> > >    - Server-Server protocol compatibility
>> > >
>> > >    - Servers of different versions can co-exist in the same cluster.
>> > >    - The wire protocol between servers is compatible.
>> > >    - Workers for distributed tasks, such as replication and log
>> > splitting,
>> > > can co-exist in the same cluster.
>> > >    - Dependent protocols (such as using ZK for coordination) will also
>> > not
>> > > be changed.
>> > >    - Example: A user can perform a rolling upgrade.
>> > >
>> > >    - File format compatibility
>> > >
>> > >    - Support file formats backward and forward compatible
>> > >    - Example: File, ZK encoding, directory layout is upgraded
>> > > automatically as part of an HBase upgrade. User can rollback to the
>> older
>> > > version and everything will continue to work.
>> > >
>> > >    - Client API compatibility
>> > >
>> > >    - Allow changing or removing existing client APIs.
>> > >    - An API needs to deprecated for a major version before we will
>> > > change/remove it.
>> > >    - Example: A user using a newly deprecated api does not need to
>> modify
>> > > application code with hbase api calls until the next major version.
>> > >
>> > >    - Client Binary compatibility
>> > >
>> > >    - Old client code can run unchanged (no recompilation needed)
>> against
>> > > new jars.
>> > >    - Example: Old compiled client code will work unchanged with the
>> new
>> > > jars.
>> > >
>> > >    - Server-Side Limited API compatibility (taken from Hadoop)
>> > >
>> > >    - Internal APIs are marked as Stable, Evolving, or Unstable
>> > >    - This implies binary compatibility for coprocessors and plugins
>> > > (pluggable classes, including replication) as long as these are only
>> > using
>> > > marked interfaces/classes.
>> > >    - Example: Old compiled Coprocessor, Filter, or Plugin code will
>> work
>> > > unchanged with the new jars.
>> > >
>> > >    - Dependency Compatibility
>> > >
>> > >    - An upgrade of HBase will not require an incompatible upgrade of a
>> > > dependent project, including the Java runtime.
>> > >    - Example: An upgrade of Hadoop will not invalidate any of the
>> > > compatibilities guarantees we made.
>> > >
>> > >    - Operational Compatibility
>> > >
>> > >    - Metric changes
>> > >    - Behavioral changes of services
>> > >    - Web page APIs
>> > >
>> > > Summary
>> > >
>> > >    - A patch upgrade is a drop-in replacement. Any change that is not
>> > Java
>> > > binary compatible would not be allowed.
>> > >    - A minor upgrade requires no application/client code modification.
>> > > Ideally it would be a drop-in replacement but client code,
>> coprocessors,
>> > > filters, etc might have to be recompiled if new jars are used.
>> > >    - A major upgrade allows the HBase community to make breaking
>> changes.
>> > >
>> > >
>> > > Compatibility Matrix
>> > >
>> > > |
>> > >  | Major | Minor | Patch |
>> > > | Client-Server wire Compatibility | N | Y | Y |
>> > > | Server-Server Compatibility | N | Y | Y |
>> > > | File Format Compatibility | N | Y | Y |
>> > > | Client API Compatibility | N | Y | Y |
>> > > | Client Binary Compatibility | N | N | Y |
>> > > | Server-Side Limited API Compatibility |
>> > >  |
>> > >  |
>> > >  |
>> > > |
>> > >    - Stable
>> > >  | N | Y | Y |
>> > > |
>> > >    - Evolving
>> > >  | N | N | Y |
>> > > |
>> > >    - Unstable
>> > >  | N | N | N |
>> > > | Dependency Compatibility | N | Y | Y |
>> > > | Operational Compatibility | N | N | Y |
>> > >
>> > > (Y means we support the compatibility. N means we can break it.)
>> > >
>> > >
>> >
>> >
>> >
>> >
>> > --
>> > Sean
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>>
>>
>>
>
>