You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by lars hofhansl <la...@apache.org> on 2014/11/16 02:06:26 UTC
HBase - Semantic Versioning
As we approach the released of HBase 1.0.0, your PMC has been working hard on documenting the exact compatibility guarantees we plan to support for HBase versions going forward.
You can find the current version of the document here:
https://docs.google.com/document/d/1p5pP7v2OuzSSaomK2S2v7sfKky1Hex6OqwsJO0sZTUY/edit?usp=sharing
For convenience I am also including the entire text at the end of this email (hopefully the formatting comes through).
Please have a look and let us know what you think.While we cannot possibly include provisions for every corner case, this should be as comprehensive as possible.Note that the semantic versioning document is itself semantically versioned :)
Thanks.
-- Lars
--------------------------------------
HBase Semantic Versioning - 1.0.4As HBase approaches its 1.0.0 version we should start using semantic versioning.In addition to the usual API versioning considerations HBase has other compatibility dimensions that we need to consider.
Compatibility Dimensions
- Client-Server wire protocol compatibility
- Allows updating client and server out of sync.
- We could only allow upgrading the server first. I.e. the server would be backward compatible to an old client, that way new APIs are OK.
- Example: A user should be able to use an old client to connect to an upgraded cluster.
- Server-Server protocol compatibility
- Servers of different versions can co-exist in the same cluster.
- The wire protocol between servers is compatible.
- Workers for distributed tasks, such as replication and log splitting, can co-exist in the same cluster.
- Dependent protocols (such as using ZK for coordination) will also not be changed.
- Example: A user can perform a rolling upgrade.
- File format compatibility
- Support file formats backward and forward compatible
- Example: File, ZK encoding, directory layout is upgraded automatically as part of an HBase upgrade. User can rollback to the older version and everything will continue to work.
- Client API compatibility
- Allow changing or removing existing client APIs.
- An API needs to deprecated for a major version before we will change/remove it.
- Example: A user using a newly deprecated api does not need to modify application code with hbase api calls until the next major version.
- Client Binary compatibility
- Old client code can run unchanged (no recompilation needed) against new jars.
- Example: Old compiled client code will work unchanged with the new jars.
- Server-Side Limited API compatibility (taken from Hadoop)
- Internal APIs are marked as Stable, Evolving, or Unstable
- This implies binary compatibility for coprocessors and plugins (pluggable classes, including replication) as long as these are only using marked interfaces/classes.
- Example: Old compiled Coprocessor, Filter, or Plugin code will work unchanged with the new jars.
- Dependency Compatibility
- An upgrade of HBase will not require an incompatible upgrade of a dependent project, including the Java runtime.
- Example: An upgrade of Hadoop will not invalidate any of the compatibilities guarantees we made.
- Operational Compatibility
- Metric changes
- Behavioral changes of services
- Web page APIs
Summary
- A patch upgrade is a drop-in replacement. Any change that is not Java binary compatible would not be allowed.
- A minor upgrade requires no application/client code modification. Ideally it would be a drop-in replacement but client code, coprocessors, filters, etc might have to be recompiled if new jars are used.
- A major upgrade allows the HBase community to make breaking changes.
Compatibility Matrix
|
| Major | Minor | Patch |
| Client-Server wire Compatibility | N | Y | Y |
| Server-Server Compatibility | N | Y | Y |
| File Format Compatibility | N | Y | Y |
| Client API Compatibility | N | Y | Y |
| Client Binary Compatibility | N | N | Y |
| Server-Side Limited API Compatibility |
|
|
|
|
- Stable
| N | Y | Y |
|
- Evolving
| N | N | Y |
|
- Unstable
| N | N | N |
| Dependency Compatibility | N | Y | Y |
| Operational Compatibility | N | N | Y |
(Y means we support the compatibility. N means we can break it.)
Re: HBase - Semantic Versioning
Posted by Nick Dimiduk <nd...@gmail.com>.
Where does the HBase Shell implementation fall in this matrix? Asking for a
friend [0] :)
[0]: https://issues.apache.org/jira/browse/HBASE-12833
On Fri, Dec 5, 2014 at 4:36 PM, Nick Dimiduk <nd...@gmail.com> wrote:
> Right. So either two sections or simply remove the specificity of
> "Server-Side".
>
> On Fri, Dec 5, 2014 at 4:19 PM, lars hofhansl <la...@apache.org> wrote:
>
>> I'd think so. So maybe the section for client API compatibility should
>> break down into the same subclasses.
>> From: Nick Dimiduk <nd...@gmail.com>
>> To: hbase-dev <de...@hbase.apache.org>; lars hofhansl <la...@apache.org>
>> Cc: Enis Söztutar <en...@gmail.com>; Sean Busbey <bu...@cloudera.com>
>> Sent: Friday, December 5, 2014 2:32 PM
>> Subject: Re: HBase - Semantic Versioning
>>
>> We have a section on Server-Side Limited API Compatibility but what about
>> similar limited compatibility on client side? We use those same annotation
>> {stable,evolving,unstable} in client-side classes. Same rules apply as
>> server-side?
>>
>> I got to thinking about this again because I recently came back to
>> thinking
>> about changes/improvements to the DataType API. It was added as Evolving,
>> but I'm thinking it should have been Unstable from the onset to give it
>> time to bake.
>>
>>
>>
>> On Fri, Dec 5, 2014 at 2:02 PM, lars hofhansl <la...@apache.org> wrote:
>>
>> > I think we've given this enough time, let's add it.Note that nothing is
>> > set in stone, and the semantic versioning doc is itself semantically
>> > versioned. :)We can always change/augment/clarify later.
>> > -- Lars
>> >
>> > From: Enis Söztutar <en...@gmail.com>
>> > To: "dev@hbase.apache.org" <de...@hbase.apache.org>; lars hofhansl <
>> > larsh@apache.org>
>> > Cc: Sean Busbey <bu...@cloudera.com>
>> > Sent: Monday, November 24, 2014 5:00 PM
>> > Subject: Re: HBase - Semantic Versioning
>> >
>> > I've created https://issues.apache.org/jira/browse/HBASE-12568 for
>> > putting this to the book, and making it official.
>> > Enis
>> >
>> >
>> > On Fri, Nov 21, 2014 at 11:15 PM, lars hofhansl <la...@apache.org>
>> wrote:
>> >
>> > Hey Sean,
>> > thanks for taking a look. All good points.Send me your gmail id offlist,
>> > I'll add you as editor (maybe in suggest mode). Might be easier that
>> way.
>> > If possible I would like to keep it to two pages total (otherwise nobody
>> > will ever read it :) )(I tried one page, but that was not possible)
>> >
>> > -- Lars
>> > From: Sean Busbey <bu...@cloudera.com>
>> > To: dev <de...@hbase.apache.org>
>> > Cc: lars hofhansl <la...@apache.org>
>> > Sent: Friday, November 21, 2014 5:19 PM
>> > Subject: Re: HBase - Semantic Versioning
>> >
>> > First, this is excellent work and will help greatly with planning around
>> > HBase deployments. I think the document is big enough, of wide enough
>> > interest, and important enough that we should keep it as a page in the
>> site
>> > outside of the ref guide. Maybe it could live near wherever we currently
>> > keep our bylaws?
>> > A very brief restatement of how major, minor, and patch map to version
>> > strings X.Y.Z would help make the compatibility matrix easier to follow
>> for
>> > those not familiar with semantic versioning. Some may go read the full
>> > reference, but most probably won't. Maybe "Summary" could be retitled
>> > "Version Changes" and the bullet points could be preceded with a couple
>> of
>> > sentences.
>> > I notice the document doesn't make any reference to our use of
>> > InterfaceAudience[1]. Can we update it to do so?
>> > Specifically, what is covered in "Client API", is it just stuff that's
>> > IA.Public in hbase-client or IA.Public anywhere? If the former, then
>> what
>> > covers the rest?
>> > I think the section on "server side limited API compatibility" is for
>> > things marked IA.LimitedPrivate, but it would be nice to have it spelled
>> > out.
>> > The document covers InterfaceStability markings but only for server
>> side.
>> > Our current ref guide[1] covers them only for things marked public. Can
>> we
>> > unify this one way or the other? I suspect the answer is that we ought
>> to
>> > have them for both.
>> > [1]: http://hbase.apache.org/book.html#d0e21466
>> >
>> >
>> > On Fri, Nov 21, 2014 at 6:15 PM, Stack <st...@duboce.net> wrote:
>> >
>> > How should we progress here? Unless objection by tomorrow -- a week --
>> > then lets just adopt this doc? I can squash it into our dev section in
>> > refguide.
>> > St.Ack
>> >
>> > On Sat, Nov 15, 2014 at 5:06 PM, lars hofhansl <la...@apache.org>
>> wrote:
>> >
>> > > As we approach the released of HBase 1.0.0, your PMC has been working
>> > hard
>> > > on documenting the exact compatibility guarantees we plan to support
>> for
>> > > HBase versions going forward.
>> > > You can find the current version of the document here:
>> > >
>> > >
>> >
>> https://docs.google.com/document/d/1p5pP7v2OuzSSaomK2S2v7sfKky1Hex6OqwsJO0sZTUY/edit?usp=sharing
>> > >
>> > > For convenience I am also including the entire text at the end of this
>> > > email (hopefully the formatting comes through).
>> > > Please have a look and let us know what you think.While we cannot
>> > possibly
>> > > include provisions for every corner case, this should be as
>> comprehensive
>> > > as possible.Note that the semantic versioning document is itself
>> > > semantically versioned :)
>> > > Thanks.
>> > > -- Lars
>> > > --------------------------------------
>> > >
>> > > HBase Semantic Versioning - 1.0.4As HBase approaches its 1.0.0
>> version we
>> > > should start using semantic versioning.In addition to the usual API
>> > > versioning considerations HBase has other compatibility dimensions
>> that
>> > we
>> > > need to consider.
>> > > Compatibility Dimensions
>> > >
>> > > - Client-Server wire protocol compatibility
>> > >
>> > > - Allows updating client and server out of sync.
>> > > - We could only allow upgrading the server first. I.e. the server
>> > would
>> > > be backward compatible to an old client, that way new APIs are OK.
>> > > - Example: A user should be able to use an old client to connect
>> to an
>> > > upgraded cluster.
>> > >
>> > > - Server-Server protocol compatibility
>> > >
>> > > - Servers of different versions can co-exist in the same cluster.
>> > > - The wire protocol between servers is compatible.
>> > > - Workers for distributed tasks, such as replication and log
>> > splitting,
>> > > can co-exist in the same cluster.
>> > > - Dependent protocols (such as using ZK for coordination) will also
>> > not
>> > > be changed.
>> > > - Example: A user can perform a rolling upgrade.
>> > >
>> > > - File format compatibility
>> > >
>> > > - Support file formats backward and forward compatible
>> > > - Example: File, ZK encoding, directory layout is upgraded
>> > > automatically as part of an HBase upgrade. User can rollback to the
>> older
>> > > version and everything will continue to work.
>> > >
>> > > - Client API compatibility
>> > >
>> > > - Allow changing or removing existing client APIs.
>> > > - An API needs to deprecated for a major version before we will
>> > > change/remove it.
>> > > - Example: A user using a newly deprecated api does not need to
>> modify
>> > > application code with hbase api calls until the next major version.
>> > >
>> > > - Client Binary compatibility
>> > >
>> > > - Old client code can run unchanged (no recompilation needed)
>> against
>> > > new jars.
>> > > - Example: Old compiled client code will work unchanged with the
>> new
>> > > jars.
>> > >
>> > > - Server-Side Limited API compatibility (taken from Hadoop)
>> > >
>> > > - Internal APIs are marked as Stable, Evolving, or Unstable
>> > > - This implies binary compatibility for coprocessors and plugins
>> > > (pluggable classes, including replication) as long as these are only
>> > using
>> > > marked interfaces/classes.
>> > > - Example: Old compiled Coprocessor, Filter, or Plugin code will
>> work
>> > > unchanged with the new jars.
>> > >
>> > > - Dependency Compatibility
>> > >
>> > > - An upgrade of HBase will not require an incompatible upgrade of a
>> > > dependent project, including the Java runtime.
>> > > - Example: An upgrade of Hadoop will not invalidate any of the
>> > > compatibilities guarantees we made.
>> > >
>> > > - Operational Compatibility
>> > >
>> > > - Metric changes
>> > > - Behavioral changes of services
>> > > - Web page APIs
>> > >
>> > > Summary
>> > >
>> > > - A patch upgrade is a drop-in replacement. Any change that is not
>> > Java
>> > > binary compatible would not be allowed.
>> > > - A minor upgrade requires no application/client code modification.
>> > > Ideally it would be a drop-in replacement but client code,
>> coprocessors,
>> > > filters, etc might have to be recompiled if new jars are used.
>> > > - A major upgrade allows the HBase community to make breaking
>> changes.
>> > >
>> > >
>> > > Compatibility Matrix
>> > >
>> > > |
>> > > | Major | Minor | Patch |
>> > > | Client-Server wire Compatibility | N | Y | Y |
>> > > | Server-Server Compatibility | N | Y | Y |
>> > > | File Format Compatibility | N | Y | Y |
>> > > | Client API Compatibility | N | Y | Y |
>> > > | Client Binary Compatibility | N | N | Y |
>> > > | Server-Side Limited API Compatibility |
>> > > |
>> > > |
>> > > |
>> > > |
>> > > - Stable
>> > > | N | Y | Y |
>> > > |
>> > > - Evolving
>> > > | N | N | Y |
>> > > |
>> > > - Unstable
>> > > | N | N | N |
>> > > | Dependency Compatibility | N | Y | Y |
>> > > | Operational Compatibility | N | N | Y |
>> > >
>> > > (Y means we support the compatibility. N means we can break it.)
>> > >
>> > >
>> >
>> >
>> >
>> >
>> > --
>> > Sean
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>>
>>
>>
>
>
Re: HBase - Semantic Versioning
Posted by Nick Dimiduk <nd...@gmail.com>.
Right. So either two sections or simply remove the specificity of
"Server-Side".
On Fri, Dec 5, 2014 at 4:19 PM, lars hofhansl <la...@apache.org> wrote:
> I'd think so. So maybe the section for client API compatibility should
> break down into the same subclasses.
> From: Nick Dimiduk <nd...@gmail.com>
> To: hbase-dev <de...@hbase.apache.org>; lars hofhansl <la...@apache.org>
> Cc: Enis Söztutar <en...@gmail.com>; Sean Busbey <bu...@cloudera.com>
> Sent: Friday, December 5, 2014 2:32 PM
> Subject: Re: HBase - Semantic Versioning
>
> We have a section on Server-Side Limited API Compatibility but what about
> similar limited compatibility on client side? We use those same annotation
> {stable,evolving,unstable} in client-side classes. Same rules apply as
> server-side?
>
> I got to thinking about this again because I recently came back to thinking
> about changes/improvements to the DataType API. It was added as Evolving,
> but I'm thinking it should have been Unstable from the onset to give it
> time to bake.
>
>
>
> On Fri, Dec 5, 2014 at 2:02 PM, lars hofhansl <la...@apache.org> wrote:
>
> > I think we've given this enough time, let's add it.Note that nothing is
> > set in stone, and the semantic versioning doc is itself semantically
> > versioned. :)We can always change/augment/clarify later.
> > -- Lars
> >
> > From: Enis Söztutar <en...@gmail.com>
> > To: "dev@hbase.apache.org" <de...@hbase.apache.org>; lars hofhansl <
> > larsh@apache.org>
> > Cc: Sean Busbey <bu...@cloudera.com>
> > Sent: Monday, November 24, 2014 5:00 PM
> > Subject: Re: HBase - Semantic Versioning
> >
> > I've created https://issues.apache.org/jira/browse/HBASE-12568 for
> > putting this to the book, and making it official.
> > Enis
> >
> >
> > On Fri, Nov 21, 2014 at 11:15 PM, lars hofhansl <la...@apache.org>
> wrote:
> >
> > Hey Sean,
> > thanks for taking a look. All good points.Send me your gmail id offlist,
> > I'll add you as editor (maybe in suggest mode). Might be easier that way.
> > If possible I would like to keep it to two pages total (otherwise nobody
> > will ever read it :) )(I tried one page, but that was not possible)
> >
> > -- Lars
> > From: Sean Busbey <bu...@cloudera.com>
> > To: dev <de...@hbase.apache.org>
> > Cc: lars hofhansl <la...@apache.org>
> > Sent: Friday, November 21, 2014 5:19 PM
> > Subject: Re: HBase - Semantic Versioning
> >
> > First, this is excellent work and will help greatly with planning around
> > HBase deployments. I think the document is big enough, of wide enough
> > interest, and important enough that we should keep it as a page in the
> site
> > outside of the ref guide. Maybe it could live near wherever we currently
> > keep our bylaws?
> > A very brief restatement of how major, minor, and patch map to version
> > strings X.Y.Z would help make the compatibility matrix easier to follow
> for
> > those not familiar with semantic versioning. Some may go read the full
> > reference, but most probably won't. Maybe "Summary" could be retitled
> > "Version Changes" and the bullet points could be preceded with a couple
> of
> > sentences.
> > I notice the document doesn't make any reference to our use of
> > InterfaceAudience[1]. Can we update it to do so?
> > Specifically, what is covered in "Client API", is it just stuff that's
> > IA.Public in hbase-client or IA.Public anywhere? If the former, then what
> > covers the rest?
> > I think the section on "server side limited API compatibility" is for
> > things marked IA.LimitedPrivate, but it would be nice to have it spelled
> > out.
> > The document covers InterfaceStability markings but only for server side.
> > Our current ref guide[1] covers them only for things marked public. Can
> we
> > unify this one way or the other? I suspect the answer is that we ought to
> > have them for both.
> > [1]: http://hbase.apache.org/book.html#d0e21466
> >
> >
> > On Fri, Nov 21, 2014 at 6:15 PM, Stack <st...@duboce.net> wrote:
> >
> > How should we progress here? Unless objection by tomorrow -- a week --
> > then lets just adopt this doc? I can squash it into our dev section in
> > refguide.
> > St.Ack
> >
> > On Sat, Nov 15, 2014 at 5:06 PM, lars hofhansl <la...@apache.org> wrote:
> >
> > > As we approach the released of HBase 1.0.0, your PMC has been working
> > hard
> > > on documenting the exact compatibility guarantees we plan to support
> for
> > > HBase versions going forward.
> > > You can find the current version of the document here:
> > >
> > >
> >
> https://docs.google.com/document/d/1p5pP7v2OuzSSaomK2S2v7sfKky1Hex6OqwsJO0sZTUY/edit?usp=sharing
> > >
> > > For convenience I am also including the entire text at the end of this
> > > email (hopefully the formatting comes through).
> > > Please have a look and let us know what you think.While we cannot
> > possibly
> > > include provisions for every corner case, this should be as
> comprehensive
> > > as possible.Note that the semantic versioning document is itself
> > > semantically versioned :)
> > > Thanks.
> > > -- Lars
> > > --------------------------------------
> > >
> > > HBase Semantic Versioning - 1.0.4As HBase approaches its 1.0.0 version
> we
> > > should start using semantic versioning.In addition to the usual API
> > > versioning considerations HBase has other compatibility dimensions that
> > we
> > > need to consider.
> > > Compatibility Dimensions
> > >
> > > - Client-Server wire protocol compatibility
> > >
> > > - Allows updating client and server out of sync.
> > > - We could only allow upgrading the server first. I.e. the server
> > would
> > > be backward compatible to an old client, that way new APIs are OK.
> > > - Example: A user should be able to use an old client to connect to
> an
> > > upgraded cluster.
> > >
> > > - Server-Server protocol compatibility
> > >
> > > - Servers of different versions can co-exist in the same cluster.
> > > - The wire protocol between servers is compatible.
> > > - Workers for distributed tasks, such as replication and log
> > splitting,
> > > can co-exist in the same cluster.
> > > - Dependent protocols (such as using ZK for coordination) will also
> > not
> > > be changed.
> > > - Example: A user can perform a rolling upgrade.
> > >
> > > - File format compatibility
> > >
> > > - Support file formats backward and forward compatible
> > > - Example: File, ZK encoding, directory layout is upgraded
> > > automatically as part of an HBase upgrade. User can rollback to the
> older
> > > version and everything will continue to work.
> > >
> > > - Client API compatibility
> > >
> > > - Allow changing or removing existing client APIs.
> > > - An API needs to deprecated for a major version before we will
> > > change/remove it.
> > > - Example: A user using a newly deprecated api does not need to
> modify
> > > application code with hbase api calls until the next major version.
> > >
> > > - Client Binary compatibility
> > >
> > > - Old client code can run unchanged (no recompilation needed)
> against
> > > new jars.
> > > - Example: Old compiled client code will work unchanged with the new
> > > jars.
> > >
> > > - Server-Side Limited API compatibility (taken from Hadoop)
> > >
> > > - Internal APIs are marked as Stable, Evolving, or Unstable
> > > - This implies binary compatibility for coprocessors and plugins
> > > (pluggable classes, including replication) as long as these are only
> > using
> > > marked interfaces/classes.
> > > - Example: Old compiled Coprocessor, Filter, or Plugin code will
> work
> > > unchanged with the new jars.
> > >
> > > - Dependency Compatibility
> > >
> > > - An upgrade of HBase will not require an incompatible upgrade of a
> > > dependent project, including the Java runtime.
> > > - Example: An upgrade of Hadoop will not invalidate any of the
> > > compatibilities guarantees we made.
> > >
> > > - Operational Compatibility
> > >
> > > - Metric changes
> > > - Behavioral changes of services
> > > - Web page APIs
> > >
> > > Summary
> > >
> > > - A patch upgrade is a drop-in replacement. Any change that is not
> > Java
> > > binary compatible would not be allowed.
> > > - A minor upgrade requires no application/client code modification.
> > > Ideally it would be a drop-in replacement but client code,
> coprocessors,
> > > filters, etc might have to be recompiled if new jars are used.
> > > - A major upgrade allows the HBase community to make breaking
> changes.
> > >
> > >
> > > Compatibility Matrix
> > >
> > > |
> > > | Major | Minor | Patch |
> > > | Client-Server wire Compatibility | N | Y | Y |
> > > | Server-Server Compatibility | N | Y | Y |
> > > | File Format Compatibility | N | Y | Y |
> > > | Client API Compatibility | N | Y | Y |
> > > | Client Binary Compatibility | N | N | Y |
> > > | Server-Side Limited API Compatibility |
> > > |
> > > |
> > > |
> > > |
> > > - Stable
> > > | N | Y | Y |
> > > |
> > > - Evolving
> > > | N | N | Y |
> > > |
> > > - Unstable
> > > | N | N | N |
> > > | Dependency Compatibility | N | Y | Y |
> > > | Operational Compatibility | N | N | Y |
> > >
> > > (Y means we support the compatibility. N means we can break it.)
> > >
> > >
> >
> >
> >
> >
> > --
> > Sean
> >
> >
> >
> >
> >
> >
> >
>
>
>
Re: HBase - Semantic Versioning
Posted by lars hofhansl <la...@apache.org>.
I'd think so. So maybe the section for client API compatibility should break down into the same subclasses.
From: Nick Dimiduk <nd...@gmail.com>
To: hbase-dev <de...@hbase.apache.org>; lars hofhansl <la...@apache.org>
Cc: Enis Söztutar <en...@gmail.com>; Sean Busbey <bu...@cloudera.com>
Sent: Friday, December 5, 2014 2:32 PM
Subject: Re: HBase - Semantic Versioning
We have a section on Server-Side Limited API Compatibility but what about
similar limited compatibility on client side? We use those same annotation
{stable,evolving,unstable} in client-side classes. Same rules apply as
server-side?
I got to thinking about this again because I recently came back to thinking
about changes/improvements to the DataType API. It was added as Evolving,
but I'm thinking it should have been Unstable from the onset to give it
time to bake.
On Fri, Dec 5, 2014 at 2:02 PM, lars hofhansl <la...@apache.org> wrote:
> I think we've given this enough time, let's add it.Note that nothing is
> set in stone, and the semantic versioning doc is itself semantically
> versioned. :)We can always change/augment/clarify later.
> -- Lars
>
> From: Enis Söztutar <en...@gmail.com>
> To: "dev@hbase.apache.org" <de...@hbase.apache.org>; lars hofhansl <
> larsh@apache.org>
> Cc: Sean Busbey <bu...@cloudera.com>
> Sent: Monday, November 24, 2014 5:00 PM
> Subject: Re: HBase - Semantic Versioning
>
> I've created https://issues.apache.org/jira/browse/HBASE-12568 for
> putting this to the book, and making it official.
> Enis
>
>
> On Fri, Nov 21, 2014 at 11:15 PM, lars hofhansl <la...@apache.org> wrote:
>
> Hey Sean,
> thanks for taking a look. All good points.Send me your gmail id offlist,
> I'll add you as editor (maybe in suggest mode). Might be easier that way.
> If possible I would like to keep it to two pages total (otherwise nobody
> will ever read it :) )(I tried one page, but that was not possible)
>
> -- Lars
> From: Sean Busbey <bu...@cloudera.com>
> To: dev <de...@hbase.apache.org>
> Cc: lars hofhansl <la...@apache.org>
> Sent: Friday, November 21, 2014 5:19 PM
> Subject: Re: HBase - Semantic Versioning
>
> First, this is excellent work and will help greatly with planning around
> HBase deployments. I think the document is big enough, of wide enough
> interest, and important enough that we should keep it as a page in the site
> outside of the ref guide. Maybe it could live near wherever we currently
> keep our bylaws?
> A very brief restatement of how major, minor, and patch map to version
> strings X.Y.Z would help make the compatibility matrix easier to follow for
> those not familiar with semantic versioning. Some may go read the full
> reference, but most probably won't. Maybe "Summary" could be retitled
> "Version Changes" and the bullet points could be preceded with a couple of
> sentences.
> I notice the document doesn't make any reference to our use of
> InterfaceAudience[1]. Can we update it to do so?
> Specifically, what is covered in "Client API", is it just stuff that's
> IA.Public in hbase-client or IA.Public anywhere? If the former, then what
> covers the rest?
> I think the section on "server side limited API compatibility" is for
> things marked IA.LimitedPrivate, but it would be nice to have it spelled
> out.
> The document covers InterfaceStability markings but only for server side.
> Our current ref guide[1] covers them only for things marked public. Can we
> unify this one way or the other? I suspect the answer is that we ought to
> have them for both.
> [1]: http://hbase.apache.org/book.html#d0e21466
>
>
> On Fri, Nov 21, 2014 at 6:15 PM, Stack <st...@duboce.net> wrote:
>
> How should we progress here? Unless objection by tomorrow -- a week --
> then lets just adopt this doc? I can squash it into our dev section in
> refguide.
> St.Ack
>
> On Sat, Nov 15, 2014 at 5:06 PM, lars hofhansl <la...@apache.org> wrote:
>
> > As we approach the released of HBase 1.0.0, your PMC has been working
> hard
> > on documenting the exact compatibility guarantees we plan to support for
> > HBase versions going forward.
> > You can find the current version of the document here:
> >
> >
> https://docs.google.com/document/d/1p5pP7v2OuzSSaomK2S2v7sfKky1Hex6OqwsJO0sZTUY/edit?usp=sharing
> >
> > For convenience I am also including the entire text at the end of this
> > email (hopefully the formatting comes through).
> > Please have a look and let us know what you think.While we cannot
> possibly
> > include provisions for every corner case, this should be as comprehensive
> > as possible.Note that the semantic versioning document is itself
> > semantically versioned :)
> > Thanks.
> > -- Lars
> > --------------------------------------
> >
> > HBase Semantic Versioning - 1.0.4As HBase approaches its 1.0.0 version we
> > should start using semantic versioning.In addition to the usual API
> > versioning considerations HBase has other compatibility dimensions that
> we
> > need to consider.
> > Compatibility Dimensions
> >
> > - Client-Server wire protocol compatibility
> >
> > - Allows updating client and server out of sync.
> > - We could only allow upgrading the server first. I.e. the server
> would
> > be backward compatible to an old client, that way new APIs are OK.
> > - Example: A user should be able to use an old client to connect to an
> > upgraded cluster.
> >
> > - Server-Server protocol compatibility
> >
> > - Servers of different versions can co-exist in the same cluster.
> > - The wire protocol between servers is compatible.
> > - Workers for distributed tasks, such as replication and log
> splitting,
> > can co-exist in the same cluster.
> > - Dependent protocols (such as using ZK for coordination) will also
> not
> > be changed.
> > - Example: A user can perform a rolling upgrade.
> >
> > - File format compatibility
> >
> > - Support file formats backward and forward compatible
> > - Example: File, ZK encoding, directory layout is upgraded
> > automatically as part of an HBase upgrade. User can rollback to the older
> > version and everything will continue to work.
> >
> > - Client API compatibility
> >
> > - Allow changing or removing existing client APIs.
> > - An API needs to deprecated for a major version before we will
> > change/remove it.
> > - Example: A user using a newly deprecated api does not need to modify
> > application code with hbase api calls until the next major version.
> >
> > - Client Binary compatibility
> >
> > - Old client code can run unchanged (no recompilation needed) against
> > new jars.
> > - Example: Old compiled client code will work unchanged with the new
> > jars.
> >
> > - Server-Side Limited API compatibility (taken from Hadoop)
> >
> > - Internal APIs are marked as Stable, Evolving, or Unstable
> > - This implies binary compatibility for coprocessors and plugins
> > (pluggable classes, including replication) as long as these are only
> using
> > marked interfaces/classes.
> > - Example: Old compiled Coprocessor, Filter, or Plugin code will work
> > unchanged with the new jars.
> >
> > - Dependency Compatibility
> >
> > - An upgrade of HBase will not require an incompatible upgrade of a
> > dependent project, including the Java runtime.
> > - Example: An upgrade of Hadoop will not invalidate any of the
> > compatibilities guarantees we made.
> >
> > - Operational Compatibility
> >
> > - Metric changes
> > - Behavioral changes of services
> > - Web page APIs
> >
> > Summary
> >
> > - A patch upgrade is a drop-in replacement. Any change that is not
> Java
> > binary compatible would not be allowed.
> > - A minor upgrade requires no application/client code modification.
> > Ideally it would be a drop-in replacement but client code, coprocessors,
> > filters, etc might have to be recompiled if new jars are used.
> > - A major upgrade allows the HBase community to make breaking changes.
> >
> >
> > Compatibility Matrix
> >
> > |
> > | Major | Minor | Patch |
> > | Client-Server wire Compatibility | N | Y | Y |
> > | Server-Server Compatibility | N | Y | Y |
> > | File Format Compatibility | N | Y | Y |
> > | Client API Compatibility | N | Y | Y |
> > | Client Binary Compatibility | N | N | Y |
> > | Server-Side Limited API Compatibility |
> > |
> > |
> > |
> > |
> > - Stable
> > | N | Y | Y |
> > |
> > - Evolving
> > | N | N | Y |
> > |
> > - Unstable
> > | N | N | N |
> > | Dependency Compatibility | N | Y | Y |
> > | Operational Compatibility | N | N | Y |
> >
> > (Y means we support the compatibility. N means we can break it.)
> >
> >
>
>
>
>
> --
> Sean
>
>
>
>
>
>
>
Re: HBase - Semantic Versioning
Posted by Nick Dimiduk <nd...@gmail.com>.
We have a section on Server-Side Limited API Compatibility but what about
similar limited compatibility on client side? We use those same annotation
{stable,evolving,unstable} in client-side classes. Same rules apply as
server-side?
I got to thinking about this again because I recently came back to thinking
about changes/improvements to the DataType API. It was added as Evolving,
but I'm thinking it should have been Unstable from the onset to give it
time to bake.
On Fri, Dec 5, 2014 at 2:02 PM, lars hofhansl <la...@apache.org> wrote:
> I think we've given this enough time, let's add it.Note that nothing is
> set in stone, and the semantic versioning doc is itself semantically
> versioned. :)We can always change/augment/clarify later.
> -- Lars
>
> From: Enis Söztutar <en...@gmail.com>
> To: "dev@hbase.apache.org" <de...@hbase.apache.org>; lars hofhansl <
> larsh@apache.org>
> Cc: Sean Busbey <bu...@cloudera.com>
> Sent: Monday, November 24, 2014 5:00 PM
> Subject: Re: HBase - Semantic Versioning
>
> I've created https://issues.apache.org/jira/browse/HBASE-12568 for
> putting this to the book, and making it official.
> Enis
>
>
> On Fri, Nov 21, 2014 at 11:15 PM, lars hofhansl <la...@apache.org> wrote:
>
> Hey Sean,
> thanks for taking a look. All good points.Send me your gmail id offlist,
> I'll add you as editor (maybe in suggest mode). Might be easier that way.
> If possible I would like to keep it to two pages total (otherwise nobody
> will ever read it :) )(I tried one page, but that was not possible)
>
> -- Lars
> From: Sean Busbey <bu...@cloudera.com>
> To: dev <de...@hbase.apache.org>
> Cc: lars hofhansl <la...@apache.org>
> Sent: Friday, November 21, 2014 5:19 PM
> Subject: Re: HBase - Semantic Versioning
>
> First, this is excellent work and will help greatly with planning around
> HBase deployments. I think the document is big enough, of wide enough
> interest, and important enough that we should keep it as a page in the site
> outside of the ref guide. Maybe it could live near wherever we currently
> keep our bylaws?
> A very brief restatement of how major, minor, and patch map to version
> strings X.Y.Z would help make the compatibility matrix easier to follow for
> those not familiar with semantic versioning. Some may go read the full
> reference, but most probably won't. Maybe "Summary" could be retitled
> "Version Changes" and the bullet points could be preceded with a couple of
> sentences.
> I notice the document doesn't make any reference to our use of
> InterfaceAudience[1]. Can we update it to do so?
> Specifically, what is covered in "Client API", is it just stuff that's
> IA.Public in hbase-client or IA.Public anywhere? If the former, then what
> covers the rest?
> I think the section on "server side limited API compatibility" is for
> things marked IA.LimitedPrivate, but it would be nice to have it spelled
> out.
> The document covers InterfaceStability markings but only for server side.
> Our current ref guide[1] covers them only for things marked public. Can we
> unify this one way or the other? I suspect the answer is that we ought to
> have them for both.
> [1]: http://hbase.apache.org/book.html#d0e21466
>
>
> On Fri, Nov 21, 2014 at 6:15 PM, Stack <st...@duboce.net> wrote:
>
> How should we progress here? Unless objection by tomorrow -- a week --
> then lets just adopt this doc? I can squash it into our dev section in
> refguide.
> St.Ack
>
> On Sat, Nov 15, 2014 at 5:06 PM, lars hofhansl <la...@apache.org> wrote:
>
> > As we approach the released of HBase 1.0.0, your PMC has been working
> hard
> > on documenting the exact compatibility guarantees we plan to support for
> > HBase versions going forward.
> > You can find the current version of the document here:
> >
> >
> https://docs.google.com/document/d/1p5pP7v2OuzSSaomK2S2v7sfKky1Hex6OqwsJO0sZTUY/edit?usp=sharing
> >
> > For convenience I am also including the entire text at the end of this
> > email (hopefully the formatting comes through).
> > Please have a look and let us know what you think.While we cannot
> possibly
> > include provisions for every corner case, this should be as comprehensive
> > as possible.Note that the semantic versioning document is itself
> > semantically versioned :)
> > Thanks.
> > -- Lars
> > --------------------------------------
> >
> > HBase Semantic Versioning - 1.0.4As HBase approaches its 1.0.0 version we
> > should start using semantic versioning.In addition to the usual API
> > versioning considerations HBase has other compatibility dimensions that
> we
> > need to consider.
> > Compatibility Dimensions
> >
> > - Client-Server wire protocol compatibility
> >
> > - Allows updating client and server out of sync.
> > - We could only allow upgrading the server first. I.e. the server
> would
> > be backward compatible to an old client, that way new APIs are OK.
> > - Example: A user should be able to use an old client to connect to an
> > upgraded cluster.
> >
> > - Server-Server protocol compatibility
> >
> > - Servers of different versions can co-exist in the same cluster.
> > - The wire protocol between servers is compatible.
> > - Workers for distributed tasks, such as replication and log
> splitting,
> > can co-exist in the same cluster.
> > - Dependent protocols (such as using ZK for coordination) will also
> not
> > be changed.
> > - Example: A user can perform a rolling upgrade.
> >
> > - File format compatibility
> >
> > - Support file formats backward and forward compatible
> > - Example: File, ZK encoding, directory layout is upgraded
> > automatically as part of an HBase upgrade. User can rollback to the older
> > version and everything will continue to work.
> >
> > - Client API compatibility
> >
> > - Allow changing or removing existing client APIs.
> > - An API needs to deprecated for a major version before we will
> > change/remove it.
> > - Example: A user using a newly deprecated api does not need to modify
> > application code with hbase api calls until the next major version.
> >
> > - Client Binary compatibility
> >
> > - Old client code can run unchanged (no recompilation needed) against
> > new jars.
> > - Example: Old compiled client code will work unchanged with the new
> > jars.
> >
> > - Server-Side Limited API compatibility (taken from Hadoop)
> >
> > - Internal APIs are marked as Stable, Evolving, or Unstable
> > - This implies binary compatibility for coprocessors and plugins
> > (pluggable classes, including replication) as long as these are only
> using
> > marked interfaces/classes.
> > - Example: Old compiled Coprocessor, Filter, or Plugin code will work
> > unchanged with the new jars.
> >
> > - Dependency Compatibility
> >
> > - An upgrade of HBase will not require an incompatible upgrade of a
> > dependent project, including the Java runtime.
> > - Example: An upgrade of Hadoop will not invalidate any of the
> > compatibilities guarantees we made.
> >
> > - Operational Compatibility
> >
> > - Metric changes
> > - Behavioral changes of services
> > - Web page APIs
> >
> > Summary
> >
> > - A patch upgrade is a drop-in replacement. Any change that is not
> Java
> > binary compatible would not be allowed.
> > - A minor upgrade requires no application/client code modification.
> > Ideally it would be a drop-in replacement but client code, coprocessors,
> > filters, etc might have to be recompiled if new jars are used.
> > - A major upgrade allows the HBase community to make breaking changes.
> >
> >
> > Compatibility Matrix
> >
> > |
> > | Major | Minor | Patch |
> > | Client-Server wire Compatibility | N | Y | Y |
> > | Server-Server Compatibility | N | Y | Y |
> > | File Format Compatibility | N | Y | Y |
> > | Client API Compatibility | N | Y | Y |
> > | Client Binary Compatibility | N | N | Y |
> > | Server-Side Limited API Compatibility |
> > |
> > |
> > |
> > |
> > - Stable
> > | N | Y | Y |
> > |
> > - Evolving
> > | N | N | Y |
> > |
> > - Unstable
> > | N | N | N |
> > | Dependency Compatibility | N | Y | Y |
> > | Operational Compatibility | N | N | Y |
> >
> > (Y means we support the compatibility. N means we can break it.)
> >
> >
>
>
>
>
> --
> Sean
>
>
>
>
>
>
>
Re: HBase - Semantic Versioning
Posted by lars hofhansl <la...@apache.org>.
I think we've given this enough time, let's add it.Note that nothing is set in stone, and the semantic versioning doc is itself semantically versioned. :)We can always change/augment/clarify later.
-- Lars
From: Enis Söztutar <en...@gmail.com>
To: "dev@hbase.apache.org" <de...@hbase.apache.org>; lars hofhansl <la...@apache.org>
Cc: Sean Busbey <bu...@cloudera.com>
Sent: Monday, November 24, 2014 5:00 PM
Subject: Re: HBase - Semantic Versioning
I've created https://issues.apache.org/jira/browse/HBASE-12568 for putting this to the book, and making it official.
Enis
On Fri, Nov 21, 2014 at 11:15 PM, lars hofhansl <la...@apache.org> wrote:
Hey Sean,
thanks for taking a look. All good points.Send me your gmail id offlist, I'll add you as editor (maybe in suggest mode). Might be easier that way.
If possible I would like to keep it to two pages total (otherwise nobody will ever read it :) )(I tried one page, but that was not possible)
-- Lars
From: Sean Busbey <bu...@cloudera.com>
To: dev <de...@hbase.apache.org>
Cc: lars hofhansl <la...@apache.org>
Sent: Friday, November 21, 2014 5:19 PM
Subject: Re: HBase - Semantic Versioning
First, this is excellent work and will help greatly with planning around HBase deployments. I think the document is big enough, of wide enough interest, and important enough that we should keep it as a page in the site outside of the ref guide. Maybe it could live near wherever we currently keep our bylaws?
A very brief restatement of how major, minor, and patch map to version strings X.Y.Z would help make the compatibility matrix easier to follow for those not familiar with semantic versioning. Some may go read the full reference, but most probably won't. Maybe "Summary" could be retitled "Version Changes" and the bullet points could be preceded with a couple of sentences.
I notice the document doesn't make any reference to our use of InterfaceAudience[1]. Can we update it to do so?
Specifically, what is covered in "Client API", is it just stuff that's IA.Public in hbase-client or IA.Public anywhere? If the former, then what covers the rest?
I think the section on "server side limited API compatibility" is for things marked IA.LimitedPrivate, but it would be nice to have it spelled out.
The document covers InterfaceStability markings but only for server side. Our current ref guide[1] covers them only for things marked public. Can we unify this one way or the other? I suspect the answer is that we ought to have them for both.
[1]: http://hbase.apache.org/book.html#d0e21466
On Fri, Nov 21, 2014 at 6:15 PM, Stack <st...@duboce.net> wrote:
How should we progress here? Unless objection by tomorrow -- a week --
then lets just adopt this doc? I can squash it into our dev section in
refguide.
St.Ack
On Sat, Nov 15, 2014 at 5:06 PM, lars hofhansl <la...@apache.org> wrote:
> As we approach the released of HBase 1.0.0, your PMC has been working hard
> on documenting the exact compatibility guarantees we plan to support for
> HBase versions going forward.
> You can find the current version of the document here:
>
> https://docs.google.com/document/d/1p5pP7v2OuzSSaomK2S2v7sfKky1Hex6OqwsJO0sZTUY/edit?usp=sharing
>
> For convenience I am also including the entire text at the end of this
> email (hopefully the formatting comes through).
> Please have a look and let us know what you think.While we cannot possibly
> include provisions for every corner case, this should be as comprehensive
> as possible.Note that the semantic versioning document is itself
> semantically versioned :)
> Thanks.
> -- Lars
> --------------------------------------
>
> HBase Semantic Versioning - 1.0.4As HBase approaches its 1.0.0 version we
> should start using semantic versioning.In addition to the usual API
> versioning considerations HBase has other compatibility dimensions that we
> need to consider.
> Compatibility Dimensions
>
> - Client-Server wire protocol compatibility
>
> - Allows updating client and server out of sync.
> - We could only allow upgrading the server first. I.e. the server would
> be backward compatible to an old client, that way new APIs are OK.
> - Example: A user should be able to use an old client to connect to an
> upgraded cluster.
>
> - Server-Server protocol compatibility
>
> - Servers of different versions can co-exist in the same cluster.
> - The wire protocol between servers is compatible.
> - Workers for distributed tasks, such as replication and log splitting,
> can co-exist in the same cluster.
> - Dependent protocols (such as using ZK for coordination) will also not
> be changed.
> - Example: A user can perform a rolling upgrade.
>
> - File format compatibility
>
> - Support file formats backward and forward compatible
> - Example: File, ZK encoding, directory layout is upgraded
> automatically as part of an HBase upgrade. User can rollback to the older
> version and everything will continue to work.
>
> - Client API compatibility
>
> - Allow changing or removing existing client APIs.
> - An API needs to deprecated for a major version before we will
> change/remove it.
> - Example: A user using a newly deprecated api does not need to modify
> application code with hbase api calls until the next major version.
>
> - Client Binary compatibility
>
> - Old client code can run unchanged (no recompilation needed) against
> new jars.
> - Example: Old compiled client code will work unchanged with the new
> jars.
>
> - Server-Side Limited API compatibility (taken from Hadoop)
>
> - Internal APIs are marked as Stable, Evolving, or Unstable
> - This implies binary compatibility for coprocessors and plugins
> (pluggable classes, including replication) as long as these are only using
> marked interfaces/classes.
> - Example: Old compiled Coprocessor, Filter, or Plugin code will work
> unchanged with the new jars.
>
> - Dependency Compatibility
>
> - An upgrade of HBase will not require an incompatible upgrade of a
> dependent project, including the Java runtime.
> - Example: An upgrade of Hadoop will not invalidate any of the
> compatibilities guarantees we made.
>
> - Operational Compatibility
>
> - Metric changes
> - Behavioral changes of services
> - Web page APIs
>
> Summary
>
> - A patch upgrade is a drop-in replacement. Any change that is not Java
> binary compatible would not be allowed.
> - A minor upgrade requires no application/client code modification.
> Ideally it would be a drop-in replacement but client code, coprocessors,
> filters, etc might have to be recompiled if new jars are used.
> - A major upgrade allows the HBase community to make breaking changes.
>
>
> Compatibility Matrix
>
> |
> | Major | Minor | Patch |
> | Client-Server wire Compatibility | N | Y | Y |
> | Server-Server Compatibility | N | Y | Y |
> | File Format Compatibility | N | Y | Y |
> | Client API Compatibility | N | Y | Y |
> | Client Binary Compatibility | N | N | Y |
> | Server-Side Limited API Compatibility |
> |
> |
> |
> |
> - Stable
> | N | Y | Y |
> |
> - Evolving
> | N | N | Y |
> |
> - Unstable
> | N | N | N |
> | Dependency Compatibility | N | Y | Y |
> | Operational Compatibility | N | N | Y |
>
> (Y means we support the compatibility. N means we can break it.)
>
>
--
Sean
Re: HBase - Semantic Versioning
Posted by Enis Söztutar <en...@gmail.com>.
I've created
https://issues.apache.org/jira/browse/HBASE-12568 for putting this to the
book, and making it official.
Enis
On Fri, Nov 21, 2014 at 11:15 PM, lars hofhansl <la...@apache.org> wrote:
> Hey Sean,
> thanks for taking a look. All good points.Send me your gmail id offlist,
> I'll add you as editor (maybe in suggest mode). Might be easier that way.
> If possible I would like to keep it to two pages total (otherwise nobody
> will ever read it :) )(I tried one page, but that was not possible)
>
> -- Lars
> From: Sean Busbey <bu...@cloudera.com>
> To: dev <de...@hbase.apache.org>
> Cc: lars hofhansl <la...@apache.org>
> Sent: Friday, November 21, 2014 5:19 PM
> Subject: Re: HBase - Semantic Versioning
>
> First, this is excellent work and will help greatly with planning around
> HBase deployments. I think the document is big enough, of wide enough
> interest, and important enough that we should keep it as a page in the site
> outside of the ref guide. Maybe it could live near wherever we currently
> keep our bylaws?
> A very brief restatement of how major, minor, and patch map to version
> strings X.Y.Z would help make the compatibility matrix easier to follow for
> those not familiar with semantic versioning. Some may go read the full
> reference, but most probably won't. Maybe "Summary" could be retitled
> "Version Changes" and the bullet points could be preceded with a couple of
> sentences.
> I notice the document doesn't make any reference to our use of
> InterfaceAudience[1]. Can we update it to do so?
> Specifically, what is covered in "Client API", is it just stuff that's
> IA.Public in hbase-client or IA.Public anywhere? If the former, then what
> covers the rest?
> I think the section on "server side limited API compatibility" is for
> things marked IA.LimitedPrivate, but it would be nice to have it spelled
> out.
> The document covers InterfaceStability markings but only for server side.
> Our current ref guide[1] covers them only for things marked public. Can we
> unify this one way or the other? I suspect the answer is that we ought to
> have them for both.
> [1]: http://hbase.apache.org/book.html#d0e21466
>
>
> On Fri, Nov 21, 2014 at 6:15 PM, Stack <st...@duboce.net> wrote:
>
> How should we progress here? Unless objection by tomorrow -- a week --
> then lets just adopt this doc? I can squash it into our dev section in
> refguide.
> St.Ack
>
> On Sat, Nov 15, 2014 at 5:06 PM, lars hofhansl <la...@apache.org> wrote:
>
> > As we approach the released of HBase 1.0.0, your PMC has been working
> hard
> > on documenting the exact compatibility guarantees we plan to support for
> > HBase versions going forward.
> > You can find the current version of the document here:
> >
> >
> https://docs.google.com/document/d/1p5pP7v2OuzSSaomK2S2v7sfKky1Hex6OqwsJO0sZTUY/edit?usp=sharing
> >
> > For convenience I am also including the entire text at the end of this
> > email (hopefully the formatting comes through).
> > Please have a look and let us know what you think.While we cannot
> possibly
> > include provisions for every corner case, this should be as comprehensive
> > as possible.Note that the semantic versioning document is itself
> > semantically versioned :)
> > Thanks.
> > -- Lars
> > --------------------------------------
> >
> > HBase Semantic Versioning - 1.0.4As HBase approaches its 1.0.0 version we
> > should start using semantic versioning.In addition to the usual API
> > versioning considerations HBase has other compatibility dimensions that
> we
> > need to consider.
> > Compatibility Dimensions
> >
> > - Client-Server wire protocol compatibility
> >
> > - Allows updating client and server out of sync.
> > - We could only allow upgrading the server first. I.e. the server
> would
> > be backward compatible to an old client, that way new APIs are OK.
> > - Example: A user should be able to use an old client to connect to an
> > upgraded cluster.
> >
> > - Server-Server protocol compatibility
> >
> > - Servers of different versions can co-exist in the same cluster.
> > - The wire protocol between servers is compatible.
> > - Workers for distributed tasks, such as replication and log
> splitting,
> > can co-exist in the same cluster.
> > - Dependent protocols (such as using ZK for coordination) will also
> not
> > be changed.
> > - Example: A user can perform a rolling upgrade.
> >
> > - File format compatibility
> >
> > - Support file formats backward and forward compatible
> > - Example: File, ZK encoding, directory layout is upgraded
> > automatically as part of an HBase upgrade. User can rollback to the older
> > version and everything will continue to work.
> >
> > - Client API compatibility
> >
> > - Allow changing or removing existing client APIs.
> > - An API needs to deprecated for a major version before we will
> > change/remove it.
> > - Example: A user using a newly deprecated api does not need to modify
> > application code with hbase api calls until the next major version.
> >
> > - Client Binary compatibility
> >
> > - Old client code can run unchanged (no recompilation needed) against
> > new jars.
> > - Example: Old compiled client code will work unchanged with the new
> > jars.
> >
> > - Server-Side Limited API compatibility (taken from Hadoop)
> >
> > - Internal APIs are marked as Stable, Evolving, or Unstable
> > - This implies binary compatibility for coprocessors and plugins
> > (pluggable classes, including replication) as long as these are only
> using
> > marked interfaces/classes.
> > - Example: Old compiled Coprocessor, Filter, or Plugin code will work
> > unchanged with the new jars.
> >
> > - Dependency Compatibility
> >
> > - An upgrade of HBase will not require an incompatible upgrade of a
> > dependent project, including the Java runtime.
> > - Example: An upgrade of Hadoop will not invalidate any of the
> > compatibilities guarantees we made.
> >
> > - Operational Compatibility
> >
> > - Metric changes
> > - Behavioral changes of services
> > - Web page APIs
> >
> > Summary
> >
> > - A patch upgrade is a drop-in replacement. Any change that is not
> Java
> > binary compatible would not be allowed.
> > - A minor upgrade requires no application/client code modification.
> > Ideally it would be a drop-in replacement but client code, coprocessors,
> > filters, etc might have to be recompiled if new jars are used.
> > - A major upgrade allows the HBase community to make breaking changes.
> >
> >
> > Compatibility Matrix
> >
> > |
> > | Major | Minor | Patch |
> > | Client-Server wire Compatibility | N | Y | Y |
> > | Server-Server Compatibility | N | Y | Y |
> > | File Format Compatibility | N | Y | Y |
> > | Client API Compatibility | N | Y | Y |
> > | Client Binary Compatibility | N | N | Y |
> > | Server-Side Limited API Compatibility |
> > |
> > |
> > |
> > |
> > - Stable
> > | N | Y | Y |
> > |
> > - Evolving
> > | N | N | Y |
> > |
> > - Unstable
> > | N | N | N |
> > | Dependency Compatibility | N | Y | Y |
> > | Operational Compatibility | N | N | Y |
> >
> > (Y means we support the compatibility. N means we can break it.)
> >
> >
>
>
>
>
> --
> Sean
>
>
>
Re: HBase - Semantic Versioning
Posted by lars hofhansl <la...@apache.org>.
Hey Sean,
thanks for taking a look. All good points.Send me your gmail id offlist, I'll add you as editor (maybe in suggest mode). Might be easier that way.
If possible I would like to keep it to two pages total (otherwise nobody will ever read it :) )(I tried one page, but that was not possible)
-- Lars
From: Sean Busbey <bu...@cloudera.com>
To: dev <de...@hbase.apache.org>
Cc: lars hofhansl <la...@apache.org>
Sent: Friday, November 21, 2014 5:19 PM
Subject: Re: HBase - Semantic Versioning
First, this is excellent work and will help greatly with planning around HBase deployments. I think the document is big enough, of wide enough interest, and important enough that we should keep it as a page in the site outside of the ref guide. Maybe it could live near wherever we currently keep our bylaws?
A very brief restatement of how major, minor, and patch map to version strings X.Y.Z would help make the compatibility matrix easier to follow for those not familiar with semantic versioning. Some may go read the full reference, but most probably won't. Maybe "Summary" could be retitled "Version Changes" and the bullet points could be preceded with a couple of sentences.
I notice the document doesn't make any reference to our use of InterfaceAudience[1]. Can we update it to do so?
Specifically, what is covered in "Client API", is it just stuff that's IA.Public in hbase-client or IA.Public anywhere? If the former, then what covers the rest?
I think the section on "server side limited API compatibility" is for things marked IA.LimitedPrivate, but it would be nice to have it spelled out.
The document covers InterfaceStability markings but only for server side. Our current ref guide[1] covers them only for things marked public. Can we unify this one way or the other? I suspect the answer is that we ought to have them for both.
[1]: http://hbase.apache.org/book.html#d0e21466
On Fri, Nov 21, 2014 at 6:15 PM, Stack <st...@duboce.net> wrote:
How should we progress here? Unless objection by tomorrow -- a week --
then lets just adopt this doc? I can squash it into our dev section in
refguide.
St.Ack
On Sat, Nov 15, 2014 at 5:06 PM, lars hofhansl <la...@apache.org> wrote:
> As we approach the released of HBase 1.0.0, your PMC has been working hard
> on documenting the exact compatibility guarantees we plan to support for
> HBase versions going forward.
> You can find the current version of the document here:
>
> https://docs.google.com/document/d/1p5pP7v2OuzSSaomK2S2v7sfKky1Hex6OqwsJO0sZTUY/edit?usp=sharing
>
> For convenience I am also including the entire text at the end of this
> email (hopefully the formatting comes through).
> Please have a look and let us know what you think.While we cannot possibly
> include provisions for every corner case, this should be as comprehensive
> as possible.Note that the semantic versioning document is itself
> semantically versioned :)
> Thanks.
> -- Lars
> --------------------------------------
>
> HBase Semantic Versioning - 1.0.4As HBase approaches its 1.0.0 version we
> should start using semantic versioning.In addition to the usual API
> versioning considerations HBase has other compatibility dimensions that we
> need to consider.
> Compatibility Dimensions
>
> - Client-Server wire protocol compatibility
>
> - Allows updating client and server out of sync.
> - We could only allow upgrading the server first. I.e. the server would
> be backward compatible to an old client, that way new APIs are OK.
> - Example: A user should be able to use an old client to connect to an
> upgraded cluster.
>
> - Server-Server protocol compatibility
>
> - Servers of different versions can co-exist in the same cluster.
> - The wire protocol between servers is compatible.
> - Workers for distributed tasks, such as replication and log splitting,
> can co-exist in the same cluster.
> - Dependent protocols (such as using ZK for coordination) will also not
> be changed.
> - Example: A user can perform a rolling upgrade.
>
> - File format compatibility
>
> - Support file formats backward and forward compatible
> - Example: File, ZK encoding, directory layout is upgraded
> automatically as part of an HBase upgrade. User can rollback to the older
> version and everything will continue to work.
>
> - Client API compatibility
>
> - Allow changing or removing existing client APIs.
> - An API needs to deprecated for a major version before we will
> change/remove it.
> - Example: A user using a newly deprecated api does not need to modify
> application code with hbase api calls until the next major version.
>
> - Client Binary compatibility
>
> - Old client code can run unchanged (no recompilation needed) against
> new jars.
> - Example: Old compiled client code will work unchanged with the new
> jars.
>
> - Server-Side Limited API compatibility (taken from Hadoop)
>
> - Internal APIs are marked as Stable, Evolving, or Unstable
> - This implies binary compatibility for coprocessors and plugins
> (pluggable classes, including replication) as long as these are only using
> marked interfaces/classes.
> - Example: Old compiled Coprocessor, Filter, or Plugin code will work
> unchanged with the new jars.
>
> - Dependency Compatibility
>
> - An upgrade of HBase will not require an incompatible upgrade of a
> dependent project, including the Java runtime.
> - Example: An upgrade of Hadoop will not invalidate any of the
> compatibilities guarantees we made.
>
> - Operational Compatibility
>
> - Metric changes
> - Behavioral changes of services
> - Web page APIs
>
> Summary
>
> - A patch upgrade is a drop-in replacement. Any change that is not Java
> binary compatible would not be allowed.
> - A minor upgrade requires no application/client code modification.
> Ideally it would be a drop-in replacement but client code, coprocessors,
> filters, etc might have to be recompiled if new jars are used.
> - A major upgrade allows the HBase community to make breaking changes.
>
>
> Compatibility Matrix
>
> |
> | Major | Minor | Patch |
> | Client-Server wire Compatibility | N | Y | Y |
> | Server-Server Compatibility | N | Y | Y |
> | File Format Compatibility | N | Y | Y |
> | Client API Compatibility | N | Y | Y |
> | Client Binary Compatibility | N | N | Y |
> | Server-Side Limited API Compatibility |
> |
> |
> |
> |
> - Stable
> | N | Y | Y |
> |
> - Evolving
> | N | N | Y |
> |
> - Unstable
> | N | N | N |
> | Dependency Compatibility | N | Y | Y |
> | Operational Compatibility | N | N | Y |
>
> (Y means we support the compatibility. N means we can break it.)
>
>
--
Sean
Re: HBase - Semantic Versioning
Posted by Sean Busbey <bu...@cloudera.com>.
First, this is excellent work and will help greatly with planning around
HBase deployments. I think the document is big enough, of wide enough
interest, and important enough that we should keep it as a page in the site
outside of the ref guide. Maybe it could live near wherever we currently
keep our bylaws?
A very brief restatement of how major, minor, and patch map to version
strings X.Y.Z would help make the compatibility matrix easier to follow for
those not familiar with semantic versioning. Some may go read the full
reference, but most probably won't. Maybe "Summary" could be retitled
"Version Changes" and the bullet points could be preceded with a couple of
sentences.
I notice the document doesn't make any reference to our use of
InterfaceAudience[1]. Can we update it to do so?
Specifically, what is covered in "Client API", is it just stuff that's
IA.Public in hbase-client or IA.Public anywhere? If the former, then what
covers the rest?
I think the section on "server side limited API compatibility" is for
things marked IA.LimitedPrivate, but it would be nice to have it spelled
out.
The document covers InterfaceStability markings but only for server side.
Our current ref guide[1] covers them only for things marked public. Can we
unify this one way or the other? I suspect the answer is that we ought to
have them for both.
[1]: http://hbase.apache.org/book.html#d0e21466
On Fri, Nov 21, 2014 at 6:15 PM, Stack <st...@duboce.net> wrote:
> How should we progress here? Unless objection by tomorrow -- a week --
> then lets just adopt this doc? I can squash it into our dev section in
> refguide.
> St.Ack
>
> On Sat, Nov 15, 2014 at 5:06 PM, lars hofhansl <la...@apache.org> wrote:
>
> > As we approach the released of HBase 1.0.0, your PMC has been working
> hard
> > on documenting the exact compatibility guarantees we plan to support for
> > HBase versions going forward.
> > You can find the current version of the document here:
> >
> >
> https://docs.google.com/document/d/1p5pP7v2OuzSSaomK2S2v7sfKky1Hex6OqwsJO0sZTUY/edit?usp=sharing
> >
> > For convenience I am also including the entire text at the end of this
> > email (hopefully the formatting comes through).
> > Please have a look and let us know what you think.While we cannot
> possibly
> > include provisions for every corner case, this should be as comprehensive
> > as possible.Note that the semantic versioning document is itself
> > semantically versioned :)
> > Thanks.
> > -- Lars
> > --------------------------------------
> >
> > HBase Semantic Versioning - 1.0.4As HBase approaches its 1.0.0 version we
> > should start using semantic versioning.In addition to the usual API
> > versioning considerations HBase has other compatibility dimensions that
> we
> > need to consider.
> > Compatibility Dimensions
> >
> > - Client-Server wire protocol compatibility
> >
> > - Allows updating client and server out of sync.
> > - We could only allow upgrading the server first. I.e. the server
> would
> > be backward compatible to an old client, that way new APIs are OK.
> > - Example: A user should be able to use an old client to connect to an
> > upgraded cluster.
> >
> > - Server-Server protocol compatibility
> >
> > - Servers of different versions can co-exist in the same cluster.
> > - The wire protocol between servers is compatible.
> > - Workers for distributed tasks, such as replication and log
> splitting,
> > can co-exist in the same cluster.
> > - Dependent protocols (such as using ZK for coordination) will also
> not
> > be changed.
> > - Example: A user can perform a rolling upgrade.
> >
> > - File format compatibility
> >
> > - Support file formats backward and forward compatible
> > - Example: File, ZK encoding, directory layout is upgraded
> > automatically as part of an HBase upgrade. User can rollback to the older
> > version and everything will continue to work.
> >
> > - Client API compatibility
> >
> > - Allow changing or removing existing client APIs.
> > - An API needs to deprecated for a major version before we will
> > change/remove it.
> > - Example: A user using a newly deprecated api does not need to modify
> > application code with hbase api calls until the next major version.
> >
> > - Client Binary compatibility
> >
> > - Old client code can run unchanged (no recompilation needed) against
> > new jars.
> > - Example: Old compiled client code will work unchanged with the new
> > jars.
> >
> > - Server-Side Limited API compatibility (taken from Hadoop)
> >
> > - Internal APIs are marked as Stable, Evolving, or Unstable
> > - This implies binary compatibility for coprocessors and plugins
> > (pluggable classes, including replication) as long as these are only
> using
> > marked interfaces/classes.
> > - Example: Old compiled Coprocessor, Filter, or Plugin code will work
> > unchanged with the new jars.
> >
> > - Dependency Compatibility
> >
> > - An upgrade of HBase will not require an incompatible upgrade of a
> > dependent project, including the Java runtime.
> > - Example: An upgrade of Hadoop will not invalidate any of the
> > compatibilities guarantees we made.
> >
> > - Operational Compatibility
> >
> > - Metric changes
> > - Behavioral changes of services
> > - Web page APIs
> >
> > Summary
> >
> > - A patch upgrade is a drop-in replacement. Any change that is not
> Java
> > binary compatible would not be allowed.
> > - A minor upgrade requires no application/client code modification.
> > Ideally it would be a drop-in replacement but client code, coprocessors,
> > filters, etc might have to be recompiled if new jars are used.
> > - A major upgrade allows the HBase community to make breaking changes.
> >
> >
> > Compatibility Matrix
> >
> > |
> > | Major | Minor | Patch |
> > | Client-Server wire Compatibility | N | Y | Y |
> > | Server-Server Compatibility | N | Y | Y |
> > | File Format Compatibility | N | Y | Y |
> > | Client API Compatibility | N | Y | Y |
> > | Client Binary Compatibility | N | N | Y |
> > | Server-Side Limited API Compatibility |
> > |
> > |
> > |
> > |
> > - Stable
> > | N | Y | Y |
> > |
> > - Evolving
> > | N | N | Y |
> > |
> > - Unstable
> > | N | N | N |
> > | Dependency Compatibility | N | Y | Y |
> > | Operational Compatibility | N | N | Y |
> >
> > (Y means we support the compatibility. N means we can break it.)
> >
> >
>
--
Sean
Re: HBase - Semantic Versioning
Posted by Stack <st...@duboce.net>.
How should we progress here? Unless objection by tomorrow -- a week --
then lets just adopt this doc? I can squash it into our dev section in
refguide.
St.Ack
On Sat, Nov 15, 2014 at 5:06 PM, lars hofhansl <la...@apache.org> wrote:
> As we approach the released of HBase 1.0.0, your PMC has been working hard
> on documenting the exact compatibility guarantees we plan to support for
> HBase versions going forward.
> You can find the current version of the document here:
>
> https://docs.google.com/document/d/1p5pP7v2OuzSSaomK2S2v7sfKky1Hex6OqwsJO0sZTUY/edit?usp=sharing
>
> For convenience I am also including the entire text at the end of this
> email (hopefully the formatting comes through).
> Please have a look and let us know what you think.While we cannot possibly
> include provisions for every corner case, this should be as comprehensive
> as possible.Note that the semantic versioning document is itself
> semantically versioned :)
> Thanks.
> -- Lars
> --------------------------------------
>
> HBase Semantic Versioning - 1.0.4As HBase approaches its 1.0.0 version we
> should start using semantic versioning.In addition to the usual API
> versioning considerations HBase has other compatibility dimensions that we
> need to consider.
> Compatibility Dimensions
>
> - Client-Server wire protocol compatibility
>
> - Allows updating client and server out of sync.
> - We could only allow upgrading the server first. I.e. the server would
> be backward compatible to an old client, that way new APIs are OK.
> - Example: A user should be able to use an old client to connect to an
> upgraded cluster.
>
> - Server-Server protocol compatibility
>
> - Servers of different versions can co-exist in the same cluster.
> - The wire protocol between servers is compatible.
> - Workers for distributed tasks, such as replication and log splitting,
> can co-exist in the same cluster.
> - Dependent protocols (such as using ZK for coordination) will also not
> be changed.
> - Example: A user can perform a rolling upgrade.
>
> - File format compatibility
>
> - Support file formats backward and forward compatible
> - Example: File, ZK encoding, directory layout is upgraded
> automatically as part of an HBase upgrade. User can rollback to the older
> version and everything will continue to work.
>
> - Client API compatibility
>
> - Allow changing or removing existing client APIs.
> - An API needs to deprecated for a major version before we will
> change/remove it.
> - Example: A user using a newly deprecated api does not need to modify
> application code with hbase api calls until the next major version.
>
> - Client Binary compatibility
>
> - Old client code can run unchanged (no recompilation needed) against
> new jars.
> - Example: Old compiled client code will work unchanged with the new
> jars.
>
> - Server-Side Limited API compatibility (taken from Hadoop)
>
> - Internal APIs are marked as Stable, Evolving, or Unstable
> - This implies binary compatibility for coprocessors and plugins
> (pluggable classes, including replication) as long as these are only using
> marked interfaces/classes.
> - Example: Old compiled Coprocessor, Filter, or Plugin code will work
> unchanged with the new jars.
>
> - Dependency Compatibility
>
> - An upgrade of HBase will not require an incompatible upgrade of a
> dependent project, including the Java runtime.
> - Example: An upgrade of Hadoop will not invalidate any of the
> compatibilities guarantees we made.
>
> - Operational Compatibility
>
> - Metric changes
> - Behavioral changes of services
> - Web page APIs
>
> Summary
>
> - A patch upgrade is a drop-in replacement. Any change that is not Java
> binary compatible would not be allowed.
> - A minor upgrade requires no application/client code modification.
> Ideally it would be a drop-in replacement but client code, coprocessors,
> filters, etc might have to be recompiled if new jars are used.
> - A major upgrade allows the HBase community to make breaking changes.
>
>
> Compatibility Matrix
>
> |
> | Major | Minor | Patch |
> | Client-Server wire Compatibility | N | Y | Y |
> | Server-Server Compatibility | N | Y | Y |
> | File Format Compatibility | N | Y | Y |
> | Client API Compatibility | N | Y | Y |
> | Client Binary Compatibility | N | N | Y |
> | Server-Side Limited API Compatibility |
> |
> |
> |
> |
> - Stable
> | N | Y | Y |
> |
> - Evolving
> | N | N | Y |
> |
> - Unstable
> | N | N | N |
> | Dependency Compatibility | N | Y | Y |
> | Operational Compatibility | N | N | Y |
>
> (Y means we support the compatibility. N means we can break it.)
>
>
Re: HBase - Semantic Versioning
Posted by lars hofhansl <la...@apache.org>.
Looks like the formatting didn't come through. Please refer to the linked document.
Thanks.
-- Lars
From: lars hofhansl <la...@apache.org>
To: Dev <de...@hbase.apache.org>
Sent: Saturday, November 15, 2014 5:06 PM
Subject: HBase - Semantic Versioning
As we approach the released of HBase 1.0.0, your PMC has been working hard on documenting the exact compatibility guarantees we plan to support for HBase versions going forward.
You can find the current version of the document here:
https://docs.google.com/document/d/1p5pP7v2OuzSSaomK2S2v7sfKky1Hex6OqwsJO0sZTUY/edit?usp=sharing
For convenience I am also including the entire text at the end of this email (hopefully the formatting comes through).
Please have a look and let us know what you think.While we cannot possibly include provisions for every corner case, this should be as comprehensive as possible.Note that the semantic versioning document is itself semantically versioned :)
Thanks.
-- Lars
--------------------------------------
HBase Semantic Versioning - 1.0.4As HBase approaches its 1.0.0 version we should start using semantic versioning.In addition to the usual API versioning considerations HBase has other compatibility dimensions that we need to consider.
Compatibility Dimensions
- Client-Server wire protocol compatibility
- Allows updating client and server out of sync.
- We could only allow upgrading the server first. I.e. the server would be backward compatible to an old client, that way new APIs are OK.
- Example: A user should be able to use an old client to connect to an upgraded cluster.
- Server-Server protocol compatibility
- Servers of different versions can co-exist in the same cluster.
- The wire protocol between servers is compatible.
- Workers for distributed tasks, such as replication and log splitting, can co-exist in the same cluster.
- Dependent protocols (such as using ZK for coordination) will also not be changed.
- Example: A user can perform a rolling upgrade.
- File format compatibility
- Support file formats backward and forward compatible
- Example: File, ZK encoding, directory layout is upgraded automatically as part of an HBase upgrade. User can rollback to the older version and everything will continue to work.
- Client API compatibility
- Allow changing or removing existing client APIs.
- An API needs to deprecated for a major version before we will change/remove it.
- Example: A user using a newly deprecated api does not need to modify application code with hbase api calls until the next major version.
- Client Binary compatibility
- Old client code can run unchanged (no recompilation needed) against new jars.
- Example: Old compiled client code will work unchanged with the new jars.
- Server-Side Limited API compatibility (taken from Hadoop)
- Internal APIs are marked as Stable, Evolving, or Unstable
- This implies binary compatibility for coprocessors and plugins (pluggable classes, including replication) as long as these are only using marked interfaces/classes.
- Example: Old compiled Coprocessor, Filter, or Plugin code will work unchanged with the new jars.
- Dependency Compatibility
- An upgrade of HBase will not require an incompatible upgrade of a dependent project, including the Java runtime.
- Example: An upgrade of Hadoop will not invalidate any of the compatibilities guarantees we made.
- Operational Compatibility
- Metric changes
- Behavioral changes of services
- Web page APIs
Summary
- A patch upgrade is a drop-in replacement. Any change that is not Java binary compatible would not be allowed.
- A minor upgrade requires no application/client code modification. Ideally it would be a drop-in replacement but client code, coprocessors, filters, etc might have to be recompiled if new jars are used.
- A major upgrade allows the HBase community to make breaking changes.
Compatibility Matrix
|
| Major | Minor | Patch |
| Client-Server wire Compatibility | N | Y | Y |
| Server-Server Compatibility | N | Y | Y |
| File Format Compatibility | N | Y | Y |
| Client API Compatibility | N | Y | Y |
| Client Binary Compatibility | N | N | Y |
| Server-Side Limited API Compatibility |
|
|
|
|
- Stable
| N | Y | Y |
|
- Evolving
| N | N | Y |
|
- Unstable
| N | N | N |
| Dependency Compatibility | N | Y | Y |
| Operational Compatibility | N | N | Y |
(Y means we support the compatibility. N means we can break it.)