[DISCUSS] Backporting Regionserver Groups (HBASE-6721) to 1.x and 0.98

[Francis Liu <to...@ymail.com.INVALID>]

Hi,
HBASE-6721 is now committed to trunk. It'd be great if it can be backported to 1.x and 0.98 so that we can use it internally as well as push up features and fixes. We have been running an internal version for around 4 years. There's seems to be interest (HW, Bloomberg, Salesforce, etc). Also given how modular the code is. There's barely any effect in existing code paths. 
Seeding the criteria with Andy's suggestions in jira:
1. Stability - Unit tests and ?2. functional3. Performance - Read/write path was not affected. Some small changes related to assignment. 
Thanks,
Francis

Re: [DISCUSS] Backporting Regionserver Groups (HBASE-6721) to 1.x and 0.98

[Enis Söztutar <en...@gmail.com>]

I think the requirements so far are fair enough (UT, doc). Other than
assignment, there is nothing in the patch that touches core code (which is
by design). So the risk is pretty low.

We can do the backport and have a couple of ITBLL + ITLAV runs with CM to
verify the stability.

Enis

On Wed, Mar 16, 2016 at 10:42 AM, Sean Busbey <bu...@cloudera.com> wrote:

> I'd like to see the docs issue (HBASE-14856) handled before we backport to
> a release line. Getting the added integration test on the nightly build
> would also be good.
>
> On Wed, Mar 16, 2016 at 12:15 PM, Francis Liu <to...@ymail.com.invalid>
> wrote:
>
> > Hi,
> > HBASE-6721 is now committed to trunk. It'd be great if it can be
> > backported to 1.x and 0.98 so that we can use it internally as well as
> push
> > up features and fixes. We have been running an internal version for
> around
> > 4 years. There's seems to be interest (HW, Bloomberg, Salesforce, etc).
> > Also given how modular the code is. There's barely any effect in existing
> > code paths.
> > Seeding the criteria with Andy's suggestions in jira:
> > 1. Stability - Unit tests and ?2. functional3. Performance - Read/write
> > path was not affected. Some small changes related to assignment.
> > Thanks,
> > Francis
> >
> >
> >
> >
>
>
> --
> busbey
>

Re: [DISCUSS] Backporting Regionserver Groups (HBASE-6721) to 1.x and 0.98

[Sean Busbey <bu...@cloudera.com>]

I'd like to see the docs issue (HBASE-14856) handled before we backport to
a release line. Getting the added integration test on the nightly build
would also be good.

On Wed, Mar 16, 2016 at 12:15 PM, Francis Liu <to...@ymail.com.invalid>
wrote:

> Hi,
> HBASE-6721 is now committed to trunk. It'd be great if it can be
> backported to 1.x and 0.98 so that we can use it internally as well as push
> up features and fixes. We have been running an internal version for around
> 4 years. There's seems to be interest (HW, Bloomberg, Salesforce, etc).
> Also given how modular the code is. There's barely any effect in existing
> code paths.
> Seeding the criteria with Andy's suggestions in jira:
> 1. Stability - Unit tests and ?2. functional3. Performance - Read/write
> path was not affected. Some small changes related to assignment.
> Thanks,
> Francis
>
>
>
>


-- 
busbey

Re: [DISCUSS] Backporting Regionserver Groups (HBASE-6721) to 1.x and 0.98

[Ted Yu <yu...@gmail.com>]

bq. we instead focus on a releasable master

+1

One of the hurdles to the above is that some unit tests in master are more
flaky compared to their counterparts in branch-1.

Smoothing out the flaky tests is non-trivial effort.

On Mon, Mar 21, 2016 at 12:52 PM, Gary Helmling <gh...@gmail.com> wrote:

> >
> > Tho based on this discussion there's a bunch of good features that users
> > want that won't fit the criteria for #1 and #2. So allowing a backport of
> > the few that can fit into the criteria shouldn't significantly affect
> > future release from trunk. This way we can have some progress on some
> > features.
> >
> >
> If we have features that don't fit criteria #1 (compatibility), then we
> need a new major release according to our semver guidelines.  Criteria #2
> (stability) I think needs to be a gating factor on any new feature coming
> in.  We need to be able to demonstrate that we're not doing any harm for
> existing users by adding new features.
>
> In both cases, I'm not sure we're very well served by committing new
> features to master, then back-porting to branch-1.  That means that
> whatever stabilization is happening is likely from testing on branch-1
> based releases and not master, giving more time for changes to diverge.  I
> think we would be better served by releasing early and often from master
> and not having a branch-1 at all.  That would serve as a forcing function
> to make sure that change going in to master have to stabilize while "fresh"
> before a release is made.
>
> For more complex efforts, it would be better to stabilize in a feature
> branch and then merge the completed effort as one.
>
> I think we're already boxing ourselves in by looking at 2.0 as a feature
> based release, which means it necessarily has an unbounded timeline.  That
> means we have to do backports for other features ready before then, since
> master can't be released.  If we instead focus on a releasable master, that
> should help us release large changes more incrementally, and save ourselves
> the hassle of so many backports at the same time.  Of course everyone is
> free to invest their time where they choose.  But I think if we focus on
> time-based releases instead it would be better for the project as a whole.
>

Re: [DISCUSS] Backporting Regionserver Groups (HBASE-6721) to 1.x and 0.98

[Gary Helmling <gh...@gmail.com>]

>
> Tho based on this discussion there's a bunch of good features that users
> want that won't fit the criteria for #1 and #2. So allowing a backport of
> the few that can fit into the criteria shouldn't significantly affect
> future release from trunk. This way we can have some progress on some
> features.
>
>
If we have features that don't fit criteria #1 (compatibility), then we
need a new major release according to our semver guidelines.  Criteria #2
(stability) I think needs to be a gating factor on any new feature coming
in.  We need to be able to demonstrate that we're not doing any harm for
existing users by adding new features.

In both cases, I'm not sure we're very well served by committing new
features to master, then back-porting to branch-1.  That means that
whatever stabilization is happening is likely from testing on branch-1
based releases and not master, giving more time for changes to diverge.  I
think we would be better served by releasing early and often from master
and not having a branch-1 at all.  That would serve as a forcing function
to make sure that change going in to master have to stabilize while "fresh"
before a release is made.

For more complex efforts, it would be better to stabilize in a feature
branch and then merge the completed effort as one.

I think we're already boxing ourselves in by looking at 2.0 as a feature
based release, which means it necessarily has an unbounded timeline.  That
means we have to do backports for other features ready before then, since
master can't be released.  If we instead focus on a releasable master, that
should help us release large changes more incrementally, and save ourselves
the hassle of so many backports at the same time.  Of course everyone is
free to invest their time where they choose.  But I think if we focus on
time-based releases instead it would be better for the project as a whole.

Re: [DISCUSS] Backporting Regionserver Groups (HBASE-6721) to 1.x and 0.98

[Francis Liu <to...@apache.org>]

I see, that is indeed undesirable. 
Tho based on this discussion there's a bunch of good features that users want that won't fit the criteria for #1 and #2. So allowing a backport of the few that can fit into the criteria shouldn't significantly affect future release from trunk. This way we can have some progress on some features. 


 

    On Monday, March 21, 2016 10:23 AM, Sean Busbey <bu...@cloudera.com> wrote:
 

 Hadoop's trunk branch hasn't had a release in a very very long time. So it continues to accumulate changes that aren't in a release while folks drive their particular desired features back into branch-2.
On Mon, Mar 21, 2016 at 12:01 PM, Francis Liu <to...@apache.org> wrote:

To summarize so far it seems the concerns for backporting are:
1. compatibility - api, wire, rolling upgradeabability2. stability - destabilizing code and deploys for those that don't want the new feature
Is there anything else? 
Elliot what happened to hadoop? Is it neither of the two?
Francis




    On Thursday, March 17, 2016 12:01 PM, Enis Söztutar <en...@apache.org> wrote:


 As long as there is interest in doing the backport work, maintaining and
experimenting with the feature on an actual release (which seems to be the
case for RSGroups and Spark at least) and keeping the code base for
branch-1 stable, there is no reason not to backport. RsGroups (and I
believe Spark support as well) is marked experimental in release notes
which means that there is some room for our client-visible guarantees.

2.0 has it's own timeline and feature set and will come in time. Of course
anybody can propose to cut a release and push for getting it sooner rather
than later. We should all help the effort, but realistically, even if after
2.0 is released, there will be people running 1.x for years after that.

Enis

On Thu, Mar 17, 2016 at 11:28 AM, Mikhail Antonov <ol...@gmail.com>
wrote:

> >>>"Why MOB and RegionServer Groups should be in a new major version and
> stuff
> like the new RPC queue (HBASE-15136), date based tiered compactions
> (HBASE-15181), special handling for system tables WALs (HBASE-13557), keep
> table state in meta (HBASE-13017) or the Region Normalizer (HBASE-13103)
> are considered for or already in 1.x?"
>
> To me the differentiator would be "how much does it change the codebase
> around". If all/most of code change is the code which is new/ignored when
> feature is turned off, and by default it's off until well-tested by various
> users - then it should be fine to include. In the list above MOBs probably
> don't satisfy that, Spark and RS Groups probably do.
>
> If we make 2.0 release with just Mobs and RS Groups, that would mean new AM
> would have to be postponed to 3.0? What about procsV2? Although we would
> want rolling upgrade to 2.0, still it's our chance to release something
> big, invasive and new (since as was mentioned, the user expectation anyway
> would be that in new major version some incompatibilities would be present
> and some migration may be required)?
>
> -Mikhail
>
> On Thu, Mar 17, 2016 at 7:48 AM, Matteo Bertozzi <th...@gmail.com>
> wrote:
>
> > Why MOB and RegionServer Groups should be in a new major version and
> stuff
> > like the new RPC queue (HBASE-15136), date based tiered compactions
> > (HBASE-15181), special handling for system tables WALs (HBASE-13557),
> keep
> > table state in meta (HBASE-13017) or the Region Normalizer (HBASE-13103)
> > are considered for or already in 1.x?
> >
> > to me, and probably most of the users, a new Major version means that
> APIs
> > will break, wire may break, there may be an upgrade of some sort and so
> on.
> > which is not true for MOB and RS groups.
> >
> > In case we do a major for RS groups and Mob will that still based on the
> > 1.x branch?
> >
> > On Wed, Mar 16, 2016 at 11:23 PM, Andrew Purtell <
> andrew.purtell@gmail.com
> > >
> > wrote:
> >
> > > I remember explicitly saying I was not against a backport of the MOB
> > > feature. You are misrepresenting my position a bit. Sure, I'm a
> skeptic.
> > > Not a big deal because I don't think we can or should seek a blanket
> > rule.
> > > Sometimes a feature will have sufficient interest and applicability
> that
> > a
> > > backport is worth considering, and then there's a question of what kind
> > of
> > > risk the changes envisioned carry. RS groups as implemented are low
> risk.
> > > Meanwhile MOB is highly invasive. I think RS groups would have two
> large
> > > production users soon after introduction into branch-1. I'm not sure
> > about
> > > MOB. They are worth consideration on their own merits and on user
> demand
> > > for them.
> > >
> > > Another thing we could do is get 2.0 started right now. Whatever major
> > > that doesn't make the cut could go into 3.0. I think the requests for
> > these
> > > backports are coming because there is no near time horizon for a 2.0
> > > release. So set it soon?
> > >
> > >
> > > > On Mar 16, 2016, at 9:27 PM, Elliott Clark <ec...@apache.org>
> wrote:
> > > >
> > > > On Wed, Mar 16, 2016 at 6:26 PM, Andrew Purtell <
> > > andrew.purtell@gmail.com>
> > > > wrote:
> > > >
> > > >> Because I for one might well want to run RS groups in production
> with
> > > >> branch-1 code without waiting for or dealing with the other stuff
> > > coming in
> > > >> any 2.0.
> > > >
> > > >
> > > > This is the same email that I sent for MOB. Which you agreed with
> then.
> > > But
> > > > not now. There's nothing substantively different about this feature
> > that
> > > > makes it different from any other feature. It's a large change that
> > > wasn't
> > > > there in 1.X line.
> > > >
> > > > I would like backups, and procedure v2 in 1.x. However even if it
> > landed
> > > > tomorrow they shouldn't be back ported as it's a large feature that's
> > not
> > > > ready. If we want anyone to ever upgrade major versions, then the new
> > > > features have to come along with the new apis. Other wise we will end
> > up
> > > in
> > > > the same state that Hadoop has.
> > >
> >
>
>
>
> --
> Thanks,
> Michael Antonov
>


  



-- 
busbey

  

Re: [DISCUSS] Backporting Regionserver Groups (HBASE-6721) to 1.x and 0.98

[Sean Busbey <bu...@cloudera.com>]

Hadoop's trunk branch hasn't had a release in a very very long time. So it
continues to accumulate changes that aren't in a release while folks drive
their particular desired features back into branch-2.

On Mon, Mar 21, 2016 at 12:01 PM, Francis Liu <to...@apache.org> wrote:

> To summarize so far it seems the concerns for backporting are:
> 1. compatibility - api, wire, rolling upgradeabability2. stability -
> destabilizing code and deploys for those that don't want the new feature
> Is there anything else?
> Elliot what happened to hadoop? Is it neither of the two?
> Francis
>
>
>
>
>     On Thursday, March 17, 2016 12:01 PM, Enis Söztutar <en...@apache.org>
> wrote:
>
>
>  As long as there is interest in doing the backport work, maintaining and
> experimenting with the feature on an actual release (which seems to be the
> case for RSGroups and Spark at least) and keeping the code base for
> branch-1 stable, there is no reason not to backport. RsGroups (and I
> believe Spark support as well) is marked experimental in release notes
> which means that there is some room for our client-visible guarantees.
>
> 2.0 has it's own timeline and feature set and will come in time. Of course
> anybody can propose to cut a release and push for getting it sooner rather
> than later. We should all help the effort, but realistically, even if after
> 2.0 is released, there will be people running 1.x for years after that.
>
> Enis
>
> On Thu, Mar 17, 2016 at 11:28 AM, Mikhail Antonov <ol...@gmail.com>
> wrote:
>
> > >>>"Why MOB and RegionServer Groups should be in a new major version and
> > stuff
> > like the new RPC queue (HBASE-15136), date based tiered compactions
> > (HBASE-15181), special handling for system tables WALs (HBASE-13557),
> keep
> > table state in meta (HBASE-13017) or the Region Normalizer (HBASE-13103)
> > are considered for or already in 1.x?"
> >
> > To me the differentiator would be "how much does it change the codebase
> > around". If all/most of code change is the code which is new/ignored when
> > feature is turned off, and by default it's off until well-tested by
> various
> > users - then it should be fine to include. In the list above MOBs
> probably
> > don't satisfy that, Spark and RS Groups probably do.
> >
> > If we make 2.0 release with just Mobs and RS Groups, that would mean new
> AM
> > would have to be postponed to 3.0? What about procsV2? Although we would
> > want rolling upgrade to 2.0, still it's our chance to release something
> > big, invasive and new (since as was mentioned, the user expectation
> anyway
> > would be that in new major version some incompatibilities would be
> present
> > and some migration may be required)?
> >
> > -Mikhail
> >
> > On Thu, Mar 17, 2016 at 7:48 AM, Matteo Bertozzi <
> theo.bertozzi@gmail.com>
> > wrote:
> >
> > > Why MOB and RegionServer Groups should be in a new major version and
> > stuff
> > > like the new RPC queue (HBASE-15136), date based tiered compactions
> > > (HBASE-15181), special handling for system tables WALs (HBASE-13557),
> > keep
> > > table state in meta (HBASE-13017) or the Region Normalizer
> (HBASE-13103)
> > > are considered for or already in 1.x?
> > >
> > > to me, and probably most of the users, a new Major version means that
> > APIs
> > > will break, wire may break, there may be an upgrade of some sort and so
> > on.
> > > which is not true for MOB and RS groups.
> > >
> > > In case we do a major for RS groups and Mob will that still based on
> the
> > > 1.x branch?
> > >
> > > On Wed, Mar 16, 2016 at 11:23 PM, Andrew Purtell <
> > andrew.purtell@gmail.com
> > > >
> > > wrote:
> > >
> > > > I remember explicitly saying I was not against a backport of the MOB
> > > > feature. You are misrepresenting my position a bit. Sure, I'm a
> > skeptic.
> > > > Not a big deal because I don't think we can or should seek a blanket
> > > rule.
> > > > Sometimes a feature will have sufficient interest and applicability
> > that
> > > a
> > > > backport is worth considering, and then there's a question of what
> kind
> > > of
> > > > risk the changes envisioned carry. RS groups as implemented are low
> > risk.
> > > > Meanwhile MOB is highly invasive. I think RS groups would have two
> > large
> > > > production users soon after introduction into branch-1. I'm not sure
> > > about
> > > > MOB. They are worth consideration on their own merits and on user
> > demand
> > > > for them.
> > > >
> > > > Another thing we could do is get 2.0 started right now. Whatever
> major
> > > > that doesn't make the cut could go into 3.0. I think the requests for
> > > these
> > > > backports are coming because there is no near time horizon for a 2.0
> > > > release. So set it soon?
> > > >
> > > >
> > > > > On Mar 16, 2016, at 9:27 PM, Elliott Clark <ec...@apache.org>
> > wrote:
> > > > >
> > > > > On Wed, Mar 16, 2016 at 6:26 PM, Andrew Purtell <
> > > > andrew.purtell@gmail.com>
> > > > > wrote:
> > > > >
> > > > >> Because I for one might well want to run RS groups in production
> > with
> > > > >> branch-1 code without waiting for or dealing with the other stuff
> > > > coming in
> > > > >> any 2.0.
> > > > >
> > > > >
> > > > > This is the same email that I sent for MOB. Which you agreed with
> > then.
> > > > But
> > > > > not now. There's nothing substantively different about this feature
> > > that
> > > > > makes it different from any other feature. It's a large change that
> > > > wasn't
> > > > > there in 1.X line.
> > > > >
> > > > > I would like backups, and procedure v2 in 1.x. However even if it
> > > landed
> > > > > tomorrow they shouldn't be back ported as it's a large feature
> that's
> > > not
> > > > > ready. If we want anyone to ever upgrade major versions, then the
> new
> > > > > features have to come along with the new apis. Other wise we will
> end
> > > up
> > > > in
> > > > > the same state that Hadoop has.
> > > >
> > >
> >
> >
> >
> > --
> > Thanks,
> > Michael Antonov
> >
>
>
>
>



-- 
busbey

Re: [DISCUSS] Backporting Regionserver Groups (HBASE-6721) to 1.x and 0.98

[Francis Liu <to...@apache.org>]

To summarize so far it seems the concerns for backporting are:
1. compatibility - api, wire, rolling upgradeabability2. stability - destabilizing code and deploys for those that don't want the new feature
Is there anything else? 
Elliot what happened to hadoop? Is it neither of the two?
Francis


 

    On Thursday, March 17, 2016 12:01 PM, Enis Söztutar <en...@apache.org> wrote:
 

 As long as there is interest in doing the backport work, maintaining and
experimenting with the feature on an actual release (which seems to be the
case for RSGroups and Spark at least) and keeping the code base for
branch-1 stable, there is no reason not to backport. RsGroups (and I
believe Spark support as well) is marked experimental in release notes
which means that there is some room for our client-visible guarantees.

2.0 has it's own timeline and feature set and will come in time. Of course
anybody can propose to cut a release and push for getting it sooner rather
than later. We should all help the effort, but realistically, even if after
2.0 is released, there will be people running 1.x for years after that.

Enis

On Thu, Mar 17, 2016 at 11:28 AM, Mikhail Antonov <ol...@gmail.com>
wrote:

> >>>"Why MOB and RegionServer Groups should be in a new major version and
> stuff
> like the new RPC queue (HBASE-15136), date based tiered compactions
> (HBASE-15181), special handling for system tables WALs (HBASE-13557), keep
> table state in meta (HBASE-13017) or the Region Normalizer (HBASE-13103)
> are considered for or already in 1.x?"
>
> To me the differentiator would be "how much does it change the codebase
> around". If all/most of code change is the code which is new/ignored when
> feature is turned off, and by default it's off until well-tested by various
> users - then it should be fine to include. In the list above MOBs probably
> don't satisfy that, Spark and RS Groups probably do.
>
> If we make 2.0 release with just Mobs and RS Groups, that would mean new AM
> would have to be postponed to 3.0? What about procsV2? Although we would
> want rolling upgrade to 2.0, still it's our chance to release something
> big, invasive and new (since as was mentioned, the user expectation anyway
> would be that in new major version some incompatibilities would be present
> and some migration may be required)?
>
> -Mikhail
>
> On Thu, Mar 17, 2016 at 7:48 AM, Matteo Bertozzi <th...@gmail.com>
> wrote:
>
> > Why MOB and RegionServer Groups should be in a new major version and
> stuff
> > like the new RPC queue (HBASE-15136), date based tiered compactions
> > (HBASE-15181), special handling for system tables WALs (HBASE-13557),
> keep
> > table state in meta (HBASE-13017) or the Region Normalizer (HBASE-13103)
> > are considered for or already in 1.x?
> >
> > to me, and probably most of the users, a new Major version means that
> APIs
> > will break, wire may break, there may be an upgrade of some sort and so
> on.
> > which is not true for MOB and RS groups.
> >
> > In case we do a major for RS groups and Mob will that still based on the
> > 1.x branch?
> >
> > On Wed, Mar 16, 2016 at 11:23 PM, Andrew Purtell <
> andrew.purtell@gmail.com
> > >
> > wrote:
> >
> > > I remember explicitly saying I was not against a backport of the MOB
> > > feature. You are misrepresenting my position a bit. Sure, I'm a
> skeptic.
> > > Not a big deal because I don't think we can or should seek a blanket
> > rule.
> > > Sometimes a feature will have sufficient interest and applicability
> that
> > a
> > > backport is worth considering, and then there's a question of what kind
> > of
> > > risk the changes envisioned carry. RS groups as implemented are low
> risk.
> > > Meanwhile MOB is highly invasive. I think RS groups would have two
> large
> > > production users soon after introduction into branch-1. I'm not sure
> > about
> > > MOB. They are worth consideration on their own merits and on user
> demand
> > > for them.
> > >
> > > Another thing we could do is get 2.0 started right now. Whatever major
> > > that doesn't make the cut could go into 3.0. I think the requests for
> > these
> > > backports are coming because there is no near time horizon for a 2.0
> > > release. So set it soon?
> > >
> > >
> > > > On Mar 16, 2016, at 9:27 PM, Elliott Clark <ec...@apache.org>
> wrote:
> > > >
> > > > On Wed, Mar 16, 2016 at 6:26 PM, Andrew Purtell <
> > > andrew.purtell@gmail.com>
> > > > wrote:
> > > >
> > > >> Because I for one might well want to run RS groups in production
> with
> > > >> branch-1 code without waiting for or dealing with the other stuff
> > > coming in
> > > >> any 2.0.
> > > >
> > > >
> > > > This is the same email that I sent for MOB. Which you agreed with
> then.
> > > But
> > > > not now. There's nothing substantively different about this feature
> > that
> > > > makes it different from any other feature. It's a large change that
> > > wasn't
> > > > there in 1.X line.
> > > >
> > > > I would like backups, and procedure v2 in 1.x. However even if it
> > landed
> > > > tomorrow they shouldn't be back ported as it's a large feature that's
> > not
> > > > ready. If we want anyone to ever upgrade major versions, then the new
> > > > features have to come along with the new apis. Other wise we will end
> > up
> > > in
> > > > the same state that Hadoop has.
> > >
> >
>
>
>
> --
> Thanks,
> Michael Antonov
>


  

Re: [DISCUSS] Backporting Regionserver Groups (HBASE-6721) to 1.x and 0.98

[Enis Söztutar <en...@apache.org>]

As long as there is interest in doing the backport work, maintaining and
experimenting with the feature on an actual release (which seems to be the
case for RSGroups and Spark at least) and keeping the code base for
branch-1 stable, there is no reason not to backport. RsGroups (and I
believe Spark support as well) is marked experimental in release notes
which means that there is some room for our client-visible guarantees.

2.0 has it's own timeline and feature set and will come in time. Of course
anybody can propose to cut a release and push for getting it sooner rather
than later. We should all help the effort, but realistically, even if after
2.0 is released, there will be people running 1.x for years after that.

Enis

On Thu, Mar 17, 2016 at 11:28 AM, Mikhail Antonov <ol...@gmail.com>
wrote:

> >>>"Why MOB and RegionServer Groups should be in a new major version and
> stuff
> like the new RPC queue (HBASE-15136), date based tiered compactions
> (HBASE-15181), special handling for system tables WALs (HBASE-13557), keep
> table state in meta (HBASE-13017) or the Region Normalizer (HBASE-13103)
> are considered for or already in 1.x?"
>
> To me the differentiator would be "how much does it change the codebase
> around". If all/most of code change is the code which is new/ignored when
> feature is turned off, and by default it's off until well-tested by various
> users - then it should be fine to include. In the list above MOBs probably
> don't satisfy that, Spark and RS Groups probably do.
>
> If we make 2.0 release with just Mobs and RS Groups, that would mean new AM
> would have to be postponed to 3.0? What about procsV2? Although we would
> want rolling upgrade to 2.0, still it's our chance to release something
> big, invasive and new (since as was mentioned, the user expectation anyway
> would be that in new major version some incompatibilities would be present
> and some migration may be required)?
>
> -Mikhail
>
> On Thu, Mar 17, 2016 at 7:48 AM, Matteo Bertozzi <th...@gmail.com>
> wrote:
>
> > Why MOB and RegionServer Groups should be in a new major version and
> stuff
> > like the new RPC queue (HBASE-15136), date based tiered compactions
> > (HBASE-15181), special handling for system tables WALs (HBASE-13557),
> keep
> > table state in meta (HBASE-13017) or the Region Normalizer (HBASE-13103)
> > are considered for or already in 1.x?
> >
> > to me, and probably most of the users, a new Major version means that
> APIs
> > will break, wire may break, there may be an upgrade of some sort and so
> on.
> > which is not true for MOB and RS groups.
> >
> > In case we do a major for RS groups and Mob will that still based on the
> > 1.x branch?
> >
> > On Wed, Mar 16, 2016 at 11:23 PM, Andrew Purtell <
> andrew.purtell@gmail.com
> > >
> > wrote:
> >
> > > I remember explicitly saying I was not against a backport of the MOB
> > > feature. You are misrepresenting my position a bit. Sure, I'm a
> skeptic.
> > > Not a big deal because I don't think we can or should seek a blanket
> > rule.
> > > Sometimes a feature will have sufficient interest and applicability
> that
> > a
> > > backport is worth considering, and then there's a question of what kind
> > of
> > > risk the changes envisioned carry. RS groups as implemented are low
> risk.
> > > Meanwhile MOB is highly invasive. I think RS groups would have two
> large
> > > production users soon after introduction into branch-1. I'm not sure
> > about
> > > MOB. They are worth consideration on their own merits and on user
> demand
> > > for them.
> > >
> > > Another thing we could do is get 2.0 started right now. Whatever major
> > > that doesn't make the cut could go into 3.0. I think the requests for
> > these
> > > backports are coming because there is no near time horizon for a 2.0
> > > release. So set it soon?
> > >
> > >
> > > > On Mar 16, 2016, at 9:27 PM, Elliott Clark <ec...@apache.org>
> wrote:
> > > >
> > > > On Wed, Mar 16, 2016 at 6:26 PM, Andrew Purtell <
> > > andrew.purtell@gmail.com>
> > > > wrote:
> > > >
> > > >> Because I for one might well want to run RS groups in production
> with
> > > >> branch-1 code without waiting for or dealing with the other stuff
> > > coming in
> > > >> any 2.0.
> > > >
> > > >
> > > > This is the same email that I sent for MOB. Which you agreed with
> then.
> > > But
> > > > not now. There's nothing substantively different about this feature
> > that
> > > > makes it different from any other feature. It's a large change that
> > > wasn't
> > > > there in 1.X line.
> > > >
> > > > I would like backups, and procedure v2 in 1.x. However even if it
> > landed
> > > > tomorrow they shouldn't be back ported as it's a large feature that's
> > not
> > > > ready. If we want anyone to ever upgrade major versions, then the new
> > > > features have to come along with the new apis. Other wise we will end
> > up
> > > in
> > > > the same state that Hadoop has.
> > >
> >
>
>
>
> --
> Thanks,
> Michael Antonov
>

Re: [DISCUSS] Backporting Regionserver Groups (HBASE-6721) to 1.x and 0.98

[Mikhail Antonov <ol...@gmail.com>]

>>>"Why MOB and RegionServer Groups should be in a new major version and
stuff
like the new RPC queue (HBASE-15136), date based tiered compactions
(HBASE-15181), special handling for system tables WALs (HBASE-13557), keep
table state in meta (HBASE-13017) or the Region Normalizer (HBASE-13103)
are considered for or already in 1.x?"

To me the differentiator would be "how much does it change the codebase
around". If all/most of code change is the code which is new/ignored when
feature is turned off, and by default it's off until well-tested by various
users - then it should be fine to include. In the list above MOBs probably
don't satisfy that, Spark and RS Groups probably do.

If we make 2.0 release with just Mobs and RS Groups, that would mean new AM
would have to be postponed to 3.0? What about procsV2? Although we would
want rolling upgrade to 2.0, still it's our chance to release something
big, invasive and new (since as was mentioned, the user expectation anyway
would be that in new major version some incompatibilities would be present
and some migration may be required)?

-Mikhail

On Thu, Mar 17, 2016 at 7:48 AM, Matteo Bertozzi <th...@gmail.com>
wrote:

> Why MOB and RegionServer Groups should be in a new major version and stuff
> like the new RPC queue (HBASE-15136), date based tiered compactions
> (HBASE-15181), special handling for system tables WALs (HBASE-13557), keep
> table state in meta (HBASE-13017) or the Region Normalizer (HBASE-13103)
> are considered for or already in 1.x?
>
> to me, and probably most of the users, a new Major version means that APIs
> will break, wire may break, there may be an upgrade of some sort and so on.
> which is not true for MOB and RS groups.
>
> In case we do a major for RS groups and Mob will that still based on the
> 1.x branch?
>
> On Wed, Mar 16, 2016 at 11:23 PM, Andrew Purtell <andrew.purtell@gmail.com
> >
> wrote:
>
> > I remember explicitly saying I was not against a backport of the MOB
> > feature. You are misrepresenting my position a bit. Sure, I'm a skeptic.
> > Not a big deal because I don't think we can or should seek a blanket
> rule.
> > Sometimes a feature will have sufficient interest and applicability that
> a
> > backport is worth considering, and then there's a question of what kind
> of
> > risk the changes envisioned carry. RS groups as implemented are low risk.
> > Meanwhile MOB is highly invasive. I think RS groups would have two large
> > production users soon after introduction into branch-1. I'm not sure
> about
> > MOB. They are worth consideration on their own merits and on user demand
> > for them.
> >
> > Another thing we could do is get 2.0 started right now. Whatever major
> > that doesn't make the cut could go into 3.0. I think the requests for
> these
> > backports are coming because there is no near time horizon for a 2.0
> > release. So set it soon?
> >
> >
> > > On Mar 16, 2016, at 9:27 PM, Elliott Clark <ec...@apache.org> wrote:
> > >
> > > On Wed, Mar 16, 2016 at 6:26 PM, Andrew Purtell <
> > andrew.purtell@gmail.com>
> > > wrote:
> > >
> > >> Because I for one might well want to run RS groups in production with
> > >> branch-1 code without waiting for or dealing with the other stuff
> > coming in
> > >> any 2.0.
> > >
> > >
> > > This is the same email that I sent for MOB. Which you agreed with then.
> > But
> > > not now. There's nothing substantively different about this feature
> that
> > > makes it different from any other feature. It's a large change that
> > wasn't
> > > there in 1.X line.
> > >
> > > I would like backups, and procedure v2 in 1.x. However even if it
> landed
> > > tomorrow they shouldn't be back ported as it's a large feature that's
> not
> > > ready. If we want anyone to ever upgrade major versions, then the new
> > > features have to come along with the new apis. Other wise we will end
> up
> > in
> > > the same state that Hadoop has.
> >
>



-- 
Thanks,
Michael Antonov

Re: [DISCUSS] Backporting Regionserver Groups (HBASE-6721) to 1.x and 0.98

[Francis Liu <to...@apache.org>]

RS groups is also pretty non-invasive. It's a cp and the core code is in a separate module. 

    On Thursday, March 17, 2016 9:06 AM, Andrew Purtell <an...@gmail.com> wrote:
 

 I also don't see why we can't have the spark connector module in a 1.x release. I know our internal users would be delighted to have it available. 

> On Mar 17, 2016, at 9:00 AM, Matteo Bertozzi <th...@gmail.com> wrote:
> 
> spark is no different than RS group for this discussion.
> We forced francis to move everything in a module and use coprocessors to be
> external as possible.
> so, if RS group is not going to a 1.x we should have spark not going in for
> the same reason.
> 
> MOB is deep into HRegion and all the other stuff even if everything is
> under if isMobCF.
> so, I see why we may not want it just by looking at the code.
> API, and usefulness is another thing to consider but it doesn't look like
> we are discussing this in this thread.
> 
> 
>> On Thu, Mar 17, 2016 at 8:53 AM, Ted Yu <yu...@gmail.com> wrote:
>> 
>> For Spark, backporting to branch-1 makes sense since
>> 
>> it is in hbase-spark module - only adds one line to root pom.xml.
>> there has been frequent user query for when hbase-spark would be in an
>> hbase release
>> 
>> On Thu, Mar 17, 2016 at 8:26 AM, Matteo Bertozzi <th...@gmail.com>
>> wrote:
>> 
>>> If we cut master now we have
>>> - no rolling upgrade (am switched to zkless only and the other code was
>>> removed)
>>> - no api compatibility (we removed the deprecated)
>>> - Offheaping on the read path
>>> - Spark
>>> - MOB
>>> - RS Groups
>>> - some other stuff...
>>> 
>>> is that worth a major?
>>> The offheaping work is probably the one that cannot be backported and it
>>> may be nice to have.
>>> but stuff like spark, mob and RS Groups can be in a 1.x and avoid
>> migration
>>> pain, and those are probably the ones that some people wants to try now.
>>> 
>>> If we have a 2.0 for features like Spark, Mob and RS Groups I'd like to
>>> have it based on branch-1. At least users can move there without having
>> to
>>> worry about compatibility, even if the version number itself will
>> probably
>>> force the users to stick with the 1.x because the assumption is that
>>> something will break or there is a migration required.
>>> 
>>> On Thu, Mar 17, 2016 at 8:05 AM, Andrew Purtell <
>> andrew.purtell@gmail.com>
>>> wrote:
>>> 
>>>> I don't think we need to do a major for RS groups.
>>>> 
>>>> I do think Elliott's points can be addressed by getting a 2.0 out the
>>> door
>>>> soon containing whatever we have on deck now to go in.
>>>> 
>>>> Probably not going to satisfy everyone here - but maybe?
>>>> 
>>>> 
>>>>> On Mar 17, 2016, at 7:48 AM, Matteo Bertozzi <
>> theo.bertozzi@gmail.com>
>>>> wrote:
>>>>> 
>>>>> Why MOB and RegionServer Groups should be in a new major version and
>>>> stuff
>>>>> like the new RPC queue (HBASE-15136), date based tiered compactions
>>>>> (HBASE-15181), special handling for system tables WALs (HBASE-13557),
>>>> keep
>>>>> table state in meta (HBASE-13017) or the Region Normalizer
>>> (HBASE-13103)
>>>>> are considered for or already in 1.x?
>>>>> 
>>>>> to me, and probably most of the users, a new Major version means that
>>>> APIs
>>>>> will break, wire may break, there may be an upgrade of some sort and
>> so
>>>> on.
>>>>> which is not true for MOB and RS groups.
>>>>> 
>>>>> In case we do a major for RS groups and Mob will that still based on
>>> the
>>>>> 1.x branch?
>>>>> 
>>>>> On Wed, Mar 16, 2016 at 11:23 PM, Andrew Purtell <
>>>> andrew.purtell@gmail.com>
>>>>> wrote:
>>>>> 
>>>>>> I remember explicitly saying I was not against a backport of the MOB
>>>>>> feature. You are misrepresenting my position a bit. Sure, I'm a
>>> skeptic.
>>>>>> Not a big deal because I don't think we can or should seek a blanket
>>>> rule.
>>>>>> Sometimes a feature will have sufficient interest and applicability
>>>> that a
>>>>>> backport is worth considering, and then there's a question of what
>>> kind
>>>> of
>>>>>> risk the changes envisioned carry. RS groups as implemented are low
>>>> risk.
>>>>>> Meanwhile MOB is highly invasive. I think RS groups would have two
>>> large
>>>>>> production users soon after introduction into branch-1. I'm not sure
>>>> about
>>>>>> MOB. They are worth consideration on their own merits and on user
>>> demand
>>>>>> for them.
>>>>>> 
>>>>>> Another thing we could do is get 2.0 started right now. Whatever
>> major
>>>>>> that doesn't make the cut could go into 3.0. I think the requests
>> for
>>>> these
>>>>>> backports are coming because there is no near time horizon for a 2.0
>>>>>> release. So set it soon?
>>>>>> 
>>>>>> 
>>>>>>> On Mar 16, 2016, at 9:27 PM, Elliott Clark <ec...@apache.org>
>>> wrote:
>>>>>>> 
>>>>>>> On Wed, Mar 16, 2016 at 6:26 PM, Andrew Purtell <
>>>>>> andrew.purtell@gmail.com>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> Because I for one might well want to run RS groups in production
>>> with
>>>>>>>> branch-1 code without waiting for or dealing with the other stuff
>>>>>> coming in
>>>>>>>> any 2.0.
>>>>>>> 
>>>>>>> 
>>>>>>> This is the same email that I sent for MOB. Which you agreed with
>>> then.
>>>>>> But
>>>>>>> not now. There's nothing substantively different about this feature
>>>> that
>>>>>>> makes it different from any other feature. It's a large change that
>>>>>> wasn't
>>>>>>> there in 1.X line.
>>>>>>> 
>>>>>>> I would like backups, and procedure v2 in 1.x. However even if it
>>>> landed
>>>>>>> tomorrow they shouldn't be back ported as it's a large feature
>> that's
>>>> not
>>>>>>> ready. If we want anyone to ever upgrade major versions, then the
>> new
>>>>>>> features have to come along with the new apis. Other wise we will
>> end
>>>> up
>>>>>> in
>>>>>>> the same state that Hadoop has.
>> 

  

Re: [DISCUSS] Backporting Regionserver Groups (HBASE-6721) to 1.x and 0.98

[Andrew Purtell <an...@gmail.com>]

I also don't see why we can't have the spark connector module in a 1.x release. I know our internal users would be delighted to have it available. 

> On Mar 17, 2016, at 9:00 AM, Matteo Bertozzi <th...@gmail.com> wrote:
> 
> spark is no different than RS group for this discussion.
> We forced francis to move everything in a module and use coprocessors to be
> external as possible.
> so, if RS group is not going to a 1.x we should have spark not going in for
> the same reason.
> 
> MOB is deep into HRegion and all the other stuff even if everything is
> under if isMobCF.
> so, I see why we may not want it just by looking at the code.
> API, and usefulness is another thing to consider but it doesn't look like
> we are discussing this in this thread.
> 
> 
>> On Thu, Mar 17, 2016 at 8:53 AM, Ted Yu <yu...@gmail.com> wrote:
>> 
>> For Spark, backporting to branch-1 makes sense since
>> 
>> it is in hbase-spark module - only adds one line to root pom.xml.
>> there has been frequent user query for when hbase-spark would be in an
>> hbase release
>> 
>> On Thu, Mar 17, 2016 at 8:26 AM, Matteo Bertozzi <th...@gmail.com>
>> wrote:
>> 
>>> If we cut master now we have
>>> - no rolling upgrade (am switched to zkless only and the other code was
>>> removed)
>>> - no api compatibility (we removed the deprecated)
>>> - Offheaping on the read path
>>> - Spark
>>> - MOB
>>> - RS Groups
>>> - some other stuff...
>>> 
>>> is that worth a major?
>>> The offheaping work is probably the one that cannot be backported and it
>>> may be nice to have.
>>> but stuff like spark, mob and RS Groups can be in a 1.x and avoid
>> migration
>>> pain, and those are probably the ones that some people wants to try now.
>>> 
>>> If we have a 2.0 for features like Spark, Mob and RS Groups I'd like to
>>> have it based on branch-1. At least users can move there without having
>> to
>>> worry about compatibility, even if the version number itself will
>> probably
>>> force the users to stick with the 1.x because the assumption is that
>>> something will break or there is a migration required.
>>> 
>>> On Thu, Mar 17, 2016 at 8:05 AM, Andrew Purtell <
>> andrew.purtell@gmail.com>
>>> wrote:
>>> 
>>>> I don't think we need to do a major for RS groups.
>>>> 
>>>> I do think Elliott's points can be addressed by getting a 2.0 out the
>>> door
>>>> soon containing whatever we have on deck now to go in.
>>>> 
>>>> Probably not going to satisfy everyone here - but maybe?
>>>> 
>>>> 
>>>>> On Mar 17, 2016, at 7:48 AM, Matteo Bertozzi <
>> theo.bertozzi@gmail.com>
>>>> wrote:
>>>>> 
>>>>> Why MOB and RegionServer Groups should be in a new major version and
>>>> stuff
>>>>> like the new RPC queue (HBASE-15136), date based tiered compactions
>>>>> (HBASE-15181), special handling for system tables WALs (HBASE-13557),
>>>> keep
>>>>> table state in meta (HBASE-13017) or the Region Normalizer
>>> (HBASE-13103)
>>>>> are considered for or already in 1.x?
>>>>> 
>>>>> to me, and probably most of the users, a new Major version means that
>>>> APIs
>>>>> will break, wire may break, there may be an upgrade of some sort and
>> so
>>>> on.
>>>>> which is not true for MOB and RS groups.
>>>>> 
>>>>> In case we do a major for RS groups and Mob will that still based on
>>> the
>>>>> 1.x branch?
>>>>> 
>>>>> On Wed, Mar 16, 2016 at 11:23 PM, Andrew Purtell <
>>>> andrew.purtell@gmail.com>
>>>>> wrote:
>>>>> 
>>>>>> I remember explicitly saying I was not against a backport of the MOB
>>>>>> feature. You are misrepresenting my position a bit. Sure, I'm a
>>> skeptic.
>>>>>> Not a big deal because I don't think we can or should seek a blanket
>>>> rule.
>>>>>> Sometimes a feature will have sufficient interest and applicability
>>>> that a
>>>>>> backport is worth considering, and then there's a question of what
>>> kind
>>>> of
>>>>>> risk the changes envisioned carry. RS groups as implemented are low
>>>> risk.
>>>>>> Meanwhile MOB is highly invasive. I think RS groups would have two
>>> large
>>>>>> production users soon after introduction into branch-1. I'm not sure
>>>> about
>>>>>> MOB. They are worth consideration on their own merits and on user
>>> demand
>>>>>> for them.
>>>>>> 
>>>>>> Another thing we could do is get 2.0 started right now. Whatever
>> major
>>>>>> that doesn't make the cut could go into 3.0. I think the requests
>> for
>>>> these
>>>>>> backports are coming because there is no near time horizon for a 2.0
>>>>>> release. So set it soon?
>>>>>> 
>>>>>> 
>>>>>>> On Mar 16, 2016, at 9:27 PM, Elliott Clark <ec...@apache.org>
>>> wrote:
>>>>>>> 
>>>>>>> On Wed, Mar 16, 2016 at 6:26 PM, Andrew Purtell <
>>>>>> andrew.purtell@gmail.com>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> Because I for one might well want to run RS groups in production
>>> with
>>>>>>>> branch-1 code without waiting for or dealing with the other stuff
>>>>>> coming in
>>>>>>>> any 2.0.
>>>>>>> 
>>>>>>> 
>>>>>>> This is the same email that I sent for MOB. Which you agreed with
>>> then.
>>>>>> But
>>>>>>> not now. There's nothing substantively different about this feature
>>>> that
>>>>>>> makes it different from any other feature. It's a large change that
>>>>>> wasn't
>>>>>>> there in 1.X line.
>>>>>>> 
>>>>>>> I would like backups, and procedure v2 in 1.x. However even if it
>>>> landed
>>>>>>> tomorrow they shouldn't be back ported as it's a large feature
>> that's
>>>> not
>>>>>>> ready. If we want anyone to ever upgrade major versions, then the
>> new
>>>>>>> features have to come along with the new apis. Other wise we will
>> end
>>>> up
>>>>>> in
>>>>>>> the same state that Hadoop has.
>> 

Re: [DISCUSS] Backporting Regionserver Groups (HBASE-6721) to 1.x and 0.98

[Matteo Bertozzi <th...@gmail.com>]

spark is no different than RS group for this discussion.
We forced francis to move everything in a module and use coprocessors to be
external as possible.
so, if RS group is not going to a 1.x we should have spark not going in for
the same reason.

MOB is deep into HRegion and all the other stuff even if everything is
under if isMobCF.
so, I see why we may not want it just by looking at the code.
API, and usefulness is another thing to consider but it doesn't look like
we are discussing this in this thread.


On Thu, Mar 17, 2016 at 8:53 AM, Ted Yu <yu...@gmail.com> wrote:

> For Spark, backporting to branch-1 makes sense since
>
> it is in hbase-spark module - only adds one line to root pom.xml.
> there has been frequent user query for when hbase-spark would be in an
> hbase release
>
> On Thu, Mar 17, 2016 at 8:26 AM, Matteo Bertozzi <th...@gmail.com>
> wrote:
>
> > If we cut master now we have
> >  - no rolling upgrade (am switched to zkless only and the other code was
> > removed)
> >  - no api compatibility (we removed the deprecated)
> >  - Offheaping on the read path
> >  - Spark
> >  - MOB
> >  - RS Groups
> >  - some other stuff...
> >
> > is that worth a major?
> > The offheaping work is probably the one that cannot be backported and it
> > may be nice to have.
> > but stuff like spark, mob and RS Groups can be in a 1.x and avoid
> migration
> > pain, and those are probably the ones that some people wants to try now.
> >
> > If we have a 2.0 for features like Spark, Mob and RS Groups I'd like to
> > have it based on branch-1. At least users can move there without having
> to
> > worry about compatibility, even if the version number itself will
> probably
> > force the users to stick with the 1.x because the assumption is that
> > something will break or there is a migration required.
> >
> > On Thu, Mar 17, 2016 at 8:05 AM, Andrew Purtell <
> andrew.purtell@gmail.com>
> > wrote:
> >
> > > I don't think we need to do a major for RS groups.
> > >
> > > I do think Elliott's points can be addressed by getting a 2.0 out the
> > door
> > > soon containing whatever we have on deck now to go in.
> > >
> > > Probably not going to satisfy everyone here - but maybe?
> > >
> > >
> > > > On Mar 17, 2016, at 7:48 AM, Matteo Bertozzi <
> theo.bertozzi@gmail.com>
> > > wrote:
> > > >
> > > > Why MOB and RegionServer Groups should be in a new major version and
> > > stuff
> > > > like the new RPC queue (HBASE-15136), date based tiered compactions
> > > > (HBASE-15181), special handling for system tables WALs (HBASE-13557),
> > > keep
> > > > table state in meta (HBASE-13017) or the Region Normalizer
> > (HBASE-13103)
> > > > are considered for or already in 1.x?
> > > >
> > > > to me, and probably most of the users, a new Major version means that
> > > APIs
> > > > will break, wire may break, there may be an upgrade of some sort and
> so
> > > on.
> > > > which is not true for MOB and RS groups.
> > > >
> > > > In case we do a major for RS groups and Mob will that still based on
> > the
> > > > 1.x branch?
> > > >
> > > > On Wed, Mar 16, 2016 at 11:23 PM, Andrew Purtell <
> > > andrew.purtell@gmail.com>
> > > > wrote:
> > > >
> > > >> I remember explicitly saying I was not against a backport of the MOB
> > > >> feature. You are misrepresenting my position a bit. Sure, I'm a
> > skeptic.
> > > >> Not a big deal because I don't think we can or should seek a blanket
> > > rule.
> > > >> Sometimes a feature will have sufficient interest and applicability
> > > that a
> > > >> backport is worth considering, and then there's a question of what
> > kind
> > > of
> > > >> risk the changes envisioned carry. RS groups as implemented are low
> > > risk.
> > > >> Meanwhile MOB is highly invasive. I think RS groups would have two
> > large
> > > >> production users soon after introduction into branch-1. I'm not sure
> > > about
> > > >> MOB. They are worth consideration on their own merits and on user
> > demand
> > > >> for them.
> > > >>
> > > >> Another thing we could do is get 2.0 started right now. Whatever
> major
> > > >> that doesn't make the cut could go into 3.0. I think the requests
> for
> > > these
> > > >> backports are coming because there is no near time horizon for a 2.0
> > > >> release. So set it soon?
> > > >>
> > > >>
> > > >>> On Mar 16, 2016, at 9:27 PM, Elliott Clark <ec...@apache.org>
> > wrote:
> > > >>>
> > > >>> On Wed, Mar 16, 2016 at 6:26 PM, Andrew Purtell <
> > > >> andrew.purtell@gmail.com>
> > > >>> wrote:
> > > >>>
> > > >>>> Because I for one might well want to run RS groups in production
> > with
> > > >>>> branch-1 code without waiting for or dealing with the other stuff
> > > >> coming in
> > > >>>> any 2.0.
> > > >>>
> > > >>>
> > > >>> This is the same email that I sent for MOB. Which you agreed with
> > then.
> > > >> But
> > > >>> not now. There's nothing substantively different about this feature
> > > that
> > > >>> makes it different from any other feature. It's a large change that
> > > >> wasn't
> > > >>> there in 1.X line.
> > > >>>
> > > >>> I would like backups, and procedure v2 in 1.x. However even if it
> > > landed
> > > >>> tomorrow they shouldn't be back ported as it's a large feature
> that's
> > > not
> > > >>> ready. If we want anyone to ever upgrade major versions, then the
> new
> > > >>> features have to come along with the new apis. Other wise we will
> end
> > > up
> > > >> in
> > > >>> the same state that Hadoop has.
> > > >>
> > >
> >
>

Re: [DISCUSS] Backporting Regionserver Groups (HBASE-6721) to 1.x and 0.98

[Ted Yu <yu...@gmail.com>]

For Spark, backporting to branch-1 makes sense since

it is in hbase-spark module - only adds one line to root pom.xml.
there has been frequent user query for when hbase-spark would be in an
hbase release

On Thu, Mar 17, 2016 at 8:26 AM, Matteo Bertozzi <th...@gmail.com>
wrote:

> If we cut master now we have
>  - no rolling upgrade (am switched to zkless only and the other code was
> removed)
>  - no api compatibility (we removed the deprecated)
>  - Offheaping on the read path
>  - Spark
>  - MOB
>  - RS Groups
>  - some other stuff...
>
> is that worth a major?
> The offheaping work is probably the one that cannot be backported and it
> may be nice to have.
> but stuff like spark, mob and RS Groups can be in a 1.x and avoid migration
> pain, and those are probably the ones that some people wants to try now.
>
> If we have a 2.0 for features like Spark, Mob and RS Groups I'd like to
> have it based on branch-1. At least users can move there without having to
> worry about compatibility, even if the version number itself will probably
> force the users to stick with the 1.x because the assumption is that
> something will break or there is a migration required.
>
> On Thu, Mar 17, 2016 at 8:05 AM, Andrew Purtell <an...@gmail.com>
> wrote:
>
> > I don't think we need to do a major for RS groups.
> >
> > I do think Elliott's points can be addressed by getting a 2.0 out the
> door
> > soon containing whatever we have on deck now to go in.
> >
> > Probably not going to satisfy everyone here - but maybe?
> >
> >
> > > On Mar 17, 2016, at 7:48 AM, Matteo Bertozzi <th...@gmail.com>
> > wrote:
> > >
> > > Why MOB and RegionServer Groups should be in a new major version and
> > stuff
> > > like the new RPC queue (HBASE-15136), date based tiered compactions
> > > (HBASE-15181), special handling for system tables WALs (HBASE-13557),
> > keep
> > > table state in meta (HBASE-13017) or the Region Normalizer
> (HBASE-13103)
> > > are considered for or already in 1.x?
> > >
> > > to me, and probably most of the users, a new Major version means that
> > APIs
> > > will break, wire may break, there may be an upgrade of some sort and so
> > on.
> > > which is not true for MOB and RS groups.
> > >
> > > In case we do a major for RS groups and Mob will that still based on
> the
> > > 1.x branch?
> > >
> > > On Wed, Mar 16, 2016 at 11:23 PM, Andrew Purtell <
> > andrew.purtell@gmail.com>
> > > wrote:
> > >
> > >> I remember explicitly saying I was not against a backport of the MOB
> > >> feature. You are misrepresenting my position a bit. Sure, I'm a
> skeptic.
> > >> Not a big deal because I don't think we can or should seek a blanket
> > rule.
> > >> Sometimes a feature will have sufficient interest and applicability
> > that a
> > >> backport is worth considering, and then there's a question of what
> kind
> > of
> > >> risk the changes envisioned carry. RS groups as implemented are low
> > risk.
> > >> Meanwhile MOB is highly invasive. I think RS groups would have two
> large
> > >> production users soon after introduction into branch-1. I'm not sure
> > about
> > >> MOB. They are worth consideration on their own merits and on user
> demand
> > >> for them.
> > >>
> > >> Another thing we could do is get 2.0 started right now. Whatever major
> > >> that doesn't make the cut could go into 3.0. I think the requests for
> > these
> > >> backports are coming because there is no near time horizon for a 2.0
> > >> release. So set it soon?
> > >>
> > >>
> > >>> On Mar 16, 2016, at 9:27 PM, Elliott Clark <ec...@apache.org>
> wrote:
> > >>>
> > >>> On Wed, Mar 16, 2016 at 6:26 PM, Andrew Purtell <
> > >> andrew.purtell@gmail.com>
> > >>> wrote:
> > >>>
> > >>>> Because I for one might well want to run RS groups in production
> with
> > >>>> branch-1 code without waiting for or dealing with the other stuff
> > >> coming in
> > >>>> any 2.0.
> > >>>
> > >>>
> > >>> This is the same email that I sent for MOB. Which you agreed with
> then.
> > >> But
> > >>> not now. There's nothing substantively different about this feature
> > that
> > >>> makes it different from any other feature. It's a large change that
> > >> wasn't
> > >>> there in 1.X line.
> > >>>
> > >>> I would like backups, and procedure v2 in 1.x. However even if it
> > landed
> > >>> tomorrow they shouldn't be back ported as it's a large feature that's
> > not
> > >>> ready. If we want anyone to ever upgrade major versions, then the new
> > >>> features have to come along with the new apis. Other wise we will end
> > up
> > >> in
> > >>> the same state that Hadoop has.
> > >>
> >
>

Re: [DISCUSS] Backporting Regionserver Groups (HBASE-6721) to 1.x and 0.98

[Matteo Bertozzi <th...@gmail.com>]

If we cut master now we have
 - no rolling upgrade (am switched to zkless only and the other code was
removed)
 - no api compatibility (we removed the deprecated)
 - Offheaping on the read path
 - Spark
 - MOB
 - RS Groups
 - some other stuff...

is that worth a major?
The offheaping work is probably the one that cannot be backported and it
may be nice to have.
but stuff like spark, mob and RS Groups can be in a 1.x and avoid migration
pain, and those are probably the ones that some people wants to try now.

If we have a 2.0 for features like Spark, Mob and RS Groups I'd like to
have it based on branch-1. At least users can move there without having to
worry about compatibility, even if the version number itself will probably
force the users to stick with the 1.x because the assumption is that
something will break or there is a migration required.

On Thu, Mar 17, 2016 at 8:05 AM, Andrew Purtell <an...@gmail.com>
wrote:

> I don't think we need to do a major for RS groups.
>
> I do think Elliott's points can be addressed by getting a 2.0 out the door
> soon containing whatever we have on deck now to go in.
>
> Probably not going to satisfy everyone here - but maybe?
>
>
> > On Mar 17, 2016, at 7:48 AM, Matteo Bertozzi <th...@gmail.com>
> wrote:
> >
> > Why MOB and RegionServer Groups should be in a new major version and
> stuff
> > like the new RPC queue (HBASE-15136), date based tiered compactions
> > (HBASE-15181), special handling for system tables WALs (HBASE-13557),
> keep
> > table state in meta (HBASE-13017) or the Region Normalizer (HBASE-13103)
> > are considered for or already in 1.x?
> >
> > to me, and probably most of the users, a new Major version means that
> APIs
> > will break, wire may break, there may be an upgrade of some sort and so
> on.
> > which is not true for MOB and RS groups.
> >
> > In case we do a major for RS groups and Mob will that still based on the
> > 1.x branch?
> >
> > On Wed, Mar 16, 2016 at 11:23 PM, Andrew Purtell <
> andrew.purtell@gmail.com>
> > wrote:
> >
> >> I remember explicitly saying I was not against a backport of the MOB
> >> feature. You are misrepresenting my position a bit. Sure, I'm a skeptic.
> >> Not a big deal because I don't think we can or should seek a blanket
> rule.
> >> Sometimes a feature will have sufficient interest and applicability
> that a
> >> backport is worth considering, and then there's a question of what kind
> of
> >> risk the changes envisioned carry. RS groups as implemented are low
> risk.
> >> Meanwhile MOB is highly invasive. I think RS groups would have two large
> >> production users soon after introduction into branch-1. I'm not sure
> about
> >> MOB. They are worth consideration on their own merits and on user demand
> >> for them.
> >>
> >> Another thing we could do is get 2.0 started right now. Whatever major
> >> that doesn't make the cut could go into 3.0. I think the requests for
> these
> >> backports are coming because there is no near time horizon for a 2.0
> >> release. So set it soon?
> >>
> >>
> >>> On Mar 16, 2016, at 9:27 PM, Elliott Clark <ec...@apache.org> wrote:
> >>>
> >>> On Wed, Mar 16, 2016 at 6:26 PM, Andrew Purtell <
> >> andrew.purtell@gmail.com>
> >>> wrote:
> >>>
> >>>> Because I for one might well want to run RS groups in production with
> >>>> branch-1 code without waiting for or dealing with the other stuff
> >> coming in
> >>>> any 2.0.
> >>>
> >>>
> >>> This is the same email that I sent for MOB. Which you agreed with then.
> >> But
> >>> not now. There's nothing substantively different about this feature
> that
> >>> makes it different from any other feature. It's a large change that
> >> wasn't
> >>> there in 1.X line.
> >>>
> >>> I would like backups, and procedure v2 in 1.x. However even if it
> landed
> >>> tomorrow they shouldn't be back ported as it's a large feature that's
> not
> >>> ready. If we want anyone to ever upgrade major versions, then the new
> >>> features have to come along with the new apis. Other wise we will end
> up
> >> in
> >>> the same state that Hadoop has.
> >>
>

Re: [DISCUSS] Backporting Regionserver Groups (HBASE-6721) to 1.x and 0.98

[Andrew Purtell <an...@gmail.com>]

I don't think we need to do a major for RS groups. 

I do think Elliott's points can be addressed by getting a 2.0 out the door soon containing whatever we have on deck now to go in. 

Probably not going to satisfy everyone here - but maybe? 


> On Mar 17, 2016, at 7:48 AM, Matteo Bertozzi <th...@gmail.com> wrote:
> 
> Why MOB and RegionServer Groups should be in a new major version and stuff
> like the new RPC queue (HBASE-15136), date based tiered compactions
> (HBASE-15181), special handling for system tables WALs (HBASE-13557), keep
> table state in meta (HBASE-13017) or the Region Normalizer (HBASE-13103)
> are considered for or already in 1.x?
> 
> to me, and probably most of the users, a new Major version means that APIs
> will break, wire may break, there may be an upgrade of some sort and so on.
> which is not true for MOB and RS groups.
> 
> In case we do a major for RS groups and Mob will that still based on the
> 1.x branch?
> 
> On Wed, Mar 16, 2016 at 11:23 PM, Andrew Purtell <an...@gmail.com>
> wrote:
> 
>> I remember explicitly saying I was not against a backport of the MOB
>> feature. You are misrepresenting my position a bit. Sure, I'm a skeptic.
>> Not a big deal because I don't think we can or should seek a blanket rule.
>> Sometimes a feature will have sufficient interest and applicability that a
>> backport is worth considering, and then there's a question of what kind of
>> risk the changes envisioned carry. RS groups as implemented are low risk.
>> Meanwhile MOB is highly invasive. I think RS groups would have two large
>> production users soon after introduction into branch-1. I'm not sure about
>> MOB. They are worth consideration on their own merits and on user demand
>> for them.
>> 
>> Another thing we could do is get 2.0 started right now. Whatever major
>> that doesn't make the cut could go into 3.0. I think the requests for these
>> backports are coming because there is no near time horizon for a 2.0
>> release. So set it soon?
>> 
>> 
>>> On Mar 16, 2016, at 9:27 PM, Elliott Clark <ec...@apache.org> wrote:
>>> 
>>> On Wed, Mar 16, 2016 at 6:26 PM, Andrew Purtell <
>> andrew.purtell@gmail.com>
>>> wrote:
>>> 
>>>> Because I for one might well want to run RS groups in production with
>>>> branch-1 code without waiting for or dealing with the other stuff
>> coming in
>>>> any 2.0.
>>> 
>>> 
>>> This is the same email that I sent for MOB. Which you agreed with then.
>> But
>>> not now. There's nothing substantively different about this feature that
>>> makes it different from any other feature. It's a large change that
>> wasn't
>>> there in 1.X line.
>>> 
>>> I would like backups, and procedure v2 in 1.x. However even if it landed
>>> tomorrow they shouldn't be back ported as it's a large feature that's not
>>> ready. If we want anyone to ever upgrade major versions, then the new
>>> features have to come along with the new apis. Other wise we will end up
>> in
>>> the same state that Hadoop has.
>> 

Re: [DISCUSS] Backporting Regionserver Groups (HBASE-6721) to 1.x and 0.98

[Matteo Bertozzi <th...@gmail.com>]

Why MOB and RegionServer Groups should be in a new major version and stuff
like the new RPC queue (HBASE-15136), date based tiered compactions
(HBASE-15181), special handling for system tables WALs (HBASE-13557), keep
table state in meta (HBASE-13017) or the Region Normalizer (HBASE-13103)
are considered for or already in 1.x?

to me, and probably most of the users, a new Major version means that APIs
will break, wire may break, there may be an upgrade of some sort and so on.
which is not true for MOB and RS groups.

In case we do a major for RS groups and Mob will that still based on the
1.x branch?

On Wed, Mar 16, 2016 at 11:23 PM, Andrew Purtell <an...@gmail.com>
wrote:

> I remember explicitly saying I was not against a backport of the MOB
> feature. You are misrepresenting my position a bit. Sure, I'm a skeptic.
> Not a big deal because I don't think we can or should seek a blanket rule.
> Sometimes a feature will have sufficient interest and applicability that a
> backport is worth considering, and then there's a question of what kind of
> risk the changes envisioned carry. RS groups as implemented are low risk.
> Meanwhile MOB is highly invasive. I think RS groups would have two large
> production users soon after introduction into branch-1. I'm not sure about
> MOB. They are worth consideration on their own merits and on user demand
> for them.
>
> Another thing we could do is get 2.0 started right now. Whatever major
> that doesn't make the cut could go into 3.0. I think the requests for these
> backports are coming because there is no near time horizon for a 2.0
> release. So set it soon?
>
>
> > On Mar 16, 2016, at 9:27 PM, Elliott Clark <ec...@apache.org> wrote:
> >
> > On Wed, Mar 16, 2016 at 6:26 PM, Andrew Purtell <
> andrew.purtell@gmail.com>
> > wrote:
> >
> >> Because I for one might well want to run RS groups in production with
> >> branch-1 code without waiting for or dealing with the other stuff
> coming in
> >> any 2.0.
> >
> >
> > This is the same email that I sent for MOB. Which you agreed with then.
> But
> > not now. There's nothing substantively different about this feature that
> > makes it different from any other feature. It's a large change that
> wasn't
> > there in 1.X line.
> >
> > I would like backups, and procedure v2 in 1.x. However even if it landed
> > tomorrow they shouldn't be back ported as it's a large feature that's not
> > ready. If we want anyone to ever upgrade major versions, then the new
> > features have to come along with the new apis. Other wise we will end up
> in
> > the same state that Hadoop has.
>

Re: [DISCUSS] Backporting Regionserver Groups (HBASE-6721) to 1.x and 0.98

[Andrew Purtell <an...@gmail.com>]

I remember explicitly saying I was not against a backport of the MOB feature. You are misrepresenting my position a bit. Sure, I'm a skeptic. Not a big deal because I don't think we can or should seek a blanket rule. Sometimes a feature will have sufficient interest and applicability that a backport is worth considering, and then there's a question of what kind of risk the changes envisioned carry. RS groups as implemented are low risk. Meanwhile MOB is highly invasive. I think RS groups would have two large production users soon after introduction into branch-1. I'm not sure about MOB. They are worth consideration on their own merits and on user demand for them. 

Another thing we could do is get 2.0 started right now. Whatever major that doesn't make the cut could go into 3.0. I think the requests for these backports are coming because there is no near time horizon for a 2.0 release. So set it soon?


> On Mar 16, 2016, at 9:27 PM, Elliott Clark <ec...@apache.org> wrote:
> 
> On Wed, Mar 16, 2016 at 6:26 PM, Andrew Purtell <an...@gmail.com>
> wrote:
> 
>> Because I for one might well want to run RS groups in production with
>> branch-1 code without waiting for or dealing with the other stuff coming in
>> any 2.0.
> 
> 
> This is the same email that I sent for MOB. Which you agreed with then. But
> not now. There's nothing substantively different about this feature that
> makes it different from any other feature. It's a large change that wasn't
> there in 1.X line.
> 
> I would like backups, and procedure v2 in 1.x. However even if it landed
> tomorrow they shouldn't be back ported as it's a large feature that's not
> ready. If we want anyone to ever upgrade major versions, then the new
> features have to come along with the new apis. Other wise we will end up in
> the same state that Hadoop has.

Re: [DISCUSS] Backporting Regionserver Groups (HBASE-6721) to 1.x and 0.98

[Elliott Clark <ec...@apache.org>]

On Wed, Mar 16, 2016 at 6:26 PM, Andrew Purtell <an...@gmail.com>
wrote:

> Because I for one might well want to run RS groups in production with
> branch-1 code without waiting for or dealing with the other stuff coming in
> any 2.0.


This is the same email that I sent for MOB. Which you agreed with then. But
not now. There's nothing substantively different about this feature that
makes it different from any other feature. It's a large change that wasn't
there in 1.X line.

I would like backups, and procedure v2 in 1.x. However even if it landed
tomorrow they shouldn't be back ported as it's a large feature that's not
ready. If we want anyone to ever upgrade major versions, then the new
features have to come along with the new apis. Other wise we will end up in
the same state that Hadoop has.

Re: [DISCUSS] Backporting Regionserver Groups (HBASE-6721) to 1.x and 0.98

[Andrew Purtell <an...@gmail.com>]

Because I for one might well want to run RS groups in production with branch-1 code without waiting for or dealing with the other stuff coming in any 2.0. I might go so far as to fork if we end up needing it and a backport is prevented by any intransigent community element. 

> On Mar 16, 2016, at 4:07 PM, Elliott Clark <ec...@apache.org> wrote:
> 
> I just don't see a why we would back port. We're going to release a 2.0
> when things are ready. It will be a major feature release. Region server
> groups is a major feature. Backporting to branch-1 seems like an end run
> around what sem ver is supposed to mean (not the api guarantees, the actual
> meaning).
> 
> Backporting major features is a bad habit that the Hadoop community seems
> to have. We shouldn't follow their lead.
> 
> On Wed, Mar 16, 2016 at 2:51 PM, Andrew Purtell <ap...@apache.org> wrote:
> 
>>> Are we being more stringent with 0.98 because it is expected to be more
>> stable than a 1.x release?
>> 
>> I am not being more stringent with 0.98. A backport to branch-1 can be
>> justified IMHO because there appears to be active interest in having it
>> there to deploy out into production somewhere. Is this also the case for
>> 0.98? ​As you may recall I accepted ZK-less assignment into 0.98 because
>> Yahoo indicated they'd run the code, so as a result it would get regular
>> use. It was a decision that wouldn't make sense if there wasn't going to be
>> active use and upkeep. Otherwise we increase risk and make some users
>> nervous (I seem to recall Cloudera did not pick up the ZK less assignment
>> change back when they were still on 0.98) without improved utility for
>> other users in trade. Just my thinking on the matter.
>> 
>> 
>> On Wed, Mar 16, 2016 at 2:24 PM, Francis Liu <to...@apache.org> wrote:
>> 
>>>> performance tests focused on the impact of the feature on those who
>>> don't want it.
>>> Andy and Ted, Given that this code does not touch the write path or the
>>> read path at all it would seem practical to skip read/write perf tests
>> (ie
>>> YCSB, PE, etc)?
>>>> where the result will go into someone's production.
>>> Are we being more stringent with 0.98 because it is expected to be more
>>> stable than a 1.x release?
>>> 
>>> 
>>>    On Wednesday, March 16, 2016 12:11 PM, Ted Yu <yu...@gmail.com>
>>> wrote:
>>> 
>>> 
>>> bq.  same things I asked for MOB: Functional, stability, and performance
>>> tests focused on the impact of the feature on those who don't want it.
>>> 
>>> +1 on the above.
>>> 
>>> On Wed, Mar 16, 2016 at 11:14 AM, Andrew Purtell <ap...@apache.org>
>>> wrote:
>>> 
>>>> I would like to see the same things I asked for MOB: Functional,
>>> stability,
>>>> and performance tests focused on the impact of the feature on those who
>>>> don't want it. Can use the usual suspects: PE, LTT, YCSB, our ITs.
>> Given
>>>> how 6721 has been implemented I suspect favorable results will be easy
>> to
>>>> obtain.
>>>> 
>>>> I think we would like to see a backport to branch-1 because we will be
>>>> bringing our production up to a 1.x soon.
>>>> 
>>>> Its fair to consider a backport to 0.98 but as RM for that branch I'd
>>> like
>>>> to see it go into branch-1 first and also have a case where the result
>>> will
>>>> go into someone's production.
>>>> 
>>>> 
>>>> On Wed, Mar 16, 2016 at 10:15 AM, Francis Liu <toffer@ymail.com.invalid
>>> 
>>>> wrote:
>>>> 
>>>>> Hi,
>>>>> HBASE-6721 is now committed to trunk. It'd be great if it can be
>>>>> backported to 1.x and 0.98 so that we can use it internally as well
>> as
>>>> push
>>>>> up features and fixes. We have been running an internal version for
>>>> around
>>>>> 4 years. There's seems to be interest (HW, Bloomberg, Salesforce,
>> etc).
>>>>> Also given how modular the code is. There's barely any effect in
>>> existing
>>>>> code paths.
>>>>> Seeding the criteria with Andy's suggestions in jira:
>>>>> 1. Stability - Unit tests and ?2. functional3. Performance -
>> Read/write
>>>>> path was not affected. Some small changes related to assignment.
>>>>> Thanks,
>>>>> Francis
>>>> 
>>>> 
>>>> --
>>>> Best regards,
>>>> 
>>>>   - Andy
>>>> 
>>>> Problems worthy of attack prove their worth by hitting back. - Piet
>> Hein
>>>> (via Tom White)
>> 
>> 
>> 
>> --
>> Best regards,
>> 
>>   - Andy
>> 
>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> (via Tom White)
>> 

Re: [DISCUSS] Backporting Regionserver Groups (HBASE-6721) to 1.x and 0.98

[Elliott Clark <ec...@apache.org>]

I just don't see a why we would back port. We're going to release a 2.0
when things are ready. It will be a major feature release. Region server
groups is a major feature. Backporting to branch-1 seems like an end run
around what sem ver is supposed to mean (not the api guarantees, the actual
meaning).

Backporting major features is a bad habit that the Hadoop community seems
to have. We shouldn't follow their lead.

On Wed, Mar 16, 2016 at 2:51 PM, Andrew Purtell <ap...@apache.org> wrote:

> > Are we being more stringent with 0.98 because it is expected to be more
> stable than a 1.x release?
>
> I am not being more stringent with 0.98. A backport to branch-1 can be
> justified IMHO because there appears to be active interest in having it
> there to deploy out into production somewhere. Is this also the case for
> 0.98? ​As you may recall I accepted ZK-less assignment into 0.98 because
> Yahoo indicated they'd run the code, so as a result it would get regular
> use. It was a decision that wouldn't make sense if there wasn't going to be
> active use and upkeep. Otherwise we increase risk and make some users
> nervous (I seem to recall Cloudera did not pick up the ZK less assignment
> change back when they were still on 0.98) without improved utility for
> other users in trade. Just my thinking on the matter.
>
>
> On Wed, Mar 16, 2016 at 2:24 PM, Francis Liu <to...@apache.org> wrote:
>
> > > performance tests focused on the impact of the feature on those who
> > don't want it.
> > Andy and Ted, Given that this code does not touch the write path or the
> > read path at all it would seem practical to skip read/write perf tests
> (ie
> > YCSB, PE, etc)?
> > > where the result will go into someone's production.
> > Are we being more stringent with 0.98 because it is expected to be more
> > stable than a 1.x release?
> >
> >
> >     On Wednesday, March 16, 2016 12:11 PM, Ted Yu <yu...@gmail.com>
> > wrote:
> >
> >
> >  bq.  same things I asked for MOB: Functional, stability, and performance
> > tests focused on the impact of the feature on those who don't want it.
> >
> > +1 on the above.
> >
> > On Wed, Mar 16, 2016 at 11:14 AM, Andrew Purtell <ap...@apache.org>
> > wrote:
> >
> > > I would like to see the same things I asked for MOB: Functional,
> > stability,
> > > and performance tests focused on the impact of the feature on those who
> > > don't want it. Can use the usual suspects: PE, LTT, YCSB, our ITs.
> Given
> > > how 6721 has been implemented I suspect favorable results will be easy
> to
> > > obtain.
> > >
> > > I think we would like to see a backport to branch-1 because we will be
> > > bringing our production up to a 1.x soon.
> > >
> > > Its fair to consider a backport to 0.98 but as RM for that branch I'd
> > like
> > > to see it go into branch-1 first and also have a case where the result
> > will
> > > go into someone's production.
> > >
> > >
> > > On Wed, Mar 16, 2016 at 10:15 AM, Francis Liu <toffer@ymail.com.invalid
> >
> > > wrote:
> > >
> > > > Hi,
> > > > HBASE-6721 is now committed to trunk. It'd be great if it can be
> > > > backported to 1.x and 0.98 so that we can use it internally as well
> as
> > > push
> > > > up features and fixes. We have been running an internal version for
> > > around
> > > > 4 years. There's seems to be interest (HW, Bloomberg, Salesforce,
> etc).
> > > > Also given how modular the code is. There's barely any effect in
> > existing
> > > > code paths.
> > > > Seeding the criteria with Andy's suggestions in jira:
> > > > 1. Stability - Unit tests and ?2. functional3. Performance -
> Read/write
> > > > path was not affected. Some small changes related to assignment.
> > > > Thanks,
> > > > Francis
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Best regards,
> > >
> > >    - Andy
> > >
> > > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > > (via Tom White)
> > >
> >
> >
> >
> >
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

Re: [DISCUSS] Backporting Regionserver Groups (HBASE-6721) to 1.x and 0.98

[Andrew Purtell <ap...@apache.org>]

> Are we being more stringent with 0.98 because it is expected to be more
stable than a 1.x release?

I am not being more stringent with 0.98. A backport to branch-1 can be
justified IMHO because there appears to be active interest in having it
there to deploy out into production somewhere. Is this also the case for
0.98? ​As you may recall I accepted ZK-less assignment into 0.98 because
Yahoo indicated they'd run the code, so as a result it would get regular
use. It was a decision that wouldn't make sense if there wasn't going to be
active use and upkeep. Otherwise we increase risk and make some users
nervous (I seem to recall Cloudera did not pick up the ZK less assignment
change back when they were still on 0.98) without improved utility for
other users in trade. Just my thinking on the matter.


On Wed, Mar 16, 2016 at 2:24 PM, Francis Liu <to...@apache.org> wrote:

> > performance tests focused on the impact of the feature on those who
> don't want it.
> Andy and Ted, Given that this code does not touch the write path or the
> read path at all it would seem practical to skip read/write perf tests (ie
> YCSB, PE, etc)?
> > where the result will go into someone's production.
> Are we being more stringent with 0.98 because it is expected to be more
> stable than a 1.x release?
>
>
>     On Wednesday, March 16, 2016 12:11 PM, Ted Yu <yu...@gmail.com>
> wrote:
>
>
>  bq.  same things I asked for MOB: Functional, stability, and performance
> tests focused on the impact of the feature on those who don't want it.
>
> +1 on the above.
>
> On Wed, Mar 16, 2016 at 11:14 AM, Andrew Purtell <ap...@apache.org>
> wrote:
>
> > I would like to see the same things I asked for MOB: Functional,
> stability,
> > and performance tests focused on the impact of the feature on those who
> > don't want it. Can use the usual suspects: PE, LTT, YCSB, our ITs. Given
> > how 6721 has been implemented I suspect favorable results will be easy to
> > obtain.
> >
> > I think we would like to see a backport to branch-1 because we will be
> > bringing our production up to a 1.x soon.
> >
> > Its fair to consider a backport to 0.98 but as RM for that branch I'd
> like
> > to see it go into branch-1 first and also have a case where the result
> will
> > go into someone's production.
> >
> >
> > On Wed, Mar 16, 2016 at 10:15 AM, Francis Liu <to...@ymail.com.invalid>
> > wrote:
> >
> > > Hi,
> > > HBASE-6721 is now committed to trunk. It'd be great if it can be
> > > backported to 1.x and 0.98 so that we can use it internally as well as
> > push
> > > up features and fixes. We have been running an internal version for
> > around
> > > 4 years. There's seems to be interest (HW, Bloomberg, Salesforce, etc).
> > > Also given how modular the code is. There's barely any effect in
> existing
> > > code paths.
> > > Seeding the criteria with Andy's suggestions in jira:
> > > 1. Stability - Unit tests and ?2. functional3. Performance - Read/write
> > > path was not affected. Some small changes related to assignment.
> > > Thanks,
> > > Francis
> > >
> > >
> > >
> > >
> >
> >
> > --
> > Best regards,
> >
> >    - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
>
>
>
>



-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Re: [DISCUSS] Backporting Regionserver Groups (HBASE-6721) to 1.x and 0.98

[Francis Liu <to...@apache.org>]

> performance tests focused on the impact of the feature on those who don't want it.
Andy and Ted, Given that this code does not touch the write path or the read path at all it would seem practical to skip read/write perf tests (ie YCSB, PE, etc)?
> where the result will go into someone's production.
Are we being more stringent with 0.98 because it is expected to be more stable than a 1.x release?


    On Wednesday, March 16, 2016 12:11 PM, Ted Yu <yu...@gmail.com> wrote:
 

 bq.  same things I asked for MOB: Functional, stability, and performance
tests focused on the impact of the feature on those who don't want it.

+1 on the above.

On Wed, Mar 16, 2016 at 11:14 AM, Andrew Purtell <ap...@apache.org>
wrote:

> I would like to see the same things I asked for MOB: Functional, stability,
> and performance tests focused on the impact of the feature on those who
> don't want it. Can use the usual suspects: PE, LTT, YCSB, our ITs. Given
> how 6721 has been implemented I suspect favorable results will be easy to
> obtain.
>
> I think we would like to see a backport to branch-1 because we will be
> bringing our production up to a 1.x soon.
>
> Its fair to consider a backport to 0.98 but as RM for that branch I'd like
> to see it go into branch-1 first and also have a case where the result will
> go into someone's production.
>
>
> On Wed, Mar 16, 2016 at 10:15 AM, Francis Liu <to...@ymail.com.invalid>
> wrote:
>
> > Hi,
> > HBASE-6721 is now committed to trunk. It'd be great if it can be
> > backported to 1.x and 0.98 so that we can use it internally as well as
> push
> > up features and fixes. We have been running an internal version for
> around
> > 4 years. There's seems to be interest (HW, Bloomberg, Salesforce, etc).
> > Also given how modular the code is. There's barely any effect in existing
> > code paths.
> > Seeding the criteria with Andy's suggestions in jira:
> > 1. Stability - Unit tests and ?2. functional3. Performance - Read/write
> > path was not affected. Some small changes related to assignment.
> > Thanks,
> > Francis
> >
> >
> >
> >
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>


  

Re: [DISCUSS] Backporting Regionserver Groups (HBASE-6721) to 1.x and 0.98

[Ted Yu <yu...@gmail.com>]

bq.  same things I asked for MOB: Functional, stability, and performance
tests focused on the impact of the feature on those who don't want it.

+1 on the above.

On Wed, Mar 16, 2016 at 11:14 AM, Andrew Purtell <ap...@apache.org>
wrote:

> I would like to see the same things I asked for MOB: Functional, stability,
> and performance tests focused on the impact of the feature on those who
> don't want it. Can use the usual suspects: PE, LTT, YCSB, our ITs. Given
> how 6721 has been implemented I suspect favorable results will be easy to
> obtain.
>
> I think we would like to see a backport to branch-1 because we will be
> bringing our production up to a 1.x soon.
>
> Its fair to consider a backport to 0.98 but as RM for that branch I'd like
> to see it go into branch-1 first and also have a case where the result will
> go into someone's production.
>
>
> On Wed, Mar 16, 2016 at 10:15 AM, Francis Liu <to...@ymail.com.invalid>
> wrote:
>
> > Hi,
> > HBASE-6721 is now committed to trunk. It'd be great if it can be
> > backported to 1.x and 0.98 so that we can use it internally as well as
> push
> > up features and fixes. We have been running an internal version for
> around
> > 4 years. There's seems to be interest (HW, Bloomberg, Salesforce, etc).
> > Also given how modular the code is. There's barely any effect in existing
> > code paths.
> > Seeding the criteria with Andy's suggestions in jira:
> > 1. Stability - Unit tests and ?2. functional3. Performance - Read/write
> > path was not affected. Some small changes related to assignment.
> > Thanks,
> > Francis
> >
> >
> >
> >
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

Re: [DISCUSS] Backporting Regionserver Groups (HBASE-6721) to 1.x and 0.98

[Andrew Purtell <ap...@apache.org>]

I would like to see the same things I asked for MOB: Functional, stability,
and performance tests focused on the impact of the feature on those who
don't want it. Can use the usual suspects: PE, LTT, YCSB, our ITs. Given
how 6721 has been implemented I suspect favorable results will be easy to
obtain.

I think we would like to see a backport to branch-1 because we will be
bringing our production up to a 1.x soon.

Its fair to consider a backport to 0.98 but as RM for that branch I'd like
to see it go into branch-1 first and also have a case where the result will
go into someone's production.


On Wed, Mar 16, 2016 at 10:15 AM, Francis Liu <to...@ymail.com.invalid>
wrote:

> Hi,
> HBASE-6721 is now committed to trunk. It'd be great if it can be
> backported to 1.x and 0.98 so that we can use it internally as well as push
> up features and fixes. We have been running an internal version for around
> 4 years. There's seems to be interest (HW, Bloomberg, Salesforce, etc).
> Also given how modular the code is. There's barely any effect in existing
> code paths.
> Seeding the criteria with Andy's suggestions in jira:
> 1. Stability - Unit tests and ?2. functional3. Performance - Read/write
> path was not affected. Some small changes related to assignment.
> Thanks,
> Francis
>
>
>
>


-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)