You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Jimmy Xiang <jx...@cloudera.com> on 2014/03/20 17:08:29 UTC
Co-locate meta and master, and beyond
Here is a heads up on some work I've been doing of late.
My goal is to make region assignment simpler, faster, more reliable
and maintainable making it so we are able to handle many more regions
more efficiently (improve mttr, and lessen impact of compactions), and
to eventually look into quorum-based master service.
In HBASE-10569, I posted a patch that makes a HMaster also a
HRegionServer. I've also done work so that we can co-locate the meta
region with the master server. These changes are based on one of the
agreements per the discussion in HBASE-5487, the master re-design.
We should be able to do a rolling restart on to this new basis (I'm
working on it).
Putting the master and regionserver processes together and making it
so meta and master are also together is a step toward making it easier
doing
region assignment without using ZK, another agreement from HBASE-5487.
Thanks.
Jimmy
Re: Co-locate meta and master, and beyond
Posted by Jimmy Xiang <jx...@cloudera.com>.
Thanks Stack for the review. I will come up a release note and commit the
patch to trunk tomorrow if no objection. I will address the follow up
issues soon.
Thanks everyone for the encouragement :)
On Mon, Mar 24, 2014 at 3:46 PM, Ted Yu <yu...@gmail.com> wrote:
> Nice work Jimmy.
>
>
> On Thu, Mar 20, 2014 at 9:08 AM, Jimmy Xiang <jx...@cloudera.com> wrote:
>
> > Here is a heads up on some work I've been doing of late.
> >
> >
> > My goal is to make region assignment simpler, faster, more reliable
> >
> > and maintainable making it so we are able to handle many more regions
> >
> > more efficiently (improve mttr, and lessen impact of compactions), and
> >
> > to eventually look into quorum-based master service.
> >
> > In HBASE-10569, I posted a patch that makes a HMaster also a
> > HRegionServer. I've also done work so that we can co-locate the meta
> > region with the master server. These changes are based on one of the
> > agreements per the discussion in HBASE-5487, the master re-design.
> >
> > We should be able to do a rolling restart on to this new basis (I'm
> >
> > working on it).
> >
> > Putting the master and regionserver processes together and making it
> >
> > so meta and master are also together is a step toward making it easier
> > doing
> > region assignment without using ZK, another agreement from HBASE-5487.
> >
> >
> > Thanks.
> >
> > Jimmy
> >
>
Re: Co-locate meta and master, and beyond
Posted by Ted Yu <yu...@gmail.com>.
Nice work Jimmy.
On Thu, Mar 20, 2014 at 9:08 AM, Jimmy Xiang <jx...@cloudera.com> wrote:
> Here is a heads up on some work I've been doing of late.
>
>
> My goal is to make region assignment simpler, faster, more reliable
>
> and maintainable making it so we are able to handle many more regions
>
> more efficiently (improve mttr, and lessen impact of compactions), and
>
> to eventually look into quorum-based master service.
>
> In HBASE-10569, I posted a patch that makes a HMaster also a
> HRegionServer. I've also done work so that we can co-locate the meta
> region with the master server. These changes are based on one of the
> agreements per the discussion in HBASE-5487, the master re-design.
>
> We should be able to do a rolling restart on to this new basis (I'm
>
> working on it).
>
> Putting the master and regionserver processes together and making it
>
> so meta and master are also together is a step toward making it easier
> doing
> region assignment without using ZK, another agreement from HBASE-5487.
>
>
> Thanks.
>
> Jimmy
>
Re: Co-locate meta and master, and beyond
Posted by ramkrishna vasudevan <ra...@gmail.com>.
+1. Great Jimmy.
On Fri, Mar 21, 2014 at 3:37 AM, Elliott Clark <ec...@apache.org> wrote:
> +1 Awesome work.
>
>
> On Thu, Mar 20, 2014 at 9:08 AM, Jimmy Xiang <jx...@cloudera.com> wrote:
>
> > Here is a heads up on some work I've been doing of late.
> >
> >
> > My goal is to make region assignment simpler, faster, more reliable
> >
> > and maintainable making it so we are able to handle many more regions
> >
> > more efficiently (improve mttr, and lessen impact of compactions), and
> >
> > to eventually look into quorum-based master service.
> >
> > In HBASE-10569, I posted a patch that makes a HMaster also a
> > HRegionServer. I've also done work so that we can co-locate the meta
> > region with the master server. These changes are based on one of the
> > agreements per the discussion in HBASE-5487, the master re-design.
> >
> > We should be able to do a rolling restart on to this new basis (I'm
> >
> > working on it).
> >
> > Putting the master and regionserver processes together and making it
> >
> > so meta and master are also together is a step toward making it easier
> > doing
> > region assignment without using ZK, another agreement from HBASE-5487.
> >
> >
> > Thanks.
> >
> > Jimmy
> >
>
Re: Co-locate meta and master, and beyond
Posted by Elliott Clark <ec...@apache.org>.
+1 Awesome work.
On Thu, Mar 20, 2014 at 9:08 AM, Jimmy Xiang <jx...@cloudera.com> wrote:
> Here is a heads up on some work I've been doing of late.
>
>
> My goal is to make region assignment simpler, faster, more reliable
>
> and maintainable making it so we are able to handle many more regions
>
> more efficiently (improve mttr, and lessen impact of compactions), and
>
> to eventually look into quorum-based master service.
>
> In HBASE-10569, I posted a patch that makes a HMaster also a
> HRegionServer. I've also done work so that we can co-locate the meta
> region with the master server. These changes are based on one of the
> agreements per the discussion in HBASE-5487, the master re-design.
>
> We should be able to do a rolling restart on to this new basis (I'm
>
> working on it).
>
> Putting the master and regionserver processes together and making it
>
> so meta and master are also together is a step toward making it easier
> doing
> region assignment without using ZK, another agreement from HBASE-5487.
>
>
> Thanks.
>
> Jimmy
>
Re: Co-locate meta and master, and beyond
Posted by Andrew Purtell <ap...@apache.org>.
+1 awesome
On Thu, Mar 20, 2014 at 9:08 AM, Jimmy Xiang <jx...@cloudera.com> wrote:
> Here is a heads up on some work I've been doing of late.
>
>
> My goal is to make region assignment simpler, faster, more reliable
>
> and maintainable making it so we are able to handle many more regions
>
> more efficiently (improve mttr, and lessen impact of compactions), and
>
> to eventually look into quorum-based master service.
>
> In HBASE-10569, I posted a patch that makes a HMaster also a
> HRegionServer. I've also done work so that we can co-locate the meta
> region with the master server. These changes are based on one of the
> agreements per the discussion in HBASE-5487, the master re-design.
>
> We should be able to do a rolling restart on to this new basis (I'm
>
> working on it).
>
> Putting the master and regionserver processes together and making it
>
> so meta and master are also together is a step toward making it easier
> doing
> region assignment without using ZK, another agreement from HBASE-5487.
>
>
> Thanks.
>
> Jimmy
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)