You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Brady Zhong <br...@gmail.com> on 2013/04/08 16:39:20 UTC
About distributed lock service in HBase
Hi all,
My name is Brady Zhong, a college student using HBase to develop our own
project. Currently I confronted with a problem. Since we need some kind of
high availability, we hope HBase can keep available even though the HMaster
goes down. Here're my questions:
1. Can HBase work during the node failure of HMaster? Can users write or
read the database before the switch and recovery of HMaster?
2. Why not use Zookeeper as distributed lock service for HBase, like Chubby
for Google Big Table?
Thanks very much for any help in advance.
Best regards,
Brady Zhong
Re: About distributed lock service in HBase
Posted by Elliott Clark <ec...@apache.org>.
1. Yes the region servers can absolutely work while the master is down (for
a while). Master is mostly there for administrative clean up duties ie,
balancing, assigning and creating tables. Additionally there can be more
than one master process running at a time. There's a leader election that
takes place in zookeeper and when a master goes down any backup masters
will elect a new functioning master.
2. We do use zookeeper for locks.
2.1 Trunk and 0.95.0 all use zookeeper for locking table operations.
See HBASE-7305
as Ted suggests.
2.2 Row locks exposed to the user don't use zookeeper since only one region
server is ever responsible for a row. With that said row locks are being
removed in HBASE-7315 since they didn't provide the guarantees that some
users were expecting (locks didn't survive region move or server restart)
On Mon, Apr 8, 2013 at 9:22 AM, Ted Yu <yu...@gmail.com> wrote:
> Brady:
> Hopefully Nicolas answered your questions.
>
> We do use zookeeper for coordinating table schema changes. See the
> following JIRA:
> HBASE-7305. ZK based Read/Write locks for table operations
>
> Cheers
>
> On Mon, Apr 8, 2013 at 8:34 AM, Nicolas Liochon <nk...@gmail.com> wrote:
>
> > HBASE-5541 is about rowlock. The master plays no role in this. This lock
> > does not have to be distributed, because, in bigtable/hbase architecture,
> > the rows are allocated to a single region / region server. This makes
> > things faster.
> >
> > Nicolas
> >
> >
> > On Mon, Apr 8, 2013 at 5:27 PM, Brady Zhong <bradyzhong2018@gmail.com
> > >wrote:
> >
> > > Hi Nicolas,
> > >
> > > Thanks for your explanation. I know HBase uses ZeeKeeper as coordinated
> > > service but not a lock management service like Chubby. Like HBASE-5541
> > > states, it seems the RegionServer controls the whole write process
> > > including providing row lock, region lock etc.. So I wonder why we
> don't
> > > use high available service like ZooKeeper creating, managing and
> > releasing
> > > lock?
> > >
> > >
> > > Brady
> > >
> > > On Mon, Apr 8, 2013 at 11:13 PM, Nicolas Liochon <nk...@gmail.com>
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > 1. Yes. Client applications can start/read/write even when there is
> no
> > > > master.
> > > > 2. HBase already uses ZooKeeper.
> > > >
> > > > You may want to have to look at the hbase reference guide (
> > > > http://hbase.apache.org/book.html).
> > > >
> > > > Nicolas
> > > >
> > > >
> > > > On Mon, Apr 8, 2013 at 4:39 PM, Brady Zhong <
> bradyzhong2018@gmail.com
> > > > >wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > My name is Brady Zhong, a college student using HBase to develop
> our
> > > own
> > > > > project. Currently I confronted with a problem. Since we need some
> > kind
> > > > of
> > > > > high availability, we hope HBase can keep available even though the
> > > > HMaster
> > > > > goes down. Here're my questions:
> > > > >
> > > > > 1. Can HBase work during the node failure of HMaster? Can users
> write
> > > or
> > > > > read the database before the switch and recovery of HMaster?
> > > > > 2. Why not use Zookeeper as distributed lock service for HBase,
> like
> > > > Chubby
> > > > > for Google Big Table?
> > > > >
> > > > > Thanks very much for any help in advance.
> > > > >
> > > > >
> > > > > Best regards,
> > > > > Brady Zhong
> > > > >
> > > >
> > >
> >
>
Re: About distributed lock service in HBase
Posted by Ted Yu <yu...@gmail.com>.
Brady:
Hopefully Nicolas answered your questions.
We do use zookeeper for coordinating table schema changes. See the
following JIRA:
HBASE-7305. ZK based Read/Write locks for table operations
Cheers
On Mon, Apr 8, 2013 at 8:34 AM, Nicolas Liochon <nk...@gmail.com> wrote:
> HBASE-5541 is about rowlock. The master plays no role in this. This lock
> does not have to be distributed, because, in bigtable/hbase architecture,
> the rows are allocated to a single region / region server. This makes
> things faster.
>
> Nicolas
>
>
> On Mon, Apr 8, 2013 at 5:27 PM, Brady Zhong <bradyzhong2018@gmail.com
> >wrote:
>
> > Hi Nicolas,
> >
> > Thanks for your explanation. I know HBase uses ZeeKeeper as coordinated
> > service but not a lock management service like Chubby. Like HBASE-5541
> > states, it seems the RegionServer controls the whole write process
> > including providing row lock, region lock etc.. So I wonder why we don't
> > use high available service like ZooKeeper creating, managing and
> releasing
> > lock?
> >
> >
> > Brady
> >
> > On Mon, Apr 8, 2013 at 11:13 PM, Nicolas Liochon <nk...@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > 1. Yes. Client applications can start/read/write even when there is no
> > > master.
> > > 2. HBase already uses ZooKeeper.
> > >
> > > You may want to have to look at the hbase reference guide (
> > > http://hbase.apache.org/book.html).
> > >
> > > Nicolas
> > >
> > >
> > > On Mon, Apr 8, 2013 at 4:39 PM, Brady Zhong <bradyzhong2018@gmail.com
> > > >wrote:
> > >
> > > > Hi all,
> > > >
> > > > My name is Brady Zhong, a college student using HBase to develop our
> > own
> > > > project. Currently I confronted with a problem. Since we need some
> kind
> > > of
> > > > high availability, we hope HBase can keep available even though the
> > > HMaster
> > > > goes down. Here're my questions:
> > > >
> > > > 1. Can HBase work during the node failure of HMaster? Can users write
> > or
> > > > read the database before the switch and recovery of HMaster?
> > > > 2. Why not use Zookeeper as distributed lock service for HBase, like
> > > Chubby
> > > > for Google Big Table?
> > > >
> > > > Thanks very much for any help in advance.
> > > >
> > > >
> > > > Best regards,
> > > > Brady Zhong
> > > >
> > >
> >
>
Re: About distributed lock service in HBase
Posted by Nicolas Liochon <nk...@gmail.com>.
HBASE-5541 is about rowlock. The master plays no role in this. This lock
does not have to be distributed, because, in bigtable/hbase architecture,
the rows are allocated to a single region / region server. This makes
things faster.
Nicolas
On Mon, Apr 8, 2013 at 5:27 PM, Brady Zhong <br...@gmail.com>wrote:
> Hi Nicolas,
>
> Thanks for your explanation. I know HBase uses ZeeKeeper as coordinated
> service but not a lock management service like Chubby. Like HBASE-5541
> states, it seems the RegionServer controls the whole write process
> including providing row lock, region lock etc.. So I wonder why we don't
> use high available service like ZooKeeper creating, managing and releasing
> lock?
>
>
> Brady
>
> On Mon, Apr 8, 2013 at 11:13 PM, Nicolas Liochon <nk...@gmail.com>
> wrote:
>
> > Hi,
> >
> > 1. Yes. Client applications can start/read/write even when there is no
> > master.
> > 2. HBase already uses ZooKeeper.
> >
> > You may want to have to look at the hbase reference guide (
> > http://hbase.apache.org/book.html).
> >
> > Nicolas
> >
> >
> > On Mon, Apr 8, 2013 at 4:39 PM, Brady Zhong <bradyzhong2018@gmail.com
> > >wrote:
> >
> > > Hi all,
> > >
> > > My name is Brady Zhong, a college student using HBase to develop our
> own
> > > project. Currently I confronted with a problem. Since we need some kind
> > of
> > > high availability, we hope HBase can keep available even though the
> > HMaster
> > > goes down. Here're my questions:
> > >
> > > 1. Can HBase work during the node failure of HMaster? Can users write
> or
> > > read the database before the switch and recovery of HMaster?
> > > 2. Why not use Zookeeper as distributed lock service for HBase, like
> > Chubby
> > > for Google Big Table?
> > >
> > > Thanks very much for any help in advance.
> > >
> > >
> > > Best regards,
> > > Brady Zhong
> > >
> >
>
Re: About distributed lock service in HBase
Posted by Brady Zhong <br...@gmail.com>.
Hi Nicolas,
Thanks for your explanation. I know HBase uses ZeeKeeper as coordinated
service but not a lock management service like Chubby. Like HBASE-5541
states, it seems the RegionServer controls the whole write process
including providing row lock, region lock etc.. So I wonder why we don't
use high available service like ZooKeeper creating, managing and releasing
lock?
Brady
On Mon, Apr 8, 2013 at 11:13 PM, Nicolas Liochon <nk...@gmail.com> wrote:
> Hi,
>
> 1. Yes. Client applications can start/read/write even when there is no
> master.
> 2. HBase already uses ZooKeeper.
>
> You may want to have to look at the hbase reference guide (
> http://hbase.apache.org/book.html).
>
> Nicolas
>
>
> On Mon, Apr 8, 2013 at 4:39 PM, Brady Zhong <bradyzhong2018@gmail.com
> >wrote:
>
> > Hi all,
> >
> > My name is Brady Zhong, a college student using HBase to develop our own
> > project. Currently I confronted with a problem. Since we need some kind
> of
> > high availability, we hope HBase can keep available even though the
> HMaster
> > goes down. Here're my questions:
> >
> > 1. Can HBase work during the node failure of HMaster? Can users write or
> > read the database before the switch and recovery of HMaster?
> > 2. Why not use Zookeeper as distributed lock service for HBase, like
> Chubby
> > for Google Big Table?
> >
> > Thanks very much for any help in advance.
> >
> >
> > Best regards,
> > Brady Zhong
> >
>
Re: About distributed lock service in HBase
Posted by Nicolas Liochon <nk...@gmail.com>.
Hi,
1. Yes. Client applications can start/read/write even when there is no
master.
2. HBase already uses ZooKeeper.
You may want to have to look at the hbase reference guide (
http://hbase.apache.org/book.html).
Nicolas
On Mon, Apr 8, 2013 at 4:39 PM, Brady Zhong <br...@gmail.com>wrote:
> Hi all,
>
> My name is Brady Zhong, a college student using HBase to develop our own
> project. Currently I confronted with a problem. Since we need some kind of
> high availability, we hope HBase can keep available even though the HMaster
> goes down. Here're my questions:
>
> 1. Can HBase work during the node failure of HMaster? Can users write or
> read the database before the switch and recovery of HMaster?
> 2. Why not use Zookeeper as distributed lock service for HBase, like Chubby
> for Google Big Table?
>
> Thanks very much for any help in advance.
>
>
> Best regards,
> Brady Zhong
>