You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by jingych <ji...@neusoft.com> on 2013/11/13 03:49:07 UTC
HBaseAdmin#checkHBaseAvailable COST ABOUT 1 MINUTE TO CHECK A DEAD(OR NOT EXISTS) HBASE MASTER
HI,
I wonder is there any way to limit the "HBaseAdmin#checkHBaseAvailable" method time cost.
As i use the "HBaseAdmin#checkHBaseAvailable" method to test if the hbase master is connectable.
But if the target master is dead or not exists at all, this method will cost 1 minute to wait the result.
jingych
2013-11-13
---------------------------------------------------------------------------------------------------
Confidentiality Notice: The information contained in this e-mail and any accompanying attachment(s)
is intended only for the use of the intended recipient and may be confidential and/or privileged of
Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader of this communication is
not the intended recipient, unauthorized use, forwarding, printing, storing, disclosure or copying
is strictly prohibited, and may be unlawful.If you have received this communication in error,please
immediately notify the sender by return e-mail, and delete the original message and all copies from
your system. Thank you.
---------------------------------------------------------------------------------------------------
Re: Re: HBaseAdmin#checkHBaseAvailable COST ABOUT 1 MINUTE TO CHECK A
DEAD(OR NOT EXISTS) HBASE MASTER
Posted by Esteban Gutierrez <es...@cloudera.com>.
jingych,
inline:
On Wed, Nov 13, 2013 at 7:06 PM, jingych <ji...@neusoft.com> wrote:
> Thanks, Esteban and Stack!
>
> As Esteban said, the problem was solved.
>
> My config is below:
> <code>
> conf.setInt("hbase.client.retries.number", 1);
> conf.setInt("zookeeper.session.timeout", 5000);
> conf.setInt("zookeeper.recovery.retry", 1);
> conf.setInt("zookeeper.recovery.retry.intervalmill", 50);
> </code>
> But it still cost 46 seconds.
> And the log printing:
> <log>
>
> org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
> org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master
>
> org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/root-region-server
>
> org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/root-region-server
>
> </log>
> It still tried to build the 4 above connections.
>
The client (via HConnectionManager) needs to set 3 watchers on each of
those znodes in ZK, each attempt will have a max timeout of 5 seconds (you
have a single zk server) plus 10 seconds of the second attempt: 3 * (5 *
2^0) + 3 * (5 * 2^1) = 45 and the extra second should come from a hardcoded
sleep in the RPC implementation during a retry.
Setting zookeeper.recovery.retry=0 can make it fail faster but in case of a
transient failure then you will have to handle the reconnection in your
code.
>
> Could you please explain why the ZK do this? ( I'm realy new to the HBase
> world.)
> If i set the ZK session timeout with 1s, is't OK?
>
you *could* but you don't want clients to overwhelm ZK by re-establishing
connections over and over.
> And what do you mean about "depending on the number of ZK servers you have
> running the socket level timeout in the client to a ZK server will be
> zookeeper.session.timeout/#ZKs"?
> It means that if i hava 3 zookeepers and zookeeper.session.timeout=5000,
> each connection will 5000/3 timeout?
>
thats correct, the timeout to establish a connection to ZK will be around
1.6 seconds (5000 milliseconds / 3) with 3 ZKs.
> I'm running ZK and HBase Master at one node as pseudo-distributed mode.
>
> Best Regards!
>
> ------------------------------
>
> jingych
>
> 2013-11-14
>
> *发件人：* Esteban Gutierrez <es...@cloudera.com>
> *发送时间：* 2013-11-14 06:10
> *收件人：* Stack <st...@duboce.net>
> *抄送：* Hbase-User <us...@hbase.apache.org>; jingych <ji...@neusoft.com>
> *主题：* Re: Re: HBaseAdmin#checkHBaseAvailable COST ABOUT 1 MINUTE TO CHECK
> A DEAD(OR NOT EXISTS) HBASE MASTER
>
> jingych,
>
> That timeout comes from ZooKeeper, are you running ZK on the same node you
> are running the HBase Master? If your environment requires to fail fast
> even for ZK connection timeouts then you need to reduce
> zookeeper.recovery.retry.intervalmill and zookeeper.recovery.retry since
> the retries are done via an exponential backoff (1 second, 2 seconds, 8
> seconds), also depending on the number of ZK servers you have running the
> socket level timeout in the client to a ZK server will be
> zookeeper.session.timeout/#ZKs
>
> cheers,
> esteban.
>
>
>
>
>
>
> --
> Cloudera, Inc.
>
>
>
> On Wed, Nov 13, 2013 at 7:21 AM, Stack <st...@duboce.net> wrote:
>
>> More of the log and the version of HBase involved please. Thanks.
>> St.Ack
>>
>>
>> On Wed, Nov 13, 2013 at 1:07 AM, jingych <ji...@neusoft.com> wrote:
>>
>>> Thanks, esteban!
>>>
>>> I'v tried. But it did not work.
>>>
>>> I first load the customer hbase-site.xml, and then try to check the
>>> hbase server.
>>> So my code is like this:
>>> <code>
>>> conf.setInt("hbase.client.retries.number", 1);
>>> conf.setInt("hbase.client.pause", 5);
>>> conf.setInt("ipc.socket.timeout", 5000);
>>> conf.setInt("hbase.rpc.timeout", 5000);
>>> </code>
>>>
>>> The log printing: Sleeping 4000ms before retry #2...
>>>
>>> If the zookeeper's quarum is the wrong address, the process will take
>>> very long time.
>>>
>>>
>>>
>>>
>>> 井玉成
>>>
>>> 基础软件事业部
>>> 东软集团股份有限公司
>>> 手机：13889491801
>>> 电话：0411-84835702
>>>
>>> 大连市甘井子区黄浦路901号 D1座217室
>>> Postcode：116085
>>> Email：jingych@neusoft.com
>>>
>>> From: Esteban Gutierrez
>>> Date: 2013-11-13 11:12
>>> To: user@hbase.apache.org; jingych
>>> Subject: Re: HBaseAdmin#checkHBaseAvailable COST ABOUT 1 MINUTE TO CHECK
>>> A DEAD(OR NOT EXISTS) HBASE MASTER
>>> jingych,
>>>
>>> The behavior is driven by the number of retries
>>> (hbase.client.retries.number), the length of the pause between retries
>>> (hbase.client.pause) and the timeout to establish a connection
>>> (ipc.socket.timeout) and the time to get some data back from HBase
>>> (hbase.rpc.timeout). Lowering the rpc timeout and the ipc socket timeout
>>> should help you to fail fast the operation when the HBase Master is not
>>> responsive.
>>>
>>> cheers,
>>> esteban.
>>>
>>>
>>>
>>>
>>> --
>>> Cloudera, Inc.
>>>
>>>
>>>
>>> On Tue, Nov 12, 2013 at 6:49 PM, jingych <ji...@neusoft.com> wrote:
>>>
>>> > HI,
>>> >
>>> > I wonder is there any way to limit the "HBaseAdmin#checkHBaseAvailable"
>>> > method time cost.
>>> >
>>> > As i use the "HBaseAdmin#checkHBaseAvailable" method to test if the
>>> hbase
>>> > master is connectable.
>>> > But if the target master is dead or not exists at all, this method will
>>> > cost 1 minute to wait the result.
>>> >
>>> >
>>> >
>>> >
>>> > jingych
>>> > 2013-11-13
>>> >
>>> >
>>> ---------------------------------------------------------------------------------------------------
>>> > Confidentiality Notice: The information contained in this e-mail and
>>> any
>>> > accompanying attachment(s)
>>> > is intended only for the use of the intended recipient and may be
>>> > confidential and/or privileged of
>>> > Neusoft Corporation, its subsidiaries and/or its affiliates. If any
>>> reader
>>> > of this communication is
>>> > not the intended recipient, unauthorized use, forwarding, printing,
>>> > storing, disclosure or copying
>>> > is strictly prohibited, and may be unlawful.If you have received this
>>> > communication in error,please
>>> > immediately notify the sender by return e-mail, and delete the original
>>> > message and all copies from
>>> > your system. Thank you.
>>> >
>>> >
>>> ---------------------------------------------------------------------------------------------------
>>> >
>>>
>>> ---------------------------------------------------------------------------------------------------
>>> Confidentiality Notice: The information contained in this e-mail and any
>>> accompanying attachment(s)
>>> is intended only for the use of the intended recipient and may be
>>> confidential and/or privileged of
>>> Neusoft Corporation, its subsidiaries and/or its affiliates. If any
>>> reader of this communication is
>>> not the intended recipient, unauthorized use, forwarding, printing,
>>> storing, disclosure or copying
>>> is strictly prohibited, and may be unlawful.If you have received this
>>> communication in error,please
>>> immediately notify the sender by return e-mail, and delete the original
>>> message and all copies from
>>> your system. Thank you.
>>>
>>> ---------------------------------------------------------------------------------------------------
>>>
>>
>>
>
> ---------------------------------------------------------------------------------------------------
> Confidentiality Notice: The information contained in this e-mail and any
> accompanying attachment(s)
> is intended only for the use of the intended recipient and may be
> confidential and/or privileged of
> Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader
> of this communication is
> not the intended recipient, unauthorized use, forwarding, printing,
> storing, disclosure or copying
> is strictly prohibited, and may be unlawful.If you have received this
> communication in error,please
> immediately notify the sender by return e-mail, and delete the original
> message and all copies from
> your system. Thank you.
>
> ---------------------------------------------------------------------------------------------------
>
回复: Re: HBaseAdmin#checkHBaseAvailable COST ABOUT 1 MINUTE TO CHECK A DEAD(OR NOT EXISTS) HBASE MASTER
Posted by jingych <ji...@neusoft.com>.
Thanks, Esteban and Stack!
As Esteban said, the problem was solved.
My config is below:
<code>
conf.setInt("hbase.client.retries.number", 1);
conf.setInt("zookeeper.session.timeout", 5000);
conf.setInt("zookeeper.recovery.retry", 1);
conf.setInt("zookeeper.recovery.retry.intervalmill", 50);
</code>
But it still cost 46 seconds.
And the log printing:
<log>
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/root-region-server
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/root-region-server
</log>
It still tried to build the 4 above connections.
Could you please explain why the ZK do this? ( I'm realy new to the HBase world.)
If i set the ZK session timeout with 1s, is't OK?
And what do you mean about "depending on the number of ZK servers you have running the socket level timeout in the client to a ZK server will be zookeeper.session.timeout/#ZKs"?
It means that if i hava 3 zookeepers and zookeeper.session.timeout=5000, each connection will 5000/3 timeout?
I'm running ZK and HBase Master at one node as pseudo-distributed mode.
Best Regards!
jingych
2013-11-14
发件人： Esteban Gutierrez
发送时间： 2013-11-14 06:10
收件人： Stack
抄送： Hbase-User; jingych
主题： Re: Re: HBaseAdmin#checkHBaseAvailable COST ABOUT 1 MINUTE TO CHECK A DEAD(OR NOT EXISTS) HBASE MASTER
jingych,
That timeout comes from ZooKeeper, are you running ZK on the same node you are running the HBase Master? If your environment requires to fail fast even for ZK connection timeouts then you need to reduce zookeeper.recovery.retry.intervalmill and zookeeper.recovery.retry since the retries are done via an exponential backoff (1 second, 2 seconds, 8 seconds), also depending on the number of ZK servers you have running the socket level timeout in the client to a ZK server will be zookeeper.session.timeout/#ZKs
cheers,
esteban.
--
Cloudera, Inc.
On Wed, Nov 13, 2013 at 7:21 AM, Stack <st...@duboce.net> wrote:
More of the log and the version of HBase involved please. Thanks.
St.Ack
On Wed, Nov 13, 2013 at 1:07 AM, jingych <ji...@neusoft.com> wrote:
Thanks, esteban!
I'v tried. But it did not work.
I first load the customer hbase-site.xml, and then try to check the hbase server.
So my code is like this:
<code>
conf.setInt("hbase.client.retries.number", 1);
conf.setInt("hbase.client.pause", 5);
conf.setInt("ipc.socket.timeout", 5000);
conf.setInt("hbase.rpc.timeout", 5000);
</code>
The log printing: Sleeping 4000ms before retry #2...
If the zookeeper's quarum is the wrong address, the process will take very long time.
井玉成
基础软件事业部
东软集团股份有限公司
手机：13889491801
电话：0411-84835702
大连市甘井子区黄浦路901号 D1座217室
Postcode：116085
Email：jingych@neusoft.com
From: Esteban Gutierrez
Date: 2013-11-13 11:12
To: user@hbase.apache.org; jingych
Subject: Re: HBaseAdmin#checkHBaseAvailable COST ABOUT 1 MINUTE TO CHECK A DEAD(OR NOT EXISTS) HBASE MASTER
jingych,
The behavior is driven by the number of retries
(hbase.client.retries.number), the length of the pause between retries
(hbase.client.pause) and the timeout to establish a connection
(ipc.socket.timeout) and the time to get some data back from HBase
(hbase.rpc.timeout). Lowering the rpc timeout and the ipc socket timeout
should help you to fail fast the operation when the HBase Master is not
responsive.
cheers,
esteban.
--
Cloudera, Inc.
On Tue, Nov 12, 2013 at 6:49 PM, jingych <ji...@neusoft.com> wrote:
> HI,
>
> I wonder is there any way to limit the "HBaseAdmin#checkHBaseAvailable"
> method time cost.
>
> As i use the "HBaseAdmin#checkHBaseAvailable" method to test if the hbase
> master is connectable.
> But if the target master is dead or not exists at all, this method will
> cost 1 minute to wait the result.
>
>
>
>
> jingych
> 2013-11-13
>
> ---------------------------------------------------------------------------------------------------
> Confidentiality Notice: The information contained in this e-mail and any
> accompanying attachment(s)
> is intended only for the use of the intended recipient and may be
> confidential and/or privileged of
> Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader
> of this communication is
> not the intended recipient, unauthorized use, forwarding, printing,
> storing, disclosure or copying
> is strictly prohibited, and may be unlawful.If you have received this
> communication in error,please
> immediately notify the sender by return e-mail, and delete the original
> message and all copies from
> your system. Thank you.
>
> ---------------------------------------------------------------------------------------------------
>
---------------------------------------------------------------------------------------------------
Confidentiality Notice: The information contained in this e-mail and any accompanying attachment(s)
is intended only for the use of the intended recipient and may be confidential and/or privileged of
Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader of this communication is
not the intended recipient, unauthorized use, forwarding, printing, storing, disclosure or copying
is strictly prohibited, and may be unlawful.If you have received this communication in error,please
immediately notify the sender by return e-mail, and delete the original message and all copies from
your system. Thank you.
---------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------
Confidentiality Notice: The information contained in this e-mail and any accompanying attachment(s)
is intended only for the use of the intended recipient and may be confidential and/or privileged of
Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader of this communication is
not the intended recipient, unauthorized use, forwarding, printing, storing, disclosure or copying
is strictly prohibited, and may be unlawful.If you have received this communication in error,please
immediately notify the sender by return e-mail, and delete the original message and all copies from
your system. Thank you.
---------------------------------------------------------------------------------------------------
Re: Re: HBaseAdmin#checkHBaseAvailable COST ABOUT 1 MINUTE TO CHECK A
DEAD(OR NOT EXISTS) HBASE MASTER
Posted by Esteban Gutierrez <es...@cloudera.com>.
jingych,
That timeout comes from ZooKeeper, are you running ZK on the same node you
are running the HBase Master? If your environment requires to fail fast
even for ZK connection timeouts then you need to reduce
zookeeper.recovery.retry.intervalmill and zookeeper.recovery.retry since
the retries are done via an exponential backoff (1 second, 2 seconds, 8
seconds), also depending on the number of ZK servers you have running the
socket level timeout in the client to a ZK server will be
zookeeper.session.timeout/#ZKs
cheers,
esteban.
--
Cloudera, Inc.
On Wed, Nov 13, 2013 at 7:21 AM, Stack <st...@duboce.net> wrote:
> More of the log and the version of HBase involved please. Thanks.
> St.Ack
>
>
> On Wed, Nov 13, 2013 at 1:07 AM, jingych <ji...@neusoft.com> wrote:
>
>> Thanks, esteban!
>>
>> I'v tried. But it did not work.
>>
>> I first load the customer hbase-site.xml, and then try to check the hbase
>> server.
>> So my code is like this:
>> <code>
>> conf.setInt("hbase.client.retries.number", 1);
>> conf.setInt("hbase.client.pause", 5);
>> conf.setInt("ipc.socket.timeout", 5000);
>> conf.setInt("hbase.rpc.timeout", 5000);
>> </code>
>>
>> The log printing: Sleeping 4000ms before retry #2...
>>
>> If the zookeeper's quarum is the wrong address, the process will take
>> very long time.
>>
>>
>>
>>
>> 井玉成
>>
>> 基础软件事业部
>> 东软集团股份有限公司
>> 手机：13889491801
>> 电话：0411-84835702
>>
>> 大连市甘井子区黄浦路901号 D1座217室
>> Postcode：116085
>> Email：jingych@neusoft.com
>>
>> From: Esteban Gutierrez
>> Date: 2013-11-13 11:12
>> To: user@hbase.apache.org; jingych
>> Subject: Re: HBaseAdmin#checkHBaseAvailable COST ABOUT 1 MINUTE TO CHECK
>> A DEAD(OR NOT EXISTS) HBASE MASTER
>> jingych,
>>
>> The behavior is driven by the number of retries
>> (hbase.client.retries.number), the length of the pause between retries
>> (hbase.client.pause) and the timeout to establish a connection
>> (ipc.socket.timeout) and the time to get some data back from HBase
>> (hbase.rpc.timeout). Lowering the rpc timeout and the ipc socket timeout
>> should help you to fail fast the operation when the HBase Master is not
>> responsive.
>>
>> cheers,
>> esteban.
>>
>>
>>
>>
>> --
>> Cloudera, Inc.
>>
>>
>>
>> On Tue, Nov 12, 2013 at 6:49 PM, jingych <ji...@neusoft.com> wrote:
>>
>> > HI,
>> >
>> > I wonder is there any way to limit the "HBaseAdmin#checkHBaseAvailable"
>> > method time cost.
>> >
>> > As i use the "HBaseAdmin#checkHBaseAvailable" method to test if the
>> hbase
>> > master is connectable.
>> > But if the target master is dead or not exists at all, this method will
>> > cost 1 minute to wait the result.
>> >
>> >
>> >
>> >
>> > jingych
>> > 2013-11-13
>> >
>> >
>> ---------------------------------------------------------------------------------------------------
>> > Confidentiality Notice: The information contained in this e-mail and any
>> > accompanying attachment(s)
>> > is intended only for the use of the intended recipient and may be
>> > confidential and/or privileged of
>> > Neusoft Corporation, its subsidiaries and/or its affiliates. If any
>> reader
>> > of this communication is
>> > not the intended recipient, unauthorized use, forwarding, printing,
>> > storing, disclosure or copying
>> > is strictly prohibited, and may be unlawful.If you have received this
>> > communication in error,please
>> > immediately notify the sender by return e-mail, and delete the original
>> > message and all copies from
>> > your system. Thank you.
>> >
>> >
>> ---------------------------------------------------------------------------------------------------
>> >
>>
>> ---------------------------------------------------------------------------------------------------
>> Confidentiality Notice: The information contained in this e-mail and any
>> accompanying attachment(s)
>> is intended only for the use of the intended recipient and may be
>> confidential and/or privileged of
>> Neusoft Corporation, its subsidiaries and/or its affiliates. If any
>> reader of this communication is
>> not the intended recipient, unauthorized use, forwarding, printing,
>> storing, disclosure or copying
>> is strictly prohibited, and may be unlawful.If you have received this
>> communication in error,please
>> immediately notify the sender by return e-mail, and delete the original
>> message and all copies from
>> your system. Thank you.
>>
>> ---------------------------------------------------------------------------------------------------
>>
>
>
Re: Re: HBaseAdmin#checkHBaseAvailable COST ABOUT 1 MINUTE TO CHECK A
DEAD(OR NOT EXISTS) HBASE MASTER
Posted by Stack <st...@duboce.net>.
More of the log and the version of HBase involved please. Thanks.
St.Ack
On Wed, Nov 13, 2013 at 1:07 AM, jingych <ji...@neusoft.com> wrote:
> Thanks, esteban!
>
> I'v tried. But it did not work.
>
> I first load the customer hbase-site.xml, and then try to check the hbase
> server.
> So my code is like this:
> <code>
> conf.setInt("hbase.client.retries.number", 1);
> conf.setInt("hbase.client.pause", 5);
> conf.setInt("ipc.socket.timeout", 5000);
> conf.setInt("hbase.rpc.timeout", 5000);
> </code>
>
> The log printing: Sleeping 4000ms before retry #2...
>
> If the zookeeper's quarum is the wrong address, the process will take very
> long time.
>
>
>
>
> 井玉成
>
> 基础软件事业部
> 东软集团股份有限公司
> 手机：13889491801
> 电话：0411-84835702
>
> 大连市甘井子区黄浦路901号 D1座217室
> Postcode：116085
> Email：jingych@neusoft.com
>
> From: Esteban Gutierrez
> Date: 2013-11-13 11:12
> To: user@hbase.apache.org; jingych
> Subject: Re: HBaseAdmin#checkHBaseAvailable COST ABOUT 1 MINUTE TO CHECK A
> DEAD(OR NOT EXISTS) HBASE MASTER
> jingych,
>
> The behavior is driven by the number of retries
> (hbase.client.retries.number), the length of the pause between retries
> (hbase.client.pause) and the timeout to establish a connection
> (ipc.socket.timeout) and the time to get some data back from HBase
> (hbase.rpc.timeout). Lowering the rpc timeout and the ipc socket timeout
> should help you to fail fast the operation when the HBase Master is not
> responsive.
>
> cheers,
> esteban.
>
>
>
>
> --
> Cloudera, Inc.
>
>
>
> On Tue, Nov 12, 2013 at 6:49 PM, jingych <ji...@neusoft.com> wrote:
>
> > HI,
> >
> > I wonder is there any way to limit the "HBaseAdmin#checkHBaseAvailable"
> > method time cost.
> >
> > As i use the "HBaseAdmin#checkHBaseAvailable" method to test if the hbase
> > master is connectable.
> > But if the target master is dead or not exists at all, this method will
> > cost 1 minute to wait the result.
> >
> >
> >
> >
> > jingych
> > 2013-11-13
> >
> >
> ---------------------------------------------------------------------------------------------------
> > Confidentiality Notice: The information contained in this e-mail and any
> > accompanying attachment(s)
> > is intended only for the use of the intended recipient and may be
> > confidential and/or privileged of
> > Neusoft Corporation, its subsidiaries and/or its affiliates. If any
> reader
> > of this communication is
> > not the intended recipient, unauthorized use, forwarding, printing,
> > storing, disclosure or copying
> > is strictly prohibited, and may be unlawful.If you have received this
> > communication in error,please
> > immediately notify the sender by return e-mail, and delete the original
> > message and all copies from
> > your system. Thank you.
> >
> >
> ---------------------------------------------------------------------------------------------------
> >
>
> ---------------------------------------------------------------------------------------------------
> Confidentiality Notice: The information contained in this e-mail and any
> accompanying attachment(s)
> is intended only for the use of the intended recipient and may be
> confidential and/or privileged of
> Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader
> of this communication is
> not the intended recipient, unauthorized use, forwarding, printing,
> storing, disclosure or copying
> is strictly prohibited, and may be unlawful.If you have received this
> communication in error,please
> immediately notify the sender by return e-mail, and delete the original
> message and all copies from
> your system. Thank you.
>
> ---------------------------------------------------------------------------------------------------
>
Re: Re: HBaseAdmin#checkHBaseAvailable COST ABOUT 1 MINUTE TO CHECK A DEAD(OR NOT EXISTS) HBASE MASTER
Posted by jingych <ji...@neusoft.com>.
Thanks, esteban!
I'v tried. But it did not work.
I first load the customer hbase-site.xml, and then try to check the hbase server.
So my code is like this:
<code>
conf.setInt("hbase.client.retries.number", 1);
conf.setInt("hbase.client.pause", 5);
conf.setInt("ipc.socket.timeout", 5000);
conf.setInt("hbase.rpc.timeout", 5000);
</code>
The log printing: Sleeping 4000ms before retry #2...
If the zookeeper's quarum is the wrong address, the process will take very long time.
井玉成
基础软件事业部
东软集团股份有限公司
手机：13889491801
电话：0411-84835702
大连市甘井子区黄浦路901号 D1座217室
Postcode：116085
Email：jingych@neusoft.com
From: Esteban Gutierrez
Date: 2013-11-13 11:12
To: user@hbase.apache.org; jingych
Subject: Re: HBaseAdmin#checkHBaseAvailable COST ABOUT 1 MINUTE TO CHECK A DEAD(OR NOT EXISTS) HBASE MASTER
jingych,
The behavior is driven by the number of retries
(hbase.client.retries.number), the length of the pause between retries
(hbase.client.pause) and the timeout to establish a connection
(ipc.socket.timeout) and the time to get some data back from HBase
(hbase.rpc.timeout). Lowering the rpc timeout and the ipc socket timeout
should help you to fail fast the operation when the HBase Master is not
responsive.
cheers,
esteban.
--
Cloudera, Inc.
On Tue, Nov 12, 2013 at 6:49 PM, jingych <ji...@neusoft.com> wrote:
> HI,
>
> I wonder is there any way to limit the "HBaseAdmin#checkHBaseAvailable"
> method time cost.
>
> As i use the "HBaseAdmin#checkHBaseAvailable" method to test if the hbase
> master is connectable.
> But if the target master is dead or not exists at all, this method will
> cost 1 minute to wait the result.
>
>
>
>
> jingych
> 2013-11-13
>
> ---------------------------------------------------------------------------------------------------
> Confidentiality Notice: The information contained in this e-mail and any
> accompanying attachment(s)
> is intended only for the use of the intended recipient and may be
> confidential and/or privileged of
> Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader
> of this communication is
> not the intended recipient, unauthorized use, forwarding, printing,
> storing, disclosure or copying
> is strictly prohibited, and may be unlawful.If you have received this
> communication in error,please
> immediately notify the sender by return e-mail, and delete the original
> message and all copies from
> your system. Thank you.
>
> ---------------------------------------------------------------------------------------------------
>
---------------------------------------------------------------------------------------------------
Confidentiality Notice: The information contained in this e-mail and any accompanying attachment(s)
is intended only for the use of the intended recipient and may be confidential and/or privileged of
Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader of this communication is
not the intended recipient, unauthorized use, forwarding, printing, storing, disclosure or copying
is strictly prohibited, and may be unlawful.If you have received this communication in error,please
immediately notify the sender by return e-mail, and delete the original message and all copies from
your system. Thank you.
---------------------------------------------------------------------------------------------------
Re: HBaseAdmin#checkHBaseAvailable COST ABOUT 1 MINUTE TO CHECK A
DEAD(OR NOT EXISTS) HBASE MASTER
Posted by Esteban Gutierrez <es...@cloudera.com>.
jingych,
The behavior is driven by the number of retries
(hbase.client.retries.number), the length of the pause between retries
(hbase.client.pause) and the timeout to establish a connection
(ipc.socket.timeout) and the time to get some data back from HBase
(hbase.rpc.timeout). Lowering the rpc timeout and the ipc socket timeout
should help you to fail fast the operation when the HBase Master is not
responsive.
cheers,
esteban.
--
Cloudera, Inc.
On Tue, Nov 12, 2013 at 6:49 PM, jingych <ji...@neusoft.com> wrote:
> HI,
>
> I wonder is there any way to limit the "HBaseAdmin#checkHBaseAvailable"
> method time cost.
>
> As i use the "HBaseAdmin#checkHBaseAvailable" method to test if the hbase
> master is connectable.
> But if the target master is dead or not exists at all, this method will
> cost 1 minute to wait the result.
>
>
>
>
> jingych
> 2013-11-13
>
> ---------------------------------------------------------------------------------------------------
> Confidentiality Notice: The information contained in this e-mail and any
> accompanying attachment(s)
> is intended only for the use of the intended recipient and may be
> confidential and/or privileged of
> Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader
> of this communication is
> not the intended recipient, unauthorized use, forwarding, printing,
> storing, disclosure or copying
> is strictly prohibited, and may be unlawful.If you have received this
> communication in error,please
> immediately notify the sender by return e-mail, and delete the original
> message and all copies from
> your system. Thank you.
>
> ---------------------------------------------------------------------------------------------------
>