You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Steve Boyle <st...@connexity.com> on 2011/12/16 00:15:00 UTC
understanding hbase client timeout settings
Hi,
I'm trying to understand what timeout controls are available in the hbase client. I'm using hbase version 0.90.4-cdh3u2. I have a client application that does gets, puts, increments and scans. I'd like to be able to have a client side timeout such that the client can clean-up in a case where the response is taking a while for whatever reason.
I've seen this jira which indicates that the feature I'm looking for is coming in a future release:
https://issues.apache.org/jira/browse/HBASE-2937
I've also seen this jira that looks related and appears to be already available:
https://issues.apache.org/jira/browse/HBASE-3154
What is the best way to control a timeout for the hbase client?
Thanks,
Steve
RE: understanding hbase client timeout settings
Posted by Steve Boyle <st...@connexity.com>.
Got it. Thank you.
-----Original Message-----
From: Shrijeet Paliwal [mailto:shrijeet@rocketfuel.com]
Sent: Thursday, December 15, 2011 9:36 PM
To: user@hbase.apache.org
Subject: Re: understanding hbase client timeout settings
Steve,
Karthick has given an explanation here :
https://reviews.apache.org/r/755/(also in Jira, but it gets lost in
comments)
On Thu, Dec 15, 2011 at 9:34 PM, Shrijeet Paliwal
<sh...@rocketfuel.com>wrote:
> We needed calls to come back (or timeout) in less than 50ms. That was
> low for hbase.rpc.timeout .
>
>
> On Thu, Dec 15, 2011 at 9:30 PM, Steve Boyle <st...@connexity.com>wrote:
>
>> Shrijeet,
>>
>> What do you consider a very low value for hbase.rpc.timeout?
>>
>> Thanks,
>> Steve
>>
>> -----Original Message-----
>> From: Shrijeet Paliwal [mailto:shrijeet@rocketfuel.com]
>> Sent: Thursday, December 15, 2011 9:03 PM
>> To: user@hbase.apache.org
>> Subject: Re: understanding hbase client timeout settings
>>
>> Steve,
>>
>> We have been using timeouts in production via two different methods:
>>
>> 1. Use RPC timeout mechanism provided by HBase client. For this you
>> will need both 2937 and 3154. We backported 2937 to 0.90.3. The
>> reason 3154 alone does not help is because if you set the conf
>> parameter introduced in
>> 3154 (hbase.rpc.timeout) to a very low value your calls to meta
>> (ROOT and
>> META) tables timeout all the time and almost no request go through.
>> That is where 2937 comes in. It uses hbase.client.operation.timeout
>> knob for all RPC calls except the ones made to meta tables.
>>
>> 2. Use an async method (example ExecutorService with timeout) to make
>> HBase client calls.
>>
>> In our experience #1 perform better (in terms of % calls timing out).
>>
>> Having said that I am still interested to hear comments from experts.
>> I personally was not able to grasp reasoning behind introduction of
>> hbase.client.operation.timeout configuration parameter.
>>
>> -Shrijeet
>>
>> On Thu, Dec 15, 2011 at 3:15 PM, Steve Boyle
>> <st...@connexity.com>
>> wrote:
>> >
>> > Hi,
>> >
>> > I'm trying to understand what timeout controls are available in the
>> hbase
>> client. I'm using hbase version 0.90.4-cdh3u2. I have a client
>> application that does gets, puts, increments and scans. I'd like to
>> be able to have a client side timeout such that the client can
>> clean-up in a case where the response is taking a while for whatever reason.
>> >
>> > I've seen this jira which indicates that the feature I'm looking
>> > for is
>> coming in a future release:
>> > https://issues.apache.org/jira/browse/HBASE-2937
>> >
>> > I've also seen this jira that looks related and appears to be
>> > already
>> available:
>> > https://issues.apache.org/jira/browse/HBASE-3154
>> >
>> > What is the best way to control a timeout for the hbase client?
>> >
>> > Thanks,
>> > Steve
>> >
>> >
>> >
>>
>
>
Re: understanding hbase client timeout settings
Posted by Shrijeet Paliwal <sh...@rocketfuel.com>.
Steve,
Karthick has given an explanation here :
https://reviews.apache.org/r/755/(also in Jira, but it gets lost in
comments)
On Thu, Dec 15, 2011 at 9:34 PM, Shrijeet Paliwal
<sh...@rocketfuel.com>wrote:
> We needed calls to come back (or timeout) in less than 50ms. That was low
> for hbase.rpc.timeout .
>
>
> On Thu, Dec 15, 2011 at 9:30 PM, Steve Boyle <st...@connexity.com>wrote:
>
>> Shrijeet,
>>
>> What do you consider a very low value for hbase.rpc.timeout?
>>
>> Thanks,
>> Steve
>>
>> -----Original Message-----
>> From: Shrijeet Paliwal [mailto:shrijeet@rocketfuel.com]
>> Sent: Thursday, December 15, 2011 9:03 PM
>> To: user@hbase.apache.org
>> Subject: Re: understanding hbase client timeout settings
>>
>> Steve,
>>
>> We have been using timeouts in production via two different methods:
>>
>> 1. Use RPC timeout mechanism provided by HBase client. For this you will
>> need both 2937 and 3154. We backported 2937 to 0.90.3. The reason 3154
>> alone does not help is because if you set the conf parameter introduced in
>> 3154 (hbase.rpc.timeout) to a very low value your calls to meta (ROOT and
>> META) tables timeout all the time and almost no request go through. That
>> is
>> where 2937 comes in. It uses hbase.client.operation.timeout knob for all
>> RPC calls except the ones made to meta tables.
>>
>> 2. Use an async method (example ExecutorService with timeout) to make
>> HBase
>> client calls.
>>
>> In our experience #1 perform better (in terms of % calls timing out).
>>
>> Having said that I am still interested to hear comments from experts. I
>> personally was not able to grasp reasoning behind introduction of
>> hbase.client.operation.timeout configuration parameter.
>>
>> -Shrijeet
>>
>> On Thu, Dec 15, 2011 at 3:15 PM, Steve Boyle <st...@connexity.com>
>> wrote:
>> >
>> > Hi,
>> >
>> > I'm trying to understand what timeout controls are available in the
>> hbase
>> client. I'm using hbase version 0.90.4-cdh3u2. I have a client
>> application that does gets, puts, increments and scans. I'd like to be
>> able to have a client side timeout such that the client can clean-up in a
>> case where the response is taking a while for whatever reason.
>> >
>> > I've seen this jira which indicates that the feature I'm looking for is
>> coming in a future release:
>> > https://issues.apache.org/jira/browse/HBASE-2937
>> >
>> > I've also seen this jira that looks related and appears to be already
>> available:
>> > https://issues.apache.org/jira/browse/HBASE-3154
>> >
>> > What is the best way to control a timeout for the hbase client?
>> >
>> > Thanks,
>> > Steve
>> >
>> >
>> >
>>
>
>
Re: understanding hbase client timeout settings
Posted by Shrijeet Paliwal <sh...@rocketfuel.com>.
We needed calls to come back (or timeout) in less than 50ms. That was low
for hbase.rpc.timeout .
On Thu, Dec 15, 2011 at 9:30 PM, Steve Boyle <st...@connexity.com>wrote:
> Shrijeet,
>
> What do you consider a very low value for hbase.rpc.timeout?
>
> Thanks,
> Steve
>
> -----Original Message-----
> From: Shrijeet Paliwal [mailto:shrijeet@rocketfuel.com]
> Sent: Thursday, December 15, 2011 9:03 PM
> To: user@hbase.apache.org
> Subject: Re: understanding hbase client timeout settings
>
> Steve,
>
> We have been using timeouts in production via two different methods:
>
> 1. Use RPC timeout mechanism provided by HBase client. For this you will
> need both 2937 and 3154. We backported 2937 to 0.90.3. The reason 3154
> alone does not help is because if you set the conf parameter introduced in
> 3154 (hbase.rpc.timeout) to a very low value your calls to meta (ROOT and
> META) tables timeout all the time and almost no request go through. That is
> where 2937 comes in. It uses hbase.client.operation.timeout knob for all
> RPC calls except the ones made to meta tables.
>
> 2. Use an async method (example ExecutorService with timeout) to make HBase
> client calls.
>
> In our experience #1 perform better (in terms of % calls timing out).
>
> Having said that I am still interested to hear comments from experts. I
> personally was not able to grasp reasoning behind introduction of
> hbase.client.operation.timeout configuration parameter.
>
> -Shrijeet
>
> On Thu, Dec 15, 2011 at 3:15 PM, Steve Boyle <st...@connexity.com>
> wrote:
> >
> > Hi,
> >
> > I'm trying to understand what timeout controls are available in the hbase
> client. I'm using hbase version 0.90.4-cdh3u2. I have a client
> application that does gets, puts, increments and scans. I'd like to be
> able to have a client side timeout such that the client can clean-up in a
> case where the response is taking a while for whatever reason.
> >
> > I've seen this jira which indicates that the feature I'm looking for is
> coming in a future release:
> > https://issues.apache.org/jira/browse/HBASE-2937
> >
> > I've also seen this jira that looks related and appears to be already
> available:
> > https://issues.apache.org/jira/browse/HBASE-3154
> >
> > What is the best way to control a timeout for the hbase client?
> >
> > Thanks,
> > Steve
> >
> >
> >
>
RE: understanding hbase client timeout settings
Posted by Steve Boyle <st...@connexity.com>.
Shrijeet,
What do you consider a very low value for hbase.rpc.timeout?
Thanks,
Steve
-----Original Message-----
From: Shrijeet Paliwal [mailto:shrijeet@rocketfuel.com]
Sent: Thursday, December 15, 2011 9:03 PM
To: user@hbase.apache.org
Subject: Re: understanding hbase client timeout settings
Steve,
We have been using timeouts in production via two different methods:
1. Use RPC timeout mechanism provided by HBase client. For this you will
need both 2937 and 3154. We backported 2937 to 0.90.3. The reason 3154
alone does not help is because if you set the conf parameter introduced in
3154 (hbase.rpc.timeout) to a very low value your calls to meta (ROOT and
META) tables timeout all the time and almost no request go through. That is
where 2937 comes in. It uses hbase.client.operation.timeout knob for all
RPC calls except the ones made to meta tables.
2. Use an async method (example ExecutorService with timeout) to make HBase
client calls.
In our experience #1 perform better (in terms of % calls timing out).
Having said that I am still interested to hear comments from experts. I
personally was not able to grasp reasoning behind introduction of
hbase.client.operation.timeout configuration parameter.
-Shrijeet
On Thu, Dec 15, 2011 at 3:15 PM, Steve Boyle <st...@connexity.com>
wrote:
>
> Hi,
>
> I'm trying to understand what timeout controls are available in the hbase
client. I'm using hbase version 0.90.4-cdh3u2. I have a client
application that does gets, puts, increments and scans. I'd like to be
able to have a client side timeout such that the client can clean-up in a
case where the response is taking a while for whatever reason.
>
> I've seen this jira which indicates that the feature I'm looking for is
coming in a future release:
> https://issues.apache.org/jira/browse/HBASE-2937
>
> I've also seen this jira that looks related and appears to be already
available:
> https://issues.apache.org/jira/browse/HBASE-3154
>
> What is the best way to control a timeout for the hbase client?
>
> Thanks,
> Steve
>
>
>
Re: understanding hbase client timeout settings
Posted by Shrijeet Paliwal <sh...@rocketfuel.com>.
Steve,
We have been using timeouts in production via two different methods:
1. Use RPC timeout mechanism provided by HBase client. For this you will
need both 2937 and 3154. We backported 2937 to 0.90.3. The reason 3154
alone does not help is because if you set the conf parameter introduced in
3154 (hbase.rpc.timeout) to a very low value your calls to meta (ROOT and
META) tables timeout all the time and almost no request go through. That is
where 2937 comes in. It uses hbase.client.operation.timeout knob for all
RPC calls except the ones made to meta tables.
2. Use an async method (example ExecutorService with timeout) to make HBase
client calls.
In our experience #1 perform better (in terms of % calls timing out).
Having said that I am still interested to hear comments from experts. I
personally was not able to grasp reasoning behind introduction of
hbase.client.operation.timeout configuration parameter.
-Shrijeet
On Thu, Dec 15, 2011 at 3:15 PM, Steve Boyle <st...@connexity.com>
wrote:
>
> Hi,
>
> I'm trying to understand what timeout controls are available in the hbase
client. I'm using hbase version 0.90.4-cdh3u2. I have a client
application that does gets, puts, increments and scans. I'd like to be
able to have a client side timeout such that the client can clean-up in a
case where the response is taking a while for whatever reason.
>
> I've seen this jira which indicates that the feature I'm looking for is
coming in a future release:
> https://issues.apache.org/jira/browse/HBASE-2937
>
> I've also seen this jira that looks related and appears to be already
available:
> https://issues.apache.org/jira/browse/HBASE-3154
>
> What is the best way to control a timeout for the hbase client?
>
> Thanks,
> Steve
>
>
>