You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Karthick Sankarachary <ka...@gmail.com> on 2010/08/30 23:29:12 UTC
Re: Review Request: HBase-2937 Facilitate Timeouts In HBase Client
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/747/
-----------------------------------------------------------
(Updated 2010-08-30 14:29:12.219496)
Review request for hbase.
Summary (updated)
-------
Currently, there is no way to force an operation on the HBase client (viz. HTable) to time out if a certain amount of time has elapsed. In other words, all invocations on the HTable class are veritable blocking calls, which will not return until a response (successful or otherwise) is received.
In general, there are two ways to handle timeouts: (a) call the operation in a separate thread, until it returns a response or the wait on the thread times out and (b) have the underlying socket unblock the operation if the read times out. The downside of the former approach is that it consumes more resources in terms of threads and callables.
Here, we describe a way to specify and handle timeouts on the HTable client, which relies on the latter approach (i.e., socket timeouts). Right now, the HBaseClient sets the socket timeout to the value of the "ipc.ping.interval" parameter, which is also how long it waits before pinging the server in case of a failure. The goal is to allow clients to set that timeout on the fly through HTable. Rather than adding an optional timeout argument to every HTable operation, we chose to make it a property of HTable which effectively applies to every method that involves a remote operation.
In order to propagate the timeout from HTable to HBaseClient, we replaced all occurrences of ServerCallable in HTable with an extension called ClientCallable, which sets the timeout on the region server interface, once it has been instantiated, through the HConnection object. The latter, in turn, asks HBaseRPC to pass that timeout to the corresponding Invoker, so that it may inject the timeout at the time the invocation is made on the region server proxy. Right before the request is sent to the server, we set the timeout specified by the client on the underlying socket.
In conclusion, this patch will afford clients the option of performing an HBase operation until it completes or a specified timeout elapses. Note that a timeout of zero is interpreted as an infinite timeout.
This addresses bug HBASE-2937.
http://issues.apache.org/jira/browse/HBASE-2937
Diffs
-----
src/main/java/org/apache/hadoop/hbase/Timeout.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/client/ClientCallable.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/client/HConnection.java 8b3b6a4
src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java ac2b99c
src/main/java/org/apache/hadoop/hbase/client/HTable.java cd60eeb
src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java 2b5eeb6
src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPC.java 9873172
Diff: http://review.cloudera.org/r/747/diff
Testing
-------
Thanks,
Karthick
Re: Review Request: HBase-2937 Facilitate Timeouts In HBase Client
Posted by st...@duboce.net.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/747/#review1497
-----------------------------------------------------------
This looks great Karthik. How does it interact with our current retry mechanism? You replace our old retrying Callable? There's a few comments below.
src/main/java/org/apache/hadoop/hbase/Timeout.java
<http://review.cloudera.org/r/747/#comment5135>
There is white space at the end of lines in this javadoc. Also, its apache or hadoop policy not to have an author tag on classes (no other classes in hbase codebase have author).
src/main/java/org/apache/hadoop/hbase/Timeout.java
<http://review.cloudera.org/r/747/#comment5137>
Why not make this class immutable? Force users to create a new one if they want to have different settings?
src/main/java/org/apache/hadoop/hbase/client/ClientCallable.java
<http://review.cloudera.org/r/747/#comment5139>
White space and author tag.
src/main/java/org/apache/hadoop/hbase/client/HConnection.java
<http://review.cloudera.org/r/747/#comment5141>
Maybe setRegionServerTimeout is a better name fort this method?
src/main/java/org/apache/hadoop/hbase/client/HTable.java
<http://review.cloudera.org/r/747/#comment5142>
Timeout could be null here?
src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java
<http://review.cloudera.org/r/747/#comment5143>
Are there tabs in here? And we use two spaces for tabs in our code base.
src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPC.java
<http://review.cloudera.org/r/747/#comment5145>
two spaces for tab spaces in hbase codebase
- stack
On 2010-08-30 14:29:12, Karthick Sankarachary wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> http://review.cloudera.org/r/747/
> -----------------------------------------------------------
>
> (Updated 2010-08-30 14:29:12)
>
>
> Review request for hbase.
>
>
> Summary
> -------
>
> Currently, there is no way to force an operation on the HBase client (viz. HTable) to time out if a certain amount of time has elapsed. In other words, all invocations on the HTable class are veritable blocking calls, which will not return until a response (successful or otherwise) is received.
>
> In general, there are two ways to handle timeouts: (a) call the operation in a separate thread, until it returns a response or the wait on the thread times out and (b) have the underlying socket unblock the operation if the read times out. The downside of the former approach is that it consumes more resources in terms of threads and callables.
>
> Here, we describe a way to specify and handle timeouts on the HTable client, which relies on the latter approach (i.e., socket timeouts). Right now, the HBaseClient sets the socket timeout to the value of the "ipc.ping.interval" parameter, which is also how long it waits before pinging the server in case of a failure. The goal is to allow clients to set that timeout on the fly through HTable. Rather than adding an optional timeout argument to every HTable operation, we chose to make it a property of HTable which effectively applies to every method that involves a remote operation.
>
> In order to propagate the timeout from HTable to HBaseClient, we replaced all occurrences of ServerCallable in HTable with an extension called ClientCallable, which sets the timeout on the region server interface, once it has been instantiated, through the HConnection object. The latter, in turn, asks HBaseRPC to pass that timeout to the corresponding Invoker, so that it may inject the timeout at the time the invocation is made on the region server proxy. Right before the request is sent to the server, we set the timeout specified by the client on the underlying socket.
>
> In conclusion, this patch will afford clients the option of performing an HBase operation until it completes or a specified timeout elapses. Note that a timeout of zero is interpreted as an infinite timeout.
>
>
> This addresses bug HBASE-2937.
> http://issues.apache.org/jira/browse/HBASE-2937
>
>
> Diffs
> -----
>
> src/main/java/org/apache/hadoop/hbase/Timeout.java PRE-CREATION
> src/main/java/org/apache/hadoop/hbase/client/ClientCallable.java PRE-CREATION
> src/main/java/org/apache/hadoop/hbase/client/HConnection.java 8b3b6a4
> src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java ac2b99c
> src/main/java/org/apache/hadoop/hbase/client/HTable.java cd60eeb
> src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java 2b5eeb6
> src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPC.java 9873172
>
> Diff: http://review.cloudera.org/r/747/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Karthick
>
>
Re: Review Request: HBase-2937 Facilitate Timeouts In HBase Client
Posted by Karthick Sankarachary <ka...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/747/
-----------------------------------------------------------
(Updated 2010-10-18 23:30:16.806654)
Review request for hbase.
Summary
-------
Currently, there is no way to force an operation on the HBase client (viz. HTable) to time out if a certain amount of time has elapsed. In other words, all invocations on the HTable class are veritable blocking calls, which will not return until a response (successful or otherwise) is received.
In general, there are two ways to handle timeouts: (a) call the operation in a separate thread, until it returns a response or the wait on the thread times out and (b) have the underlying socket unblock the operation if the read times out. The downside of the former approach is that it consumes more resources in terms of threads and callables.
Here, we describe a way to specify and handle timeouts on the HTable client, which relies on the latter approach (i.e., socket timeouts). Right now, the HBaseClient sets the socket timeout to the value of the "ipc.ping.interval" parameter, which is also how long it waits before pinging the server in case of a failure. The goal is to allow clients to set that timeout on the fly through HTable. Rather than adding an optional timeout argument to every HTable operation, we chose to make it a property of HTable which effectively applies to every method that involves a remote operation.
In order to propagate the timeout from HTable to HBaseClient, we replaced all occurrences of ServerCallable in HTable with an extension called ClientCallable, which sets the timeout on the region server interface, once it has been instantiated, through the HConnection object. The latter, in turn, asks HBaseRPC to pass that timeout to the corresponding Invoker, so that it may inject the timeout at the time the invocation is made on the region server proxy. Right before the request is sent to the server, we set the timeout specified by the client on the underlying socket.
In conclusion, this patch will afford clients the option of performing an HBase operation until it completes or a specified timeout elapses. Note that a timeout of zero is interpreted as an infinite timeout.
This addresses bug HBASE-2937.
http://issues.apache.org/jira/browse/HBASE-2937
Diffs (updated)
-----
src/main/java/org/apache/hadoop/hbase/TimeoutPolicy.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/client/ClientCallable.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/client/HConnection.java d9164b6
src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java d07201c
src/main/java/org/apache/hadoop/hbase/client/HTable.java 4d60a17
src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java 2b5eeb6
src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPC.java e23a629
src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java bdefb2c
Diff: http://review.cloudera.org/r/747/diff
Testing
-------
Thanks,
Karthick
Re: Review Request: HBase-2937 Facilitate Timeouts In HBase Client
Posted by Karthick Sankarachary <ka...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/747/
-----------------------------------------------------------
(Updated 2010-10-14 22:26:21.319560)
Review request for hbase.
Summary
-------
Currently, there is no way to force an operation on the HBase client (viz. HTable) to time out if a certain amount of time has elapsed. In other words, all invocations on the HTable class are veritable blocking calls, which will not return until a response (successful or otherwise) is received.
In general, there are two ways to handle timeouts: (a) call the operation in a separate thread, until it returns a response or the wait on the thread times out and (b) have the underlying socket unblock the operation if the read times out. The downside of the former approach is that it consumes more resources in terms of threads and callables.
Here, we describe a way to specify and handle timeouts on the HTable client, which relies on the latter approach (i.e., socket timeouts). Right now, the HBaseClient sets the socket timeout to the value of the "ipc.ping.interval" parameter, which is also how long it waits before pinging the server in case of a failure. The goal is to allow clients to set that timeout on the fly through HTable. Rather than adding an optional timeout argument to every HTable operation, we chose to make it a property of HTable which effectively applies to every method that involves a remote operation.
In order to propagate the timeout from HTable to HBaseClient, we replaced all occurrences of ServerCallable in HTable with an extension called ClientCallable, which sets the timeout on the region server interface, once it has been instantiated, through the HConnection object. The latter, in turn, asks HBaseRPC to pass that timeout to the corresponding Invoker, so that it may inject the timeout at the time the invocation is made on the region server proxy. Right before the request is sent to the server, we set the timeout specified by the client on the underlying socket.
In conclusion, this patch will afford clients the option of performing an HBase operation until it completes or a specified timeout elapses. Note that a timeout of zero is interpreted as an infinite timeout.
This addresses bug HBASE-2937.
http://issues.apache.org/jira/browse/HBASE-2937
Diffs (updated)
-----
src/main/java/org/apache/hadoop/hbase/TimeoutPolicy.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/client/ClientCallable.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/client/HConnection.java d9164b6
src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java fbffd0b
src/main/java/org/apache/hadoop/hbase/client/HTable.java 4d60a17
src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java 2b5eeb6
src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPC.java e23a629
Diff: http://review.cloudera.org/r/747/diff
Testing
-------
Thanks,
Karthick
Re: Review Request: HBase-2937 Facilitate Timeouts In HBase Client
Posted by Karthick Sankarachary <ka...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/747/
-----------------------------------------------------------
(Updated 2010-10-13 22:44:50.353740)
Review request for hbase.
Summary
-------
Currently, there is no way to force an operation on the HBase client (viz. HTable) to time out if a certain amount of time has elapsed. In other words, all invocations on the HTable class are veritable blocking calls, which will not return until a response (successful or otherwise) is received.
In general, there are two ways to handle timeouts: (a) call the operation in a separate thread, until it returns a response or the wait on the thread times out and (b) have the underlying socket unblock the operation if the read times out. The downside of the former approach is that it consumes more resources in terms of threads and callables.
Here, we describe a way to specify and handle timeouts on the HTable client, which relies on the latter approach (i.e., socket timeouts). Right now, the HBaseClient sets the socket timeout to the value of the "ipc.ping.interval" parameter, which is also how long it waits before pinging the server in case of a failure. The goal is to allow clients to set that timeout on the fly through HTable. Rather than adding an optional timeout argument to every HTable operation, we chose to make it a property of HTable which effectively applies to every method that involves a remote operation.
In order to propagate the timeout from HTable to HBaseClient, we replaced all occurrences of ServerCallable in HTable with an extension called ClientCallable, which sets the timeout on the region server interface, once it has been instantiated, through the HConnection object. The latter, in turn, asks HBaseRPC to pass that timeout to the corresponding Invoker, so that it may inject the timeout at the time the invocation is made on the region server proxy. Right before the request is sent to the server, we set the timeout specified by the client on the underlying socket.
In conclusion, this patch will afford clients the option of performing an HBase operation until it completes or a specified timeout elapses. Note that a timeout of zero is interpreted as an infinite timeout.
This addresses bug HBASE-2937.
http://issues.apache.org/jira/browse/HBASE-2937
Diffs (updated)
-----
src/main/java/org/apache/hadoop/hbase/Timeout.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/client/ClientCallable.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/client/HConnection.java d9164b6
src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java fbffd0b
src/main/java/org/apache/hadoop/hbase/client/HTable.java 4d60a17
src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java 2b5eeb6
src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPC.java e23a629
Diff: http://review.cloudera.org/r/747/diff
Testing
-------
Thanks,
Karthick