You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Karthick Sankarachary (JIRA)" <ji...@apache.org> on 2010/08/27 21:11:54 UTC
[jira] Created: (HBASE-2938) Add Thread-Local Behavior To HTable
Pool
Add Thread-Local Behavior To HTable Pool
----------------------------------------
Key: HBASE-2938
URL: https://issues.apache.org/jira/browse/HBASE-2938
Project: HBase
Issue Type: Improvement
Components: client
Affects Versions: 0.89.20100621
Reporter: Karthick Sankarachary
Attachments: HBASE-2938.patch
It is a well-documented fact that the HBase table client (viz., HTable) is not thread-safe. Hence, the recommendation has been to use a HTablePool or a ThreadLocal to manage access to tables. The downside of the latter is that it (a) requires the user to reinvent the wheel in terms of mapping table names to tables and (b) forces the user to maintain the thread-local objects. Ideally, it would be nice if we could make the HTablePool handle thread-local objects as well. That way, it not only becomes the "one stop shop" for all client-side tables, but also insulates the user from the ThreadLocal object.
Here, we propose a way to generalize the HTablePool so that the underlying pool type is either "reusable" or "thread-local". To make this possible, we introdudce the concept of a SharedMap, which essentially, maps a key to a collection of values, the elements of which are managed by a pool. In effect, that collection acts as a shared pool of resources, access to which is closely controlled as dictated by the particular semantics of the pool.
Furthermore, to simplify the construction of HTablePools, we added a couple of parameters (viz. "hbase.client.htable.pool.type" and "hbase.client.hbase.pool.size") to control the default behavior of a HTablePool.
In case the size of the pool is set to a non-zero positive number, that is used to cap the number of resources that a pool may contain for any given key. A size of Integer#MAX_VALUE is interpreted to mean an unbounded pool.
Currently, the SharedMap supports the following types of pools:
* A ThreadLocalPool, which represents a pool that builds on the ThreadLocal class. It essentially binds the resource to the thread from which it is accessed.
* A ReusablePool, which represents a pool that builds on the LinkedList class. It essentially allows resources to be checked out, at which point it is (temporarily) removed from the pool. When the resource is no longer required, it should be returned to the pool in order to be reused.
* A RoundRobinPool, which represents a pool that stores its resources in an ArrayList. It load-balances access to its resources by returning a different resource every time a given key is looked up.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-2938) Add Thread-Local Behavior To HTable
Pool
Posted by "Karthick Sankarachary (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karthick Sankarachary updated HBASE-2938:
-----------------------------------------
Status: Patch Available (was: Open)
> Add Thread-Local Behavior To HTable Pool
> ----------------------------------------
>
> Key: HBASE-2938
> URL: https://issues.apache.org/jira/browse/HBASE-2938
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.89.20100621
> Reporter: Karthick Sankarachary
> Attachments: HBASE-2938.patch
>
>
> It is a well-documented fact that the HBase table client (viz., HTable) is not thread-safe. Hence, the recommendation has been to use a HTablePool or a ThreadLocal to manage access to tables. The downside of the latter is that it (a) requires the user to reinvent the wheel in terms of mapping table names to tables and (b) forces the user to maintain the thread-local objects. Ideally, it would be nice if we could make the HTablePool handle thread-local objects as well. That way, it not only becomes the "one stop shop" for all client-side tables, but also insulates the user from the ThreadLocal object.
>
> Here, we propose a way to generalize the HTablePool so that the underlying pool type is either "reusable" or "thread-local". To make this possible, we introdudce the concept of a SharedMap, which essentially, maps a key to a collection of values, the elements of which are managed by a pool. In effect, that collection acts as a shared pool of resources, access to which is closely controlled as dictated by the particular semantics of the pool.
> Furthermore, to simplify the construction of HTablePools, we added a couple of parameters (viz. "hbase.client.htable.pool.type" and "hbase.client.hbase.pool.size") to control the default behavior of a HTablePool.
>
> In case the size of the pool is set to a non-zero positive number, that is used to cap the number of resources that a pool may contain for any given key. A size of Integer#MAX_VALUE is interpreted to mean an unbounded pool.
>
> Currently, the SharedMap supports the following types of pools:
> * A ThreadLocalPool, which represents a pool that builds on the ThreadLocal class. It essentially binds the resource to the thread from which it is accessed.
> * A ReusablePool, which represents a pool that builds on the LinkedList class. It essentially allows resources to be checked out, at which point it is (temporarily) removed from the pool. When the resource is no longer required, it should be returned to the pool in order to be reused.
> * A RoundRobinPool, which represents a pool that stores its resources in an ArrayList. It load-balances access to its resources by returning a different resource every time a given key is looked up.
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-2938) Add Thread-Local Behavior To HTable
Pool
Posted by "Karthick Sankarachary (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karthick Sankarachary updated HBASE-2938:
-----------------------------------------
Attachment: HBASE-2938.patch
> Add Thread-Local Behavior To HTable Pool
> ----------------------------------------
>
> Key: HBASE-2938
> URL: https://issues.apache.org/jira/browse/HBASE-2938
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.89.20100621
> Reporter: Karthick Sankarachary
> Attachments: HBASE-2938.patch
>
>
> It is a well-documented fact that the HBase table client (viz., HTable) is not thread-safe. Hence, the recommendation has been to use a HTablePool or a ThreadLocal to manage access to tables. The downside of the latter is that it (a) requires the user to reinvent the wheel in terms of mapping table names to tables and (b) forces the user to maintain the thread-local objects. Ideally, it would be nice if we could make the HTablePool handle thread-local objects as well. That way, it not only becomes the "one stop shop" for all client-side tables, but also insulates the user from the ThreadLocal object.
>
> Here, we propose a way to generalize the HTablePool so that the underlying pool type is either "reusable" or "thread-local". To make this possible, we introdudce the concept of a SharedMap, which essentially, maps a key to a collection of values, the elements of which are managed by a pool. In effect, that collection acts as a shared pool of resources, access to which is closely controlled as dictated by the particular semantics of the pool.
> Furthermore, to simplify the construction of HTablePools, we added a couple of parameters (viz. "hbase.client.htable.pool.type" and "hbase.client.hbase.pool.size") to control the default behavior of a HTablePool.
>
> In case the size of the pool is set to a non-zero positive number, that is used to cap the number of resources that a pool may contain for any given key. A size of Integer#MAX_VALUE is interpreted to mean an unbounded pool.
>
> Currently, the SharedMap supports the following types of pools:
> * A ThreadLocalPool, which represents a pool that builds on the ThreadLocal class. It essentially binds the resource to the thread from which it is accessed.
> * A ReusablePool, which represents a pool that builds on the LinkedList class. It essentially allows resources to be checked out, at which point it is (temporarily) removed from the pool. When the resource is no longer required, it should be returned to the pool in order to be reused.
> * A RoundRobinPool, which represents a pool that stores its resources in an ArrayList. It load-balances access to its resources by returning a different resource every time a given key is looked up.
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.