You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Lars Hofhansl (Created) (JIRA)" <ji...@apache.org> on 2012/01/10 02:29:39 UTC
[jira] [Created] (HBASE-5164) Better HTable resource consumption in
CoprocessorHost
Better HTable resource consumption in CoprocessorHost
-----------------------------------------------------
Key: HBASE-5164
URL: https://issues.apache.org/jira/browse/HBASE-5164
Project: HBase
Issue Type: Sub-task
Components: coprocessors
Reporter: Lars Hofhansl
Priority: Minor
Fix For: 0.94.0
HBASE-4805 allows for more control over HTable's resource consumption.
This is currently not used by CoprocessorHost (even though it would even be more critical to control this inside the RegionServer).
It's not immediate obvious how to do that.
Maybe CoprocessorHost should maintain a lazy ExecutorService and HConnection and reuse both for all HTables retrieved via CoprocessorEnvironment.getTable(...).
Not sure how critical this is, but I feel without this it is dangerous to use getTable, as it would lead to all resource consumption problems we find in the client, but inside a crucial part of the HBase servers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5164) Better HTable resource consumption
in CoprocessorHost
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13183100#comment-13183100 ]
Lars Hofhansl commented on HBASE-5164:
--------------------------------------
Thanks Andrew,
absolutely agree that CPs need table access beyond the local region scope.
What I have in mind with this is essentially a stop gap measure, using the extended HTable API (introduced in HBASE-4805) that allows for externally (from the HTable's viewpoint) managed ExecutorService/HConnection. The CPhost would manage these and getTable would return a HTable created with the HTable(byte[], HConnection, ExecutorService) constructor.
Do you think it is worth doing this?
> Better HTable resource consumption in CoprocessorHost
> -----------------------------------------------------
>
> Key: HBASE-5164
> URL: https://issues.apache.org/jira/browse/HBASE-5164
> Project: HBase
> Issue Type: Sub-task
> Components: coprocessors
> Reporter: Lars Hofhansl
> Priority: Minor
> Fix For: 0.94.0
>
>
> HBASE-4805 allows for more control over HTable's resource consumption.
> This is currently not used by CoprocessorHost (even though it would even be more critical to control this inside the RegionServer).
> It's not immediate obvious how to do that.
> Maybe CoprocessorHost should maintain a lazy ExecutorService and HConnection and reuse both for all HTables retrieved via CoprocessorEnvironment.getTable(...).
> Not sure how critical this is, but I feel without this it is dangerous to use getTable, as it would lead to all resource consumption problems we find in the client, but inside a crucial part of the HBase servers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5164) Better HTable resource consumption
in CoprocessorHost
Posted by "Andrew Purtell (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13183069#comment-13183069 ]
Andrew Purtell commented on HBASE-5164:
---------------------------------------
We added getTable to the CP API with that risk in mind. The notion is CPs are not allowed to just instantiate HTables, instead they ask for a more constrained server side IPC. No such implementation yet exists, so getTable essentially is a placeholder.
If we consider coprocessors hosted in an external process, but still on the server, the risks are not much lower than in process. A stuck table client in the external CP will hold up the initiator anyway, and in fact this makes external CP handling more problematic than if table clients were simply not allowed server side at all. However I think many useful CP use cases must have a table access scope wider than the single region.
What we need is a "lightweight" server to server IPC? Would present HTableInterface but lack the fat client semantics.
> Better HTable resource consumption in CoprocessorHost
> -----------------------------------------------------
>
> Key: HBASE-5164
> URL: https://issues.apache.org/jira/browse/HBASE-5164
> Project: HBase
> Issue Type: Sub-task
> Components: coprocessors
> Reporter: Lars Hofhansl
> Priority: Minor
> Fix For: 0.94.0
>
>
> HBASE-4805 allows for more control over HTable's resource consumption.
> This is currently not used by CoprocessorHost (even though it would even be more critical to control this inside the RegionServer).
> It's not immediate obvious how to do that.
> Maybe CoprocessorHost should maintain a lazy ExecutorService and HConnection and reuse both for all HTables retrieved via CoprocessorEnvironment.getTable(...).
> Not sure how critical this is, but I feel without this it is dangerous to use getTable, as it would lead to all resource consumption problems we find in the client, but inside a crucial part of the HBase servers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5164) Better HTable resource consumption in
CoprocessorHost
Posted by "Lars Hofhansl (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-5164:
---------------------------------
Fix Version/s: (was: 0.94.0)
0.96.0
> Better HTable resource consumption in CoprocessorHost
> -----------------------------------------------------
>
> Key: HBASE-5164
> URL: https://issues.apache.org/jira/browse/HBASE-5164
> Project: HBase
> Issue Type: Sub-task
> Components: coprocessors
> Reporter: Lars Hofhansl
> Priority: Minor
> Fix For: 0.96.0
>
>
> HBASE-4805 allows for more control over HTable's resource consumption.
> This is currently not used by CoprocessorHost (even though it would even be more critical to control this inside the RegionServer).
> It's not immediate obvious how to do that.
> Maybe CoprocessorHost should maintain a lazy ExecutorService and HConnection and reuse both for all HTables retrieved via CoprocessorEnvironment.getTable(...).
> Not sure how critical this is, but I feel without this it is dangerous to use getTable, as it would lead to all resource consumption problems we find in the client, but inside a crucial part of the HBase servers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira