You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@hbase.apache.org by "Andrew Purtell (JIRA)" <ji...@apache.org> on 2010/01/17 21:33:54 UTC

[jira] Created: (HBASE-2141) Flexible thread pool for RPC server

Flexible thread pool for RPC server
-----------------------------------

                 Key: HBASE-2141
                 URL: https://issues.apache.org/jira/browse/HBASE-2141
             Project: Hadoop HBase
          Issue Type: Improvement
            Reporter: Andrew Purtell
             Fix For: 0.21.0


Having a bounded thread pool is important, so availability can degrade gracefully (more or less), as opposed to the whole regionserver becoming livelocked. But, we don't need for the pool to be preallocated as is done currently. How about changing the RPC server thread pool such that the user can specify a minimum and maximum number of handler threads? The pool would start with the minimum, allocate more up to the max to handle additional concurrency, then terminate unused threads after some time back down to the minimum. Then we can do things like set a maximum of 100 handlers or such without taking on the overhead of 100 threads until it is needed. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HBASE-2141) Flexible thread pool for RPC server

Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.

    [ https://issues.apache.org/jira/browse/HBASE-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12801497#action_12801497 ] 

Jean-Daniel Cryans commented on HBASE-2141:
-------------------------------------------

This also helps with debugging jstacks so that if the user never uses 100 handlers but configured 100 then it's more pain to read through.

> Flexible thread pool for RPC server
> -----------------------------------
>
>                 Key: HBASE-2141
>                 URL: https://issues.apache.org/jira/browse/HBASE-2141
>             Project: Hadoop HBase
>          Issue Type: Improvement
>            Reporter: Andrew Purtell
>             Fix For: 0.21.0
>
>
> Having a bounded thread pool is important, so availability can degrade gracefully (more or less), as opposed to the whole regionserver becoming livelocked. But, we don't need for the pool to be preallocated as is done currently. How about changing the RPC server thread pool such that the user can specify a minimum and maximum number of handler threads? The pool would start with the minimum, allocate more up to the max to handle additional concurrency, then terminate unused threads after some time back down to the minimum. Then we can do things like set a maximum of 100 handlers or such without taking on the overhead of 100 threads until it is needed. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HBASE-2141) Flexible thread pool for RPC server

Posted by "stack (JIRA)" <ji...@apache.org>.

    [ https://issues.apache.org/jira/browse/HBASE-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12801498#action_12801498 ] 

stack commented on HBASE-2141:
------------------------------

For me, I'm not that mad about doing dev on hadoop rpc, or our subclass of it.  I think we should start over.  Start with an nio2-based rpc.  Growing the children as per apache1 or bounding the threads allowed in as per apache2 can be a feature of the new rpc.



> Flexible thread pool for RPC server
> -----------------------------------
>
>                 Key: HBASE-2141
>                 URL: https://issues.apache.org/jira/browse/HBASE-2141
>             Project: Hadoop HBase
>          Issue Type: Improvement
>            Reporter: Andrew Purtell
>             Fix For: 0.21.0
>
>
> Having a bounded thread pool is important, so availability can degrade gracefully (more or less), as opposed to the whole regionserver becoming livelocked. But, we don't need for the pool to be preallocated as is done currently. How about changing the RPC server thread pool such that the user can specify a minimum and maximum number of handler threads? The pool would start with the minimum, allocate more up to the max to handle additional concurrency, then terminate unused threads after some time back down to the minimum. Then we can do things like set a maximum of 100 handlers or such without taking on the overhead of 100 threads until it is needed. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.