[jira] [Comment Edited] (HBASE-29141) Default maxQueueLength used for call queues is too high

["Umesh Kumar Kumawat (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-29141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18041287#comment-18041287 ] 

Umesh Kumar Kumawat edited comment on HBASE-29141 at 11/28/25 12:48 PM:
------------------------------------------------------------------------

From the [^HBASE-15306-v1.patch] when DEFAULT_CALL_QUEUE_SIZE_HARD_LIMIT was introduced it was a MAX hard limit


was (Author: JIRAUSER298625):
From the [^HBASE-15306-v1.patch] when DEFAULT_CALL_QUEUE_SIZE_HARD_LIMIT was introduced it was a MAX hard limit

> Default maxQueueLength used for call queues is too high
> -------------------------------------------------------
>
>                 Key: HBASE-29141
>                 URL: https://issues.apache.org/jira/browse/HBASE-29141
>             Project: HBase
>          Issue Type: Bug
>          Components: master, regionserver
>    Affects Versions: 3.0.0-beta-1, 2.5.11, 2.6.2
>            Reporter: Umesh Kumar Kumawat
>            Assignee: Umesh Kumar Kumawat
>            Priority: Major
>
> Regarding the handler and queues HBase have these config. 
>  * {{hbase.regionserver.handler.count}}  => Number of handlers, Default is 30.
>  * {{hbase.ipc.server.callqueue.handler.factor}}  => Queue-to-handler ratio. Default it is 0.1. Means 10 handler per queue.
>  ** {{number of call queues}} = {{handlerCount}} * {{callQueuesHandlersFactor(0.1)}}  ([code|https://github.com/apache/hbase/blob/777010361abb203b8b17673d84acf4f7f1d0283a/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcExecutor.java#L195])
>  * {{hbase.ipc.server.max.callqueue.length}}  => Max callqueue Length. Default is {{DEFAULT_MAX_CALLQUEUE_LENGTH_PER_HANDLER(10) * {color:#ff0000}*total handlerCount*{color}}}  ([code|https://github.com/apache/hbase/blob/777010361abb203b8b17673d84acf4f7f1d0283a/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java#L71])
>  ** {*}Issue{*}: I think there’s a small issue here. We are currently treating this as the length per queue, but it's calculated using the {_}total handler count{_}, not the number of handlers per queue.
>  * In addition we also have a hard limit of ({{{}DEFAULT_CALL_QUEUE_SIZE_HARD_LIMIT{}}}) 250 ([code|https://github.com/apache/hbase/blob/777010361abb203b8b17673d84acf4f7f1d0283a/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcExecutor.java#L228]) this is hard limit on *minimum* value of queue size. Queue size can not be less then 250.
>  
> In codel queue type, we have a config {{hbase.ipc.server.callqueue.codel.lifo.threshold}} (default 0.8). Having 100 handlers, by default all the 10 queues will be of size 1000 and queue processing will not change to lifo unless there are 800 calls waiitng in a single queue. Don't you think it is too much.
>  
>  # I believe we may have unintentionally created a dependency between the *total number of handlers* and the call queue length. Should we reconsider this logic?
>  # Regarding the hard limit: should we be enforcing a _minimum_ value for the queue size or a _maximum_ value for the queue size ?



--
This message was sent by Atlassian Jira
(v8.20.10#820010)