You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Duo Zhang (Jira)" <ji...@apache.org> on 2026/06/15 07:54:00 UTC
[jira] [Updated] (HBASE-29141) Calculate default maxQueueLength call queues correctly
[ https://issues.apache.org/jira/browse/HBASE-29141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Duo Zhang updated HBASE-29141:
------------------------------
Fix Version/s: (was: 4.0.0-alpha-1)
> Calculate default maxQueueLength call queues correctly
> ------------------------------------------------------
>
> Key: HBASE-29141
> URL: https://issues.apache.org/jira/browse/HBASE-29141
> Project: HBase
> Issue Type: Bug
> Components: master, regionserver
> Affects Versions: 3.0.0-beta-1, 2.5.11, 2.6.2
> Reporter: Umesh Kumar Kumawat
> Assignee: Umesh Kumar Kumawat
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.7.0, 3.0.0-beta-2, 2.6.5, 2.5.14
>
>
> *TLDR*
> In hbase for SimpleRPCScheduler, we have call queues and handlers. For each queue there are some handler assigned. We have a config, {{hbase.ipc.server.max.callqueue.length}} to control the maximum length of the queue. If we don't define this config HBase tries calculate default maxLength for the queue using {{{}DEFAULT_MAX_CALLQUEUE_LENGTH_PER_HANDLER{}}}. But there was one issue here each queue was assigned to a portion of handler but while calculating the max queue length we were using the count for all the handlers.
> *Description using configs and code links*
> _____________________________________________________________________________________________________________________
> Regarding the handler and queues HBase have these config.
> * {{hbase.regionserver.handler.count}} => Number of handlers, Default is 30.
> * {{hbase.ipc.server.callqueue.handler.factor}} => Queue-to-handler ratio. Default it is 0.1. Means 10 handler per queue.
> ** {{number of call queues}} = {{handlerCount}} * {{callQueuesHandlersFactor(0.1)}} ([code|https://github.com/apache/hbase/blob/777010361abb203b8b17673d84acf4f7f1d0283a/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcExecutor.java#L195])
> * {{hbase.ipc.server.max.callqueue.length}} => Max callqueue Length. Default is {{DEFAULT_MAX_CALLQUEUE_LENGTH_PER_HANDLER(10) * {color:#ff0000}*total handlerCount*{color}}} ([code|https://github.com/apache/hbase/blob/777010361abb203b8b17673d84acf4f7f1d0283a/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java#L71])
> ** {*}Issue{*}: I think there’s a small issue here. We are currently treating this as the length per queue, but it's calculated using the {_}total handler count{_}, not the number of handlers per queue.
> _____________________________________________________________________________________________________________________
> *Other concerns*
> In codel queue type, we have a config {{hbase.ipc.server.callqueue.codel.lifo.threshold}} (default 0.8). Having 100 handlers, by default all the 10 queues will be of size 1000 and queue processing will not change to lifo unless there are 800 calls waiitng in a single queue. Don't you think it is too much.
>
> # I believe we may have unintentionally created a dependency between the *total number of handlers* and the call queue length. Should we reconsider this logic?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)