[jira] [Comment Edited] (HBASE-27962) Introduce an AdaptiveFastPathRWRpcExecutor to make the W/R/S separations fit various workloads

["Yutong Xiao (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-27962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17743132#comment-17743132 ] 

Yutong Xiao edited comment on HBASE-27962 at 7/14/23 12:24 PM:
---------------------------------------------------------------

There is one thing I need to figure out that the shared handlers will run the queued requests first (that is to say, a shared write handler will process write requests with higher priority), and it only run other type requests when it is idling, when there is no idling handlers it could be regarded as an RWQueueExecutor. Cannot agree with you that this is a serious problem. And also not get the point that how it make slow requests hard to debug, we have a lot of metrics for debugging.  

Further, HBASE-27766 has no fastpath feature. The mode need to suffer the cost of queue lock competing. 

As for the isolate model of reads and write, we can also introduce two ratios, one control shared writers, one control shared readers. This could let the client escalate the handler allocation.

HBASE-27766 only consider idling scan handlers run gets, but HBASE-27962 also covers idling write handlers and idling get handlers. From my point of view, HBASE-27766 is redundant when we employed HBASE-27962.

Thank you~


was (Author: xytss123):
There is one thing I need to figure out that the shared handlers will run the queued requests first (that is to say, a shared write handler will process write requests with high priority), and it only run other type requests when it is idling, when there is no idling handlers it could be regarded as an RWQueueExecutor. Cannot agree with you that this is a serious problem. And also not get the point that how it make slow requests hard to debug, we have a lot of metrics for debugging.  

Further, HBASE-27766 has no fastpath feature. The mode need to suffer the cost of queue lock competing. 

As for the isolate model of reads and write, we can also introduce two ratios, one control shared writers, one control shared readers. This could let the client escalate the handler allocation.

HBASE-27766 only consider idling scan handlers run gets, but HBASE-27962 also covers idling write handlers and idling get handlers. From my point of view, HBASE-27766 is redundant when we employed HBASE-27962.

Thank you~

> Introduce an AdaptiveFastPathRWRpcExecutor to make the W/R/S separations fit various workloads 
> -----------------------------------------------------------------------------------------------
>
>                 Key: HBASE-27962
>                 URL: https://issues.apache.org/jira/browse/HBASE-27962
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Yutong Xiao
>            Assignee: Yutong Xiao
>            Priority: Major
>
> We currently use the FastPathRWQueueRpcExecutor. But the numbers of read/write handlers are fixed, which make the RegionServer performance not so good in our prod env.
> The logic of it is described below:
>  * The basic architecture is same as FastPathRWRpcExecutor.
>  * Introduced a float shared_ratio (0, 1.0), to indicate the ratio of the number of shared handlers. (for example, when we set the ratio to 0.2, if we have 100 handlers, 50 for write, 25 for get, 25 for scan, there will be 10 + 5 + 5 shared handlers and 40 isolated handlers for write, 20 for get and 20 for scan).
>  * Shared handler could run all the three kinds of requests.
>  * Shared handler will be shared only when it is idle.
>  * Shared handler is also bounded to a kind of RPCQueue, it will process the requests in that queue first.
> This improvement will improve the resource utility under various workloads and guarantee a level of R/W/S isolation for requests processing at the same time.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)