You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Reid Chan (Jira)" <ji...@apache.org> on 2022/01/18 03:14:00 UTC
[jira] [Resolved] (HBASE-26551) Add FastPath feature to HBase RWQueueRpcExecutor
[ https://issues.apache.org/jira/browse/HBASE-26551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Reid Chan resolved HBASE-26551.
-------------------------------
Hadoop Flags: Reviewed
Resolution: Fixed
> Add FastPath feature to HBase RWQueueRpcExecutor
> ------------------------------------------------
>
> Key: HBASE-26551
> URL: https://issues.apache.org/jira/browse/HBASE-26551
> Project: HBase
> Issue Type: Task
> Components: rpc, Scheduler
> Reporter: Yutong Xiao
> Assignee: Yutong Xiao
> Priority: Major
> Fix For: 2.5.0, 1.7.2, 2.6.0, 3.0.0-alpha-3
>
> Attachments: QueueTimeComparison.png, QueueTimeComparisonWithMax.png
>
>
> In ticket [HBASE-17808|https://issues.apache.org/jira/browse/HBASE-17808], the author introduced a fastpath implementation for RWQueueRpcExecutor. It aggregated 3 different independent RpcExecutor to implement the mechanism. This redundancy costed more memory and from its own performance test, it cannot outperform the original implementation. This time, I directly extended RWQueueRpcExecutor to implement the fast path mechanism. From my test result, it has a better queue time performance than before.
> YCSB Test:
> Constant Configurations:
> hbase.regionserver.handler.count: 1000
> hbase.ipc.server.callqueue.read.ratio: 0.5
> hbase.ipc.server.callqueue.handler.factor: 0.2
> Test Workload:
> YCSB: 50% Write, 25% Get, 25% Scan. Max Scan length: 1000.
> Client Threads: 100
> ||FastPathRWQueueRpcExecutor||RWQueueRpcExecutor||
> |[OVERALL], RunTime(ms), 909365
> [OVERALL], Throughput(ops/sec), 5498.3422498116815
> [TOTAL_GCS_PS_Scavenge], Count, 1208
> [TOTAL_GC_TIME_PS_Scavenge], Time(ms), 8006
> [TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 0.8803945610398465
> [TOTAL_GCS_PS_MarkSweep], Count, 2
> [TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 96
> [TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.010556817119638429
> [TOTAL_GCs], Count, 1210
> [TOTAL_GC_TIME], Time(ms), 8102
> [TOTAL_GC_TIME_%], Time(%), 0.8909513781594849
> [READ], Operations, 1248885
> [READ], AverageLatency(us), 14080.154160711354
> [READ], MinLatency(us), 269
> [READ], MaxLatency(us), 180735
> [READ], 95thPercentileLatency(us), 29775
> [READ], 99thPercentileLatency(us), 39391
> [READ], Return=OK, 1248885
> [CLEANUP], Operations, 200
> [CLEANUP], AverageLatency(us), 311.78
> [CLEANUP], MinLatency(us), 1
> [CLEANUP], MaxLatency(us), 59647
> [CLEANUP], 95thPercentileLatency(us), 26
> [CLEANUP], 99thPercentileLatency(us), 173
> [INSERT], Operations, 1251067
> [INSERT], AverageLatency(us), 14235.898240461942
> [INSERT], MinLatency(us), 393
> [INSERT], MaxLatency(us), 204159
> [INSERT], 95thPercentileLatency(us), 29919
> [INSERT], 99thPercentileLatency(us), 39647
> [INSERT], Return=OK, 1251067
> [UPDATE], Operations, 1249582
> [UPDATE], AverageLatency(us), 14166.923049467741
> [UPDATE], MinLatency(us), 321
> [UPDATE], MaxLatency(us), 203647
> [UPDATE], 95thPercentileLatency(us), 29855
> [UPDATE], 99thPercentileLatency(us), 39551
> [UPDATE], Return=OK, 1249582
> [SCAN], Operations, 1250466
> [SCAN], AverageLatency(us), 30056.68854251135
> [SCAN], MinLatency(us), 787
> [SCAN], MaxLatency(us), 509183
> [SCAN], 95thPercentileLatency(us), 57823
> [SCAN], 99thPercentileLatency(us), 74751
> [SCAN], Return=OK, 1250466|[OVERALL], RunTime(ms), 958763
> [OVERALL], Throughput(ops/sec), 5215.053146606617
> [TOTAL_GCS_PS_Scavenge], Count, 1264
> [TOTAL_GC_TIME_PS_Scavenge], Time(ms), 8680
> [TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 0.9053332262509086
> [TOTAL_GCS_PS_MarkSweep], Count, 1
> [TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 38
> [TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.00396344039142103
> [TOTAL_GCs], Count, 1265
> [TOTAL_GC_TIME], Time(ms), 8718
> [TOTAL_GC_TIME_%], Time(%), 0.9092966666423298
> [READ], Operations, 1250961
> [READ], AverageLatency(us), 14663.084518222391
> [READ], MinLatency(us), 320
> [READ], MaxLatency(us), 204415
> [READ], 95thPercentileLatency(us), 30815
> [READ], 99thPercentileLatency(us), 43071
> [READ], Return=OK, 1250961
> [CLEANUP], Operations, 200
> [CLEANUP], AverageLatency(us), 366.845
> [CLEANUP], MinLatency(us), 1
> [CLEANUP], MaxLatency(us), 70719
> [CLEANUP], 95thPercentileLatency(us), 36
> [CLEANUP], 99thPercentileLatency(us), 80
> [INSERT], Operations, 1248183
> [INSERT], AverageLatency(us), 14334.938754974231
> [INSERT], MinLatency(us), 390
> [INSERT], MaxLatency(us), 2828287
> [INSERT], 95thPercentileLatency(us), 30271
> [INSERT], 99thPercentileLatency(us), 41919
> [INSERT], Return=OK, 1248183
> [UPDATE], Operations, 1250212
> [UPDATE], AverageLatency(us), 14283.836318960304
> [UPDATE], MinLatency(us), 337
> [UPDATE], MaxLatency(us), 2828287
> [UPDATE], 95thPercentileLatency(us), 30255
> [UPDATE], 99thPercentileLatency(us), 41855
> [UPDATE], Return=OK, 1250212
> [SCAN], Operations, 1250644
> [SCAN], AverageLatency(us), 33153.01709839091
> [SCAN], MinLatency(us), 742
> [SCAN], MaxLatency(us), 645119
> [SCAN], 95thPercentileLatency(us), 62879
> [SCAN], 99thPercentileLatency(us), 80447
> [SCAN], Return=OK, 1250644|
> In the metrics screenshot, the first interval is the performance of RWQueueRpcExecutor and the second interval is FastPathRWQueueRpcExecutor.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)