[jira] [Commented] (HBASE-26945) Quotas causes too much load on meta for large clusters

["Duo Zhang (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-26945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17520851#comment-17520851 ] 

Duo Zhang commented on HBASE-26945:
-----------------------------------

Just go to master to ask the region count? In AssignmentManager we have the information in memory so it should be very fast to just get a count. And if master is dead, we can not split or merge regions then, so the region count of a table can not be changed?

> Quotas causes too much load on meta for large clusters
> ------------------------------------------------------
>
>                 Key: HBASE-26945
>                 URL: https://issues.apache.org/jira/browse/HBASE-26945
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Bryan Beaudreault
>            Priority: Minor
>
> We've been upgrading larger clusters to hbase 2, and ran into this issue where two equivalent clusters (one on 1.2 and the other on 2.4.x) running almost identical workloads, but the hbase2 cluster was doing way more meta requests.
> One of the reasons seems to be from https://issues.apache.org/jira/browse/HBASE-21820, which added an updateQuotaFactors method to the QuotaCache QuotaRefreshChore. This new method  calls MetaTableAcessor.scanMeta for every table in the cluster. This is happening on every regionserver, so if you have many tables and/or many regionservers you can easily start sending lots of traffic to meta. One way to offset this is to reduce frequency of hbase.quota.refresh.period, but this means you are less able to quickly respond to changes in load.
> We should figure out a caching or notification mechanism that can reduce the need scan meta so much. It seems like we're mainly scanning meta to get a count of total regions per table, which should not be changing so often.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)