You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Hudson (Jira)" <ji...@apache.org> on 2020/03/23 17:52:00 UTC
[jira] [Commented] (HBASE-23949) refactor loadBalancer implements
for rsgroup balance by table to achieve overallbalanced
[ https://issues.apache.org/jira/browse/HBASE-23949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17064977#comment-17064977 ]
Hudson commented on HBASE-23949:
--------------------------------
Results for branch master
[build #1676 on builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1676/]: (x) *{color:red}-1 overall{color}*
----
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general report|https://builds.apache.org/job/HBase%20Nightly/job/master/1676//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) report|https://builds.apache.org/job/HBase%20Nightly/job/master/1676//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) report|https://builds.apache.org/job/HBase%20Nightly/job/master/1676//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11 report|https://builds.apache.org/job/HBase%20Nightly/job/master/1676//JDK11_Nightly_Build_Report/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for details|https://builds.apache.org/job/HBase%20Nightly/job/master/1676//artifact/output-integration/hadoop-2.log]. (note that this means we didn't run on Hadoop 3)
> refactor loadBalancer implements for rsgroup balance by table to achieve overallbalanced
> ------------------------------------------------------------------------------------------
>
> Key: HBASE-23949
> URL: https://issues.apache.org/jira/browse/HBASE-23949
> Project: HBase
> Issue Type: Bug
> Components: rsgroup
> Affects Versions: 2.2.0
> Reporter: niuyulin
> Assignee: niuyulin
> Priority: Major
>
> now can not achieve overallbalanced when use rsgroup balancer and by table is on,
> because balance every table actually use the clusterload only contain one table's load.
> we should use clusterload contain all this rsgroup table's load to balance overall
>
> hbase/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
> {code:java}
> public boolean balance(boolean force) throws IOException {
> ......
> boolean isByTable = getConfiguration().getBoolean("hbase.master.loadbalance.bytable", false);
> Map<TableName, Map<ServerName, List<RegionInfo>>> assignments =
> this.assignmentManager.getRegionStates()
> .getAssignmentsForBalancer(tableStateManager, this.serverManager.getOnlineServersList(),
> isByTable);
> for (Map<ServerName, List<RegionInfo>> serverMap : assignments.values()) {
> serverMap.keySet().removeAll(this.serverManager.getDrainingServersList());
> }
> //Give the balancer the current cluster state.
> this.balancer.setClusterMetrics(getClusterMetricsWithoutCoprocessor());
> this.balancer.setClusterLoad(assignments);
> List<RegionPlan> plans = new ArrayList<>();
> for (Entry<TableName, Map<ServerName, List<RegionInfo>>> e : assignments.entrySet()) {
> List<RegionPlan> partialPlans = this.balancer.balanceCluster(e.getKey(), e.getValue());
> if (partialPlans != null) {
> plans.addAll(partialPlans);
> }
> }
> {code}
> now do refactor:
> # add method 'balanceTable' in interface LoadBalancer
> # SimpleLoadBalancer and StochasticLoadBalancer do the real 'balanceTable' , and 'balanceTable' is not support in BaseLoadBalancer and RSGroupBasedLoadBalancer
> # RSGroupBasedLoadBalancer invoke balanceCluster , and pass GroupClusterLoad to internal balacer by group
> # internal balancer balance cluster invoke 'balanceTable'
--
This message was sent by Atlassian Jira
(v8.3.4#803005)