You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2020/03/12 07:34:06 UTC
[GitHub] [hbase] nyl3532016 opened a new pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
nyl3532016 opened a new pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279
…able
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] infraio merged pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
infraio merged pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] infraio commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
infraio commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#discussion_r391521272
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/LoadBalancer.java
##########
@@ -89,18 +89,18 @@
void setMasterServices(MasterServices masterServices);
/**
- * Perform the major balance operation
+ * Perform the major balance operation for cluster
* @return List of plans
*/
Review comment:
Add javadoc for the parameter of balanceCluster and balanceTable?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-598141706
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 0s | Docker mode activated. |
| -1 :x: | docker | 0m 11s | Docker failed to build yetus/hbase:c5aa2bc121. |
| Subsystem | Report/Notes |
|----------:|:-------------|
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/3/console |
| versions | git=2.17.1 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-602189613
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 2s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 7m 3s | master passed |
| +1 :green_heart: | compile | 1m 13s | master passed |
| -1 :x: | shadedjars | 0m 9s | branch has 7 errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 46s | hbase-server in master failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 45s | the patch passed |
| +1 :green_heart: | compile | 1m 14s | the patch passed |
| +1 :green_heart: | javac | 1m 14s | the patch passed |
| -1 :x: | shadedjars | 0m 9s | patch has 7 errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 43s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| -0 :warning: | unit | 5m 36s | hbase-server in the patch failed. |
| | | 26m 8s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/15/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 748e8433b014 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / ca533a77cb |
| Default Java | 2020-01-14 |
| shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/15/artifact/yetus-jdk11-hadoop3-check/output/branch-shadedjars.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/15/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/15/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/15/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/15/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/15/testReport/ |
| Max. process+thread count | 720 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/15/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-600068149
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 3m 57s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 44s | master passed |
| +1 :green_heart: | checkstyle | 1m 17s | master passed |
| -1 :x: | spotbugs | 2m 3s | hbase-server in master has 1 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 18s | the patch passed |
| -0 :warning: | checkstyle | 1m 17s | hbase-server: The patch generated 37 new + 408 unchanged - 9 fixed = 445 total (was 417) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 16m 8s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. |
| +1 :green_heart: | spotbugs | 2m 9s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. |
| | | 45m 7s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/11/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux b0685ed431e7 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 60de4c2e31 |
| spotbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/11/artifact/yetus-general-check/output/branch-spotbugs-hbase-server-warnings.html |
| checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/11/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| Max. process+thread count | 93 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/11/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-599443227
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 9s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 58s | master passed |
| +1 :green_heart: | compile | 0m 59s | master passed |
| +1 :green_heart: | shadedjars | 5m 20s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 41s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 14s | the patch passed |
| +1 :green_heart: | compile | 1m 20s | the patch passed |
| +1 :green_heart: | javac | 1m 20s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 30s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 44s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 95m 2s | hbase-server in the patch passed. |
| | | 124m 34s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/7/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 586fe2f3873b 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / f66cbe1a40 |
| Default Java | 1.8.0_232 |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/7/testReport/ |
| Max. process+thread count | 3958 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/7/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] infraio commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
infraio commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#discussion_r393023486
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupBasedLoadBalancer.java
##########
@@ -111,51 +109,44 @@ public void setMasterServices(MasterServices masterServices) {
this.masterServices = masterServices;
}
+ /**
+ * Override to balance by RSGroup
+ * not invoke {@link #balanceTable(TableName, Map)}
+ */
@Override
- public List<RegionPlan> balanceCluster(TableName tableName, Map<ServerName, List<RegionInfo>>
- clusterState) throws IOException {
- return balanceCluster(clusterState);
- }
-
- @Override
- public List<RegionPlan> balanceCluster(Map<ServerName, List<RegionInfo>> clusterState)
- throws IOException {
+ public List<RegionPlan> balanceCluster(
+ Map<TableName, Map<ServerName, List<RegionInfo>>> loadOfAllTable) throws IOException {
if (!isOnline()) {
throw new ConstraintException(
RSGroupInfoManager.class.getSimpleName() + " is not online, unable to perform balance");
}
// Calculate correct assignments and a list of RegionPlan for mis-placed regions
- Pair<Map<ServerName,List<RegionInfo>>, List<RegionPlan>> correctedStateAndRegionPlans =
- correctAssignments(clusterState);
- Map<ServerName,List<RegionInfo>> correctedState = correctedStateAndRegionPlans.getFirst();
+ Pair<Map<TableName, Map<ServerName, List<RegionInfo>>>, List<RegionPlan>> correctedStateAndRegionPlans =
+ correctAssignments(loadOfAllTable);
+ Map<TableName, Map<ServerName, List<RegionInfo>>> correctedLoadOfAllTable =
+ correctedStateAndRegionPlans.getFirst();
List<RegionPlan> regionPlans = correctedStateAndRegionPlans.getSecond();
-
+ RSGroupInfo defaultInfo = rsGroupInfoManager.getRSGroup(RSGroupInfo.DEFAULT_GROUP);
// Add RegionPlan
// for the regions which have been placed according to the region server group assignment
// into the movement list
try {
- // Record which region servers have been processed，so as to skip them after processed
- HashSet<ServerName> processedServers = new HashSet<>();
-
// For each rsgroup
for (RSGroupInfo rsgroup : rsGroupInfoManager.listRSGroups()) {
- Map<ServerName, List<RegionInfo>> groupClusterState = new HashMap<>();
- Map<TableName, Map<ServerName, List<RegionInfo>>> groupClusterLoad = new HashMap<>();
- for (ServerName server : clusterState.keySet()) { // for each region server
- if (!processedServers.contains(server) // server is not processed yet
- && rsgroup.containsServer(server.getAddress())) { // server belongs to this rsgroup
- List<RegionInfo> regionsOnServer = correctedState.get(server);
- groupClusterState.put(server, regionsOnServer);
-
- processedServers.add(server);
+ Map<TableName, Map<ServerName, List<RegionInfo>>> LoadOfTableInGroup =
Review comment:
LoadOfTableInGroup => loadOfTablesInGroup
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-598081833
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 38s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 47s | master passed |
| +1 :green_heart: | compile | 0m 59s | master passed |
| +1 :green_heart: | shadedjars | 4m 51s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 37s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 12s | the patch passed |
| +1 :green_heart: | compile | 0m 56s | the patch passed |
| +1 :green_heart: | javac | 0m 56s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 48s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 36s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 64m 46s | hbase-server in the patch passed. |
| | | 91m 18s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 502140d5dcab 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 26f3702735 |
| Default Java | 1.8.0_232 |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/1/testReport/ |
| Max. process+thread count | 5611 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/1/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-598121805
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 29s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 8s | master passed |
| +1 :green_heart: | checkstyle | 1m 24s | master passed |
| -1 :x: | spotbugs | 2m 13s | hbase-server in master has 1 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 38s | the patch passed |
| -0 :warning: | checkstyle | 1m 21s | hbase-server: The patch generated 34 new + 414 unchanged - 3 fixed = 448 total (was 417) |
| -0 :warning: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | hadoopcheck | 17m 51s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. |
| -1 :x: | spotbugs | 2m 22s | hbase-server generated 2 new + 1 unchanged - 0 fixed = 3 total (was 1) |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. |
| | | 46m 2s | |
| Reason | Tests |
|-------:|:------|
| FindBugs | module:hbase-server |
| | Inconsistent synchronization of org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.isByTable; locked 66% of time Unsynchronized access at BaseLoadBalancer.java:66% of time Unsynchronized access at BaseLoadBalancer.java:[line 1051] |
| | org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.balanceCluster(Map) makes inefficient use of keySet iterator instead of entrySet iterator At RSGroupBasedLoadBalancer.java:keySet iterator instead of entrySet iterator At RSGroupBasedLoadBalancer.java:[line 140] |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/2/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux 6c0da658ab55 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 26f3702735 |
| spotbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/2/artifact/yetus-general-check/output/branch-spotbugs-hbase-server-warnings.html |
| checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| whitespace | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/2/artifact/yetus-general-check/output/whitespace-eol.txt |
| spotbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/2/artifact/yetus-general-check/output/new-spotbugs-hbase-server.html |
| Max. process+thread count | 83 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/2/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-599648005
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 32s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 29s | master passed |
| +1 :green_heart: | compile | 1m 6s | master passed |
| +1 :green_heart: | shadedjars | 5m 18s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 36s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 50s | the patch passed |
| +1 :green_heart: | compile | 1m 2s | the patch passed |
| +1 :green_heart: | javac | 1m 2s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 19s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 38s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 122m 44s | hbase-server in the patch failed. |
| | | 152m 32s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/9/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux d942572d3fba 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / f66cbe1a40 |
| Default Java | 1.8.0_232 |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/9/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/9/testReport/ |
| Max. process+thread count | 4171 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/9/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] infraio commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
infraio commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#discussion_r391649307
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupBasedLoadBalancer.java
##########
@@ -427,4 +419,9 @@ public void postMasterStartupInitialize() {
public void updateBalancerStatus(boolean status) {
internalBalancer.updateBalancerStatus(status);
}
+
+ @Override
+ public List<RegionPlan> balanceTable(TableName tableName, Map<ServerName, List<RegionInfo>> clusterLoad){
Review comment:
Why not support here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-599228628
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 32s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 37s | master passed |
| +1 :green_heart: | compile | 0m 57s | master passed |
| +1 :green_heart: | shadedjars | 4m 46s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 36s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 13s | the patch passed |
| +1 :green_heart: | compile | 0m 59s | the patch passed |
| +1 :green_heart: | javac | 0m 59s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 44s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 36s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 61m 8s | hbase-server in the patch passed. |
| | | 87m 19s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/5/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux ff83c580e3f6 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / f66cbe1a40 |
| Default Java | 1.8.0_232 |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/5/testReport/ |
| Max. process+thread count | 5570 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/5/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-600409298
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 42s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 59s | master passed |
| +1 :green_heart: | checkstyle | 1m 21s | master passed |
| -1 :x: | spotbugs | 2m 7s | hbase-server in master has 1 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 26s | the patch passed |
| -0 :warning: | checkstyle | 1m 17s | hbase-server: The patch generated 37 new + 408 unchanged - 9 fixed = 445 total (was 417) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 11m 2s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2. |
| +1 :green_heart: | spotbugs | 2m 14s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. |
| | | 37m 33s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/12/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux 76249862803e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 9190a6b59a |
| spotbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/12/artifact/yetus-general-check/output/branch-spotbugs-hbase-server-warnings.html |
| checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/12/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| Max. process+thread count | 93 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/12/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] infraio commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
infraio commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#discussion_r391647557
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/SimpleLoadBalancer.java
##########
@@ -106,6 +106,12 @@ void setNextRegionForUnload(int nextRegionForUnload) {
}
+ @Override
+ public synchronized void setConf(Configuration conf) {
Review comment:
No need this?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-599510299
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 40s | Docker mode activated. |
| -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 25s | master passed |
| +1 :green_heart: | compile | 0m 59s | master passed |
| +1 :green_heart: | shadedjars | 4m 51s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 36s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 16s | the patch passed |
| +1 :green_heart: | compile | 0m 59s | the patch passed |
| +1 :green_heart: | javac | 0m 59s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 51s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 37s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 59m 21s | hbase-server in the patch passed. |
| | | 85m 38s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/8/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux ecf5c4ff319e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / f66cbe1a40 |
| Default Java | 1.8.0_232 |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/8/testReport/ |
| Max. process+thread count | 5927 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/8/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] nyl3532016 commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
nyl3532016 commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#discussion_r391999041
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupBasedLoadBalancer.java
##########
@@ -427,4 +419,9 @@ public void postMasterStartupInitialize() {
public void updateBalancerStatus(boolean status) {
internalBalancer.updateBalancerStatus(status);
}
+
+ @Override
+ public List<RegionPlan> balanceTable(TableName tableName, Map<ServerName, List<RegionInfo>> clusterLoad){
Review comment:
yes , it should support
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-602186092
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 8s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 7m 21s | master passed |
| +1 :green_heart: | compile | 1m 16s | master passed |
| -1 :x: | shadedjars | 0m 10s | branch has 7 errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 46s | hbase-server in master failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 7m 20s | the patch passed |
| +1 :green_heart: | compile | 1m 18s | the patch passed |
| +1 :green_heart: | javac | 1m 18s | the patch passed |
| -1 :x: | shadedjars | 0m 9s | patch has 7 errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 41s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 103m 10s | hbase-server in the patch passed. |
| | | 125m 10s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/14/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 3f2f73fdaef5 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / ca533a77cb |
| Default Java | 2020-01-14 |
| shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/14/artifact/yetus-jdk11-hadoop3-check/output/branch-shadedjars.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/14/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/14/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/14/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/14/testReport/ |
| Max. process+thread count | 4585 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/14/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] infraio commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
infraio commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#discussion_r392779299
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/SimpleLoadBalancer.java
##########
@@ -594,10 +597,13 @@ private void addRegionPlan(final MinMaxPriorityQueue<RegionPlan> regionsToMove,
regionsToReturn.add(rp);
}
+ /**
+ * Override to invoke {@link #setClusterLoad} before balance
+ */
@Override
- public List<RegionPlan> balanceCluster(TableName tableName,
- Map<ServerName, List<RegionInfo>> clusterState) throws HBaseIOException {
- LOG.debug("Start Generate Balance plan for table: " + tableName);
- return balanceCluster(clusterState);
+ public synchronized List<RegionPlan>
+ balanceCluster(Map<TableName, Map<ServerName, List<RegionInfo>>> clusterLoadPerTable) {
+ setClusterLoad(clusterLoadPerTable);
Review comment:
Add reason for why need to setClusterLoad?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-598159403
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 36s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 10s | master passed |
| +1 :green_heart: | checkstyle | 1m 21s | master passed |
| -1 :x: | spotbugs | 2m 13s | hbase-server in master has 1 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 38s | the patch passed |
| -0 :warning: | checkstyle | 1m 23s | hbase-server: The patch generated 34 new + 414 unchanged - 3 fixed = 448 total (was 417) |
| -0 :warning: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | hadoopcheck | 20m 26s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. |
| -1 :x: | spotbugs | 2m 30s | hbase-server generated 2 new + 1 unchanged - 0 fixed = 3 total (was 1) |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. |
| | | 49m 41s | |
| Reason | Tests |
|-------:|:------|
| FindBugs | module:hbase-server |
| | Inconsistent synchronization of org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.isByTable; locked 66% of time Unsynchronized access at BaseLoadBalancer.java:66% of time Unsynchronized access at BaseLoadBalancer.java:[line 1051] |
| | org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.balanceCluster(Map) makes inefficient use of keySet iterator instead of entrySet iterator At RSGroupBasedLoadBalancer.java:keySet iterator instead of entrySet iterator At RSGroupBasedLoadBalancer.java:[line 140] |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/3/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux bab5dffd69ba 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 26f3702735 |
| spotbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/3/artifact/yetus-general-check/output/branch-spotbugs-hbase-server-warnings.html |
| checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| whitespace | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/3/artifact/yetus-general-check/output/whitespace-eol.txt |
| spotbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/3/artifact/yetus-general-check/output/new-spotbugs-hbase-server.html |
| Max. process+thread count | 83 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/3/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-599223401
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 6s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 50s | master passed |
| +1 :green_heart: | checkstyle | 1m 23s | master passed |
| -1 :x: | spotbugs | 2m 12s | hbase-server in master has 1 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 39s | the patch passed |
| -0 :warning: | checkstyle | 1m 19s | hbase-server: The patch generated 32 new + 411 unchanged - 6 fixed = 443 total (was 417) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 17m 38s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. |
| -1 :x: | spotbugs | 2m 22s | hbase-server generated 2 new + 1 unchanged - 0 fixed = 3 total (was 1) |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. |
| | | 44m 58s | |
| Reason | Tests |
|-------:|:------|
| FindBugs | module:hbase-server |
| | Inconsistent synchronization of org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.isByTable; locked 66% of time Unsynchronized access at BaseLoadBalancer.java:66% of time Unsynchronized access at BaseLoadBalancer.java:[line 1051] |
| | org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.balanceCluster(Map) makes inefficient use of keySet iterator instead of entrySet iterator At RSGroupBasedLoadBalancer.java:keySet iterator instead of entrySet iterator At RSGroupBasedLoadBalancer.java:[line 143] |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/5/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux 0e750ab30175 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / f66cbe1a40 |
| spotbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/5/artifact/yetus-general-check/output/branch-spotbugs-hbase-server-warnings.html |
| checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/5/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| spotbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/5/artifact/yetus-general-check/output/new-spotbugs-hbase-server.html |
| Max. process+thread count | 83 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/5/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-598065709
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 37s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 42s | master passed |
| +1 :green_heart: | checkstyle | 1m 16s | master passed |
| -1 :x: | spotbugs | 2m 7s | hbase-server in master has 1 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 18s | the patch passed |
| -0 :warning: | checkstyle | 1m 15s | hbase-server: The patch generated 35 new + 414 unchanged - 3 fixed = 449 total (was 417) |
| -0 :warning: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | hadoopcheck | 20m 37s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. |
| -1 :x: | spotbugs | 3m 24s | hbase-server generated 2 new + 1 unchanged - 0 fixed = 3 total (was 1) |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 21s | The patch does not generate ASF License warnings. |
| | | 50m 23s | |
| Reason | Tests |
|-------:|:------|
| FindBugs | module:hbase-server |
| | Inconsistent synchronization of org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.isByTable; locked 66% of time Unsynchronized access at BaseLoadBalancer.java:66% of time Unsynchronized access at BaseLoadBalancer.java:[line 1051] |
| | org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.balanceCluster(Map) makes inefficient use of keySet iterator instead of entrySet iterator At RSGroupBasedLoadBalancer.java:keySet iterator instead of entrySet iterator At RSGroupBasedLoadBalancer.java:[line 140] |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux 60a57c042270 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 26f3702735 |
| spotbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/1/artifact/yetus-general-check/output/branch-spotbugs-hbase-server-warnings.html |
| checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| whitespace | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/1/artifact/yetus-general-check/output/whitespace-eol.txt |
| spotbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/1/artifact/yetus-general-check/output/new-spotbugs-hbase-server.html |
| Max. process+thread count | 93 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/1/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-598559537
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 7m 37s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 11s | master passed |
| +1 :green_heart: | checkstyle | 1m 23s | master passed |
| -1 :x: | spotbugs | 2m 10s | hbase-server in master has 1 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 37s | the patch passed |
| -0 :warning: | checkstyle | 1m 24s | hbase-server: The patch generated 33 new + 414 unchanged - 3 fixed = 447 total (was 417) |
| -0 :warning: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | hadoopcheck | 17m 37s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. |
| -1 :x: | spotbugs | 2m 21s | hbase-server generated 2 new + 1 unchanged - 0 fixed = 3 total (was 1) |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. |
| | | 51m 44s | |
| Reason | Tests |
|-------:|:------|
| FindBugs | module:hbase-server |
| | Inconsistent synchronization of org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.isByTable; locked 66% of time Unsynchronized access at BaseLoadBalancer.java:66% of time Unsynchronized access at BaseLoadBalancer.java:[line 1051] |
| | org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.balanceCluster(Map) makes inefficient use of keySet iterator instead of entrySet iterator At RSGroupBasedLoadBalancer.java:keySet iterator instead of entrySet iterator At RSGroupBasedLoadBalancer.java:[line 140] |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/4/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux 19ada0536889 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 1b163d98b9 |
| spotbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/4/artifact/yetus-general-check/output/branch-spotbugs-hbase-server-warnings.html |
| checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/4/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| whitespace | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/4/artifact/yetus-general-check/output/whitespace-eol.txt |
| spotbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/4/artifact/yetus-general-check/output/new-spotbugs-hbase-server.html |
| Max. process+thread count | 83 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/4/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] nyl3532016 commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
nyl3532016 commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#discussion_r391997216
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
##########
@@ -1639,6 +1641,44 @@ private void roundRobinAssignment(Cluster cluster, List<RegionInfo> regions,
}
}
+ private Map<ServerName, List<RegionInfo>> toEnsumbleTableClusterLoad(
+ Map<TableName, Map<ServerName, List<RegionInfo>>> clusterLoadPerTable) {
+ Map<ServerName, List<RegionInfo>> returnMap = new TreeMap<>();
+ for (Map<ServerName, List<RegionInfo>> serverNameListMap : clusterLoadPerTable.values()) {
+ serverNameListMap.forEach((serverName, regionInfoList) -> {
+ List<RegionInfo> regionInfos =
+ returnMap.computeIfAbsent(serverName, k -> new ArrayList<>());
+ regionInfos.addAll(regionInfoList);
+ });
+ }
+ return returnMap;
+ }
+
+ @Override
+ public List<RegionPlan> balanceTable(TableName tableName, Map<ServerName, List<RegionInfo>> clusterLoad){
Review comment:
yes ,it should be abstrat method
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-599229521
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 31s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 34s | master passed |
| +1 :green_heart: | compile | 1m 7s | master passed |
| +1 :green_heart: | shadedjars | 4m 58s | branch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 41s | hbase-server in master failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 15s | the patch passed |
| +1 :green_heart: | compile | 1m 7s | the patch passed |
| +1 :green_heart: | javac | 1m 7s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 55s | patch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 39s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| -0 :warning: | unit | 65m 1s | hbase-server in the patch failed. |
| | | 94m 8s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux ffc70fd8749a 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / f66cbe1a40 |
| Default Java | 2020-01-14 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/5/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/5/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/5/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/5/testReport/ |
| Max. process+thread count | 6152 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/5/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-599368251
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 48s | Docker mode activated. |
| -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 37s | master passed |
| +1 :green_heart: | compile | 1m 8s | master passed |
| +1 :green_heart: | shadedjars | 5m 8s | branch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 41s | hbase-server in master failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 6s | the patch passed |
| +1 :green_heart: | compile | 1m 6s | the patch passed |
| +1 :green_heart: | javac | 1m 6s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 0s | patch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 41s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| -0 :warning: | unit | 64m 5s | hbase-server in the patch failed. |
| | | 93m 32s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/6/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux a93350ce36c3 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / f66cbe1a40 |
| Default Java | 2020-01-14 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/6/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/6/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/6/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/6/testReport/ |
| Max. process+thread count | 6093 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/6/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-602028506
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 29s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 13s | master passed |
| +1 :green_heart: | checkstyle | 1m 16s | master passed |
| -1 :x: | spotbugs | 1m 58s | hbase-server in master has 1 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 13s | the patch passed |
| -0 :warning: | checkstyle | 1m 17s | hbase-server: The patch generated 37 new + 408 unchanged - 9 fixed = 445 total (was 417) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 10m 47s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2. |
| +1 :green_heart: | spotbugs | 2m 8s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 12s | The patch does not generate ASF License warnings. |
| | | 35m 38s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/13/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux 73b3ff39dd04 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 5592d0c191 |
| spotbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/13/artifact/yetus-general-check/output/branch-spotbugs-hbase-server-warnings.html |
| checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/13/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| Max. process+thread count | 93 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/13/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] nyl3532016 commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
nyl3532016 commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#discussion_r395982033
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupBasedLoadBalancer.java
##########
@@ -111,51 +109,45 @@ public void setMasterServices(MasterServices masterServices) {
this.masterServices = masterServices;
}
+ /**
+ * Override to balance by RSGroup
+ * not invoke {@link #balanceTable(TableName, Map)}
+ */
@Override
- public List<RegionPlan> balanceCluster(TableName tableName, Map<ServerName, List<RegionInfo>>
- clusterState) throws IOException {
- return balanceCluster(clusterState);
- }
-
- @Override
- public List<RegionPlan> balanceCluster(Map<ServerName, List<RegionInfo>> clusterState)
- throws IOException {
+ public List<RegionPlan> balanceCluster(
+ Map<TableName, Map<ServerName, List<RegionInfo>>> loadOfAllTable) throws IOException {
if (!isOnline()) {
throw new ConstraintException(
RSGroupInfoManager.class.getSimpleName() + " is not online, unable to perform balance");
}
// Calculate correct assignments and a list of RegionPlan for mis-placed regions
- Pair<Map<ServerName,List<RegionInfo>>, List<RegionPlan>> correctedStateAndRegionPlans =
- correctAssignments(clusterState);
- Map<ServerName,List<RegionInfo>> correctedState = correctedStateAndRegionPlans.getFirst();
+ Pair<Map<TableName, Map<ServerName, List<RegionInfo>>>, List<RegionPlan>> correctedStateAndRegionPlans =
+ correctAssignments(loadOfAllTable);
+ Map<TableName, Map<ServerName, List<RegionInfo>>> correctedLoadOfAllTable =
+ correctedStateAndRegionPlans.getFirst();
List<RegionPlan> regionPlans = correctedStateAndRegionPlans.getSecond();
-
+ RSGroupInfo defaultInfo = rsGroupInfoManager.getRSGroup(RSGroupInfo.DEFAULT_GROUP);
// Add RegionPlan
// for the regions which have been placed according to the region server group assignment
// into the movement list
try {
- // Record which region servers have been processed，so as to skip them after processed
- HashSet<ServerName> processedServers = new HashSet<>();
-
// For each rsgroup
for (RSGroupInfo rsgroup : rsGroupInfoManager.listRSGroups()) {
- Map<ServerName, List<RegionInfo>> groupClusterState = new HashMap<>();
- Map<TableName, Map<ServerName, List<RegionInfo>>> groupClusterLoad = new HashMap<>();
- for (ServerName server : clusterState.keySet()) { // for each region server
- if (!processedServers.contains(server) // server is not processed yet
- && rsgroup.containsServer(server.getAddress())) { // server belongs to this rsgroup
- List<RegionInfo> regionsOnServer = correctedState.get(server);
- groupClusterState.put(server, regionsOnServer);
-
- processedServers.add(server);
+ Map<TableName, Map<ServerName, List<RegionInfo>>> loadOfTablesInGroup =
+ new HashMap<>();
+ for (Map.Entry<TableName, Map<ServerName, List<RegionInfo>>> entry: correctedLoadOfAllTable.entrySet()) {
+ TableName tableName = entry.getKey();
+ RSGroupInfo targetRSGInfo = RSGroupUtil
+ .getRSGroupInfo(masterServices, rsGroupInfoManager, tableName).orElse(defaultInfo);
+ if (targetRSGInfo.getName().equals(rsgroup.getName())) {
+ loadOfTablesInGroup.put(tableName, entry.getValue());
}
}
-
- groupClusterLoad.put(HConstants.ENSEMBLE_TABLE_NAME, groupClusterState);
- this.internalBalancer.setClusterLoad(groupClusterLoad);
- List<RegionPlan> groupPlans = this.internalBalancer
- .balanceCluster(groupClusterState);
+ List<RegionPlan> groupPlans = null;
+ if (!loadOfTablesInGroup.isEmpty()) {
Review comment:
well , I add some log
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-602040582
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 2m 44s | Docker mode activated. |
| -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 11m 8s | master passed |
| +1 :green_heart: | compile | 1m 59s | master passed |
| -1 :x: | shadedjars | 0m 13s | branch has 7 errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 1m 2s | hbase-server in master failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 10m 31s | the patch passed |
| +1 :green_heart: | compile | 1m 58s | the patch passed |
| +1 :green_heart: | javac | 1m 58s | the patch passed |
| -1 :x: | shadedjars | 0m 23s | patch has 7 errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 1m 12s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 110m 46s | hbase-server in the patch passed. |
| | | 144m 1s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/13/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux dce3cd43d759 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 5592d0c191 |
| Default Java | 2020-01-14 |
| shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/13/artifact/yetus-jdk11-hadoop3-check/output/branch-shadedjars.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/13/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/13/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/13/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/13/testReport/ |
| Max. process+thread count | 4318 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/13/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] nyl3532016 commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
nyl3532016 commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#discussion_r391999041
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupBasedLoadBalancer.java
##########
@@ -427,4 +419,9 @@ public void postMasterStartupInitialize() {
public void updateBalancerStatus(boolean status) {
internalBalancer.updateBalancerStatus(status);
}
+
+ @Override
+ public List<RegionPlan> balanceTable(TableName tableName, Map<ServerName, List<RegionInfo>> clusterLoad){
Review comment:
yes , I think 'balanceTable' should not be method of LoadBalancer , it should be method of BaseLoadBalancer , and RSGroupBasedLoadBalancer no need implement it
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-600090342
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 30s | Docker mode activated. |
| -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 7m 12s | master passed |
| +1 :green_heart: | compile | 1m 9s | master passed |
| -1 :x: | shadedjars | 0m 12s | branch has 7 errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 49s | hbase-server in master failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 7m 26s | the patch passed |
| +1 :green_heart: | compile | 1m 13s | the patch passed |
| +1 :green_heart: | javac | 1m 13s | the patch passed |
| -1 :x: | shadedjars | 0m 10s | patch has 7 errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 40s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 66m 42s | hbase-server in the patch passed. |
| | | 88m 15s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/11/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 1bb908d4782e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 60de4c2e31 |
| Default Java | 2020-01-14 |
| shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/11/artifact/yetus-jdk11-hadoop3-check/output/branch-shadedjars.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/11/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/11/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/11/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/11/testReport/ |
| Max. process+thread count | 7218 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/11/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] nyl3532016 commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
nyl3532016 commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#discussion_r392779758
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/SimpleLoadBalancer.java
##########
@@ -594,10 +597,13 @@ private void addRegionPlan(final MinMaxPriorityQueue<RegionPlan> regionsToMove,
regionsToReturn.add(rp);
}
+ /**
+ * Override to invoke {@link #setClusterLoad} before balance
+ */
@Override
- public List<RegionPlan> balanceCluster(TableName tableName,
- Map<ServerName, List<RegionInfo>> clusterState) throws HBaseIOException {
- LOG.debug("Start Generate Balance plan for table: " + tableName);
- return balanceCluster(clusterState);
+ public synchronized List<RegionPlan>
+ balanceCluster(Map<TableName, Map<ServerName, List<RegionInfo>>> clusterLoadPerTable) {
+ setClusterLoad(clusterLoadPerTable);
Review comment:
ok
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] nyl3532016 commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
nyl3532016 commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#discussion_r395981969
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
##########
@@ -1639,6 +1637,44 @@ private void roundRobinAssignment(Cluster cluster, List<RegionInfo> regions,
}
}
+ private Map<ServerName, List<RegionInfo>> toEnsumbleTableLoad(
+ Map<TableName, Map<ServerName, List<RegionInfo>>> LoadOfAllTable) {
+ Map<ServerName, List<RegionInfo>> returnMap = new TreeMap<>();
+ for (Map<ServerName, List<RegionInfo>> serverNameListMap : LoadOfAllTable.values()) {
+ serverNameListMap.forEach((serverName, regionInfoList) -> {
+ List<RegionInfo> regionInfos =
+ returnMap.computeIfAbsent(serverName, k -> new ArrayList<>());
+ regionInfos.addAll(regionInfoList);
+ });
+ }
+ return returnMap;
+ }
+
+ @Override
+ public abstract List<RegionPlan> balanceTable(TableName tableName,
+ Map<ServerName, List<RegionInfo>> loadOfOneTable);
+
+
+ @Override
+ public List<RegionPlan>
+ balanceCluster(Map<TableName, Map<ServerName, List<RegionInfo>>> loadOfAllTable) {
+ if (isByTable) {
+ List<RegionPlan> result = new ArrayList<>();
+ loadOfAllTable.forEach((tableName, loadOfOneTable) -> {
+ LOG.debug("Start Generate Balance plan for table: " + tableName);
Review comment:
ok , I will change this
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-599351608
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 51s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 16s | master passed |
| +1 :green_heart: | compile | 1m 6s | master passed |
| +1 :green_heart: | shadedjars | 5m 24s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 39s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 50s | the patch passed |
| +1 :green_heart: | compile | 1m 4s | the patch passed |
| +1 :green_heart: | javac | 1m 4s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 19s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 37s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 6m 33s | hbase-server in the patch failed. |
| | | 35m 54s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/6/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 695d26bcb1ed 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / f66cbe1a40 |
| Default Java | 1.8.0_232 |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/6/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/6/testReport/ |
| Max. process+thread count | 763 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/6/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-598571421
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 42s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 8m 0s | master passed |
| +1 :green_heart: | compile | 1m 11s | master passed |
| +1 :green_heart: | shadedjars | 5m 14s | branch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 42s | hbase-server in master failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 11s | the patch passed |
| +1 :green_heart: | compile | 1m 5s | the patch passed |
| +1 :green_heart: | javac | 1m 5s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 53s | patch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 40s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 66m 0s | hbase-server in the patch passed. |
| | | 96m 57s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux e8efaf0e5267 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 1b163d98b9 |
| Default Java | 2020-01-14 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/4/testReport/ |
| Max. process+thread count | 5829 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/4/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-599580813
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 37s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 57s | master passed |
| +1 :green_heart: | checkstyle | 1m 17s | master passed |
| -1 :x: | spotbugs | 2m 11s | hbase-server in master has 1 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 22s | the patch passed |
| -0 :warning: | checkstyle | 1m 15s | hbase-server: The patch generated 36 new + 408 unchanged - 9 fixed = 444 total (was 417) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 17m 9s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. |
| -1 :x: | spotbugs | 2m 27s | hbase-server generated 2 new + 1 unchanged - 0 fixed = 3 total (was 1) |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. |
| | | 43m 37s | |
| Reason | Tests |
|-------:|:------|
| FindBugs | module:hbase-server |
| | Inconsistent synchronization of org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.isByTable; locked 66% of time Unsynchronized access at BaseLoadBalancer.java:66% of time Unsynchronized access at BaseLoadBalancer.java:[line 1051] |
| | org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.balanceCluster(Map) makes inefficient use of keySet iterator instead of entrySet iterator At RSGroupBasedLoadBalancer.java:keySet iterator instead of entrySet iterator At RSGroupBasedLoadBalancer.java:[line 143] |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/9/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux a1c45d3c9069 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / f66cbe1a40 |
| spotbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/9/artifact/yetus-general-check/output/branch-spotbugs-hbase-server-warnings.html |
| checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/9/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| spotbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/9/artifact/yetus-general-check/output/new-spotbugs-hbase-server.html |
| Max. process+thread count | 93 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/9/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-602038399
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 8s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 5s | master passed |
| +1 :green_heart: | compile | 1m 2s | master passed |
| +1 :green_heart: | shadedjars | 5m 26s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 36s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 52s | the patch passed |
| +1 :green_heart: | compile | 1m 2s | the patch passed |
| +1 :green_heart: | javac | 1m 2s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 29s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 37s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 93m 32s | hbase-server in the patch passed. |
| | | 122m 39s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/13/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 567b1fd86bdc 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 5592d0c191 |
| Default Java | 1.8.0_232 |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/13/testReport/ |
| Max. process+thread count | 4322 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/13/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-600043333
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 4m 7s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 33s | master passed |
| +1 :green_heart: | compile | 1m 5s | master passed |
| -1 :x: | shadedjars | 0m 10s | branch has 7 errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 43s | hbase-server in master failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 31s | the patch passed |
| +1 :green_heart: | compile | 1m 9s | the patch passed |
| +1 :green_heart: | javac | 1m 9s | the patch passed |
| -1 :x: | shadedjars | 0m 11s | patch has 7 errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 44s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| -0 :warning: | unit | 63m 17s | hbase-server in the patch failed. |
| | | 86m 59s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/10/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux f9de7489d5c8 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 60de4c2e31 |
| Default Java | 2020-01-14 |
| shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/10/artifact/yetus-jdk11-hadoop3-check/output/branch-shadedjars.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/10/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/10/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/10/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/10/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/10/testReport/ |
| Max. process+thread count | 6888 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/10/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] nyl3532016 commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
nyl3532016 commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#discussion_r395981938
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
##########
@@ -1639,6 +1637,44 @@ private void roundRobinAssignment(Cluster cluster, List<RegionInfo> regions,
}
}
+ private Map<ServerName, List<RegionInfo>> toEnsumbleTableLoad(
+ Map<TableName, Map<ServerName, List<RegionInfo>>> LoadOfAllTable) {
+ Map<ServerName, List<RegionInfo>> returnMap = new TreeMap<>();
+ for (Map<ServerName, List<RegionInfo>> serverNameListMap : LoadOfAllTable.values()) {
+ serverNameListMap.forEach((serverName, regionInfoList) -> {
+ List<RegionInfo> regionInfos =
+ returnMap.computeIfAbsent(serverName, k -> new ArrayList<>());
+ regionInfos.addAll(regionInfoList);
+ });
+ }
+ return returnMap;
+ }
+
+ @Override
+ public abstract List<RegionPlan> balanceTable(TableName tableName,
+ Map<ServerName, List<RegionInfo>> loadOfOneTable);
+
+
+ @Override
+ public List<RegionPlan>
+ balanceCluster(Map<TableName, Map<ServerName, List<RegionInfo>>> loadOfAllTable) {
+ if (isByTable) {
+ List<RegionPlan> result = new ArrayList<>();
+ loadOfAllTable.forEach((tableName, loadOfOneTable) -> {
+ LOG.debug("Start Generate Balance plan for table: " + tableName);
+ List<RegionPlan> partialPlans = balanceTable(tableName, loadOfOneTable);
+ if (partialPlans != null) {
+ result.addAll(partialPlans);
+ }
+ });
+ return result;
+ } else {
+ LOG.debug("Start Generate Balance plan for table: " + HConstants.ENSEMBLE_TABLE_NAME);
Review comment:
yes, you are right
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-602186008
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 6s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 43s | master passed |
| +1 :green_heart: | compile | 1m 0s | master passed |
| +1 :green_heart: | shadedjars | 5m 21s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 36s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 50s | the patch passed |
| +1 :green_heart: | compile | 1m 1s | the patch passed |
| +1 :green_heart: | javac | 1m 1s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 27s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 35s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 96m 3s | hbase-server in the patch passed. |
| | | 124m 35s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/14/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 1bef44f9cf53 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / ca533a77cb |
| Default Java | 1.8.0_232 |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/14/testReport/ |
| Max. process+thread count | 4275 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/14/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-602175176
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 33s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 9s | master passed |
| +1 :green_heart: | checkstyle | 1m 23s | master passed |
| -1 :x: | spotbugs | 2m 11s | hbase-server in master has 1 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 31s | the patch passed |
| -0 :warning: | checkstyle | 1m 17s | hbase-server: The patch generated 3 new + 402 unchanged - 15 fixed = 405 total (was 417) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 11m 5s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2. |
| +1 :green_heart: | spotbugs | 2m 17s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. |
| | | 38m 4s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/14/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux 1de3d3d287da 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / ca533a77cb |
| spotbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/14/artifact/yetus-general-check/output/branch-spotbugs-hbase-server-warnings.html |
| checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/14/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| Max. process+thread count | 93 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/14/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] infraio commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
infraio commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#discussion_r391650288
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/favored/FavoredNodeLoadBalancer.java
##########
@@ -87,7 +87,8 @@ public synchronized void initialize() throws HBaseIOException {
}
@Override
- public List<RegionPlan> balanceCluster(Map<ServerName, List<RegionInfo>> clusterState) {
+ public List<RegionPlan>
+ balanceCluster(Map<TableName, Map<ServerName, List<RegionInfo>>> clusterLoadPerTable) {
Review comment:
The balancer should only need to override balanceTable method?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-602191456
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 5s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 55s | master passed |
| +1 :green_heart: | checkstyle | 1m 21s | master passed |
| -1 :x: | spotbugs | 2m 11s | hbase-server in master has 1 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 48s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 19s | hbase-server: The patch generated 0 new + 401 unchanged - 16 fixed = 401 total (was 417) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 11m 57s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2. |
| +1 :green_heart: | spotbugs | 2m 19s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 11s | The patch does not generate ASF License warnings. |
| | | 39m 34s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/15/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux f2ef028aafcb 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / ca533a77cb |
| spotbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/15/artifact/yetus-general-check/output/branch-spotbugs-hbase-server-warnings.html |
| Max. process+thread count | 83 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/15/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-599593401
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 51s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 7m 39s | master passed |
| +1 :green_heart: | compile | 1m 22s | master passed |
| +1 :green_heart: | shadedjars | 5m 44s | branch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 44s | hbase-server in master failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 8m 5s | the patch passed |
| +1 :green_heart: | compile | 1m 53s | the patch passed |
| +1 :green_heart: | javac | 1m 53s | the patch passed |
| +1 :green_heart: | shadedjars | 7m 59s | patch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 1m 4s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| -0 :warning: | unit | 11m 40s | hbase-server in the patch failed. |
| | | 49m 10s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/9/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux c4f16b0b4825 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / f66cbe1a40 |
| Default Java | 2020-01-14 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/9/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/9/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/9/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/9/testReport/ |
| Max. process+thread count | 549 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/9/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-599534748
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 2m 44s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 9m 34s | master passed |
| +1 :green_heart: | compile | 1m 18s | master passed |
| +1 :green_heart: | shadedjars | 5m 41s | branch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 43s | hbase-server in master failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 7m 7s | the patch passed |
| +1 :green_heart: | compile | 1m 13s | the patch passed |
| +1 :green_heart: | javac | 1m 13s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 33s | patch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 41s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 97m 7s | hbase-server in the patch passed. |
| | | 133m 23s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/8/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux fa031b82234d 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / f66cbe1a40 |
| Default Java | 2020-01-14 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/8/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/8/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/8/testReport/ |
| Max. process+thread count | 4555 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/8/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-600065856
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 6m 33s | Docker mode activated. |
| -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 20s | master passed |
| +1 :green_heart: | compile | 1m 1s | master passed |
| +1 :green_heart: | shadedjars | 5m 25s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 39s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 48s | the patch passed |
| +1 :green_heart: | compile | 0m 59s | the patch passed |
| +1 :green_heart: | javac | 0m 59s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 30s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 41s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 6m 7s | hbase-server in the patch failed. |
| | | 40m 24s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/11/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 9c587d883b84 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 60de4c2e31 |
| Default Java | 1.8.0_232 |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/11/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/11/testReport/ |
| Max. process+thread count | 555 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/11/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] nyl3532016 commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
nyl3532016 commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#discussion_r392014575
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/favored/FavoredNodeLoadBalancer.java
##########
@@ -87,7 +87,8 @@ public synchronized void initialize() throws HBaseIOException {
}
@Override
- public List<RegionPlan> balanceCluster(Map<ServerName, List<RegionInfo>> clusterState) {
+ public List<RegionPlan>
+ balanceCluster(Map<TableName, Map<ServerName, List<RegionInfo>>> clusterLoadPerTable) {
Review comment:
yes , but method 'balanceCluster' of this class not invoke 'balanceTable' , so just override balanceTable with null return
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-599354567
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 51s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 29s | master passed |
| +1 :green_heart: | checkstyle | 1m 30s | master passed |
| -1 :x: | spotbugs | 2m 19s | hbase-server in master has 1 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 50s | the patch passed |
| -0 :warning: | checkstyle | 1m 21s | hbase-server: The patch generated 32 new + 411 unchanged - 6 fixed = 443 total (was 417) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 19m 50s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. |
| -1 :x: | spotbugs | 2m 22s | hbase-server generated 2 new + 1 unchanged - 0 fixed = 3 total (was 1) |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. |
| | | 49m 3s | |
| Reason | Tests |
|-------:|:------|
| FindBugs | module:hbase-server |
| | Inconsistent synchronization of org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.isByTable; locked 66% of time Unsynchronized access at BaseLoadBalancer.java:66% of time Unsynchronized access at BaseLoadBalancer.java:[line 1051] |
| | org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.balanceCluster(Map) makes inefficient use of keySet iterator instead of entrySet iterator At RSGroupBasedLoadBalancer.java:keySet iterator instead of entrySet iterator At RSGroupBasedLoadBalancer.java:[line 143] |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/6/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux e882dd811d56 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / f66cbe1a40 |
| spotbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/6/artifact/yetus-general-check/output/branch-spotbugs-hbase-server-warnings.html |
| checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/6/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| spotbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/6/artifact/yetus-general-check/output/new-spotbugs-hbase-server.html |
| Max. process+thread count | 83 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/6/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] nyl3532016 commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
nyl3532016 commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#discussion_r392002784
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
##########
@@ -360,10 +356,9 @@ protected boolean needsBalance(Cluster cluster) {
}
@Override
- public synchronized List<RegionPlan> balanceCluster(TableName tableName, Map<ServerName,
- List<RegionInfo>> clusterState) {
- this.tableName = tableName;
- return balanceCluster(clusterState);
+ public synchronized List<RegionPlan> balanceCluster(Map<TableName, Map<ServerName,
+ List<RegionInfo>>> clusterLoadPerTable) {
+ return super.balanceCluster(clusterLoadPerTable);
Review comment:
yes, I remove it
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] nyl3532016 commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
nyl3532016 commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#discussion_r391996237
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
##########
@@ -1639,6 +1641,44 @@ private void roundRobinAssignment(Cluster cluster, List<RegionInfo> regions,
}
}
+ private Map<ServerName, List<RegionInfo>> toEnsumbleTableClusterLoad(
+ Map<TableName, Map<ServerName, List<RegionInfo>>> clusterLoadPerTable) {
+ Map<ServerName, List<RegionInfo>> returnMap = new TreeMap<>();
+ for (Map<ServerName, List<RegionInfo>> serverNameListMap : clusterLoadPerTable.values()) {
+ serverNameListMap.forEach((serverName, regionInfoList) -> {
+ List<RegionInfo> regionInfos =
+ returnMap.computeIfAbsent(serverName, k -> new ArrayList<>());
+ regionInfos.addAll(regionInfoList);
+ });
+ }
+ return returnMap;
+ }
+
+ @Override
+ public List<RegionPlan> balanceTable(TableName tableName, Map<ServerName, List<RegionInfo>> clusterLoad){
+ throw new UnsupportedOperationException("balanceTable is not support in BaseLoadBalancer");
+ }
+
+ @Override
+ public synchronized List<RegionPlan>
Review comment:
method of the extends class need be synchronized
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] binlijin commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
binlijin commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#discussion_r395399054
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
##########
@@ -1639,6 +1637,44 @@ private void roundRobinAssignment(Cluster cluster, List<RegionInfo> regions,
}
}
+ private Map<ServerName, List<RegionInfo>> toEnsumbleTableLoad(
+ Map<TableName, Map<ServerName, List<RegionInfo>>> LoadOfAllTable) {
+ Map<ServerName, List<RegionInfo>> returnMap = new TreeMap<>();
+ for (Map<ServerName, List<RegionInfo>> serverNameListMap : LoadOfAllTable.values()) {
+ serverNameListMap.forEach((serverName, regionInfoList) -> {
+ List<RegionInfo> regionInfos =
+ returnMap.computeIfAbsent(serverName, k -> new ArrayList<>());
+ regionInfos.addAll(regionInfoList);
+ });
+ }
+ return returnMap;
+ }
+
+ @Override
+ public abstract List<RegionPlan> balanceTable(TableName tableName,
+ Map<ServerName, List<RegionInfo>> loadOfOneTable);
+
+
+ @Override
+ public List<RegionPlan>
+ balanceCluster(Map<TableName, Map<ServerName, List<RegionInfo>>> loadOfAllTable) {
+ if (isByTable) {
+ List<RegionPlan> result = new ArrayList<>();
+ loadOfAllTable.forEach((tableName, loadOfOneTable) -> {
+ LOG.debug("Start Generate Balance plan for table: " + tableName);
+ List<RegionPlan> partialPlans = balanceTable(tableName, loadOfOneTable);
+ if (partialPlans != null) {
+ result.addAll(partialPlans);
+ }
+ });
+ return result;
+ } else {
+ LOG.debug("Start Generate Balance plan for table: " + HConstants.ENSEMBLE_TABLE_NAME);
Review comment:
There is no Table ENSEMBLE_TABLE_NAME actually, need to change this log?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-600418645
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 30s | Docker mode activated. |
| -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 35s | master passed |
| +1 :green_heart: | compile | 1m 9s | master passed |
| -1 :x: | shadedjars | 0m 11s | branch has 7 errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 41s | hbase-server in master failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 17s | the patch passed |
| +1 :green_heart: | compile | 1m 5s | the patch passed |
| +1 :green_heart: | javac | 1m 5s | the patch passed |
| -1 :x: | shadedjars | 0m 12s | patch has 7 errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 39s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| -0 :warning: | unit | 62m 43s | hbase-server in the patch failed. |
| | | 82m 31s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/12/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 3c1c83cf74d3 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 9190a6b59a |
| Default Java | 2020-01-14 |
| shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/12/artifact/yetus-jdk11-hadoop3-check/output/branch-shadedjars.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/12/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/12/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/12/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/12/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/12/testReport/ |
| Max. process+thread count | 5912 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/12/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-600426181
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 3s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 19s | master passed |
| +1 :green_heart: | compile | 1m 2s | master passed |
| +1 :green_heart: | shadedjars | 5m 26s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 37s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 52s | the patch passed |
| +1 :green_heart: | compile | 1m 2s | the patch passed |
| +1 :green_heart: | javac | 1m 2s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 22s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 35s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 88m 50s | hbase-server in the patch passed. |
| | | 118m 0s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/12/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux a37a67f33d79 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 9190a6b59a |
| Default Java | 1.8.0_232 |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/12/testReport/ |
| Max. process+thread count | 4458 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/12/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] infraio commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
infraio commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#discussion_r391648185
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
##########
@@ -360,10 +356,9 @@ protected boolean needsBalance(Cluster cluster) {
}
@Override
- public synchronized List<RegionPlan> balanceCluster(TableName tableName, Map<ServerName,
- List<RegionInfo>> clusterState) {
- this.tableName = tableName;
- return balanceCluster(clusterState);
+ public synchronized List<RegionPlan> balanceCluster(Map<TableName, Map<ServerName,
+ List<RegionInfo>>> clusterLoadPerTable) {
+ return super.balanceCluster(clusterLoadPerTable);
Review comment:
If call super method directly, no need to add this method here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-598178857
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 36s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 11s | master passed |
| +1 :green_heart: | compile | 1m 6s | master passed |
| +1 :green_heart: | shadedjars | 5m 1s | branch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 40s | hbase-server in master failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 12s | the patch passed |
| +1 :green_heart: | compile | 1m 7s | the patch passed |
| +1 :green_heart: | javac | 1m 7s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 55s | patch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 39s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| -0 :warning: | unit | 65m 48s | hbase-server in the patch failed. |
| | | 94m 26s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 311b23cf659d 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 26f3702735 |
| Default Java | 2020-01-14 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/3/testReport/ |
| Max. process+thread count | 6635 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/3/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-598101970
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 8s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 7m 16s | master passed |
| +1 :green_heart: | compile | 1m 12s | master passed |
| +1 :green_heart: | shadedjars | 5m 36s | branch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 42s | hbase-server in master failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 7m 2s | the patch passed |
| +1 :green_heart: | compile | 1m 12s | the patch passed |
| +1 :green_heart: | javac | 1m 12s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 26s | patch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 39s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| -0 :warning: | unit | 107m 3s | hbase-server in the patch failed. |
| | | 139m 14s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 0a05c6d80616 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 26f3702735 |
| Default Java | 2020-01-14 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/1/testReport/ |
| Max. process+thread count | 4293 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/1/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] infraio commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
infraio commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#discussion_r395969064
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
##########
@@ -1639,6 +1637,44 @@ private void roundRobinAssignment(Cluster cluster, List<RegionInfo> regions,
}
}
+ private Map<ServerName, List<RegionInfo>> toEnsumbleTableLoad(
+ Map<TableName, Map<ServerName, List<RegionInfo>>> LoadOfAllTable) {
+ Map<ServerName, List<RegionInfo>> returnMap = new TreeMap<>();
+ for (Map<ServerName, List<RegionInfo>> serverNameListMap : LoadOfAllTable.values()) {
+ serverNameListMap.forEach((serverName, regionInfoList) -> {
+ List<RegionInfo> regionInfos =
+ returnMap.computeIfAbsent(serverName, k -> new ArrayList<>());
+ regionInfos.addAll(regionInfoList);
+ });
+ }
+ return returnMap;
+ }
+
+ @Override
+ public abstract List<RegionPlan> balanceTable(TableName tableName,
+ Map<ServerName, List<RegionInfo>> loadOfOneTable);
+
+
+ @Override
+ public List<RegionPlan>
+ balanceCluster(Map<TableName, Map<ServerName, List<RegionInfo>>> loadOfAllTable) {
+ if (isByTable) {
+ List<RegionPlan> result = new ArrayList<>();
+ loadOfAllTable.forEach((tableName, loadOfOneTable) -> {
+ LOG.debug("Start Generate Balance plan for table: " + tableName);
Review comment:
LOG.info("Start generate balance plan for table: {}", tableName);
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] infraio commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
infraio commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#discussion_r392781215
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupBasedLoadBalancer.java
##########
@@ -427,4 +421,19 @@ public void postMasterStartupInitialize() {
public void updateBalancerStatus(boolean status) {
internalBalancer.updateBalancerStatus(status);
}
+
+ /**
+ * can achieve table balanced rather than overall balanced
+ */
+ @Override
+ public List<RegionPlan> balanceTable(TableName tableName,
+ Map<ServerName, List<RegionInfo>> clusterLoad) {
+ Map<TableName, Map<ServerName, List<RegionInfo>>> clusterLoadThisTable = new HashMap<>();
+ clusterLoadThisTable.put(tableName, clusterLoad);
+ try {
+ return balanceCluster(clusterLoadThisTable);
Review comment:
It is wired to call balanceClsuter here. Maybe abstract a new method balanceTableInRSGroup. And balanceTable and balanceClsuter both call the new method.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-599407618
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 31s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 6s | master passed |
| +1 :green_heart: | checkstyle | 1m 13s | master passed |
| -1 :x: | spotbugs | 2m 12s | hbase-server in master has 1 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 17s | the patch passed |
| -0 :warning: | checkstyle | 1m 18s | hbase-server: The patch generated 36 new + 408 unchanged - 9 fixed = 444 total (was 417) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 16m 44s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. |
| -1 :x: | spotbugs | 2m 32s | hbase-server generated 2 new + 1 unchanged - 0 fixed = 3 total (was 1) |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 12s | The patch does not generate ASF License warnings. |
| | | 41m 49s | |
| Reason | Tests |
|-------:|:------|
| FindBugs | module:hbase-server |
| | Inconsistent synchronization of org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.isByTable; locked 66% of time Unsynchronized access at BaseLoadBalancer.java:66% of time Unsynchronized access at BaseLoadBalancer.java:[line 1051] |
| | org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.balanceCluster(Map) makes inefficient use of keySet iterator instead of entrySet iterator At RSGroupBasedLoadBalancer.java:keySet iterator instead of entrySet iterator At RSGroupBasedLoadBalancer.java:[line 143] |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/7/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux add793689b4a 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / f66cbe1a40 |
| spotbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/7/artifact/yetus-general-check/output/branch-spotbugs-hbase-server-warnings.html |
| checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/7/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| spotbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/7/artifact/yetus-general-check/output/new-spotbugs-hbase-server.html |
| Max. process+thread count | 93 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/7/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-599501070
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 2m 35s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 8m 32s | master passed |
| +1 :green_heart: | checkstyle | 1m 58s | master passed |
| -1 :x: | spotbugs | 3m 16s | hbase-server in master has 1 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 8m 2s | the patch passed |
| -0 :warning: | checkstyle | 1m 54s | hbase-server: The patch generated 36 new + 408 unchanged - 9 fixed = 444 total (was 417) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 24m 33s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. |
| -1 :x: | spotbugs | 3m 35s | hbase-server generated 2 new + 1 unchanged - 0 fixed = 3 total (was 1) |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 22s | The patch does not generate ASF License warnings. |
| | | 63m 44s | |
| Reason | Tests |
|-------:|:------|
| FindBugs | module:hbase-server |
| | Inconsistent synchronization of org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.isByTable; locked 66% of time Unsynchronized access at BaseLoadBalancer.java:66% of time Unsynchronized access at BaseLoadBalancer.java:[line 1051] |
| | org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.balanceCluster(Map) makes inefficient use of keySet iterator instead of entrySet iterator At RSGroupBasedLoadBalancer.java:keySet iterator instead of entrySet iterator At RSGroupBasedLoadBalancer.java:[line 143] |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/8/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux 916cf4e18b6f 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / f66cbe1a40 |
| spotbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/8/artifact/yetus-general-check/output/branch-spotbugs-hbase-server-warnings.html |
| checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/8/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| spotbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/8/artifact/yetus-general-check/output/new-spotbugs-hbase-server.html |
| Max. process+thread count | 83 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/8/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] infraio commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
infraio commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#discussion_r391646091
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
##########
@@ -1639,6 +1641,44 @@ private void roundRobinAssignment(Cluster cluster, List<RegionInfo> regions,
}
}
+ private Map<ServerName, List<RegionInfo>> toEnsumbleTableClusterLoad(
+ Map<TableName, Map<ServerName, List<RegionInfo>>> clusterLoadPerTable) {
+ Map<ServerName, List<RegionInfo>> returnMap = new TreeMap<>();
+ for (Map<ServerName, List<RegionInfo>> serverNameListMap : clusterLoadPerTable.values()) {
+ serverNameListMap.forEach((serverName, regionInfoList) -> {
+ List<RegionInfo> regionInfos =
+ returnMap.computeIfAbsent(serverName, k -> new ArrayList<>());
+ regionInfos.addAll(regionInfoList);
+ });
+ }
+ return returnMap;
+ }
+
+ @Override
+ public List<RegionPlan> balanceTable(TableName tableName, Map<ServerName, List<RegionInfo>> clusterLoad){
+ throw new UnsupportedOperationException("balanceTable is not support in BaseLoadBalancer");
+ }
+
+ @Override
+ public synchronized List<RegionPlan>
Review comment:
Why synchronized?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] infraio commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
infraio commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#discussion_r391646646
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
##########
@@ -1639,6 +1641,44 @@ private void roundRobinAssignment(Cluster cluster, List<RegionInfo> regions,
}
}
+ private Map<ServerName, List<RegionInfo>> toEnsumbleTableClusterLoad(
+ Map<TableName, Map<ServerName, List<RegionInfo>>> clusterLoadPerTable) {
+ Map<ServerName, List<RegionInfo>> returnMap = new TreeMap<>();
+ for (Map<ServerName, List<RegionInfo>> serverNameListMap : clusterLoadPerTable.values()) {
+ serverNameListMap.forEach((serverName, regionInfoList) -> {
+ List<RegionInfo> regionInfos =
+ returnMap.computeIfAbsent(serverName, k -> new ArrayList<>());
+ regionInfos.addAll(regionInfoList);
+ });
+ }
+ return returnMap;
+ }
+
+ @Override
+ public List<RegionPlan> balanceTable(TableName tableName, Map<ServerName, List<RegionInfo>> clusterLoad){
Review comment:
Unsupport? Should be abstrat method?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-602203157
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 18s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 8s | master passed |
| +1 :green_heart: | compile | 1m 6s | master passed |
| +1 :green_heart: | shadedjars | 5m 37s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 42s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 6s | the patch passed |
| +1 :green_heart: | compile | 1m 1s | the patch passed |
| +1 :green_heart: | javac | 1m 1s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 29s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 36s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 92m 7s | hbase-server in the patch passed. |
| | | 122m 7s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/15/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 9b71bd1702fd 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / ca533a77cb |
| Default Java | 1.8.0_232 |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/15/testReport/ |
| Max. process+thread count | 3945 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/15/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-600046479
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 4m 44s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 55s | master passed |
| +1 :green_heart: | compile | 0m 58s | master passed |
| +1 :green_heart: | shadedjars | 5m 3s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 38s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 21s | the patch passed |
| +1 :green_heart: | compile | 1m 0s | the patch passed |
| +1 :green_heart: | javac | 1m 0s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 53s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 35s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 62m 50s | hbase-server in the patch passed. |
| | | 94m 17s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/10/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 4deebd827a76 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 60de4c2e31 |
| Default Java | 1.8.0_232 |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/10/testReport/ |
| Max. process+thread count | 6347 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/10/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] infraio commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
infraio commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#discussion_r395969257
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupBasedLoadBalancer.java
##########
@@ -111,51 +109,45 @@ public void setMasterServices(MasterServices masterServices) {
this.masterServices = masterServices;
}
+ /**
+ * Override to balance by RSGroup
+ * not invoke {@link #balanceTable(TableName, Map)}
+ */
@Override
- public List<RegionPlan> balanceCluster(TableName tableName, Map<ServerName, List<RegionInfo>>
- clusterState) throws IOException {
- return balanceCluster(clusterState);
- }
-
- @Override
- public List<RegionPlan> balanceCluster(Map<ServerName, List<RegionInfo>> clusterState)
- throws IOException {
+ public List<RegionPlan> balanceCluster(
+ Map<TableName, Map<ServerName, List<RegionInfo>>> loadOfAllTable) throws IOException {
if (!isOnline()) {
throw new ConstraintException(
RSGroupInfoManager.class.getSimpleName() + " is not online, unable to perform balance");
}
// Calculate correct assignments and a list of RegionPlan for mis-placed regions
- Pair<Map<ServerName,List<RegionInfo>>, List<RegionPlan>> correctedStateAndRegionPlans =
- correctAssignments(clusterState);
- Map<ServerName,List<RegionInfo>> correctedState = correctedStateAndRegionPlans.getFirst();
+ Pair<Map<TableName, Map<ServerName, List<RegionInfo>>>, List<RegionPlan>> correctedStateAndRegionPlans =
+ correctAssignments(loadOfAllTable);
+ Map<TableName, Map<ServerName, List<RegionInfo>>> correctedLoadOfAllTable =
+ correctedStateAndRegionPlans.getFirst();
List<RegionPlan> regionPlans = correctedStateAndRegionPlans.getSecond();
-
+ RSGroupInfo defaultInfo = rsGroupInfoManager.getRSGroup(RSGroupInfo.DEFAULT_GROUP);
// Add RegionPlan
// for the regions which have been placed according to the region server group assignment
// into the movement list
try {
- // Record which region servers have been processed，so as to skip them after processed
- HashSet<ServerName> processedServers = new HashSet<>();
-
// For each rsgroup
for (RSGroupInfo rsgroup : rsGroupInfoManager.listRSGroups()) {
- Map<ServerName, List<RegionInfo>> groupClusterState = new HashMap<>();
- Map<TableName, Map<ServerName, List<RegionInfo>>> groupClusterLoad = new HashMap<>();
- for (ServerName server : clusterState.keySet()) { // for each region server
- if (!processedServers.contains(server) // server is not processed yet
- && rsgroup.containsServer(server.getAddress())) { // server belongs to this rsgroup
- List<RegionInfo> regionsOnServer = correctedState.get(server);
- groupClusterState.put(server, regionsOnServer);
-
- processedServers.add(server);
+ Map<TableName, Map<ServerName, List<RegionInfo>>> loadOfTablesInGroup =
+ new HashMap<>();
+ for (Map.Entry<TableName, Map<ServerName, List<RegionInfo>>> entry: correctedLoadOfAllTable.entrySet()) {
+ TableName tableName = entry.getKey();
+ RSGroupInfo targetRSGInfo = RSGroupUtil
+ .getRSGroupInfo(masterServices, rsGroupInfoManager, tableName).orElse(defaultInfo);
+ if (targetRSGInfo.getName().equals(rsgroup.getName())) {
+ loadOfTablesInGroup.put(tableName, entry.getValue());
}
}
-
- groupClusterLoad.put(HConstants.ENSEMBLE_TABLE_NAME, groupClusterState);
- this.internalBalancer.setClusterLoad(groupClusterLoad);
- List<RegionPlan> groupPlans = this.internalBalancer
- .balanceCluster(groupClusterState);
+ List<RegionPlan> groupPlans = null;
+ if (!loadOfTablesInGroup.isEmpty()) {
Review comment:
add log here, too?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-600030810
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 4m 44s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 56s | master passed |
| +1 :green_heart: | checkstyle | 1m 17s | master passed |
| -1 :x: | spotbugs | 2m 7s | hbase-server in master has 1 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 30s | the patch passed |
| -0 :warning: | checkstyle | 1m 14s | hbase-server: The patch generated 37 new + 408 unchanged - 9 fixed = 445 total (was 417) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 20m 15s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. |
| -1 :x: | spotbugs | 3m 38s | hbase-server generated 5 new + 1 unchanged - 0 fixed = 6 total (was 1) |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 22s | The patch does not generate ASF License warnings. |
| | | 55m 2s | |
| Reason | Tests |
|-------:|:------|
| FindBugs | module:hbase-server |
| | Inconsistent synchronization of org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.config; locked 50% of time Unsynchronized access at BaseLoadBalancer.java:50% of time Unsynchronized access at BaseLoadBalancer.java:[line 1144] |
| | Inconsistent synchronization of org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.onlySystemTablesOnMaster; locked 50% of time Unsynchronized access at BaseLoadBalancer.java:50% of time Unsynchronized access at BaseLoadBalancer.java:[line 1067] |
| | Inconsistent synchronization of org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.overallSlop; locked 62% of time Unsynchronized access at SimpleLoadBalancer.java:62% of time Unsynchronized access at SimpleLoadBalancer.java:[line 134] |
| | Inconsistent synchronization of org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.slop; locked 58% of time Unsynchronized access at BaseLoadBalancer.java:58% of time Unsynchronized access at BaseLoadBalancer.java:[line 1198] |
| | org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.getConf() is unsynchronized, org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.setConf(Configuration) is synchronized At BaseLoadBalancer.java:synchronized At BaseLoadBalancer.java:[line 1144] |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/10/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux aaf5b55c6d11 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 60de4c2e31 |
| spotbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/10/artifact/yetus-general-check/output/branch-spotbugs-hbase-server-warnings.html |
| checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/10/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| spotbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/10/artifact/yetus-general-check/output/new-spotbugs-hbase-server.html |
| Max. process+thread count | 93 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/10/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-598579611
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 6m 40s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 4s | master passed |
| +1 :green_heart: | compile | 1m 1s | master passed |
| +1 :green_heart: | shadedjars | 5m 15s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 38s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 43s | the patch passed |
| +1 :green_heart: | compile | 1m 1s | the patch passed |
| +1 :green_heart: | javac | 1m 1s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 9s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 36s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 94m 26s | hbase-server in the patch passed. |
| | | 128m 11s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/4/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 8c508f19ca7d 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 1b163d98b9 |
| Default Java | 1.8.0_232 |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/4/testReport/ |
| Max. process+thread count | 4282 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/4/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] infraio commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
infraio commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#discussion_r395969067
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
##########
@@ -1639,6 +1637,44 @@ private void roundRobinAssignment(Cluster cluster, List<RegionInfo> regions,
}
}
+ private Map<ServerName, List<RegionInfo>> toEnsumbleTableLoad(
+ Map<TableName, Map<ServerName, List<RegionInfo>>> LoadOfAllTable) {
+ Map<ServerName, List<RegionInfo>> returnMap = new TreeMap<>();
+ for (Map<ServerName, List<RegionInfo>> serverNameListMap : LoadOfAllTable.values()) {
+ serverNameListMap.forEach((serverName, regionInfoList) -> {
+ List<RegionInfo> regionInfos =
+ returnMap.computeIfAbsent(serverName, k -> new ArrayList<>());
+ regionInfos.addAll(regionInfoList);
+ });
+ }
+ return returnMap;
+ }
+
+ @Override
+ public abstract List<RegionPlan> balanceTable(TableName tableName,
+ Map<ServerName, List<RegionInfo>> loadOfOneTable);
+
+
+ @Override
+ public List<RegionPlan>
+ balanceCluster(Map<TableName, Map<ServerName, List<RegionInfo>>> loadOfAllTable) {
+ if (isByTable) {
+ List<RegionPlan> result = new ArrayList<>();
+ loadOfAllTable.forEach((tableName, loadOfOneTable) -> {
+ LOG.debug("Start Generate Balance plan for table: " + tableName);
+ List<RegionPlan> partialPlans = balanceTable(tableName, loadOfOneTable);
+ if (partialPlans != null) {
+ result.addAll(partialPlans);
+ }
+ });
+ return result;
+ } else {
+ LOG.debug("Start Generate Balance plan for table: " + HConstants.ENSEMBLE_TABLE_NAME);
Review comment:
May be good to log ""Start generate balance plan for cluster". No need log the ENSEMBLE_TABLE_NAME. And this log can be info?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] nyl3532016 commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
nyl3532016 commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#discussion_r395420805
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
##########
@@ -1639,6 +1637,44 @@ private void roundRobinAssignment(Cluster cluster, List<RegionInfo> regions,
}
}
+ private Map<ServerName, List<RegionInfo>> toEnsumbleTableLoad(
+ Map<TableName, Map<ServerName, List<RegionInfo>>> LoadOfAllTable) {
+ Map<ServerName, List<RegionInfo>> returnMap = new TreeMap<>();
+ for (Map<ServerName, List<RegionInfo>> serverNameListMap : LoadOfAllTable.values()) {
+ serverNameListMap.forEach((serverName, regionInfoList) -> {
+ List<RegionInfo> regionInfos =
+ returnMap.computeIfAbsent(serverName, k -> new ArrayList<>());
+ regionInfos.addAll(regionInfoList);
+ });
+ }
+ return returnMap;
+ }
+
+ @Override
+ public abstract List<RegionPlan> balanceTable(TableName tableName,
+ Map<ServerName, List<RegionInfo>> loadOfOneTable);
+
+
+ @Override
+ public List<RegionPlan>
+ balanceCluster(Map<TableName, Map<ServerName, List<RegionInfo>>> loadOfAllTable) {
+ if (isByTable) {
+ List<RegionPlan> result = new ArrayList<>();
+ loadOfAllTable.forEach((tableName, loadOfOneTable) -> {
+ LOG.debug("Start Generate Balance plan for table: " + tableName);
+ List<RegionPlan> partialPlans = balanceTable(tableName, loadOfOneTable);
+ if (partialPlans != null) {
+ result.addAll(partialPlans);
+ }
+ });
+ return result;
+ } else {
+ LOG.debug("Start Generate Balance plan for table: " + HConstants.ENSEMBLE_TABLE_NAME);
Review comment:
it convenient to debug ,balance for Table ENSEMBLE_TABLE_NAME , mean balanceByTable is false, we just consider all region as one table's
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] infraio commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
infraio commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#discussion_r393025640
##########
File path: hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestDefaultLoadBalancer.java
##########
@@ -58,7 +57,7 @@
private static final Logger LOG = LoggerFactory.getLogger(TestDefaultLoadBalancer.class);
- private static LoadBalancer loadBalancer;
+ private static SimpleLoadBalancer loadBalancer;
Review comment:
This test is for SimpleLoadBalancer? Rename TestDefaultLoadBalancer to TestSimpleLoadBalancer?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] infraio commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
infraio commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#discussion_r392065248
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/favored/FavoredNodeLoadBalancer.java
##########
@@ -87,7 +87,8 @@ public synchronized void initialize() throws HBaseIOException {
}
@Override
- public List<RegionPlan> balanceCluster(Map<ServerName, List<RegionInfo>> clusterState) {
+ public List<RegionPlan>
+ balanceCluster(Map<TableName, Map<ServerName, List<RegionInfo>>> clusterLoadPerTable) {
Review comment:
Didn't get your point. Please add javadoc comment about why retrun null here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-598103371
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 0s | Docker mode activated. |
| -1 :x: | docker | 0m 15s | Docker failed to build yetus/hbase:c5aa2bc121. |
| Subsystem | Report/Notes |
|----------:|:-------------|
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/2/console |
| versions | git=2.17.1 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-598140058
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 33s | Docker mode activated. |
| -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 29s | master passed |
| +1 :green_heart: | compile | 1m 7s | master passed |
| +1 :green_heart: | shadedjars | 5m 0s | branch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 41s | hbase-server in master failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 12s | the patch passed |
| +1 :green_heart: | compile | 1m 7s | the patch passed |
| +1 :green_heart: | javac | 1m 7s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 52s | patch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 39s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| -0 :warning: | unit | 65m 57s | hbase-server in the patch failed. |
| | | 94m 51s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 68ac6474729f 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 26f3702735 |
| Default Java | 2020-01-14 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/2/testReport/ |
| Max. process+thread count | 6805 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/2/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#issuecomment-599433502
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 34s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 36s | master passed |
| +1 :green_heart: | compile | 1m 10s | master passed |
| +1 :green_heart: | shadedjars | 5m 19s | branch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 44s | hbase-server in master failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 29s | the patch passed |
| +1 :green_heart: | compile | 1m 9s | the patch passed |
| +1 :green_heart: | javac | 1m 9s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 12s | patch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 41s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| -0 :warning: | unit | 71m 8s | hbase-server in the patch failed. |
| | | 101m 11s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/7/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1279 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 4305ea61be17 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / f66cbe1a40 |
| Default Java | 2020-01-14 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/7/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/7/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/7/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/7/testReport/ |
| Max. process+thread count | 6071 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1279/7/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] nyl3532016 commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
Posted by GitBox <gi...@apache.org>.
nyl3532016 commented on a change in pull request #1279: HBASE-23949 refactor loadBalancer implements for rsgroup balance by t…
URL: https://github.com/apache/hbase/pull/1279#discussion_r391997762
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/SimpleLoadBalancer.java
##########
@@ -106,6 +106,12 @@ void setNextRegionForUnload(int nextRegionForUnload) {
}
+ @Override
+ public synchronized void setConf(Configuration conf) {
Review comment:
yes, I remove it
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services