You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2021/03/04 01:40:36 UTC
[GitHub] [hbase] clarax opened a new pull request #3011: HBASE-25625 StochasticBalancer CostFunctions needs a better way to ev…
clarax opened a new pull request #3011:
URL: https://github.com/apache/hbase/pull/3011
…aluate resource distribution
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3011: HBASE-25625 StochasticBalancer CostFunctions needs a better way to ev…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3011:
URL: https://github.com/apache/hbase/pull/3011#issuecomment-791851307
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 44s | Docker mode activated. |
| -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 42s | master passed |
| +1 :green_heart: | compile | 1m 23s | master passed |
| +1 :green_heart: | shadedjars | 8m 46s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 49s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 51s | the patch passed |
| +1 :green_heart: | compile | 1m 40s | the patch passed |
| +1 :green_heart: | javac | 1m 40s | the patch passed |
| +1 :green_heart: | shadedjars | 8m 58s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 52s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 226m 58s | hbase-server in the patch failed. |
| | | 264m 33s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3011 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 05654c226f00 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 830d2895b2 |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/5/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/5/testReport/ |
| Max. process+thread count | 3147 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/5/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3011: HBASE-25625 StochasticBalancer CostFunctions needs a better way to ev…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3011:
URL: https://github.com/apache/hbase/pull/3011#issuecomment-791022192
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 6s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 51s | master passed |
| +1 :green_heart: | compile | 0m 58s | master passed |
| +1 :green_heart: | shadedjars | 7m 14s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 35s | master passed |
||| _ Patch Compile Tests _ |
| -1 :x: | mvninstall | 2m 11s | root in the patch failed. |
| -1 :x: | compile | 0m 55s | hbase-server in the patch failed. |
| -0 :warning: | javac | 0m 55s | hbase-server in the patch failed. |
| -1 :x: | shadedjars | 5m 38s | patch has 14 errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 35s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 0m 54s | hbase-server in the patch failed. |
| | | 25m 5s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3011 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux aa563d95dd2d 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 830d2895b2 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-jdk8-hadoop3-check/output/patch-mvninstall-root.txt |
| compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-jdk8-hadoop3-check/output/patch-compile-hbase-server.txt |
| javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-jdk8-hadoop3-check/output/patch-compile-hbase-server.txt |
| shadedjars | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-jdk8-hadoop3-check/output/patch-shadedjars.txt |
| unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/testReport/ |
| Max. process+thread count | 78 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3011: HBASE-25625 StochasticBalancer CostFunctions needs a better way to ev…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3011:
URL: https://github.com/apache/hbase/pull/3011#issuecomment-790309567
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 10s | Docker mode activated. |
| -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 10s | master passed |
| +1 :green_heart: | compile | 1m 0s | master passed |
| +1 :green_heart: | shadedjars | 7m 10s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 37s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 52s | the patch passed |
| +1 :green_heart: | compile | 0m 59s | the patch passed |
| +1 :green_heart: | javac | 0m 59s | the patch passed |
| +1 :green_heart: | shadedjars | 7m 14s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 36s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 211m 57s | hbase-server in the patch failed. |
| | | 240m 38s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3011 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 3f7895912066 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 5d9a6ed1fe |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/1/testReport/ |
| Max. process+thread count | 3481 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3011: HBASE-25625 StochasticBalancer CostFunctions needs a better way to ev…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3011:
URL: https://github.com/apache/hbase/pull/3011#issuecomment-790956119
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 10s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 41s | master passed |
| +1 :green_heart: | compile | 1m 0s | master passed |
| +1 :green_heart: | shadedjars | 7m 11s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 40s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 50s | the patch passed |
| +1 :green_heart: | compile | 1m 0s | the patch passed |
| +1 :green_heart: | javac | 1m 0s | the patch passed |
| +1 :green_heart: | shadedjars | 7m 13s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 37s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 212m 5s | hbase-server in the patch failed. |
| | | 241m 19s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3011 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux c34eb45f8c70 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 830d2895b2 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/3/testReport/ |
| Max. process+thread count | 3400 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/3/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] sunhelly commented on a change in pull request #3011: HBASE-25625 StochasticBalancer CostFunctions needs a better way to ev…
Posted by GitBox <gi...@apache.org>.
sunhelly commented on a change in pull request #3011:
URL: https://github.com/apache/hbase/pull/3011#discussion_r588808384
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
##########
@@ -383,20 +384,19 @@ protected Cluster(
}
}
+ regionsPerTable = new int[numTables];
Review comment:
What if we use balance by table, which makes Cluster objects only contains one table ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3011: HBASE-25625 StochasticBalancer CostFunctions needs a better way to ev…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3011:
URL: https://github.com/apache/hbase/pull/3011#issuecomment-790550810
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 7m 19s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 53s | master passed |
| +1 :green_heart: | compile | 0m 58s | master passed |
| +1 :green_heart: | shadedjars | 7m 13s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 36s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 52s | the patch passed |
| +1 :green_heart: | compile | 0m 57s | the patch passed |
| +1 :green_heart: | javac | 0m 57s | the patch passed |
| +1 :green_heart: | shadedjars | 7m 13s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 36s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 216m 43s | hbase-server in the patch failed. |
| | | 251m 15s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3011 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux be50297a1d42 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 830d2895b2 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/2/testReport/ |
| Max. process+thread count | 3472 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/2/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani edited a comment on pull request #3011: HBASE-25625 StochasticBalancer CostFunctions needs a better way to ev…
Posted by GitBox <gi...@apache.org>.
virajjasani edited a comment on pull request #3011:
URL: https://github.com/apache/hbase/pull/3011#issuecomment-790379492
@clarax could you please check compilation issue [here](https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/1/artifact/yetus-general-check/output/patch-compile-hbase-server.txt)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3011: HBASE-25625 StochasticBalancer CostFunctions needs a better way to ev…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3011:
URL: https://github.com/apache/hbase/pull/3011#issuecomment-790421658
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 7m 6s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 12s | master passed |
| +1 :green_heart: | compile | 3m 22s | master passed |
| +1 :green_heart: | checkstyle | 1m 13s | master passed |
| +1 :green_heart: | spotbugs | 2m 10s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 49s | the patch passed |
| +1 :green_heart: | compile | 3m 19s | the patch passed |
| +1 :green_heart: | javac | 3m 19s | the patch passed |
| -0 :warning: | checkstyle | 1m 12s | hbase-server: The patch generated 2 new + 100 unchanged - 1 fixed = 102 total (was 101) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 19m 5s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. |
| +1 :green_heart: | spotbugs | 2m 17s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. |
| | | 55m 58s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/2/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3011 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile |
| uname | Linux e77e0872c24b 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 830d2895b2 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| Max. process+thread count | 85 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/2/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] clarax commented on pull request #3011: HBASE-25625 StochasticBalancer CostFunctions needs a better way to ev…
Posted by GitBox <gi...@apache.org>.
clarax commented on pull request #3011:
URL: https://github.com/apache/hbase/pull/3011#issuecomment-816054326
> Is this PR live still @clarax ? Or superceded by #3067 ? Thanks.
The new heuristic needs to be evaluated by simulation at different scales. so this pr is currently pending the validation. 3067 is a scaled back fix for table skew only as a sub-task https://issues.apache.org/jira/browse/HBASE-25739?filter=-2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] saintstack commented on pull request #3011: HBASE-25625 StochasticBalancer CostFunctions needs a better way to ev…
Posted by GitBox <gi...@apache.org>.
saintstack commented on pull request #3011:
URL: https://github.com/apache/hbase/pull/3011#issuecomment-816010433
Is this PR live still @clarax ? Or superceded by #3067 ? Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] d-c-manning commented on a change in pull request #3011: HBASE-25625 StochasticBalancer CostFunctions needs a better way to ev…
Posted by GitBox <gi...@apache.org>.
d-c-manning commented on a change in pull request #3011:
URL: https://github.com/apache/hbase/pull/3011#discussion_r587954225
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
##########
@@ -513,6 +515,75 @@ private void registerRegion(RegionInfo region, int regionIndex,
}
}
+ private void reComputeRegionStDevPerTable() {
+ for (int i = 0; i < numTables; i++) {
+ regionStDevPerTable[i] = 0;
+ }
+
+ for (int[] aNumRegionsPerServerPerTable : numRegionsPerServerPerTable) {
+ for (int tableIndex = 0; tableIndex < aNumRegionsPerServerPerTable.length; tableIndex++) {
+ double deviation = aNumRegionsPerServerPerTable[tableIndex]
+ - regionsPerTable[tableIndex] / numServers;
+ regionStDevPerTable[tableIndex] += deviation * deviation;
+ }
+ }
+
+ for (int i = 0; i < numTables; i++) {
+ regionStDevPerTable[i] = scale(getMinStDev(regionsPerTable[i], numServers),
+ getMaxStDev(regionsPerTable[i], numServers),
+ Math.sqrt(regionStDevPerTable[i] / numServers));
+ }
+ }
+
+ /**
+ * Return the max standard deviation of distribution of regions
+ * Compute max as if all region servers had 0 and one had the sum of all costs. This must be
+ * a zero sum cost for this to make sense.
+ */
+ public double getMaxStDev(double total, double numServers) {
Review comment:
can this be static?
```suggestion
public static double getMaxStDev(double total, double numServers) {
```
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
##########
@@ -837,19 +908,8 @@ void regionMoved(int region, int oldServer, int newServer) {
}
numRegionsPerServerPerTable[newServer][tableIndex]++;
- //check whether this caused maxRegionsPerTable in the new Server to be updated
- if (numRegionsPerServerPerTable[newServer][tableIndex] > numMaxRegionsPerTable[tableIndex]) {
- numMaxRegionsPerTable[tableIndex] = numRegionsPerServerPerTable[newServer][tableIndex];
- } else if (oldServer >= 0 && (numRegionsPerServerPerTable[oldServer][tableIndex] + 1)
- == numMaxRegionsPerTable[tableIndex]) {
- //recompute maxRegionsPerTable since the previous value was coming from the old server
- numMaxRegionsPerTable[tableIndex] = 0;
- for (int[] aNumRegionsPerServerPerTable : numRegionsPerServerPerTable) {
- if (aNumRegionsPerServerPerTable[tableIndex] > numMaxRegionsPerTable[tableIndex]) {
- numMaxRegionsPerTable[tableIndex] = aNumRegionsPerServerPerTable[tableIndex];
- }
- }
- }
+ // recalculate stdev
+ reComputeRegionStDevPerTable();
Review comment:
It seems like here we should only be recomputing the stdev for the affected `tableIndex`. I don't want to over-optimize if it makes the code less readable... but in clusters with large numbers of tables, we would be doing unnecessary work.
I'm thinking in the context of [HBASE-22300](https://issues.apache.org/jira/browse/HBASE-22300) where large clusters no longer run through enough iterations of the balancer. In that case, the issue is number of regions, which will always be larger than number of tables... and likely much larger. So it may not be a major issue here, but I'm bringing it up just in case.
##########
File path: hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancerLargeCluster.java
##########
@@ -38,6 +38,8 @@ public void testLargeCluster() {
int numRegionsPerServer = 80; // all servers except one
int numTables = 100;
int replication = 1;
- testWithCluster(numNodes, numRegions, numRegionsPerServer, replication, numTables, true, true);
+
+ // we need to capture the outlier and generate a move
Review comment:
I don't understand this comment.
##########
File path: hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancerLargeCluster.java
##########
@@ -38,6 +38,8 @@ public void testLargeCluster() {
int numRegionsPerServer = 80; // all servers except one
int numTables = 100;
int replication = 1;
- testWithCluster(numNodes, numRegions, numRegionsPerServer, replication, numTables, true, true);
+
+ // we need to capture the outlier and generate a move
+ testWithCluster(numNodes, numRegions, numRegionsPerServer, replication, numTables, false, false);
Review comment:
The change of code suggests that the balancer will no longer fully balance the cluster in one run... why is that? Are we making the balancer less effective, even in this test with large cluster with one outlier node? I thought this is the type of case that the fix is supposed to improve?
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
##########
@@ -168,7 +168,9 @@ private void createRegionFinder() {
int[] initialRegionIndexToServerIndex; //regionIndex -> serverIndex (initial cluster state)
int[] regionIndexToTableIndex; //regionIndex -> tableIndex
int[][] numRegionsPerServerPerTable; //serverIndex -> tableIndex -> # regions
- int[] numMaxRegionsPerTable; //tableIndex -> max number of regions in a single RS
+ int[] regionsPerTable; // count of regions per table
+ double[] regionStDevPerTable; //tableIndex -> standard deviation of region distribution
Review comment:
nit: should we rename to mention that this value is already scaled?
```suggestion
double[] regionStDevPerTableScaled; //tableIndex -> scaled standard deviation of region distribution
```
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
##########
@@ -383,20 +385,20 @@ protected Cluster(
}
}
+ regionsPerTable = new int[numTables];
+ for (int i = 0; i < numTables; i++) {
+ regionsPerTable[i] = 0;
+ }
+
Review comment:
nit: maybe it's nice to be explicit like the code above, but default values are already 0
```suggestion
```
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
##########
@@ -513,6 +515,75 @@ private void registerRegion(RegionInfo region, int regionIndex,
}
}
+ private void reComputeRegionStDevPerTable() {
+ for (int i = 0; i < numTables; i++) {
+ regionStDevPerTable[i] = 0;
+ }
+
+ for (int[] aNumRegionsPerServerPerTable : numRegionsPerServerPerTable) {
+ for (int tableIndex = 0; tableIndex < aNumRegionsPerServerPerTable.length; tableIndex++) {
+ double deviation = aNumRegionsPerServerPerTable[tableIndex]
+ - regionsPerTable[tableIndex] / numServers;
+ regionStDevPerTable[tableIndex] += deviation * deviation;
+ }
+ }
+
+ for (int i = 0; i < numTables; i++) {
+ regionStDevPerTable[i] = scale(getMinStDev(regionsPerTable[i], numServers),
+ getMaxStDev(regionsPerTable[i], numServers),
+ Math.sqrt(regionStDevPerTable[i] / numServers));
+ }
+ }
+
+ /**
+ * Return the max standard deviation of distribution of regions
Review comment:
```suggestion
* Return the max standard deviation of distribution of costs
```
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
##########
@@ -513,6 +515,75 @@ private void registerRegion(RegionInfo region, int regionIndex,
}
}
+ private void reComputeRegionStDevPerTable() {
+ for (int i = 0; i < numTables; i++) {
+ regionStDevPerTable[i] = 0;
+ }
+
+ for (int[] aNumRegionsPerServerPerTable : numRegionsPerServerPerTable) {
+ for (int tableIndex = 0; tableIndex < aNumRegionsPerServerPerTable.length; tableIndex++) {
+ double deviation = aNumRegionsPerServerPerTable[tableIndex]
+ - regionsPerTable[tableIndex] / numServers;
+ regionStDevPerTable[tableIndex] += deviation * deviation;
+ }
+ }
+
+ for (int i = 0; i < numTables; i++) {
+ regionStDevPerTable[i] = scale(getMinStDev(regionsPerTable[i], numServers),
+ getMaxStDev(regionsPerTable[i], numServers),
+ Math.sqrt(regionStDevPerTable[i] / numServers));
+ }
+ }
+
+ /**
+ * Return the max standard deviation of distribution of regions
+ * Compute max as if all region servers had 0 and one had the sum of all costs. This must be
+ * a zero sum cost for this to make sense.
+ */
+ public double getMaxStDev(double total, double numServers) {
+ double mean = total / numServers;
+ return Math.sqrt(((total - mean) * (total - mean)
+ + (numServers - 1) * mean * mean)
+ / numServers);
+ }
+
+ /**
+ * Return the min standard deviation of distribution of regions
+ */
+ public double getMinStDev(double total, double numServers) {
Review comment:
can this be static?
```suggestion
public static double getMinStDev(double total, double numServers) {
```
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
##########
@@ -513,6 +515,75 @@ private void registerRegion(RegionInfo region, int regionIndex,
}
}
+ private void reComputeRegionStDevPerTable() {
+ for (int i = 0; i < numTables; i++) {
+ regionStDevPerTable[i] = 0;
+ }
+
+ for (int[] aNumRegionsPerServerPerTable : numRegionsPerServerPerTable) {
+ for (int tableIndex = 0; tableIndex < aNumRegionsPerServerPerTable.length; tableIndex++) {
+ double deviation = aNumRegionsPerServerPerTable[tableIndex]
+ - regionsPerTable[tableIndex] / numServers;
+ regionStDevPerTable[tableIndex] += deviation * deviation;
+ }
+ }
+
+ for (int i = 0; i < numTables; i++) {
+ regionStDevPerTable[i] = scale(getMinStDev(regionsPerTable[i], numServers),
+ getMaxStDev(regionsPerTable[i], numServers),
+ Math.sqrt(regionStDevPerTable[i] / numServers));
+ }
+ }
+
+ /**
+ * Return the max standard deviation of distribution of regions
+ * Compute max as if all region servers had 0 and one had the sum of all costs. This must be
+ * a zero sum cost for this to make sense.
+ */
+ public double getMaxStDev(double total, double numServers) {
+ double mean = total / numServers;
+ return Math.sqrt(((total - mean) * (total - mean)
+ + (numServers - 1) * mean * mean)
+ / numServers);
+ }
+
+ /**
+ * Return the min standard deviation of distribution of regions
Review comment:
```suggestion
* Return the min standard deviation of distribution of costs
```
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
##########
@@ -513,6 +515,75 @@ private void registerRegion(RegionInfo region, int regionIndex,
}
}
+ private void reComputeRegionStDevPerTable() {
+ for (int i = 0; i < numTables; i++) {
+ regionStDevPerTable[i] = 0;
+ }
+
+ for (int[] aNumRegionsPerServerPerTable : numRegionsPerServerPerTable) {
+ for (int tableIndex = 0; tableIndex < aNumRegionsPerServerPerTable.length; tableIndex++) {
+ double deviation = aNumRegionsPerServerPerTable[tableIndex]
+ - regionsPerTable[tableIndex] / numServers;
+ regionStDevPerTable[tableIndex] += deviation * deviation;
+ }
+ }
+
+ for (int i = 0; i < numTables; i++) {
+ regionStDevPerTable[i] = scale(getMinStDev(regionsPerTable[i], numServers),
+ getMaxStDev(regionsPerTable[i], numServers),
+ Math.sqrt(regionStDevPerTable[i] / numServers));
+ }
+ }
+
+ /**
+ * Return the max standard deviation of distribution of regions
+ * Compute max as if all region servers had 0 and one had the sum of all costs. This must be
+ * a zero sum cost for this to make sense.
+ */
+ public double getMaxStDev(double total, double numServers) {
+ double mean = total / numServers;
+ return Math.sqrt(((total - mean) * (total - mean)
+ + (numServers - 1) * mean * mean)
+ / numServers);
+ }
+
+ /**
+ * Return the min standard deviation of distribution of regions
+ */
+ public double getMinStDev(double total, double numServers) {
+ double mean = total / numServers;
+ // It's possible that there aren't enough regions to go around
+ double min;
+ if (numServers > total) {
+ min = Math.sqrt(((numServers - total) * mean * mean
+ + (1 - mean) * (1 - mean) * total) / numServers);
+ } else {
+ // Some will have 1 more than everything else.
+ int numHigh = (int) (total - (Math.floor(mean) * numServers));
+ int numLow = (int) (numServers - numHigh);
+ min = Math.sqrt(((numHigh * (Math.ceil(mean) - mean) * (Math.ceil(mean) - mean))
+ + (numLow * (mean - Math.floor(mean)) * (mean - Math.floor(mean)))) / numServers);
+ }
+ return min;
+ }
+
+ /**
+ * Scale the value between 0 and 1.
+ *
+ * @param min Min value
+ * @param max The Max value
+ * @param value The value to be scaled.
+ * @return The scaled value.
+ */
+ public double scale(double min, double max, double value) {
Review comment:
can this be static?
```suggestion
public static double scale(double min, double max, double value) {
```
##########
File path: hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestBaseLoadBalancer.java
##########
@@ -391,7 +391,7 @@ public void testRegionAvailabilityWithRegionMoves() throws Exception {
// now move region1 from servers[0] to servers[2]
cluster.doAction(new MoveRegionAction(0, 0, 2));
// check that the numMaxRegionsPerTable for "table" has increased to 2
Review comment:
comment no longer applies
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on pull request #3011: HBASE-25625 StochasticBalancer CostFunctions needs a better way to ev…
Posted by GitBox <gi...@apache.org>.
virajjasani commented on pull request #3011:
URL: https://github.com/apache/hbase/pull/3011#issuecomment-790379492
Compilation error:
```
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-3011/yetus-general-check/src/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestBaseLoadBalancer.java:[394,16] error: [DoNotCall] This method always throws java.lang.AssertionError. Use assertEquals(expected, actual, delta) to compare floating-point numbers
(see https://errorprone.info/bugpattern/DoNotCall)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] clarax commented on a change in pull request #3011: HBASE-25625 StochasticBalancer CostFunctions needs a better way to ev…
Posted by GitBox <gi...@apache.org>.
clarax commented on a change in pull request #3011:
URL: https://github.com/apache/hbase/pull/3011#discussion_r612690357
##########
File path: hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancerLargeCluster.java
##########
@@ -38,6 +38,8 @@ public void testLargeCluster() {
int numRegionsPerServer = 80; // all servers except one
int numTables = 100;
int replication = 1;
- testWithCluster(numNodes, numRegions, numRegionsPerServer, replication, numTables, true, true);
+
+ // we need to capture the outlier and generate a move
+ testWithCluster(numNodes, numRegions, numRegionsPerServer, replication, numTables, false, false);
Review comment:
It turns out to be related to https://issues.apache.org/jira/browse/HBASE-25767. Will merge the fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] clarax closed pull request #3011: HBASE-25625 StochasticBalancer CostFunctions needs a better way to ev…
Posted by GitBox <gi...@apache.org>.
clarax closed pull request #3011:
URL: https://github.com/apache/hbase/pull/3011
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3011: HBASE-25625 StochasticBalancer CostFunctions needs a better way to ev…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3011:
URL: https://github.com/apache/hbase/pull/3011#issuecomment-791022252
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 28s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 18s | master passed |
| +1 :green_heart: | compile | 1m 5s | master passed |
| +1 :green_heart: | shadedjars | 6m 44s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 42s | master passed |
||| _ Patch Compile Tests _ |
| -1 :x: | mvninstall | 2m 29s | root in the patch failed. |
| -1 :x: | compile | 1m 5s | hbase-server in the patch failed. |
| -0 :warning: | javac | 1m 5s | hbase-server in the patch failed. |
| -1 :x: | shadedjars | 5m 21s | patch has 14 errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 39s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 1m 6s | hbase-server in the patch failed. |
| | | 25m 12s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3011 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux d7d5ef3a6c42 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 830d2895b2 |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-jdk11-hadoop3-check/output/patch-mvninstall-root.txt |
| compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-jdk11-hadoop3-check/output/patch-compile-hbase-server.txt |
| javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-jdk11-hadoop3-check/output/patch-compile-hbase-server.txt |
| shadedjars | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt |
| unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/testReport/ |
| Max. process+thread count | 103 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3011: HBASE-25625 StochasticBalancer CostFunctions needs a better way to ev…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3011:
URL: https://github.com/apache/hbase/pull/3011#issuecomment-791847533
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 13s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 0s | master passed |
| +1 :green_heart: | compile | 1m 1s | master passed |
| +1 :green_heart: | shadedjars | 7m 20s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 37s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 52s | the patch passed |
| +1 :green_heart: | compile | 0m 59s | the patch passed |
| +1 :green_heart: | javac | 0m 59s | the patch passed |
| +1 :green_heart: | shadedjars | 7m 15s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 37s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 211m 4s | hbase-server in the patch failed. |
| | | 239m 36s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/5/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3011 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 5ccd0d41708f 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 830d2895b2 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/5/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/5/testReport/ |
| Max. process+thread count | 3657 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/5/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] clarax commented on a change in pull request #3011: HBASE-25625 StochasticBalancer CostFunctions needs a better way to ev…
Posted by GitBox <gi...@apache.org>.
clarax commented on a change in pull request #3011:
URL: https://github.com/apache/hbase/pull/3011#discussion_r609035598
##########
File path: hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancerLargeCluster.java
##########
@@ -38,6 +38,8 @@ public void testLargeCluster() {
int numRegionsPerServer = 80; // all servers except one
int numTables = 100;
int replication = 1;
- testWithCluster(numNodes, numRegions, numRegionsPerServer, replication, numTables, true, true);
+
+ // we need to capture the outlier and generate a move
+ testWithCluster(numNodes, numRegions, numRegionsPerServer, replication, numTables, false, false);
Review comment:
This is one of the side impact. the computation gets more complicated. I will work on it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3011: HBASE-25625 StochasticBalancer CostFunctions needs a better way to ev…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3011:
URL: https://github.com/apache/hbase/pull/3011#issuecomment-790239037
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 6s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 12s | master passed |
| +1 :green_heart: | compile | 3m 20s | master passed |
| +1 :green_heart: | checkstyle | 1m 14s | master passed |
| +1 :green_heart: | spotbugs | 2m 9s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 50s | the patch passed |
| -1 :x: | compile | 3m 14s | hbase-server in the patch failed. |
| -0 :warning: | javac | 3m 14s | hbase-server in the patch failed. |
| -0 :warning: | checkstyle | 1m 12s | hbase-server: The patch generated 2 new + 100 unchanged - 1 fixed = 102 total (was 101) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 19m 10s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. |
| +1 :green_heart: | spotbugs | 2m 18s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. |
| | | 49m 50s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3011 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile |
| uname | Linux a74798b76a14 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 5d9a6ed1fe |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/1/artifact/yetus-general-check/output/patch-compile-hbase-server.txt |
| javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/1/artifact/yetus-general-check/output/patch-compile-hbase-server.txt |
| checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| Max. process+thread count | 86 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/1/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3011: HBASE-25625 StochasticBalancer CostFunctions needs a better way to ev…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3011:
URL: https://github.com/apache/hbase/pull/3011#issuecomment-791023674
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 3m 58s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 53s | master passed |
| +1 :green_heart: | compile | 3m 13s | master passed |
| +1 :green_heart: | checkstyle | 1m 9s | master passed |
| +1 :green_heart: | spotbugs | 2m 2s | master passed |
||| _ Patch Compile Tests _ |
| -1 :x: | mvninstall | 2m 3s | root in the patch failed. |
| -1 :x: | compile | 1m 52s | hbase-server in the patch failed. |
| -0 :warning: | javac | 1m 52s | hbase-server in the patch failed. |
| -0 :warning: | checkstyle | 1m 5s | hbase-server: The patch generated 3 new + 134 unchanged - 1 fixed = 137 total (was 135) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| -1 :x: | hadoopcheck | 2m 9s | The patch causes 14 errors with Hadoop v3.1.2. |
| -1 :x: | hadoopcheck | 4m 21s | The patch causes 14 errors with Hadoop v3.2.1. |
| -1 :x: | hadoopcheck | 6m 43s | The patch causes 14 errors with Hadoop v3.3.0. |
| -1 :x: | spotbugs | 0m 32s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. |
| | | 28m 23s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3011 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile |
| uname | Linux b84641460c04 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 830d2895b2 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-general-check/output/patch-mvninstall-root.txt |
| compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-general-check/output/patch-compile-hbase-server.txt |
| javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-general-check/output/patch-compile-hbase-server.txt |
| checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| hadoopcheck | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-general-check/output/patch-javac-3.1.2.txt |
| hadoopcheck | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-general-check/output/patch-javac-3.2.1.txt |
| hadoopcheck | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-general-check/output/patch-javac-3.3.0.txt |
| spotbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-general-check/output/patch-spotbugs-hbase-server.txt |
| Max. process+thread count | 96 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3011: HBASE-25625 StochasticBalancer CostFunctions needs a better way to ev…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3011:
URL: https://github.com/apache/hbase/pull/3011#issuecomment-790549571
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 7m 42s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 55s | master passed |
| +1 :green_heart: | compile | 1m 17s | master passed |
| +1 :green_heart: | shadedjars | 8m 27s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 45s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 38s | the patch passed |
| +1 :green_heart: | compile | 1m 19s | the patch passed |
| +1 :green_heart: | javac | 1m 19s | the patch passed |
| +1 :green_heart: | shadedjars | 7m 36s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 43s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 209m 42s | hbase-server in the patch failed. |
| | | 248m 58s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3011 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 3dabc5f1454a 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 830d2895b2 |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/2/testReport/ |
| Max. process+thread count | 2685 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/2/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3011: HBASE-25625 StochasticBalancer CostFunctions needs a better way to ev…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3011:
URL: https://github.com/apache/hbase/pull/3011#issuecomment-790311039
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 19s | Docker mode activated. |
| -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 50s | master passed |
| +1 :green_heart: | compile | 1m 13s | master passed |
| +1 :green_heart: | shadedjars | 7m 28s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 43s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 42s | the patch passed |
| +1 :green_heart: | compile | 1m 13s | the patch passed |
| +1 :green_heart: | javac | 1m 13s | the patch passed |
| +1 :green_heart: | shadedjars | 7m 25s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 41s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 212m 29s | hbase-server in the patch failed. |
| | | 243m 55s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3011 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 897dd517b4a7 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 5d9a6ed1fe |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/1/testReport/ |
| Max. process+thread count | 3121 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] clarax commented on a change in pull request #3011: HBASE-25625 StochasticBalancer CostFunctions needs a better way to ev…
Posted by GitBox <gi...@apache.org>.
clarax commented on a change in pull request #3011:
URL: https://github.com/apache/hbase/pull/3011#discussion_r609033192
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
##########
@@ -383,20 +384,19 @@ protected Cluster(
}
}
+ regionsPerTable = new int[numTables];
Review comment:
Thank you for the comments. Sorry I am not sure what the issue is. Could you elaborate? The new changes scale with the number of tables. If there is only one table, numTables=1. only one table is being looked at.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] clarax commented on pull request #3011: HBASE-25625 StochasticBalancer CostFunctions needs a better way to ev…
Posted by GitBox <gi...@apache.org>.
clarax commented on pull request #3011:
URL: https://github.com/apache/hbase/pull/3011#issuecomment-902884096
Close the pr for now since the other pr #3067 landed and relieved most of the problem. Will revisit when the problem arises again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] clarax commented on a change in pull request #3011: HBASE-25625 StochasticBalancer CostFunctions needs a better way to ev…
Posted by GitBox <gi...@apache.org>.
clarax commented on a change in pull request #3011:
URL: https://github.com/apache/hbase/pull/3011#discussion_r609034878
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
##########
@@ -837,19 +908,8 @@ void regionMoved(int region, int oldServer, int newServer) {
}
numRegionsPerServerPerTable[newServer][tableIndex]++;
- //check whether this caused maxRegionsPerTable in the new Server to be updated
- if (numRegionsPerServerPerTable[newServer][tableIndex] > numMaxRegionsPerTable[tableIndex]) {
- numMaxRegionsPerTable[tableIndex] = numRegionsPerServerPerTable[newServer][tableIndex];
- } else if (oldServer >= 0 && (numRegionsPerServerPerTable[oldServer][tableIndex] + 1)
- == numMaxRegionsPerTable[tableIndex]) {
- //recompute maxRegionsPerTable since the previous value was coming from the old server
- numMaxRegionsPerTable[tableIndex] = 0;
- for (int[] aNumRegionsPerServerPerTable : numRegionsPerServerPerTable) {
- if (aNumRegionsPerServerPerTable[tableIndex] > numMaxRegionsPerTable[tableIndex]) {
- numMaxRegionsPerTable[tableIndex] = aNumRegionsPerServerPerTable[tableIndex];
- }
- }
- }
+ // recalculate stdev
+ reComputeRegionStDevPerTable();
Review comment:
Thank you for bringing up this. reComputeRegionStDevPerTable() can be optimized to only compute the impacted table index to reduce computation cost.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3011: HBASE-25625 StochasticBalancer CostFunctions needs a better way to ev…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3011:
URL: https://github.com/apache/hbase/pull/3011#issuecomment-790823516
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 5s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 48s | master passed |
| +1 :green_heart: | compile | 3m 22s | master passed |
| +1 :green_heart: | checkstyle | 1m 12s | master passed |
| +1 :green_heart: | spotbugs | 2m 8s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 51s | the patch passed |
| +1 :green_heart: | compile | 3m 21s | the patch passed |
| +1 :green_heart: | javac | 3m 21s | the patch passed |
| -0 :warning: | checkstyle | 1m 12s | hbase-server: The patch generated 2 new + 134 unchanged - 1 fixed = 136 total (was 135) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 18m 56s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. |
| +1 :green_heart: | spotbugs | 2m 17s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 11s | The patch does not generate ASF License warnings. |
| | | 49m 20s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/3/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3011 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 2a1ab759d5f3 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 830d2895b2 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| Max. process+thread count | 85 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/3/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3011: HBASE-25625 StochasticBalancer CostFunctions needs a better way to ev…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3011:
URL: https://github.com/apache/hbase/pull/3011#issuecomment-790959819
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 10s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 39s | master passed |
| +1 :green_heart: | compile | 1m 14s | master passed |
| +1 :green_heart: | shadedjars | 7m 26s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 42s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 37s | the patch passed |
| +1 :green_heart: | compile | 1m 13s | the patch passed |
| +1 :green_heart: | javac | 1m 13s | the patch passed |
| +1 :green_heart: | shadedjars | 7m 50s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 47s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 215m 21s | hbase-server in the patch failed. |
| | | 246m 53s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3011 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux f22b7fab953b 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 830d2895b2 |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/3/testReport/ |
| Max. process+thread count | 2840 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/3/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3011: HBASE-25625 StochasticBalancer CostFunctions needs a better way to ev…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3011:
URL: https://github.com/apache/hbase/pull/3011#issuecomment-791791323
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 9s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 56s | master passed |
| +1 :green_heart: | compile | 3m 21s | master passed |
| +1 :green_heart: | checkstyle | 1m 13s | master passed |
| +1 :green_heart: | spotbugs | 2m 10s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 49s | the patch passed |
| +1 :green_heart: | compile | 3m 16s | the patch passed |
| +1 :green_heart: | javac | 3m 16s | the patch passed |
| -0 :warning: | checkstyle | 1m 12s | hbase-server: The patch generated 4 new + 134 unchanged - 1 fixed = 138 total (was 135) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 19m 2s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. |
| +1 :green_heart: | spotbugs | 2m 23s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 11s | The patch does not generate ASF License warnings. |
| | | 49m 27s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/5/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3011 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 43ab0b3af714 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 830d2895b2 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/5/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| Max. process+thread count | 85 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/5/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] saintstack commented on pull request #3011: HBASE-25625 StochasticBalancer CostFunctions needs a better way to ev…
Posted by GitBox <gi...@apache.org>.
saintstack commented on pull request #3011:
URL: https://github.com/apache/hbase/pull/3011#issuecomment-816060690
Should this or the other PR have their own JIRA then @clarax Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org