You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2022/04/26 07:31:11 UTC
[GitHub] [hbase] d-c-manning opened a new pull request, #4371: HBASE-22349 slop in StochasticLoadBalancer
d-c-manning opened a new pull request, #4371:
URL: https://github.com/apache/hbase/pull/4371
[HBASE-22349](https://issues.apache.org/jira/browse/HBASE-22349)
Slop is a concept used within `SimpleLoadBalancer` but has always been referenced in `StochasticLoadBalancer` without any effect. This change proposes that we start using the concept of slop, as another trigger for a balancer run. This is similar to HBASE-24139, where a server with 0 regions is considered a trigger to run the balancer.
In this way we can guarantee balancer attempts to run in other cases that a balancer run is desirable, such as only 1 region on a server, or 2x or 3x the average region count on a server.
I remove the `normalizeSlop` method for two reasons:
1. A negative value will indicate the slop value should not be used at all
2. Values greater than 1 can still be useful (setting slop to 2 would mean to trigger a balancer run if a server had >3x the average region count for the cluster.)
As far as I can tell, the default slop of `0.001f` never had any meaning for the `StochasticLoadBalancer` before this change. I choose to reuse the default slop of `0.2f` as used in the `SimpleLoadBalancer`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] d-c-manning commented on pull request #4371: HBASE-22349 slop in StochasticLoadBalancer
Posted by GitBox <gi...@apache.org>.
d-c-manning commented on PR #4371:
URL: https://github.com/apache/hbase/pull/4371#issuecomment-1112576300
Oops, missed the whitespace comment. Will fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4371: HBASE-22349 slop in StochasticLoadBalancer
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4371:
URL: https://github.com/apache/hbase/pull/4371#issuecomment-1109700900
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 6s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 45s | master passed |
| +1 :green_heart: | compile | 1m 20s | master passed |
| +1 :green_heart: | shadedjars | 3m 42s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 55s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 33s | the patch passed |
| +1 :green_heart: | compile | 1m 19s | the patch passed |
| +1 :green_heart: | javac | 1m 19s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 1s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 56s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 38s | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 11m 36s | hbase-balancer in the patch passed. |
| -1 :x: | unit | 221m 58s | hbase-server in the patch failed. |
| | | 257m 5s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4371/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4371 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux ab83f86587a7 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / af13c6d4c6 |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4371/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4371/1/testReport/ |
| Max. process+thread count | 2672 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-balancer hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4371/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] d-c-manning commented on pull request #4371: HBASE-22349 slop in StochasticLoadBalancer
Posted by GitBox <gi...@apache.org>.
d-c-manning commented on PR #4371:
URL: https://github.com/apache/hbase/pull/4371#issuecomment-1112567802
test failures don't appear to be related.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4371: HBASE-22349 slop in StochasticLoadBalancer
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4371:
URL: https://github.com/apache/hbase/pull/4371#issuecomment-1109691984
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 44s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 23s | master passed |
| +1 :green_heart: | compile | 1m 0s | master passed |
| +1 :green_heart: | shadedjars | 3m 59s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 44s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 7s | the patch passed |
| +1 :green_heart: | compile | 1m 0s | the patch passed |
| +1 :green_heart: | javac | 1m 0s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 1s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 48s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 26s | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 11m 11s | hbase-balancer in the patch passed. |
| +1 :green_heart: | unit | 213m 7s | hbase-server in the patch passed. |
| | | 246m 1s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4371/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4371 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 83283906398a 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / af13c6d4c6 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4371/1/testReport/ |
| Max. process+thread count | 2718 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-balancer hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4371/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4371: HBASE-22349 slop in StochasticLoadBalancer
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4371:
URL: https://github.com/apache/hbase/pull/4371#issuecomment-1109496909
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 41s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 44s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 4s | master passed |
| +1 :green_heart: | compile | 2m 59s | master passed |
| +1 :green_heart: | checkstyle | 0m 52s | master passed |
| +0 :ok: | refguide | 2m 3s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. |
| +1 :green_heart: | spotbugs | 1m 55s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 13s | the patch passed |
| +1 :green_heart: | compile | 3m 11s | the patch passed |
| +1 :green_heart: | javac | 3m 11s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 12s | The patch passed checkstyle in hbase-common |
| +1 :green_heart: | checkstyle | 0m 7s | hbase-balancer: The patch generated 0 new + 15 unchanged - 64 fixed = 15 total (was 79) |
| +1 :green_heart: | checkstyle | 0m 33s | The patch passed checkstyle in hbase-server |
| -0 :warning: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. |
| +0 :ok: | refguide | 1m 38s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. |
| +1 :green_heart: | hadoopcheck | 14m 30s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. |
| +1 :green_heart: | spotbugs | 3m 20s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 26s | The patch does not generate ASF License warnings. |
| | | 44m 40s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4371/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4371 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile refguide xml |
| uname | Linux a6ce397cd769 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / af13c6d4c6 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| refguide | https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-4371/1/yetus-general-check/output/branch-site/book.html |
| whitespace | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4371/1/artifact/yetus-general-check/output/whitespace-eol.txt |
| refguide | https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-4371/1/yetus-general-check/output/patch-site/book.html |
| Max. process+thread count | 64 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-balancer hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4371/1/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] apurtell merged pull request #4371: HBASE-22349 slop in StochasticLoadBalancer
Posted by GitBox <gi...@apache.org>.
apurtell merged PR #4371:
URL: https://github.com/apache/hbase/pull/4371
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org