You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2020/08/23 09:22:41 UTC
[GitHub] [hbase] rda3mon opened a new pull request #2293: HBASE-24931 Fix for Candidate generator getAction method ignoring 0th…
rda3mon opened a new pull request #2293:
URL: https://github.com/apache/hbase/pull/2293
… index
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] rda3mon commented on pull request #2293: HBASE-24931 Fix for Candidate generator getAction method ignoring 0th…
Posted by GitBox <gi...@apache.org>.
rda3mon commented on pull request #2293:
URL: https://github.com/apache/hbase/pull/2293#issuecomment-678783527
Fixing broken tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #2293: HBASE-24931 Fix for Candidate generator getAction method ignoring 0th…
Posted by GitBox <gi...@apache.org>.
Apache9 commented on pull request #2293:
URL: https://github.com/apache/hbase/pull/2293#issuecomment-799419960
FIxed by #3024
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] rda3mon commented on pull request #2293: HBASE-24931 Fix for Candidate generator getAction method ignoring 0th…
Posted by GitBox <gi...@apache.org>.
rda3mon commented on pull request #2293:
URL: https://github.com/apache/hbase/pull/2293#issuecomment-678760008
Blocks to dead favorednodes spill over to any other datanode without rsgroup barrier, which wasn't acceptable as wanted a tighter isolation
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2293: HBASE-24931 Fix for Candidate generator getAction method ignoring 0th…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2293:
URL: https://github.com/apache/hbase/pull/2293#issuecomment-678765389
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 25s | Docker mode activated. |
| -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 19s | master passed |
| +1 :green_heart: | compile | 1m 6s | master passed |
| +1 :green_heart: | shadedjars | 5m 49s | branch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 42s | hbase-server in master failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 1s | the patch passed |
| +1 :green_heart: | compile | 1m 3s | the patch passed |
| +1 :green_heart: | javac | 1m 3s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 45s | patch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 39s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| -1 :x: | unit | 130m 2s | hbase-server in the patch failed. |
| | | 156m 1s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2293/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2293 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 4254ce565be3 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 9f62a82334 |
| Default Java | 2020-01-14 |
| javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2293/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2293/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2293/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2293/1/testReport/ |
| Max. process+thread count | 3890 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2293/1/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on pull request #2293: HBASE-24931 Fix for Candidate generator getAction method ignoring 0th…
Posted by GitBox <gi...@apache.org>.
virajjasani commented on pull request #2293:
URL: https://github.com/apache/hbase/pull/2293#issuecomment-687808943
Thanks @rda3mon for all your efforts. IMHO, test might require modification in this case. However, to cross verify, you can initiate a mail thread on `dev@hbase` asking if anyone with better understanding of `TestStochasticLoadBalancerHeterogeneousCost` can confirm whether your source code change is directly proportional to test change and also if it is fine increasing `ALLOWED_WINDOW` or any better approach should help.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2293: HBASE-24931 Fix for Candidate generator getAction method ignoring 0th…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2293:
URL: https://github.com/apache/hbase/pull/2293#issuecomment-678753871
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 28s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 41s | master passed |
| +1 :green_heart: | checkstyle | 1m 5s | master passed |
| +1 :green_heart: | spotbugs | 1m 58s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 26s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 1s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 11m 15s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. |
| +1 :green_heart: | spotbugs | 2m 7s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. |
| | | 32m 25s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2293/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2293 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux 70200f6d7cc5 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 9f62a82334 |
| Max. process+thread count | 94 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2293/1/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] rda3mon removed a comment on pull request #2293: HBASE-24931 Fix for Candidate generator getAction method ignoring 0th…
Posted by GitBox <gi...@apache.org>.
rda3mon removed a comment on pull request #2293:
URL: https://github.com/apache/hbase/pull/2293#issuecomment-678760008
Blocks to dead favorednodes spill over to any other datanode without rsgroup barrier, which wasn't acceptable as wanted a tighter isolation
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] rda3mon commented on pull request #2293: HBASE-24931 Fix for Candidate generator getAction method ignoring 0th…
Posted by GitBox <gi...@apache.org>.
rda3mon commented on pull request #2293:
URL: https://github.com/apache/hbase/pull/2293#issuecomment-678759833
@virajjasani I noticed this bug while writing a custom Candidate generator for reassigning those regions which had one of their favorednodes as dead. Cost Multiplier was on the higher side, and hence it was always balancing while 1 region was never reassigned.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on pull request #2293: HBASE-24931 Fix for Candidate generator getAction method ignoring 0th…
Posted by GitBox <gi...@apache.org>.
virajjasani commented on pull request #2293:
URL: https://github.com/apache/hbase/pull/2293#issuecomment-678807715
@busbey would you like to take a look?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2293: HBASE-24931 Fix for Candidate generator getAction method ignoring 0th…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2293:
URL: https://github.com/apache/hbase/pull/2293#issuecomment-678767453
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 47s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 23s | master passed |
| +1 :green_heart: | compile | 1m 8s | master passed |
| +1 :green_heart: | shadedjars | 6m 49s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 38s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 56s | the patch passed |
| +1 :green_heart: | compile | 1m 7s | the patch passed |
| +1 :green_heart: | javac | 1m 7s | the patch passed |
| +1 :green_heart: | shadedjars | 6m 30s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 39s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 149m 24s | hbase-server in the patch failed. |
| | | 177m 17s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2293/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2293 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 73532a5fcfd0 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 9f62a82334 |
| Default Java | 1.8.0_232 |
| unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2293/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2293/1/testReport/ |
| Max. process+thread count | 3701 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2293/1/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on pull request #2293: HBASE-24931 Fix for Candidate generator getAction method ignoring 0th…
Posted by GitBox <gi...@apache.org>.
virajjasani commented on pull request #2293:
URL: https://github.com/apache/hbase/pull/2293#issuecomment-678954465
@rda3mon did you miss git push by any chance when you said "Fixing broken tests" ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on pull request #2293: HBASE-24931 Fix for Candidate generator getAction method ignoring 0th…
Posted by GitBox <gi...@apache.org>.
virajjasani commented on pull request #2293:
URL: https://github.com/apache/hbase/pull/2293#issuecomment-678979251
No worries, please take your time.
Thanks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] rda3mon commented on pull request #2293: HBASE-24931 Fix for Candidate generator getAction method ignoring 0th…
Posted by GitBox <gi...@apache.org>.
rda3mon commented on pull request #2293:
URL: https://github.com/apache/hbase/pull/2293#issuecomment-688038129
@virajjasani Original author @PierreZ will be able to look at this post 14th Sep as he doesn't have access to laptop until then.. So, will wait till then.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] rda3mon commented on pull request #2293: HBASE-24931 Fix for Candidate generator getAction method ignoring 0th…
Posted by GitBox <gi...@apache.org>.
rda3mon commented on pull request #2293:
URL: https://github.com/apache/hbase/pull/2293#issuecomment-688039152
@virajjasani Increase in ALLOWED_WINDOW will not be of much help. Because, the deviation isn't small. Expected 50 * 0.4 = 20. Actual is 49.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] rda3mon commented on pull request #2293: HBASE-24931 Fix for Candidate generator getAction method ignoring 0th…
Posted by GitBox <gi...@apache.org>.
rda3mon commented on pull request #2293:
URL: https://github.com/apache/hbase/pull/2293#issuecomment-760676633
@virajjasani I am closing this as I couldn't fix the test cases and it was very tricky to narrow down the issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] rda3mon commented on pull request #2293: HBASE-24931 Fix for Candidate generator getAction method ignoring 0th…
Posted by GitBox <gi...@apache.org>.
rda3mon commented on pull request #2293:
URL: https://github.com/apache/hbase/pull/2293#issuecomment-678960422
@virajjasani I need another day or so. It requires more involved understanding to fix broken tests. I will get back once I fix them
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] rda3mon commented on pull request #2293: HBASE-24931 Fix for Candidate generator getAction method ignoring 0th…
Posted by GitBox <gi...@apache.org>.
rda3mon commented on pull request #2293:
URL: https://github.com/apache/hbase/pull/2293#issuecomment-687619263
@virajjasani I spent a good day and half. Couldn't solve this issue. Tagging origin author to help me on this @PierreZ and have written a detailed email.
Brief detail:
On correcting this line `if (fromRegion >= 0 && toRegion >= 0) {` from `if (fromRegion > 0 && toRegion > 0) {`. `TestStochasticLoadBalancerHeterogeneousCost` tests broke down.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] rda3mon edited a comment on pull request #2293: HBASE-24931 Fix for Candidate generator getAction method ignoring 0th…
Posted by GitBox <gi...@apache.org>.
rda3mon edited a comment on pull request #2293:
URL: https://github.com/apache/hbase/pull/2293#issuecomment-678759833
@virajjasani I noticed this bug while writing a custom Candidate generator for reassigning those regions which had one of their favorednodes as dead. Cost Multiplier was on the higher side, and hence it was always balancing while 1 region was never reassigned.
Blocks to dead favorednodes spill over to any other datanode without rsgroup barrier, which wasn't acceptable as wanted a tighter isolation
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 closed pull request #2293: HBASE-24931 Fix for Candidate generator getAction method ignoring 0th…
Posted by GitBox <gi...@apache.org>.
Apache9 closed pull request #2293:
URL: https://github.com/apache/hbase/pull/2293
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org