You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2020/12/10 08:37:32 UTC
[GitHub] [hbase] caroliney14 opened a new pull request #2762: HBASE-25329 Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
caroliney14 opened a new pull request #2762:
URL: https://github.com/apache/hbase/pull/2762
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2762: HBASE-25329 Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2762:
URL: https://github.com/apache/hbase/pull/2762#issuecomment-759358586
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 6m 44s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ branch-1 Compile Tests _ |
| +0 :ok: | mvndep | 2m 24s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 7m 56s | branch-1 passed |
| +1 :green_heart: | compile | 1m 9s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | compile | 1m 20s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +1 :green_heart: | checkstyle | 2m 10s | branch-1 passed |
| +1 :green_heart: | shadedjars | 3m 10s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 13s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javadoc | 1m 20s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +0 :ok: | spotbugs | 2m 47s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 4m 15s | branch-1 passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 2s | the patch passed |
| +1 :green_heart: | compile | 1m 14s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javac | 1m 14s | the patch passed |
| +1 :green_heart: | compile | 1m 20s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +1 :green_heart: | javac | 1m 20s | the patch passed |
| +1 :green_heart: | checkstyle | 2m 6s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedjars | 2m 54s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | hadoopcheck | 5m 26s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. |
| +1 :green_heart: | javadoc | 1m 1s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javadoc | 1m 27s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| -1 :x: | findbugs | 3m 32s | hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 28s | hbase-hadoop-compat in the patch passed. |
| +1 :green_heart: | unit | 0m 40s | hbase-hadoop2-compat in the patch passed. |
| +1 :green_heart: | unit | 101m 29s | hbase-server in the patch passed. |
| +1 :green_heart: | asflicense | 1m 14s | The patch does not generate ASF License warnings. |
| | | 163m 19s | |
| Reason | Tests |
|-------:|:------|
| FindBugs | module:hbase-server |
| | org.apache.hadoop.hbase.master.AssignmentManager.updateRegionsInTransitionMetrics() makes inefficient use of keySet iterator instead of entrySet iterator At AssignmentManager.java:keySet iterator instead of entrySet iterator At AssignmentManager.java:[line 3561] |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/5/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2762 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux c941e1ef10ee 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2762/out/precommit/personality/provided.sh |
| git revision | branch-1 / a6b0d5b |
| Default Java | Azul Systems, Inc.-1.7.0_272-b10 |
| Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10 |
| findbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/5/artifact/out/new-findbugs-hbase-server.html |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/5/testReport/ |
| Max. process+thread count | 4264 (vs. ulimit of 10000) |
| modules | C: hbase-hadoop-compat hbase-hadoop2-compat hbase-server U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/5/console |
| versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2762: HBASE-25329 Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2762:
URL: https://github.com/apache/hbase/pull/2762#issuecomment-745882510
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 35s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ branch-1 Compile Tests _ |
| +0 :ok: | mvndep | 2m 23s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 7m 59s | branch-1 passed |
| +1 :green_heart: | compile | 1m 11s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | compile | 1m 19s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +1 :green_heart: | checkstyle | 2m 16s | branch-1 passed |
| +1 :green_heart: | shadedjars | 3m 2s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 9s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javadoc | 1m 17s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +0 :ok: | spotbugs | 2m 43s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 4m 6s | branch-1 passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 52s | the patch passed |
| +1 :green_heart: | compile | 1m 11s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javac | 1m 11s | the patch passed |
| +1 :green_heart: | compile | 1m 19s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +1 :green_heart: | javac | 1m 19s | the patch passed |
| +1 :green_heart: | checkstyle | 2m 8s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedjars | 2m 51s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | hadoopcheck | 4m 33s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. |
| +1 :green_heart: | javadoc | 1m 0s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javadoc | 1m 13s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +1 :green_heart: | findbugs | 4m 26s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 29s | hbase-hadoop-compat in the patch passed. |
| +1 :green_heart: | unit | 0m 39s | hbase-hadoop2-compat in the patch passed. |
| +1 :green_heart: | unit | 113m 19s | hbase-server in the patch passed. |
| +1 :green_heart: | asflicense | 1m 25s | The patch does not generate ASF License warnings. |
| | | 166m 9s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/4/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2762 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 79e82eb440e3 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2762/out/precommit/personality/provided.sh |
| git revision | branch-1 / 8e45165 |
| Default Java | Azul Systems, Inc.-1.7.0_272-b10 |
| Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/4/testReport/ |
| Max. process+thread count | 4202 (vs. ulimit of 10000) |
| modules | C: hbase-hadoop-compat hbase-hadoop2-compat hbase-server U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/4/console |
| versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #2762: HBASE-25329 Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
Posted by GitBox <gi...@apache.org>.
virajjasani commented on a change in pull request #2762:
URL: https://github.com/apache/hbase/pull/2762#discussion_r556370161
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
##########
@@ -3538,18 +3538,31 @@ public void updateRegionsInTransitionMetrics() {
int totalRITs = 0;
int totalRITsOverThreshold = 0;
long oldestRITTime = 0;
+ HashMap<String, RegionState> ritsOverThreshold = null;
Review comment:
nit: `Map<String, RegionState> ritsOverThreshold = null`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #2762: HBASE-25329 Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
Posted by GitBox <gi...@apache.org>.
virajjasani commented on a change in pull request #2762:
URL: https://github.com/apache/hbase/pull/2762#discussion_r560702118
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
##########
@@ -3538,18 +3538,32 @@ public void updateRegionsInTransitionMetrics() {
int totalRITs = 0;
int totalRITsOverThreshold = 0;
long oldestRITTime = 0;
+ Map<String, RegionState> ritsOverThreshold = null;
int ritThreshold = this.server.getConfiguration().
getInt(HConstants.METRICS_RIT_STUCK_WARNING_THRESHOLD, 60000);
for (RegionState state: regionStates.getRegionsInTransition()) {
totalRITs++;
long ritTime = currentTime - state.getStamp();
if (ritTime > ritThreshold) { // more than the threshold
totalRITsOverThreshold++;
+ if (ritsOverThreshold == null) {
+ ritsOverThreshold = new HashMap<>();
+ }
+ ritsOverThreshold.put(state.getRegion().getEncodedName(), state);
}
if (oldestRITTime < ritTime) {
oldestRITTime = ritTime;
}
}
+ if (LOG.isDebugEnabled() && ritsOverThreshold != null && !ritsOverThreshold.isEmpty()) {
+ StringBuilder sb = new StringBuilder();
+ for (Map.Entry<String, RegionState> regionName: ritsOverThreshold.entrySet()) {
+ sb.append(regionName).append(":")
+ .append(ritsOverThreshold.get(regionName).getState().name()).append("\n");
Review comment:
This should be:
```
sb.append(regionName.getKey()).append(":")
.append(regionName.getValue().getState().name()).append("\n");
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2762: HBASE-25329 Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2762:
URL: https://github.com/apache/hbase/pull/2762#issuecomment-763098345
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 48s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ branch-1 Compile Tests _ |
| +0 :ok: | mvndep | 2m 25s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 7m 57s | branch-1 passed |
| +1 :green_heart: | compile | 1m 14s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | compile | 1m 21s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +1 :green_heart: | checkstyle | 2m 12s | branch-1 passed |
| +1 :green_heart: | shadedjars | 3m 4s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 11s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javadoc | 1m 15s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +0 :ok: | spotbugs | 2m 41s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 4m 6s | branch-1 passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 55s | the patch passed |
| +1 :green_heart: | compile | 1m 10s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javac | 1m 10s | the patch passed |
| +1 :green_heart: | compile | 1m 20s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +1 :green_heart: | javac | 1m 20s | the patch passed |
| +1 :green_heart: | checkstyle | 2m 11s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedjars | 2m 55s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | hadoopcheck | 4m 44s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. |
| +1 :green_heart: | javadoc | 1m 1s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javadoc | 1m 16s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| -1 :x: | findbugs | 2m 57s | hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 29s | hbase-hadoop-compat in the patch passed. |
| +1 :green_heart: | unit | 0m 39s | hbase-hadoop2-compat in the patch passed. |
| -1 :x: | unit | 48m 10s | hbase-server in the patch failed. |
| +1 :green_heart: | asflicense | 0m 50s | The patch does not generate ASF License warnings. |
| | | 101m 7s | |
| Reason | Tests |
|-------:|:------|
| FindBugs | module:hbase-server |
| | java.util.Map$Entry<java.lang.String,org.apache.hadoop.hbase.master.RegionState> is incompatible with expected argument type String in org.apache.hadoop.hbase.master.AssignmentManager.updateRegionsInTransitionMetrics() At AssignmentManager.java:argument type String in org.apache.hadoop.hbase.master.AssignmentManager.updateRegionsInTransitionMetrics() At AssignmentManager.java:[line 3561] |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/7/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2762 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 63e88abccdce 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2762/out/precommit/personality/provided.sh |
| git revision | branch-1 / a6b0d5b |
| Default Java | Azul Systems, Inc.-1.7.0_272-b10 |
| Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10 |
| findbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/7/artifact/out/new-findbugs-hbase-server.html |
| unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/7/artifact/out/patch-unit-hbase-server.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/7/testReport/ |
| Max. process+thread count | 768 (vs. ulimit of 10000) |
| modules | C: hbase-hadoop-compat hbase-hadoop2-compat hbase-server U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/7/console |
| versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2762: HBASE-25329 Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2762:
URL: https://github.com/apache/hbase/pull/2762#issuecomment-760660854
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 42s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ branch-1 Compile Tests _ |
| +0 :ok: | mvndep | 2m 24s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 7m 49s | branch-1 passed |
| +1 :green_heart: | compile | 1m 10s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | compile | 1m 19s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +1 :green_heart: | checkstyle | 2m 15s | branch-1 passed |
| +1 :green_heart: | shadedjars | 3m 4s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 8s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javadoc | 1m 18s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +0 :ok: | spotbugs | 2m 43s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 4m 5s | branch-1 passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 56s | the patch passed |
| +1 :green_heart: | compile | 1m 9s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javac | 1m 9s | the patch passed |
| +1 :green_heart: | compile | 1m 20s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +1 :green_heart: | javac | 1m 20s | the patch passed |
| +1 :green_heart: | checkstyle | 2m 7s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedjars | 2m 50s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | hadoopcheck | 4m 49s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. |
| +1 :green_heart: | javadoc | 1m 0s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javadoc | 1m 14s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| -1 :x: | findbugs | 2m 54s | hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 28s | hbase-hadoop-compat in the patch passed. |
| +1 :green_heart: | unit | 0m 40s | hbase-hadoop2-compat in the patch passed. |
| +1 :green_heart: | unit | 107m 28s | hbase-server in the patch passed. |
| +1 :green_heart: | asflicense | 1m 10s | The patch does not generate ASF License warnings. |
| | | 160m 19s | |
| Reason | Tests |
|-------:|:------|
| FindBugs | module:hbase-server |
| | org.apache.hadoop.hbase.master.AssignmentManager.updateRegionsInTransitionMetrics() makes inefficient use of keySet iterator instead of entrySet iterator At AssignmentManager.java:keySet iterator instead of entrySet iterator At AssignmentManager.java:[line 3561] |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/6/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2762 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux e8dbcb6414de 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2762/out/precommit/personality/provided.sh |
| git revision | branch-1 / a6b0d5b |
| Default Java | Azul Systems, Inc.-1.7.0_272-b10 |
| Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10 |
| findbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/6/artifact/out/new-findbugs-hbase-server.html |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/6/testReport/ |
| Max. process+thread count | 4147 (vs. ulimit of 10000) |
| modules | C: hbase-hadoop-compat hbase-hadoop2-compat hbase-server U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/6/console |
| versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] caroliney14 commented on a change in pull request #2762: HBASE-25329 Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
Posted by GitBox <gi...@apache.org>.
caroliney14 commented on a change in pull request #2762:
URL: https://github.com/apache/hbase/pull/2762#discussion_r543846000
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
##########
@@ -3549,11 +3554,24 @@ public void updateRegionsInTransitionMetrics() {
if (oldestRITTime < ritTime) {
oldestRITTime = ritTime;
}
+ if (counter < 500) { // Record 500 oldest RITs
+ oldestRITHashesAndStates.add(
+ state.getRegion().getRegionNameAsString() + ":" + state.getState().name()
Review comment:
as for `state`, that is an enum, and we want the state name, which is why I added `.name()`. see [this](https://stackoverflow.com/a/17741770) SO
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] d-c-manning commented on a change in pull request #2762: HBASE-25329 Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
Posted by GitBox <gi...@apache.org>.
d-c-manning commented on a change in pull request #2762:
URL: https://github.com/apache/hbase/pull/2762#discussion_r541445488
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
##########
@@ -3549,11 +3554,24 @@ public void updateRegionsInTransitionMetrics() {
if (oldestRITTime < ritTime) {
oldestRITTime = ritTime;
}
+ if (counter < 500) { // Record 500 oldest RITs
Review comment:
500 seems high to me. Would 20 be enough? Should this be a constant in the file, even if not configurable?
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
##########
@@ -3549,11 +3554,24 @@ public void updateRegionsInTransitionMetrics() {
if (oldestRITTime < ritTime) {
oldestRITTime = ritTime;
}
+ if (counter < 500) { // Record 500 oldest RITs
+ oldestRITHashesAndStates.add(
+ state.getRegion().getRegionNameAsString() + ":" + state.getState().name()
+ );
+ }
+ counter += 1;
}
if (this.metricsAssignmentManager != null) {
this.metricsAssignmentManager.updateRITOldestAge(oldestRITTime);
this.metricsAssignmentManager.updateRITCount(totalRITs);
this.metricsAssignmentManager.updateRITCountOverThreshold(totalRITsOverThreshold);
+
+ LOG.debug("Oldest RIT hashes and states: " + oldestRITHashesAndStates.toString());
+ long time = EnvironmentEdgeManager.currentTime();
+ if ((time - ritThreshold / 2) >= this.lastRITHashMetricUpdate) {
Review comment:
Why do we update the metrics hashes only if it's been more than `ritThreshold / 2` time since last update? If we do the work to find the oldest, it seems like we should update the metrics always. Then any query to get metrics will always have the most recent results (~3 seconds old at max, with default `hbase.regionserver.msginterval`)
Though I do think it's a good idea for limiting the logging. Should the `LOG.debug` statement be in here?
Then this statement may deserve a comment.
```suggestion
// Only log if it has been long enough since the last update (default 30 seconds)
if ((time - ritThreshold / 2) >= this.lastRITHashMetricUpdate) {
```
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
##########
@@ -3538,9 +3541,11 @@ public void updateRegionsInTransitionMetrics() {
int totalRITs = 0;
int totalRITsOverThreshold = 0;
long oldestRITTime = 0;
+ Set<String> oldestRITHashesAndStates = Sets.newHashSet(); // set of <rit hash>:<rit state>
int ritThreshold = this.server.getConfiguration().
getInt(HConstants.METRICS_RIT_STUCK_WARNING_THRESHOLD, 60000);
- for (RegionState state: regionStates.getRegionsInTransition()) {
+ int counter = 0;
+ for (RegionState state: regionStates.getRegionsInTransitionOrderedByDuration()) {
Review comment:
since this method uses the `state.getStamp()` to determine whether the RIT is over the threshold or the oldest time... should we just use `getRegionsInTransitionOrderedByTimestamp` instead?
This new method `getRegionsInTransitionOrderedByDuration` is tracking total duration instead of duration since last state transition, so the longest RITs may not necessarily be those that are reported over threshold. I think it's probably good to be consistent... I'm not sure which way is better to be consistent, but since these metrics in this method are reporting time since last change instead of total duration, I'd favor that approach.
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
##########
@@ -3549,11 +3554,24 @@ public void updateRegionsInTransitionMetrics() {
if (oldestRITTime < ritTime) {
oldestRITTime = ritTime;
}
+ if (counter < 500) { // Record 500 oldest RITs
+ oldestRITHashesAndStates.add(
+ state.getRegion().getRegionNameAsString() + ":" + state.getState().name()
+ );
+ }
+ counter += 1;
}
if (this.metricsAssignmentManager != null) {
this.metricsAssignmentManager.updateRITOldestAge(oldestRITTime);
this.metricsAssignmentManager.updateRITCount(totalRITs);
this.metricsAssignmentManager.updateRITCountOverThreshold(totalRITsOverThreshold);
+
+ LOG.debug("Oldest RIT hashes and states: " + oldestRITHashesAndStates.toString());
Review comment:
we should only log if `oldestRITHashesAndStates` is non-empty.
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
##########
@@ -3549,11 +3554,24 @@ public void updateRegionsInTransitionMetrics() {
if (oldestRITTime < ritTime) {
oldestRITTime = ritTime;
}
+ if (counter < 500) { // Record 500 oldest RITs
+ oldestRITHashesAndStates.add(
+ state.getRegion().getRegionNameAsString() + ":" + state.getState().name()
+ );
+ }
+ counter += 1;
}
if (this.metricsAssignmentManager != null) {
this.metricsAssignmentManager.updateRITOldestAge(oldestRITTime);
this.metricsAssignmentManager.updateRITCount(totalRITs);
this.metricsAssignmentManager.updateRITCountOverThreshold(totalRITsOverThreshold);
+
+ LOG.debug("Oldest RIT hashes and states: " + oldestRITHashesAndStates.toString());
+ long time = EnvironmentEdgeManager.currentTime();
Review comment:
can we just reuse `currentTime` here from the beginning of the method?
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
##########
@@ -3549,11 +3554,24 @@ public void updateRegionsInTransitionMetrics() {
if (oldestRITTime < ritTime) {
oldestRITTime = ritTime;
}
+ if (counter < 500) { // Record 500 oldest RITs
+ oldestRITHashesAndStates.add(
+ state.getRegion().getRegionNameAsString() + ":" + state.getState().name()
Review comment:
Do we have to use `name()` as I see other references use `state.getState()` directly.
region names could be quite long - when you refer to hash, did you want the md5 encoded name?
```suggestion
state.getRegion().getEncodedName() + ":" + state.getState()
```
Or should the entire region name be available?
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
##########
@@ -3549,11 +3554,24 @@ public void updateRegionsInTransitionMetrics() {
if (oldestRITTime < ritTime) {
oldestRITTime = ritTime;
}
+ if (counter < 500) { // Record 500 oldest RITs
+ oldestRITHashesAndStates.add(
Review comment:
should we also filter these to only those RITs that are over the threshold? Or do we want to display every RIT, even if it's only been transitioning for 100ms, if it's the longest/only RIT in the cluster?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] bharathv commented on pull request #2762: HBASE-25329 Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
Posted by GitBox <gi...@apache.org>.
bharathv commented on pull request #2762:
URL: https://github.com/apache/hbase/pull/2762#issuecomment-755010117
Thanks for putting this together. Definitely useful to have more insights into RIT states. I added a comment in the jira. Would like to know your thoughts.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on pull request #2762: HBASE-25329 Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
Posted by GitBox <gi...@apache.org>.
virajjasani commented on pull request #2762:
URL: https://github.com/apache/hbase/pull/2762#issuecomment-756002541
I just realized that this branch-1 backport PR also has review comments.
Since consolidated discussion is happening on Jira, when @caroliney14 refresh PRs, we can start reviewing master branch PR #2761 first else it might be difficult to track separate reviews happening on separate PRs :)
Let me mark this PR with backport label. Thanks @caroliney14
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] caroliney14 commented on pull request #2762: HBASE-25329 Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
Posted by GitBox <gi...@apache.org>.
caroliney14 commented on pull request #2762:
URL: https://github.com/apache/hbase/pull/2762#issuecomment-754316224
@virajjasani @bharathv do you mind taking a look at this? thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2762: HBASE-25329 Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2762:
URL: https://github.com/apache/hbase/pull/2762#issuecomment-742879654
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 45s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ branch-1 Compile Tests _ |
| +0 :ok: | mvndep | 2m 21s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 8m 22s | branch-1 passed |
| +1 :green_heart: | compile | 1m 18s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | compile | 1m 27s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +1 :green_heart: | checkstyle | 2m 26s | branch-1 passed |
| +1 :green_heart: | shadedjars | 3m 37s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 16s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javadoc | 1m 24s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +0 :ok: | spotbugs | 3m 29s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 5m 0s | branch-1 passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 20s | the patch passed |
| +1 :green_heart: | compile | 1m 14s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javac | 1m 14s | the patch passed |
| +1 :green_heart: | compile | 1m 23s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +1 :green_heart: | javac | 1m 23s | the patch passed |
| -1 :x: | checkstyle | 1m 38s | hbase-server: The patch generated 2 new + 251 unchanged - 0 fixed = 253 total (was 251) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedjars | 2m 56s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | hadoopcheck | 4m 50s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. |
| +1 :green_heart: | javadoc | 1m 0s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javadoc | 1m 15s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +1 :green_heart: | findbugs | 4m 29s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 28s | hbase-hadoop-compat in the patch passed. |
| +1 :green_heart: | unit | 0m 37s | hbase-hadoop2-compat in the patch passed. |
| +1 :green_heart: | unit | 105m 14s | hbase-server in the patch passed. |
| +1 :green_heart: | asflicense | 1m 1s | The patch does not generate ASF License warnings. |
| | | 161m 48s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2762 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 71f6463d256d 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2762/out/precommit/personality/provided.sh |
| git revision | branch-1 / 8e45165 |
| Default Java | Azul Systems, Inc.-1.7.0_272-b10 |
| Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10 |
| checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/2/artifact/out/diff-checkstyle-hbase-server.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/2/testReport/ |
| Max. process+thread count | 4262 (vs. ulimit of 10000) |
| modules | C: hbase-hadoop-compat hbase-hadoop2-compat hbase-server U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/2/console |
| versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2762: HBASE-25329 Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2762:
URL: https://github.com/apache/hbase/pull/2762#issuecomment-745771727
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 36s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ branch-1 Compile Tests _ |
| +0 :ok: | mvndep | 2m 24s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 7m 53s | branch-1 passed |
| +1 :green_heart: | compile | 1m 8s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | compile | 1m 17s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +1 :green_heart: | checkstyle | 2m 12s | branch-1 passed |
| +1 :green_heart: | shadedjars | 3m 2s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 8s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javadoc | 1m 15s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +0 :ok: | spotbugs | 2m 43s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 4m 9s | branch-1 passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 56s | the patch passed |
| +1 :green_heart: | compile | 1m 9s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javac | 1m 9s | the patch passed |
| +1 :green_heart: | compile | 1m 19s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +1 :green_heart: | javac | 1m 19s | the patch passed |
| -1 :x: | checkstyle | 1m 37s | hbase-server: The patch generated 2 new + 251 unchanged - 0 fixed = 253 total (was 251) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedjars | 2m 50s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | hadoopcheck | 4m 39s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. |
| +1 :green_heart: | javadoc | 1m 1s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javadoc | 1m 17s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +1 :green_heart: | findbugs | 4m 20s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 29s | hbase-hadoop-compat in the patch passed. |
| +1 :green_heart: | unit | 0m 39s | hbase-hadoop2-compat in the patch passed. |
| -1 :x: | unit | 107m 2s | hbase-server in the patch failed. |
| +1 :green_heart: | asflicense | 1m 13s | The patch does not generate ASF License warnings. |
| | | 159m 31s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hbase.master.TestAssignmentManagerMetrics |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2762 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux b3d95c899b03 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2762/out/precommit/personality/provided.sh |
| git revision | branch-1 / 8e45165 |
| Default Java | Azul Systems, Inc.-1.7.0_272-b10 |
| Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10 |
| checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/3/artifact/out/diff-checkstyle-hbase-server.txt |
| unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/3/artifact/out/patch-unit-hbase-server.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/3/testReport/ |
| Max. process+thread count | 4001 (vs. ulimit of 10000) |
| modules | C: hbase-hadoop-compat hbase-hadoop2-compat hbase-server U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/3/console |
| versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani merged pull request #2762: HBASE-25329 Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
Posted by GitBox <gi...@apache.org>.
virajjasani merged pull request #2762:
URL: https://github.com/apache/hbase/pull/2762
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2762: HBASE-25329 Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2762:
URL: https://github.com/apache/hbase/pull/2762#issuecomment-763191977
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 36s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ branch-1 Compile Tests _ |
| +0 :ok: | mvndep | 2m 23s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 7m 52s | branch-1 passed |
| +1 :green_heart: | compile | 0m 55s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | compile | 1m 1s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +1 :green_heart: | checkstyle | 1m 58s | branch-1 passed |
| +1 :green_heart: | shadedjars | 3m 2s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 55s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javadoc | 0m 58s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +0 :ok: | spotbugs | 2m 41s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 26s | branch-1 passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 56s | the patch passed |
| +1 :green_heart: | compile | 0m 53s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javac | 0m 53s | the patch passed |
| +1 :green_heart: | compile | 0m 59s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +1 :green_heart: | javac | 0m 59s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 53s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedjars | 2m 52s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | hadoopcheck | 4m 39s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. |
| +1 :green_heart: | javadoc | 0m 46s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javadoc | 0m 58s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| -1 :x: | findbugs | 2m 57s | hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 30s | hbase-hadoop-compat in the patch passed. |
| -1 :x: | unit | 105m 48s | hbase-server in the patch failed. |
| +1 :green_heart: | asflicense | 0m 53s | The patch does not generate ASF License warnings. |
| | | 152m 57s | |
| Reason | Tests |
|-------:|:------|
| FindBugs | module:hbase-server |
| | java.util.Map$Entry<java.lang.String,org.apache.hadoop.hbase.master.RegionState> is incompatible with expected argument type String in org.apache.hadoop.hbase.master.AssignmentManager.updateRegionsInTransitionMetrics() At AssignmentManager.java:argument type String in org.apache.hadoop.hbase.master.AssignmentManager.updateRegionsInTransitionMetrics() At AssignmentManager.java:[line 3561] |
| Failed junit tests | hadoop.hbase.master.TestAssignmentManagerMetrics |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/8/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2762 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 03271de6b187 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2762/out/precommit/personality/provided.sh |
| git revision | branch-1 / a6b0d5b |
| Default Java | Azul Systems, Inc.-1.7.0_272-b10 |
| Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10 |
| findbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/8/artifact/out/new-findbugs-hbase-server.html |
| unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/8/artifact/out/patch-unit-hbase-server.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/8/testReport/ |
| Max. process+thread count | 4227 (vs. ulimit of 10000) |
| modules | C: hbase-hadoop-compat hbase-server U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/8/console |
| versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] caroliney14 commented on a change in pull request #2762: HBASE-25329 Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
Posted by GitBox <gi...@apache.org>.
caroliney14 commented on a change in pull request #2762:
URL: https://github.com/apache/hbase/pull/2762#discussion_r543842413
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
##########
@@ -3549,11 +3554,24 @@ public void updateRegionsInTransitionMetrics() {
if (oldestRITTime < ritTime) {
oldestRITTime = ritTime;
}
+ if (counter < 500) { // Record 500 oldest RITs
+ oldestRITHashesAndStates.add(
Review comment:
I think it makes sense to only record RITs over threshold, since those would be the problematic ones
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] caroliney14 commented on a change in pull request #2762: HBASE-25329 Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
Posted by GitBox <gi...@apache.org>.
caroliney14 commented on a change in pull request #2762:
URL: https://github.com/apache/hbase/pull/2762#discussion_r543848809
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
##########
@@ -3549,11 +3554,24 @@ public void updateRegionsInTransitionMetrics() {
if (oldestRITTime < ritTime) {
oldestRITTime = ritTime;
}
+ if (counter < 500) { // Record 500 oldest RITs
+ oldestRITHashesAndStates.add(
+ state.getRegion().getRegionNameAsString() + ":" + state.getState().name()
+ );
+ }
+ counter += 1;
}
if (this.metricsAssignmentManager != null) {
this.metricsAssignmentManager.updateRITOldestAge(oldestRITTime);
this.metricsAssignmentManager.updateRITCount(totalRITs);
this.metricsAssignmentManager.updateRITCountOverThreshold(totalRITsOverThreshold);
+
+ LOG.debug("Oldest RIT hashes and states: " + oldestRITHashesAndStates.toString());
+ long time = EnvironmentEdgeManager.currentTime();
+ if ((time - ritThreshold / 2) >= this.lastRITHashMetricUpdate) {
Review comment:
> Though I do think it's a good idea for limiting the logging. Should the LOG.debug statement be in here?
yes it is for the reason you stated. will add the comment and move `LOG.debug` under this condition. maybe the other stuff can be moved outside.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] caroliney14 commented on a change in pull request #2762: HBASE-25329 Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
Posted by GitBox <gi...@apache.org>.
caroliney14 commented on a change in pull request #2762:
URL: https://github.com/apache/hbase/pull/2762#discussion_r543848809
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
##########
@@ -3549,11 +3554,24 @@ public void updateRegionsInTransitionMetrics() {
if (oldestRITTime < ritTime) {
oldestRITTime = ritTime;
}
+ if (counter < 500) { // Record 500 oldest RITs
+ oldestRITHashesAndStates.add(
+ state.getRegion().getRegionNameAsString() + ":" + state.getState().name()
+ );
+ }
+ counter += 1;
}
if (this.metricsAssignmentManager != null) {
this.metricsAssignmentManager.updateRITOldestAge(oldestRITTime);
this.metricsAssignmentManager.updateRITCount(totalRITs);
this.metricsAssignmentManager.updateRITCountOverThreshold(totalRITsOverThreshold);
+
+ LOG.debug("Oldest RIT hashes and states: " + oldestRITHashesAndStates.toString());
+ long time = EnvironmentEdgeManager.currentTime();
+ if ((time - ritThreshold / 2) >= this.lastRITHashMetricUpdate) {
Review comment:
> Though I do think it's a good idea for limiting the logging. Should the LOG.debug statement be in here?
yes it is for the reason you stated. will add the comment and move `LOG.debug` under this condition
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
##########
@@ -3549,11 +3554,24 @@ public void updateRegionsInTransitionMetrics() {
if (oldestRITTime < ritTime) {
oldestRITTime = ritTime;
}
+ if (counter < 500) { // Record 500 oldest RITs
+ oldestRITHashesAndStates.add(
+ state.getRegion().getRegionNameAsString() + ":" + state.getState().name()
+ );
+ }
+ counter += 1;
}
if (this.metricsAssignmentManager != null) {
this.metricsAssignmentManager.updateRITOldestAge(oldestRITTime);
this.metricsAssignmentManager.updateRITCount(totalRITs);
this.metricsAssignmentManager.updateRITCountOverThreshold(totalRITsOverThreshold);
+
+ LOG.debug("Oldest RIT hashes and states: " + oldestRITHashesAndStates.toString());
+ long time = EnvironmentEdgeManager.currentTime();
+ if ((time - ritThreshold / 2) >= this.lastRITHashMetricUpdate) {
Review comment:
> Though I do think it's a good idea for limiting the logging. Should the LOG.debug statement be in here?
yes it is for the reason you stated. will add the comment and move `LOG.debug` under this condition
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2762: HBASE-25329 Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2762:
URL: https://github.com/apache/hbase/pull/2762#issuecomment-763455431
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 35s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ branch-1 Compile Tests _ |
| +0 :ok: | mvndep | 2m 22s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 8m 2s | branch-1 passed |
| +1 :green_heart: | compile | 0m 53s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | compile | 0m 59s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +1 :green_heart: | checkstyle | 1m 58s | branch-1 passed |
| +1 :green_heart: | shadedjars | 3m 2s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 54s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javadoc | 0m 57s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +0 :ok: | spotbugs | 2m 41s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 27s | branch-1 passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 59s | the patch passed |
| +1 :green_heart: | compile | 0m 56s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javac | 0m 56s | the patch passed |
| +1 :green_heart: | compile | 0m 59s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +1 :green_heart: | javac | 0m 59s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 52s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedjars | 2m 50s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | hadoopcheck | 4m 41s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. |
| +1 :green_heart: | javadoc | 0m 46s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javadoc | 1m 0s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +1 :green_heart: | findbugs | 3m 34s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 28s | hbase-hadoop-compat in the patch passed. |
| +1 :green_heart: | unit | 108m 56s | hbase-server in the patch passed. |
| +1 :green_heart: | asflicense | 0m 53s | The patch does not generate ASF License warnings. |
| | | 156m 10s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/9/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2762 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux df3d4acc9a8c 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2762/out/precommit/personality/provided.sh |
| git revision | branch-1 / a6b0d5b |
| Default Java | Azul Systems, Inc.-1.7.0_272-b10 |
| Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/9/testReport/ |
| Max. process+thread count | 4160 (vs. ulimit of 10000) |
| modules | C: hbase-hadoop-compat hbase-server U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/9/console |
| versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #2762: HBASE-25329 Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
Posted by GitBox <gi...@apache.org>.
virajjasani commented on a change in pull request #2762:
URL: https://github.com/apache/hbase/pull/2762#discussion_r556369037
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
##########
@@ -3538,18 +3538,31 @@ public void updateRegionsInTransitionMetrics() {
int totalRITs = 0;
int totalRITsOverThreshold = 0;
long oldestRITTime = 0;
+ HashMap<String, RegionState> ritsOverThreshold = null;
int ritThreshold = this.server.getConfiguration().
getInt(HConstants.METRICS_RIT_STUCK_WARNING_THRESHOLD, 60000);
for (RegionState state: regionStates.getRegionsInTransition()) {
totalRITs++;
long ritTime = currentTime - state.getStamp();
if (ritTime > ritThreshold) { // more than the threshold
totalRITsOverThreshold++;
+ if (ritsOverThreshold == null) {
+ ritsOverThreshold = new HashMap<String, RegionState>();
+ }
+ ritsOverThreshold.put(state.getRegion().getEncodedName(), state);
}
if (oldestRITTime < ritTime) {
oldestRITTime = ritTime;
}
}
+ if (LOG.isDebugEnabled() && ritsOverThreshold != null && !ritsOverThreshold.isEmpty()) {
+ StringBuilder sb = new StringBuilder();
+ for (String regionName: ritsOverThreshold.keySet()) {
+ sb.append(regionName + ":" + ritsOverThreshold.get(regionName).getState().name() + "\n");
+ }
+ sb.delete(sb.length()-2, sb.length());
Review comment:
Do we want to remove last appended new line char? If so, start index should be `sb.length()-1` right?
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
##########
@@ -3538,18 +3538,31 @@ public void updateRegionsInTransitionMetrics() {
int totalRITs = 0;
int totalRITsOverThreshold = 0;
long oldestRITTime = 0;
+ HashMap<String, RegionState> ritsOverThreshold = null;
int ritThreshold = this.server.getConfiguration().
getInt(HConstants.METRICS_RIT_STUCK_WARNING_THRESHOLD, 60000);
for (RegionState state: regionStates.getRegionsInTransition()) {
totalRITs++;
long ritTime = currentTime - state.getStamp();
if (ritTime > ritThreshold) { // more than the threshold
totalRITsOverThreshold++;
+ if (ritsOverThreshold == null) {
+ ritsOverThreshold = new HashMap<String, RegionState>();
+ }
+ ritsOverThreshold.put(state.getRegion().getEncodedName(), state);
}
if (oldestRITTime < ritTime) {
oldestRITTime = ritTime;
}
}
+ if (LOG.isDebugEnabled() && ritsOverThreshold != null && !ritsOverThreshold.isEmpty()) {
+ StringBuilder sb = new StringBuilder();
+ for (String regionName: ritsOverThreshold.keySet()) {
+ sb.append(regionName + ":" + ritsOverThreshold.get(regionName).getState().name() + "\n");
Review comment:
nit: use chain of append() to cover String concatenation?
```
sb.append(regionName).append(":")
.append(ritsOverThreshold.get(regionName).getState().name()).append("\n");
```
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
##########
@@ -3538,18 +3538,31 @@ public void updateRegionsInTransitionMetrics() {
int totalRITs = 0;
int totalRITsOverThreshold = 0;
long oldestRITTime = 0;
+ HashMap<String, RegionState> ritsOverThreshold = null;
int ritThreshold = this.server.getConfiguration().
getInt(HConstants.METRICS_RIT_STUCK_WARNING_THRESHOLD, 60000);
for (RegionState state: regionStates.getRegionsInTransition()) {
totalRITs++;
long ritTime = currentTime - state.getStamp();
if (ritTime > ritThreshold) { // more than the threshold
totalRITsOverThreshold++;
+ if (ritsOverThreshold == null) {
+ ritsOverThreshold = new HashMap<String, RegionState>();
Review comment:
nit: `new HashMap<>()`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #2762: HBASE-25329 Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
Posted by GitBox <gi...@apache.org>.
virajjasani commented on a change in pull request #2762:
URL: https://github.com/apache/hbase/pull/2762#discussion_r558358500
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
##########
@@ -3538,18 +3538,32 @@ public void updateRegionsInTransitionMetrics() {
int totalRITs = 0;
int totalRITsOverThreshold = 0;
long oldestRITTime = 0;
+ Map<String, RegionState> ritsOverThreshold = null;
int ritThreshold = this.server.getConfiguration().
getInt(HConstants.METRICS_RIT_STUCK_WARNING_THRESHOLD, 60000);
for (RegionState state: regionStates.getRegionsInTransition()) {
totalRITs++;
long ritTime = currentTime - state.getStamp();
if (ritTime > ritThreshold) { // more than the threshold
totalRITsOverThreshold++;
+ if (ritsOverThreshold == null) {
+ ritsOverThreshold = new HashMap<>();
+ }
+ ritsOverThreshold.put(state.getRegion().getEncodedName(), state);
}
if (oldestRITTime < ritTime) {
oldestRITTime = ritTime;
}
}
+ if (LOG.isDebugEnabled() && ritsOverThreshold != null && !ritsOverThreshold.isEmpty()) {
+ StringBuilder sb = new StringBuilder();
+ for (String regionName: ritsOverThreshold.keySet()) {
Review comment:
If you could convert this to entrySet(), that would be great (as per findbugs)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] caroliney14 commented on a change in pull request #2762: HBASE-25329 Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
Posted by GitBox <gi...@apache.org>.
caroliney14 commented on a change in pull request #2762:
URL: https://github.com/apache/hbase/pull/2762#discussion_r543844166
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
##########
@@ -3549,11 +3554,24 @@ public void updateRegionsInTransitionMetrics() {
if (oldestRITTime < ritTime) {
oldestRITTime = ritTime;
}
+ if (counter < 500) { // Record 500 oldest RITs
+ oldestRITHashesAndStates.add(
+ state.getRegion().getRegionNameAsString() + ":" + state.getState().name()
Review comment:
if the md5 encoded name is enough info for debugging (which I think yes), then it should suffice to use that 👍
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2762: HBASE-25329 Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2762:
URL: https://github.com/apache/hbase/pull/2762#issuecomment-742809234
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 6m 42s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ branch-1 Compile Tests _ |
| +0 :ok: | mvndep | 2m 23s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 8m 7s | branch-1 passed |
| +1 :green_heart: | compile | 1m 12s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | compile | 1m 17s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +1 :green_heart: | checkstyle | 2m 10s | branch-1 passed |
| +1 :green_heart: | shadedjars | 3m 0s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 9s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javadoc | 1m 13s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +0 :ok: | spotbugs | 2m 58s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 4m 21s | branch-1 passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 4s | the patch passed |
| +1 :green_heart: | compile | 1m 12s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javac | 1m 12s | the patch passed |
| +1 :green_heart: | compile | 1m 26s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +1 :green_heart: | javac | 1m 26s | the patch passed |
| -1 :x: | checkstyle | 1m 45s | hbase-server: The patch generated 2 new + 251 unchanged - 0 fixed = 253 total (was 251) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedjars | 2m 51s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | hadoopcheck | 4m 46s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. |
| +1 :green_heart: | javadoc | 0m 59s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javadoc | 1m 16s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +1 :green_heart: | findbugs | 4m 25s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 29s | hbase-hadoop-compat in the patch passed. |
| +1 :green_heart: | unit | 0m 40s | hbase-hadoop2-compat in the patch passed. |
| -1 :x: | unit | 121m 34s | hbase-server in the patch failed. |
| +1 :green_heart: | asflicense | 1m 14s | The patch does not generate ASF License warnings. |
| | | 181m 27s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hbase.security.visibility.TestVisibilityLabelsWithACL |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2762 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux ac24713412f8 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2762/out/precommit/personality/provided.sh |
| git revision | branch-1 / 8e45165 |
| Default Java | Azul Systems, Inc.-1.7.0_272-b10 |
| Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10 |
| checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/1/artifact/out/diff-checkstyle-hbase-server.txt |
| unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/1/artifact/out/patch-unit-hbase-server.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/1/testReport/ |
| Max. process+thread count | 4200 (vs. ulimit of 10000) |
| modules | C: hbase-hadoop-compat hbase-hadoop2-compat hbase-server U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/1/console |
| versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2762: HBASE-25329 Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2762:
URL: https://github.com/apache/hbase/pull/2762#issuecomment-763551454
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 6m 59s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ branch-1 Compile Tests _ |
| +0 :ok: | mvndep | 2m 23s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 7m 55s | branch-1 passed |
| +1 :green_heart: | compile | 0m 55s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | compile | 0m 59s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +1 :green_heart: | checkstyle | 1m 57s | branch-1 passed |
| +1 :green_heart: | shadedjars | 3m 2s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 55s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javadoc | 0m 58s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +0 :ok: | spotbugs | 2m 41s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 26s | branch-1 passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 59s | the patch passed |
| +1 :green_heart: | compile | 0m 55s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javac | 0m 55s | the patch passed |
| +1 :green_heart: | compile | 1m 0s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +1 :green_heart: | javac | 1m 0s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 52s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedjars | 2m 51s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | hadoopcheck | 4m 42s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. |
| +1 :green_heart: | javadoc | 0m 44s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javadoc | 0m 59s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +1 :green_heart: | findbugs | 3m 35s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 29s | hbase-hadoop-compat in the patch passed. |
| +1 :green_heart: | unit | 105m 44s | hbase-server in the patch passed. |
| +1 :green_heart: | asflicense | 0m 47s | The patch does not generate ASF License warnings. |
| | | 159m 15s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/10/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2762 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux d6684fcee9fa 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2762/out/precommit/personality/provided.sh |
| git revision | branch-1 / a6b0d5b |
| Default Java | Azul Systems, Inc.-1.7.0_272-b10 |
| Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/10/testReport/ |
| Max. process+thread count | 4249 (vs. ulimit of 10000) |
| modules | C: hbase-hadoop-compat hbase-server U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2762/10/console |
| versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org