You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2021/05/05 02:25:37 UTC

[GitHub] [hbase] apurtell opened a new pull request #3230: HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore

apurtell opened a new pull request #3230:
URL: https://github.com/apache/hbase/pull/3230


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #3230: HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3230:
URL: https://github.com/apache/hbase/pull/3230#issuecomment-832448402


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   1m 21s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  2s |  Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   4m 32s |  master passed  |
   | +1 :green_heart: |  compile  |   1m  3s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   9m 51s |  branch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 54s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m 15s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 18s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 18s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   9m  9s |  patch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 38s |  the patch passed  |
   ||| _ Other Tests _ |
   | -1 :x: |  unit  | 212m 53s |  hbase-server in the patch failed.  |
   |  |   | 248m 53s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/3230 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux a2f612fd551f 4.15.0-126-generic #129-Ubuntu SMP Mon Nov 23 18:53:38 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 17193dae58 |
   | Default Java | AdoptOpenJDK-1.8.0_282-b08 |
   | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
   |  Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/1/testReport/ |
   | Max. process+thread count | 3188 (vs. ulimit of 30000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/1/console |
   | versions | git=2.17.1 maven=3.6.3 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hbase] apurtell edited a comment on pull request #3230: HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore

Posted by GitBox <gi...@apache.org>.
apurtell edited a comment on pull request #3230:
URL: https://github.com/apache/hbase/pull/3230#issuecomment-832853177


   Just the logging changes now, and the bug fix to ensure log lines emit correct information. 
   
   The CatalogJanitor cleanup proposal is now this PR: https://github.com/apache/hbase/pull/3234
   I have submitted it without a test change so we get the same test results there and can carry the discussion forward. Will work on a test change too. 
   
   @virajjasani The next test report will be clean. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hbase] Apache9 commented on pull request #3230: HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore

Posted by GitBox <gi...@apache.org>.
Apache9 commented on pull request #3230:
URL: https://github.com/apache/hbase/pull/3230#issuecomment-832378369


   Some background
   
   https://issues.apache.org/jira/browse/HBASE-24942
   
   I've found a bug when refactoring CatalogJanitor because of we may clean the merge qualifiers in MergeTableRegionsProcedure. Theoretically we should only do this in GCRegion related procedures(as what you proposed here), but doing this in other places could speed up later process which could be blocked by merge qualifiers.
   
   For me I'm +1 on removing the redundant removal in CatalogJanitor, but let's wait for @saintstack 's opinon too?
   
   Thanks.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hbase] apurtell commented on a change in pull request #3230: HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore

Posted by GitBox <gi...@apache.org>.
apurtell commented on a change in pull request #3230:
URL: https://github.com/apache/hbase/pull/3230#discussion_r626988853



##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/janitor/CatalogJanitor.java
##########
@@ -165,13 +168,19 @@ public int scan() throws IOException {
       this.lastReport = scanForReport();
       if (!this.lastReport.isEmpty()) {
         LOG.warn(this.lastReport.toString());
+      } else {
+        LOG.debug(this.lastReport.toString());

Review comment:
       We used to abide by that guidance but I don't see that done often in new code. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hbase] apurtell edited a comment on pull request #3230: HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore

Posted by GitBox <gi...@apache.org>.
apurtell edited a comment on pull request #3230:
URL: https://github.com/apache/hbase/pull/3230#issuecomment-832841115


   With these changes the ingestion test completes successfully and without weird artifacts in the logging. All split regions are GCed by procedures. In memory state aligns with filesystem state. 
   
       /hbase/logs/hbase-apurtell-master-ip-172-31-58-47.us-west-2.compute.internal.log:130972:2021-05-05 15:47:39,190 INFO  [master/ip-172-31-58-47:8100.Chore.1] 
       master.HbckChore: Loaded 230 regions (0 disabled, 0 split parents) from in-memory state
       /hbase/logs/hbase-apurtell-master-ip-172-31-58-47.us-west-2.compute.internal.log:130973:2021-05-05 15:47:39,190 DEBUG [master/ip-172-31-58-47:8100.Chore.1] 
       master.HbckChore: Regions by state: OPEN=230
       /hbase/logs/hbase-apurtell-master-ip-172-31-58-47.us-west-2.compute.internal.log:130974:2021-05-05 15:47:39,190 INFO  [master/ip-172-31-58-47:8100.Chore.1] 
       master.HbckChore: Loaded 230 regions from 5 regionservers' reports and found 0 orphan regions
       /hbase/logs/hbase-apurtell-master-ip-172-31-58-47.us-west-2.compute.internal.log:130986:2021-05-05 15:47:39,200 INFO  [master/ip-172-31-58-47:8100.Chore.1] 
       master.HbckChore: Loaded 3 tables 230 regions from filesyetem and found 0 orphan regions
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hbase] apurtell commented on pull request #3230: HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore

Posted by GitBox <gi...@apache.org>.
apurtell commented on pull request #3230:
URL: https://github.com/apache/hbase/pull/3230#issuecomment-832843132


   Well since what I want here is "More DEBUG and TRACE level logging" I will remove the additional changes so the test failure is fixed and address that on a separate PR. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hbase] apurtell commented on pull request #3230: HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore

Posted by GitBox <gi...@apache.org>.
apurtell commented on pull request #3230:
URL: https://github.com/apache/hbase/pull/3230#issuecomment-832853177


   Just the logging changes now, and the bug fix to ensure log lines emit correct information. 
   
   The CatalogJanitor cleanup proposal is now this PR: https://github.com/apache/hbase/pull/3234
   I have submitted it without a test change so we get the same test results there and can carry the discussion forward. Will work on a test change too. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hbase] saintstack commented on a change in pull request #3230: HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore

Posted by GitBox <gi...@apache.org>.
saintstack commented on a change in pull request #3230:
URL: https://github.com/apache/hbase/pull/3230#discussion_r626778648



##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/HbckChore.java
##########
@@ -223,15 +223,37 @@ private void loadRegionsFromInMemoryState() {
           .isTableState(regionInfo.getTable(), TableState.State.DISABLED)) {
         disabledTableRegions.add(regionInfo.getRegionNameAsString());
       }
-      if (regionInfo.isSplitParent()) {
+      if (regionState.isSplit()) {

Review comment:
       +1 on this change.
   
   Was always odd have a Regions' split state recorded in its RegionInfo; RegionInfo does not keep internally if it is being merged or if open, etc. Regions are manipulated by other entities so state belongs elsewhere. +1 on move to regionstate checking instead.

##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/janitor/CatalogJanitor.java
##########
@@ -165,13 +168,19 @@ public int scan() throws IOException {
       this.lastReport = scanForReport();
       if (!this.lastReport.isEmpty()) {
         LOG.warn(this.lastReport.toString());
+      } else {
+        LOG.debug(this.lastReport.toString());

Review comment:
       +1

##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/janitor/CatalogJanitor.java
##########
@@ -165,13 +168,19 @@ public int scan() throws IOException {
       this.lastReport = scanForReport();
       if (!this.lastReport.isEmpty()) {
         LOG.warn(this.lastReport.toString());
+      } else {
+        LOG.debug(this.lastReport.toString());
       }
+
       updateAssignmentManagerMetrics();
 
       Map<RegionInfo, Result> mergedRegions = this.lastReport.mergedRegions;
       for (Map.Entry<RegionInfo, Result> e : mergedRegions.entrySet()) {
         if (this.services.isInMaintenanceMode()) {
           // Stop cleaning if the master is in maintenance mode
+          if (LOG.isDebugEnabled()) {
+            LOG.debug("In maintenence mode, not cleaning");
+          }

Review comment:
       We need the if (LOG.isDebug.... ?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hbase] virajjasani commented on pull request #3230: HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore

Posted by GitBox <gi...@apache.org>.
virajjasani commented on pull request #3230:
URL: https://github.com/apache/hbase/pull/3230#issuecomment-832472753


   `TestCatalogJanitorInMemoryStates` failure might be relevant? Just retriggered the build, will get back to this after 2nd run completes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hbase] virajjasani commented on pull request #3230: HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore

Posted by GitBox <gi...@apache.org>.
virajjasani commented on pull request #3230:
URL: https://github.com/apache/hbase/pull/3230#issuecomment-832622591


   The error is persistent and it is relevant
   ```
   java.lang.AssertionError: Parent region should have been removed from RegionStates
   	at org.apache.hadoop.hbase.master.janitor.TestCatalogJanitorInMemoryStates.testInMemoryParentCleanup(TestCatalogJanitorInMemoryStates.java:123)
   
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #3230: HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3230:
URL: https://github.com/apache/hbase/pull/3230#issuecomment-832956987


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   0m 29s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   4m 31s |  master passed  |
   | +1 :green_heart: |  compile  |   1m 12s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   8m 14s |  branch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 42s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   4m 17s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 12s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 12s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   8m  6s |  patch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 38s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  | 140m  8s |  hbase-server in the patch passed.  |
   |  |   | 171m 48s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/3230 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 30a6e24d4786 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / eb9b54304e |
   | Default Java | AdoptOpenJDK-11.0.10+9 |
   |  Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/3/testReport/ |
   | Max. process+thread count | 4161 (vs. ulimit of 30000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/3/console |
   | versions | git=2.17.1 maven=3.6.3 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hbase] bharathv commented on a change in pull request #3230: HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore

Posted by GitBox <gi...@apache.org>.
bharathv commented on a change in pull request #3230:
URL: https://github.com/apache/hbase/pull/3230#discussion_r626763927



##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/janitor/CatalogJanitor.java
##########
@@ -165,13 +168,19 @@ public int scan() throws IOException {
       this.lastReport = scanForReport();
       if (!this.lastReport.isEmpty()) {
         LOG.warn(this.lastReport.toString());
+      } else {
+        LOG.debug(this.lastReport.toString());

Review comment:
       nit: toString() might be an expensive call if debug is not enabled, might want to switch to a isDebugEnabled() guard, (later versions of log4j supports log.debug(Object)/log.debug(Supplier) so that toString() is eventually evaluated after the debug guard check)..




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hbase] apurtell commented on a change in pull request #3230: HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore

Posted by GitBox <gi...@apache.org>.
apurtell commented on a change in pull request #3230:
URL: https://github.com/apache/hbase/pull/3230#discussion_r626219664



##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/janitor/CatalogJanitor.java
##########
@@ -317,11 +329,19 @@ static boolean cleanParent(MasterServices services, RegionInfo parent, Result ro
       LOG.debug("Deleting region " + parent.getShortNameToLog() + " because daughters -- " +
         daughterA + ", " + daughterB + " -- no longer hold references");
       ProcedureExecutor<MasterProcedureEnv> pe = services.getMasterProcedureExecutor();
-      pe.submitProcedure(new GCRegionProcedure(pe.getEnvironment(), parent));
-      // Remove from in-memory states

Review comment:
       GCRegionProcedure does this. It is not needed here too. I think it better to let the procedure do it. We expect the procedure to take care of such details, so let's in fact do that.
   Keeping this code around would be harmless, though. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hbase] apurtell merged pull request #3230: HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore

Posted by GitBox <gi...@apache.org>.
apurtell merged pull request #3230:
URL: https://github.com/apache/hbase/pull/3230


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #3230: HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3230:
URL: https://github.com/apache/hbase/pull/3230#issuecomment-833003042


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   1m 11s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   4m  0s |  master passed  |
   | +1 :green_heart: |  compile  |   1m  3s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   8m 58s |  branch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 38s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   4m  3s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m  2s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m  2s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   8m 56s |  patch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 36s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  | 218m 59s |  hbase-server in the patch passed.  |
   |  |   | 251m 20s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/3230 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 8ba4f254b55c 4.15.0-126-generic #129-Ubuntu SMP Mon Nov 23 18:53:38 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / eb9b54304e |
   | Default Java | AdoptOpenJDK-1.8.0_282-b08 |
   |  Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/3/testReport/ |
   | Max. process+thread count | 3157 (vs. ulimit of 30000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/3/console |
   | versions | git=2.17.1 maven=3.6.3 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hbase] apurtell commented on pull request #3230: HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore

Posted by GitBox <gi...@apache.org>.
apurtell commented on pull request #3230:
URL: https://github.com/apache/hbase/pull/3230#issuecomment-832841115


   With these changes the ingestion test completes successfully and without weird artifacts in the logging. All split regions are GCed by procedures. In memory state aligns with filesystem state. 
   
       /hbase/logs/hbase-apurtell-master-ip-172-31-58-47.us-west-2.compute.internal.log:130972:2021-05-05 15:47:39,190 INFO  [master/ip-172-31-58-47:8100.Chore.1] master.HbckChore: Loaded 230 regions (0 disabled, 0 split parents) from in-memory state
       /hbase/logs/hbase-apurtell-master-ip-172-31-58-47.us-west-2.compute.internal.log:130973:2021-05-05 15:47:39,190 DEBUG [master/ip-172-31-58-47:8100.Chore.1] master.HbckChore: Regions by state: OPEN=230


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #3230: HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3230:
URL: https://github.com/apache/hbase/pull/3230#issuecomment-833160561


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   1m  9s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  2s |  Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   4m  6s |  master passed  |
   | +1 :green_heart: |  compile  |   1m  4s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   9m  6s |  branch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 37s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   4m  3s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m  2s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m  2s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   8m 58s |  patch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 37s |  the patch passed  |
   ||| _ Other Tests _ |
   | -1 :x: |  unit  | 239m 41s |  hbase-server in the patch failed.  |
   |  |   | 273m 29s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/3230 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux df228ca31218 4.15.0-126-generic #129-Ubuntu SMP Mon Nov 23 18:53:38 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / eb9b54304e |
   | Default Java | AdoptOpenJDK-1.8.0_282-b08 |
   | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/4/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
   |  Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/4/testReport/ |
   | Max. process+thread count | 3399 (vs. ulimit of 30000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/4/console |
   | versions | git=2.17.1 maven=3.6.3 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #3230: HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3230:
URL: https://github.com/apache/hbase/pull/3230#issuecomment-832385925


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   1m  8s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any @author tags.  |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   4m 38s |  master passed  |
   | +1 :green_heart: |  compile  |   3m 13s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   1m  3s |  master passed  |
   | +1 :green_heart: |  spotbugs  |   2m  7s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   3m 35s |  the patch passed  |
   | +1 :green_heart: |  compile  |   3m 12s |  the patch passed  |
   | +1 :green_heart: |  javac  |   3m 12s |  the patch passed  |
   | -0 :warning: |  checkstyle  |   1m  2s |  hbase-server: The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1)  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace issues.  |
   | +1 :green_heart: |  hadoopcheck  |  18m 18s |  Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.  |
   | +1 :green_heart: |  spotbugs  |   2m 16s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 16s |  The patch does not generate ASF License warnings.  |
   |  |   |  48m 42s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/1/artifact/yetus-general-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/3230 |
   | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile |
   | uname | Linux 480c24ef7e98 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 17193dae58 |
   | Default Java | AdoptOpenJDK-1.8.0_282-b08 |
   | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
   | Max. process+thread count | 96 (vs. ulimit of 30000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/1/console |
   | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hbase] apurtell commented on a change in pull request #3230: HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore

Posted by GitBox <gi...@apache.org>.
apurtell commented on a change in pull request #3230:
URL: https://github.com/apache/hbase/pull/3230#discussion_r626989255



##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/janitor/CatalogJanitor.java
##########
@@ -165,13 +168,19 @@ public int scan() throws IOException {
       this.lastReport = scanForReport();
       if (!this.lastReport.isEmpty()) {
         LOG.warn(this.lastReport.toString());
+      } else {
+        LOG.debug(this.lastReport.toString());
       }
+
       updateAssignmentManagerMetrics();
 
       Map<RegionInfo, Result> mergedRegions = this.lastReport.mergedRegions;
       for (Map.Entry<RegionInfo, Result> e : mergedRegions.entrySet()) {
         if (this.services.isInMaintenanceMode()) {
           // Stop cleaning if the master is in maintenance mode
+          if (LOG.isDebugEnabled()) {
+            LOG.debug("In maintenence mode, not cleaning");
+          }

Review comment:
       Ok both you and @bharathv mentioned this, will do. 
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #3230: HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3230:
URL: https://github.com/apache/hbase/pull/3230#issuecomment-833042756


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   1m 11s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any @author tags.  |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   3m 39s |  master passed  |
   | +1 :green_heart: |  compile  |   3m 12s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   1m  2s |  master passed  |
   | +1 :green_heart: |  spotbugs  |   2m  0s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   3m 36s |  the patch passed  |
   | +1 :green_heart: |  compile  |   3m 11s |  the patch passed  |
   | +1 :green_heart: |  javac  |   3m 11s |  the patch passed  |
   | -0 :warning: |  checkstyle  |   1m  1s |  hbase-server: The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1)  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace issues.  |
   | +1 :green_heart: |  hadoopcheck  |  18m  6s |  Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.  |
   | +1 :green_heart: |  spotbugs  |   2m 15s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 13s |  The patch does not generate ASF License warnings.  |
   |  |   |  47m  9s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/4/artifact/yetus-general-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/3230 |
   | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile |
   | uname | Linux 2b2ba1cc9ec6 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / eb9b54304e |
   | Default Java | AdoptOpenJDK-1.8.0_282-b08 |
   | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/4/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
   | Max. process+thread count | 96 (vs. ulimit of 30000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/4/console |
   | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hbase] apurtell commented on a change in pull request #3230: HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore

Posted by GitBox <gi...@apache.org>.
apurtell commented on a change in pull request #3230:
URL: https://github.com/apache/hbase/pull/3230#discussion_r626219018



##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/HbckChore.java
##########
@@ -223,15 +223,37 @@ private void loadRegionsFromInMemoryState() {
           .isTableState(regionInfo.getTable(), TableState.State.DISABLED)) {
         disabledTableRegions.add(regionInfo.getRegionNameAsString());
       }
-      if (regionInfo.isSplitParent()) {
+      if (regionState.isSplit()) {

Review comment:
       Most of this patch is just more logging. This is a real bug fix, though. Right now it just affects logging (numbers printed in a debug log line) but if in the future code depends on `splitParentRegions` to be properly populated, this will be needed. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #3230: HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3230:
URL: https://github.com/apache/hbase/pull/3230#issuecomment-832422235


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   0m 31s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   8m 27s |  master passed  |
   | +1 :green_heart: |  compile  |   1m 17s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   8m 51s |  branch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 46s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   4m 37s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 17s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 17s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   8m 57s |  patch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 46s |  the patch passed  |
   ||| _ Other Tests _ |
   | -1 :x: |  unit  | 146m 42s |  hbase-server in the patch failed.  |
   |  |   | 184m 18s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/3230 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux f2b9ed735d44 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 17193dae58 |
   | Default Java | AdoptOpenJDK-11.0.10+9 |
   | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
   |  Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/1/testReport/ |
   | Max. process+thread count | 3881 (vs. ulimit of 30000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/1/console |
   | versions | git=2.17.1 maven=3.6.3 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hbase] apurtell commented on a change in pull request #3230: HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore

Posted by GitBox <gi...@apache.org>.
apurtell commented on a change in pull request #3230:
URL: https://github.com/apache/hbase/pull/3230#discussion_r626219557



##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/janitor/CatalogJanitor.java
##########
@@ -252,14 +265,11 @@ private boolean cleanMergeRegion(final RegionInfo mergedRegion, List<RegionInfo>
           .stream().map(r -> RegionInfo.getShortNameToLog(r)).collect(Collectors.joining(", ")),
         mergedRegion);
       ProcedureExecutor<MasterProcedureEnv> pe = this.services.getMasterProcedureExecutor();
-      pe.submitProcedure(
-        new GCMultipleMergedRegionsProcedure(pe.getEnvironment(), mergedRegion, parents));
-      for (RegionInfo ri : parents) {
-        // The above scheduled GCMultipleMergedRegionsProcedure does the below.

Review comment:
       GCMultipleMergedRegionsProcedure does this. It is not needed here too. I think it better to let the procedure do it. We expect the procedure to take care of such details, so let's in fact do that.
   Keeping this code around would be harmless, though. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hbase] apurtell commented on pull request #3230: HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore

Posted by GitBox <gi...@apache.org>.
apurtell commented on pull request #3230:
URL: https://github.com/apache/hbase/pull/3230#issuecomment-833124523


   Added log level guards as requested before merge. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #3230: HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3230:
URL: https://github.com/apache/hbase/pull/3230#issuecomment-832498562


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   1m  6s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any @author tags.  |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   3m 58s |  master passed  |
   | +1 :green_heart: |  compile  |   3m 15s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   1m  2s |  master passed  |
   | +1 :green_heart: |  spotbugs  |   2m  5s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   3m 36s |  the patch passed  |
   | +1 :green_heart: |  compile  |   3m 12s |  the patch passed  |
   | +1 :green_heart: |  javac  |   3m 12s |  the patch passed  |
   | -0 :warning: |  checkstyle  |   1m  2s |  hbase-server: The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1)  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace issues.  |
   | +1 :green_heart: |  hadoopcheck  |  18m  1s |  Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.  |
   | +1 :green_heart: |  spotbugs  |   2m 14s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 16s |  The patch does not generate ASF License warnings.  |
   |  |   |  47m 36s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/2/artifact/yetus-general-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/3230 |
   | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile |
   | uname | Linux f5fae22b4931 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 17193dae58 |
   | Default Java | AdoptOpenJDK-1.8.0_282-b08 |
   | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
   | Max. process+thread count | 95 (vs. ulimit of 30000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/2/console |
   | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #3230: HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3230:
URL: https://github.com/apache/hbase/pull/3230#issuecomment-833153234


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   1m  1s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m  4s |  master passed  |
   | +1 :green_heart: |  compile  |   1m 18s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   9m  3s |  branch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 44s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   4m 48s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 17s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 17s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   8m 58s |  patch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 42s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  | 214m 56s |  hbase-server in the patch passed.  |
   |  |   | 250m 27s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/3230 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux ed52943ad64c 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / eb9b54304e |
   | Default Java | AdoptOpenJDK-11.0.10+9 |
   |  Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/4/testReport/ |
   | Max. process+thread count | 3203 (vs. ulimit of 30000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/4/console |
   | versions | git=2.17.1 maven=3.6.3 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hbase] apurtell edited a comment on pull request #3230: HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore

Posted by GitBox <gi...@apache.org>.
apurtell edited a comment on pull request #3230:
URL: https://github.com/apache/hbase/pull/3230#issuecomment-832853177


   Just the logging changes now, and the bug fix to ensure log lines emit correct information. 
   
   The CatalogJanitor cleanup proposal is now this PR: https://github.com/apache/hbase/pull/3234
   I have submitted it without a test change so we get the same test results there and can carry the discussion forward. Will work on a test change too. 
   
   @virajjasani The next test report here should be good.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #3230: HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3230:
URL: https://github.com/apache/hbase/pull/3230#issuecomment-832612958


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   1m  7s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   4m 18s |  master passed  |
   | +1 :green_heart: |  compile  |   1m  2s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   8m 54s |  branch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 38s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   4m  0s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m  2s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m  2s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   8m 55s |  patch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 38s |  the patch passed  |
   ||| _ Other Tests _ |
   | -1 :x: |  unit  | 212m  5s |  hbase-server in the patch failed.  |
   |  |   | 244m 34s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/3230 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux f83ce92d60e4 4.15.0-126-generic #129-Ubuntu SMP Mon Nov 23 18:53:38 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 17193dae58 |
   | Default Java | AdoptOpenJDK-1.8.0_282-b08 |
   | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
   |  Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/2/testReport/ |
   | Max. process+thread count | 3243 (vs. ulimit of 30000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/2/console |
   | versions | git=2.17.1 maven=3.6.3 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hbase] apurtell edited a comment on pull request #3230: HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore

Posted by GitBox <gi...@apache.org>.
apurtell edited a comment on pull request #3230:
URL: https://github.com/apache/hbase/pull/3230#issuecomment-832841115


   With these changes the ingestion test completes successfully and without weird artifacts in the logging. All split regions are GCed by procedures. In memory state aligns with filesystem state. 
   
       /hbase/logs/hbase-apurtell-master-ip-172-31-58-47.us-west-2.compute.internal.log:130972:2021-05-05 15:47:39,190 INFO  [master/ip-172-31-58-47:8100.Chore.1] 
       master.HbckChore: Loaded 230 regions (0 disabled, 0 split parents) from in-memory state
       /hbase/logs/hbase-apurtell-master-ip-172-31-58-47.us-west-2.compute.internal.log:130973:2021-05-05 15:47:39,190 DEBUG [master/ip-172-31-58-47:8100.Chore.1] 
       master.HbckChore: Regions by state: OPEN=230


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #3230: HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3230:
URL: https://github.com/apache/hbase/pull/3230#issuecomment-832878005


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   1m  5s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any @author tags.  |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   3m 34s |  master passed  |
   | +1 :green_heart: |  compile  |   3m 11s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   1m  3s |  master passed  |
   | +1 :green_heart: |  spotbugs  |   2m  2s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   3m 38s |  the patch passed  |
   | +1 :green_heart: |  compile  |   3m  8s |  the patch passed  |
   | +1 :green_heart: |  javac  |   3m  8s |  the patch passed  |
   | -0 :warning: |  checkstyle  |   1m  4s |  hbase-server: The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1)  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace issues.  |
   | +1 :green_heart: |  hadoopcheck  |  17m 51s |  Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.  |
   | +1 :green_heart: |  spotbugs  |   2m 11s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 13s |  The patch does not generate ASF License warnings.  |
   |  |   |  46m 45s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/3/artifact/yetus-general-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/3230 |
   | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile |
   | uname | Linux d37628d83ab8 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / eb9b54304e |
   | Default Java | AdoptOpenJDK-1.8.0_282-b08 |
   | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
   | Max. process+thread count | 95 (vs. ulimit of 30000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/3/console |
   | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #3230: HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3230:
URL: https://github.com/apache/hbase/pull/3230#issuecomment-832584044


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   0m 28s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   4m 36s |  master passed  |
   | +1 :green_heart: |  compile  |   1m 13s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   8m 16s |  branch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 44s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   4m 20s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 14s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 14s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   8m 49s |  patch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 44s |  the patch passed  |
   ||| _ Other Tests _ |
   | -1 :x: |  unit  | 156m 27s |  hbase-server in the patch failed.  |
   |  |   | 188m 53s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/3230 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 486137bc9384 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 17193dae58 |
   | Default Java | AdoptOpenJDK-11.0.10+9 |
   | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
   |  Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/2/testReport/ |
   | Max. process+thread count | 3930 (vs. ulimit of 30000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3230/2/console |
   | versions | git=2.17.1 maven=3.6.3 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org