You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2020/08/11 03:21:38 UTC

[GitHub] [hbase] WenFeiYi opened a new pull request #2194: HBASE-24849 Branch-1 Backport : HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll

WenFeiYi opened a new pull request #2194:
URL: https://github.com/apache/hbase/pull/2194


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Apache-HBase commented on pull request #2194: HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2194:
URL: https://github.com/apache/hbase/pull/2194#issuecomment-670447154


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   1m 18s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  The patch appears to include 1 new or modified test files.  |
   ||| _ branch-1 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   9m 57s |  branch-1 passed  |
   | +1 :green_heart: |  compile  |   0m 38s |  branch-1 passed with JDK v1.8.0_262  |
   | +1 :green_heart: |  compile  |   0m 43s |  branch-1 passed with JDK v1.7.0_272  |
   | +1 :green_heart: |  checkstyle  |   1m 38s |  branch-1 passed  |
   | +1 :green_heart: |  shadedjars  |   3m  9s |  branch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 50s |  branch-1 passed with JDK v1.8.0_262  |
   | +1 :green_heart: |  javadoc  |   0m 42s |  branch-1 passed with JDK v1.7.0_272  |
   | +0 :ok: |  spotbugs  |   3m 12s |  Used deprecated FindBugs config; considering switching to SpotBugs.  |
   | +1 :green_heart: |  findbugs  |   3m  8s |  branch-1 passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   1m 54s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 40s |  the patch passed with JDK v1.8.0_262  |
   | +1 :green_heart: |  javac  |   0m 40s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 42s |  the patch passed with JDK v1.7.0_272  |
   | +1 :green_heart: |  javac  |   0m 42s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   1m 27s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace issues.  |
   | +1 :green_heart: |  shadedjars  |   2m 47s |  patch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  hadoopcheck  |   4m 34s |  Patch does not cause any errors with Hadoop 2.8.5 2.9.2.  |
   | +1 :green_heart: |  javadoc  |   0m 32s |  the patch passed with JDK v1.8.0_262  |
   | +1 :green_heart: |  javadoc  |   0m 45s |  the patch passed with JDK v1.7.0_272  |
   | -1 :x: |  findbugs  |   3m  9s |  hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)  |
   ||| _ Other Tests _ |
   | -1 :x: |  unit  | 166m  3s |  hbase-server in the patch failed.  |
   | +1 :green_heart: |  asflicense  |   0m 35s |  The patch does not generate ASF License warnings.  |
   |  |   | 208m 47s |   |
   
   
   | Reason | Tests |
   |-------:|:------|
   | FindBugs | module:hbase-server |
   |  |  Return value of putIfAbsent is ignored, but controller is reused in org.apache.hadoop.hbase.regionserver.LogRoller$1.logRollRequested(WALActionsListener$RollRequestReason)  At LogRoller.java:ignored, but controller is reused in org.apache.hadoop.hbase.regionserver.LogRoller$1.logRollRequested(WALActionsListener$RollRequestReason)  At LogRoller.java:[line 79] |
   | Failed junit tests | hadoop.hbase.master.normalizer.TestSimpleRegionNormalizerOnCluster |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/3/artifact/out/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/2194 |
   | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
   | uname | Linux 39eea9ecdefc 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2194/out/precommit/personality/provided.sh |
   | git revision | branch-1 / 839ce8e |
   | Default Java | 1.7.0_272 |
   | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_262 /usr/lib/jvm/zulu-7-amd64:1.7.0_272 |
   | findbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/3/artifact/out/new-findbugs-hbase-server.html |
   | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/3/artifact/out/patch-unit-hbase-server.txt |
   |  Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/3/testReport/ |
   | Max. process+thread count | 3815 (vs. ulimit of 10000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/3/console |
   | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] WenFeiYi commented on a change in pull request #2194: HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll

Posted by GitBox <gi...@apache.org>.
WenFeiYi commented on a change in pull request #2194:
URL: https://github.com/apache/hbase/pull/2194#discussion_r468301161



##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/LogRoller.java
##########
@@ -229,11 +244,52 @@ private void scheduleFlush(final byte [] encodedRegionName) {
    */
   @VisibleForTesting
   public boolean walRollFinished() {
-    for (boolean needRoll : walNeedsRoll.values()) {
-      if (needRoll) {
+    long now = System.currentTimeMillis();
+    for (RollController controller : wals.values()) {
+      if (controller.needsRoll(now)) {
         return false;
       }
     }
     return true;
   }
+
+
+  /**
+   * Independently control the roll of each wal. When use multiwal,
+   * can avoid all wal roll together. see HBASE-24665 for detail
+   */
+  protected class RollController {
+    private final WAL wal;
+    private final AtomicBoolean rollRequest;
+    private long lastRollTime;
+
+    RollController(WAL wal) {
+      this.wal = wal;
+      this.rollRequest = new AtomicBoolean(false);
+      this.lastRollTime = System.currentTimeMillis();
+    }
+
+    public void requestRoll() {
+      this.rollRequest.set(true);
+    }
+
+    public byte[][] rollWal(long now) throws IOException {
+      this.lastRollTime = now;
+      byte[][] regionsToFlush = wal.rollWriter(true);
+      this.rollRequest.set(false);

Review comment:
       In order to be consistent with the previous




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Reidddddd commented on pull request #2194: HBASE-24849 Branch-1 Backport : HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll

Posted by GitBox <gi...@apache.org>.
Reidddddd commented on pull request #2194:
URL: https://github.com/apache/hbase/pull/2194#issuecomment-700505084


   One question, i don't get the solution, why this change can avoid all wals roll?
   the original version `ConcurrentHashMap<WAL, Boolean> walNeedsRoll` is also a per wal data structure?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Apache-HBase commented on pull request #2194: HBASE-24849 Branch-1 Backport : HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2194:
URL: https://github.com/apache/hbase/pull/2194#issuecomment-708910313


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |  17m 49s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  The patch appears to include 1 new or modified test files.  |
   ||| _ branch-1 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  11m 34s |  branch-1 passed  |
   | +1 :green_heart: |  compile  |   1m  8s |  branch-1 passed with JDK v1.8.0_262  |
   | +1 :green_heart: |  compile  |   1m 15s |  branch-1 passed with JDK v1.7.0_272  |
   | +1 :green_heart: |  checkstyle  |   2m 32s |  branch-1 passed  |
   | +1 :green_heart: |  shadedjars  |   4m 41s |  branch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   1m  9s |  branch-1 passed with JDK v1.8.0_262  |
   | +1 :green_heart: |  javadoc  |   1m  3s |  branch-1 passed with JDK v1.7.0_272  |
   | +0 :ok: |  spotbugs  |   4m 42s |  Used deprecated FindBugs config; considering switching to SpotBugs.  |
   | +1 :green_heart: |  findbugs  |   4m 37s |  branch-1 passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   3m  2s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m  2s |  the patch passed with JDK v1.8.0_262  |
   | +1 :green_heart: |  javac  |   1m  2s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m  7s |  the patch passed with JDK v1.7.0_272  |
   | +1 :green_heart: |  javac  |   1m  7s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   2m 19s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace issues.  |
   | +1 :green_heart: |  shadedjars  |   4m 30s |  patch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  hadoopcheck  |   7m 21s |  Patch does not cause any errors with Hadoop 2.8.5 2.9.2.  |
   | +1 :green_heart: |  javadoc  |   0m 51s |  the patch passed with JDK v1.8.0_262  |
   | +1 :green_heart: |  javadoc  |   1m  1s |  the patch passed with JDK v1.7.0_272  |
   | -1 :x: |  findbugs  |   5m  8s |  hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)  |
   ||| _ Other Tests _ |
   | -1 :x: |  unit  | 164m 30s |  hbase-server in the patch failed.  |
   | +1 :green_heart: |  asflicense  |   0m 38s |  The patch does not generate ASF License warnings.  |
   |  |   | 242m 28s |   |
   
   
   | Reason | Tests |
   |-------:|:------|
   | FindBugs | module:hbase-server |
   |  |  Sequence of calls to java.util.concurrent.ConcurrentHashMap may not be atomic in org.apache.hadoop.hbase.regionserver.LogRoller$1.logRollRequested(WALActionsListener$RollRequestReason)  At LogRoller.java:may not be atomic in org.apache.hadoop.hbase.regionserver.LogRoller$1.logRollRequested(WALActionsListener$RollRequestReason)  At LogRoller.java:[line 81] |
   | Failed junit tests | hadoop.hbase.regionserver.TestRegionServerAbort |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/7/artifact/out/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/2194 |
   | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
   | uname | Linux fe0fbfa9ff66 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2194/out/precommit/personality/provided.sh |
   | git revision | branch-1 / e4a8dcc |
   | Default Java | 1.7.0_272 |
   | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_262 /usr/lib/jvm/zulu-7-amd64:1.7.0_272 |
   | findbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/7/artifact/out/new-findbugs-hbase-server.html |
   | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/7/artifact/out/patch-unit-hbase-server.txt |
   |  Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/7/testReport/ |
   | Max. process+thread count | 4294 (vs. ulimit of 10000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/7/console |
   | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Reidddddd commented on pull request #2194: HBASE-24849 Branch-1 Backport : HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll

Posted by GitBox <gi...@apache.org>.
Reidddddd commented on pull request #2194:
URL: https://github.com/apache/hbase/pull/2194#issuecomment-700629560


   Please take a look at the `FindBugs` warning.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Apache-HBase commented on pull request #2194: HBASE-24849 Branch-1 Backport : HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2194:
URL: https://github.com/apache/hbase/pull/2194#issuecomment-709988778


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   1m  6s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  The patch appears to include 1 new or modified test files.  |
   ||| _ branch-1 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  10m  1s |  branch-1 passed  |
   | +1 :green_heart: |  compile  |   0m 46s |  branch-1 passed with JDK v1.8.0_262  |
   | +1 :green_heart: |  compile  |   0m 46s |  branch-1 passed with JDK v1.7.0_272  |
   | +1 :green_heart: |  checkstyle  |   1m 45s |  branch-1 passed  |
   | +1 :green_heart: |  shadedjars  |   3m 12s |  branch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 52s |  branch-1 passed with JDK v1.8.0_262  |
   | +1 :green_heart: |  javadoc  |   0m 44s |  branch-1 passed with JDK v1.7.0_272  |
   | +0 :ok: |  spotbugs  |   3m 11s |  Used deprecated FindBugs config; considering switching to SpotBugs.  |
   | +1 :green_heart: |  findbugs  |   3m  7s |  branch-1 passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   2m  4s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 46s |  the patch passed with JDK v1.8.0_262  |
   | +1 :green_heart: |  javac  |   0m 46s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 47s |  the patch passed with JDK v1.7.0_272  |
   | +1 :green_heart: |  javac  |   0m 47s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   1m 31s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace issues.  |
   | +1 :green_heart: |  shadedjars  |   3m  2s |  patch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  hadoopcheck  |   5m  5s |  Patch does not cause any errors with Hadoop 2.8.5 2.9.2.  |
   | +1 :green_heart: |  javadoc  |   0m 34s |  the patch passed with JDK v1.8.0_262  |
   | +1 :green_heart: |  javadoc  |   0m 44s |  the patch passed with JDK v1.7.0_272  |
   | +1 :green_heart: |  findbugs  |   3m  3s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  | 121m 25s |  hbase-server in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 36s |  The patch does not generate ASF License warnings.  |
   |  |   | 165m 47s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/10/artifact/out/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/2194 |
   | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
   | uname | Linux c94d4d5a346c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2194/out/precommit/personality/provided.sh |
   | git revision | branch-1 / fb25a7d |
   | Default Java | 1.7.0_272 |
   | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_262 /usr/lib/jvm/zulu-7-amd64:1.7.0_272 |
   |  Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/10/testReport/ |
   | Max. process+thread count | 3741 (vs. ulimit of 10000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/10/console |
   | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Reidddddd merged pull request #2194: HBASE-24849 Branch-1 Backport : HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll

Posted by GitBox <gi...@apache.org>.
Reidddddd merged pull request #2194:
URL: https://github.com/apache/hbase/pull/2194


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] WenFeiYi commented on a change in pull request #2194: HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll

Posted by GitBox <gi...@apache.org>.
WenFeiYi commented on a change in pull request #2194:
URL: https://github.com/apache/hbase/pull/2194#discussion_r468301336



##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/LogRoller.java
##########
@@ -141,9 +150,14 @@ public void run() {
     while (!server.isStopped()) {
       long now = System.currentTimeMillis();
       checkLowReplication(now);
-      boolean periodic = false;
       if (!rollLog.get()) {
-        periodic = (now - this.lastrolltime) > this.rollperiod;
+        boolean periodic = false;
+        for (RollController controller : wals.values()) {
+          if (controller.needsPeriodicRoll(now)) {
+            periodic = true;
+            break;
+          }
+        }

Review comment:
       In order to be compatible with jdk1.7




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Reidddddd commented on pull request #2194: HBASE-24849 Branch-1 Backport : HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll

Posted by GitBox <gi...@apache.org>.
Reidddddd commented on pull request #2194:
URL: https://github.com/apache/hbase/pull/2194#issuecomment-708952307


   Let's wait for QA


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Apache-HBase commented on pull request #2194: HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2194:
URL: https://github.com/apache/hbase/pull/2194#issuecomment-671245385


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   6m 40s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  The patch appears to include 1 new or modified test files.  |
   ||| _ branch-1 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   9m 47s |  branch-1 passed  |
   | +1 :green_heart: |  compile  |   0m 40s |  branch-1 passed with JDK v1.8.0_262  |
   | +1 :green_heart: |  compile  |   0m 44s |  branch-1 passed with JDK v1.7.0_272  |
   | +1 :green_heart: |  checkstyle  |   1m 40s |  branch-1 passed  |
   | +1 :green_heart: |  shadedjars  |   3m  0s |  branch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 49s |  branch-1 passed with JDK v1.8.0_262  |
   | +1 :green_heart: |  javadoc  |   0m 41s |  branch-1 passed with JDK v1.7.0_272  |
   | +0 :ok: |  spotbugs  |   2m 59s |  Used deprecated FindBugs config; considering switching to SpotBugs.  |
   | +1 :green_heart: |  findbugs  |   2m 55s |  branch-1 passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   1m 54s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 41s |  the patch passed with JDK v1.8.0_262  |
   | +1 :green_heart: |  javac  |   0m 41s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 43s |  the patch passed with JDK v1.7.0_272  |
   | +1 :green_heart: |  javac  |   0m 43s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   1m 29s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace issues.  |
   | +1 :green_heart: |  shadedjars  |   2m 47s |  patch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  hadoopcheck  |   4m 34s |  Patch does not cause any errors with Hadoop 2.8.5 2.9.2.  |
   | +1 :green_heart: |  javadoc  |   0m 30s |  the patch passed with JDK v1.8.0_262  |
   | +1 :green_heart: |  javadoc  |   0m 42s |  the patch passed with JDK v1.7.0_272  |
   | -1 :x: |  findbugs  |   2m 56s |  hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)  |
   ||| _ Other Tests _ |
   | -1 :x: |  unit  | 112m 28s |  hbase-server in the patch failed.  |
   | +1 :green_heart: |  asflicense  |   0m 35s |  The patch does not generate ASF License warnings.  |
   |  |   | 159m 39s |   |
   
   
   | Reason | Tests |
   |-------:|:------|
   | FindBugs | module:hbase-server |
   |  |  Sequence of calls to java.util.concurrent.ConcurrentHashMap may not be atomic in org.apache.hadoop.hbase.regionserver.LogRoller$1.logRollRequested(WALActionsListener$RollRequestReason)  At LogRoller.java:may not be atomic in org.apache.hadoop.hbase.regionserver.LogRoller$1.logRollRequested(WALActionsListener$RollRequestReason)  At LogRoller.java:[line 80] |
   | Failed junit tests | hadoop.hbase.procedure.TestFailedProcCleanup |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/5/artifact/out/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/2194 |
   | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
   | uname | Linux 9c5e1b730967 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2194/out/precommit/personality/provided.sh |
   | git revision | branch-1 / 839ce8e |
   | Default Java | 1.7.0_272 |
   | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_262 /usr/lib/jvm/zulu-7-amd64:1.7.0_272 |
   | findbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/5/artifact/out/new-findbugs-hbase-server.html |
   | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/5/artifact/out/patch-unit-hbase-server.txt |
   |  Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/5/testReport/ |
   | Max. process+thread count | 4437 (vs. ulimit of 10000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/5/console |
   | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Reidddddd commented on pull request #2194: HBASE-24849 Branch-1 Backport : HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll

Posted by GitBox <gi...@apache.org>.
Reidddddd commented on pull request #2194:
URL: https://github.com/apache/hbase/pull/2194#issuecomment-708997085


   can use this SuppressWarnings, with value and justification.
   {code}
   edu.umd.cs.findbugs.annotations.SuppressWarnings(
         value="",
         justification="")
   {code}


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] WenFeiYi closed pull request #2194: Branch-1 Backport : HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll

Posted by GitBox <gi...@apache.org>.
WenFeiYi closed pull request #2194:
URL: https://github.com/apache/hbase/pull/2194


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] anoopsjohn commented on a change in pull request #2194: HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll

Posted by GitBox <gi...@apache.org>.
anoopsjohn commented on a change in pull request #2194:
URL: https://github.com/apache/hbase/pull/2194#discussion_r467850190



##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/LogRoller.java
##########
@@ -229,11 +244,52 @@ private void scheduleFlush(final byte [] encodedRegionName) {
    */
   @VisibleForTesting
   public boolean walRollFinished() {
-    for (boolean needRoll : walNeedsRoll.values()) {
-      if (needRoll) {
+    long now = System.currentTimeMillis();
+    for (RollController controller : wals.values()) {
+      if (controller.needsRoll(now)) {
         return false;
       }
     }
     return true;
   }
+
+
+  /**
+   * Independently control the roll of each wal. When use multiwal,
+   * can avoid all wal roll together. see HBASE-24665 for detail
+   */
+  protected class RollController {
+    private final WAL wal;
+    private final AtomicBoolean rollRequest;
+    private long lastRollTime;
+
+    RollController(WAL wal) {
+      this.wal = wal;
+      this.rollRequest = new AtomicBoolean(false);
+      this.lastRollTime = System.currentTimeMillis();
+    }
+
+    public void requestRoll() {
+      this.rollRequest.set(true);
+    }
+
+    public byte[][] rollWal(long now) throws IOException {
+      this.lastRollTime = now;
+      byte[][] regionsToFlush = wal.rollWriter(true);
+      this.rollRequest.set(false);

Review comment:
       This is not as per other branch patches. Why?

##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/LogRoller.java
##########
@@ -141,9 +150,14 @@ public void run() {
     while (!server.isStopped()) {
       long now = System.currentTimeMillis();
       checkLowReplication(now);
-      boolean periodic = false;
       if (!rollLog.get()) {
-        periodic = (now - this.lastrolltime) > this.rollperiod;
+        boolean periodic = false;
+        for (RollController controller : wals.values()) {
+          if (controller.needsPeriodicRoll(now)) {
+            periodic = true;
+            break;
+          }
+        }

Review comment:
       This logic is not same as per branch-2 patches.  What is different here? why?

##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/LogRoller.java
##########
@@ -56,23 +56,32 @@
   private static final Log LOG = LogFactory.getLog(LogRoller.class);
   private final ReentrantLock rollLock = new ReentrantLock();
   private final AtomicBoolean rollLog = new AtomicBoolean(false);
-  private final ConcurrentHashMap<WAL, Boolean> walNeedsRoll =
-      new ConcurrentHashMap<WAL, Boolean>();
+  private final ConcurrentHashMap<WAL, RollController> wals =
+      new ConcurrentHashMap<WAL, RollController>();
   private final Server server;
   protected final RegionServerServices services;
-  private volatile long lastrolltime = System.currentTimeMillis();
   // Period to roll log.
-  private final long rollperiod;
+  private final long rollPeriod;
   private final int threadWakeFrequency;
   // The interval to check low replication on hlog's pipeline
-  private long checkLowReplicationInterval;
+  private final long checkLowReplicationInterval;
 
   public void addWAL(final WAL wal) {
-    if (null == walNeedsRoll.putIfAbsent(wal, Boolean.FALSE)) {
+    if (null == wals.putIfAbsent(wal, new RollController(wal))) {
       wal.registerWALActionsListener(new WALActionsListener.Base() {
         @Override
         public void logRollRequested(WALActionsListener.RollRequestReason reason) {
-          walNeedsRoll.put(wal, Boolean.TRUE);
+          RollController controller = wals.get(wal);

Review comment:
       I see.. because of jdk 1.7 support, we have to use this way and that is what Findbugs complain. 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Apache-HBase commented on pull request #2194: HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2194:
URL: https://github.com/apache/hbase/pull/2194#issuecomment-670322376


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |  12m  1s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  1s |  No case conflicting files found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  The patch appears to include 1 new or modified test files.  |
   ||| _ branch-1 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   9m 42s |  branch-1 passed  |
   | +1 :green_heart: |  compile  |   0m 40s |  branch-1 passed with JDK v1.8.0_262  |
   | +1 :green_heart: |  compile  |   0m 44s |  branch-1 passed with JDK v1.7.0_272  |
   | +1 :green_heart: |  checkstyle  |   1m 38s |  branch-1 passed  |
   | +1 :green_heart: |  shadedjars  |   3m  2s |  branch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 47s |  branch-1 passed with JDK v1.8.0_262  |
   | +1 :green_heart: |  javadoc  |   0m 41s |  branch-1 passed with JDK v1.7.0_272  |
   | +0 :ok: |  spotbugs  |   3m  0s |  Used deprecated FindBugs config; considering switching to SpotBugs.  |
   | +1 :green_heart: |  findbugs  |   2m 56s |  branch-1 passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   1m 52s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 40s |  the patch passed with JDK v1.8.0_262  |
   | +1 :green_heart: |  javac  |   0m 40s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 43s |  the patch passed with JDK v1.7.0_272  |
   | +1 :green_heart: |  javac  |   0m 43s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   1m 27s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace issues.  |
   | +1 :green_heart: |  shadedjars  |   2m 45s |  patch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  hadoopcheck  |   4m 33s |  Patch does not cause any errors with Hadoop 2.8.5 2.9.2.  |
   | +1 :green_heart: |  javadoc  |   0m 31s |  the patch passed with JDK v1.8.0_262  |
   | +1 :green_heart: |  javadoc  |   0m 42s |  the patch passed with JDK v1.7.0_272  |
   | -1 :x: |  findbugs  |   2m 54s |  hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  | 131m 34s |  hbase-server in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 33s |  The patch does not generate ASF License warnings.  |
   |  |   | 183m 51s |   |
   
   
   | Reason | Tests |
   |-------:|:------|
   | FindBugs | module:hbase-server |
   |  |  Sequence of calls to java.util.concurrent.ConcurrentHashMap may not be atomic in org.apache.hadoop.hbase.regionserver.LogRoller$1.logRollRequested(WALActionsListener$RollRequestReason)  At LogRoller.java:may not be atomic in org.apache.hadoop.hbase.regionserver.LogRoller$1.logRollRequested(WALActionsListener$RollRequestReason)  At LogRoller.java:[line 77] |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/2/artifact/out/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/2194 |
   | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
   | uname | Linux 42f2940b77ba 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2194/out/precommit/personality/provided.sh |
   | git revision | branch-1 / 839ce8e |
   | Default Java | 1.7.0_272 |
   | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_262 /usr/lib/jvm/zulu-7-amd64:1.7.0_272 |
   | findbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/2/artifact/out/new-findbugs-hbase-server.html |
   |  Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/2/testReport/ |
   | Max. process+thread count | 3967 (vs. ulimit of 10000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/2/console |
   | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Apache-HBase commented on pull request #2194: HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2194:
URL: https://github.com/apache/hbase/pull/2194#issuecomment-671169872


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |  11m 38s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  The patch appears to include 1 new or modified test files.  |
   ||| _ branch-1 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   9m 50s |  branch-1 passed  |
   | +1 :green_heart: |  compile  |   0m 43s |  branch-1 passed with JDK v1.8.0_262  |
   | +1 :green_heart: |  compile  |   0m 48s |  branch-1 passed with JDK v1.7.0_272  |
   | +1 :green_heart: |  checkstyle  |   1m 52s |  branch-1 passed  |
   | +1 :green_heart: |  shadedjars  |   3m 18s |  branch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 48s |  branch-1 passed with JDK v1.8.0_262  |
   | +1 :green_heart: |  javadoc  |   0m 42s |  branch-1 passed with JDK v1.7.0_272  |
   | +0 :ok: |  spotbugs  |   3m 13s |  Used deprecated FindBugs config; considering switching to SpotBugs.  |
   | +1 :green_heart: |  findbugs  |   3m  9s |  branch-1 passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   2m  3s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 42s |  the patch passed with JDK v1.8.0_262  |
   | +1 :green_heart: |  javac  |   0m 42s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 49s |  the patch passed with JDK v1.7.0_272  |
   | +1 :green_heart: |  javac  |   0m 49s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   1m 42s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace issues.  |
   | +1 :green_heart: |  shadedjars  |   3m  4s |  patch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  hadoopcheck  |   4m 56s |  Patch does not cause any errors with Hadoop 2.8.5 2.9.2.  |
   | +1 :green_heart: |  javadoc  |   0m 32s |  the patch passed with JDK v1.8.0_262  |
   | +1 :green_heart: |  javadoc  |   0m 42s |  the patch passed with JDK v1.7.0_272  |
   | -1 :x: |  findbugs  |   3m 10s |  hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)  |
   ||| _ Other Tests _ |
   | -1 :x: |  unit  | 131m  5s |  hbase-server in the patch failed.  |
   | +1 :green_heart: |  asflicense  |   0m 29s |  The patch does not generate ASF License warnings.  |
   |  |   | 185m 35s |   |
   
   
   | Reason | Tests |
   |-------:|:------|
   | FindBugs | module:hbase-server |
   |  |  Sequence of calls to java.util.concurrent.ConcurrentHashMap may not be atomic in org.apache.hadoop.hbase.regionserver.LogRoller$1.logRollRequested(WALActionsListener$RollRequestReason)  At LogRoller.java:may not be atomic in org.apache.hadoop.hbase.regionserver.LogRoller$1.logRollRequested(WALActionsListener$RollRequestReason)  At LogRoller.java:[line 79] |
   | Failed junit tests | hadoop.hbase.client.TestReplicasClient |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/4/artifact/out/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/2194 |
   | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
   | uname | Linux 5b01a2a2e1b4 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2194/out/precommit/personality/provided.sh |
   | git revision | branch-1 / 839ce8e |
   | Default Java | 1.7.0_272 |
   | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_262 /usr/lib/jvm/zulu-7-amd64:1.7.0_272 |
   | findbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/4/artifact/out/new-findbugs-hbase-server.html |
   | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/4/artifact/out/patch-unit-hbase-server.txt |
   |  Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/4/testReport/ |
   | Max. process+thread count | 4068 (vs. ulimit of 10000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/4/console |
   | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] WenFeiYi closed pull request #2194: HBASE-24849 Branch-1 Backport : HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll

Posted by GitBox <gi...@apache.org>.
WenFeiYi closed pull request #2194:
URL: https://github.com/apache/hbase/pull/2194


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Reidddddd commented on pull request #2194: HBASE-24849 Branch-1 Backport : HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll

Posted by GitBox <gi...@apache.org>.
Reidddddd commented on pull request #2194:
URL: https://github.com/apache/hbase/pull/2194#issuecomment-710021123


   We are cool now, thx for the contribution, this is nice @WenFeiYi 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Reidddddd commented on pull request #2194: HBASE-24849 Branch-1 Backport : HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll

Posted by GitBox <gi...@apache.org>.
Reidddddd commented on pull request #2194:
URL: https://github.com/apache/hbase/pull/2194#issuecomment-701321474


   I read @anoopsjohn's comment. If we can make sure the `FindBugs` is fine here, please add `SuppressWarnings`


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] anoopsjohn commented on pull request #2194: HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll

Posted by GitBox <gi...@apache.org>.
anoopsjohn commented on pull request #2194:
URL: https://github.com/apache/hbase/pull/2194#issuecomment-669665564


   branch-1 also had issue with precommit QA.  This is fixed now.  Can u pls do a commit (new line or so) to trigger QA


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Apache-HBase commented on pull request #2194: HBASE-24849 Branch-1 Backport : HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2194:
URL: https://github.com/apache/hbase/pull/2194#issuecomment-709274667


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   6m 45s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  The patch appears to include 1 new or modified test files.  |
   ||| _ branch-1 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   9m 50s |  branch-1 passed  |
   | +1 :green_heart: |  compile  |   0m 42s |  branch-1 passed with JDK v1.8.0_262  |
   | +1 :green_heart: |  compile  |   0m 44s |  branch-1 passed with JDK v1.7.0_272  |
   | +1 :green_heart: |  checkstyle  |   1m 40s |  branch-1 passed  |
   | +1 :green_heart: |  shadedjars  |   3m  1s |  branch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 49s |  branch-1 passed with JDK v1.8.0_262  |
   | +1 :green_heart: |  javadoc  |   0m 42s |  branch-1 passed with JDK v1.7.0_272  |
   | +0 :ok: |  spotbugs  |   3m  1s |  Used deprecated FindBugs config; considering switching to SpotBugs.  |
   | +1 :green_heart: |  findbugs  |   2m 58s |  branch-1 passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   1m 56s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 42s |  the patch passed with JDK v1.8.0_262  |
   | +1 :green_heart: |  javac  |   0m 42s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 44s |  the patch passed with JDK v1.7.0_272  |
   | +1 :green_heart: |  javac  |   0m 44s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   1m 28s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace issues.  |
   | +1 :green_heart: |  shadedjars  |   2m 51s |  patch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  hadoopcheck  |   4m 36s |  Patch does not cause any errors with Hadoop 2.8.5 2.9.2.  |
   | +1 :green_heart: |  javadoc  |   0m 31s |  the patch passed with JDK v1.8.0_262  |
   | +1 :green_heart: |  javadoc  |   0m 42s |  the patch passed with JDK v1.7.0_272  |
   | -1 :x: |  findbugs  |   2m 58s |  hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)  |
   ||| _ Other Tests _ |
   | -1 :x: |  unit  | 101m 21s |  hbase-server in the patch failed.  |
   | +1 :green_heart: |  asflicense  |   0m 35s |  The patch does not generate ASF License warnings.  |
   |  |   | 149m  1s |   |
   
   
   | Reason | Tests |
   |-------:|:------|
   | FindBugs | module:hbase-server |
   |  |  Sequence of calls to java.util.concurrent.ConcurrentHashMap may not be atomic in org.apache.hadoop.hbase.regionserver.LogRoller$1.logRollRequested(WALActionsListener$RollRequestReason)  At LogRoller.java:may not be atomic in org.apache.hadoop.hbase.regionserver.LogRoller$1.logRollRequested(WALActionsListener$RollRequestReason)  At LogRoller.java:[line 84] |
   | Failed junit tests | hadoop.hbase.procedure.TestZKProcedure |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/9/artifact/out/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/2194 |
   | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
   | uname | Linux aa425ef495e0 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2194/out/precommit/personality/provided.sh |
   | git revision | branch-1 / e4a8dcc |
   | Default Java | 1.7.0_272 |
   | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_262 /usr/lib/jvm/zulu-7-amd64:1.7.0_272 |
   | findbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/9/artifact/out/new-findbugs-hbase-server.html |
   | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/9/artifact/out/patch-unit-hbase-server.txt |
   |  Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/9/testReport/ |
   | Max. process+thread count | 3774 (vs. ulimit of 10000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/9/console |
   | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Reidddddd edited a comment on pull request #2194: HBASE-24849 Branch-1 Backport : HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll

Posted by GitBox <gi...@apache.org>.
Reidddddd edited a comment on pull request #2194:
URL: https://github.com/apache/hbase/pull/2194#issuecomment-708997085


   can use this SuppressWarnings, with value and justification.
   ```
   edu.umd.cs.findbugs.annotations.SuppressWarnings(
         value="",
         justification="")
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Apache-HBase commented on pull request #2194: HBASE-24849 Branch-1 Backport : HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2194:
URL: https://github.com/apache/hbase/pull/2194#issuecomment-700628756


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   8m  2s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  The patch appears to include 1 new or modified test files.  |
   ||| _ branch-1 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  10m 45s |  branch-1 passed  |
   | +1 :green_heart: |  compile  |   1m  3s |  branch-1 passed with JDK v1.8.0_262  |
   | +1 :green_heart: |  compile  |   0m 53s |  branch-1 passed with JDK v1.7.0_272  |
   | +1 :green_heart: |  checkstyle  |   2m 24s |  branch-1 passed  |
   | +1 :green_heart: |  shadedjars  |   5m 44s |  branch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   1m 16s |  branch-1 passed with JDK v1.8.0_262  |
   | +1 :green_heart: |  javadoc  |   1m  6s |  branch-1 passed with JDK v1.7.0_272  |
   | +0 :ok: |  spotbugs  |   4m 39s |  Used deprecated FindBugs config; considering switching to SpotBugs.  |
   | +1 :green_heart: |  findbugs  |   4m 32s |  branch-1 passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   3m  7s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m  8s |  the patch passed with JDK v1.8.0_262  |
   | +1 :green_heart: |  javac  |   1m  8s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 11s |  the patch passed with JDK v1.7.0_272  |
   | +1 :green_heart: |  javac  |   1m 11s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   2m 26s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace issues.  |
   | +1 :green_heart: |  shadedjars  |   4m 24s |  patch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  hadoopcheck  |   6m 46s |  Patch does not cause any errors with Hadoop 2.8.5 2.9.2.  |
   | +1 :green_heart: |  javadoc  |   0m 44s |  the patch passed with JDK v1.8.0_262  |
   | +1 :green_heart: |  javadoc  |   1m  3s |  the patch passed with JDK v1.7.0_272  |
   | -1 :x: |  findbugs  |   4m 22s |  hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  | 162m 33s |  hbase-server in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 37s |  The patch does not generate ASF License warnings.  |
   |  |   | 229m 31s |   |
   
   
   | Reason | Tests |
   |-------:|:------|
   | FindBugs | module:hbase-server |
   |  |  Sequence of calls to java.util.concurrent.ConcurrentHashMap may not be atomic in org.apache.hadoop.hbase.regionserver.LogRoller$1.logRollRequested(WALActionsListener$RollRequestReason)  At LogRoller.java:may not be atomic in org.apache.hadoop.hbase.regionserver.LogRoller$1.logRollRequested(WALActionsListener$RollRequestReason)  At LogRoller.java:[line 81] |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/6/artifact/out/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/2194 |
   | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
   | uname | Linux f4e8d0bd9fb5 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2194/out/precommit/personality/provided.sh |
   | git revision | branch-1 / 93b76fd |
   | Default Java | 1.7.0_272 |
   | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_262 /usr/lib/jvm/zulu-7-amd64:1.7.0_272 |
   | findbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/6/artifact/out/new-findbugs-hbase-server.html |
   |  Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/6/testReport/ |
   | Max. process+thread count | 4580 (vs. ulimit of 10000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/6/console |
   | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Reidddddd commented on pull request #2194: HBASE-24849 Branch-1 Backport : HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll

Posted by GitBox <gi...@apache.org>.
Reidddddd commented on pull request #2194:
URL: https://github.com/apache/hbase/pull/2194#issuecomment-700514955


   There should be one key point in old version which leads to all WALs rolled, can you point that out.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] WenFeiYi commented on pull request #2194: HBASE-24849 Branch-1 Backport : HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll

Posted by GitBox <gi...@apache.org>.
WenFeiYi commented on pull request #2194:
URL: https://github.com/apache/hbase/pull/2194#issuecomment-708916809


   @Reidddddd In old version, it is not judged which wal triggered before roll, and all wals use the same roll period.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org