You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2020/09/24 04:22:35 UTC

[GitHub] [hbase] saintstack opened a new pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…

saintstack opened a new pull request #2451:
URL: https://github.com/apache/hbase/pull/2451


   …hen hbase:meta assigned to RS
   
   Fill in gap left by HBASE-11183 'Timeline Consistent region replicas - Phase 2 design'.
   HBASE-11183 left off implementing 'async WAL Replication' on the hbase:meta
   Table; hbase:meta Table could only do Phase 1 Region Replicas reading
   the primary Regions' hfiles. Here we add 'async WAL Replication' to
   hbase:meta so Replicas can be more current with the primary's changes.
   
   Adds a 'special' ReplicationSource that reads hbase:meta WAL files and replicates
   all edits to the configured in-cluster endpoint (Defaults to the
   RegionReadReplicaEndpoint.class -- set hbase.region.meta.replica.replication to
   target a different endpoint implementation).
   
   Set hbase.region.meta.replica.replication.enabled to enable async WAL
   Replication for hbase:meta region replicas. Its off by default.
   
   The HBaseMetaReplicationSource for async WAL Replication of hbase:meta does
   NOT need to keep up WAL offset or a queue of WALs-to-replicate in the
   replication queue store as is done in other ReplicationSource implementations;
   the HBaseMetaReplicationSource is for Region Replicas only. General
   Replication does not replicate hbase:meta. hbase:meta Region Replicas reset
   on crash of the primary replica so there is no need to 'recover'
   replication that was running on the crashed server.
   
   Because it so different in operation, the HBaseMetaReplicationSource is bolted
   on to the side of the ReplicationSourceManager. It is lazily
   instantiated to match the lazy instantiation of the hbase:meta
   WALProvider, created and started on the open of the first Region of an
   hbase:meta table. Thereafter it stays up till the process dies, even if
   all hbase:meta Regions have moved off the server, in case a hbase:meta
   Region is moved back (Doing this latter simplifies the implementation)
   
   hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
     Read configuration to see if we need to wait on setting a Region read-enabled
     (if so, replicas will only flip to enable read after confirming a
     flush of the primary so they for sure are a replica of a known point)
   
   hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/AssignRegionHandler.java
    If configured, on open of hbase:meta, ask the ReplicationSourceManager
    to add a ReplicationSource (if it hasn't already).
   
   hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/RegionReplicaFlushHandler.java
    Edit log message.
   
   hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/UnassignRegionHandler.java
    If configured, on close of hbase:meta, update ReplicationSourceManager
    that a source Region has closed.
   
   hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceFactory.java
    javadoc and make constructor private.
   
   hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceInterface.java
    Add logPositionAndCleanOldLogs w/ default of the old behavior so
    HBaseMetaReplicationSource can bypass updating store with WAL position,
    etc.
   
   hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java
    Add creation and start of an HBaseMetaReplicationSource.
   
   hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceShipper.java
    Go via ReplicationSource when calling logPostionAndCleanOldLogs.
   
   hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/WALFileLengthProvider.java
    Javadoc.
   
   hbase-server/src/main/java/org/apache/hadoop/hbase/util/ServerRegionReplicaUtil.java
    Add utility for reading configurations for hbase:meta region replicas.
   
   hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALEdit.java
    Add an isFlushMarker utility.
   
   hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALFactory.java
    Javadoc.
   
   hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestRegionReplicaReplicationEndpoint.java
    Use define.
   
   hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HBaseMetaReplicationSource.java
    Specical version of ReplicationSource for Region Replicas on
    hbase:meta.
   
   hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HBaseMetaReplicationSourcePeer.java
    Needs a special peer too (peers are baked into replication though we
    don't use 'peers' here)
   
   hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestMetaRegionReplicaReplicationEndpoint.java
   hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestWALProvider.java
    Tests.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Apache-HBase commented on pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2451:
URL: https://github.com/apache/hbase/pull/2451#issuecomment-701651114


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   0m 32s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  4s |  Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ HBASE-18070 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   4m 46s |  HBASE-18070 passed  |
   | +1 :green_heart: |  compile  |   1m 14s |  HBASE-18070 passed  |
   | +1 :green_heart: |  shadedjars  |   7m 13s |  branch has no errors when building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 48s |  hbase-server in HBASE-18070 failed.  |
   | -0 :warning: |  patch  |   8m 16s |  Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   4m 29s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 14s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 14s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   7m  7s |  patch has no errors when building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 44s |  hbase-server in the patch failed.  |
   ||| _ Other Tests _ |
   | -1 :x: |  unit  | 144m 54s |  hbase-server in the patch failed.  |
   |  |   | 175m  7s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/2451 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux a9ad485350f3 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | HBASE-18070 / 03e8e5b517 |
   | Default Java | 2020-01-14 |
   | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/5/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
   | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/5/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
   | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/5/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
   |  Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/5/testReport/ |
   | Max. process+thread count | 3953 (vs. ulimit of 30000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/5/console |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] huaxiangsun commented on pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…

Posted by GitBox <gi...@apache.org>.
huaxiangsun commented on pull request #2451:
URL: https://github.com/apache/hbase/pull/2451#issuecomment-702785331


   > I was trying to avoid synchronizing on the ReplicationSourceManager... seemed brutish.
   
   Good.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Apache-HBase commented on pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2451:
URL: https://github.com/apache/hbase/pull/2451#issuecomment-702981650


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   1m  3s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  4s |  Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ HBASE-18070 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   3m 42s |  HBASE-18070 passed  |
   | +1 :green_heart: |  compile  |   0m 56s |  HBASE-18070 passed  |
   | +1 :green_heart: |  shadedjars  |   6m 35s |  branch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 39s |  HBASE-18070 passed  |
   | -0 :warning: |  patch  |   7m 26s |  Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   3m 32s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 55s |  the patch passed  |
   | +1 :green_heart: |  javac  |   0m 55s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   6m 30s |  patch has no errors when building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 37s |  hbase-server generated 1 new + 29 unchanged - 0 fixed = 30 total (was 29)  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  | 147m 23s |  hbase-server in the patch passed.  |
   |  |   | 173m 58s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/7/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/2451 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 8ca223171c4a 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | HBASE-18070 / 03e8e5b517 |
   | Default Java | 1.8.0_232 |
   | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/7/artifact/yetus-jdk8-hadoop3-check/output/diff-javadoc-javadoc-hbase-server.txt |
   |  Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/7/testReport/ |
   | Max. process+thread count | 4112 (vs. ulimit of 30000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/7/console |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Apache-HBase commented on pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2451:
URL: https://github.com/apache/hbase/pull/2451#issuecomment-698114608


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   2m 33s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  4s |  Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ HBASE-18070 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   7m  7s |  HBASE-18070 passed  |
   | +1 :green_heart: |  compile  |   1m 45s |  HBASE-18070 passed  |
   | +1 :green_heart: |  shadedjars  |  11m 47s |  branch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   1m  7s |  HBASE-18070 passed  |
   ||| _ Patch Compile Tests _ |
   | -1 :x: |  mvninstall  |   3m 13s |  root in the patch failed.  |
   | -1 :x: |  compile  |   1m 11s |  hbase-server in the patch failed.  |
   | -0 :warning: |  javac  |   1m 11s |  hbase-server in the patch failed.  |
   | -1 :x: |  shadedjars  |   8m 55s |  patch has 28 errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   1m  3s |  the patch passed  |
   ||| _ Other Tests _ |
   | -1 :x: |  unit  |   1m  8s |  hbase-server in the patch failed.  |
   |  |   |  41m 28s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/2451 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 8ac364d9574c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | HBASE-18070 / 03e8e5b517 |
   | Default Java | 1.8.0_232 |
   | mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/1/artifact/yetus-jdk8-hadoop3-check/output/patch-mvninstall-root.txt |
   | compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/1/artifact/yetus-jdk8-hadoop3-check/output/patch-compile-hbase-server.txt |
   | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/1/artifact/yetus-jdk8-hadoop3-check/output/patch-compile-hbase-server.txt |
   | shadedjars | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/1/artifact/yetus-jdk8-hadoop3-check/output/patch-shadedjars.txt |
   | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
   |  Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/1/testReport/ |
   | Max. process+thread count | 78 (vs. ulimit of 30000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/1/console |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] saintstack commented on pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…

Posted by GitBox <gi...@apache.org>.
saintstack commented on pull request #2451:
URL: https://github.com/apache/hbase/pull/2451#issuecomment-701552064


   I ran the failed tests locally and they passed. Rerunning.
   
   @Apache9  Mind taking a look sir? This the direction you suggested?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Apache-HBase commented on pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2451:
URL: https://github.com/apache/hbase/pull/2451#issuecomment-701056437


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   3m 25s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any @author tags.  |
   ||| _ HBASE-18070 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   6m 10s |  HBASE-18070 passed  |
   | +1 :green_heart: |  checkstyle  |   1m 28s |  HBASE-18070 passed  |
   | +1 :green_heart: |  spotbugs  |   2m 54s |  HBASE-18070 passed  |
   | -0 :warning: |  patch  |   3m  4s |  Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m  9s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   1m 45s |  hbase-server: The patch generated 0 new + 42 unchanged - 11 fixed = 42 total (was 53)  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace issues.  |
   | +1 :green_heart: |  hadoopcheck  |  12m 10s |  Patch does not cause any errors with Hadoop 3.1.2 3.2.1.  |
   | +1 :green_heart: |  spotbugs  |   2m 22s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 16s |  The patch does not generate ASF License warnings.  |
   |  |   |  43m 26s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/3/artifact/yetus-general-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/2451 |
   | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
   | uname | Linux 02c9af434ec1 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | HBASE-18070 / 03e8e5b517 |
   | Max. process+thread count | 94 (vs. ulimit of 30000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/3/console |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] huaxiangsun commented on pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…

Posted by GitBox <gi...@apache.org>.
huaxiangsun commented on pull request #2451:
URL: https://github.com/apache/hbase/pull/2451#issuecomment-700193476


   Start reading, will be  back.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] huaxiangsun commented on a change in pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…

Posted by GitBox <gi...@apache.org>.
huaxiangsun commented on a change in pull request #2451:
URL: https://github.com/apache/hbase/pull/2451#discussion_r497745552



##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/util/ServerRegionReplicaUtil.java
##########
@@ -154,47 +162,60 @@ public static StoreFileInfo getStoreFileInfo(Configuration conf, FileSystem fs,
   }
 
   /**
-   * Create replication peer for replicating to region replicas if needed.
+   * Create replication peer for replicating user-space Region Read Replicas.
    * @param conf configuration to use
-   * @throws IOException
    */
   public static void setupRegionReplicaReplication(Configuration conf) throws IOException {
-    if (!isRegionReplicaReplicationEnabled(conf)) {
+    if (!conf.getBoolean(REGION_REPLICA_REPLICATION_CONF_KEY, DEFAULT_REGION_REPLICA_REPLICATION)) {
       return;
     }
-
+    String peerId = REGION_REPLICA_REPLICATION_PEER;
     try (Connection connection = ConnectionFactory.createConnection(conf);
       Admin admin = connection.getAdmin()) {
       ReplicationPeerConfig peerConfig = null;
       try {
-        peerConfig = admin.getReplicationPeerConfig(REGION_REPLICA_REPLICATION_PEER);
+        peerConfig = admin.getReplicationPeerConfig(peerId);
       } catch (ReplicationPeerNotFoundException e) {
-        LOG.warn(
-          "Region replica replication peer id=" + REGION_REPLICA_REPLICATION_PEER + " not exist",
-          e);
+        LOG.warn("Region replica peer id={} does not exist", peerId, e);
       }
-
       if (peerConfig == null) {
-        LOG.info("Region replica replication peer id=" + REGION_REPLICA_REPLICATION_PEER
-          + " not exist. Creating...");
-        peerConfig = new ReplicationPeerConfig();
-        peerConfig.setClusterKey(ZKConfig.getZooKeeperClusterKey(conf));
-        peerConfig.setReplicationEndpointImpl(RegionReplicaReplicationEndpoint.class.getName());
-        admin.addReplicationPeer(REGION_REPLICA_REPLICATION_PEER, peerConfig);
+        LOG.info("Region Read Replica peerId={} does not exist; creating...", peerId);
+        peerConfig = ReplicationPeerConfig.newBuilder().
+          setClusterKey(ZKConfig.getZooKeeperClusterKey(conf)).
+          setReplicationEndpointImpl(RegionReplicaReplicationEndpoint.class.getName()).build();
+        admin.addReplicationPeer(peerId, peerConfig);
       }
     }
   }
 
-  public static boolean isRegionReplicaReplicationEnabled(Configuration conf) {
-    return conf.getBoolean(REGION_REPLICA_REPLICATION_CONF_KEY,
-      DEFAULT_REGION_REPLICA_REPLICATION);
+  /**
+   * @return True if Region Read Replica is enabled for <code>tn</code>.
+   */
+  public static boolean isRegionReplicaReplicationEnabled(Configuration conf, TableName tn) {
+    return isMetaRegionReplicaReplicationEnabled(conf, tn) ||
+      conf.getBoolean(REGION_REPLICA_REPLICATION_CONF_KEY, DEFAULT_REGION_REPLICA_REPLICATION);

Review comment:
       Nit: can we have another utility method of 
   conf.getBoolean(REGION_REPLICA_REPLICATION_CONF_KEY, DEFAULT_REGION_REPLICA_REPLICATION)
   as is isUserRegionReplicaReplicationEnabled(***)?
   
   It will be easy to read.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] saintstack merged pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…

Posted by GitBox <gi...@apache.org>.
saintstack merged pull request #2451:
URL: https://github.com/apache/hbase/pull/2451


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Apache-HBase commented on pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2451:
URL: https://github.com/apache/hbase/pull/2451#issuecomment-698110263


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   1m  6s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ HBASE-18070 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   4m 51s |  HBASE-18070 passed  |
   | +1 :green_heart: |  compile  |   1m 22s |  HBASE-18070 passed  |
   | +1 :green_heart: |  shadedjars  |   7m 40s |  branch has no errors when building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 42s |  hbase-server in HBASE-18070 failed.  |
   ||| _ Patch Compile Tests _ |
   | -1 :x: |  mvninstall  |   2m 25s |  root in the patch failed.  |
   | -1 :x: |  compile  |   0m 46s |  hbase-server in the patch failed.  |
   | -0 :warning: |  javac  |   0m 46s |  hbase-server in the patch failed.  |
   | -1 :x: |  shadedjars  |   5m 35s |  patch has 28 errors when building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 40s |  hbase-server in the patch failed.  |
   ||| _ Other Tests _ |
   | -1 :x: |  unit  |   0m 48s |  hbase-server in the patch failed.  |
   |  |   |  27m  6s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/2451 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 00247b1156d1 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | HBASE-18070 / 03e8e5b517 |
   | Default Java | 2020-01-14 |
   | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
   | mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/1/artifact/yetus-jdk11-hadoop3-check/output/patch-mvninstall-root.txt |
   | compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/1/artifact/yetus-jdk11-hadoop3-check/output/patch-compile-hbase-server.txt |
   | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/1/artifact/yetus-jdk11-hadoop3-check/output/patch-compile-hbase-server.txt |
   | shadedjars | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/1/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt |
   | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
   | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
   |  Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/1/testReport/ |
   | Max. process+thread count | 89 (vs. ulimit of 30000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/1/console |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] saintstack commented on pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…

Posted by GitBox <gi...@apache.org>.
saintstack commented on pull request #2451:
URL: https://github.com/apache/hbase/pull/2451#issuecomment-698105096


   This replaces #2413 (and the action closed the original PR). This PR addresses coments there. Let me close out the #2413 by pointing here and addressing any outstanding remarks there.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] huaxiangsun commented on pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…

Posted by GitBox <gi...@apache.org>.
huaxiangsun commented on pull request #2451:
URL: https://github.com/apache/hbase/pull/2451#issuecomment-702790612


   > 
   > This seems like a bug. I can file a follow-on to write a test to repro and to fix?
   
   Yeah, that is what I read from the code. A follow-on is good as we can create a unitest to show if the issue really exists.
   > 
   > > It will be better that the source can be removed and re-added back when meta region is moved back, it is much cleaner.
   > 
   > This I think this could prove difficult if split meta and for some reason, many meta Regions on a single server w/ some opening and come closing. The open/close are each in their own handlers so could end up making for some interesting create/destroy scenarios. Keeping up the ReplicationSource once created seems easy to reason about.
   
   Agreed.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Apache-HBase commented on pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2451:
URL: https://github.com/apache/hbase/pull/2451#issuecomment-702994670


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   1m  3s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ HBASE-18070 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   4m 37s |  HBASE-18070 passed  |
   | +1 :green_heart: |  compile  |   1m 12s |  HBASE-18070 passed  |
   | +1 :green_heart: |  shadedjars  |   7m 20s |  branch has no errors when building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 43s |  hbase-server in HBASE-18070 failed.  |
   | -0 :warning: |  patch  |   8m 14s |  Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   4m 32s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 10s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 10s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   7m 21s |  patch has no errors when building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 41s |  hbase-server in the patch failed.  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  | 195m  5s |  hbase-server in the patch passed.  |
   |  |   | 225m 26s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/7/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/2451 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 99675009caa5 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | HBASE-18070 / 03e8e5b517 |
   | Default Java | 2020-01-14 |
   | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/7/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
   | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/7/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
   |  Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/7/testReport/ |
   | Max. process+thread count | 3146 (vs. ulimit of 30000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/7/console |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Apache-HBase commented on pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2451:
URL: https://github.com/apache/hbase/pull/2451#issuecomment-701189108


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   7m 56s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ HBASE-18070 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m 27s |  HBASE-18070 passed  |
   | +1 :green_heart: |  compile  |   1m 20s |  HBASE-18070 passed  |
   | +1 :green_heart: |  shadedjars  |   9m 49s |  branch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 51s |  HBASE-18070 passed  |
   | -0 :warning: |  patch  |  10m 52s |  Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m 13s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 21s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 21s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   9m 40s |  patch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 48s |  the patch passed  |
   ||| _ Other Tests _ |
   | -1 :x: |  unit  | 201m 36s |  hbase-server in the patch failed.  |
   |  |   | 246m 34s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/2451 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux d9a9bf7b5d6a 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | HBASE-18070 / 03e8e5b517 |
   | Default Java | 1.8.0_232 |
   | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/4/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
   |  Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/4/testReport/ |
   | Max. process+thread count | 3789 (vs. ulimit of 30000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/4/console |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Apache-HBase commented on pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2451:
URL: https://github.com/apache/hbase/pull/2451#issuecomment-701579934


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   0m 36s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any @author tags.  |
   ||| _ HBASE-18070 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   4m 45s |  HBASE-18070 passed  |
   | +1 :green_heart: |  checkstyle  |   1m 17s |  HBASE-18070 passed  |
   | +1 :green_heart: |  spotbugs  |   2m 13s |  HBASE-18070 passed  |
   | -0 :warning: |  patch  |   2m 22s |  Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   3m 22s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   1m  3s |  hbase-server: The patch generated 0 new + 42 unchanged - 11 fixed = 42 total (was 53)  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace issues.  |
   | +1 :green_heart: |  hadoopcheck  |  11m 18s |  Patch does not cause any errors with Hadoop 3.1.2 3.2.1.  |
   | +1 :green_heart: |  spotbugs  |   2m 12s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 15s |  The patch does not generate ASF License warnings.  |
   |  |   |  34m 11s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/5/artifact/yetus-general-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/2451 |
   | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
   | uname | Linux f68357939a39 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | HBASE-18070 / 03e8e5b517 |
   | Max. process+thread count | 94 (vs. ulimit of 30000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/5/console |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Apache-HBase commented on pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2451:
URL: https://github.com/apache/hbase/pull/2451#issuecomment-701702857


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   1m 27s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  4s |  Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ HBASE-18070 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m 12s |  HBASE-18070 passed  |
   | +1 :green_heart: |  compile  |   1m 28s |  HBASE-18070 passed  |
   | +1 :green_heart: |  shadedjars  |  10m  6s |  branch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 55s |  HBASE-18070 passed  |
   | -0 :warning: |  patch  |  11m 15s |  Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   4m 58s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 26s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 26s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   9m 29s |  patch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 54s |  the patch passed  |
   ||| _ Other Tests _ |
   | -1 :x: |  unit  | 281m  5s |  hbase-server in the patch failed.  |
   |  |   | 319m 14s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/5/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/2451 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux d99f785bdd38 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | HBASE-18070 / 03e8e5b517 |
   | Default Java | 1.8.0_232 |
   | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/5/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
   |  Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/5/testReport/ |
   | Max. process+thread count | 3084 (vs. ulimit of 30000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/5/console |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Apache-HBase commented on pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2451:
URL: https://github.com/apache/hbase/pull/2451#issuecomment-698107974


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   0m 28s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any @author tags.  |
   ||| _ HBASE-18070 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   4m  1s |  HBASE-18070 passed  |
   | +1 :green_heart: |  checkstyle  |   1m 23s |  HBASE-18070 passed  |
   | +1 :green_heart: |  spotbugs  |   2m 22s |  HBASE-18070 passed  |
   ||| _ Patch Compile Tests _ |
   | -1 :x: |  mvninstall  |   2m  4s |  root in the patch failed.  |
   | -0 :warning: |  checkstyle  |   1m 15s |  hbase-server: The patch generated 1 new + 42 unchanged - 11 fixed = 43 total (was 53)  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace issues.  |
   | -1 :x: |  hadoopcheck  |   2m 19s |  The patch causes 28 errors with Hadoop v3.1.2.  |
   | -1 :x: |  hadoopcheck  |   4m 39s |  The patch causes 28 errors with Hadoop v3.2.1.  |
   | -1 :x: |  spotbugs  |   0m 38s |  hbase-server in the patch failed.  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 13s |  The patch does not generate ASF License warnings.  |
   |  |   |  18m 42s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/1/artifact/yetus-general-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/2451 |
   | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
   | uname | Linux 6b28ca70613b 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | HBASE-18070 / 03e8e5b517 |
   | mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/1/artifact/yetus-general-check/output/patch-mvninstall-root.txt |
   | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
   | hadoopcheck | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/1/artifact/yetus-general-check/output/patch-javac-3.1.2.txt |
   | hadoopcheck | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/1/artifact/yetus-general-check/output/patch-javac-3.2.1.txt |
   | spotbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/1/artifact/yetus-general-check/output/patch-spotbugs-hbase-server.txt |
   | Max. process+thread count | 94 (vs. ulimit of 30000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/1/console |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] saintstack commented on pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…

Posted by GitBox <gi...@apache.org>.
saintstack commented on pull request #2451:
URL: https://github.com/apache/hbase/pull/2451#issuecomment-698105096


   This replaces #2413 (and the action closed the original PR). This PR addresses coments there. Let me close out the #2413 by pointing here and addressing any outstanding remarks there.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] saintstack commented on pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…

Posted by GitBox <gi...@apache.org>.
saintstack commented on pull request #2451:
URL: https://github.com/apache/hbase/pull/2451#issuecomment-702506783


   @huaxiangsun I added HBASE-25145 "WALReader quits if nothing to replicate (and won't restart)" to look at the issue you found above sir.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] saintstack commented on a change in pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…

Posted by GitBox <gi...@apache.org>.
saintstack commented on a change in pull request #2451:
URL: https://github.com/apache/hbase/pull/2451#discussion_r498424906



##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/util/ServerRegionReplicaUtil.java
##########
@@ -154,47 +162,60 @@ public static StoreFileInfo getStoreFileInfo(Configuration conf, FileSystem fs,
   }
 
   /**
-   * Create replication peer for replicating to region replicas if needed.
+   * Create replication peer for replicating user-space Region Read Replicas.
    * @param conf configuration to use
-   * @throws IOException
    */
   public static void setupRegionReplicaReplication(Configuration conf) throws IOException {
-    if (!isRegionReplicaReplicationEnabled(conf)) {
+    if (!conf.getBoolean(REGION_REPLICA_REPLICATION_CONF_KEY, DEFAULT_REGION_REPLICA_REPLICATION)) {
       return;
     }
-
+    String peerId = REGION_REPLICA_REPLICATION_PEER;
     try (Connection connection = ConnectionFactory.createConnection(conf);
       Admin admin = connection.getAdmin()) {
       ReplicationPeerConfig peerConfig = null;
       try {
-        peerConfig = admin.getReplicationPeerConfig(REGION_REPLICA_REPLICATION_PEER);
+        peerConfig = admin.getReplicationPeerConfig(peerId);
       } catch (ReplicationPeerNotFoundException e) {
-        LOG.warn(
-          "Region replica replication peer id=" + REGION_REPLICA_REPLICATION_PEER + " not exist",
-          e);
+        LOG.warn("Region replica peer id={} does not exist", peerId, e);
       }
-
       if (peerConfig == null) {
-        LOG.info("Region replica replication peer id=" + REGION_REPLICA_REPLICATION_PEER
-          + " not exist. Creating...");
-        peerConfig = new ReplicationPeerConfig();
-        peerConfig.setClusterKey(ZKConfig.getZooKeeperClusterKey(conf));
-        peerConfig.setReplicationEndpointImpl(RegionReplicaReplicationEndpoint.class.getName());
-        admin.addReplicationPeer(REGION_REPLICA_REPLICATION_PEER, peerConfig);
+        LOG.info("Region Read Replica peerId={} does not exist; creating...", peerId);
+        peerConfig = ReplicationPeerConfig.newBuilder().
+          setClusterKey(ZKConfig.getZooKeeperClusterKey(conf)).
+          setReplicationEndpointImpl(RegionReplicaReplicationEndpoint.class.getName()).build();
+        admin.addReplicationPeer(peerId, peerConfig);
       }
     }
   }
 
-  public static boolean isRegionReplicaReplicationEnabled(Configuration conf) {
-    return conf.getBoolean(REGION_REPLICA_REPLICATION_CONF_KEY,
-      DEFAULT_REGION_REPLICA_REPLICATION);
+  /**
+   * @return True if Region Read Replica is enabled for <code>tn</code>.
+   */
+  public static boolean isRegionReplicaReplicationEnabled(Configuration conf, TableName tn) {
+    return isMetaRegionReplicaReplicationEnabled(conf, tn) ||
+      conf.getBoolean(REGION_REPLICA_REPLICATION_CONF_KEY, DEFAULT_REGION_REPLICA_REPLICATION);

Review comment:
       np. sounds good.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Apache9 commented on a change in pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…

Posted by GitBox <gi...@apache.org>.
Apache9 commented on a change in pull request #2451:
URL: https://github.com/apache/hbase/pull/2451#discussion_r498885329



##########
File path: hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestMetaRegionReplicaReplicationEndpoint.java
##########
@@ -0,0 +1,334 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.replication.regionserver;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertNotNull;
+import static org.junit.Assert.assertTrue;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Objects;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.CellScanner;
+import org.apache.hadoop.hbase.CellUtil;
+import org.apache.hadoop.hbase.ClientMetaTableAccessor;
+import org.apache.hadoop.hbase.HBaseClassTestRule;
+import org.apache.hadoop.hbase.HBaseTestingUtility;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.HRegionLocation;
+import org.apache.hadoop.hbase.MetaTableAccessor;
+import org.apache.hadoop.hbase.MiniHBaseCluster;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.Waiter;
+import org.apache.hadoop.hbase.client.RegionLocator;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.regionserver.HRegion;
+import org.apache.hadoop.hbase.regionserver.HRegionServer;
+import org.apache.hadoop.hbase.regionserver.Region;
+import org.apache.hadoop.hbase.regionserver.RegionScanner;
+import org.apache.hadoop.hbase.testclassification.FlakeyTests;
+import org.apache.hadoop.hbase.testclassification.LargeTests;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.util.ServerRegionReplicaUtil;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.rules.TestName;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Tests RegionReplicaReplicationEndpoint class for hbase:meta by setting up region replicas and
+ * verifying async wal replication replays the edits to the secondary region in various scenarios.
+ * @see TestRegionReplicaReplicationEndpoint
+ */
+@Category({FlakeyTests.class, LargeTests.class})

Review comment:
       Why FlakeyTests?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Apache-HBase commented on pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2451:
URL: https://github.com/apache/hbase/pull/2451#issuecomment-698134480


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   2m 12s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ HBASE-18070 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m  9s |  HBASE-18070 passed  |
   | +1 :green_heart: |  compile  |   1m 14s |  HBASE-18070 passed  |
   | +1 :green_heart: |  shadedjars  |   9m  2s |  branch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   1m  0s |  HBASE-18070 passed  |
   | -0 :warning: |  patch  |  10m 14s |  Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m 26s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 29s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 29s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |  10m  3s |  patch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 55s |  the patch passed  |
   ||| _ Other Tests _ |
   | -1 :x: |  unit  |  12m 46s |  hbase-server in the patch failed.  |
   |  |   |  50m 31s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/2451 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux f9bc4142f32e 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | HBASE-18070 / 03e8e5b517 |
   | Default Java | 1.8.0_232 |
   | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
   |  Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/2/testReport/ |
   | Max. process+thread count | 666 (vs. ulimit of 30000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/2/console |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Apache-HBase commented on pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2451:
URL: https://github.com/apache/hbase/pull/2451#issuecomment-698107974






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Apache-HBase commented on pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2451:
URL: https://github.com/apache/hbase/pull/2451#issuecomment-701120252


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   0m 38s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  4s |  Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ HBASE-18070 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m 26s |  HBASE-18070 passed  |
   | +1 :green_heart: |  compile  |   1m 29s |  HBASE-18070 passed  |
   | +1 :green_heart: |  shadedjars  |   9m 30s |  branch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   1m  1s |  HBASE-18070 passed  |
   | -0 :warning: |  patch  |  10m 46s |  Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   4m 40s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m  2s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m  2s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   6m 56s |  patch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 39s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  | 155m  5s |  hbase-server in the patch passed.  |
   |  |   | 188m 19s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/2451 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 9415ee7fb138 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | HBASE-18070 / 03e8e5b517 |
   | Default Java | 1.8.0_232 |
   |  Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/3/testReport/ |
   | Max. process+thread count | 4314 (vs. ulimit of 30000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/3/console |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Apache-HBase commented on pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2451:
URL: https://github.com/apache/hbase/pull/2451#issuecomment-702932202


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   0m 31s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any @author tags.  |
   ||| _ HBASE-18070 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   4m 17s |  HBASE-18070 passed  |
   | +1 :green_heart: |  checkstyle  |   1m 15s |  HBASE-18070 passed  |
   | +1 :green_heart: |  spotbugs  |   2m 27s |  HBASE-18070 passed  |
   | -0 :warning: |  patch  |   2m 37s |  Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   3m 54s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   1m 13s |  hbase-server: The patch generated 0 new + 42 unchanged - 11 fixed = 42 total (was 53)  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace issues.  |
   | +1 :green_heart: |  hadoopcheck  |  12m 36s |  Patch does not cause any errors with Hadoop 3.1.2 3.2.1.  |
   | +1 :green_heart: |  spotbugs  |   2m 34s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 17s |  The patch does not generate ASF License warnings.  |
   |  |   |  37m  1s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/7/artifact/yetus-general-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/2451 |
   | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
   | uname | Linux 86130bfb3678 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | HBASE-18070 / 03e8e5b517 |
   | Max. process+thread count | 94 (vs. ulimit of 30000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/7/console |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Apache-HBase commented on pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2451:
URL: https://github.com/apache/hbase/pull/2451#issuecomment-702516156


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   1m  2s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any @author tags.  |
   ||| _ HBASE-18070 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   3m 46s |  HBASE-18070 passed  |
   | +1 :green_heart: |  checkstyle  |   1m  8s |  HBASE-18070 passed  |
   | +1 :green_heart: |  spotbugs  |   2m 12s |  HBASE-18070 passed  |
   | -0 :warning: |  patch  |   2m 21s |  Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   3m 37s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   1m  8s |  hbase-server: The patch generated 0 new + 42 unchanged - 11 fixed = 42 total (was 53)  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace issues.  |
   | +1 :green_heart: |  hadoopcheck  |  11m 52s |  Patch does not cause any errors with Hadoop 3.1.2 3.2.1.  |
   | +1 :green_heart: |  spotbugs  |   2m 22s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 14s |  The patch does not generate ASF License warnings.  |
   |  |   |  34m 45s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/6/artifact/yetus-general-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/2451 |
   | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
   | uname | Linux e2aabc6d26a9 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | HBASE-18070 / 03e8e5b517 |
   | Max. process+thread count | 94 (vs. ulimit of 30000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/6/console |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Apache-HBase commented on pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2451:
URL: https://github.com/apache/hbase/pull/2451#issuecomment-698129584


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   0m 45s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any @author tags.  |
   ||| _ HBASE-18070 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   3m 57s |  HBASE-18070 passed  |
   | +1 :green_heart: |  checkstyle  |   1m 15s |  HBASE-18070 passed  |
   | +1 :green_heart: |  spotbugs  |   2m 15s |  HBASE-18070 passed  |
   | -0 :warning: |  patch  |   2m 24s |  Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   3m 50s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   1m 11s |  hbase-server: The patch generated 0 new + 42 unchanged - 11 fixed = 42 total (was 53)  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace issues.  |
   | +1 :green_heart: |  hadoopcheck  |  13m 22s |  Patch does not cause any errors with Hadoop 3.1.2 3.2.1.  |
   | +1 :green_heart: |  spotbugs  |   2m 36s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 12s |  The patch does not generate ASF License warnings.  |
   |  |   |  37m 23s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/2/artifact/yetus-general-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/2451 |
   | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
   | uname | Linux 2a47305d6ff7 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | HBASE-18070 / 03e8e5b517 |
   | Max. process+thread count | 94 (vs. ulimit of 30000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/2/console |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Apache-HBase commented on pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2451:
URL: https://github.com/apache/hbase/pull/2451#issuecomment-701129348


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   1m  2s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any @author tags.  |
   ||| _ HBASE-18070 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   3m 38s |  HBASE-18070 passed  |
   | +1 :green_heart: |  checkstyle  |   1m  8s |  HBASE-18070 passed  |
   | +1 :green_heart: |  spotbugs  |   2m 20s |  HBASE-18070 passed  |
   | -0 :warning: |  patch  |   2m 29s |  Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   3m 39s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   1m  6s |  hbase-server: The patch generated 0 new + 42 unchanged - 11 fixed = 42 total (was 53)  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace issues.  |
   | +1 :green_heart: |  hadoopcheck  |  11m 57s |  Patch does not cause any errors with Hadoop 3.1.2 3.2.1.  |
   | +1 :green_heart: |  spotbugs  |   2m 25s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 13s |  The patch does not generate ASF License warnings.  |
   |  |   |  34m 53s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/4/artifact/yetus-general-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/2451 |
   | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
   | uname | Linux 59e135895f9e 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | HBASE-18070 / 03e8e5b517 |
   | Max. process+thread count | 94 (vs. ulimit of 30000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/4/console |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] saintstack commented on a change in pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…

Posted by GitBox <gi...@apache.org>.
saintstack commented on a change in pull request #2451:
URL: https://github.com/apache/hbase/pull/2451#discussion_r498900957



##########
File path: hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestMetaRegionReplicaReplicationEndpoint.java
##########
@@ -0,0 +1,334 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.replication.regionserver;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertNotNull;
+import static org.junit.Assert.assertTrue;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Objects;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.CellScanner;
+import org.apache.hadoop.hbase.CellUtil;
+import org.apache.hadoop.hbase.ClientMetaTableAccessor;
+import org.apache.hadoop.hbase.HBaseClassTestRule;
+import org.apache.hadoop.hbase.HBaseTestingUtility;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.HRegionLocation;
+import org.apache.hadoop.hbase.MetaTableAccessor;
+import org.apache.hadoop.hbase.MiniHBaseCluster;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.Waiter;
+import org.apache.hadoop.hbase.client.RegionLocator;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.regionserver.HRegion;
+import org.apache.hadoop.hbase.regionserver.HRegionServer;
+import org.apache.hadoop.hbase.regionserver.Region;
+import org.apache.hadoop.hbase.regionserver.RegionScanner;
+import org.apache.hadoop.hbase.testclassification.FlakeyTests;
+import org.apache.hadoop.hbase.testclassification.LargeTests;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.util.ServerRegionReplicaUtil;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.rules.TestName;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Tests RegionReplicaReplicationEndpoint class for hbase:meta by setting up region replicas and
+ * verifying async wal replication replays the edits to the secondary region in various scenarios.
+ * @see TestRegionReplicaReplicationEndpoint
+ */
+@Category({FlakeyTests.class, LargeTests.class})

Review comment:
       Copy/Paste. Let me fix. Thanks.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Apache-HBase commented on pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2451:
URL: https://github.com/apache/hbase/pull/2451#issuecomment-701117900


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   1m 22s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ HBASE-18070 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   4m 45s |  HBASE-18070 passed  |
   | +1 :green_heart: |  compile  |   1m 12s |  HBASE-18070 passed  |
   | +1 :green_heart: |  shadedjars  |   7m 11s |  branch has no errors when building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 48s |  hbase-server in HBASE-18070 failed.  |
   | -0 :warning: |  patch  |   8m 12s |  Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   4m 23s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 14s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 14s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   7m 12s |  patch has no errors when building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 45s |  hbase-server in the patch failed.  |
   ||| _ Other Tests _ |
   | -1 :x: |  unit  | 148m 14s |  hbase-server in the patch failed.  |
   |  |   | 179m  6s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/2451 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux bc9c4691394f 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | HBASE-18070 / 03e8e5b517 |
   | Default Java | 2020-01-14 |
   | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
   | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
   | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
   |  Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/3/testReport/ |
   | Max. process+thread count | 4123 (vs. ulimit of 30000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/3/console |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] Apache-HBase commented on pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2451:
URL: https://github.com/apache/hbase/pull/2451#issuecomment-701208280


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   1m 33s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  4s |  Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ HBASE-18070 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   6m 20s |  HBASE-18070 passed  |
   | +1 :green_heart: |  compile  |   1m 52s |  HBASE-18070 passed  |
   | +1 :green_heart: |  shadedjars  |   9m 20s |  branch has no errors when building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 52s |  hbase-server in HBASE-18070 failed.  |
   | -0 :warning: |  patch  |  10m 24s |  Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m 16s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 34s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 34s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   8m 56s |  patch has no errors when building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 56s |  hbase-server in the patch failed.  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  | 248m 17s |  hbase-server in the patch passed.  |
   |  |   | 287m 32s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/2451 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux a60eeb66e73b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | HBASE-18070 / 03e8e5b517 |
   | Default Java | 2020-01-14 |
   | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
   | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
   |  Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/4/testReport/ |
   | Max. process+thread count | 2970 (vs. ulimit of 30000) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/4/console |
   | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hbase] huaxiangsun commented on pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…

Posted by GitBox <gi...@apache.org>.
huaxiangsun commented on pull request #2451:
URL: https://github.com/apache/hbase/pull/2451#issuecomment-702788348


   > I hear you.
   > 
   > But take a look at the calls to storage. Most calls to queueStorage bypass the ReplicationSource instance and the call goes direct to ReplicationSourceManager and its queue storage. What we pass on init here is not used by ReplicationSource instances.
   > 
   > Let me remove it. Passing it gives the wrong impression.
   
   Great! I also confused a bit as I thought most of queueStorage access is from Source.
   > 
   > I removed passing queueStorage to ReplicationSource#init. It is never used. I suppose we could go the other way and refactor so queueStorage access always goes via a ReplicationSource instance. This would be a big change. Even then it looks like there are usages of queueStorage that bypass ReplicationStorage instance -- which is fine, but it just makes the model of going via ReplicationStorage to do queueStorage incomplete/unclean. Opinions appreciated.
   > 
   > I took a look at refactoring so we created queueStorage in ReplicationSourceManager Constructor but thereafter, all access to queueStorage had to go via a replicationSource instance. The transition is easy in perhaps 50% of cases. In other contexts, we don't have a replicationSource but just a peerId (could see if could pull ReplicationSource if only a peerId), and then in other places there is no replicationSource instance around. A refactor here is probably warranted but a follow-on AND given that the new CatalogReplicationSource is not party to most of these accesses, the refactor is for sure off-topic. I can file a follow-on... Thanks for the suggestion Huaxiang
   
   Thanks for looking into the details! Yeah, we can do refactoring later. 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org