You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2020/06/02 03:21:35 UTC
[GitHub] [hbase] sguggilam opened a new pull request #1828: HBASE-24446 Use EnvironmentEdgeManager to compute clock skew in Master
sguggilam opened a new pull request #1828:
URL: https://github.com/apache/hbase/pull/1828
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1828: HBASE-24446 Use EnvironmentEdgeManager to compute clock skew in Master
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1828:
URL: https://github.com/apache/hbase/pull/1828#issuecomment-637257780
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 2m 55s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 4s | master passed |
| +1 :green_heart: | checkstyle | 1m 11s | master passed |
| +1 :green_heart: | spotbugs | 2m 6s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 45s | the patch passed |
| -0 :warning: | checkstyle | 1m 10s | hbase-server: The patch generated 4 new + 15 unchanged - 0 fixed = 19 total (was 15) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 12m 13s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. |
| +1 :green_heart: | spotbugs | 2m 15s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. |
| | | 37m 31s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1828 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux 5f8850c79c28 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / bad2d4e409 |
| checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| Max. process+thread count | 84 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/1/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1828: HBASE-24446 Use EnvironmentEdgeManager to compute clock skew in Master
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1828:
URL: https://github.com/apache/hbase/pull/1828#issuecomment-639159863
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 52s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 47s | master passed |
| +1 :green_heart: | checkstyle | 1m 7s | master passed |
| +1 :green_heart: | spotbugs | 2m 3s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 28s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 4s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 11m 44s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. |
| +1 :green_heart: | spotbugs | 2m 38s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. |
| | | 37m 1s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/4/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1828 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux 531f64d09c09 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 61a7468df7 |
| Max. process+thread count | 94 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/4/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1828: HBASE-24446 Use EnvironmentEdgeManager to compute clock skew in Master
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1828:
URL: https://github.com/apache/hbase/pull/1828#issuecomment-637828445
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 11s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 49s | master passed |
| +1 :green_heart: | compile | 0m 57s | master passed |
| +1 :green_heart: | shadedjars | 6m 11s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 38s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 53s | the patch passed |
| +1 :green_heart: | compile | 0m 57s | the patch passed |
| +1 :green_heart: | javac | 0m 57s | the patch passed |
| +1 :green_heart: | shadedjars | 6m 12s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 36s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 207m 37s | hbase-server in the patch passed. |
| | | 233m 40s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1828 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux bc783d7ff429 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / a8b3cad6c5 |
| Default Java | 1.8.0_232 |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/3/testReport/ |
| Max. process+thread count | 2927 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/3/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #1828: HBASE-24446 Use EnvironmentEdgeManager to compute clock skew in Master
Posted by GitBox <gi...@apache.org>.
virajjasani commented on a change in pull request #1828:
URL: https://github.com/apache/hbase/pull/1828#discussion_r434062695
##########
File path: hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerReportForDuty.java
##########
@@ -223,6 +226,41 @@ public void testReportForDutyWithMasterChange() throws Exception {
}
+ /**
+ * Tests region sever reportForDuty with manual environment edge
+ */
+ @Test(timeout = 60000)
Review comment:
Yeah, this looks good 👍
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java
##########
@@ -233,13 +234,19 @@ public void run() {
private int numberOfAttemptsSoFar = 0;
private long maxWaitTime = -1;
+ private final long rsRpcRetryInterval;
+ public static final String RS_RPC_RETRY_INTERVAL_CONF_KEY =
Review comment:
Can be `private`?
##########
File path: hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerReportForDuty.java
##########
@@ -223,6 +226,41 @@ public void testReportForDutyWithMasterChange() throws Exception {
}
+ /**
+ * Tests region sever reportForDuty with manual environment edge
+ */
+ @Test(timeout = 60000)
+ public void testReportForDutyWithEnvironmentEdge() throws Exception {
+ // Start a master and wait for it to become the active/primary master.
+ // Use a random unique port
+ cluster.getConfiguration().setInt(HConstants.MASTER_PORT, HBaseTestingUtility.randomFreePort());
+ // Set the dispatch and retry delay to 0 since we want the rpc request to be sent immediately
+ cluster.getConfiguration().setInt("hbase.procedure.remote.dispatcher.delay.msec", 0);
+ cluster.getConfiguration().setInt("hbase.regionserver.rpc.retry.interval", 0);
+
+ // master has a rs. defaultMinToStart = 2
+ boolean tablesOnMaster = LoadBalancer.isTablesOnMaster(testUtil.getConfiguration());
+ cluster.getConfiguration().setInt(ServerManager.WAIT_ON_REGIONSERVERS_MINTOSTART,
+ tablesOnMaster ? 2 : 1);
+ cluster.getConfiguration().setInt(ServerManager.WAIT_ON_REGIONSERVERS_MAXTOSTART,
+ tablesOnMaster ? 2 : 1);
+
+ EnvironmentEdge previousEdge = EnvironmentEdgeManager.getDelegate();
+ // Inject manual environment edge for clock skew computation between RS and master
+ ManualEnvironmentEdge edge = new ManualEnvironmentEdge();
+ EnvironmentEdgeManager.injectEdge(edge);
+ master = cluster.addMaster();
+ rs = cluster.addRegionServer();
+ LOG.debug("Starting master: " + master.getMaster().getServerName());
+ master.start();
+ rs.start();
+
+ waitForClusterOnline(master);
+
+ // Reset the manual environment edge
+ EnvironmentEdgeManager.injectEdge(previousEdge);
Review comment:
This reset is done so that other tests don't get our manual edge right?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1828: HBASE-24446 Use EnvironmentEdgeManager to compute clock skew in Master
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1828:
URL: https://github.com/apache/hbase/pull/1828#issuecomment-637429906
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 32s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 21s | master passed |
| +1 :green_heart: | compile | 0m 55s | master passed |
| +1 :green_heart: | shadedjars | 5m 36s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 35s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 25s | the patch passed |
| +1 :green_heart: | compile | 0m 54s | the patch passed |
| +1 :green_heart: | javac | 0m 54s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 33s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 35s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 135m 51s | hbase-server in the patch passed. |
| | | 159m 16s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1828 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 2773fd367e41 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / bad2d4e409 |
| Default Java | 1.8.0_232 |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/2/testReport/ |
| Max. process+thread count | 4739 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/2/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #1828: HBASE-24446 Use EnvironmentEdgeManager to compute clock skew in Master
Posted by GitBox <gi...@apache.org>.
Apache9 commented on a change in pull request #1828:
URL: https://github.com/apache/hbase/pull/1828#discussion_r435789124
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java
##########
@@ -233,13 +233,19 @@ public void run() {
private int numberOfAttemptsSoFar = 0;
private long maxWaitTime = -1;
+ private final long rsRpcRetryInterval;
Review comment:
This is not related to the title here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] apurtell commented on a change in pull request #1828: HBASE-24446 Use EnvironmentEdgeManager to compute clock skew in Master
Posted by GitBox <gi...@apache.org>.
apurtell commented on a change in pull request #1828:
URL: https://github.com/apache/hbase/pull/1828#discussion_r436053515
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java
##########
@@ -233,13 +233,19 @@ public void run() {
private int numberOfAttemptsSoFar = 0;
private long maxWaitTime = -1;
+ private final long rsRpcRetryInterval;
Review comment:
Agree, this doesn't fit the scope implied by the JIRA and PR description. Remove it here, submit as new issue/PR?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #1828: HBASE-24446 Use EnvironmentEdgeManager to compute clock skew in Master
Posted by GitBox <gi...@apache.org>.
virajjasani commented on a change in pull request #1828:
URL: https://github.com/apache/hbase/pull/1828#discussion_r433859189
##########
File path: hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerReportForDuty.java
##########
@@ -223,6 +226,41 @@ public void testReportForDutyWithMasterChange() throws Exception {
}
+ /**
+ * Tests region sever reportForDuty with manual environment edge
+ */
+ @Test(timeout = 60000)
Review comment:
timeout is not required here. This belongs to `LargeTests`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] sguggilam commented on pull request #1828: HBASE-24446 Use EnvironmentEdgeManager to compute clock skew in Master
Posted by GitBox <gi...@apache.org>.
sguggilam commented on pull request #1828:
URL: https://github.com/apache/hbase/pull/1828#issuecomment-638270416
@apurtell
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] sguggilam commented on a change in pull request #1828: HBASE-24446 Use EnvironmentEdgeManager to compute clock skew in Master
Posted by GitBox <gi...@apache.org>.
sguggilam commented on a change in pull request #1828:
URL: https://github.com/apache/hbase/pull/1828#discussion_r434072305
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java
##########
@@ -233,13 +234,19 @@ public void run() {
private int numberOfAttemptsSoFar = 0;
private long maxWaitTime = -1;
+ private final long rsRpcRetryInterval;
+ public static final String RS_RPC_RETRY_INTERVAL_CONF_KEY =
Review comment:
Done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1828: HBASE-24446 Use EnvironmentEdgeManager to compute clock skew in Master
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1828:
URL: https://github.com/apache/hbase/pull/1828#issuecomment-637467123
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 24s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 25s | master passed |
| +1 :green_heart: | compile | 1m 21s | master passed |
| +1 :green_heart: | shadedjars | 6m 35s | branch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 45s | hbase-server in master failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 43s | the patch passed |
| +1 :green_heart: | compile | 1m 21s | the patch passed |
| +1 :green_heart: | javac | 1m 21s | the patch passed |
| +1 :green_heart: | shadedjars | 6m 33s | patch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 42s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| -1 :x: | unit | 205m 4s | hbase-server in the patch failed. |
| | | 235m 41s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1828 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 70301b9afe1e 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / bad2d4e409 |
| Default Java | 2020-01-14 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/2/testReport/ |
| Max. process+thread count | 2880 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/2/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1828: HBASE-24446 Use EnvironmentEdgeManager to compute clock skew in Master
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1828:
URL: https://github.com/apache/hbase/pull/1828#issuecomment-637357953
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 30s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 46s | master passed |
| +1 :green_heart: | checkstyle | 1m 4s | master passed |
| +1 :green_heart: | spotbugs | 2m 40s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 10s | the patch passed |
| -0 :warning: | checkstyle | 1m 3s | hbase-server: The patch generated 4 new + 15 unchanged - 0 fixed = 19 total (was 15) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 11m 6s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. |
| +1 :green_heart: | spotbugs | 2m 4s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. |
| | | 33m 52s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/2/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1828 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux 02ece4f3a729 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / bad2d4e409 |
| checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| Max. process+thread count | 94 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/2/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] sguggilam commented on a change in pull request #1828: HBASE-24446 Use EnvironmentEdgeManager to compute clock skew in Master
Posted by GitBox <gi...@apache.org>.
sguggilam commented on a change in pull request #1828:
URL: https://github.com/apache/hbase/pull/1828#discussion_r434049475
##########
File path: hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerReportForDuty.java
##########
@@ -223,6 +226,41 @@ public void testReportForDutyWithMasterChange() throws Exception {
}
+ /**
+ * Tests region sever reportForDuty with manual environment edge
+ */
+ @Test(timeout = 60000)
Review comment:
The change basically allows you to set configuration parameter for the retry of RPC to go through successfully in the case where is some custom delegate for the environment edge. We can use this even for any tests in future where there is manual environment edge injected. Let me know your thoughts on this
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1828: HBASE-24446 Use EnvironmentEdgeManager to compute clock skew in Master
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1828:
URL: https://github.com/apache/hbase/pull/1828#issuecomment-637308250
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 32s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 26s | master passed |
| +1 :green_heart: | compile | 1m 5s | master passed |
| +1 :green_heart: | shadedjars | 5m 54s | branch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 42s | hbase-server in master failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 9s | the patch passed |
| +1 :green_heart: | compile | 1m 6s | the patch passed |
| +1 :green_heart: | javac | 1m 6s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 57s | patch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 51s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| -1 :x: | unit | 158m 39s | hbase-server in the patch failed. |
| | | 185m 34s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1828 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux ea4e4a7359dc 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / bad2d4e409 |
| Default Java | 2020-01-14 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/1/testReport/ |
| Max. process+thread count | 4145 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/1/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] sguggilam commented on a change in pull request #1828: HBASE-24446 Use EnvironmentEdgeManager to compute clock skew in Master
Posted by GitBox <gi...@apache.org>.
sguggilam commented on a change in pull request #1828:
URL: https://github.com/apache/hbase/pull/1828#discussion_r436061999
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java
##########
@@ -233,13 +233,19 @@ public void run() {
private int numberOfAttemptsSoFar = 0;
private long maxWaitTime = -1;
+ private final long rsRpcRetryInterval;
Review comment:
Sure @apurtell , created a new JIRA to track this : https://issues.apache.org/jira/browse/HBASE-24511
Will close this PR for now and send two follow up PRs for the newly created PR followed by the PR for this JIRA
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1828: HBASE-24446 Use EnvironmentEdgeManager to compute clock skew in Master
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1828:
URL: https://github.com/apache/hbase/pull/1828#issuecomment-637309319
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 32s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 51s | master passed |
| +1 :green_heart: | compile | 0m 57s | master passed |
| +1 :green_heart: | shadedjars | 5m 35s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 38s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 30s | the patch passed |
| +1 :green_heart: | compile | 0m 54s | the patch passed |
| +1 :green_heart: | javac | 0m 54s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 36s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 37s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 164m 9s | hbase-server in the patch failed. |
| | | 188m 28s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1828 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux de0b7ef1b54c 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / bad2d4e409 |
| Default Java | 1.8.0_232 |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/1/testReport/ |
| Max. process+thread count | 3885 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/1/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1828: HBASE-24446 Use EnvironmentEdgeManager to compute clock skew in Master
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1828:
URL: https://github.com/apache/hbase/pull/1828#issuecomment-639214219
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 19s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 53s | master passed |
| +1 :green_heart: | compile | 1m 10s | master passed |
| +1 :green_heart: | shadedjars | 6m 19s | branch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 43s | hbase-server in master failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 33s | the patch passed |
| +1 :green_heart: | compile | 1m 9s | the patch passed |
| +1 :green_heart: | javac | 1m 9s | the patch passed |
| +1 :green_heart: | shadedjars | 6m 23s | patch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 41s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 195m 25s | hbase-server in the patch passed. |
| | | 224m 33s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1828 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux b6913c50e92b 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 61a7468df7 |
| Default Java | 2020-01-14 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/4/testReport/ |
| Max. process+thread count | 3556 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/4/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] sguggilam closed pull request #1828: HBASE-24446 Use EnvironmentEdgeManager to compute clock skew in Master
Posted by GitBox <gi...@apache.org>.
sguggilam closed pull request #1828:
URL: https://github.com/apache/hbase/pull/1828
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] sguggilam commented on a change in pull request #1828: HBASE-24446 Use EnvironmentEdgeManager to compute clock skew in Master
Posted by GitBox <gi...@apache.org>.
sguggilam commented on a change in pull request #1828:
URL: https://github.com/apache/hbase/pull/1828#discussion_r434041771
##########
File path: hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerReportForDuty.java
##########
@@ -223,6 +226,41 @@ public void testReportForDutyWithMasterChange() throws Exception {
}
+ /**
+ * Tests region sever reportForDuty with manual environment edge
+ */
+ @Test(timeout = 60000)
Review comment:
Makes sense, I will remove the timeout
I have seen TestClockSkewDetection, the tests basically mocks a server and just tests the registration between RS and master. The current test in TestRegionServerReportForDuty also checks if the master comes online and finishes the initialization without any issue when a manual edge is inserted which is where i could see the gap in RSProcedureDispatcher and made the change. Please let me know if you think otherwise
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1828: HBASE-24446 Use EnvironmentEdgeManager to compute clock skew in Master
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1828:
URL: https://github.com/apache/hbase/pull/1828#issuecomment-637806312
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 40s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 54s | master passed |
| +1 :green_heart: | compile | 1m 30s | master passed |
| +1 :green_heart: | shadedjars | 8m 12s | branch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 53s | hbase-server in master failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 39s | the patch passed |
| +1 :green_heart: | compile | 1m 31s | the patch passed |
| +1 :green_heart: | javac | 1m 31s | the patch passed |
| +1 :green_heart: | shadedjars | 8m 22s | patch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 51s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 145m 14s | hbase-server in the patch passed. |
| | | 180m 48s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1828 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 071a1aee1190 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / a8b3cad6c5 |
| Default Java | 2020-01-14 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/3/testReport/ |
| Max. process+thread count | 3622 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/3/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] sguggilam commented on a change in pull request #1828: HBASE-24446 Use EnvironmentEdgeManager to compute clock skew in Master
Posted by GitBox <gi...@apache.org>.
sguggilam commented on a change in pull request #1828:
URL: https://github.com/apache/hbase/pull/1828#discussion_r436018339
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java
##########
@@ -233,13 +233,19 @@ public void run() {
private int numberOfAttemptsSoFar = 0;
private long maxWaitTime = -1;
+ private final long rsRpcRetryInterval;
Review comment:
An example would be Master trying to open the meta region even before the RS is online, In this case the retry of opening the meta region would never happen in the case of injected environment edge and master would never finish the initialization
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] sguggilam commented on a change in pull request #1828: HBASE-24446 Use EnvironmentEdgeManager to compute clock skew in Master
Posted by GitBox <gi...@apache.org>.
sguggilam commented on a change in pull request #1828:
URL: https://github.com/apache/hbase/pull/1828#discussion_r434073102
##########
File path: hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerReportForDuty.java
##########
@@ -223,6 +226,41 @@ public void testReportForDutyWithMasterChange() throws Exception {
}
+ /**
+ * Tests region sever reportForDuty with manual environment edge
+ */
+ @Test(timeout = 60000)
+ public void testReportForDutyWithEnvironmentEdge() throws Exception {
+ // Start a master and wait for it to become the active/primary master.
+ // Use a random unique port
+ cluster.getConfiguration().setInt(HConstants.MASTER_PORT, HBaseTestingUtility.randomFreePort());
+ // Set the dispatch and retry delay to 0 since we want the rpc request to be sent immediately
+ cluster.getConfiguration().setInt("hbase.procedure.remote.dispatcher.delay.msec", 0);
+ cluster.getConfiguration().setInt("hbase.regionserver.rpc.retry.interval", 0);
+
+ // master has a rs. defaultMinToStart = 2
+ boolean tablesOnMaster = LoadBalancer.isTablesOnMaster(testUtil.getConfiguration());
+ cluster.getConfiguration().setInt(ServerManager.WAIT_ON_REGIONSERVERS_MINTOSTART,
+ tablesOnMaster ? 2 : 1);
+ cluster.getConfiguration().setInt(ServerManager.WAIT_ON_REGIONSERVERS_MAXTOSTART,
+ tablesOnMaster ? 2 : 1);
+
+ EnvironmentEdge previousEdge = EnvironmentEdgeManager.getDelegate();
+ // Inject manual environment edge for clock skew computation between RS and master
+ ManualEnvironmentEdge edge = new ManualEnvironmentEdge();
+ EnvironmentEdgeManager.injectEdge(edge);
+ master = cluster.addMaster();
+ rs = cluster.addRegionServer();
+ LOG.debug("Starting master: " + master.getMaster().getServerName());
+ master.start();
+ rs.start();
+
+ waitForClusterOnline(master);
+
+ // Reset the manual environment edge
+ EnvironmentEdgeManager.injectEdge(previousEdge);
Review comment:
Yes thats right but after a closer look , I see that we are destroying the cluster after each test case anyways. Have reverted this change
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] sguggilam commented on a change in pull request #1828: HBASE-24446 Use EnvironmentEdgeManager to compute clock skew in Master
Posted by GitBox <gi...@apache.org>.
sguggilam commented on a change in pull request #1828:
URL: https://github.com/apache/hbase/pull/1828#discussion_r436014652
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java
##########
@@ -233,13 +233,19 @@ public void run() {
private int numberOfAttemptsSoFar = 0;
private long maxWaitTime = -1;
+ private final long rsRpcRetryInterval;
Review comment:
@Apache9 This is useful for the cases where there is an injected environment edge and when the first RS RPC request fails causing it to go to the retry block.
In the absence of this , the default timeout would be set to 100 ms and the DelayedUtil class is meant to execute the retry after 100 ms. However as per the getRemainingTime() logic here (https://github.com/apache/hbase/blob/5b01e613fbbb92e243e99a1d199b4ffbb21ed2d9/hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/util/DelayedUtil.java#L94), the equation would be evaluated to
EnvironmentEdgeManager.currentTime() >= EnvironmentEdgeManager.currentTime() + 100 which would never get evaluated to true in case of injected edge and retry never happens. Hence this config helps you to override it to 0 in cases we want to test with a manual injected environment edge
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1828: HBASE-24446 Use EnvironmentEdgeManager to compute clock skew in Master
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1828:
URL: https://github.com/apache/hbase/pull/1828#issuecomment-637733790
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 30s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 42s | master passed |
| +1 :green_heart: | checkstyle | 1m 7s | master passed |
| +1 :green_heart: | spotbugs | 1m 58s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 20s | the patch passed |
| -0 :warning: | checkstyle | 1m 4s | hbase-server: The patch generated 4 new + 15 unchanged - 0 fixed = 19 total (was 15) |
| +1 :green_heart: | whitespace | 0m 1s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 11m 3s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. |
| +1 :green_heart: | spotbugs | 2m 7s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. |
| | | 31m 58s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/3/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1828 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux db3b04fdb75f 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / a8b3cad6c5 |
| checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| Max. process+thread count | 94 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/3/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1828: HBASE-24446 Use EnvironmentEdgeManager to compute clock skew in Master
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1828:
URL: https://github.com/apache/hbase/pull/1828#issuecomment-639196962
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 52s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 34s | master passed |
| +1 :green_heart: | compile | 0m 55s | master passed |
| +1 :green_heart: | shadedjars | 5m 31s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 38s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 30s | the patch passed |
| +1 :green_heart: | compile | 0m 55s | the patch passed |
| +1 :green_heart: | javac | 0m 55s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 28s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 36s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 134m 51s | hbase-server in the patch passed. |
| | | 159m 59s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1828 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 22e1443806c8 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 61a7468df7 |
| Default Java | 1.8.0_232 |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/4/testReport/ |
| Max. process+thread count | 4713 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1828/4/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] sguggilam commented on a change in pull request #1828: HBASE-24446 Use EnvironmentEdgeManager to compute clock skew in Master
Posted by GitBox <gi...@apache.org>.
sguggilam commented on a change in pull request #1828:
URL: https://github.com/apache/hbase/pull/1828#discussion_r434041771
##########
File path: hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerReportForDuty.java
##########
@@ -223,6 +226,41 @@ public void testReportForDutyWithMasterChange() throws Exception {
}
+ /**
+ * Tests region sever reportForDuty with manual environment edge
+ */
+ @Test(timeout = 60000)
Review comment:
@virajjasani Makes sense, I will remove the timeout
I have seen TestClockSkewDetection, the tests basically mocks a server and just tests the registration between RS and master. The current test in TestRegionServerReportForDuty also checks if the master comes online and finishes the initialization without any issue when a manual edge is inserted which is where i could see the gap in RSProcedureDispatcher and made the change. Please let me know if you think otherwise
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org