You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2020/12/02 06:15:54 UTC
[GitHub] [hbase] saintstack opened a new pull request #2731: HBASE-25349 [Flakey Tests] branch-2 TestRefreshRecoveredReplication.t…
saintstack opened a new pull request #2731:
URL: https://github.com/apache/hbase/pull/2731
…estReplicationRefreshSource:141 Waiting timed out after [60,000] msec
Start the check for recovered queue presence earlier.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] ndimiduk commented on a change in pull request #2731: HBASE-25349 [Flakey Tests] branch-2 TestRefreshRecoveredReplication.t…
Posted by GitBox <gi...@apache.org>.
ndimiduk commented on a change in pull request #2731:
URL: https://github.com/apache/hbase/pull/2731#discussion_r534357346
##########
File path: hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestRefreshRecoveredReplication.java
##########
@@ -121,22 +127,25 @@ public void testReplicationRefreshSource() throws Exception {
table1.put(new Put(r).addColumn(famName, famName, r));
}
- // kill rs holding table region
- Optional<RegionServerThread> server = UTIL1.getMiniHBaseCluster().getLiveRegionServerThreads()
- .stream()
+ // Kill rs holding table region. There are only TWO servers. We depend on it.
+ List<RegionServerThread> rss = UTIL1.getMiniHBaseCluster().getLiveRegionServerThreads();
+ assertEquals(2, rss.size());
+ Optional<RegionServerThread> server = rss.stream()
.filter(rst -> CollectionUtils.isNotEmpty(rst.getRegionServer().getRegions(tablename)))
.findAny();
Assert.assertTrue(server.isPresent());
+ HRegionServer otherServer = rss.get(0).getRegionServer() == server.get().getRegionServer()?
+ rss.get(1).getRegionServer(): rss.get(0).getRegionServer();
server.get().getRegionServer().abort("stopping for test");
+ // waiting for recovered peer to appear.
+ Replication replication = (Replication)otherServer.getReplicationSourceService();
+ UTIL1.waitFor(60000, () -> !replication.getReplicationManager().getOldSources().isEmpty());
+ // Wait on only one server being up.
UTIL1.waitFor(60000, () ->
- UTIL1.getMiniHBaseCluster().getLiveRegionServerThreads().size() == NUM_SLAVES1 - 1);
+ // Have to go back to source here because getLiveRegionServerThreads makes new array each time
+ UTIL1.getMiniHBaseCluster().getLiveRegionServerThreads().size() == NUM_SLAVES1 - 1);
Review comment:
Instead of waiting on strict equality by count, can you wait for `getLiveRegionServerThreads()` to contain only `otherServer` by identity?
##########
File path: hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestRefreshRecoveredReplication.java
##########
@@ -75,6 +80,7 @@
@BeforeClass
public static void setUpBeforeClass() throws Exception {
+ // NUM_SLAVES1 is presumed 2 in below.
Review comment:
Can `NUM_SLAVES1` be made final, and referenced wherever it's presumed to be 2? Maybe rename it if you're in there making that change...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #2731: HBASE-25349 [Flakey Tests] branch-2 TestRefreshRecoveredReplication.t…
Posted by GitBox <gi...@apache.org>.
saintstack commented on a change in pull request #2731:
URL: https://github.com/apache/hbase/pull/2731#discussion_r534367135
##########
File path: hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestRefreshRecoveredReplication.java
##########
@@ -75,6 +80,7 @@
@BeforeClass
public static void setUpBeforeClass() throws Exception {
+ // NUM_SLAVES1 is presumed 2 in below.
Review comment:
The NUM_SLAVES1 comes from base class.
This class and the subclasses need major refactoring. I avoid it in here trying to keep change minimal.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] saintstack merged pull request #2731: HBASE-25349 [Flakey Tests] branch-2 TestRefreshRecoveredReplication.t…
Posted by GitBox <gi...@apache.org>.
saintstack merged pull request #2731:
URL: https://github.com/apache/hbase/pull/2731
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2731: HBASE-25349 [Flakey Tests] branch-2 TestRefreshRecoveredReplication.t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2731:
URL: https://github.com/apache/hbase/pull/2731#issuecomment-737033253
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 20s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ branch-2 Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 1s | branch-2 passed |
| +1 :green_heart: | checkstyle | 1m 16s | branch-2 passed |
| +1 :green_heart: | spotbugs | 2m 6s | branch-2 passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 34s | the patch passed |
| -0 :warning: | checkstyle | 1m 14s | hbase-server: The patch generated 3 new + 2 unchanged - 0 fixed = 5 total (was 2) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 12m 54s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. |
| +1 :green_heart: | spotbugs | 2m 15s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. |
| | | 36m 45s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2731/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2731 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux d746a0b5bbed 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2 / 745a3a9ab7 |
| checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2731/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| Max. process+thread count | 84 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2731/1/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #2731: HBASE-25349 [Flakey Tests] branch-2 TestRefreshRecoveredReplication.t…
Posted by GitBox <gi...@apache.org>.
saintstack commented on a change in pull request #2731:
URL: https://github.com/apache/hbase/pull/2731#discussion_r534367571
##########
File path: hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestRefreshRecoveredReplication.java
##########
@@ -121,22 +127,25 @@ public void testReplicationRefreshSource() throws Exception {
table1.put(new Put(r).addColumn(famName, famName, r));
}
- // kill rs holding table region
- Optional<RegionServerThread> server = UTIL1.getMiniHBaseCluster().getLiveRegionServerThreads()
- .stream()
+ // Kill rs holding table region. There are only TWO servers. We depend on it.
+ List<RegionServerThread> rss = UTIL1.getMiniHBaseCluster().getLiveRegionServerThreads();
+ assertEquals(2, rss.size());
+ Optional<RegionServerThread> server = rss.stream()
.filter(rst -> CollectionUtils.isNotEmpty(rst.getRegionServer().getRegions(tablename)))
.findAny();
Assert.assertTrue(server.isPresent());
+ HRegionServer otherServer = rss.get(0).getRegionServer() == server.get().getRegionServer()?
+ rss.get(1).getRegionServer(): rss.get(0).getRegionServer();
server.get().getRegionServer().abort("stopping for test");
+ // waiting for recovered peer to appear.
+ Replication replication = (Replication)otherServer.getReplicationSourceService();
+ UTIL1.waitFor(60000, () -> !replication.getReplicationManager().getOldSources().isEmpty());
+ // Wait on only one server being up.
UTIL1.waitFor(60000, () ->
- UTIL1.getMiniHBaseCluster().getLiveRegionServerThreads().size() == NUM_SLAVES1 - 1);
+ // Have to go back to source here because getLiveRegionServerThreads makes new array each time
+ UTIL1.getMiniHBaseCluster().getLiveRegionServerThreads().size() == NUM_SLAVES1 - 1);
Review comment:
I could but skipping as nit if you don't mind.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2731: HBASE-25349 [Flakey Tests] branch-2 TestRefreshRecoveredReplication.t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2731:
URL: https://github.com/apache/hbase/pull/2731#issuecomment-737090532
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 35s | Docker mode activated. |
| -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ branch-2 Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 17s | branch-2 passed |
| +1 :green_heart: | compile | 1m 6s | branch-2 passed |
| +1 :green_heart: | shadedjars | 6m 37s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 43s | branch-2 passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 58s | the patch passed |
| +1 :green_heart: | compile | 1m 5s | the patch passed |
| +1 :green_heart: | javac | 1m 5s | the patch passed |
| +1 :green_heart: | shadedjars | 6m 37s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 41s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 134m 57s | hbase-server in the patch passed. |
| | | 162m 48s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2731/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2731 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 06ae0f2ec6da 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2 / 745a3a9ab7 |
| Default Java | AdoptOpenJDK-11.0.6+10 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2731/1/testReport/ |
| Max. process+thread count | 4296 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2731/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2731: HBASE-25349 [Flakey Tests] branch-2 TestRefreshRecoveredReplication.t…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2731:
URL: https://github.com/apache/hbase/pull/2731#issuecomment-737144329
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 2m 17s | Docker mode activated. |
| -0 :warning: | yetus | 0m 8s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ branch-2 Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 33s | branch-2 passed |
| +1 :green_heart: | compile | 1m 20s | branch-2 passed |
| +1 :green_heart: | shadedjars | 8m 24s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 58s | branch-2 passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 22s | the patch passed |
| +1 :green_heart: | compile | 1m 15s | the patch passed |
| +1 :green_heart: | javac | 1m 15s | the patch passed |
| +1 :green_heart: | shadedjars | 6m 52s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 37s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 225m 21s | hbase-server in the patch passed. |
| | | 260m 6s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2731/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2731 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 8d549924fead 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2 / 745a3a9ab7 |
| Default Java | AdoptOpenJDK-1.8.0_232-b09 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2731/1/testReport/ |
| Max. process+thread count | 2488 (vs. ulimit of 12500) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2731/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org