You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2021/03/19 01:42:18 UTC
[GitHub] [hadoop] hdaikoku opened a new pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.
hdaikoku opened a new pull request #2787:
URL: https://github.com/apache/hadoop/pull/2787
https://issues.apache.org/jira/browse/HDFS-15900
## NOTICE
Please create an issue in ASF JIRA before opening a pull request,
and you need to set the title of the pull request which starts with
the corresponding JIRA issue number. (e.g. HADOOP-XXXXX. Fix a typo in YYY.)
For more details, please see https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] goiri commented on a change in pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.
Posted by GitBox <gi...@apache.org>.
goiri commented on a change in pull request #2787:
URL: https://github.com/apache/hadoop/pull/2787#discussion_r598843870
##########
File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/store/TestStateStoreMembershipState.java
##########
@@ -473,6 +478,53 @@ public void testRegistrationExpiredAndDeletion()
}, 100, 3000);
}
+ @Test
+ public void testNamespaceInfoWithUnavailableNameNodeRegistration() throws IOException {
+ // Populate the state store with one ACTIVE NameNode entry and one UNAVAILABLE NameNode entry
+ // 1) ns0:nn0 - ACTIVE
+ // 2) ns0:nn1 - UNAVAILABLE
+ List<MembershipState> registrationList = new ArrayList<>();
+ String router = ROUTERS[0];
+ String ns = NAMESERVICES[0];
+ String rpcAddress = "testrpcaddress";
+ String serviceAddress = "testserviceaddress";
+ String lifelineAddress = "testlifelineaddress";
+ String blockPoolId = "testblockpool";
+ String clusterId = "testcluster";
+ String webScheme = "http";
+ String webAddress = "testwebaddress";
+ boolean safemode = false;
+
+ MembershipState record = MembershipState.newInstance(
+ router, ns, NAMENODES[0], clusterId, blockPoolId,
+ rpcAddress, serviceAddress, lifelineAddress, webScheme,
+ webAddress, FederationNamenodeServiceState.ACTIVE, safemode);
+ registrationList.add(record);
+
+ // Set empty clusterId and blockPoolId for UNAVAILABLE NameNode
+ record = MembershipState.newInstance(
+ router, ns, NAMENODES[1], "", "",
+ rpcAddress, serviceAddress, lifelineAddress, webScheme,
+ webAddress, FederationNamenodeServiceState.UNAVAILABLE, safemode);
+ registrationList.add(record);
+
+ registerAndLoadRegistrations(registrationList);
+
+ GetNamespaceInfoRequest request = GetNamespaceInfoRequest.newInstance();
+ GetNamespaceInfoResponse response = membershipStore.getNamespaceInfo(request);
+ Set<FederationNamespaceInfo> namespaces = response.getNamespaceInfo();
+
+ // Verify only one namespace is registered
+ assertThat(namespaces).hasSize(1);
Review comment:
Why is this better than assertEquals(1, namespaces.size())?
Is the error message friendlier?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2787:
URL: https://github.com/apache/hadoop/pull/2787#issuecomment-805663042
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 37s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 2 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 39s | | trunk passed |
| +1 :green_heart: | compile | 0m 42s | | trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 37s | | trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 28s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 43s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 42s | | trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 55s | | trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 14s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 11s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 33s | | the patch passed |
| +1 :green_heart: | compile | 0m 34s | | the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 35s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 0m 29s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 17s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 32s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 30s | | the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 47s | | the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 16s | | the patch passed |
| +1 :green_heart: | shadedclient | 13m 59s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| -1 :x: | unit | 17m 46s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2787/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt) | hadoop-hdfs-rbf in the patch passed. |
| +1 :green_heart: | asflicense | 0m 35s | | The patch does not generate ASF License warnings. |
| | | 91m 49s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.server.federation.router.TestRouterRpcMultiDestination |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2787/4/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2787 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 213e3e38c69b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 6f9fc3eca23e2a1cba7238ff49a8d4ecd7f9c0ef |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2787/4/testReport/ |
| Max. process+thread count | 2184 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2787/4/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hdaikoku commented on pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.
Posted by GitBox <gi...@apache.org>.
hdaikoku commented on pull request #2787:
URL: https://github.com/apache/hadoop/pull/2787#issuecomment-805605184
@tasanuma I have fixed the style issues and rebased: 6f9fc3e
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hdaikoku commented on a change in pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.
Posted by GitBox <gi...@apache.org>.
hdaikoku commented on a change in pull request #2787:
URL: https://github.com/apache/hadoop/pull/2787#discussion_r599167124
##########
File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/store/TestStateStoreMembershipState.java
##########
@@ -473,6 +478,53 @@ public void testRegistrationExpiredAndDeletion()
}, 100, 3000);
}
+ @Test
+ public void testNamespaceInfoWithUnavailableNameNodeRegistration() throws IOException {
+ // Populate the state store with one ACTIVE NameNode entry and one UNAVAILABLE NameNode entry
+ // 1) ns0:nn0 - ACTIVE
+ // 2) ns0:nn1 - UNAVAILABLE
+ List<MembershipState> registrationList = new ArrayList<>();
+ String router = ROUTERS[0];
+ String ns = NAMESERVICES[0];
+ String rpcAddress = "testrpcaddress";
+ String serviceAddress = "testserviceaddress";
+ String lifelineAddress = "testlifelineaddress";
+ String blockPoolId = "testblockpool";
+ String clusterId = "testcluster";
+ String webScheme = "http";
+ String webAddress = "testwebaddress";
+ boolean safemode = false;
+
+ MembershipState record = MembershipState.newInstance(
+ router, ns, NAMENODES[0], clusterId, blockPoolId,
+ rpcAddress, serviceAddress, lifelineAddress, webScheme,
+ webAddress, FederationNamenodeServiceState.ACTIVE, safemode);
+ registrationList.add(record);
+
+ // Set empty clusterId and blockPoolId for UNAVAILABLE NameNode
+ record = MembershipState.newInstance(
+ router, ns, NAMENODES[1], "", "",
+ rpcAddress, serviceAddress, lifelineAddress, webScheme,
+ webAddress, FederationNamenodeServiceState.UNAVAILABLE, safemode);
+ registrationList.add(record);
+
+ registerAndLoadRegistrations(registrationList);
+
+ GetNamespaceInfoRequest request = GetNamespaceInfoRequest.newInstance();
+ GetNamespaceInfoResponse response = membershipStore.getNamespaceInfo(request);
+ Set<FederationNamespaceInfo> namespaces = response.getNamespaceInfo();
+
+ // Verify only one namespace is registered
+ assertThat(namespaces).hasSize(1);
+
+ // Verify the registered namespace has a valid pair of clusterId and blockPoolId derived from ACTIVE NameNode
+ for (FederationNamespaceInfo namespace : namespaces) {
Review comment:
Agreed. Modified to verify only the first element from the iterator: 04ab61c
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tasanuma commented on pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.
Posted by GitBox <gi...@apache.org>.
tasanuma commented on pull request #2787:
URL: https://github.com/apache/hadoop/pull/2787#issuecomment-804532574
@hdaikoku Could you fix the checkstyle warnings in the CI report?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hdaikoku commented on pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.
Posted by GitBox <gi...@apache.org>.
hdaikoku commented on pull request #2787:
URL: https://github.com/apache/hadoop/pull/2787#issuecomment-803768020
Needed to override `compareTo` as well and now the failed tests are ok: 2b776b2
I have confirmed locally that all the unit tests under hadoop-hdfs-rbf pass except the following three cases which I don't think are caused by this PR.
```
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] TestRouterFederationRename.testCounter:440->Object.wait:502->Object.wait:-2 ? TestTimedOut
[ERROR] TestRouterFederationRename.testSetup:145 ? Remote The directory /src cannot be...
[ERROR] TestRouterFederationRename.testSetup:145 ? Remote The directory /src cannot be...
[INFO]
[ERROR] Tests run: 781, Failures: 0, Errors: 3, Skipped: 2
[INFO]
[ERROR] There are test failures.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tasanuma commented on pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.
Posted by GitBox <gi...@apache.org>.
tasanuma commented on pull request #2787:
URL: https://github.com/apache/hadoop/pull/2787#issuecomment-804537584
The failed tests succeeded in my local environment and seem not to related.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hdaikoku commented on pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.
Posted by GitBox <gi...@apache.org>.
hdaikoku commented on pull request #2787:
URL: https://github.com/apache/hadoop/pull/2787#issuecomment-809044463
Thank you all for the review!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tasanuma commented on pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.
Posted by GitBox <gi...@apache.org>.
tasanuma commented on pull request #2787:
URL: https://github.com/apache/hadoop/pull/2787#issuecomment-804510471
+1, pending jenkins.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tasanuma merged pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.
Posted by GitBox <gi...@apache.org>.
tasanuma merged pull request #2787:
URL: https://github.com/apache/hadoop/pull/2787
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2787:
URL: https://github.com/apache/hadoop/pull/2787#issuecomment-804529045
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 34s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 2 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 53s | | trunk passed |
| +1 :green_heart: | compile | 0m 41s | | trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 38s | | trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 28s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 43s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 41s | | trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 57s | | trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 16s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 14s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 34s | | the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 34s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 1s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 0m 17s | [/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2787/3/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt) | hadoop-hdfs-project/hadoop-hdfs-rbf: The patch generated 5 new + 0 unchanged - 0 fixed = 5 total (was 0) |
| +1 :green_heart: | mvnsite | 0m 32s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 30s | | the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 48s | | the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 16s | | the patch passed |
| +1 :green_heart: | shadedclient | 13m 50s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| -1 :x: | unit | 17m 50s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2787/3/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt) | hadoop-hdfs-rbf in the patch passed. |
| +1 :green_heart: | asflicense | 0m 33s | | The patch does not generate ASF License warnings. |
| | | 91m 32s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.server.federation.router.TestRouterRpc |
| | hadoop.fs.contract.router.web.TestRouterWebHDFSContractCreate |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2787/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2787 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 03adef16d5ff 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / d07a0b02c29fa37c40b5e6cd1de729acb895bad3 |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2787/3/testReport/ |
| Max. process+thread count | 2190 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2787/3/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hdaikoku commented on a change in pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.
Posted by GitBox <gi...@apache.org>.
hdaikoku commented on a change in pull request #2787:
URL: https://github.com/apache/hadoop/pull/2787#discussion_r598426671
##########
File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/resolver/FederationNamespaceInfo.java
##########
@@ -75,4 +76,27 @@ public String getBlockPoolId() {
public String toString() {
return this.nameserviceId + "->" + this.blockPoolId + ":" + this.clusterId;
}
-}
\ No newline at end of file
+
+ @Override
+ public boolean equals(Object obj) {
+ if (this == obj) {
+ return true;
+ }
+ if (obj instanceof FederationNamespaceInfo) {
Review comment:
Thank you for pointing out.
Reimplemented `hashCode` and `equals` with commons: 6124e71
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] goiri commented on a change in pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.
Posted by GitBox <gi...@apache.org>.
goiri commented on a change in pull request #2787:
URL: https://github.com/apache/hadoop/pull/2787#discussion_r598845045
##########
File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/store/TestStateStoreMembershipState.java
##########
@@ -473,6 +478,53 @@ public void testRegistrationExpiredAndDeletion()
}, 100, 3000);
}
+ @Test
+ public void testNamespaceInfoWithUnavailableNameNodeRegistration() throws IOException {
+ // Populate the state store with one ACTIVE NameNode entry and one UNAVAILABLE NameNode entry
+ // 1) ns0:nn0 - ACTIVE
+ // 2) ns0:nn1 - UNAVAILABLE
+ List<MembershipState> registrationList = new ArrayList<>();
+ String router = ROUTERS[0];
+ String ns = NAMESERVICES[0];
+ String rpcAddress = "testrpcaddress";
+ String serviceAddress = "testserviceaddress";
+ String lifelineAddress = "testlifelineaddress";
+ String blockPoolId = "testblockpool";
+ String clusterId = "testcluster";
+ String webScheme = "http";
+ String webAddress = "testwebaddress";
+ boolean safemode = false;
+
+ MembershipState record = MembershipState.newInstance(
+ router, ns, NAMENODES[0], clusterId, blockPoolId,
+ rpcAddress, serviceAddress, lifelineAddress, webScheme,
+ webAddress, FederationNamenodeServiceState.ACTIVE, safemode);
+ registrationList.add(record);
+
+ // Set empty clusterId and blockPoolId for UNAVAILABLE NameNode
+ record = MembershipState.newInstance(
+ router, ns, NAMENODES[1], "", "",
+ rpcAddress, serviceAddress, lifelineAddress, webScheme,
+ webAddress, FederationNamenodeServiceState.UNAVAILABLE, safemode);
+ registrationList.add(record);
+
+ registerAndLoadRegistrations(registrationList);
+
+ GetNamespaceInfoRequest request = GetNamespaceInfoRequest.newInstance();
+ GetNamespaceInfoResponse response = membershipStore.getNamespaceInfo(request);
+ Set<FederationNamespaceInfo> namespaces = response.getNamespaceInfo();
+
+ // Verify only one namespace is registered
+ assertThat(namespaces).hasSize(1);
+
+ // Verify the registered namespace has a valid pair of clusterId and blockPoolId derived from ACTIVE NameNode
+ for (FederationNamespaceInfo namespace : namespaces) {
Review comment:
Given that we've already checked for the size to be 1, could we just get the element first?
It is a little counter intuitive to see this for loop checking for all elements.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tasanuma commented on pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.
Posted by GitBox <gi...@apache.org>.
tasanuma commented on pull request #2787:
URL: https://github.com/apache/hadoop/pull/2787#issuecomment-809027425
Merged to trunk.
Thanks for your contribution and discussion, @hdaikoku and @aajisaka. Thanks for your review, @goiri.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2787:
URL: https://github.com/apache/hadoop/pull/2787#issuecomment-803801256
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 37s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 2 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 52s | | trunk passed |
| +1 :green_heart: | compile | 0m 41s | | trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 35s | | trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 24s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 42s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 39s | | trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 57s | | trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 24s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 54s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 33s | | the patch passed |
| +1 :green_heart: | compile | 0m 32s | | the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 0m 29s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 0m 18s | [/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2787/2/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt) | hadoop-hdfs-project/hadoop-hdfs-rbf: The patch generated 5 new + 0 unchanged - 0 fixed = 5 total (was 0) |
| +1 :green_heart: | mvnsite | 0m 34s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 32s | | the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 49s | | the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 21s | | the patch passed |
| +1 :green_heart: | shadedclient | 15m 7s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| -1 :x: | unit | 17m 59s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2787/2/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt) | hadoop-hdfs-rbf in the patch passed. |
| +1 :green_heart: | asflicense | 0m 32s | | The patch does not generate ASF License warnings. |
| | | 94m 56s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.fs.contract.router.web.TestRouterWebHDFSContractCreate |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2787/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2787 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 36bd6387d6af 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / c51d8f07a68648bd6dc0c52c5a77cad7b8c02f05 |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2787/2/testReport/ |
| Max. process+thread count | 2183 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2787/2/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hdaikoku commented on a change in pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.
Posted by GitBox <gi...@apache.org>.
hdaikoku commented on a change in pull request #2787:
URL: https://github.com/apache/hadoop/pull/2787#discussion_r598427620
##########
File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/impl/MembershipStoreImpl.java
##########
@@ -213,12 +213,14 @@ public boolean loadCache(boolean force) throws IOException {
nnRegistrations.put(nnId, nnRegistration);
}
nnRegistration.add(membership);
- String bpId = membership.getBlockPoolId();
- String cId = membership.getClusterId();
- String nsId = membership.getNameserviceId();
- FederationNamespaceInfo nsInfo =
- new FederationNamespaceInfo(bpId, cId, nsId);
- this.activeNamespaces.add(nsInfo);
+ if (membership.getState() != FederationNamenodeServiceState.UNAVAILABLE) {
+ String bpId = membership.getBlockPoolId();
Review comment:
Added a test case in TestStateStoreMembershipState: c51d8f0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hdaikoku commented on a change in pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.
Posted by GitBox <gi...@apache.org>.
hdaikoku commented on a change in pull request #2787:
URL: https://github.com/apache/hadoop/pull/2787#discussion_r599170545
##########
File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/store/TestStateStoreMembershipState.java
##########
@@ -473,6 +478,53 @@ public void testRegistrationExpiredAndDeletion()
}, 100, 3000);
}
+ @Test
+ public void testNamespaceInfoWithUnavailableNameNodeRegistration() throws IOException {
+ // Populate the state store with one ACTIVE NameNode entry and one UNAVAILABLE NameNode entry
+ // 1) ns0:nn0 - ACTIVE
+ // 2) ns0:nn1 - UNAVAILABLE
+ List<MembershipState> registrationList = new ArrayList<>();
+ String router = ROUTERS[0];
+ String ns = NAMESERVICES[0];
+ String rpcAddress = "testrpcaddress";
+ String serviceAddress = "testserviceaddress";
+ String lifelineAddress = "testlifelineaddress";
+ String blockPoolId = "testblockpool";
+ String clusterId = "testcluster";
+ String webScheme = "http";
+ String webAddress = "testwebaddress";
+ boolean safemode = false;
+
+ MembershipState record = MembershipState.newInstance(
+ router, ns, NAMENODES[0], clusterId, blockPoolId,
+ rpcAddress, serviceAddress, lifelineAddress, webScheme,
+ webAddress, FederationNamenodeServiceState.ACTIVE, safemode);
+ registrationList.add(record);
+
+ // Set empty clusterId and blockPoolId for UNAVAILABLE NameNode
+ record = MembershipState.newInstance(
+ router, ns, NAMENODES[1], "", "",
+ rpcAddress, serviceAddress, lifelineAddress, webScheme,
+ webAddress, FederationNamenodeServiceState.UNAVAILABLE, safemode);
+ registrationList.add(record);
+
+ registerAndLoadRegistrations(registrationList);
+
+ GetNamespaceInfoRequest request = GetNamespaceInfoRequest.newInstance();
+ GetNamespaceInfoResponse response = membershipStore.getNamespaceInfo(request);
+ Set<FederationNamespaceInfo> namespaces = response.getNamespaceInfo();
+
+ // Verify only one namespace is registered
+ assertThat(namespaces).hasSize(1);
Review comment:
Didn't have any particular reason so I switched it to assertEquals: d07a0b0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] goiri commented on a change in pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.
Posted by GitBox <gi...@apache.org>.
goiri commented on a change in pull request #2787:
URL: https://github.com/apache/hadoop/pull/2787#discussion_r597386195
##########
File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/resolver/FederationNamespaceInfo.java
##########
@@ -75,4 +76,27 @@ public String getBlockPoolId() {
public String toString() {
return this.nameserviceId + "->" + this.blockPoolId + ":" + this.clusterId;
}
-}
\ No newline at end of file
+
+ @Override
+ public boolean equals(Object obj) {
+ if (this == obj) {
+ return true;
+ }
+ if (obj instanceof FederationNamespaceInfo) {
Review comment:
There is also EqualsBuilder in commons.
##########
File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/impl/MembershipStoreImpl.java
##########
@@ -213,12 +213,14 @@ public boolean loadCache(boolean force) throws IOException {
nnRegistrations.put(nnId, nnRegistration);
}
nnRegistration.add(membership);
- String bpId = membership.getBlockPoolId();
- String cId = membership.getClusterId();
- String nsId = membership.getNameserviceId();
- FederationNamespaceInfo nsInfo =
- new FederationNamespaceInfo(bpId, cId, nsId);
- this.activeNamespaces.add(nsInfo);
+ if (membership.getState() != FederationNamenodeServiceState.UNAVAILABLE) {
+ String bpId = membership.getBlockPoolId();
Review comment:
Is there any test we can do for this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hdaikoku commented on a change in pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.
Posted by GitBox <gi...@apache.org>.
hdaikoku commented on a change in pull request #2787:
URL: https://github.com/apache/hadoop/pull/2787#discussion_r599170545
##########
File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/store/TestStateStoreMembershipState.java
##########
@@ -473,6 +478,53 @@ public void testRegistrationExpiredAndDeletion()
}, 100, 3000);
}
+ @Test
+ public void testNamespaceInfoWithUnavailableNameNodeRegistration() throws IOException {
+ // Populate the state store with one ACTIVE NameNode entry and one UNAVAILABLE NameNode entry
+ // 1) ns0:nn0 - ACTIVE
+ // 2) ns0:nn1 - UNAVAILABLE
+ List<MembershipState> registrationList = new ArrayList<>();
+ String router = ROUTERS[0];
+ String ns = NAMESERVICES[0];
+ String rpcAddress = "testrpcaddress";
+ String serviceAddress = "testserviceaddress";
+ String lifelineAddress = "testlifelineaddress";
+ String blockPoolId = "testblockpool";
+ String clusterId = "testcluster";
+ String webScheme = "http";
+ String webAddress = "testwebaddress";
+ boolean safemode = false;
+
+ MembershipState record = MembershipState.newInstance(
+ router, ns, NAMENODES[0], clusterId, blockPoolId,
+ rpcAddress, serviceAddress, lifelineAddress, webScheme,
+ webAddress, FederationNamenodeServiceState.ACTIVE, safemode);
+ registrationList.add(record);
+
+ // Set empty clusterId and blockPoolId for UNAVAILABLE NameNode
+ record = MembershipState.newInstance(
+ router, ns, NAMENODES[1], "", "",
+ rpcAddress, serviceAddress, lifelineAddress, webScheme,
+ webAddress, FederationNamenodeServiceState.UNAVAILABLE, safemode);
+ registrationList.add(record);
+
+ registerAndLoadRegistrations(registrationList);
+
+ GetNamespaceInfoRequest request = GetNamespaceInfoRequest.newInstance();
+ GetNamespaceInfoResponse response = membershipStore.getNamespaceInfo(request);
+ Set<FederationNamespaceInfo> namespaces = response.getNamespaceInfo();
+
+ // Verify only one namespace is registered
+ assertThat(namespaces).hasSize(1);
Review comment:
Didn't have any particular reason or preference so I switched it to assertEquals: d07a0b0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2787:
URL: https://github.com/apache/hadoop/pull/2787#issuecomment-802513718
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 37s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 35s | | trunk passed |
| +1 :green_heart: | compile | 0m 42s | | trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 36s | | trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 28s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 42s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 42s | | trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 55s | | trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 17s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 13s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 33s | | the patch passed |
| +1 :green_heart: | compile | 0m 33s | | the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 33s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 0m 29s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 0m 18s | [/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2787/1/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt) | hadoop-hdfs-project/hadoop-hdfs-rbf: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
| +1 :green_heart: | mvnsite | 0m 31s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 32s | | the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 47s | | the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 15s | | the patch passed |
| +1 :green_heart: | shadedclient | 14m 5s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| -1 :x: | unit | 17m 37s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2787/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt) | hadoop-hdfs-rbf in the patch passed. |
| +1 :green_heart: | asflicense | 0m 33s | | The patch does not generate ASF License warnings. |
| | | 91m 38s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.server.federation.router.TestRouterRpcMultiDestination |
| | hadoop.hdfs.server.federation.router.TestRouterRpc |
| | hadoop.hdfs.server.federation.resolver.TestFederationNamespaceInfo |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2787/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2787 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 52e780355c42 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 8983b9f309e260d69c4fa932e38aaf418da515c2 |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2787/1/testReport/ |
| Max. process+thread count | 2189 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2787/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] aajisaka commented on a change in pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.
Posted by GitBox <gi...@apache.org>.
aajisaka commented on a change in pull request #2787:
URL: https://github.com/apache/hadoop/pull/2787#discussion_r599190892
##########
File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/store/TestStateStoreMembershipState.java
##########
@@ -473,6 +478,53 @@ public void testRegistrationExpiredAndDeletion()
}, 100, 3000);
}
+ @Test
+ public void testNamespaceInfoWithUnavailableNameNodeRegistration() throws IOException {
+ // Populate the state store with one ACTIVE NameNode entry and one UNAVAILABLE NameNode entry
+ // 1) ns0:nn0 - ACTIVE
+ // 2) ns0:nn1 - UNAVAILABLE
+ List<MembershipState> registrationList = new ArrayList<>();
+ String router = ROUTERS[0];
+ String ns = NAMESERVICES[0];
+ String rpcAddress = "testrpcaddress";
+ String serviceAddress = "testserviceaddress";
+ String lifelineAddress = "testlifelineaddress";
+ String blockPoolId = "testblockpool";
+ String clusterId = "testcluster";
+ String webScheme = "http";
+ String webAddress = "testwebaddress";
+ boolean safemode = false;
+
+ MembershipState record = MembershipState.newInstance(
+ router, ns, NAMENODES[0], clusterId, blockPoolId,
+ rpcAddress, serviceAddress, lifelineAddress, webScheme,
+ webAddress, FederationNamenodeServiceState.ACTIVE, safemode);
+ registrationList.add(record);
+
+ // Set empty clusterId and blockPoolId for UNAVAILABLE NameNode
+ record = MembershipState.newInstance(
+ router, ns, NAMENODES[1], "", "",
+ rpcAddress, serviceAddress, lifelineAddress, webScheme,
+ webAddress, FederationNamenodeServiceState.UNAVAILABLE, safemode);
+ registrationList.add(record);
+
+ registerAndLoadRegistrations(registrationList);
+
+ GetNamespaceInfoRequest request = GetNamespaceInfoRequest.newInstance();
+ GetNamespaceInfoResponse response = membershipStore.getNamespaceInfo(request);
+ Set<FederationNamespaceInfo> namespaces = response.getNamespaceInfo();
+
+ // Verify only one namespace is registered
+ assertThat(namespaces).hasSize(1);
Review comment:
> Is the error message friendlier?
Yes. If the expected size is set to 2, the error message will be:
```
Expected size:<2> but was:<1> in:
<[ns0->testblockpool:nn0]>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] goiri commented on a change in pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.
Posted by GitBox <gi...@apache.org>.
goiri commented on a change in pull request #2787:
URL: https://github.com/apache/hadoop/pull/2787#discussion_r599702122
##########
File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/store/TestStateStoreMembershipState.java
##########
@@ -473,6 +478,53 @@ public void testRegistrationExpiredAndDeletion()
}, 100, 3000);
}
+ @Test
+ public void testNamespaceInfoWithUnavailableNameNodeRegistration() throws IOException {
+ // Populate the state store with one ACTIVE NameNode entry and one UNAVAILABLE NameNode entry
+ // 1) ns0:nn0 - ACTIVE
+ // 2) ns0:nn1 - UNAVAILABLE
+ List<MembershipState> registrationList = new ArrayList<>();
+ String router = ROUTERS[0];
+ String ns = NAMESERVICES[0];
+ String rpcAddress = "testrpcaddress";
+ String serviceAddress = "testserviceaddress";
+ String lifelineAddress = "testlifelineaddress";
+ String blockPoolId = "testblockpool";
+ String clusterId = "testcluster";
+ String webScheme = "http";
+ String webAddress = "testwebaddress";
+ boolean safemode = false;
+
+ MembershipState record = MembershipState.newInstance(
+ router, ns, NAMENODES[0], clusterId, blockPoolId,
+ rpcAddress, serviceAddress, lifelineAddress, webScheme,
+ webAddress, FederationNamenodeServiceState.ACTIVE, safemode);
+ registrationList.add(record);
+
+ // Set empty clusterId and blockPoolId for UNAVAILABLE NameNode
+ record = MembershipState.newInstance(
+ router, ns, NAMENODES[1], "", "",
+ rpcAddress, serviceAddress, lifelineAddress, webScheme,
+ webAddress, FederationNamenodeServiceState.UNAVAILABLE, safemode);
+ registrationList.add(record);
+
+ registerAndLoadRegistrations(registrationList);
+
+ GetNamespaceInfoRequest request = GetNamespaceInfoRequest.newInstance();
+ GetNamespaceInfoResponse response = membershipStore.getNamespaceInfo(request);
+ Set<FederationNamespaceInfo> namespaces = response.getNamespaceInfo();
+
+ // Verify only one namespace is registered
+ assertThat(namespaces).hasSize(1);
Review comment:
I guess that the assertThat then it's better.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hdaikoku commented on a change in pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.
Posted by GitBox <gi...@apache.org>.
hdaikoku commented on a change in pull request #2787:
URL: https://github.com/apache/hadoop/pull/2787#discussion_r597352892
##########
File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/resolver/TestFederationNamespaceInfo.java
##########
@@ -0,0 +1,39 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdfs.server.federation.resolver;
+
+import org.junit.Test;
+
+import java.util.Set;
+import java.util.TreeSet;
+
+import static org.assertj.core.api.Assertions.assertThat;
+
+public class TestFederationNamespaceInfo {
+ /**
+ * Regression test for HDFS-15900.
Review comment:
Provided courtesy of @aajisaka
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org