You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "will-sh (via GitHub)" <gi...@apache.org> on 2024/03/08 02:32:53 UTC
[PR] HDDS-10090.Clean up TestStorageContainerManager [ozone]
will-sh opened a new pull request, #6347:
URL: https://github.com/apache/ozone/pull/6347
## What changes were proposed in this pull request?
HDDS-10090.Clean up TestStorageContainerManager
Please describe your PR in detail:
Several goals in this jira have been achieved:
* Disabled test cases related to non-HA to HA change. This used to be not supported scenario, but was recently implemented in [HDDS-9897](https://issues.apache.org/jira/browse/HDDS-9897). Since that task also added a new test case, testSCMReinitializationWithHAEnabled and testSCMReinitializationWithHAUpgrade can be removed now.
* Thread.currentThread().sleep() can be replaced with Thread.sleep().
* Replace usage of GenericTestUtils.getTempPath with @TempDir.
The below items are not fixed yet, I will post some uncertain parts to the PR discussion.
* Test cases can now use MiniOzoneCluster wrapped in try-with-resources blocks to ensure the cluster is shut down even if waitForClusterToBeReady() and other operations fail.
* Convert try-catch blocks to assertThrows() where applicable (where exception is expected and try block calls fail().
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-10090
## How was this patch tested?
The below tests are passed
- manual tests: `mvn test -Dtest=TestStorageContainerManager`
- CI workflow run on the fork git repo (https://github.com/will-sh/ozone/actions/runs/8191346419)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10090.Clean up TestStorageContainerManager [ozone]
Posted by "will-sh (via GitHub)" <gi...@apache.org>.
will-sh commented on PR #6347:
URL: https://github.com/apache/ozone/pull/6347#issuecomment-1984948061
Regarding the remaining tasks, I'm not sure if my understanding is correct:
* Test cases can now use MiniOzoneCluster wrapped in try-with-resources blocks to ensure the cluster is shut down even if waitForClusterToBeReady() and other operations fail.
Before:
```
private void testRpcPermissionWithConf(
OzoneConfiguration ozoneConf,
Predicate<String> isAdmin,
String... usernames) throws Exception {
MiniOzoneCluster cluster = MiniOzoneCluster.newBuilder(ozoneConf).build();
try {
cluster.waitForClusterToBeReady();
for (String username : usernames) {
testRpcPermission(cluster, username,
!isAdmin.test(username));
}
} finally {
cluster.shutdown();
}
}
```
After:
```
private void testRpcPermissionWithConf(
OzoneConfiguration ozoneConf,
Predicate<String> isAdmin,
String... usernames) throws Exception {
try (MiniOzoneCluster cluster = MiniOzoneCluster.newBuilder(ozoneConf).build()) {
cluster.waitForClusterToBeReady();
for (String username : usernames) {
testRpcPermission(cluster, username,
!isAdmin.test(username));
}
} // The cluster is automatically closed here, no need to call cluster.shutdown()
}
```
* Convert try-catch blocks to assertThrows() where applicable (where exception is expected and try block calls fail().
I only found one place that might need to be refactored
Before:
```
try {
ContainerWithPipeline container2 = mockClientServer.allocateContainer(
HddsProtos.ReplicationType.RATIS,
HddsProtos.ReplicationFactor.ONE, OzoneConsts.OZONE);
if (expectPermissionDenied) {
fail("Operation should fail, expecting an IOException here.");
} else {
assertEquals(1, container2.getPipeline().getNodes().size());
}
} catch (Exception e) {
verifyPermissionDeniedException(e, fakeRemoteUsername);
}
```
After:
```
if (expectPermissionDenied) {
Exception ex = assertThrows(Exception.class, () -> {
ContainerWithPipeline container2 = mockClientServer.allocateContainer(
HddsProtos.ReplicationType.RATIS,
HddsProtos.ReplicationFactor.ONE, OzoneConsts.OZONE);
});
verifyPermissionDeniedException(ex, fakeRemoteUsername);
} else {
ContainerWithPipeline container2 = mockClientServer.allocateContainer(
HddsProtos.ReplicationType.RATIS,
HddsProtos.ReplicationFactor.ONE, OzoneConsts.OZONE);
assertEquals(1, container2.getPipeline().getNodes().size());
}
```
Could you please advise?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10090. Clean up TestStorageContainerManager [ozone]
Posted by "will-sh (via GitHub)" <gi...@apache.org>.
will-sh commented on PR #6347:
URL: https://github.com/apache/ozone/pull/6347#issuecomment-1986587400
Thanks for reviewing @adoroszlai
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10090.Clean up TestStorageContainerManager [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on code in PR #6347:
URL: https://github.com/apache/ozone/pull/6347#discussion_r1517129410
##########
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/TestStorageContainerManager.java:
##########
@@ -212,7 +211,7 @@ private void testRpcPermissionWithConf(
}
private void testRpcPermission(MiniOzoneCluster cluster,
- String fakeRemoteUsername, boolean expectPermissionDenied) {
+ String fakeRemoteUsername, boolean expectPermissionDenied) {
Review Comment:
nit: please keep original indent of method parameters (there are a few more similar cases below)
##########
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/TestStorageContainerManager.java:
##########
@@ -779,7 +707,7 @@ public void testScmProcessDatanodeHeartbeat() throws Exception {
conf.setClass(NET_TOPOLOGY_NODE_SWITCH_MAPPING_IMPL_KEY,
StaticMapping.class, DNSToSwitchMapping.class);
StaticMapping.addNodeToRack(NetUtils.normalizeHostNames(
- Collections.singleton(HddsUtils.getHostName(conf))).get(0),
+ Collections.singleton(HddsUtils.getHostName(conf))).get(0),
"/rack1");
Review Comment:
Now I noticed this can be simplified by using `normalizeHostName(String)` instead of `normalizeHostNames(Collection)`:
```
StaticMapping.addNodeToRack(NetUtils.normalizeHostName(HddsUtils.getHostName(conf)),
"/rack1");
```
Note: `import ...Collections` becomes unused.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10090.Clean up TestStorageContainerManager [ozone]
Posted by "will-sh (via GitHub)" <gi...@apache.org>.
will-sh commented on code in PR #6347:
URL: https://github.com/apache/ozone/pull/6347#discussion_r1517181138
##########
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/TestStorageContainerManager.java:
##########
@@ -212,7 +211,7 @@ private void testRpcPermissionWithConf(
}
private void testRpcPermission(MiniOzoneCluster cluster,
- String fakeRemoteUsername, boolean expectPermissionDenied) {
+ String fakeRemoteUsername, boolean expectPermissionDenied) {
Review Comment:
looks like the indent is auto-changed by intellij. I will try to figure out why and fix it :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10090. Clean up TestStorageContainerManager [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai merged PR #6347:
URL: https://github.com/apache/ozone/pull/6347
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org