You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2021/09/20 12:24:10 UTC
[GitHub] [ozone] sodonnel opened a new pull request #2662: HDDS-5766. Speed up some OM tests by skipping SCM safemode check
sodonnel opened a new pull request #2662:
URL: https://github.com/apache/ozone/pull/2662
## What changes were proposed in this pull request?
Many of the OzoneManager integration tests do not really use SCM or the datanodes. Therefore we can skip the safemode check in SCM and start zero datanodes to make the mini-cluster come up faster.
With just this change, on my laptop:
```
TestOzoneManagerConfiguration went from 6min -> 3min 40s.
TestOzoneManagerHAMetadataOnly went from 4min 19s -> 1min 9s.
TestOzoneManagerListVolumes went from 40s -> 20s
TestOMMetrics went from 2min 6 -> 39s 55s
```
The runtime for these tests on Github is:
```
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 273.591 s - in org.apache.hadoop.ozone.om.TestOzoneManagerConfiguration
Warning: Tests run: 10, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 363.67 s - in org.apache.hadoop.ozone.om.TestOzoneManagerHAMetadataOnly
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 181.937 s - in org.apache.hadoop.ozone.om.TestOmMetrics
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 50.156 s - in org.apache.hadoop.ozone.om.TestOzoneManagerListVolumes
```
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-5766
## How was this patch tested?
Existing tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] sodonnel commented on a change in pull request #2662: HDDS-5766. Speed up some OM tests by skipping SCM safemode check
Posted by GitBox <gi...@apache.org>.
sodonnel commented on a change in pull request #2662:
URL: https://github.com/apache/ozone/pull/2662#discussion_r712258299
##########
File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOmMetrics.java
##########
@@ -367,6 +376,13 @@ public void testAclOperations() throws IOException {
@Test
public void testAclOperationsHA() throws Exception {
+ // This test needs a cluster with DNs and SCM to wait on safemode
+ cluster.shutdown();
Review comment:
Yea I think you are correct. I have pushed a new commit that refactors things to ensure we only get the cluster we need for each test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] sodonnel commented on pull request #2662: HDDS-5766. Speed up some OM tests by skipping SCM safemode check
Posted by GitBox <gi...@apache.org>.
sodonnel commented on pull request #2662:
URL: https://github.com/apache/ozone/pull/2662#issuecomment-922990358
Updated tests ran in:
```
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 65.194 s - in org.apache.hadoop.ozone.om.TestOzoneManagerConfiguration
Warning: Tests run: 10, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 72.674 s - in org.apache.hadoop.ozone.om.TestOzoneManagerHAMetadataOnly
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.88 s - in org.apache.hadoop.ozone.om.TestOzoneManagerListVolumes
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 46.888 s - in org.apache.hadoop.ozone.om.TestOmMetrics
```
Saving 208 + 291 + 134 + 29 = 662 seconds saved (~ 11 minutes).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on a change in pull request #2662: HDDS-5766. Speed up some OM tests by skipping SCM safemode check
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on a change in pull request #2662:
URL: https://github.com/apache/ozone/pull/2662#discussion_r712240983
##########
File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOmMetrics.java
##########
@@ -367,6 +376,13 @@ public void testAclOperations() throws IOException {
@Test
public void testAclOperationsHA() throws Exception {
+ // This test needs a cluster with DNs and SCM to wait on safemode
+ cluster.shutdown();
Review comment:
This looks a bit strange. Would it be better to get rid of `@Before` and explicitly create cluster with desired properties for each test? (Similar to what `TestOzoneManagerListVolumes` [did before](https://github.com/apache/ozone/blob/aae9ebd948e2b57bb947446d3b8fda9969cd6d47/hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOzoneManagerListVolumes.java#L197-L200) HDDS-5717.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai merged pull request #2662: HDDS-5766. Speed up some OM tests by skipping SCM safemode check
Posted by GitBox <gi...@apache.org>.
adoroszlai merged pull request #2662:
URL: https://github.com/apache/ozone/pull/2662
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org