You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "adoroszlai (via GitHub)" <gi...@apache.org> on 2024/06/11 11:07:13 UTC
[PR] HDDS-11001. Speed up some tests that restart datanode [ozone]
adoroszlai opened a new pull request, #6803:
URL: https://github.com/apache/ozone/pull/6803
## What changes were proposed in this pull request?
Restarting datanode in integration test with `waitForDatanode=true` waits for the node to become stale. Reducing stale node period (like most other tests already do) can improve test execution time.
https://issues.apache.org/jira/browse/HDDS-11001
## How was this patch tested?
Before:
```
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 129.058 s - in org.apache.hadoop.ozone.shell.TestReconfigShell
[INFO] org.apache.hadoop.ozone.shell.TestReconfigShell.testStorageContainerManagerGetReconfigurationProperties Time elapsed: 0.097 s
[INFO] org.apache.hadoop.ozone.shell.TestReconfigShell.testDataNodeGetReconfigurableProperties Time elapsed: 0.076 s
[INFO] org.apache.hadoop.ozone.shell.TestReconfigShell.testDatanodeBulkReconfig Time elapsed: 107.224 s
[INFO] org.apache.hadoop.ozone.shell.TestReconfigShell.testOzoneManagerGetReconfigurationProperties Time elapsed: 0.022 s
[INFO] Running org.apache.hadoop.ozone.shell.TestOzoneContainerUpgradeShell
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 133.043 s - in org.apache.hadoop.ozone.shell.TestOzoneContainerUpgradeShell
[INFO] org.apache.hadoop.ozone.shell.TestOzoneContainerUpgradeShell.testNormalContainerUpgrade Time elapsed: 115.725 s
[INFO] Running org.apache.hadoop.hdds.scm.storage.TestContainerCommandsEC
[INFO] Tests run: 33, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 170.121 s - in org.apache.hadoop.hdds.scm.storage.TestContainerCommandsEC
[INFO] org.apache.hadoop.hdds.scm.storage.TestContainerCommandsEC.testECReconstructionCoordinatorShouldCleanupContainersOnFailure Time elapsed: 30.298 s
[INFO] org.apache.hadoop.hdds.scm.storage.TestContainerCommandsEC.testECReconstructionCoordinatorWithFullAndPartialStripe(List)[1] Time elapsed: 0.292 s
...
[INFO] org.apache.hadoop.hdds.scm.storage.TestContainerCommandsEC.testECReconstructionCoordinatorWithFullAndPartialStripe(List)[9] Time elapsed: 0.185 s
[INFO] org.apache.hadoop.hdds.scm.storage.TestContainerCommandsEC.testCreateRecoveryContainer Time elapsed: 0.024 s
[INFO] org.apache.hadoop.hdds.scm.storage.TestContainerCommandsEC.testCreateRecoveryContainerAfterDNRestart Time elapsed: 108.108 s
[INFO] org.apache.hadoop.hdds.scm.storage.TestContainerCommandsEC.testECReconstructionCoordinatorWithPartialStripe(List)[1] Time elapsed: 0.15 s
...
[INFO] org.apache.hadoop.hdds.scm.storage.TestContainerCommandsEC.testECReconstructionCoordinatorWithPartialStripe(List)[9] Time elapsed: 0.115 s
[INFO] org.apache.hadoop.hdds.scm.storage.TestContainerCommandsEC.testListBlock Time elapsed: 0.048 s
[INFO] org.apache.hadoop.hdds.scm.storage.TestContainerCommandsEC.testECReconstructionCoordinatorWithMissingIndexes135 Time elapsed: 0.105 s
[INFO] org.apache.hadoop.hdds.scm.storage.TestContainerCommandsEC.testECReconstructionCoordinatorWith(List)[1] Time elapsed: 0.1 s
...
[INFO] org.apache.hadoop.hdds.scm.storage.TestContainerCommandsEC.testECReconstructionCoordinatorWith(List)[9] Time elapsed: 0.118 s
[INFO] org.apache.hadoop.hdds.scm.storage.TestContainerCommandsEC.testOrphanBlock Time elapsed: 6.578 s
```
After:
```
[INFO] Running org.apache.hadoop.ozone.shell.TestReconfigShell
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 32.261 s - in org.apache.hadoop.ozone.shell.TestReconfigShell
[INFO] org.apache.hadoop.ozone.shell.TestReconfigShell.testStorageContainerManagerGetReconfigurationProperties Time elapsed: 0.095 s
[INFO] org.apache.hadoop.ozone.shell.TestReconfigShell.testDataNodeGetReconfigurableProperties Time elapsed: 0.074 s
[INFO] org.apache.hadoop.ozone.shell.TestReconfigShell.testDatanodeBulkReconfig Time elapsed: 10.145 s
[INFO] org.apache.hadoop.ozone.shell.TestReconfigShell.testOzoneManagerGetReconfigurationProperties Time elapsed: 0.024 s
[INFO] Running org.apache.hadoop.ozone.shell.TestOzoneContainerUpgradeShell
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 37.499 s - in org.apache.hadoop.ozone.shell.TestOzoneContainerUpgradeShell
[INFO] org.apache.hadoop.ozone.shell.TestOzoneContainerUpgradeShell.testNormalContainerUpgrade Time elapsed: 19.96 s
[INFO] Running org.apache.hadoop.hdds.scm.storage.TestContainerCommandsEC
[INFO] Tests run: 33, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 72.319 s - in org.apache.hadoop.hdds.scm.storage.TestContainerCommandsEC
[INFO] org.apache.hadoop.hdds.scm.storage.TestContainerCommandsEC.testECReconstructionCoordinatorShouldCleanupContainersOnFailure Time elapsed: 30.309 s
[INFO] org.apache.hadoop.hdds.scm.storage.TestContainerCommandsEC.testECReconstructionCoordinatorWithFullAndPartialStripe(List)[1] Time elapsed: 0.308 s
...
[INFO] org.apache.hadoop.hdds.scm.storage.TestContainerCommandsEC.testECReconstructionCoordinatorWithFullAndPartialStripe(List)[9] Time elapsed: 0.189 s
[INFO] org.apache.hadoop.hdds.scm.storage.TestContainerCommandsEC.testCreateRecoveryContainer Time elapsed: 0.022 s
[INFO] org.apache.hadoop.hdds.scm.storage.TestContainerCommandsEC.testCreateRecoveryContainerAfterDNRestart Time elapsed: 10.926 s
[INFO] org.apache.hadoop.hdds.scm.storage.TestContainerCommandsEC.testECReconstructionCoordinatorWithPartialStripe(List)[1] Time elapsed: 0.138 s
...
[INFO] org.apache.hadoop.hdds.scm.storage.TestContainerCommandsEC.testECReconstructionCoordinatorWithPartialStripe(List)[9] Time elapsed: 0.12 s
[INFO] org.apache.hadoop.hdds.scm.storage.TestContainerCommandsEC.testListBlock Time elapsed: 0.078 s
[INFO] org.apache.hadoop.hdds.scm.storage.TestContainerCommandsEC.testECReconstructionCoordinatorWithMissingIndexes135 Time elapsed: 0.115 s
[INFO] org.apache.hadoop.hdds.scm.storage.TestContainerCommandsEC.testECReconstructionCoordinatorWith(List)[1] Time elapsed: 0.095 s
...
[INFO] org.apache.hadoop.hdds.scm.storage.TestContainerCommandsEC.testECReconstructionCoordinatorWith(List)[9] Time elapsed: 0.115 s
[INFO] org.apache.hadoop.hdds.scm.storage.TestContainerCommandsEC.testOrphanBlock Time elapsed: 5.569 s
```
CI:
https://github.com/adoroszlai/ozone/actions/runs/9463724905
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-11001. Speed up some tests that restart datanode [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai merged PR #6803:
URL: https://github.com/apache/ozone/pull/6803
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-11001. Speed up some tests that restart datanode [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #6803:
URL: https://github.com/apache/ozone/pull/6803#issuecomment-2162252915
Thanks @hemantk-12 for the review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org