You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "cychiu8 (via GitHub)" <gi...@apache.org> on 2026/07/24 07:32:54 UTC
[PR] HDDS-10315. Speed up TestContainerReplication [ozone]
cychiu8 opened a new pull request, #10858:
URL: https://github.com/apache/ozone/pull/10858
## What changes were proposed in this pull request?
Refactors the `TestContainerReplication` integration test to reduce runtime and duplication.
Reduce runtime:
- The `legacyEnabled` parameter is no longer used, so drop the `legacyEnabled` parameter dimension:
- `createConfiguration(boolean enableLegacy)` becomes `createConfiguration()`.
- `containerReplicationArguments()` now yields `(policyClass)` instead of (`policyClass, legacyEnabled)`, halving the parameterized runs.
- Renamed the test from `testContainerReplication` to `testRatisContainerReReplicationAfterDatanodeShutdown` and added a Javadoc describing what it verifies.
Cleanup:
- Both createTestData overloads reuse the existing `TestDataUtil.createVolumeAndBucket(...)` helper instead of hand-rolled volume/bucket setup.
- Replaced `OzoneClientFactory.getRpcClient(conf)` with `cluster.newClient()`.
- Replaced the one-off `newCluster` helper with the inline `MiniOzoneCluster` builder.
- Removed `lookupKey(...)` helper and folded into the existing `lookupKeyFirstLocation(...)`. Dropped unused imports and a stale TODO.
- Simplify `testECContainerReplication` datanode setup.
- fixes a Overiding → Overriding typo.
### Original running time (223.7 s)
```
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.ozone.container.TestContainerReplication
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 223.7 s -- in org.apache.hadoop.ozone.container.TestContainerReplication
[INFO] org.apache.hadoop.ozone.container.TestContainerReplication.testECContainerReplication -- Time elapsed: 40.69 s
[INFO] org.apache.hadoop.ozone.container.TestContainerReplication.testImportedContainerIsClosed -- Time elapsed: 32.48 s
[INFO] org.apache.hadoop.ozone.container.TestContainerReplication.testContainerReplication(String, boolean)[1] -- Time elapsed: 24.52 s
[INFO] org.apache.hadoop.ozone.container.TestContainerReplication.testContainerReplication(String, boolean)[2] -- Time elapsed: 24.35 s
[INFO] org.apache.hadoop.ozone.container.TestContainerReplication.testContainerReplication(String, boolean)[3] -- Time elapsed: 25.54 s
[INFO] org.apache.hadoop.ozone.container.TestContainerReplication.testContainerReplication(String, boolean)[4] -- Time elapsed: 24.42 s
[INFO] org.apache.hadoop.ozone.container.TestContainerReplication.testContainerReplication(String, boolean)[5] -- Time elapsed: 24.91 s
[INFO] org.apache.hadoop.ozone.container.TestContainerReplication.testContainerReplication(String, boolean)[6] -- Time elapsed: 26.51 s
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 04:00 min
[INFO] Finished at: 2026-07-23T10:30:16+09:00
[INFO] ------------------------------------------------------------------------
```
### Running time After Refactor (140.8s): 82.9 s faster
```
[INFO] Running org.apache.hadoop.ozone.container.TestContainerReplication
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 140.8 s -- in org.apache.hadoop.ozone.container.TestContainerReplication
[INFO] org.apache.hadoop.ozone.container.TestContainerReplication.testECContainerReplication -- Time elapsed: 35.28 s
[INFO] org.apache.hadoop.ozone.container.TestContainerReplication.testImportedContainerIsClosed -- Time elapsed: 31.83 s
[INFO] org.apache.hadoop.ozone.container.TestContainerReplication.testRatisContainerReReplicationAfterDatanodeShutdown(String)[1] -- Time elapsed: 24.40 s
[INFO] org.apache.hadoop.ozone.container.TestContainerReplication.testRatisContainerReReplicationAfterDatanodeShutdown(String)[2] -- Time elapsed: 25.11 s
[INFO] org.apache.hadoop.ozone.container.TestContainerReplication.testRatisContainerReReplicationAfterDatanodeShutdown(String)[3] -- Time elapsed: 24.11 s
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
```
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-10315
## How was this patch tested?
Ran the test suite:
```
mvn -pl :ozone-integration-test test \
-Dtest='org.apache.hadoop.ozone.container.TestContainerReplication' \
-Dsurefire.reportFormat=plain \
-Dsurefire.useFile=false \
-DskipShade -DskipRecon
```
Result: all tests passed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10315. Speed up TestContainerReplication [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #10858:
URL: https://github.com/apache/ozone/pull/10858#issuecomment-5078343716
Thanks @cychiu8 for the patch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10315. Speed up TestContainerReplication [ozone]
Posted by "cychiu8 (via GitHub)" <gi...@apache.org>.
cychiu8 commented on PR #10858:
URL: https://github.com/apache/ozone/pull/10858#issuecomment-5075808123
Hello @adoroszlai, Regarding [HDDS-10315](https://issues.apache.org/jira/browse/HDDS-10315) you reported, here is the PR. Happy to have your feedback whenever you get a chance :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10315. Speed up TestContainerReplication [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai merged PR #10858:
URL: https://github.com/apache/ozone/pull/10858
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org