You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "adoroszlai (via GitHub)" <gi...@apache.org> on 2026/10/09 18:08:09 UTC

[PR] HDDS-16820. Speed up TestBlockOutputStreamCorrectness [ozone]

adoroszlai opened a new pull request, #11457:
URL: https://github.com/apache/ozone/pull/11457

   ## What changes were proposed in this pull request?
   
   `TestBlockOutputStreamCorrectness` takes ~70 seconds in CI.  The single slowest test case is writing data 1-byte at a time.
   
   Reduce both data size and buffer sizes in `TestBlockOutputStreamCorrectness` by a factor of 16 to reduce runtime.
   
   https://issues.apache.org/jira/browse/HDDS-16820
   
   ## How was this patch tested?
   
   ### Local run
   
   Before:
   
   ```
   Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 61.33 s -- in org.apache.hadoop.hdds.scm.storage.TestBlockOutputStreamCorrectness
   org.apache.hadoop.hdds.scm.storage.TestBlockOutputStreamCorrectness.testEcReconstructionStreamDisablesContainerAutoCreate -- Time elapsed: 0.573 s
   org.apache.hadoop.hdds.scm.storage.TestBlockOutputStreamCorrectness.testEcClientStreamAllowsContainerAutoCreate -- Time elapsed: 0.002 s
   org.apache.hadoop.hdds.scm.storage.TestBlockOutputStreamCorrectness.testMissingStripeChecksumDoesNotMakeExecutePutBlockFailDuringECReconstruction -- Time elapsed: 0.136 s
   org.apache.hadoop.hdds.scm.storage.TestBlockOutputStreamCorrectness.test(int)[1] -- Time elapsed: 49.31 s
   org.apache.hadoop.hdds.scm.storage.TestBlockOutputStreamCorrectness.test(int)[2] -- Time elapsed: 5.425 s
   org.apache.hadoop.hdds.scm.storage.TestBlockOutputStreamCorrectness.test(int)[3] -- Time elapsed: 5.298 s
   ```
   
   After:
   
   ```
   Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.911 s -- in org.apache.hadoop.hdds.scm.storage.TestBlockOutputStreamCorrectness
   org.apache.hadoop.hdds.scm.storage.TestBlockOutputStreamCorrectness.testEcReconstructionStreamDisablesContainerAutoCreate -- Time elapsed: 0.623 s
   org.apache.hadoop.hdds.scm.storage.TestBlockOutputStreamCorrectness.testEcClientStreamAllowsContainerAutoCreate -- Time elapsed: 0.005 s
   org.apache.hadoop.hdds.scm.storage.TestBlockOutputStreamCorrectness.testMissingStripeChecksumDoesNotMakeExecutePutBlockFailDuringECReconstruction -- Time elapsed: 0.104 s
   org.apache.hadoop.hdds.scm.storage.TestBlockOutputStreamCorrectness.test(int)[1] -- Time elapsed: 2.678 s
   org.apache.hadoop.hdds.scm.storage.TestBlockOutputStreamCorrectness.test(int)[2] -- Time elapsed: 0.213 s
   org.apache.hadoop.hdds.scm.storage.TestBlockOutputStreamCorrectness.test(int)[3] -- Time elapsed: 0.173 s
   ```
   
   ### CI
   
   [Before](https://github.com/apache/ozone/actions/runs/37919334183/job/113787874112#step:13:1621):
   
   ```
   Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 71.42 s -- in org.apache.hadoop.hdds.scm.storage.TestBlockOutputStreamCorrectness
   ```
   
   [After](https://github.com/adoroszlai/ozone/actions/runs/37957214926/job/113920149092#step:13:1617):
   
   ```
   Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.142 s -- in org.apache.hadoop.hdds.scm.storage.TestBlockOutputStreamCorrectness
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-16820. Speed up TestBlockOutputStreamCorrectness [ozone]

Posted by "peterxcli (via GitHub)" <gi...@apache.org>.
peterxcli commented on PR #11457:
URL: https://github.com/apache/ozone/pull/11457#issuecomment-6095848112

   thanks @adoroszlai for the patch!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-16820. Speed up TestBlockOutputStreamCorrectness [ozone]

Posted by "peterxcli (via GitHub)" <gi...@apache.org>.
peterxcli merged PR #11457:
URL: https://github.com/apache/ozone/pull/11457


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-16820. Speed up TestBlockOutputStreamCorrectness [ozone]

Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #11457:
URL: https://github.com/apache/ozone/pull/11457#issuecomment-6097879851

   Thanks @peterxcli for the review.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org