[GitHub] [hadoop-ozone] bshashikant commented on pull request #1374: HDDS-4185. Remove IncrementalByteBuffer from Ozone client

[GitBox <gi...@apache.org>]

bshashikant commented on pull request #1374:
URL: https://github.com/apache/hadoop-ozone/pull/1374#issuecomment-688760210


   > @bshashikant Do you have any feedback?
   
   Thanks @elek for putting up the patch together. The patch looks good to me in general. I am just thinking of a scenario like this:
   
   1) A stream(KeyOutputStream) is open and write is initiated
   2) Let's say "write(stream)" is invoked multiple times on the stream each time with length of 100 bytes or such in multiple iterations.
   3) In this case, key size overall can be greater than a block but key is written in segments of let's say 100 bytes
   4) Stream is closed
   
   In cases like this, we will end up having too many small buffers (100 bytes in this cases) created (with each write call).
   Buffer pool is reused for multiple blocks of a key so the similar patten will be followed for all the blocks.
   
   Although, the overall key is larger than size, this pattern of write still will lead to allocation of very small buffers which is not desirable for performance as pointed by the perf tests.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org