[jira] [Created] (HDDS-2375) Refactor BlockOutputStream to allow flexible buffering

["Tsz-wo Sze (Jira)" <ji...@apache.org>]

Tsz-wo Sze created HDDS-2375:
--------------------------------

             Summary: Refactor BlockOutputStream to allow flexible buffering
                 Key: HDDS-2375
                 URL: https://issues.apache.org/jira/browse/HDDS-2375
             Project: Hadoop Distributed Data Store
          Issue Type: Improvement
          Components: Ozone Client
            Reporter: Tsz-wo Sze
            Assignee: Tsz-wo Sze


In HDDS-2331, we found that Ozone client allocates a ByteBuffer with chunk size (e.g. 16MB ) to store data, unregarded the actual data size.  The ByteBuffer will create a  byte[] with chunk size.  When the ByteBuffer is wrapped to a ByteString the byte[] remains in the ByteString.

As a result, when the actual data size is small (e.g. 1MB), a lot of memory spaces (15MB) are wasted.

In this JIRA, we refactor BlockOutputStream so that the buffering becomes more flexible.  In a later JIRA, we could implement chunk buffer using a list of smaller buffers which are allocated only if needed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org