You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by "Xiaoyu Yao (Jira)" <ji...@apache.org> on 2019/11/15 22:21:00 UTC
[jira] [Resolved] (HDDS-2375) Refactor BlockOutputStream to allow
flexible buffering
[ https://issues.apache.org/jira/browse/HDDS-2375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Xiaoyu Yao resolved HDDS-2375.
------------------------------
Resolution: Fixed
Thanks [~szetszwo] for the contribution and all for the reviews. I've merged the PR to master.
> Refactor BlockOutputStream to allow flexible buffering
> ------------------------------------------------------
>
> Key: HDDS-2375
> URL: https://issues.apache.org/jira/browse/HDDS-2375
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Components: Ozone Client
> Reporter: Tsz-wo Sze
> Assignee: Tsz-wo Sze
> Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> In HDDS-2331, we found that Ozone client allocates a ByteBuffer with chunk size (e.g. 16MB ) to store data, unregarded the actual data size. The ByteBuffer will create a byte[] with chunk size. When the ByteBuffer is wrapped to a ByteString the byte[] remains in the ByteString.
> As a result, when the actual data size is small (e.g. 1MB), a lot of memory spaces (15MB) are wasted.
> In this JIRA, we refactor BlockOutputStream so that the buffering becomes more flexible. In a later JIRA (HDDS-2386), we implement chunk buffer using a list of smaller buffers which are allocated only if needed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org