You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Shashikant Banerjee (JIRA)" <ji...@apache.org> on 2018/02/06 09:33:01 UTC
[jira] [Commented] (HDFS-12794) Ozone: Parallelize ChunkOutputSream
Writes to container
[ https://issues.apache.org/jira/browse/HDFS-12794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353639#comment-16353639 ]
Shashikant Banerjee commented on HDFS-12794:
--------------------------------------------
patch v10 addresses the OOM issue by limiting the size of ozone stream buffer size config to 32 MB for unit tests.
> Ozone: Parallelize ChunkOutputSream Writes to container
> -------------------------------------------------------
>
> Key: HDFS-12794
> URL: https://issues.apache.org/jira/browse/HDFS-12794
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
> Affects Versions: HDFS-7240
> Reporter: Shashikant Banerjee
> Assignee: Shashikant Banerjee
> Priority: Major
> Fix For: HDFS-7240
>
> Attachments: HDFS-12794-HDFS-7240.001.patch, HDFS-12794-HDFS-7240.002.patch, HDFS-12794-HDFS-7240.003.patch, HDFS-12794-HDFS-7240.004.patch, HDFS-12794-HDFS-7240.005.patch, HDFS-12794-HDFS-7240.006.patch, HDFS-12794-HDFS-7240.007.patch, HDFS-12794-HDFS-7240.008.patch, HDFS-12794-HDFS-7240.009.patch, HDFS-12794-HDFS-7240.010.patch
>
>
> The chunkOutPutStream Write are sync in nature .Once one chunk of data gets written, the next chunk write is blocked until the previous chunk is written to the container.
> The ChunkOutputWrite Stream writes should be made async and Close on the OutputStream should ensure flushing of all dirty buffers to the container.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org