You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Shashikant Banerjee (JIRA)" <ji...@apache.org> on 2017/11/09 18:56:00 UTC
[jira] [Updated] (HDFS-12794) Ozone: Parallelize ChunkOutputSream
Writes to container
[ https://issues.apache.org/jira/browse/HDFS-12794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shashikant Banerjee updated HDFS-12794:
---------------------------------------
Attachment: HDFS-12794-HDFS-7240.001.patch
In patch v1, an arrayBlocking queue of size determined by ozone_max_chunk_data_limit/chunkSize is used which will basically determine the no of outstanding async chunkoutputstream write requests at any point of time. The entry put in the queue is the traceId of the chunk for which the async write is triggered. When the async write completes, its traceid is removed from the queue . Close on the stream ensures the response gets completed from XceiverClient and entry is removed from the queue.
> Ozone: Parallelize ChunkOutputSream Writes to container
> -------------------------------------------------------
>
> Key: HDFS-12794
> URL: https://issues.apache.org/jira/browse/HDFS-12794
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
> Affects Versions: HDFS-7240
> Reporter: Shashikant Banerjee
> Assignee: Shashikant Banerjee
> Fix For: HDFS-7240
>
> Attachments: HDFS-12794-HDFS-7240.001.patch
>
>
> The chunkOutPutStream Write are sync in nature .Once one chunk of data gets written, the next chunk write is blocked until the previous chunk is written to the container.
> The ChunkOutputWrite Stream writes should be made async and Close on the OutputStream should ensure flushing of all dirty buffers to the container.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org