You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Kaijie Chen (Jira)" <ji...@apache.org> on 2021/09/29 07:49:00 UTC
[jira] [Resolved] (HDDS-5752) Add FileRegion API for Ozone
Streaming
[ https://issues.apache.org/jira/browse/HDDS-5752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kaijie Chen resolved HDDS-5752.
-------------------------------
Resolution: Won't Do
The applicable conditions of API is limited. Maybe it's better to just use it for performance tests, and not add it.
> Add FileRegion API for Ozone Streaming
> --------------------------------------
>
> Key: HDDS-5752
> URL: https://issues.apache.org/jira/browse/HDDS-5752
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Kaijie Chen
> Assignee: Kaijie Chen
> Priority: Major
> Labels: pull-request-available
>
> Ratis streaming supports {{writeAsync(File, position, count)}} API, which is backed by Netty {{FileRegion}} and {{FileChannel.transferTo}}.
> Let's introduce a new interface {{FileRegionStreamOutput}} to implement zero copy in {{OzoneDataStreamOutput}}, {{KeyDataStreamOutput}}, {{BlockDataStreamOutputEntry}} and {{BlockDataStreamOutput}}.
> Our tests shows Streaming(FileRegion) API have better performance over Async API.
> And it's a good performance reference for the Streaming(ByteBuffer) API.
> https://issues.apache.org/jira/browse/HDDS-5685?focusedCommentId=17416557&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17416557
> Test A (single 10GB key)
> ||API||Min||Max||Avg||
> |Async|64s|72s|68s|
> |Streaming (MappedByteBuffer)|55s|68s|62s|
> |Streaming (FileRegion)|55s|64s|61s|
> Test B (24x 1GB keys from 1 client)
> ||API||Min||Max||Avg||
> |Async|48s|54s|51s|
> |Streaming (MappedByteBuffer)|32s|>630s|33s|
> |Streaming (FileRegion)|36s|39s|38s|
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org