You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Kaijie Chen (Jira)" <ji...@apache.org> on 2021/09/16 11:58:00 UTC
[jira] [Assigned] (HDDS-5752) Support zero copy in streaming API
[ https://issues.apache.org/jira/browse/HDDS-5752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kaijie Chen reassigned HDDS-5752:
---------------------------------
Assignee: Kaijie Chen
> Support zero copy in streaming API
> ----------------------------------
>
> Key: HDDS-5752
> URL: https://issues.apache.org/jira/browse/HDDS-5752
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Kaijie Chen
> Assignee: Kaijie Chen
> Priority: Major
> Labels: pull-request-available
>
> Ratis streaming supports {{writeAsync(File, position, count)}} API, which is backed by Netty {{FileRegion}} and {{FileChannel.transferTo}}.
> Let's introduce a new interface {{FileRegionStreamOutput}} to implement zero copy in {{OzoneDataStreamOutput}}, {{KeyDataStreamOutput}}, {{BlockDataStreamOutputEntry}} and {{BlockDataStreamOutput}}.
> Our tests shows Streaming(FileRegion) API have better performance over Async API and Streaming(MappedByteBuffer).
>
> Writing a single 10GB key from 1 client to a 3-node cluster.
> ||Test ID||Async||Streaming(MappedByteBuffer)||Streaming(FileRegion)||
> |1|87252|78698|66844|
> |2|88853|80203|70198|
> |3|90292|81529|70343|
>
> Writing 24 keys (1GB each) in parallel, from 1 client to a 3-node cluster.
> ||Test ID||Async||Streaming(MappedByteBuffer)||Streaming(FileRegion)||
> |1|54357|50228|39384|
> |2|52807|48714|39528|
> |3|49238|49603|36618|
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org