You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Kaijie Chen (Jira)" <ji...@apache.org> on 2021/09/16 11:11:00 UTC
[jira] [Created] (HDDS-5752) Support zero copy in streaming API
Kaijie Chen created HDDS-5752:
---------------------------------
Summary: Support zero copy in streaming API
Key: HDDS-5752
URL: https://issues.apache.org/jira/browse/HDDS-5752
Project: Apache Ozone
Issue Type: Sub-task
Reporter: Kaijie Chen
Ratis streaming supports {{writeAsync(File, position, count)}} API, which is backed by Netty {{FileRegion}} and {{FileChannel.transferTo}}.
Let's introduce a new interface {{FileRegionStreamOutput}} to implement zero copy in {{OzoneDataStreamOutput}}, {{KeyDataStreamOutput}}, {{BlockDataStreamOutputEntry}} and {{BlockDataStreamOutput}}.
Our tests shows Streaming(FileRegion) API have better performance over Async API and Streaming(MappedByteBuffer).
Writing a single 10GB key from 1 client to a 3-node cluster.
||Test ID||Async||Streaming(MappedByteBuffer)||Streaming(FileRegion)||
|1|87252|78698|66844|
|2|88853|80203|70198|
|3|90292|81529|70343|
Writing 24 keys (1GB each) in parallel, from 1 client to a 3-node cluster.
||Test ID||Async||Streaming(MappedByteBuffer)||Streaming(FileRegion)||
|1|54357|50228|39384|
|2|52807|48714|39528|
|3|49238|49603|36618|
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org