[jira] [Commented] (HDDS-9843) Ozone client high memory utilization during write - should use pooled direct buffer for chunks

["Duong (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HDDS-9843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17814008#comment-17814008 ] 

Duong commented on HDDS-9843:
-----------------------------

So, HDDS-9536 is abandoned. 
We should move the write chunk buffer allocations to pooled direct buffers because:
- Pool buffers avoid the cost of allocation.
- Combined with direct buffers GC cost is eliminated.
- Direct buffers avoid one extra copy when writing data to network via GRPC.


> Ozone client high memory utilization during write - should use pooled direct buffer for chunks
> ----------------------------------------------------------------------------------------------
>
>                 Key: HDDS-9843
>                 URL: https://issues.apache.org/jira/browse/HDDS-9843
>             Project: Apache Ozone
>          Issue Type: Improvement
>          Components: Ozone Client, S3, s3gateway
>            Reporter: Duong
>            Assignee: Duong
>            Priority: Major
>              Labels: Performance, performance, pull-request-available
>         Attachments: buffer_allocation.png, image-2023-12-05-16-35-37-893.png, image-2023-12-05-16-36-17-726.png, s3_heap_metric.png
>
>
> Found this issue when working on S3.
> S3G memory footprint is too high when reading/writing small objects.
> Ran a test to write thousand of small (400 bytes) objects per seconds and memory utilization and gc is too high.
> !image-2023-12-05-16-36-17-726.png|width=527,height=489!
> This is issue is not S3 specific, but happens on ozone-client in general.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org