You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Tsz-wo Sze (Jira)" <ji...@apache.org> on 2024/02/09 18:05:00 UTC
[jira] [Updated] (HDDS-9843) Ozone client high memory (heap) utilization
[ https://issues.apache.org/jira/browse/HDDS-9843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz-wo Sze updated HDDS-9843:
-----------------------------
Description:
Ozone client high memory utilization during write - should use pooled direct buffer for chunks
Found this issue when working on S3.
S3G memory footprint is too high when reading/writing small objects.
Ran a test to write thousand of small (400 bytes) objects per seconds and memory utilization and gc is too high.
!image-2023-12-05-16-36-17-726.png|width=527,height=489!
This is issue is not S3 specific, but happens on ozone-client in general.
was:
Found this issue when working on S3.
S3G memory footprint is too high when reading/writing small objects.
Ran a test to write thousand of small (400 bytes) objects per seconds and memory utilization and gc is too high.
!image-2023-12-05-16-36-17-726.png|width=527,height=489!
This is issue is not S3 specific, but happens on ozone-client in general.
Summary: Ozone client high memory (heap) utilization (was: Ozone client high memory utilization during write - should use pooled direct buffer for chunks)
> Ozone client high memory (heap) utilization
> -------------------------------------------
>
> Key: HDDS-9843
> URL: https://issues.apache.org/jira/browse/HDDS-9843
> Project: Apache Ozone
> Issue Type: Improvement
> Components: Ozone Client, S3, s3gateway
> Reporter: Duong
> Assignee: Duong
> Priority: Major
> Labels: Performance, performance, pull-request-available
> Attachments: buffer_allocation.png, image-2023-12-05-16-35-37-893.png, image-2023-12-05-16-36-17-726.png, s3_heap_metric.png
>
>
> Ozone client high memory utilization during write - should use pooled direct buffer for chunks
> Found this issue when working on S3.
> S3G memory footprint is too high when reading/writing small objects.
> Ran a test to write thousand of small (400 bytes) objects per seconds and memory utilization and gc is too high.
> !image-2023-12-05-16-36-17-726.png|width=527,height=489!
> This is issue is not S3 specific, but happens on ozone-client in general.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org