You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Lokesh Jain (Jira)" <ji...@apache.org> on 2021/04/09 11:32:00 UTC
[jira] [Comment Edited] (HDDS-5053) RocksDB block cache capacity is
wrongly configured
[ https://issues.apache.org/jira/browse/HDDS-5053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17317927#comment-17317927 ]
Lokesh Jain edited comment on HDDS-5053 at 4/9/21, 11:31 AM:
-------------------------------------------------------------
Initially I thought that excessive usage was because block cache was not being shared by the containers. https://github.com/apache/ozone/pull/2126 included change for that. But it did not help in reducing the memory usage.
It turns out block cache was wrongly configured to use 64MB*MB bytes. Created a PR to address the same. This was causing block cache memory to increase in an unbounded manner.
It would be still be good to include PR 2126. Will create a separate jira for it.
was (Author: ljain):
Initially I thought that excessive usage was because block cache was not being shared by the containers. https://github.com/apache/ozone/pull/2126 included change for that. But it did not help in reducing the memory usage.
It turns out block cache was wrongly configured to use 64MB*MB bytes. Created a PR to address the same. This was causing block cache memory to increase in an unbounded manner.
> RocksDB block cache capacity is wrongly configured
> --------------------------------------------------
>
> Key: HDDS-5053
> URL: https://issues.apache.org/jira/browse/HDDS-5053
> Project: Apache Ozone
> Issue Type: Bug
> Components: Ozone Datanode
> Reporter: Lokesh Jain
> Assignee: Lokesh Jain
> Priority: Major
> Labels: pull-request-available
>
> The DN process uses around 20 GB of RSS after startup and does not release the memory even though rocksdb is closed for every container.
> The capacity used by AbstractDatanodeStore is default capacity * MB. Since default capacity is 64MB therefore block cache is passed a capacity of 64TB. After the fix the DN memory usage drops to ~3GB.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org