You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "mingchao zhao (Jira)" <ji...@apache.org> on 2022/08/11 08:29:00 UTC
[jira] [Commented] (HDDS-6722) Memory leak after updating to RocksDB 7
[ https://issues.apache.org/jira/browse/HDDS-6722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17578321#comment-17578321 ]
mingchao zhao commented on HDDS-6722:
-------------------------------------
Thanks [~weichiu] for your feedback. I agree with you. To make Ozone 1.3.0 more stable, we can use RocksDB 6 in this release. We can upgrade to RocksDB 7 in the next release.
> Memory leak after updating to RocksDB 7
> ---------------------------------------
>
> Key: HDDS-6722
> URL: https://issues.apache.org/jira/browse/HDDS-6722
> Project: Apache Ozone
> Issue Type: Bug
> Affects Versions: 1.3.0
> Reporter: Wei-Chiu Chuang
> Assignee: Duong
> Priority: Blocker
> Labels: pull-request-available
> Attachments: om rocksdb crash.png
>
>
> After HDDS-6456 updated RocksDB version from 6.25.3 to 7.0.4, we started to experience OOM at Ozone Manager very quickly.
> In our test environment, the OM memory stayed at around 3GB. But after that, OM memory quickly increase to over 100GB and crashed within 2 hours.
> After some investigation and discussion with the RocksDB developers (https://github.com/facebook/rocksdb/issues/9962) we were able to isolate the issue to this particular breaking change: https://github.com/facebook/rocksdb/commit/99d86252b
> Prior to the change, RocksDB 6 automatically reclaims native memory when the corresponding Java object is garbage collected. After the change (RocksDB 7), applications must proactively close the objects unused.
> The biggest offenders are the RocksIterator. But there are many others. I'll file jira to clean up their usage. Unfortunately, SonarCloud doesn't seem to detect these unclosed objects.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org