You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ozone.apache.org by 617072740 <61...@qq.com> on 2020/06/12 08:20:04 UTC
Merge or Remove Rocksdb in datanode ?
Hi all,
One rocksdb for each container in datanode cost too much off-heap memory. There are too choices to handle this: Merge Rocksdb or Remove Rocksdb, the design doc of these two choices: https://docs.google.com/document/d/18Ybg-NjyU602c-MYXaJHP6yrg-dVMZKGyoK5C_pp1mM/edit#. And the jira is https://issues.apache.org/jira/browse/HDDS-3630.
I would prefre to Merge Rocksdb, because Remove Rocksdb really is a big work and also need to do some work which could be done by rocksdb. The shortage of Remove Rocksdb is in the design doc.
If you have any suggestions, please let me know. Thank you very much.
Thanks,
runzhiwang
Re: Merge or Remove Rocksdb in datanode ?
Posted by Arpit Agarwal <aa...@cloudera.com.INVALID>.
Hi runzhiwang,
Thanks for highlighting this issue. Can we look at limiting the number of open RocksDB handles and also reducing cache allocation for each RocksDB?
> On Jun 12, 2020, at 1:20 AM, 617072740 <61...@qq.com> wrote:
>
> Hi all,
> One rocksdb for each container in datanode cost too much off-heap memory. There are too choices to handle this: Merge Rocksdb or Remove Rocksdb, the design doc of these two choices: https://docs.google.com/document/d/18Ybg-NjyU602c-MYXaJHP6yrg-dVMZKGyoK5C_pp1mM/edit#. And the jira is https://issues.apache.org/jira/browse/HDDS-3630.
> I would prefre to Merge Rocksdb, because Remove Rocksdb really is a big work and also need to do some work which could be done by rocksdb. The shortage of Remove Rocksdb is in the design doc.
> If you have any suggestions, please let me know. Thank you very much.
>
>
>
>
> Thanks,
> runzhiwang
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-dev-help@hadoop.apache.org