[jira] [Resolved] (HDDS-3160) Disable index and filter block cache for RocksDB

["Marton Elek (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HDDS-3160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Marton Elek resolved HDDS-3160.
-------------------------------
    Fix Version/s: 0.6.0
       Resolution: Fixed

> Disable index and filter block cache for RocksDB
> ------------------------------------------------
>
>                 Key: HDDS-3160
>                 URL: https://issues.apache.org/jira/browse/HDDS-3160
>             Project: Hadoop Distributed Data Store
>          Issue Type: Improvement
>            Reporter: Marton Elek
>            Assignee: Marton Elek
>            Priority: Minor
>              Labels: pull-request-available
>             Fix For: 0.6.0
>
>         Attachments: key_allocation_after.png, key_allocation_before.png, profile.png
>
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> During preformance tests It was noticed that the OM performance is dropped after 10-20 million of keys. (see the screenshot).
> By default cache_index_and_filter_blocks is enabled for all of our RocksDB instances (see DBProfile) which is not the best option. (For example see this thread: https://github.com/facebook/rocksdb/issues/3961#)
> With turning on this cache the indexes and bloom filters are cached **inside the block cache** which makes slower the cache when we have significant data.
> Without turning it on (based on my understanding) all the indexes will remain open without any cache. With our current settings we have only a few number of sst files (even with million of keys) therefore it seems to be safe to turn this option off.
> With turning this option of I was able to write >100M keys with high throughput. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org