You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "Adam Faris (Jira)" <ji...@apache.org> on 2021/05/12 22:40:00 UTC

[jira] [Created] (SAMZA-2655) Expose RocksDB maxOpenFiles setting as Samza parameter

Adam Faris created SAMZA-2655:
---------------------------------

             Summary: Expose RocksDB maxOpenFiles setting as Samza parameter
                 Key: SAMZA-2655
                 URL: https://issues.apache.org/jira/browse/SAMZA-2655
             Project: Samza
          Issue Type: Improvement
          Components: kv-store
            Reporter: Adam Faris


This is a feature request to expose RocksDB's maxOpenFiles setting as a Samza Parameter.  It's needed to prevent large databases from opening more files than what's allowed by ulimit settings and causing the container to fail.

https://github.com/facebook/rocksdb/blob/master/include/rocksdb/options.h#L460
{noformat}
// Number of open files that can be used by the DB.  You may need to  
// increase this if your database has a large working set. Value -1 means  
// files opened are always kept open. You can estimate number of files based 
// on target_file_size_base and target_file_size_multiplier for level-based  
// compaction. For universal-style compaction, you can usually set it to -1.  
// {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)