[jira] [Created] (KAFKA-7023) Kafka Streams RocksDB bulk loading config may not be honored with customized RocksDBConfigSetter

["Liquan Pei (JIRA)" <ji...@apache.org>]

Liquan Pei created KAFKA-7023:
---------------------------------

             Summary: Kafka Streams RocksDB bulk loading config may not be honored with customized RocksDBConfigSetter 
                 Key: KAFKA-7023
                 URL: https://issues.apache.org/jira/browse/KAFKA-7023
             Project: Kafka
          Issue Type: Bug
          Components: streams
    Affects Versions: 1.1.0
            Reporter: Liquan Pei
            Assignee: Liquan Pei


We observed frequent L0 -> L1 compaction during Kafka Streams state recovery. 

During bulk loading, the following options are set: [https://github.com/facebook/rocksdb/blob/master/options/options.cc] 

Options*
Options::PrepareForBulkLoad()
{
 // never slowdown ingest.
 level0_file_num_compaction_trigger = (1<<30);
 level0_slowdown_writes_trigger = (1<<30);
 level0_stop_writes_trigger = (1<<30);
 soft_pending_compaction_bytes_limit = 0;
 hard_pending_compaction_bytes_limit = 0;

// no auto compactions please. The application should issue a
 // manual compaction after all data is loaded into L0.
 disable_auto_compactions = true;
 // A manual compaction run should pick all files in L0 in
 // a single compaction run.
 max_compaction_bytes = (static_cast<uint64_t>(1) << 60);

// It is better to have only 2 levels, otherwise a manual
 // compaction would compact at every possible level, thereby
 // increasing the total time needed for compactions.
 num_levels = 2;

// Need to allow more write buffers to allow more parallism
 // of flushes.
 max_write_buffer_number = 6;
 min_write_buffer_number_to_merge = 1;

// When compaction is disabled, more parallel flush threads can
 // help with write throughput.
 max_background_flushes = 4;

// Prevent a memtable flush to automatically promote files
 // to L1. This is helpful so that all files that are
 // input to the manual compaction are all at L0.
 max_background_compactions = 2;

// The compaction would create large files in L1.
 target_file_size_base = 256 * 1024 * 1024;
 return this;
}

Especially, those values are set to a very large number to avoid compactions and ensures files are all on L0. 

level0_file_num_compaction_trigger = (1<<30);
level0_slowdown_writes_trigger = (1<<30);
level0_stop_writes_trigger = (1<<30);

However, in RockDBStore.java, openDB code, we first call 

options.prepareForBulkLoad() and then use the configs from the customized customized RocksDBConfigSetter. This may overwrite the configs set in prepareBulkLoad call. The fix is to move prepareBulkLoad call after applying configs customized RocksDBConfigSetter. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)