[jira] [Created] (KAFKA-14432) RocksDBStore relies on finalizers to not leak memory

["Lucas Brutschy (Jira)" <ji...@apache.org>]

Lucas Brutschy created KAFKA-14432:
--------------------------------------

             Summary: RocksDBStore relies on finalizers to not leak memory
                 Key: KAFKA-14432
                 URL: https://issues.apache.org/jira/browse/KAFKA-14432
             Project: Kafka
          Issue Type: Bug
            Reporter: Lucas Brutschy
            Assignee: Lucas Brutschy


Relying on finalizers in RocksDB has been deprecated for a long time, and starting with rocksdb 7, finalizers are removed completely (see [https://github.com/facebook/rocksdb/pull/9523]). 

Kafka Streams currently relies on finalizers in parts to not leak memory. This needs to be resolved before we can upgrade to RocksDB 7.

See  [https://github.com/apache/kafka/pull/12809] .

This is a native heap profile after running Kafka Streams without finalizers for a few hours:
Total: 13547.5 MB
 12936.3  95.5%  95.5%  12936.3  95.5% rocksdb::port::cacheline_aligned_alloc
   438.5   3.2%  98.7%    438.5   3.2% rocksdb::BlockFetcher::ReadBlockContents
    84.0   0.6%  99.3%     84.2   0.6% rocksdb::Arena::AllocateNewBlock
    45.9   0.3%  99.7%     45.9   0.3% prof_backtrace_impl
     8.1   0.1%  99.7%     14.6   0.1% rocksdb::BlockBasedTable::PutDataBlockToCache
     6.4   0.0%  99.8%  12941.4  95.5% Java_org_rocksdb_Statistics_newStatistics___3BJ
     6.1   0.0%  99.8%      6.9   0.1% rocksdb::LRUCacheShard::Insert@2d8b20
     5.1   0.0%  99.9%      6.5   0.0% rocksdb::VersionSet::ProcessManifestWrites
     3.9   0.0%  99.9%      3.9   0.0% rocksdb::WritableFileWriter::WritableFileWriter
     3.2   0.0%  99.9%      3.2   0.0% std::string::_Rep::_S_create



--
This message was sent by Atlassian Jira
(v8.20.10#820010)