[jira] [Created] (KAFKA-15084) Remove lock contention in RemoteIndexCache

["Divij Vaidya (Jira)" <ji...@apache.org>]

Divij Vaidya created KAFKA-15084:
------------------------------------

             Summary: Remove lock contention in RemoteIndexCache
                 Key: KAFKA-15084
                 URL: https://issues.apache.org/jira/browse/KAFKA-15084
             Project: Kafka
          Issue Type: Sub-task
          Components: core
    Affects Versions: 3.6.0
            Reporter: Divij Vaidya
            Assignee: Divij Vaidya
             Fix For: 3.6.0


RemoteIndexCache cache is accessed from multiple threads concurrently in the fetch from consumer code path [1]. 

Currently, the RemoteIndexCache uses LinkedHashMap as the cache implementation internally. Since LinkedHashMap is not a thread safe data structure, we use coarse grained lock on the entire map/cache when writing to the cache.

This means that if a thread if fetching information from a particular segment from RemoteStorageManager, other threads who are trying to access a different segment from the cache will also wait for the former thread to complete. This is due to the usage of global lock in the cache.

This lock contentions leads to decrease in throughput for fetch from consumer for cases where RSM network call may take more time.

As a goal for this JIRA, we would like to ensure that the threads reading existing values in the cache do not get blocked when thread updating the cache is fetching data.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)