Will Cache will get invalidated if i read lots of messages using random offsets and partitions

[Rupesh Kumar <Ru...@walmartlabs.com.INVALID>]

Hi Team,

I understand that Kafka is meant for sequential access, but I have use case of accessing the messages from Kafka based on random offsets and partitions.

For Example

There is topic called “topic-A” some consumers are listening from this topic, it may happen that processing of these messages gets failed inside consumer so in that case we will have to reprocess those messages.
We store the offset and partition details for the failed messages.

Another consumer will process these failed messages and put again to the topic, so to reprocess those messages again we will be fetching those messages from respective
topics based on offsets and partitions. So in this case we will have lots of random offsets to access from different topics (our application should support 1000 messages/second including all topics).


So now my doubt comes here
     Will accessing messages using random offsets and partitions invalidate the cache for the topic that was used for sequential access (other consumers was reading sequentially).

     I want to understand the impact of this random offset, partition access on kafka,


  1.  Will it slow down Kafka ?
  2.  Will it read messages from cache or from disk ?
  3.  If messages is not in cache then will it load from disk and store in cache … in that case what will happen to the existing data that was in cache and was being used for sequential access ?

I went through Kafka documentation but couldn’t  answer to all my questions.
Need your help here.

Please let me know if my question is not clear or you need more info.

Regards
Rupesh