[jira] [Assigned] (KAFKA-19971) Tiered Storage Fetches Fail with IllegalStateException During Index Cache Eviction Window

["Nandini Singhal (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-19971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Nandini Singhal reassigned KAFKA-19971:
---------------------------------------

    Assignee: Nandini Singhal

> Tiered Storage Fetches Fail with IllegalStateException During Index Cache Eviction Window
> -----------------------------------------------------------------------------------------
>
>                 Key: KAFKA-19971
>                 URL: https://issues.apache.org/jira/browse/KAFKA-19971
>             Project: Kafka
>          Issue Type: Bug
>          Components: Tiered-Storage
>            Reporter: Nandini Singhal
>            Assignee: Nandini Singhal
>            Priority: Major
>
> Remote log fetches can fail with IllegalStateException: This entry is marked for cleanup when the index cache is under pressure and evicting entries. A race condition exists between entry eviction and fetch operations, causing requests to fail even though the data could be re-fetched from remote storage.
> When the remote index cache reaches its capacity and needs to evict entries, a multi-step eviction process creates a window where in-flight fetch requests can fail unexpectedly:
>   1. Cache eviction triggers removal of an index entry
>   2. Entry is marked for cleanup (markedForCleanup = true)
>   3. Index files are renamed to .deleted suffix
>   4. Cleanup is scheduled to run after a delay (default: 10 seconds)
>   5. After the delay, cleanup sets cleanStarted = true and deletes files
>   6. During this window, if a fetch request references this entry, it fails with:
>   java.lang.IllegalStateException: This entry is marked for cleanup
>  
> Expected behavior: Should treat as cache miss and re-fetch from remote storage (or handle gracefully).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)