[jira] [Created] (KAFKA-16073) Tiered Storage Bug: Incorrect Handling of Offset Ranges During Segment Deletion

["hzh0425 (Jira)" <ji...@apache.org>]

hzh0425 created KAFKA-16073:
-------------------------------

             Summary: Tiered Storage Bug: Incorrect Handling of Offset Ranges During Segment Deletion
                 Key: KAFKA-16073
                 URL: https://issues.apache.org/jira/browse/KAFKA-16073
             Project: Kafka
          Issue Type: Bug
          Components: core, Tiered-Storage
    Affects Versions: 3.6.1
            Reporter: hzh0425
            Assignee: hzh0425
             Fix For: 3.6.1


This bug pertains to Apache Kafka's tiered storage functionality. Specifically, it involves a timing issue in the {{UnifiedLog.deleteSegments}} method. The method first deletes segments from memory but delays updating the {{{}localLogStartOffset{}}}. Meanwhile, in {{{}ReplicaManager.handleOffsetOutOfRangeError{}}}, if the fetch offset is less than {{{}localLogStartOffset{}}}, it triggers the read remote process. However, if it's greater, an {{OffsetOutOfRangeException}} is sent to the client.

Consider a scenario with concurrent operations, where {{{}offset1 < offset2 < offset3{}}}. A client requests {{offset2}} while a background thread is deleting segments. The segments are deleted in memory, but {{LocalLogStartOffset}} is still at {{offset1}} and not yet updated to {{{}offset3{}}}. In this state, since {{offset2}} is greater than {{{}offset1{}}}, {{ReplicaManager.handleOffsetOutOfRangeError}} erroneously returns an {{OffsetOutOfRangeException}} to the client. This happens because the system has not yet recognized the new starting offset ({{{}offset3{}}}), leading to incorrect handling of fetch requests.
 
 
 
 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)