You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Divij Vaidya (Jira)" <ji...@apache.org> on 2023/09/07 14:42:00 UTC

[jira] [Commented] (KAFKA-15352) Ensure consistency while deleting the remote log segments

    [ https://issues.apache.org/jira/browse/KAFKA-15352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17762768#comment-17762768 ] 

Divij Vaidya commented on KAFKA-15352:
--------------------------------------

PR: 
[https://github.com/apache/kafka/pull/14349] 

> Ensure consistency while deleting the remote log segments
> ---------------------------------------------------------
>
>                 Key: KAFKA-15352
>                 URL: https://issues.apache.org/jira/browse/KAFKA-15352
>             Project: Kafka
>          Issue Type: Sub-task
>            Reporter: Kamal Chandraprakash
>            Assignee: Christo Lolov
>            Priority: Blocker
>             Fix For: 3.6.0
>
>
> In Kafka-14888, the remote log segments are deleted which breaches the retention time/size before updating the log-start-offset. In middle of deletion, if the consumer starts to read from the beginning of the topic, then it will fail to read the messages and UNKNOWN_SERVER_ERROR will be thrown back to the consumer.
> To ensure consistency, similar to local log segments where the actual segments are deleted after {{{}segment.delete.delay.ms{}}}, we should update the log-start-offset first before deleting the remote log segment.
> See the [PR#13561|https://github.com/apache/kafka/pull/13561] and [comment|https://github.com/apache/kafka/pull/13561#discussion_r1293086543] for more details.
> Case-2:
> The old-leader (follower) can delete the remote log segment in the middle of leader election. We need to update the log-start-offset metadata for this case.
> See this comment [https://github.com/apache/kafka/pull/13561#discussion_r1293081560]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)