You are viewing a plain text version of this content. The canonical link for it is here.

Posted to jira@kafka.apache.org by "Divij Vaidya (Jira)" <ji...@apache.org> on 2023/09/19 16:59:00 UTC

[jira] [Comment Edited] (KAFKA-15477) Kafka won't shutdown when deleting remote segments

    [ https://issues.apache.org/jira/browse/KAFKA-15477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17766880#comment-17766880 ] 

Divij Vaidya edited comment on KAFKA-15477 at 9/19/23 4:58 PM:
---------------------------------------------------------------

Ideally, deletion of remote segments (from Kafka perspective) is simply deleting the metadata from RLMM (which should be fast and not time consuming). But in 3.6 implementation, we synchronously delete the segments from remote storage, that is why you are facing this problem. The fix is scheduled for next version release. You can track it at https://issues.apache.org/jira/browse/KAFKA-15313 

The fix basically is to make deletion faster by having the process of deleting data from remote storage independent of Kafka broker and making it as the responsibility of RSM plugin. 


was (Author: divijvaidya):
Ideally, deletion of remote segments (from Kafka perspective) is simply deleting the metadata from RLMM. But in 3.6 implementation, we synchronously delete the segments from remote storage, that is why you are facing this problem. The fix is scheduled for next version release. You can track it at https://issues.apache.org/jira/browse/KAFKA-15313 

The fix basically is to make deletion faster by having the process of deleting data from remote storage independent of Kafka broker and making it as the responsibility of RSM plugin. 

> Kafka won't shutdown when deleting remote segments
> --------------------------------------------------
>
>                 Key: KAFKA-15477
>                 URL: https://issues.apache.org/jira/browse/KAFKA-15477
>             Project: Kafka
>          Issue Type: Bug
>          Components: core
>    Affects Versions: 3.6.0
>            Reporter: Francois Visconte
>            Priority: Minor
>
> When brokers are busy deleting a bunch of segments (following a topic removal using tiered storage), brokers won't respond to sigterm signal and cleanly shutdown. 
> Intead, they keep removing remote segment until it's fully completed (which can take time for topics with long retention).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)