You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Vahid Hashemian (JIRA)" <ji...@apache.org> on 2018/07/17 01:25:00 UTC
[jira] [Commented] (KAFKA-7167) old segments not deleted despite
short retention.ms
[ https://issues.apache.org/jira/browse/KAFKA-7167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16545920#comment-16545920 ]
Vahid Hashemian commented on KAFKA-7167:
----------------------------------------
I tried the 1.1.0 release with the config you specified (and a fresh topic), and was not able to reproduce the issue. Log segments are getting deleted as expected.
> old segments not deleted despite short retention.ms
> ---------------------------------------------------
>
> Key: KAFKA-7167
> URL: https://issues.apache.org/jira/browse/KAFKA-7167
> Project: Kafka
> Issue Type: Bug
> Components: log
> Affects Versions: 1.1.0
> Reporter: Christian Tramnitz
> Priority: Critical
>
> We have a three node Kafka cluster running 1.1.0 and (global) log.retentions.ms configured to 7200000 (2h). (global) log.roll.ms=300000 is set to ensure we have new segments every 5 minutes, to clean up old ones granularly.
> The topic policy has cleanup.policy = delete.
>
> I can see some old segments being deleted, i.e.
> Jul 16 08:59:17 server460 kafka[26553]: [2018-07-16 08:59:17,321] INFO [Log partition=test-topic-6, dir=/kafka] Deleting segment 18702312 (kafka.log.Log)
> Jul 16 08:59:17 server460 kafka[26553]: [2018-07-16 08:59:17,329] INFO Deleted log /kafka/test-topic-6/00000000000018702312.log.deleted. (kafka.log.LogSegment)
> Jul 16 08:59:17 server460 kafka[26553]: [2018-07-16 08:59:17,329] INFO Deleted offset index /kafka/test-topic-6/00000000000018702312.index.deleted. (kafka.log.LogSegment)
> Jul 16 08:59:17 server460 kafka[26553]: [2018-07-16 08:59:17,329] INFO Deleted time index /kafka/test-topic-6/00000000000018702312.timeindex.deleted. (kafka.log.LogSegment)
> But apparently not all of them. There are still old(er!) segments in the topic's dir, ultimately growing to fill-up the entire disk.
>
> Are there any other configuration values that may affect old segment deletion or is this a bug?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)