[jira] [Commented] (KAFKA-1379) Partition reassignment resets clock for time-based retention

["Xavier Léauté (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-1379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14903212#comment-14903212 ] 

Xavier Léauté commented on KAFKA-1379:
--------------------------------------

This is a huge issue for us as well, since it requires we keep double the disk capacity on hand, in case one of our brokers or disks fails, which happens relatively often at our scale.

Alternatively, we have to go in and remove expired segments by hand, by comparing replicated segments with the partition leader, before disks run out of space.


> Partition reassignment resets clock for time-based retention
> ------------------------------------------------------------
>
>                 Key: KAFKA-1379
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1379
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Joel Koshy
>
> Since retention is driven off mod-times reassigned partitions will result in
> data that has been on a leader to be retained for another full retention
> cycle. E.g., if retention is seven days and you reassign partitions on the
> sixth day then those partitions will remain on the replicas for another
> seven days.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)