[jira] [Commented] (KAFKA-1379) Partition reassignment resets clock for time-based retention

["Luca Toscano (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-1379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15310068#comment-15310068 ] 

Luca Toscano commented on KAFKA-1379:
-------------------------------------

Hi Moritz,

thanks a lot for pointing us to this Jira in users@. At the moment we use a similar trick to resolve disk partitions filling up (retention.ms):
https://wikitech.wikimedia.org/wiki/Analytics/Cluster/Kafka/Administration#Temporarily_Modify_Per_Topic_Retention_Settings

I also opened a Phabricator task to track this problem https://phabricator.wikimedia.org/T136690

retention.bytes is definitely worth to try, but is there anything else that can mitigate this issue?

> Partition reassignment resets clock for time-based retention
> ------------------------------------------------------------
>
>                 Key: KAFKA-1379
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1379
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Joel Koshy
>
> Since retention is driven off mod-times reassigned partitions will result in
> data that has been on a leader to be retained for another full retention
> cycle. E.g., if retention is seven days and you reassign partitions on the
> sixth day then those partitions will remain on the replicas for another
> seven days.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)