[jira] [Commented] (KAFKA-1379) Partition reassignment resets clock for time-based retention

["Moritz Siuts (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-1379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301936#comment-15301936 ] 

Moritz Siuts commented on KAFKA-1379:
-------------------------------------

From the user-mailinglist:

{quote}
We’ve recently upgraded to 0.9.  In 0.8, when we restarted a broker, data
log file mtimes were not changed.  In 0.9, any data log file that was on
disk before the broker has it’s mtime modified to the time of the broker
restart.
{quote}

A workaround can be to set {{retention.bytes}} on a topic level, like this:

{noformat}
./bin/kafka-topics.sh --zookeeper X.X.X.X:2181/kafka -alter --config retention.bytes=5000000 –topic my_topic
{noformat}

The settings controls the max size in bytes of a partition oft he specified topic. So you can find a good size by checking the size of a partition with {{du -b}} and use this value.

> Partition reassignment resets clock for time-based retention
> ------------------------------------------------------------
>
>                 Key: KAFKA-1379
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1379
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Joel Koshy
>
> Since retention is driven off mod-times reassigned partitions will result in
> data that has been on a leader to be retained for another full retention
> cycle. E.g., if retention is seven days and you reassign partitions on the
> sixth day then those partitions will remain on the replicas for another
> seven days.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)