[jira] [Resolved] (KAFKA-12177) Retention is not idempotent

["Jun Rao (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jun Rao resolved KAFKA-12177.
-----------------------------
    Fix Version/s: 3.0.0
         Assignee: Lucas Bradstreet
       Resolution: Fixed

merged the PR to trunk

> Retention is not idempotent
> ---------------------------
>
>                 Key: KAFKA-12177
>                 URL: https://issues.apache.org/jira/browse/KAFKA-12177
>             Project: Kafka
>          Issue Type: Improvement
>            Reporter: Lucas Bradstreet
>            Assignee: Lucas Bradstreet
>            Priority: Minor
>             Fix For: 3.0.0
>
>
> Kafka today applies retention in the following order:
>  # Time
>  # Size
>  # Log start offset
> Today it is possible for a segment with offsets less than the log start offset to contain data that is not deletable due to time retention. This means that it's possible for log start offset retention to unblock further deletions as a result of time based retention. Note that this does require a case where the max timestamp for each segment increases, decreases and then increases again. Even so it would be nice to make retention idempotent by applying log start offset retention first, followed by size and time. This would also be potentially cheaper to perform as neither log start offset and size retention require the maxTimestamp for a segment to be loaded from disk after a broker restart.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)