[jira] [Created] (KAFKA-7910) Document retention.ms behavior with record timestamp

["Stanislav Kozlovski (JIRA)" <ji...@apache.org>]

Stanislav Kozlovski created KAFKA-7910:
------------------------------------------

             Summary: Document retention.ms behavior with record timestamp
                 Key: KAFKA-7910
                 URL: https://issues.apache.org/jira/browse/KAFKA-7910
             Project: Kafka
          Issue Type: Improvement
            Reporter: Stanislav Kozlovski


It is intuitive to believe that `log.retention.ms` starts applying once a log file is closed.
The documentation says:
> This configuration controls the maximum time we will retain a log before we will discard old log segments to free up space if we are using the "delete" retention policy.
Yet, the actual behavior is that we take into account the largest timestamp of that segment file ([https://github.com/apache/kafka/blob/4cdbb3e5c19142d118f0f3999dd3e21deccb3643/core/src/main/scala/kafka/log/Log.scala#L1246)|https://github.com/apache/kafka/blob/4cdbb3e5c19142d118f0f3999dd3e21deccb3643/core/src/main/scala/kafka/log/Log.scala#L1246).] and then consider `retention.ms` on top of that.

This means that if Kafka is configured with `log.message.timestamp.type=CreateTime` (as it is by default), any records that have a future timestamp set by the producer will not get deleted as expected by the initial intuition (and documentation) of the `log.retention.ms`.

We should document the behavior of `retention.ms` with the record timestamp.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)