You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@kafka.apache.org by "Jun Rao (Jira)" <ji...@apache.org> on 2022/10/19 00:49:00 UTC

[jira] [Resolved] (KAFKA-14321) max.compaction.lag.ms is not enforced accurately

     [ https://issues.apache.org/jira/browse/KAFKA-14321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jun Rao resolved KAFKA-14321.
-----------------------------
    Resolution: Duplicate

This actually duplicates KAFKA-10760. Closing this one.

> max.compaction.lag.ms is not enforced accurately
> ------------------------------------------------
>
>                 Key: KAFKA-14321
>                 URL: https://issues.apache.org/jira/browse/KAFKA-14321
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Jun Rao
>            Priority: Major
>
> Compaction only cleans data in non-active segments. When max.compaction.lag.ms is set, we use it to set segment.ms to force segment rolling by time. However, the current implementation of time-based segment roll is not precise. It only rolls a segment if the new record's timestamp differs from the timestamp of the first record in the segment by more than segment.ms. If we have a bunch of records appended within segment.ms and then stop producing new records, all those records could remain in the active segments forever, which prevents the records to be cleaned.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)