You are viewing a plain text version of this content. The canonical link for it is here.

Posted to jira@kafka.apache.org by " Brajesh Kumar (Jira)" <ji...@apache.org> on 2020/12/14 21:18:00 UTC

[jira] [Assigned] (KAFKA-10760) In compacted topic with max.compaction.lag.ms, the segments are not rolled until new messages arrive

     [ https://issues.apache.org/jira/browse/KAFKA-10760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

 Brajesh Kumar reassigned KAFKA-10760:
--------------------------------------

    Assignee:  Brajesh Kumar

> In compacted topic with max.compaction.lag.ms, the segments are not rolled until new messages arrive
> ----------------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-10760
>                 URL: https://issues.apache.org/jira/browse/KAFKA-10760
>             Project: Kafka
>          Issue Type: Bug
>          Components: core
>            Reporter: Sarwar Bhuiyan
>            Assignee:  Brajesh Kumar
>            Priority: Major
>
> Currently, if a compacted topic has min.cleanable.dirty.ratio set to something low and max.compaction.lag.ms set to a small time, according to KIP [https://cwiki.apache.org/confluence/display/KAFKA/KIP-354] the expectation is that the active segment will be rolled regardless or segment.ms or whether new data has come in to "advance" the time. However, in practice, the current implementation only rolls the segment when new data which means that there are situations where the topic is not fully compacted until new data arrives which may not be until a while later. The implementation can be improved by rolling the segment just purely based on the max.compaction.lag.ms setting. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)