[jira] [Created] (KAFKA-10760) In compacted topic with max.compaction.lag.ms, the segments are not rolled until new messages arrive

["Sarwar Bhuiyan (Jira)" <ji...@apache.org>]

Sarwar Bhuiyan created KAFKA-10760:
--------------------------------------

             Summary: In compacted topic with max.compaction.lag.ms, the segments are not rolled until new messages arrive
                 Key: KAFKA-10760
                 URL: https://issues.apache.org/jira/browse/KAFKA-10760
             Project: Kafka
          Issue Type: Bug
          Components: core
            Reporter: Sarwar Bhuiyan


Currently, if a compacted topic has min.cleanable.dirty.ratio set to something low and max.compaction.lag.ms set to a small time, according to KIP [https://cwiki.apache.org/confluence/display/KAFKA/KIP-354] the expectation is that the active segment will be rolled regardless or segment.ms or whether new data has come in to "advance" the time. However, in practice, the current implementation only rolls the segment when new data which means that there are situations where the topic is not fully compacted until new data arrives which may not be until a while later. The implementation can be improved by rolling the segment just purely based on the max.compaction.lag.ms setting. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)