You are viewing a plain text version of this content. The canonical link for it is here.

Posted to jira@kafka.apache.org by "Jun Rao (Jira)" <ji...@apache.org> on 2022/03/09 20:05:00 UTC

[jira] [Created] (KAFKA-13723) max.compaction.lag.ms implemented incorrectly

Jun Rao created KAFKA-13723:
-------------------------------

             Summary: max.compaction.lag.ms implemented incorrectly
                 Key: KAFKA-13723
                 URL: https://issues.apache.org/jira/browse/KAFKA-13723
             Project: Kafka
          Issue Type: Bug
          Components: core
    Affects Versions: 2.3.0
            Reporter: Jun Rao


In https://issues.apache.org/jira/browse/KAFKA-7321, we introduced max.compaction.lag.ms to guarantee that a record be cleaned before a certain time. 

 

The implementation in LogCleanerManager has the following code. The path for earliestDirtySegmentTimestamp < cleanUntilTime seems incorrect. In that case, it seems that we should set the delay to 0 so that we could trigger cleaning immediately since the segment has been dirty for longer than max.compaction.lag.ms. 

 

 
{code:java}
def maxCompactionDelay(log: UnifiedLog, firstDirtyOffset: Long, now: Long) : Long = {

...

val maxCompactionLagMs = math.max(log.config.maxCompactionLagMs, 0L)
val cleanUntilTime = now - maxCompactionLagMs

if (earliestDirtySegmentTimestamp < cleanUntilTime)
cleanUntilTime - earliestDirtySegmentTimestamp
else
0L
}{code}
 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)