[jira] [Commented] (KAFKA-13723) max.compaction.lag.ms implemented incorrectly

["Jun Rao (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-13723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17503813#comment-17503813 ] 

Jun Rao commented on KAFKA-13723:
---------------------------------

[~xiongqiwu] and [~jjkoshy]  : Could you check if this is a real issue? Thanks.

> max.compaction.lag.ms implemented incorrectly
> ---------------------------------------------
>
>                 Key: KAFKA-13723
>                 URL: https://issues.apache.org/jira/browse/KAFKA-13723
>             Project: Kafka
>          Issue Type: Bug
>          Components: core
>    Affects Versions: 2.3.0
>            Reporter: Jun Rao
>            Priority: Major
>
> In https://issues.apache.org/jira/browse/KAFKA-7321, we introduced max.compaction.lag.ms to guarantee that a record be cleaned before a certain time. 
>  
> The implementation in LogCleanerManager has the following code. The path for earliestDirtySegmentTimestamp < cleanUntilTime seems incorrect. In that case, it seems that we should set the delay to 0 so that we could trigger cleaning immediately since the segment has been dirty for longer than max.compaction.lag.ms. 
>  
>  
> {code:java}
> def maxCompactionDelay(log: UnifiedLog, firstDirtyOffset: Long, now: Long) : Long = {
> ...
> val maxCompactionLagMs = math.max(log.config.maxCompactionLagMs, 0L)
> val cleanUntilTime = now - maxCompactionLagMs
> if (earliestDirtySegmentTimestamp < cleanUntilTime)
> cleanUntilTime - earliestDirtySegmentTimestamp
> else
> 0L
> }{code}
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)