Question about [KIP-354]: Compaction can be delayed indefinitely in logs without traffic

[Christian Apolloni <ch...@gmail.com>]

Hello,

It's my understanding that [KIP-354] should ensure eligibility for compaction after the min.compaction.lag.ms is elapsed. The parameter should ensure that a new segment is rolled within the given time. By doing some tests and looking into the code, I think this is not always the case.

As far as I understand, the trigger for eventually rolling a new segment is only called when a new message is appended into the log. In low-traffic topics this could mean that no new message is received for a while. In the meantime the min.compaction.lag.ms can elapse but since no new message arrives, no new segment is rolled, causing the last active segment to remain uncompacted.

Is my understanding correct? If yes, is this behaviour unexpected or a known limitation of the compaction mechanism?

Kind regards,

-- 
Christian Apolloni