[jira] [Resolved] (KAFKA-6854) Log cleaner fails with transaction markers that are deleted during clean

["Jason Gustafson (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-6854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jason Gustafson resolved KAFKA-6854.
------------------------------------
    Resolution: Fixed

> Log cleaner fails with transaction markers that are deleted during clean
> ------------------------------------------------------------------------
>
>                 Key: KAFKA-6854
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6854
>             Project: Kafka
>          Issue Type: Task
>          Components: core
>    Affects Versions: 1.1.0
>            Reporter: Rajini Sivaram
>            Assignee: Rajini Sivaram
>            Priority: Blocker
>             Fix For: 2.0.0, 1.0.2, 1.1.1
>
>
> Log cleaner grows buffers when `result.messagesRead` is zero. In a typical scenario, this is a result of source buffer being too small to read the first batch. The buffer is then doubled in size until one batch can be read, up to a maximum of `max.message.size`. There are issues with the maximum message size used in calculations as reported in KAFKA-6834. But there is a separate issue with the use of `result.messagesRead` when transactions are used. This contains the number of filtered messages read from source which can be zero when a transaction control marker is discarded. Log cleaner incorrectly assumes that messages were not read because the buffer was too small. This can result in the log cleaner attempting to grow buffers to double the buffer size unnecessarily, failing with an exception if the buffer is already `max.message.bytes`. This kills the log cleaner.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)