[jira] [Commented] (KAFKA-5321) MemoryRecords.filterTo can return corrupt data if output buffer is not large enough

["Ismael Juma (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-5321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16030703#comment-16030703 ] 

Ismael Juma commented on KAFKA-5321:
------------------------------------

[~hachikuji], this was fixed as part of KAFKA-5316, right?

> MemoryRecords.filterTo can return corrupt data if output buffer is not large enough
> -----------------------------------------------------------------------------------
>
>                 Key: KAFKA-5321
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5321
>             Project: Kafka
>          Issue Type: Bug
>          Components: log
>            Reporter: Jason Gustafson
>            Assignee: Jason Gustafson
>            Priority: Blocker
>             Fix For: 0.11.0.0
>
>
> Due to KAFKA-5316, it is possible for a record set to grow during cleaning and overflow the output buffer allocated for writing. When we reach the record set which is doomed to overflow the buffer, there are two possibilities:
> 1. No records were removed and the original entry is directly appended to the log. This results in the overflow reported in KAFKA-5316.
> 2. Records were removed and a new record set is built. 
> Here we are concerned with the latter case.The problem is that the builder code automatically allocates a new buffer when we reach the end of the existing buffer and does not reset the position in the original buffer. Since {{MemoryRecords.filterTo}} continues using the old buffer, this can lead to data corruption after cleaning (the data left in the overflowed buffer is garbage). 
> Note that this issue could get fixed as part of a general solution KAFKA-5316, but if that seems too risk, we might fix this separately. A simple solution is to make both paths consistent and ensure that we raise an exception.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)