[jira] [Resolved] (KAFKA-21000) Potential data corruption on clean broker restart

["Jun Rao (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-21000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jun Rao resolved KAFKA-21000.
-----------------------------
    Fix Version/s: 4.5.0
         Assignee: Tushar Saini  (was: Ken Huang)
       Resolution: Fixed

merged the PR to trunk.

> Potential data corruption on clean broker restart
> -------------------------------------------------
>
>                 Key: KAFKA-21000
>                 URL: https://issues.apache.org/jira/browse/KAFKA-21000
>             Project: Kafka
>          Issue Type: Improvement
>          Components: core
>            Reporter: Jun Rao
>            Assignee: Tushar Saini
>            Priority: Major
>             Fix For: 4.5.0
>
>
> A similar issue to https://issues.apache.org/jira/browse/KAFKA-20979 could be exposed in the .log file when preallocation is used.
> When the active segment is closed, FileRecords.close() calls the following.
> {code:java}
> flush();
> trim();{code}
> When preallocation is used, trim() resizes the file length to what's actually being used, but without flushing the length metadata. The broker cleanly shuts down and crashes before the metadata is flushed. When the broker is restarted, the on-disk file length still has the preallocation length but won't go through recovery (since the broker was shutdown cleanly). When the broker tries to calculate the next offset, it will hit the uninitialized bytes and cause an exception.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)