You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Michael Axiak (JIRA)" <ji...@apache.org> on 2019/05/20 15:39:00 UTC
[jira] [Updated] (KAFKA-8395) Add an ability to backup log segment
files on truncation
[ https://issues.apache.org/jira/browse/KAFKA-8395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Axiak updated KAFKA-8395:
---------------------------------
Flags: Patch
> Add an ability to backup log segment files on truncation
> --------------------------------------------------------
>
> Key: KAFKA-8395
> URL: https://issues.apache.org/jira/browse/KAFKA-8395
> Project: Kafka
> Issue Type: New Feature
> Components: core
> Affects Versions: 2.2.0
> Reporter: Michael Axiak
> Priority: Minor
> Fix For: 2.2.1
>
>
> At HubSpot, we believe we hit a combination of bugs [1] [2], which may have caused us to lose data. In this scenario, as part of metadata conflict resolution a slowly starting up broker recovered an offset of zero and truncated segment files.
> As part of a belt-and-suspenders approach to reducing this risk in the future, I propose adding the ability to rename/backup these files and allowing kafka to move on. Note that this breaks the ordering guarantees, but allows one to recover the data and decide later how to approach it.
> This feature should be turned off by default but enabled with a configuration option.
> (A pull request is following soon on Github)
> 1: https://issues.apache.org/jira/browse/KAFKA-2178
> 2: https://issues.apache.org/jira/browse/KAFKA-1120
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)