[jira] [Commented] (KAFKA-3788) Potential message lost when switching to new segment

["Bharat Viswanadham (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-3788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15321014#comment-15321014 ] 

Bharat Viswanadham commented on KAFKA-3788:
-------------------------------------------

Can I take this task, to work on it?

> Potential message lost when switching to new segment
> ----------------------------------------------------
>
>                 Key: KAFKA-3788
>                 URL: https://issues.apache.org/jira/browse/KAFKA-3788
>             Project: Kafka
>          Issue Type: Bug
>          Components: log
>    Affects Versions: 0.9.0.0, 0.9.0.1, 0.10.0.0
>            Reporter: Arkadiusz Firus
>            Assignee: Jay Kreps
>            Priority: Minor
>              Labels: easyfix
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> If a new segment is needed method roll() from class kafka.log.Log is invoked. It prepares new segment and schedules _asynchronous_ flush of the previous segment.
> Asynchronous call can lead to a problematic situation. As far as I know neither Linux nor Windows guarantees that the order of files persisted to disk will be the same as the order of writes to files. This means that records from the new segment can be flushed before the old ones which in case of power outage can lead to gaps between records.
> Changing asynchronous invocation to synchronous one will solve the problem because we have guarantee that all records from the previous segment will be persisted to hard drive before we write any record to the new segment.
> I am guessing that asynchronous invocation was chosen to increase performance but switching between segments is not so often. So it is not a big gain.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)