[jira] [Commented] (KAFKA-2997) Synchronous write to disk

["Grant Henke (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-2997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15060256#comment-15060256 ] 

Grant Henke commented on KAFKA-2997:
------------------------------------

I am curious about the motivation for synchronous writing to the file system. What benefit/functionality are you trying to achieve?

> Synchronous write to disk
> -------------------------
>
>                 Key: KAFKA-2997
>                 URL: https://issues.apache.org/jira/browse/KAFKA-2997
>             Project: Kafka
>          Issue Type: Improvement
>          Components: core
>    Affects Versions: 0.9.0.0
>            Reporter: Arkadiusz Firus
>            Priority: Minor
>              Labels: features, patch
>
> Hi All,
> I am currently work on a mechanism which allows to do an efficient synchronous writing to the file system. My idea is to gather few write requests for one partition and after that call the fsync.
> As I read the code I find out that the best place to do it is to modify:
> kafka.log.Log.append
> method. Currently at the end of the method (line 368) there is a verification if the number of unflushed messages is greater than the flush interval (configuration parameter).
> I am thinking of extending this condition. I want to add additional boolean configuration parameter (sync write or something like this). If this parameter is set to true at the end of this method the thread should hang on a lock. On the other hand there will be another timer thread (for every partition) which will be invoked every 10ms (configuration parameter). During invocation the thread will call flush method and after that will be releasing all hanged threads.
> I am writing here because I would like to know your opinion about such approach. Do you think this one is good or maybe someone have a better (more permanent) one. I would also like to know if such approach is according to general Kafka architecture.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)