[jira] [Resolved] (FLUME-503) Use HDFS sync API instead of rolling for durability

["Ashish Paliwal (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLUME-503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ashish Paliwal resolved FLUME-503.
----------------------------------
       Resolution: Won't Fix
    Fix Version/s: v0.9.5

Won't fix. 0.X branch not maintained anymore

> Use HDFS sync API instead of rolling for durability
> ---------------------------------------------------
>
>                 Key: FLUME-503
>                 URL: https://issues.apache.org/jira/browse/FLUME-503
>             Project: Flume
>          Issue Type: New Feature
>          Components: Sinks+Sources
>    Affects Versions: v0.9.4
>            Reporter: Disabled imported user
>             Fix For: v0.9.5
>
>
> Some versions of Hadoop (CDH3>b2 or 0.20-append branch) support a sync() API that guarantees data has been flushed to all of the nodes in the write pipeline. This should be equally as durable as closing an HDFS file.
> Flume should allow the use of sync() to make data durable on a regular basis without having to create lots of tiny files on HDFS.
> Related is the ability to use the getNumCurrentReplicas() API to detect when the number of replicas falls below the desired replication factor, and roll at that point (to pick up a new DN)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)