[jira] [Updated] (FLUME-1432) FileChannel should replay logs in order of them being written

["Brock Noland (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLUME-1432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Brock Noland updated FLUME-1432:
--------------------------------

    Affects Version/s: v1.2.0
    
> FileChannel should replay logs in order of them being written
> -------------------------------------------------------------
>
>                 Key: FLUME-1432
>                 URL: https://issues.apache.org/jira/browse/FLUME-1432
>             Project: Flume
>          Issue Type: Bug
>          Components: Channel
>    Affects Versions: v1.2.0
>            Reporter: Brock Noland
>            Assignee: Brock Noland
>
> Currently we replay the logs one at a time causing us to build large queue of pending takes. Additionally, there maybe scenerios where this simply will not work. Take a queue which is full (via checkpoint) and two files:
> 1:
> put
> commit
> put
> commit
> 2:
> take
> commit
> take
> commit
> take
> commit
> Replaying these logs in the current form will not work because we will we try and reply the puts first and exceed our queue size. For these reasons, we should replay them in the order they were written.
> However, at present there is no way to do this. Currently we have two identifers in each record we write, a transaction id and a timestamp. Neither can be used in replaying logs in order because the transaction id is created when we create the transaction not when we write to the log. Someone could create transaction, sleep, and then do work. The timestamp its not granular enough as we could have duplicates.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira