[jira] [Updated] (FLUME-3101) taildir source may endless loop when tail a file

["Ferenc Szabo (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLUME-3101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ferenc Szabo updated FLUME-3101:
--------------------------------
    Fix Version/s:     (was: 1.8.0)
                   1.9.0

> taildir source may endless loop when tail a file
> ------------------------------------------------
>
>                 Key: FLUME-3101
>                 URL: https://issues.apache.org/jira/browse/FLUME-3101
>             Project: Flume
>          Issue Type: Bug
>          Components: Sinks+Sources
>    Affects Versions: 1.7.0
>            Reporter: hunshenshi
>              Labels: patch, taildirsource
>             Fix For: 1.9.0
>
>         Attachments: FLUME-3101-0.patch, FLUME-3101-1.patch, FLUME-3101-2.patch, FLUME-3101-3.patch
>
>
> If there are many files in the path that need to be tail, and there is a file written by *high frequency* (for example, there are file a, file b and file c in the path, file a is written at high frequency), *taildir can read the batchSize size event from file a everytime*, then taildir will only read data from file a, other files will not to be read, because in TaildirSource.tailFileProcess will into an endless loop.
> code:
> {code:title=TaildirSource.java|borderStyle=solid}
> private void tailFileProcess(TailFile tf, boolean backoffWithoutNL)
>     throws IOException, InterruptedException {
>   while (true) {
>     // if events.size >= batchSize will not break while, 
>     // then into endless loop to only read tf
>     if (events.size() < batchSize) {
>       break;
>     }
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)