You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "hunshenshi (JIRA)" <ji...@apache.org> on 2017/06/01 10:14:04 UTC
[jira] [Created] (FLUME-3101) taildir source may endless loop when
tail a file
hunshenshi created FLUME-3101:
---------------------------------
Summary: taildir source may endless loop when tail a file
Key: FLUME-3101
URL: https://issues.apache.org/jira/browse/FLUME-3101
Project: Flume
Issue Type: Bug
Components: Sinks+Sources
Affects Versions: 1.7.0
Reporter: hunshenshi
If there are many files in the path that need to be tail, and there is a file written by *high frequency* (for example, there are file a, file b and file c in the path, file a is written at high frequency), *taildir can read the batchSize size event from file a everytime*, then taildir will only read data from file a, other files will not to be read, because in TaildirSource.tailFileProcess will into an endless loop.
code:
{code:title=Bar.java|borderStyle=solid}
private void tailFileProcess(TailFile tf, boolean backoffWithoutNL)
throws IOException, InterruptedException {
while (true) {
// if events.size >= batchSize will not break while,
// then into endless loop to only read tf
if (events.size() < batchSize) {
break;
}
}
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)