Several taildir issues.

[Feng Longda <hu...@gmail.com>]

Hi,



Recently, I want one tail logs tools, I found Flume is the right
tools. But when I study taildir source code. I found several issues.



1.  TailFile.readEvent just read one line every time, but in fact it
read BUFFER_SIZE, Sometimes it contain multiple lines, but the next
line will be read in the next time, it waste time.

2.  If there are huge files in the monitored directory, it will cost
much time on scanning directory every time when new file is coming or
one file has been removed, at the same time updating position to
positionFile will cost much time too.

3.  Flume doesn’t support monitory subdirectory, for example
/home/hadoop/logs/*/*.log， it will occur exception in the
TaildirMatcher.getMatchingFilesNoCache.





Regards

Longda