[GitHub] [flume] mingzhekai opened a new pull request #295: FLUME-3341 Fix:Taildir source may cause file handle leak and data duplication

[GitBox <gi...@apache.org>]

mingzhekai opened a new pull request #295: FLUME-3341 Fix:Taildir source may cause file handle leak and data duplication
URL: https://github.com/apache/flume/pull/295
 
 
   As is described in Flume-3342, renaming a file may cause data duplication. But moreover, if the original file was opened by flume before and not closed yet, flume reopens it while not freeing its original file handles. And then I delete the new file, but after the idle time, flume just closes the new file handle and forgets to close its original file handles, leading to a handle leak. It can be found by "lsof | grep "deleted".
   
   This bug is triggered by using Log4j to roll log files. To avoid losing data when rolling, I use regex to include both current log file and old log files in the taildir path.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services