[jira] [Created] (FLUME-2777) Tail Dir Source leads to duplicate events on rolling the tailed file

["Johny Rufus (JIRA)" <ji...@apache.org>]

Johny Rufus created FLUME-2777:
----------------------------------

             Summary: Tail Dir Source leads to duplicate events on rolling the tailed file
                 Key: FLUME-2777
                 URL: https://issues.apache.org/jira/browse/FLUME-2777
             Project: Flume
          Issue Type: Bug
          Components: Sinks+Sources
    Affects Versions: 1.7
            Reporter: Johny Rufus
            Assignee: Johny Rufus


I have a simple setup, where I write 200 events to logfile1. [TailSrc is on the lookout for logfile* ]
Then I rename logfile1 to logfile2.
I create a new logfile1 and write 100 events to it.

Typically I should see 300 events in my channel. But I see 500 events.
I was able to trace the duplicates to ReliableTaildirEventReader.java updateFiles(boolean) to the way renamed files are handled , by specifying starting position as 0. [This starting position should be obtained from tf.getPosition()]

I am attaching a proposed fix, would be great if one of you guys [~iijima_satoshi] / [~hshreedharan]/ [~roshan_naik] can take a look at the fix and validate the issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)