[jira] [Created] (ASTERIXDB-1898) Processed files handling issue in localfs feed after restart

["Xikui Wang (JIRA)" <ji...@apache.org>]

Xikui Wang created ASTERIXDB-1898:
-------------------------------------

             Summary: Processed files handling issue in localfs feed after restart
                 Key: ASTERIXDB-1898
                 URL: https://issues.apache.org/jira/browse/ASTERIXDB-1898
             Project: Apache AsterixDB
          Issue Type: Improvement
            Reporter: Xikui Wang


If we create a localfs feed monitoring a local directory, start, stop then start feed again will cause the second start feed to read those files that have been processed by the first start feed statement, including the one is being processed. 

Currently we don't have a very good way to keep track of the files that have been processed. The FileSystemWatcher is re-created with every start feed statement as it creates a new job specification. Even we keep a single instance file watcher, it's very hard to track all processed files.

For now, we could only warn users to remove those processed files out of the directory before the start the feed again.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)