You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flume.apache.org by "Daniel Lanza García (JIRA)" <ji...@apache.org> on 2018/03/02 09:22:00 UTC
[jira] [Updated] (FLUME-3219) Taildir source: if file is renamed,
it is consumed again
[ https://issues.apache.org/jira/browse/FLUME-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Lanza García updated FLUME-3219:
---------------------------------------
Issue Type: Improvement (was: Bug)
> Taildir source: if file is renamed, it is consumed again
> --------------------------------------------------------
>
> Key: FLUME-3219
> URL: https://issues.apache.org/jira/browse/FLUME-3219
> Project: Flume
> Issue Type: Improvement
> Components: Sinks+Sources
> Affects Versions: 1.8.0
> Reporter: Daniel Lanza García
> Priority: Major
>
> Current behavior of Taildir is such that if a file is renamed (eg log rotated) it is consumed again.
> https://github.com/apache/flume/blob/d1f24f56ce9714bb3e1edc671da290c75a17dead/flume-ng-sources/flume-taildir-source/src/main/java/org/apache/flume/source/taildir/ReliableTaildirEventReader.java#L247
> Would not be better if the inode is followed, and if that inode has been consumed do not consume it again? With current implementation, once file is rotated, you get duplicates in the case you include in the path previous days's data (you want to do that if agent fails and needs to consume data from previous days).
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@flume.apache.org
For additional commands, e-mail: issues-help@flume.apache.org