You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flume.apache.org by Lior Harel <ha...@gmail.com> on 2011/08/31 12:59:50 UTC

agent restart resend tailed file data

Hi,

we're using flume agent to tail a directory of log files, we noticed that
when the node is restarted it resends the tailed files. is there any
configuration that can overcome this problem?

Thanks

Re: agent restart resend tailed file data

Posted by NerdyNick <ne...@gmail.com>.

right now the only way to do this is to set the startFromEnd = true.
This will trip tail to just pick back up at the end of the file vs the
start on restart. Do note you can loose events during the restart
because of this. I believe there is a bug in to extend this feature
and record the position in the file to disk, but do not recall the
status of it.

On Wed, Aug 31, 2011 at 4:59 AM, Lior Harel <ha...@gmail.com> wrote:
> Hi,
> we're using flume agent to tail a directory of log files, we noticed that
> when the node is restarted it resends the tailed files. is there any
> configuration that can overcome this problem?
> Thanks

-- 
Nick Verbeck - NerdyNick
----------------------------------------------------
NerdyNick.com
Coloco.ubuntu-rocks.org