Events are lost by "flume node stop" using fail over sink

["Y. Sakamoto" <ks...@muc.biglobe.ne.jp>]

Hi,
Using fail over sink on agent side ( < agentBESink ? text > ) ,
I stopped collector by "flume node stop" command.
( Agent is still working. )

Then agent became writing to fail over sink,
but some events were lost.

I think following phenomenon :
  "flume node stop" command kills collector process,
  then agent will detects and switch sink to fail over.
  But the events, which are already send
  and which are not written on collector's sink yet, will be lost.


How can I prevent from losting events on stopping collector ?
Is there any setup to flush events on stopping collector ?


Thanks,
  Y. Sakamoto.

Re: Events are lost by "flume node stop" using fail over sink

[Lior Harel <ha...@gmail.com>]

are you sure you're using disk fail over sink?
in your mail you wrote agentBESink, that stands for best effort not disk
fail over.
from the docs:
Best Effort. This agent does not write to disk at all, and drops messages
in the event of collector failures.


2011/11/16 Y. Sakamoto <ks...@muc.biglobe.ne.jp>

> Hi,
> Using fail over sink on agent side ( < agentBESink ? text > ) ,
> I stopped collector by "flume node stop" command.
> ( Agent is still working. )
>
> Then agent became writing to fail over sink,
> but some events were lost.
>
> I think following phenomenon :
>  "flume node stop" command kills collector process,
>  then agent will detects and switch sink to fail over.
>  But the events, which are already send
>  and which are not written on collector's sink yet, will be lost.
>
>
> How can I prevent from losting events on stopping collector ?
> Is there any setup to flush events on stopping collector ?
>
>
> Thanks,
>   Y. Sakamoto.
>
>