You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by Jay Stricks <ja...@wapolabs.com> on 2012/02/09 16:02:47 UTC

Processing Data During Master Outage

Hi folks,

I'm having stability problems with my master, and until we get it resolved,
I am curious how you would recommend getting Flume to handle the data that
was generated on the agent 1) during the outage, and then 2) until the flow
configuration settings were processed by the master once it was back up. Is
this data actively being collected by the agents until it gets the configs
from the Master and then forwards it on to the collectors? If not, is there
a way to take access_log files and have them be processed from the missing
time stamp?  Any advice would be greatly appreciated.

The agents have the following untranslated sources and sinks:

*Source: *tail("/var/log/httpd/access_log","true")
*Sink: *{value("app","ngnaccesslogs") => autoE2EChain }

*Source: *syslogUdp(5140)
*Sink: *{value("app","ngnfb") => autoE2EChain }




Thanks,

Jay Stricks