You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by Matt Wise <ma...@nextdoor.com> on 2014/03/31 17:55:14 UTC
Safety of Automatic Reload...?
Is there a document anywhere describing what settings can/can't be expected
to update dynamically just by modifying the Flume NG agent config file
(1.4+)?
Matt Wise
Sr. Systems Architect
Nextdoor.com
Re: Safety of Automatic Reload...?
Posted by Matt Wise <ma...@nextdoor.com>.
Thanks for the help Hari... we switched to a non-restart version of our
puppet code push (we just push the config file, and let Flume handle it)
and things are working reasonably well!
Matt Wise
Sr. Systems Architect
Nextdoor.com
On Mon, Mar 31, 2014 at 1:09 PM, Hari Shreedharan <hshreedharan@cloudera.com
> wrote:
> I just re-read your question. On touching the config file (or modifying
> it), all parameters get updated, except the metrics/monitoring (for which
> you need to shutdown the Flume agent and restart with the new parameters on
> the command line).
>
> File Channel cannot be resized since the checkpoint is a fixed size file.
> You can delete your checkpoint files and their backups if you want to
> resize the file channel (make sure you shutdown the agent first). Be aware
> that a full replay without checkpoint will take time to complete, depending
> on how many data files you have and how many events are present in that.
>
>
> Thanks,
> Hari
>
> On Monday, March 31, 2014 at 11:12 AM, Hari Shreedharan wrote:
>
> I assume you don't want Flume to reload when the file is updated? If
> yes, just pass "--no-reload-conf" to the flume-ng command when starting the
> agent.
>
>
> Thanks,
> Hari
>
> On Monday, March 31, 2014 at 8:55 AM, Matt Wise wrote:
>
> Is there a document anywhere describing what settings can/can't be
> expected to update dynamically just by modifying the Flume NG agent config
> file (1.4+)?
>
> Matt Wise
> Sr. Systems Architect
> Nextdoor.com
>
>
>
>
Re: Safety of Automatic Reload...?
Posted by Hari Shreedharan <hs...@cloudera.com>.
I just re-read your question. On touching the config file (or modifying it), all parameters get updated, except the metrics/monitoring (for which you need to shutdown the Flume agent and restart with the new parameters on the command line).
File Channel cannot be resized since the checkpoint is a fixed size file. You can delete your checkpoint files and their backups if you want to resize the file channel (make sure you shutdown the agent first). Be aware that a full replay without checkpoint will take time to complete, depending on how many data files you have and how many events are present in that.
Thanks,
Hari
On Monday, March 31, 2014 at 11:12 AM, Hari Shreedharan wrote:
> I assume you don’t want Flume to reload when the file is updated? If yes, just pass "--no-reload-conf" to the flume-ng command when starting the agent.
>
>
> Thanks,
> Hari
>
>
> On Monday, March 31, 2014 at 8:55 AM, Matt Wise wrote:
>
> > Is there a document anywhere describing what settings can/can't be expected to update dynamically just by modifying the Flume NG agent config file (1.4+)?
> >
> > Matt Wise
> > Sr. Systems Architect
> > Nextdoor.com (http://Nextdoor.com)
> >
> >
> >
> >
> >
> >
> >
>
>
Re: Safety of Automatic Reload...?
Posted by Hari Shreedharan <hs...@cloudera.com>.
I assume you don’t want Flume to reload when the file is updated? If yes, just pass "--no-reload-conf" to the flume-ng command when starting the agent.
Thanks,
Hari
On Monday, March 31, 2014 at 8:55 AM, Matt Wise wrote:
> Is there a document anywhere describing what settings can/can't be expected to update dynamically just by modifying the Flume NG agent config file (1.4+)?
>
> Matt Wise
> Sr. Systems Architect
> Nextdoor.com (http://Nextdoor.com)
>
>
>
>
>
>