You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by Lior Harel <ha...@gmail.com> on 2011/08/29 12:11:58 UTC
nodes configuration is lost after master is restarted
Hi All,
I am running a single master with a few agents and a single collector, I
noticed that after restarting the flume master it seems to lost all previous
configuration.
I am running in ZBCS mode, without any changes to the default configuration.
can you help me making the configuration persistent?
Thanks
Lior
Re: nodes configuration is lost after master is restarted
Posted by Jonathan Hsieh <jo...@cloudera.com>.
I've updated the FAQ,
https://cwiki.apache.org/confluence/display/FLUME/Troubleshooting+FAQ
Feel free to update with more answers to problems if you encounter them!
Thanks,
Jon.
On Wed, Sep 7, 2011 at 10:09 AM, Mike <mi...@gmail.com> wrote:
> Thanks for posting this - I just used it to solve the same problem.
>
> On Tue, Aug 30, 2011 at 12:21 PM, Lior Harel <ha...@gmail.com> wrote:
> > you're right, changing the configuration solved my problems. should the
> > documentation note that the default dir is a temporary one?
> >
> > On Tue, Aug 30, 2011 at 4:33 PM, Joe Crobak <jo...@gmail.com> wrote:
> >>
> >> Per the user guide [1], I suspect the property that you need to update
> is
> >> flume.master.zk.logdir. It defaults to:
> >> <property>
> >> <name>flume.master.zk.logdir</name>
> >> <value>/tmp/flume-${user.name}-zk</value>
> >> <description>The base directory in which the ZBCS stores
> >> data.</description>
> >> </property>
> >> which might be cleaned up at any time by the OS.
> >> You probably also want to set FLUME_PID_DIR in flume-env.sh, because
> this
> >> also defaults to a location in
> >> /tmp: flume:MASTERPID=${FLUME_PID_DIR:="/tmp"}/flumemaster.pid
> >>
> >>
> >> [1]
> http://archive.cloudera.com/cdh/3/flume/UserGuide/index.html#_configuring_the_zbcs
> >>
> >> On Mon, Aug 29, 2011 at 6:11 AM, Lior Harel <ha...@gmail.com>
> wrote:
> >>>
> >>> Hi All,
> >>> I am running a single master with a few agents and a single collector,
> I
> >>> noticed that after restarting the flume master it seems to lost all
> previous
> >>> configuration.
> >>> I am running in ZBCS mode, without any changes to the default
> >>> configuration. can you help me making the configuration persistent?
> >>> Thanks
> >>> Lior
> >
> >
>
--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// jon@cloudera.com
Re: nodes configuration is lost after master is restarted
Posted by Mike <mi...@gmail.com>.
Thanks for posting this - I just used it to solve the same problem.
On Tue, Aug 30, 2011 at 12:21 PM, Lior Harel <ha...@gmail.com> wrote:
> you're right, changing the configuration solved my problems. should the
> documentation note that the default dir is a temporary one?
>
> On Tue, Aug 30, 2011 at 4:33 PM, Joe Crobak <jo...@gmail.com> wrote:
>>
>> Per the user guide [1], I suspect the property that you need to update is
>> flume.master.zk.logdir. It defaults to:
>> <property>
>> <name>flume.master.zk.logdir</name>
>> <value>/tmp/flume-${user.name}-zk</value>
>> <description>The base directory in which the ZBCS stores
>> data.</description>
>> </property>
>> which might be cleaned up at any time by the OS.
>> You probably also want to set FLUME_PID_DIR in flume-env.sh, because this
>> also defaults to a location in
>> /tmp: flume:MASTERPID=${FLUME_PID_DIR:="/tmp"}/flumemaster.pid
>>
>>
>> [1] http://archive.cloudera.com/cdh/3/flume/UserGuide/index.html#_configuring_the_zbcs
>>
>> On Mon, Aug 29, 2011 at 6:11 AM, Lior Harel <ha...@gmail.com> wrote:
>>>
>>> Hi All,
>>> I am running a single master with a few agents and a single collector, I
>>> noticed that after restarting the flume master it seems to lost all previous
>>> configuration.
>>> I am running in ZBCS mode, without any changes to the default
>>> configuration. can you help me making the configuration persistent?
>>> Thanks
>>> Lior
>
>
Re: nodes configuration is lost after master is restarted
Posted by Lior Harel <ha...@gmail.com>.
you're right, changing the configuration solved my problems. should the
documentation note that the default dir is a temporary one?
On Tue, Aug 30, 2011 at 4:33 PM, Joe Crobak <jo...@gmail.com> wrote:
> Per the user guide [1], I suspect the property that you need to update is
> flume.master.zk.logdir. It defaults to:
>
> <property>
> <name>flume.master.zk.logdir</name>
> <value>/tmp/flume-${user.name}-zk</value>
> <description>The base directory in which the ZBCS stores
> data.</description>
> </property>
>
> which might be cleaned up at any time by the OS.
>
> You probably also want to set FLUME_PID_DIR in flume-env.sh, because this
> also defaults to a location in
> /tmp: flume:MASTERPID=${FLUME_PID_DIR:="/tmp"}/flumemaster.pid
>
>
> [1]
> http://archive.cloudera.com/cdh/3/flume/UserGuide/index.html#_configuring_the_zbcs
>
>
> On Mon, Aug 29, 2011 at 6:11 AM, Lior Harel <ha...@gmail.com> wrote:
>
>> Hi All,
>> I am running a single master with a few agents and a single collector, I
>> noticed that after restarting the flume master it seems to lost all previous
>> configuration.
>> I am running in ZBCS mode, without any changes to the default
>> configuration. can you help me making the configuration persistent?
>>
>> Thanks
>> Lior
>>
>
>
Re: nodes configuration is lost after master is restarted
Posted by Joe Crobak <jo...@gmail.com>.
Per the user guide [1], I suspect the property that you need to update is
flume.master.zk.logdir. It defaults to:
<property>
<name>flume.master.zk.logdir</name>
<value>/tmp/flume-${user.name}-zk</value>
<description>The base directory in which the ZBCS stores
data.</description>
</property>
which might be cleaned up at any time by the OS.
You probably also want to set FLUME_PID_DIR in flume-env.sh, because this
also defaults to a location in
/tmp: flume:MASTERPID=${FLUME_PID_DIR:="/tmp"}/flumemaster.pid
[1]
http://archive.cloudera.com/cdh/3/flume/UserGuide/index.html#_configuring_the_zbcs
On Mon, Aug 29, 2011 at 6:11 AM, Lior Harel <ha...@gmail.com> wrote:
> Hi All,
> I am running a single master with a few agents and a single collector, I
> noticed that after restarting the flume master it seems to lost all previous
> configuration.
> I am running in ZBCS mode, without any changes to the default
> configuration. can you help me making the configuration persistent?
>
> Thanks
> Lior
>
Re: nodes configuration is lost after master is restarted
Posted by Matthew Rathbone <ma...@foursquare.com>.
This happens to me a lot. It's random, sometimes it forgets, sometimes it remembers, I have no idea why. I just make sure to keep a script for running the configurations back into the master.
--
Matthew Rathbone
Foursquare | Software Engineer | Server Engineering Team
matthew@foursquare.com (mailto:matthew@foursquare.com) | @rathboma (http://twitter.com/rathboma) | 4sq (http://foursquare.com/rathboma)
On Monday, August 29, 2011 at 5:11 AM, Lior Harel wrote:
> Hi All,
> I am running a single master with a few agents and a single collector, I noticed that after restarting the flume master it seems to lost all previous configuration.
> I am running in ZBCS mode, without any changes to the default configuration. can you help me making the configuration persistent?
>
> Thanks
> Lior
>
>
>