You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by GuoWei <we...@wbkit.com> on 2013/01/02 06:45:44 UTC
where to Find flume-ng logs
Hello,
I use flume-mg 1.3 to collect logs and user Hbase Sink to store data to
Hbase. And I want to see flume-ng logs. But I can not find log under
flume-ng root directory. My log4j.properties configure the log directory as
default "./logs".
And I start flume-ng as following:
> Flume-ng agent name host1 conf-file /home/hadoop/flume-ng/conf/razor.conf
Thanks a lot
Regards
Re: where to Find flume-ng logs
Posted by GuoWei <we...@wbkit.com>.
Thanks Connor and Alexandra.
I got the log file.
Regards
GuoWei
From: Connor Woodson <cw...@gmail.com>
Reply-To: <us...@flume.apache.org>
Date: Wed, 2 Jan 2013 14:41:12 -0800
To: <us...@flume.apache.org>
Subject: Re: where to Find flume-ng logs
Here's the command you should use to run flume:
/path/to/flume-ng agent —name host1 --conf /home/hadoop/flume-ng/conf -f
/home/hadoop/flume-ng/conf/razor.conf
And if you want to manually specify the location of the logs, you can either
hard code it into the log4j.properties or you can remove the line
"flume.log.dir=./log" from that file (don't know if that's necessary) and
then add the following to the end of your flume command:
-Dflume.log.dir=/log/dir
- Connor
On Wed, Jan 2, 2013 at 3:17 AM, Alexander Alten-Lorenz <wg...@gmail.com>
wrote:
> Hi,
>
> You've to add the log4j.properties into the classpath:
>
> bin/flume-ng agent -n <agent> -f <configfile> --conf <config directory>
>
> <config directory> means the directory where log4j.properties is located
>
> In log4j.properties be sure you've the correct path:
>
> flume.root.logger=INFO,LOGFILE
> flume.log.dir=./logs
> flume.log.file=flume.log
>
> - Alex
>
> On Jan 2, 2013, at 11:00 AM, GuoWei <we...@wbkit.com> wrote:
>
>> > Hello, Alexander,
>> >
>> > I start flume from the root directory ex: /home/hadoop/flume-ng.
>> > And I also make logs directory in flume-mg directory. And give the logs
>> > folder 777 permission.
>> > But I still can not see the log files under logs directory.
>> >
>> > I can view the log from the console. But never get the log file.
>> >
>> > Any wrong configurations ?
>> >
>> > Thanks
>> >
>> >
>> >
>> >
>> > On 13-1-2 下午3:23, "Alexander Alten-Lorenz" <wg...@gmail.com> wrote:
>> >
>>> >> Hi.
>>> >>
>>> >> "./logs" means in the directory, from where you start flume, in the
>>> >> subdirectory "logs".
>>> >>
>>> >> Best,
>>> >> Alex
>>> >>
>>> >> On Jan 2, 2013, at 6:45 AM, GuoWei <we...@wbkit.com> wrote:
>>> >>
>>>> >>> Hello,
>>>> >>>
>>>> >>> I use flume-mg 1.3 to collect logs and user Hbase Sink to store data to
>>>> >>> Hbase. And I want to see flume-ng logs. But I can not find log under
>>>> >>> flume-ng root directory. My log4j.properties configure the log
>>>> >>> directory as
>>>> >>> default "./logs".
>>>> >>>
>>>> >>> And I start flume-ng as following:
>>>>> >>>> Flume-ng agent ‹name host1 ‹conf-file
>>>>> >>>> /home/hadoop/flume-ng/conf/razor.conf
>>>> >>>
>>>> >>> Thanks a lot
>>>> >>> Regards
>>>> >>>
>>>> >>>
>>> >>
>>> >>
>> >
>> >
>
> --
> Alexander Alten-Lorenz
> http://mapredit.blogspot.com
> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>
Re: where to Find flume-ng logs
Posted by Connor Woodson <cw...@gmail.com>.
Here's the command you should use to run flume:
*/path/to/flume-ng agent —name host1 **--conf /home/hadoop/flume-ng/conf **
-f /home/hadoop/flume-ng/conf/razor.conf *
And if you want to manually specify the location of the logs, you can
either hard code it into the log4j.properties or you can remove the line
"flume.log.dir=./log" from that file (don't know if that's necessary) and
then add the following to the end of your flume command:
-Dflume.log.dir=/log/dir
- Connor
On Wed, Jan 2, 2013 at 3:17 AM, Alexander Alten-Lorenz
<wg...@gmail.com>wrote:
> Hi,
>
> You've to add the log4j.properties into the classpath:
>
> bin/flume-ng agent -n <agent> -f <configfile> --conf <config directory>
>
> <config directory> means the directory where log4j.properties is located
>
> In log4j.properties be sure you've the correct path:
>
> flume.root.logger=INFO,LOGFILE
> flume.log.dir=./logs
> flume.log.file=flume.log
>
> - Alex
>
> On Jan 2, 2013, at 11:00 AM, GuoWei <we...@wbkit.com> wrote:
>
> > Hello, Alexander,
> >
> > I start flume from the root directory ex: /home/hadoop/flume-ng.
> > And I also make logs directory in flume-mg directory. And give the logs
> > folder 777 permission.
> > But I still can not see the log files under logs directory.
> >
> > I can view the log from the console. But never get the log file.
> >
> > Any wrong configurations ?
> >
> > Thanks
> >
> >
> >
> >
> > On 13-1-2 下午3:23, "Alexander Alten-Lorenz" <wg...@gmail.com> wrote:
> >
> >> Hi.
> >>
> >> "./logs" means in the directory, from where you start flume, in the
> >> subdirectory "logs".
> >>
> >> Best,
> >> Alex
> >>
> >> On Jan 2, 2013, at 6:45 AM, GuoWei <we...@wbkit.com> wrote:
> >>
> >>> Hello,
> >>>
> >>> I use flume-mg 1.3 to collect logs and user Hbase Sink to store data to
> >>> Hbase. And I want to see flume-ng logs. But I can not find log under
> >>> flume-ng root directory. My log4j.properties configure the log
> >>> directory as
> >>> default "./logs".
> >>>
> >>> And I start flume-ng as following:
> >>>> Flume-ng agent ‹name host1 ‹conf-file
> >>>> /home/hadoop/flume-ng/conf/razor.conf
> >>>
> >>> Thanks a lot
> >>> Regards
> >>>
> >>>
> >>
> >>
> >
> >
>
> --
> Alexander Alten-Lorenz
> http://mapredit.blogspot.com
> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>
>
Re: where to Find flume-ng logs
Posted by Alexander Alten-Lorenz <wg...@gmail.com>.
You could pass per agent a different log4j.property file as well as a different flume-env.sh by setting up different directories and point each agent into.
On Jan 4, 2013, at 11:01 AM, Abhijeet Pathak <Ab...@kpitcummins.com> wrote:
> Using that method, if we have multiple agents, they all append to the same flume.log file.
>
> Is there way to create separate log file for each agent without creating separate folder for each flume-Ng configuration?
>
> Sent from my Windows Phone
> ________________________________
> From: Alexander Alten-Lorenz
> Sent: 02-01-2013 16:48
> To: user@flume.apache.org
> Subject: Re: where to Find flume-ng logs
>
> Hi,
>
> You've to add the log4j.properties into the classpath:
>
> bin/flume-ng agent -n <agent> -f <configfile> --conf <config directory>
>
> <config directory> means the directory where log4j.properties is located
>
> In log4j.properties be sure you've the correct path:
>
> flume.root.logger=INFO,LOGFILE
> flume.log.dir=./logs
> flume.log.file=flume.log
>
> - Alex
>
> On Jan 2, 2013, at 11:00 AM, GuoWei <we...@wbkit.com> wrote:
>
>> Hello, Alexander,
>>
>> I start flume from the root directory ex: /home/hadoop/flume-ng.
>> And I also make logs directory in flume-mg directory. And give the logs
>> folder 777 permission.
>> But I still can not see the log files under logs directory.
>>
>> I can view the log from the console. But never get the log file.
>>
>> Any wrong configurations ?
>>
>> Thanks
>>
>>
>>
>>
>> On 13-1-2 下午3:23, "Alexander Alten-Lorenz" <wg...@gmail.com> wrote:
>>
>>> Hi.
>>>
>>> "./logs" means in the directory, from where you start flume, in the
>>> subdirectory "logs".
>>>
>>> Best,
>>> Alex
>>>
>>> On Jan 2, 2013, at 6:45 AM, GuoWei <we...@wbkit.com> wrote:
>>>
>>>> Hello,
>>>>
>>>> I use flume-mg 1.3 to collect logs and user Hbase Sink to store data to
>>>> Hbase. And I want to see flume-ng logs. But I can not find log under
>>>> flume-ng root directory. My log4j.properties configure the log
>>>> directory as
>>>> default "./logs".
>>>>
>>>> And I start flume-ng as following:
>>>>> Flume-ng agent ‹name host1 ‹conf-file
>>>>> /home/hadoop/flume-ng/conf/razor.conf
>>>>
>>>> Thanks a lot
>>>> Regards
>>>>
>>>>
>>>
>>>
>>
>>
>
> --
> Alexander Alten-Lorenz
> http://mapredit.blogspot.com
> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>
>
>
>
--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF
RE: where to Find flume-ng logs
Posted by Abhijeet Pathak <Ab...@kpitcummins.com>.
Using that method, if we have multiple agents, they all append to the same flume.log file.
Is there way to create separate log file for each agent without creating separate folder for each flume-Ng configuration?
Sent from my Windows Phone
________________________________
From: Alexander Alten-Lorenz
Sent: 02-01-2013 16:48
To: user@flume.apache.org
Subject: Re: where to Find flume-ng logs
Hi,
You've to add the log4j.properties into the classpath:
bin/flume-ng agent -n <agent> -f <configfile> --conf <config directory>
<config directory> means the directory where log4j.properties is located
In log4j.properties be sure you've the correct path:
flume.root.logger=INFO,LOGFILE
flume.log.dir=./logs
flume.log.file=flume.log
- Alex
On Jan 2, 2013, at 11:00 AM, GuoWei <we...@wbkit.com> wrote:
> Hello, Alexander,
>
> I start flume from the root directory ex: /home/hadoop/flume-ng.
> And I also make logs directory in flume-mg directory. And give the logs
> folder 777 permission.
> But I still can not see the log files under logs directory.
>
> I can view the log from the console. But never get the log file.
>
> Any wrong configurations ?
>
> Thanks
>
>
>
>
> On 13-1-2 下午3:23, "Alexander Alten-Lorenz" <wg...@gmail.com> wrote:
>
>> Hi.
>>
>> "./logs" means in the directory, from where you start flume, in the
>> subdirectory "logs".
>>
>> Best,
>> Alex
>>
>> On Jan 2, 2013, at 6:45 AM, GuoWei <we...@wbkit.com> wrote:
>>
>>> Hello,
>>>
>>> I use flume-mg 1.3 to collect logs and user Hbase Sink to store data to
>>> Hbase. And I want to see flume-ng logs. But I can not find log under
>>> flume-ng root directory. My log4j.properties configure the log
>>> directory as
>>> default "./logs".
>>>
>>> And I start flume-ng as following:
>>>> Flume-ng agent ‹name host1 ‹conf-file
>>>> /home/hadoop/flume-ng/conf/razor.conf
>>>
>>> Thanks a lot
>>> Regards
>>>
>>>
>>
>>
>
>
--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF
Re: where to Find flume-ng logs
Posted by Alexander Alten-Lorenz <wg...@gmail.com>.
Hi,
You've to add the log4j.properties into the classpath:
bin/flume-ng agent -n <agent> -f <configfile> --conf <config directory>
<config directory> means the directory where log4j.properties is located
In log4j.properties be sure you've the correct path:
flume.root.logger=INFO,LOGFILE
flume.log.dir=./logs
flume.log.file=flume.log
- Alex
On Jan 2, 2013, at 11:00 AM, GuoWei <we...@wbkit.com> wrote:
> Hello, Alexander,
>
> I start flume from the root directory ex: /home/hadoop/flume-ng.
> And I also make logs directory in flume-mg directory. And give the logs
> folder 777 permission.
> But I still can not see the log files under logs directory.
>
> I can view the log from the console. But never get the log file.
>
> Any wrong configurations ?
>
> Thanks
>
>
>
>
> On 13-1-2 下午3:23, "Alexander Alten-Lorenz" <wg...@gmail.com> wrote:
>
>> Hi.
>>
>> "./logs" means in the directory, from where you start flume, in the
>> subdirectory "logs".
>>
>> Best,
>> Alex
>>
>> On Jan 2, 2013, at 6:45 AM, GuoWei <we...@wbkit.com> wrote:
>>
>>> Hello,
>>>
>>> I use flume-mg 1.3 to collect logs and user Hbase Sink to store data to
>>> Hbase. And I want to see flume-ng logs. But I can not find log under
>>> flume-ng root directory. My log4j.properties configure the log
>>> directory as
>>> default "./logs".
>>>
>>> And I start flume-ng as following:
>>>> Flume-ng agent ‹name host1 ‹conf-file
>>>> /home/hadoop/flume-ng/conf/razor.conf
>>>
>>> Thanks a lot
>>> Regards
>>>
>>>
>>
>>
>
>
--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF
Re: where to Find flume-ng logs
Posted by GuoWei <we...@wbkit.com>.
Hello, Alexander,
I start flume from the root directory ex: /home/hadoop/flume-ng.
And I also make logs directory in flume-mg directory. And give the logs
folder 777 permission.
But I still can not see the log files under logs directory.
I can view the log from the console. But never get the log file.
Any wrong configurations ?
Thanks
On 13-1-2 下午3:23, "Alexander Alten-Lorenz" <wg...@gmail.com> wrote:
>Hi.
>
>"./logs" means in the directory, from where you start flume, in the
>subdirectory "logs".
>
>Best,
> Alex
>
>On Jan 2, 2013, at 6:45 AM, GuoWei <we...@wbkit.com> wrote:
>
>> Hello,
>>
>> I use flume-mg 1.3 to collect logs and user Hbase Sink to store data to
>> Hbase. And I want to see flume-ng logs. But I can not find log under
>> flume-ng root directory. My log4j.properties configure the log
>>directory as
>> default "./logs".
>>
>> And I start flume-ng as following:
>>> Flume-ng agent ‹name host1 ‹conf-file
>>>/home/hadoop/flume-ng/conf/razor.conf
>>
>> Thanks a lot
>> Regards
>>
>>
>
>
Re: where to Find flume-ng logs
Posted by Alexander Alten-Lorenz <wg...@gmail.com>.
Hi.
"./logs" means in the directory, from where you start flume, in the subdirectory "logs".
Best,
Alex
On Jan 2, 2013, at 6:45 AM, GuoWei <we...@wbkit.com> wrote:
> Hello,
>
> I use flume-mg 1.3 to collect logs and user Hbase Sink to store data to
> Hbase. And I want to see flume-ng logs. But I can not find log under
> flume-ng root directory. My log4j.properties configure the log directory as
> default "./logs".
>
> And I start flume-ng as following:
>> Flume-ng agent ‹name host1 ‹conf-file /home/hadoop/flume-ng/conf/razor.conf
>
> Thanks a lot
> Regards
>
>