You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by higkoohk <hi...@gmail.com> on 2013/05/15 06:02:02 UTC
Why would flume create one file per milliseconds?
Hello ,all !
I'm a new flumer , today I use flume to collector web server logs.
My flume config is 'flume.conf' in attachment,and it's logs named
'flume.log' in attachment.
I found that flume would create one file per milliseconds ?
Thanks for help !
Re: Why would flume create one file per milliseconds?
Posted by higkoohk <hi...@gmail.com>.
Maybe duplite 'o' when create new line in 'vi' .
2013/5/15 higkoohk <hi...@gmail.com>
> Ah,Just it is !!
>
> Many thanks @ Hari Shreedharan
>
>
> 2013/5/15 Hari Shreedharan <hs...@cloudera.com>
>
>> Your config has a typo:
>> tengine.sinks.hdfs4log.hdfs.rollCouont = 3600
>>
>> It should be:
>> tengine.sinks.hdfs4log.hdfs.rollCount = 3600
>>
>> Due to this typo, your files are being rolled once 10 events are written.
>>
>>
>> Hari
>>
>> --
>> Hari Shreedharan
>>
>> On Tuesday, May 14, 2013 at 10:54 PM, higkoohk wrote:
>>
>> I try change the source to cat or spooldir ,the effect is the same!
>> 1. Many HDFS file like:
>>
>> 2. Many created logs like:
>>
>>
>>
>>
>> 2013/5/15 higkoohk <hi...@gmail.com>
>>
>> Hello ,all !
>>
>> I'm a new flumer , today I use flume to collector web server logs.
>>
>> My flume config is 'flume.conf' in attachment,and it's logs named
>> 'flume.log' in attachment.
>>
>> I found that flume would create one file per milliseconds ?
>>
>> Thanks for help !
>>
>>
Re: Why would flume create one file per milliseconds?
Posted by higkoohk <hi...@gmail.com>.
Ah,Just it is !!
Many thanks @ Hari Shreedharan
2013/5/15 Hari Shreedharan <hs...@cloudera.com>
> Your config has a typo:
> tengine.sinks.hdfs4log.hdfs.rollCouont = 3600
>
> It should be:
> tengine.sinks.hdfs4log.hdfs.rollCount = 3600
>
> Due to this typo, your files are being rolled once 10 events are written.
>
>
> Hari
>
> --
> Hari Shreedharan
>
> On Tuesday, May 14, 2013 at 10:54 PM, higkoohk wrote:
>
> I try change the source to cat or spooldir ,the effect is the same!
> 1. Many HDFS file like:
>
> 2. Many created logs like:
>
>
>
>
> 2013/5/15 higkoohk <hi...@gmail.com>
>
> Hello ,all !
>
> I'm a new flumer , today I use flume to collector web server logs.
>
> My flume config is 'flume.conf' in attachment,and it's logs named
> 'flume.log' in attachment.
>
> I found that flume would create one file per milliseconds ?
>
> Thanks for help !
>
>
>
>
Re: Why would flume create one file per milliseconds?
Posted by Hari Shreedharan <hs...@cloudera.com>.
Your config has a typo:
tengine.sinks.hdfs4log.hdfs.rollCouont = 3600
It should be:
tengine.sinks.hdfs4log.hdfs.rollCount = 3600
Due to this typo, your files are being rolled once 10 events are written.
Hari
--
Hari Shreedharan
On Tuesday, May 14, 2013 at 10:54 PM, higkoohk wrote:
> I try change the source to cat or spooldir ,the effect is the same!
> 1. Many HDFS file like:
>
> 2. Many created logs like:
>
>
>
>
> 2013/5/15 higkoohk <higkoohk@gmail.com (mailto:higkoohk@gmail.com)>
> > Hello ,all !
> >
> > I'm a new flumer , today I use flume to collector web server logs.
> >
> > My flume config is 'flume.conf' in attachment,and it's logs named 'flume.log' in attachment.
> >
> > I found that flume would create one file per milliseconds ?
> >
> > Thanks for help !
>
Re: Why would flume create one file per milliseconds?
Posted by higkoohk <hi...@gmail.com>.
I try change the source to cat or spooldir ,the effect is the same!
1. Many HDFS file like:
[image: 内嵌图片 1]
2. Many created logs like:
[image: 内嵌图片 2]
2013/5/15 higkoohk <hi...@gmail.com>
> Hello ,all !
>
> I'm a new flumer , today I use flume to collector web server logs.
>
> My flume config is 'flume.conf' in attachment,and it's logs named
> 'flume.log' in attachment.
>
> I found that flume would create one file per milliseconds ?
>
> Thanks for help !
>
>