How chuwka identity the tailing log generated by java log4j?

[IvyTang <iv...@gmail.com>]

I want to use chuwka to collect the log generated by java log4j.

The log file has the size limition ,when the size reaches to 200M , the
log4j will rename this file to .1 and renew a log file.

If chukwa is tailing the current log , and the log size reaches to 200M but
the chukwa just collect < 200M . And the the log is renamed , a new log
file is generated.

I thought this log will miss , the chukwa will read the new log .But i
found the chukwa adaptor file offset doesn't change to 0 .

How chukwa handle this situation?

thx

-- 
Best regards,

Ivy Tang

Re: How chuwka identity the tailing log generated by java log4j?

[Eric Yang <er...@gmail.com>]

FileTailingAdaptors track the rotation of the log file by having to
compare the most recent read offset, and file size.  If the file size
is smaller than most recent read offset, adaptor detects this as file
rotation.  The adaptor read to the end of the first file, and send all
remaining data to collector.  Next, the adaptor discovers the similar
pattern file name and track the creation date of the rotated log
files.  Base on creation date, and file pointer offset tracking, data
are sent to collectors in the ordered sequence.

regards,
Eric

On Thu, Mar 8, 2012 at 10:18 PM, IvyTang <iv...@gmail.com> wrote:
> I want to use chuwka to collect the log generated by java log4j.
>
> The log file has the size limition ,when the size reaches to 200M , the
> log4j will rename this file to .1 and renew a log file.
>
> If chukwa is tailing the current log , and the log size reaches to 200M but
> the chukwa just collect < 200M . And the the log is renamed , a new log file
> is generated.
>
> I thought this log will miss , the chukwa will read the new log .But i found
> the chukwa adaptor file offset doesn't change to 0 .
>
> How chukwa handle this situation?
>
> thx
>
> --
> Best regards,
>
> Ivy Tang
>
>
>