You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@chukwa.apache.org by Eric Yang <er...@gmail.com> on 2010/10/03 02:05:47 UTC
Planning for SyslogAdaptor
Hi,
UDPAdaptor can record syslog logs but the log entry is tagged with
Chukwa internal data type at adaptor start up time. I am thinking to
extend UDPAdaptor to record syslog messages with small enhancements.
SyslogAdaptor can extract facility name field from syslog message to
tag facility name as the data type. This would provide the ability to
invoke different demux parsers base on the facility name in syslog.
HOSTNAME can also be mapped to source for Chukwa chunk meta data.
My use case is to configure hadoop log4j.properties to log everything
through org.apache.log4j.net.SyslogAppender to one SysLogAdaptor on
the local Chukwa agent, and invokes different parsers for audit log
and name node log. Facility name is some what limited, but it should
be enough type left to manage hadoop log files. Suggestion and
feedbacks are welcome.
regards,
Eric
Re: Planning for SyslogAdaptor
Posted by Eric Yang <er...@gmail.com>.
Yes, I plan to refactor UDPAdaptor run method. The while loop logic
is slightly different for SyslogAdaptor, and the rest are the same.
regards,
Eric
On Sat, Oct 2, 2010 at 8:30 PM, Ariel Rabkin <as...@gmail.com> wrote:
> Yes. That sounds like a good idea. Can you reuse any of the
> UDPAdaptor code by inheritance? (Or by refactoring, and then using
> inheritance?)
>
> --Ari
>
> On Sat, Oct 2, 2010 at 5:05 PM, Eric Yang <er...@gmail.com> wrote:
>> Hi,
>>
>> UDPAdaptor can record syslog logs but the log entry is tagged with
>> Chukwa internal data type at adaptor start up time. I am thinking to
>> extend UDPAdaptor to record syslog messages with small enhancements.
>> SyslogAdaptor can extract facility name field from syslog message to
>> tag facility name as the data type. This would provide the ability to
>> invoke different demux parsers base on the facility name in syslog.
>> HOSTNAME can also be mapped to source for Chukwa chunk meta data.
>>
>> My use case is to configure hadoop log4j.properties to log everything
>> through org.apache.log4j.net.SyslogAppender to one SysLogAdaptor on
>> the local Chukwa agent, and invokes different parsers for audit log
>> and name node log. Facility name is some what limited, but it should
>> be enough type left to manage hadoop log files. Suggestion and
>> feedbacks are welcome.
>>
>> regards,
>> Eric
>>
>
>
>
> --
> Ari Rabkin asrabkin@gmail.com
> UC Berkeley Computer Science Department
>
Re: Planning for SyslogAdaptor
Posted by Ariel Rabkin <as...@gmail.com>.
Yes. That sounds like a good idea. Can you reuse any of the
UDPAdaptor code by inheritance? (Or by refactoring, and then using
inheritance?)
--Ari
On Sat, Oct 2, 2010 at 5:05 PM, Eric Yang <er...@gmail.com> wrote:
> Hi,
>
> UDPAdaptor can record syslog logs but the log entry is tagged with
> Chukwa internal data type at adaptor start up time. I am thinking to
> extend UDPAdaptor to record syslog messages with small enhancements.
> SyslogAdaptor can extract facility name field from syslog message to
> tag facility name as the data type. This would provide the ability to
> invoke different demux parsers base on the facility name in syslog.
> HOSTNAME can also be mapped to source for Chukwa chunk meta data.
>
> My use case is to configure hadoop log4j.properties to log everything
> through org.apache.log4j.net.SyslogAppender to one SysLogAdaptor on
> the local Chukwa agent, and invokes different parsers for audit log
> and name node log. Facility name is some what limited, but it should
> be enough type left to manage hadoop log files. Suggestion and
> feedbacks are welcome.
>
> regards,
> Eric
>
--
Ari Rabkin asrabkin@gmail.com
UC Berkeley Computer Science Department