You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by Flinkster <fl...@gmail.com> on 2012/04/17 20:24:56 UTC

log4j via flume or syslog appender?

Hi there,

Any advice on flume log4j appenders: flume or syslog?

With the log4j syslog appender there's no new jars to install and
syslog daemon is already running on the box. Just change the
log4j.properties and point syslog.conf to flume collector.

If I choose to use the log4j flume appender I need to install flume
agent and couple of new jars to the system.

Are there any benefits to use log4j flume appender vs log4j syslog appender?

Thanks,

-ari

Re: log4j via flume or syslog appender?

Posted by Ralph Goers <ra...@dslextreme.com>.
I'm not sure how rsyslog can help when the syslog protocol itself isn't reliable (no response is expected). In addition, while rsyslog has accepted the message if it was to crash at that point the event would be lost yet the client would believe it was delivered.

As I said, what you gain is a) one less moving part and b) guaranteed delivery.

Ralph

On Apr 17, 2012, at 1:24 PM, Flinkster wrote:

> What if you use rsyslog? It has a reliable TCP transport.
> 
> log4j -> syslog appender -> rsyslog -> flume
> 
> rsyslog extensions:
> 
> - ISO 8601 timestamp with millisecond granularity and timezone information
> - the addition of the name of relays in the host fields to make it
> possible to track the path a given message has traversed
> - reliable transport using TCP
> - support GSS-API and TLS
> - logging directly into various database engines.
> - support for the upcoming new IETF syslog RFC series
> - support for buffered operation modes where messages are buffered
> locally if the receiver is not ready
> 
> The question: what would I gain by using flume log4j appender?
> 
> -ari
> 
> 
> On Tue, Apr 17, 2012 at 12:55 PM, Ralph Goers
> <ra...@dslextreme.com> wrote:
>> The syslog protocol doesn't guarantee delivery. Using the flume appender connecting to an agent should.  That said, I'm not sure log4j itself will help. Log4j2 will.
>> 
>> Ralph
>> 
>> On Apr 17, 2012, at 11:24 AM, Flinkster wrote:
>> 
>>> Hi there,
>>> 
>>> Any advice on flume log4j appenders: flume or syslog?
>>> 
>>> With the log4j syslog appender there's no new jars to install and
>>> syslog daemon is already running on the box. Just change the
>>> log4j.properties and point syslog.conf to flume collector.
>>> 
>>> If I choose to use the log4j flume appender I need to install flume
>>> agent and couple of new jars to the system.
>>> 
>>> Are there any benefits to use log4j flume appender vs log4j syslog appender?
>>> 
>>> Thanks,
>>> 
>>> -ari
>> 


Re: log4j via flume or syslog appender?

Posted by Flinkster <fl...@gmail.com>.
What if you use rsyslog? It has a reliable TCP transport.

log4j -> syslog appender -> rsyslog -> flume

rsyslog extensions:

- ISO 8601 timestamp with millisecond granularity and timezone information
- the addition of the name of relays in the host fields to make it
possible to track the path a given message has traversed
- reliable transport using TCP
- support GSS-API and TLS
- logging directly into various database engines.
- support for the upcoming new IETF syslog RFC series
- support for buffered operation modes where messages are buffered
locally if the receiver is not ready

The question: what would I gain by using flume log4j appender?

-ari


On Tue, Apr 17, 2012 at 12:55 PM, Ralph Goers
<ra...@dslextreme.com> wrote:
> The syslog protocol doesn't guarantee delivery. Using the flume appender connecting to an agent should.  That said, I'm not sure log4j itself will help. Log4j2 will.
>
> Ralph
>
> On Apr 17, 2012, at 11:24 AM, Flinkster wrote:
>
>> Hi there,
>>
>> Any advice on flume log4j appenders: flume or syslog?
>>
>> With the log4j syslog appender there's no new jars to install and
>> syslog daemon is already running on the box. Just change the
>> log4j.properties and point syslog.conf to flume collector.
>>
>> If I choose to use the log4j flume appender I need to install flume
>> agent and couple of new jars to the system.
>>
>> Are there any benefits to use log4j flume appender vs log4j syslog appender?
>>
>> Thanks,
>>
>> -ari
>

Re: log4j via flume or syslog appender?

Posted by Ralph Goers <ra...@dslextreme.com>.
The syslog protocol doesn't guarantee delivery. Using the flume appender connecting to an agent should.  That said, I'm not sure log4j itself will help. Log4j2 will.

Ralph

On Apr 17, 2012, at 11:24 AM, Flinkster wrote:

> Hi there,
> 
> Any advice on flume log4j appenders: flume or syslog?
> 
> With the log4j syslog appender there's no new jars to install and
> syslog daemon is already running on the box. Just change the
> log4j.properties and point syslog.conf to flume collector.
> 
> If I choose to use the log4j flume appender I need to install flume
> agent and couple of new jars to the system.
> 
> Are there any benefits to use log4j flume appender vs log4j syslog appender?
> 
> Thanks,
> 
> -ari