Kafka log4j appender infinite logging cycle question

[David Black <da...@workday.com>]

Hi,


I have the following scenario:

- using the Kafka log4j appender (in fact its a variant on the OOTB
appender)

- configuring the appender for the root log4j category

- appender is using the async producer

I am wondering about what happens when e.g. the producer cannot connect to
any broker and it logs error messages, which (I’m guessing) get passed by
log4j back into the (kafka) appender. I think there is a possibility of a
kind of infinite cycle, where any log message emitted by the kafka producer
framework may result in more messages being emitted by the kafka producer
framework, due to being pushed back through the appender and so on.

To an extent I can control this with log4j.additivity e.g. by having a
separate category for the kafka root package (direct its output to my other
appenders but not the kafka appender), and setting additivity for kafka
root package to false. But I think messages from other packages (from other
libraries) that the kafka producer uses may still cause this cycling. Would
I have to identify all those packages and also exclude them? I think this
would not work because other parts of my process use some of those same
jars and I need their output to go to kafka via the kafka log4j appender.

I’m wondering has anyone else encountered / considered / dealt with this
and if so how?


Any help much appreciated


David Black

Workday Inc.

Re: Kafka log4j appender infinite logging cycle question

[Jun Rao <ju...@gmail.com>]

Yes, there is this self loop, but it's not always a 1-to-1 mapping. Since
you use async producers, each N messages corresponds to a Kafka produce
request. So, if you encounter an error while processing the request, you
add one more message (for every N messages). What you have to avoid is per
message level error (e.g., serialization) since that will add one more
message for each incoming message.

Thanks,

Jun


On Thu, Aug 1, 2013 at 3:41 AM, David Black <da...@workday.com> wrote:

> Hi,
>
>
> I have the following scenario:
>
> - using the Kafka log4j appender (in fact its a variant on the OOTB
> appender)
>
> - configuring the appender for the root log4j category
>
> - appender is using the async producer
>
> I am wondering about what happens when e.g. the producer cannot connect to
> any broker and it logs error messages, which (I’m guessing) get passed by
> log4j back into the (kafka) appender. I think there is a possibility of a
> kind of infinite cycle, where any log message emitted by the kafka producer
> framework may result in more messages being emitted by the kafka producer
> framework, due to being pushed back through the appender and so on.
>
> To an extent I can control this with log4j.additivity e.g. by having a
> separate category for the kafka root package (direct its output to my other
> appenders but not the kafka appender), and setting additivity for kafka
> root package to false. But I think messages from other packages (from other
> libraries) that the kafka producer uses may still cause this cycling. Would
> I have to identify all those packages and also exclude them? I think this
> would not work because other parts of my process use some of those same
> jars and I need their output to go to kafka via the kafka log4j appender.
>
> I’m wondering has anyone else encountered / considered / dealt with this
> and if so how?
>
>
> Any help much appreciated
>
>
> David Black
>
> Workday Inc.
>