Re: Alternative Asynchronous Appender

[Simon Park <si...@yahoo.co.uk>]

Hi Curt,

Thanks for taking the time to reply.

I'm afraid I don't fully understand the design decision to enable logging events to be discarded.  If a piece of information is sufficiently expendable, then why bother logging it in the first place?  I guess the community pressure to incorporate this feature must have been quite strong.  Fair enough.

On the end-of-lifecycle behaviour, I guess a well-behaved application should use the LogManager to close all appenders cleanly, so you're right, it isn't such an issue.  I had been thinking that shutdown hooks might be easier to write with references available to appenders somewhat more readily than for the dispatcher, but again, the LogManager could conceivably be called from a shutdown hook.

I have been doing some performance work to see how the AsyncAppender compares with my own implementation.  On a single-processor, single-core machine, the AsyncAppender is about 15-20% faster for 10 threads doing nothing but logging (yeah, I know, slightly unrealistic, but I wanted to stress the appender code).  I plan to test on a dual-processor, dual-core machine, although upon reflection I expect similar results.  The brake is a heavily-contended lock on a home-brewed deque, backed by a LinkedList.  Basically the many thrreads that are logging outnumber the single dispatch thread, the net effect being that the buffer fills up and never really empties, further worsening performance since a gate closes at this point to give the dispatch thread opportunity to clear at least one event from the buffer.

The solution I'm going to look at is to use Doug Lea's WaitFreeQueue (http://gee.cs.oswego.edu/dl/classes/EDU/oswego/cs/dl/util/concurrent/WaitFreeQueue.java), which should dramatically ease contention due to locks being on the head and tail nodes in the queue, rather than on the whole queue.  Since Doug's code is compatible with Java 1.3, there should still be the good compatibility one has with Log4J 1.2.x.

If you're interested, I'll let you know how I get on.

Best Regards,

Simon

----- Original Message ----
From: Curt Arnold <ca...@apache.org>
To: Log4J Developers List <lo...@logging.apache.org>
Sent: Tuesday, 30 October, 2007 9:53:03 PM
Subject: Re: Alternative Asynchronous Appender


On Oct 30, 2007, at 3:01 PM, Simon Park wrote:

> Hi,
>
> I've posted code (Apache license, Log4J coding standards) that I  
> hope will offer an alternative to the standard Log4J  
> AsyncAppender.  The design intention is to facilitate reduced  
> blocking for logging operations, thereby allowing higher throughput  
> for heavily-threaded applications.  More at http:// 
> simonsiteblog.blogspot.com/2007/10/ive-been-working-on-alternative- 
> to.html.
>
> Is this useful?  I would welcome feedback.
>
> Simon
>
>

Haven't looked at the code, but I see your point.  The log4j 1.2.14  
and later AsyncAppender does delay blocking relative to the prior  
AsyncAppender (and can skip blocking all together is blocking=false),  
but when it does block, it will block longer.  The GC pause metaphor  
is pretty right on the mark.

The earlier AsyncAppender did pull an event at a time off the buffer,  
but I found that difficult to do the non-blocking option with that  
approach.  However, you could likely modify the loop at the end of  
Dispatcher.run to pull one event off the queue for each completed  
append when the appender is blocking=true and the appenders buffer is  
already full and blocking.

AsyncAppender.close() will joins with the dispatcher thread,  
essentially blocking until all pending log requests are handled.  So  
I don't think the more reliable logging at end of application is an  
issue.  Obviously, any abnormal termination may result in lost  
logging requests, but that will be proportional to the buffer size. 

---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-dev-help@logging.apache.org







      ___________________________________________________________ 
Want ideas for reducing your carbon footprint? Visit Yahoo! For Good  http://uk.promotions.yahoo.com/forgood/environment.html