You are viewing a plain text version of this content. The canonical link for it is here.
Posted to log4j-dev@logging.apache.org by Stuart Robertson <st...@gmail.com> on 2006/06/27 23:39:13 UTC

log4j features and general readyness for use

We're considering dipping our toes into log4j 1.3 waters for a not-far-off
production application, and would like to get a read on the developers'
opinion on what areas are stable for production.  I understand there are no
guarantees, we won't hold you to it, etc. etc., but still, you probably know
which areas are in flux and which are already probably how they'll look
whenever 1.3 goes live.

We'd actually be using it initially as a backward compatible replacement for
1.2 in an application where we're experiencing the dreaded deadlock issues,
which it sounds like 1.3 may help resolve.

Anyway, thanks for any input.

Stu

Re: log4j features and general readyness for use

Posted by Elias Ross <er...@m-qube.com>.
On Wed, 2006-06-28 at 12:34 -0500, Curt Arnold wrote:
> 
> 
> Elias, I would appreciate your reviewing bug 33855 and see if it  
> should be marked as a dup of 24159.  You did submit some patches  
> against 24159, but I believe the patches in 33855 obsoleted them.

Actually, it's the other way around.  I'll mark patches in 33855 as
obsolete.

> At that time the rules at  http://www.apache.org/foundation/ 
> voting.html and http://logging.apache.org/site/bylaws.html come in
> to  
> play.

Thanks for reminding me of the process.  I'll see if I can borrow a soap
box and then call for a vote.



---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-dev-help@logging.apache.org


Re: log4j features and general readyness for use

Posted by Curt Arnold <ca...@apache.org>.
On Jun 28, 2006, at 11:01 AM, Elias Ross wrote:

> On Wed, 2006-06-28 at 00:04 -0500, Curt Arnold wrote:
>
>> My personal feeling is that 1.3 is in a no-win situation.  Backwards
>> compatibility was not rigidly enforced during development.  While
>> there has been some success reworking the code to be compatible with
>> 1.2.x, I do not think that it will ever be close enough to 100%
>> compatible to be a universally recommended replacement for deployed
>> applications that are working fine with 1.2.x.  However, there are
>> many issues (primarily involving concurrency) that will require
>> breaking binary compatibility with 1.2.x to solve, but log4j 1.3.
>> does not solve them.
>
> I did offer the Log4J team a better designed logging appender package,
> which would be nice to have bundled 1.3, which does solve deadlock
> issues.  Then users who have issues can use appenders from this  
> package
> instead, which are compatible from a configuration point of view.
>
> So, instead of this:
>
> log4j.rootCategory=debug, stdout
> log4j.appender.stdout=org.apache.log4j.ConsoleAppender
> log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
>
> you write:
>
> log4j.rootCategory=debug, stdout
> log4j.appender.stdout=org.apache.log4j.concurrent.ConsoleAppender
> log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
>
> I don't know what the position is for their inclusion.  I'd like to  
> know
> either way where the community stands, a simple Yes or No would be
> appreciated.
>

Your submission does address a specific legitimate problem of logging  
calls during toString() on the message parameter, however that wasn't  
what I was alluding to when I mentioned concurrency issues that  
required breaking changes and could only be addressed in log4j 2.0.

I was primary concerned with log4j 1.2 common approach of having  
internal operations that are not safe for concurrency and wrapping  
them all within a coarse-grained lock.  The toString() issue is a  
symptom, but not the only, symptom of that approach.  The major  
downsides of that approach are that log4j does not scale well on  
multi-processor systems and it is maddeningly complex to do any  
analysis of concurrency issues since very few significant operations  
can be determined to be safe by code review.

However, if the original poster is running into deadlocks due to the  
toString() issue, then using log4j 1.3 and the concurrent appender  
contribution in the sandbox may be an reasonable solution until log4j  
2.0 is available.

For those just coming in, see:

http://issues.apache.org/bugzilla/show_bug.cgi?id=24159
http://issues.apache.org/bugzilla/show_bug.cgi?id=33855

Elias, I would appreciate your reviewing bug 33855 and see if it  
should be marked as a dup of 24159.  You did submit some patches  
against 24159, but I believe the patches in 33855 obsoleted them.

Elias's contribution was been added to the log4j sandbox for review  
and feedback back in February.  Source can be checked out using:

svn co http://svn.apache.org/repos/asf/logging/sandbox/log4j/ 
concurrent log4j-concurrent

Can be built with either Maven or Ant but each require log4j 1.3 to  
be provided.  Ant build:

ant -Dlog4j.jar=/foo/log4j-1.3alpha-8.jar

Maven build requires manually adding a log4j-1.3alpha8 entry to the  
local repository.  Maven doesn't
like the hyphen in "1.3alpha-8".

You would need to add the log4j-concurrent.jar from the sandbox  
project to the classpath in addition to the log4j 1.3 jars.

The contribution was placed in the sandbox to allow the community to  
provide feedback on the contribution, but I'm not aware of any.  If  
there are people who want to step up and say that it worked well for  
them,  that would help.  It might help to have a little more  
discussion, but if you want a simple Yes or No, then the appropriate  
way to get that started would be to call for a vote.  Ideally, you'd  
start with a message that described your proposal in adequate detail  
with a subject starting with [DISCUSS], something like:

[DISCUSS] Migrate log4j-concurrent from sandbox to trunk

When the issue seems to be adequately understood, then call for a  
vote with:

[VOTE] Migrate log4j-concurrent from sandbox to trunk

At that time the rules at  http://www.apache.org/foundation/ 
voting.html and http://logging.apache.org/site/bylaws.html come in to  
play.


---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-dev-help@logging.apache.org


Re: log4j features and general readyness for use

Posted by Elias Ross <er...@m-qube.com>.
On Wed, 2006-06-28 at 00:04 -0500, Curt Arnold wrote:

> My personal feeling is that 1.3 is in a no-win situation.  Backwards  
> compatibility was not rigidly enforced during development.  While  
> there has been some success reworking the code to be compatible with  
> 1.2.x, I do not think that it will ever be close enough to 100%  
> compatible to be a universally recommended replacement for deployed  
> applications that are working fine with 1.2.x.  However, there are  
> many issues (primarily involving concurrency) that will require  
> breaking binary compatibility with 1.2.x to solve, but log4j 1.3.  
> does not solve them.

I did offer the Log4J team a better designed logging appender package,
which would be nice to have bundled 1.3, which does solve deadlock
issues.  Then users who have issues can use appenders from this package
instead, which are compatible from a configuration point of view.

So, instead of this:

log4j.rootCategory=debug, stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout

you write:

log4j.rootCategory=debug, stdout
log4j.appender.stdout=org.apache.log4j.concurrent.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout

I don't know what the position is for their inclusion.  I'd like to know
either way where the community stands, a simple Yes or No would be
appreciated.



---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-dev-help@logging.apache.org


Re: log4j features and general readyness for use

Posted by Stuart Robertson <st...@gmail.com>.
Thanks Curt.  We're not using AsyncAppender, and the deadlock we're seeing
involves a custom appender extending AppenderSkeleton, in the doAppend()
method.  From reading of others' issues like this it sounds like it's
related to the RollingAppender issues.  Unfortunately the thread dump we
have doesn't include one vital stacktrace.

But here's what I do know.  The custom appender thread is stuck here:

Thread Name : Servlet.Engine.Transports : 1
State : Deadlock/Waiting on condition
Owns Monitor Lock on
com.hcsc.eas.framework.services.logging.ExceptionAppender@1CB39288/1CB39290
Waiting for Monitor Lock on org.apache.log4j.spi.RootCategory@1C7ED468
/1C7ED470
Java Stack
at com.hcsc.eas.framework.services.logging.ExceptionAppender.append(
ExceptionAppender.java:212)
at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java(Compiled
Code))
at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(
AppenderAttachableImpl.java(Compiled Code))
at org.apache.log4j.Category.callAppenders(Category.java(Compiled Code))
at org.apache.log4j.Category.forcedLog(Category.java:372)
at org.apache.log4j.Category.log(Category.java:834)


The monitor it's waiting for is held by the thread we don't have a
stacktrace for, but what we do know is this:

Thread Name : ORB.thread.pool : 2
State : Deadlock/Waiting on condition
Owns Monitor Lock on org.apache.log4j.spi.RootCategory@1C7ED468/1C7ED470
Waiting for Monitor Lock on
com.hcsc.eas.framework.services.logging.ExceptionAppender@1CB39288/1CB39290


The first thread apparently also owns a lock on a particular logger which
all of the other threads are waiting on - the threads all look like the
following.

Thread Name : ORB.thread.pool : 7
State : Waiting on condition
Waiting for Monitor Lock on org.apache.log4j.Logger@1CB50468/1CB50470


Anyway, I'm sure this isn't enough for you to understand the issue - I
haven't really untangled it myself and unfortuantely don't have full
stacktrace info.  But your answer about the pros/cons of log4j 1.3 was
helpful.  I'll try out the current alpha snapshot with our application and
see if it a) works at all and b) helps with our issue.

Thanks again,

Stu

On 6/28/06, Curt Arnold <ca...@apache.org> wrote:
>
>
> On Jun 27, 2006, at 4:39 PM, Stuart Robertson wrote:
>
> > We're considering dipping our toes into log4j 1.3 waters for a not-
> > far-off production application, and would like to get a read on the
> > developers' opinion on what areas are stable for production.  I
> > understand there are no guarantees, we won't hold you to it, etc.
> > etc., but still, you probably know which areas are in flux and
> > which are already probably how they'll look whenever 1.3 goes live.
> >
> > We'd actually be using it initially as a backward compatible
> > replacement for 1.2 in an application where we're experiencing the
> > dreaded deadlock issues, which it sounds like 1.3 may help resolve.
> >
> > Anyway, thanks for any input.
> >
> > Stu
>
> Depends on what issue that you are running into.
>
> If you are running into AsyncAppender related deadlocks, the log4j
> 1.2 branch should have the recent AsyncAppender rewrite in it.  I'm
> hesitant to support a log4j 1.2.x release with the new AsyncAppender
> since there has been very little feedback on it.  The log4j 1.2
> AsyncAppender was seriously broken, but had been for a long time and
> people either weren't using it, weren't running into the problems or
> had worked around them.  Though there has been no plans for
> additional 1.2.x. releases, there is nothing preventing that decision
> being revisited.
>
> If you are talking about deadlocks due the message parameter's
> toString() method making log4j calls itself, then I'm not sure that
> 1.3 buys you anything.  I haven't seen a way to solve the problem
> that doesn't introduce a potential compatibility problem.  Though it
> wasn't documented, log4j has never (as far as I know) supported that
> scenario and so potentially breaking deployed applications to allow
> an improvement for new applications is not something that I could
> support in a 1.x release.
>
> My personal feeling is that 1.3 is in a no-win situation.  Backwards
> compatibility was not rigidly enforced during development.  While
> there has been some success reworking the code to be compatible with
> 1.2.x, I do not think that it will ever be close enough to 100%
> compatible to be a universally recommended replacement for deployed
> applications that are working fine with 1.2.x.  However, there are
> many issues (primarily involving concurrency) that will require
> breaking binary compatibility with 1.2.x to solve, but log4j 1.3.
> does not solve them.
>
> I have outline some of my thoughts on log4j 2.0 development in the
> past and would love to spend some time brainstorming and getting an
> outline and some sample code together.  However, I've been resource
> constrained due to revenue generating projects.  However, I think it
> is close to time to opening up a log4j 2.0 branch and getting some
> development started.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
> For additional commands, e-mail: log4j-dev-help@logging.apache.org
>
>

Re: log4j features and general readyness for use

Posted by Curt Arnold <ca...@apache.org>.
On Jun 27, 2006, at 4:39 PM, Stuart Robertson wrote:

> We're considering dipping our toes into log4j 1.3 waters for a not- 
> far-off production application, and would like to get a read on the  
> developers' opinion on what areas are stable for production.  I  
> understand there are no guarantees, we won't hold you to it, etc.  
> etc., but still, you probably know which areas are in flux and  
> which are already probably how they'll look whenever 1.3 goes live.
>
> We'd actually be using it initially as a backward compatible  
> replacement for 1.2 in an application where we're experiencing the  
> dreaded deadlock issues, which it sounds like 1.3 may help resolve.
>
> Anyway, thanks for any input.
>
> Stu

Depends on what issue that you are running into.

If you are running into AsyncAppender related deadlocks, the log4j  
1.2 branch should have the recent AsyncAppender rewrite in it.  I'm  
hesitant to support a log4j 1.2.x release with the new AsyncAppender  
since there has been very little feedback on it.  The log4j 1.2  
AsyncAppender was seriously broken, but had been for a long time and  
people either weren't using it, weren't running into the problems or  
had worked around them.  Though there has been no plans for  
additional 1.2.x. releases, there is nothing preventing that decision  
being revisited.

If you are talking about deadlocks due the message parameter's  
toString() method making log4j calls itself, then I'm not sure that  
1.3 buys you anything.  I haven't seen a way to solve the problem  
that doesn't introduce a potential compatibility problem.  Though it  
wasn't documented, log4j has never (as far as I know) supported that  
scenario and so potentially breaking deployed applications to allow  
an improvement for new applications is not something that I could  
support in a 1.x release.

My personal feeling is that 1.3 is in a no-win situation.  Backwards  
compatibility was not rigidly enforced during development.  While  
there has been some success reworking the code to be compatible with  
1.2.x, I do not think that it will ever be close enough to 100%  
compatible to be a universally recommended replacement for deployed  
applications that are working fine with 1.2.x.  However, there are  
many issues (primarily involving concurrency) that will require  
breaking binary compatibility with 1.2.x to solve, but log4j 1.3.  
does not solve them.

I have outline some of my thoughts on log4j 2.0 development in the  
past and would love to spend some time brainstorming and getting an  
outline and some sample code together.  However, I've been resource  
constrained due to revenue generating projects.  However, I think it  
is close to time to opening up a log4j 2.0 branch and getting some  
development started.


---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-dev-help@logging.apache.org