You are viewing a plain text version of this content. The canonical link for it is here.
Posted to log4j-dev@logging.apache.org by Ceki Gülcü <ce...@qos.ch> on 2005/02/09 13:20:50 UTC
Taxonomy of JCL problems
Hello,
For those interested I've been busy writing an article about the problems
encountered when using JCL.
http://www.qos.ch/logging/classloader.jsp
Your comments and critical reviews are welcome.
--
Ceki Gülcü
The complete log4j manual: http://www.qos.ch/log4j/
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-dev-help@logging.apache.org
Re: Taxonomy of JCL problems
Posted by Ceki Gülcü <ce...@qos.ch>.
At 04:01 PM 2/9/2005, Niclas Hedhman wrote:
>On Wednesday 09 February 2005 20:20, Ceki Gülcü wrote:
> > Hello,
> >
> > For those interested I've been busy writing an article about the problems
> > encountered when using JCL.
> >
> > http://www.qos.ch/logging/classloader.jsp
> >
> > Your comments and critical reviews are welcome.
>
>Great work, BUT on *my* browser the 'code' sections are all on a single line,
>which is most probably a bug in this browser's CSS handling of your
>white-space: pre; style.
>
>Any chance that you can easily insert a <pre> tag for those paragraphs??
Done. Thanks Niclas.
>TIA,
>
>Niclas
--
Ceki Gülcü
The complete log4j manual: http://www.qos.ch/log4j/
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-dev-help@logging.apache.org
Re: Taxonomy of JCL problems
Posted by Ceki Gülcü <ce...@qos.ch>.
At 04:01 PM 2/9/2005, you wrote:
>On Wednesday 09 February 2005 20:20, Ceki Gülcü wrote:
> > Hello,
> >
> > For those interested I've been busy writing an article about the problems
> > encountered when using JCL.
> >
> > http://www.qos.ch/logging/classloader.jsp
> >
> > Your comments and critical reviews are welcome.
>
>Great work, BUT on *my* browser the 'code' sections are all on a single line,
>which is most probably a bug in this browser's CSS handling of your
>white-space: pre; style.
Ooops. It's broken on IE.
>Any chance that you can easily insert a <pre> tag for those paragraphs??
>
>TIA,
>
>Niclas
--
Ceki Gülcü
The complete log4j manual: http://www.qos.ch/log4j/
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-dev-help@logging.apache.org
Re: Taxonomy of JCL problems
Posted by Niclas Hedhman <ni...@hedhman.org>.
On Wednesday 09 February 2005 20:20, Ceki Gülcü wrote:
> Hello,
>
> For those interested I've been busy writing an article about the problems
> encountered when using JCL.
>
> http://www.qos.ch/logging/classloader.jsp
>
> Your comments and critical reviews are welcome.
Great work, BUT on *my* browser the 'code' sections are all on a single line,
which is most probably a bug in this browser's CSS handling of your
white-space: pre; style.
Any chance that you can easily insert a <pre> tag for those paragraphs??
TIA,
Niclas
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-dev-help@logging.apache.org
Re: Taxonomy of JCL problems
Posted by Ceki Gülcü <ce...@qos.ch>.
At 04:54 PM 2/10/2005, Niclas Hedhman wrote:
>Is there any way we could get the Jakarta PMC and/or the ASF Board to kill
>the
>Commons-Logging project once and for all?
Escalating this issue to the Board would require a huge amount of
effort, cause much aggravation and generate a lot of bad blood.
I hope that when people realize that severity of the bugs in JCL,
either of two things happen.
1) The bugs in JCL get corrected and as a result log4j suffers less
at the hands of bugs originating in JCL.
2) The bugs in JCL do not get corrected. People slowly migrate to
UGLI.
Trust natural selection to do its wonders.
>Cheers
>Niclas
--
Ceki Gülcü
The complete log4j manual: http://www.qos.ch/log4j/
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-dev-help@logging.apache.org
RE: Taxonomy of JCL problems
Posted by Yoav Shapira <yo...@MIT.EDU>.
Hi,
>Is there any way we could get the Jakarta PMC and/or the ASF Board to kill
>the Commons-Logging project once and for all?
I doubt it. There are strong supporters for JCL within and outside the ASF.
As usual, the critics tend to be louder than the supporters, but people are
voting by taking the action of using JCL.
Whatever strategy we propose, e.g. UGLI, should also account for a gradual
transfer from today's JCL-based approaches to the approach we propose. A
solution that calls for today's projects to suddenly just make the switch
won't fly. But the organized and well-thought-out effort we're starting
here is certainly the way to go.
Yoav
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-dev-help@logging.apache.org
Re: Taxonomy of JCL problems
Posted by Niclas Hedhman <ni...@hedhman.org>.
On Wednesday 09 February 2005 20:20, Ceki Gülcü wrote:
> Hello,
>
> For those interested I've been busy writing an article about the problems
> encountered when using JCL.
>
> http://www.qos.ch/logging/classloader.jsp
Is there any way we could get the Jakarta PMC and/or the ASF Board to kill the
Commons-Logging project once and for all?
As late as a few minutes ago, the new JDO project (yes, the real JDO folks are
now at Apache) entered into their wiki that they will use commons-logging in
the Reference Implementation.
http://wiki.apache.org/jdo/LoggingDetails
Cheers
Niclas
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-dev-help@logging.apache.org
RE: Taxonomy of JCL problems
Posted by Ceki Gülcü <ce...@qos.ch>.
At 04:26 PM 2/9/2005, Yoav Shapira wrote:
>Hi,
>Awesome article. Thanks for writing it and for fixing the code sections. I
>suggest you invite comments on commons-dev to see what the JCL people think.
The comments I received so far have not been very supportive but that is to
be expected.
>Another useful addition would be an expanded section on how UGLI
>approaches/solves these problems.
OK, I added a paragraph or two on this just before the conclusion.
>Yoav
--
Ceki Gülcü
The complete log4j manual: http://www.qos.ch/log4j/
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-dev-help@logging.apache.org
Re: Taxonomy of JCL problems
Posted by Niclas Hedhman <ni...@hedhman.org>.
On Thursday 10 February 2005 18:27, Ceki Gülcü wrote:
> UGLI is rather modest in its goals. It does not have the pretension of
> initiating any major technological leap. JCL promises more than what it can
> deliver. UGLI does what JCL does today but without the headaches.
This is a *very* good assessment.
> Given the above, I am quite willing to support any third-party initiative,
> going well beyond UGLI, that provides a reliable and well-tested solution.
> Metro may be such a solution.
That's really generous :o)
We (Metro) have had some 'issues' with Log4J in the following areas;
1. API / management+spi / impl separation. --> Something that has been started
in Log4J and will give us a lot of benfits. So all kudos to that. It is a BIG
step in the right direction.
2. Logger "discovery" is the second area we are struggling with. As you know,
I am a big fan of Inversion of Control, which basically disallow objects to
'get' and 'create' its own stuff, and should be handed objects to use.
I see that a lot of stuff (RepositorySelectors) have happened in this field,
which I have not studied (maybe that was what the request some 8-10months ago
to change to 1.3 was all about?) and need to assess the implications of.
Briefly, it looks like we can work with this from the "discovery" point of
view. I really need to study this.
3. Runtime upgrades of Log4J itself!
Metro has an ambition that most subsystems should be upgradable without
shutting the whole thing down. "Upgrade" in this case is both a matter of
getting the implementation of the various parts and core updated (e.g.
bugfixes) but also that I can add custom stuff, such as new appenders or
formatters. Static methods and final classes are generally a real PITA, since
it prevents interception (without bytecode manipulation) of the references,
making it difficult to get rid of the classes from memory.
This issue will probably remain, and we can get back to this after the rest is
in place (see below).
I think that we will be able to create a LoggerRepository that ties into the
Transit Plugin system, together with a clever build system add-on that can
package Log4J in a way so that this works.
Question; Does the LoggerRepository and RepositorySelector system have to be
part of the configuration system (PropertiesConfigurator, DOMConfigurator et
al), or are they responsible for their own configuration of Loggers,
Appenders and Formatters?
If the latter, then I think we have "a good deal" that we can work with,
including getting a fairly good plugin system for Log4J (with the JDK1.4
constraint). If the configuration system is 'core', it may make it a bit
trickier, as appenders/formatters are referenced directly with classnames. I
still think some 'hack' could overcome that.
However, as I mentioned privately, I am pretty busy with other stuff right
now, so it will probably take 6-12months before something materializes...
Cheers
Niclas
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-dev-help@logging.apache.org
Re: Taxonomy of JCL problems
Posted by Ceki Gülcü <ce...@qos.ch>.
At 09:17 AM 2/10/2005, Niclas Hedhman wrote:
>On Thursday 10 February 2005 03:34, Ceki Gülcü wrote:
> > nless the ugli-xxx.jar file is corrupt, UGLI cannot never cause bugs of
> > Type-I, Type-II or Type-III for the simple reason that all the required
> > files are bundled together in the ugli-xxx jar file. UGLI's LoggerFactory
> > only uses the class laoder that loaded LoggerFactory itself to instantiate
> > other classes. In future versions of UGLI, the binding will done at compile
> > time, so such questions won't even need to be asked.
>
>Hmmmm....
>So you are saying, that if I for each commons-logging.jar includes log4j.jar
>in the same classloader, I still have TypeII problems??
Yes. JCL suffers from Type-II problems but only in child-first trees.
>That is a case that your page doesn't cover, and right now I can't see that
>something bad would happen (in parent-first CLs. IMHO Child-First are plainly
>bad.), and would be interested to be informed of such scenario.
Example 4 covers this case (in a child-first tree).
>Don't get me wrong, I don't like JCL, but IMHO we are dealing with 3
>different
>types of problems;
>1. Architectural.
>2. Deployment and misconfigurations.
>3. Bugs.
>
>And it is at times difficult to separate them apart. I can probably agree
>that
>UGLI addresses 2 and 3. 3. is IMO a non-issue, and just a matter of kicking
>the JCL's butt. 2. is the matter of UGLI forcing the proper bundling.
>1. is what really interests *me* as a 'container author'.
UGLI is rather modest in its goals. It does not have the pretension of
initiating any major technological leap. JCL promises more than what it can
deliver. UGLI does what JCL does today but without the headaches.
Given the above, I am quite willing to support any third-party initiative,
going well beyond UGLI, that provides a reliable and well-tested solution.
Metro may be such a solution.
>Cheers
>Niclas
--
Ceki Gülcü
The complete log4j manual: http://www.qos.ch/log4j/
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-dev-help@logging.apache.org
Re: Taxonomy of JCL problems
Posted by Niclas Hedhman <ni...@hedhman.org>.
On Thursday 10 February 2005 03:34, Ceki Gülcü wrote:
> nless the ugli-xxx.jar file is corrupt, UGLI cannot never cause bugs of
> Type-I, Type-II or Type-III for the simple reason that all the required
> files are bundled together in the ugli-xxx jar file. UGLI's LoggerFactory
> only uses the class laoder that loaded LoggerFactory itself to instantiate
> other classes. In future versions of UGLI, the binding will done at compile
> time, so such questions won't even need to be asked.
Hmmmm....
So you are saying, that if I for each commons-logging.jar includes log4j.jar
in the same classloader, I still have TypeII problems??
That is a case that your page doesn't cover, and right now I can't see that
something bad would happen (in parent-first CLs. IMHO Child-First are plainly
bad.), and would be interested to be informed of such scenario.
Don't get me wrong, I don't like JCL, but IMHO we are dealing with 3 different
types of problems;
1. Architectural.
2. Deployment and misconfigurations.
3. Bugs.
And it is at times difficult to separate them apart. I can probably agree that
UGLI addresses 2 and 3. 3. is IMO a non-issue, and just a matter of kicking
the JCL's butt. 2. is the matter of UGLI forcing the proper bundling.
1. is what really interests *me* as a 'container author'.
Cheers
Niclas
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-dev-help@logging.apache.org
Re: Taxonomy of JCL problems
Posted by Ceki Gülcü <ce...@qos.ch>.
At 05:46 PM 2/9/2005, Niclas Hedhman wrote:
>On Wednesday 09 February 2005 23:34, you wrote:
> > As you are very knowledgeable in class loader
> > problems, I'd appreciate your comments and/or critical reviews.
>
>First of all, I am very happy to see that someone else care, and care to such
>an extent and goes through all the pain to explain this to everyone to such
>detail. There are some minor mistakes, like
><quote>
>Class loading problems encountered when using JCL fall into two main
>categories:
></quote>
LOL.
>JCL has been a favourite hate-object of mine for quite a long time, and I am
>surprised that it is still around and even seems to gain popularity... :o(
It's both surprising and fascinating at the same time.
>But, the same problem as exists with the JCL <-> Log4J relationship, exist
>between "Log4J core" <-> Appenders (e.g.), although at a much smaller scale.
Absolutely.
>The Appenders must be loaded by the same classloader as Log4J, and that being
>at start-up.
>This is basically the same requirement that exist in JCL as well.
><quote>
>this API must be located at the same level or higher in the class loader tree.
></quote>
Yes and no. JCL's discovery mechanism (surprisingly enough) stands in the
way of several configurations which otherwise would have been possible by
just breaking up the core and the extensions. Log4j 1.3 is broken into
several pieces allowing log4j extensions and their dependencies to reside
in child class loader while log4j-core resides at the same level or higher
up. Unfortunately, log4j tries to load appender through the TCCL, a
practice that can be euphemistically qualified as "problematic".
>In relation to JCL, you can argue that both containers and webapps uses both
>JCL and Log4J so the problem is extremely common, whereas "my scenarios" are
>less common, but in fact the same issue, that being that blindly loading
>classes from java.lang.String is not a good practice and a ClassLoader
>argument should be provided in such contexts.
Whoa. That's a big paradigm shift.
>It is also necessary to have a proper API/Impl split, and basically push the
>impl classes as far down as possible (out of view) and keep the stable API
>classes higher up and shared.
+1
>If I use UGLI's ULogger interface I can do this, push the API (ULogger) high
>up and push the implementation(!) (Log4J) far down, then Log4J needs to be
>treated as a big blob of implementation, with stuff like the mail.jar as a
>real dependency, since configuration changes in running servers, could later
>include SMTPAender and crash (have not checked how severly lately, but used
>to be pretty badly.).
This problem "fixed" in 1.3. The current solution is not entirely
satisfactory because it relies on the TCCL.
>Now, I don't like to treat Log4J as a static monolithic blob of
>implementation, but ATM I have no choice. But this could change, and Log4J
>become a more dynamic creature, capable of adapting in non-stoppable servers.
>
>If a proper plugin model is employed for all things that are loaded
>dynamically through the configuration files, all this could be solved (not
>the JCL mess!)
>That would basically mean that the configuration specified a "plugin URI",
>which described the resources needed to load the plugin into Log4J, and Log4J
>managed the loading and unloading of those. That is what we do in DPML
>(http://www.dpml.net) for just about everything, and is not that hard. Log4J
>ends up being a 'big monolithic blob of implementation', but that is far from
>ideal.
>Unfortunately, the solution we have, called Transit, requires JDK1.4 and not
>generally available to Log4J, but something similar should be developed here.
>Current API and impl of our plugin system can be found here;
>
>http://paris.dpml.net/svn/development/main/home/transit/handler/src/main/net/dpml/transit/repository/Repository.java
>
>http://paris.dpml.net/svn/development/main/home/transit/handler/src/main/net/dpml/transit/repository/StandardLoader.java
>if you want to study what we have done.
>
>The StandardLoader uses a properties file to understand the plugin structure,
>esp what are API resources and what are the Impl resources. Find attached (if
>it goes through the mailing list) such descriptor for the Log4J provider in
>our logging system. (Note; this plugin does not define any API resources)
Before trying out more sophisticated approaches, I'd like to ascertain that
the approach adopted in 1.3 does not work. I currently don't have the time
to devote more time to this topic in any more depth.
>Hope I didn't bore everyone to death :o)
On the contrary.
>Cheers
>Niclas
>
>P.S. The paragraph that was just added about UGLI doesn't have these kinds of
>problems doesn't convince me at all. If you bundle JCL with Log4J similarily
>to how you bundle UGLI+Log4J, wouldn't the same be true for JCL?
Bugs of Type-I would be solved but JCL also suffers of Type-II even if
log4j and JCL are bundled together.
>I don't see
>anything dealing with classloaders...just like in JCL. I understand why it
>doesn't, but I don't understand how you claim there is a difference in this
>respect (except for the hash table).
Unless the ugli-xxx.jar file is corrupt, UGLI cannot never cause bugs of
Type-I, Type-II or Type-III for the simple reason that all the required
files are bundled together in the ugli-xxx jar file. UGLI's LoggerFactory
only uses the class laoder that loaded LoggerFactory itself to instantiate
other classes. In future versions of UGLI, the binding will done at compile
time, so such questions won't even need to be asked.
--
Ceki Gülcü
The complete log4j manual: http://www.qos.ch/log4j/
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-dev-help@logging.apache.org
Re: Taxonomy of JCL problems
Posted by Niclas Hedhman <ni...@hedhman.org>.
On Wednesday 09 February 2005 23:34, you wrote:
> As you are very knowledgeable in class loader
> problems, I'd appreciate your comments and/or critical reviews.
First of all, I am very happy to see that someone else care, and care to such
an extent and goes through all the pain to explain this to everyone to such
detail. There are some minor mistakes, like
<quote>
Class loading problems encountered when using JCL fall into two main
categories:
</quote>
(There exist three types of people in the world; those who can count, and
those who can't.)
Otherwise, I have not yet found any factual mistakes.
And I would love to see the Example-7 completed ;o)
JCL has been a favourite hate-object of mine for quite a long time, and I am
surprised that it is still around and even seems to gain popularity... :o(
Now, if you extend your own little pet into the same line of thinking just a
little bit further, I think you will get the point of 'previous talks' on
inherent issues in Log4J.
Log4J as it is today is both an API and an implementation, and it is somewhat
'diffuse' what is what. This is being addressed, and that is good news.
But, the same problem as exists with the JCL <-> Log4J relationship, exist
between "Log4J core" <-> Appenders (e.g.), although at a much smaller scale.
The Appenders must be loaded by the same classloader as Log4J, and that being
at start-up.
This is basically the same requirement that exist in JCL as well.
<quote>
this API must be located at the same level or higher in the class loader tree.
</quote>
In relation to JCL, you can argue that both containers and webapps uses both
JCL and Log4J so the problem is extremely common, whereas "my scenarios" are
less common, but in fact the same issue, that being that blindly loading
classes from java.lang.String is not a good practice and a ClassLoader
argument should be provided in such contexts.
It is also necessary to have a proper API/Impl split, and basically push the
impl classes as far down as possible (out of view) and keep the stable API
classes higher up and shared.
If I use UGLI's ULogger interface I can do this, push the API (ULogger) high
up and push the implementation(!) (Log4J) far down, then Log4J needs to be
treated as a big blob of implementation, with stuff like the mail.jar as a
real dependency, since configuration changes in running servers, could later
include SMTPAender and crash (have not checked how severly lately, but used
to be pretty badly.).
Now, I don't like to treat Log4J as a static monolithic blob of
implementation, but ATM I have no choice. But this could change, and Log4J
become a more dynamic creature, capable of adapting in non-stoppable servers.
If a proper plugin model is employed for all things that are loaded
dynamically through the configuration files, all this could be solved (not
the JCL mess!)
That would basically mean that the configuration specified a "plugin URI",
which described the resources needed to load the plugin into Log4J, and Log4J
managed the loading and unloading of those. That is what we do in DPML
(http://www.dpml.net) for just about everything, and is not that hard. Log4J
ends up being a 'big monolithic blob of implementation', but that is far from
ideal.
Unfortunately, the solution we have, called Transit, requires JDK1.4 and not
generally available to Log4J, but something similar should be developed here.
Current API and impl of our plugin system can be found here;
http://paris.dpml.net/svn/development/main/home/transit/handler/src/main/net/dpml/transit/repository/Repository.java
http://paris.dpml.net/svn/development/main/home/transit/handler/src/main/net/dpml/transit/repository/StandardLoader.java
if you want to study what we have done.
The StandardLoader uses a properties file to understand the plugin structure,
esp what are API resources and what are the Impl resources. Find attached (if
it goes through the mailing list) such descriptor for the Log4J provider in
our logging system. (Note; this plugin does not define any API resources)
Maybe I have strayed far too off-topic by now, and should halt it here.
If Log4J wants to improve (irregardless of JCL issues), it should not blindly
load classes either, but define a proper plugin mechanism which the
configuration system uses (mind you, I have not looked at anything in Log4J
1.3 yet!).
Hope I didn't bore everyone to death :o)
Cheers
Niclas
P.S. The paragraph that was just added about UGLI doesn't have these kinds of
problems doesn't convince me at all. If you bundle JCL with Log4J similarily
to how you bundle UGLI+Log4J, wouldn't the same be true for JCL? I don't see
anything dealing with classloaders...just like in JCL. I understand why it
doesn't, but I don't understand how you claim there is a difference in this
respect (except for the hash table).
RE: Taxonomy of JCL problems
Posted by Yoav Shapira <yo...@MIT.EDU>.
Hi,
Awesome article. Thanks for writing it and for fixing the code sections. I
suggest you invite comments on commons-dev to see what the JCL people think.
Another useful addition would be an expanded section on how UGLI
approaches/solves these problems.
Yoav
-----Original Message-----
From: Ceki Gülcü [mailto:ceki@qos.ch]
Sent: Wednesday, February 09, 2005 7:21 AM
To: log4j-dev@logging.apache.org
Subject: Taxonomy of JCL problems
Hello,
For those interested I've been busy writing an article about the problems
encountered when using JCL.
http://www.qos.ch/logging/classloader.jsp
Your comments and critical reviews are welcome.
--
Ceki Gülcü
The complete log4j manual: http://www.qos.ch/log4j/
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-dev-help@logging.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-dev-help@logging.apache.org