You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by Roland Weber <ht...@dubioso.net> on 2006/03/04 20:08:44 UTC
The Logging Problem
Hello all,
I am in doubt what to do about logging. It is one of our project goals
to minimize the dependencies on external libraries, including logging.
http-core currently does that in a very straight-forward way: it does
not log at all. Only the HttpService has some logging statements, which
call methods that do nothing and have to be overridden in a derived
class in order to generate output.
I don't think this approach will last. Wire logs are our primary means
to help people with strange server behavior, or to detect faults in the
requests built by their applications. The non-wire logs are important
to track problems in the HttpClient behavior. I believe we'll need both
for the HTTP components, too. I sure don't want to waste my time with
guessing what the components do in absence of log files.
Within http-core, we can implement some workarounds. For example, we
can use different implementations of connections or data reader/writer
to get a wire log, and different implementations of HttpMessage and
derived interfaces to log information about header updates. But that
is second-rate logging, because the log statements are not where the
action takes place. It also requires users to change their application
code in order to generate log output. Or to use and reconfigure a
Spring style mechanism. We can't tell them anymore to just add a few
properties here or there to enable logging.
I understand the desire to not have to configure commons-logging. But
as a developer, I do have a strong desire to get an informative log if
people expect me to help them. And if the decision is between my time
for analyzing their problems, or their time for installing a logging
component, you can guess what I prefer :-)
Even if we can live without logging in http-core, I expect most people
to use some other components as well, like http-auth or http-client.
The desire to not install a logging component will hardly be addressed
if frequently used non-core components do require one. Are we going to
have a discussion for each component whether it is complex enough to
require log statements, or is used so frequently that it should not
depend on logging?
The best idea I've had so far to address both concerns - and I'm not
claiming it is a good idea - is to compile two versions of each component.
The default one with logging statements, and an alternative nolog version
where all log statements and other dependencies on logging (imports)
have been removed from the source.
Technically, that requires preprocessing to generate the nolog version.
I have found this blog entry about using Ant filters for preprocessing:
http://weblogs.java.net/blog/schaefa/archive/2005/01/how_to_do_condi_1.html
The charm of this idea is that the source code remains valid Java source
and can always be compiled without preprocessing. Even the line numbers
of the preprocessed source will match those of the original source, which
is important for interpreting stack traces.
A second logging question is which logging framework we should use. There
is still a decision pending about whether to switch from commons-logging
to Log4j UGLI: http://issues.apache.org/bugzilla/show_bug.cgi?id=32937
I implicitly assume that we want to use the same logging framework for
all HTTP components that require logging, so this should be discussed
before I put logging into http-async. Currently, I am not convinced of
the advantages of UGLI over commons-logging, but I haven't yet studied
UGLI in detail.
I intend to have another programming stint on http-async next weekend and
a few days of the week after. I hope for feedback on the logging questions
until then. If we come to a common understanding, and if time permits,
I would like to make http-async the pilot for all our components that are
going to use logging. But considering the effort it would take, I don't
want to rush ahead and implement something that will not be accepted as
the basis for other HTTP components.
Please let me know what you think.
cheers,
Roland
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: The Logging Problem
Posted by Ortwin Glück <od...@odi.ch>.
Dion Gillard wrote:
> I'm with you. Keep the commons-logging dependency.
Thanks. You know, sometimes I wish the stock JDK classes would make use
of logging. Especially those that do networking, so you could figure out
why something just doesn't work.
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: The Logging Problem
Posted by Dion Gillard <di...@gmail.com>.
I'm with you. Keep the commons-logging dependency.
On 3/6/06, Ortwin Glück <od...@odi.ch> wrote:
>
> Guys,
>
> I really don't see the problem with the commons-logging dependency.
> Dependencies to other libraries are a completely normal thing in modern
> software engineering. And commons logging is not an exotic one but very
> common. I simply don't want to make complicated workarounds to eliminate
> the dependency. Neither do I want to live without logging.
>
> Odi
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
>
>
--
http://www.multitask.com.au/people/dion/
Chuck Norris sleeps with a night light. Not because Chuck Norris is afraid
of the dark, but because the dark is afraid of Chuck Norris
Re: The Logging Problem
Posted by Ortwin Glück <od...@odi.ch>.
Guys,
I really don't see the problem with the commons-logging dependency.
Dependencies to other libraries are a completely normal thing in modern
software engineering. And commons logging is not an exotic one but very
common. I simply don't want to make complicated workarounds to eliminate
the dependency. Neither do I want to live without logging.
Odi
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: The Logging Non-Problem
Posted by Dion Gillard <di...@gmail.com>.
+1
On 3/6/06, Michael Becke <mb...@gmail.com> wrote:
>
> So how does this sound?
>
> - If/when we do end up using a logging package we will use Commons Logging
> - Commons Logging will be a dependency for HttpClient and most likely
> http-async and http-conn
> - Wire logging will be accomplished via a custom sender/receiver
> wrapper. This wrapper will be included as a part of HttpClient, and
> can always be used independently from SVN.
> - Trace logging is of no real help and will not be included in any of
> the packages. People who really want it could always add via aspects,
> etc.
> - Our goal is to do without logging for the core APIs. This will
> continue to be our plan until we run into a situation that changes our
> mind.
>
> Mike
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
>
>
--
http://www.multitask.com.au/people/dion/
Chuck Norris sleeps with a night light. Not because Chuck Norris is afraid
of the dark, but because the dark is afraid of Chuck Norris
Re: The Logging Non-Problem
Posted by Roland Weber <ht...@dubioso.net>.
Hi Mike,
> Yes, this cannot really be done without a logging package. As I
> mentioned, seems like http-async will need logging.
Then I am at ease again. I was a little concerned about:
> - Trace logging is of no real help and will not be included in any of
> the packages.
You were obviously referring to the entry/exit traces only,
and for those I agree. I just wanted to make sure that I am
at liberty to distribute log statements across http-async
as I feel is necessary.
cheers,
Roland
PS: Thanks for the lyrics - really fitting :-)
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: The Logging Non-Problem
Posted by Michael Becke <mb...@gmail.com>.
On 3/6/06, Roland Weber <ht...@dubioso.net> wrote:
> Well, maybe I'm a little behind on those modern techniques. The last thing
> I heard was that aspects can add things at the beginning or end of methods,
> but not within. In other words, aspects would give me entry/exit traces.
> Those are the traces I hardly care about. They mostly are textual noise
> that drowns the important information in the log.
Agreed.
> What I want to use, and what I am using on an almost daily basis at work,
> are traces within a method that tell me whether the code executes the
> ...
> expect multithreading problems to be the major part of problems with
> http-async (assuming that anyone is going to use it in the first place :-)
> Neither do I trust sophisticated, nebulous applications to instrument my
> code with really useful trace logging, because only I (or other humans
> that dig into the source code) know where the few crucial parts of the
> code are, and what information needs to be logged in order to trace their
> behavior with little noise.
Yes, this cannot really be done without a logging package. As I
mentioned, seems like http-async will need logging.
Mike
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: The Logging Non-Problem
Posted by Roland Weber <ht...@dubioso.net>.
Michael Becke wrote:
> So how does this sound?
>
> - If/when we do end up using a logging package we will use Commons Logging
> - Commons Logging will be a dependency for HttpClient and most likely
> http-async and http-conn
> - Wire logging will be accomplished via a custom sender/receiver
> wrapper. This wrapper will be included as a part of HttpClient, and
> can always be used independently from SVN.
That sounds good so far.
> - Trace logging is of no real help and will not be included in any of
> the packages. People who really want it could always add via aspects,
> etc.
Well, maybe I'm a little behind on those modern techniques. The last thing
I heard was that aspects can add things at the beginning or end of methods,
but not within. In other words, aspects would give me entry/exit traces.
Those are the traces I hardly care about. They mostly are textual noise
that drowns the important information in the log.
What I want to use, and what I am using on an almost daily basis at work,
are traces within a method that tell me whether the code executes the
"if" or the "else" branch, or which values some interesting variables have,
and so on. In other words, trace logging.
Apart from manually inserted System.out.println statements, trace logging
is my primary means of debugging my code. It is the only help on which I
can rely once the code is running as part of somebody else's application.
And I find it to be of immeasurable and very real help.
I've noticed before that you do not really make use of such logging:
http://mail-archives.apache.org/mod_mbox/jakarta-commons-httpclient-dev/200404.mbox/%3c9FF54B46-998A-11D8-8D93-00306557E112@u.washington.edu%3e
[...] We generally don't use the different loggers to filter logging[...]
But my development style very much depends on it. Of course I could code
locally with trace logging and remove the log statements for the patches.
But I have no intention of wasting my time by chasing user's multithreading
problems in the absence of log output from "my" background threads. And I
expect multithreading problems to be the major part of problems with
http-async (assuming that anyone is going to use it in the first place :-)
Neither do I trust sophisticated, nebulous applications to instrument my
code with really useful trace logging, because only I (or other humans
that dig into the source code) know where the few crucial parts of the
code are, and what information needs to be logged in order to trace their
behavior with little noise.
That's why I suggested to compile two versions of the component(s) in the
first place: one without logging for the people who don't want the
dependency, and one with logging for the people who want me to help them.
> - Our goal is to do without logging for the core APIs. This will
> continue to be our plan until we run into a situation that changes our
> mind.
I have no problem with that.
cheers,
Roland
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: The Logging Non-Problem
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Sun, 2006-03-05 at 17:25 -0500, Michael Becke wrote:
> So how does this sound?
>
> - If/when we do end up using a logging package we will use Commons Logging
> - Commons Logging will be a dependency for HttpClient and most likely
> http-async and http-conn
> - Wire logging will be accomplished via a custom sender/receiver
> wrapper. This wrapper will be included as a part of HttpClient, and
> can always be used independently from SVN.
> - Trace logging is of no real help and will not be included in any of
> the packages. People who really want it could always add via aspects,
> etc.
> - Our goal is to do without logging for the core APIs. This will
> continue to be our plan until we run into a situation that changes our
> mind.
>
> Mike
>
+1
Oleg
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: The Logging Non-Problem
Posted by Ortwin Glück <od...@odi.ch>.
+1
Sounds very reasonable, now. Thanks.
Michael Becke wrote:
> So how does this sound?
>
> - If/when we do end up using a logging package we will use Commons Logging
> - Commons Logging will be a dependency for HttpClient and most likely
> http-async and http-conn
> - Wire logging will be accomplished via a custom sender/receiver
> wrapper. This wrapper will be included as a part of HttpClient, and
> can always be used independently from SVN.
> - Trace logging is of no real help and will not be included in any of
> the packages. People who really want it could always add via aspects,
> etc.
> - Our goal is to do without logging for the core APIs. This will
> continue to be our plan until we run into a situation that changes our
> mind.
>
> Mike
--
[web] http://www.odi.ch/
[blog] http://www.odi.ch/weblog/
[pgp] key 0x81CF3416
finger print F2B1 B21F F056 D53E 5D79 A5AF 02BE 70F5 81CF 3416
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: The Logging Non-Problem
Posted by Michael Becke <mb...@gmail.com>.
So how does this sound?
- If/when we do end up using a logging package we will use Commons Logging
- Commons Logging will be a dependency for HttpClient and most likely
http-async and http-conn
- Wire logging will be accomplished via a custom sender/receiver
wrapper. This wrapper will be included as a part of HttpClient, and
can always be used independently from SVN.
- Trace logging is of no real help and will not be included in any of
the packages. People who really want it could always add via aspects,
etc.
- Our goal is to do without logging for the core APIs. This will
continue to be our plan until we run into a situation that changes our
mind.
Mike
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: The Logging Non-Problem
Posted by Michael Becke <mb...@gmail.com>.
On 3/6/06, Roland Weber <ht...@dubioso.net> wrote:
> I don't trust tools either :-)
> Have you seen "Conan the Barbarian"? Little Conan's father
> points to the steel sword and says: "This, you can trust."
> http://www.imdb.com/title/tt0082198/quotes
Couldn't help but join in:
It's Log, Log, it's big, it's heavy, it's wood.
It's Log, Log, it's better than bad, it's good!
Everyone wants a log! You're gonna love it, Log!
Come on and get your log! Everyone needs a Log!"
http://www.redbrick.dcu.ie/~fatwa/ren/songs.html
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: The Logging Non-Problem
Posted by Roland Weber <ht...@dubioso.net>.
Hi Oleg,
> What if you do Thread#join(some_ms) on the background thread from the
> controller thread and then throw an exception?
I don't expect to implement a controller thread in http-async, except in
the examples section. The background thread(s) will be controlled by the
applications via the dispatcher and notification interfaces. I hope to
have a patch ready early next week, then you'll see what I mean.
> I suppose you have seen 'The Fifth Element'? Remember what Zorg (the Bad
> Guy) said? "You want something done, do it yourself!"
"My favorite." (after burning a puppet with a flame thrower :-)
cheers,
Roland
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: The Logging Non-Problem
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Mon, 2006-03-06 at 20:14 +0100, Roland Weber wrote:
> Hi Oleg,
>
Hi Roland
As I said, I have no intention of imposing any design constraints on
http-async and your work.
> >>In http-async, I have background threads switching states and (beginning
> >>in March) performing callbacks to application code. The operations of
> >>these threads can not be logged appropriately by http-client. Throwing
> >>exceptions is also pointless in background threads.
> >
> > I respectfully disagree. There are very simple but effective mechanisms
> > to propagate exceptions from background threads to the controller
> > thread.
>
> Any that do not rely on the controller thread to pick up the exception?
> You seem to be trustful regarding the skills of application developers,
> but I have a different mindset. I expect at least some of the application
> developers to completely screw up thread synchronization, to deadlock
> their threads or loose notifications, and then to blame http-async for
> their freezing applications. I trust the log statements I put into my
> code, but never the application. The log output will prove *them* wrong :-)
>
> >> Besides, I can't
> >>throw an exception in case of a deadlock or similar misbehavior.
> >
> > Why not?
>
> Because the thread that should throw the exception is locked dead.
>
What if you do Thread#join(some_ms) on the background thread from the
controller thread and then throw an exception?
> > Neither do I. There are enough tools that can take care of all the dirty
> > work for us.
>
> I don't trust tools either :-)
> Have you seen "Conan the Barbarian"?
In between my 'Das Kapital' reading sessions ;-)
> Little Conan's father
> points to the steel sword and says: "This, you can trust."
> http://www.imdb.com/title/tt0082198/quotes
>
>
I seriously doubt one should put the same trust into logging though
> >>hope that the developers didn't forget to put in the one log statement
> >>that would help me.
> >
> > What if they did? ;-)
>
> Then I curse the developers, start up the laptop and have a look directly
> into the source code of our product to see what might have gone wrong or
> what kind of second-rate logging information is available ;-)
>
I suppose you have seen 'The Fifth Element'? Remember what Zorg (the Bad
Guy) said? "You want something done, do it yourself!"
Oleg
> cheers,
> Roland
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: The Logging Non-Problem
Posted by Roland Weber <ht...@dubioso.net>.
Hi Oleg,
>>In http-async, I have background threads switching states and (beginning
>>in March) performing callbacks to application code. The operations of
>>these threads can not be logged appropriately by http-client. Throwing
>>exceptions is also pointless in background threads.
>
> I respectfully disagree. There are very simple but effective mechanisms
> to propagate exceptions from background threads to the controller
> thread.
Any that do not rely on the controller thread to pick up the exception?
You seem to be trustful regarding the skills of application developers,
but I have a different mindset. I expect at least some of the application
developers to completely screw up thread synchronization, to deadlock
their threads or loose notifications, and then to blame http-async for
their freezing applications. I trust the log statements I put into my
code, but never the application. The log output will prove *them* wrong :-)
>> Besides, I can't
>>throw an exception in case of a deadlock or similar misbehavior.
>
> Why not?
Because the thread that should throw the exception is locked dead.
> Neither do I. There are enough tools that can take care of all the dirty
> work for us.
I don't trust tools either :-)
Have you seen "Conan the Barbarian"? Little Conan's father
points to the steel sword and says: "This, you can trust."
http://www.imdb.com/title/tt0082198/quotes
>>hope that the developers didn't forget to put in the one log statement
>>that would help me.
>
> What if they did? ;-)
Then I curse the developers, start up the laptop and have a look directly
into the source code of our product to see what might have gone wrong or
what kind of second-rate logging information is available ;-)
cheers,
Roland
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: The Logging Non-Problem
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Sun, 2006-03-05 at 18:36 +0100, Roland Weber wrote:
> Hi Oleg,
>
> > Low level components such as http-core, http-nio, http-auth and, imo,
> > http-async must throw exceptions and never log messages either directly
> > to stdout or via a logging toolkit.
>
> In http-async, I have background threads switching states and (beginning
> in March) performing callbacks to application code. The operations of
> these threads can not be logged appropriately by http-client. Throwing
> exceptions is also pointless in background threads.
I respectfully disagree. There are very simple but effective mechanisms
to propagate exceptions from background threads to the controller
thread.
> Besides, I can't
> throw an exception in case of a deadlock or similar misbehavior.
>
Why not?
> Connection management is another example where I would hope for logging
> information about the internal state of the managed data. Though that can
> probably be addressed by providing specific toString() or similar methods,
> to be called from http-client and/or http-async. Or else not, considering
> that the thread supposed to log can be blocked when allocating a connection.
> I'm not sure about http-auth for stateful or connection based authentication.
>
I am not saying logging is not useful. I just see very little reason to
insert log traces manually. There are bytecode enhancers that can do
that for you.
> > We must not impose a specific
> > logging toolkit on the users of these components.
>
> commons-logging is a wrapper for an arbitrary logging toolkit. The one
> time I had to integrate HttpClient into an application, I was quite happy
> with it. No big problem to write an adapter, to JLog 2.x in that case.
>
This issue has been beaten to death on numerous occasions. To some the
glass will always be half-empty rather than half-full.
> > Logging per ce does not solve any problem. I regularly spend quite a bit
> > of my time analyzing huge log files when trying to help people
> > troubleshoot their problems with HttpClient. I usually find 99% of the
> > context log entries completely useless. Usually only HTTP headers are of
> > any use. Moreover, 90% of problems are caused by cookie or
> > authentication headers. A tiny fraction of the time I have spent these
> > years on grepping through all those wire logs could have been spent on
> > writing a tutorial on how to enhance HttpClient bytecode to log just
> > specific type of information.
>
> Fair concern. I agree that the overwhelming share of the problems we
> see can be handled with a wire log and/or information about the headers.
> But I have to prepare for a different kind of problems in http-async.
>
> Bytecode fuddling is something I don't want to get into. I haven't
> written assembler code since my diploma thesis, and bytecode manipulation
> is close to that level :-)
Neither do I. There are enough tools that can take care of all the dirty
work for us.
> The advantage of log statements in the code is
> that you don't have to put them there when problems show up. But given
> the interactive nature of the support we're giving, that may indeed be
> the better options.
> I'm used to a different kind of troubleshooting at work. When I am sent
> to a customer, they usually have a huge chunk of complex products
> installed, and I need to track down something without knowing where
> exactly the problem comes from. Playing with the bytecode is out of the
> question, and even adding debug code to the systems is usually a problem.
> So I enable logging of some components about which I am suspicious, and
> hope that the developers didn't forget to put in the one log statement
> that would help me.
>
What if they did? ;-)
> > I think we should eat our own dog food. Besides, there is a lot of
> > activity going out around commons [logging] these days. There are
> > indications many sound concepts originating from UGLI will get
> > incorporated into commons [logging]
>
> OK, commons-logging it is. That's my preference too.
>
I am afraid the final decision will still have to settled by a formal
vote. My preference is to stay away from the logging wars ... by not
using logging at all (whenever feasible)
> >>I intend to have another programming stint on http-async next weekend and
> >>a few days of the week after. I hope for feedback on the logging questions
> >>until then. If we come to a common understanding, and if time permits,
> >>I would like to make http-async the pilot for all our components that are
> >>going to use logging. But considering the effort it would take, I don't
> >>want to rush ahead and implement something that will not be accepted as
> >>the basis for other HTTP components.
>
> Assuming that only http-client and http-async will use logging,
> I'll forget about preprocessing and compiling different versions.
>
By no means I want to impose any design constraints on http-async.
Oleg
> cheers,
> Roland
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: The Logging Non-Problem
Posted by Roland Weber <ht...@dubioso.net>.
Hi Oleg,
> Low level components such as http-core, http-nio, http-auth and, imo,
> http-async must throw exceptions and never log messages either directly
> to stdout or via a logging toolkit.
In http-async, I have background threads switching states and (beginning
in March) performing callbacks to application code. The operations of
these threads can not be logged appropriately by http-client. Throwing
exceptions is also pointless in background threads. Besides, I can't
throw an exception in case of a deadlock or similar misbehavior.
Connection management is another example where I would hope for logging
information about the internal state of the managed data. Though that can
probably be addressed by providing specific toString() or similar methods,
to be called from http-client and/or http-async. Or else not, considering
that the thread supposed to log can be blocked when allocating a connection.
I'm not sure about http-auth for stateful or connection based authentication.
> We must not impose a specific
> logging toolkit on the users of these components.
commons-logging is a wrapper for an arbitrary logging toolkit. The one
time I had to integrate HttpClient into an application, I was quite happy
with it. No big problem to write an adapter, to JLog 2.x in that case.
> Logging per ce does not solve any problem. I regularly spend quite a bit
> of my time analyzing huge log files when trying to help people
> troubleshoot their problems with HttpClient. I usually find 99% of the
> context log entries completely useless. Usually only HTTP headers are of
> any use. Moreover, 90% of problems are caused by cookie or
> authentication headers. A tiny fraction of the time I have spent these
> years on grepping through all those wire logs could have been spent on
> writing a tutorial on how to enhance HttpClient bytecode to log just
> specific type of information.
Fair concern. I agree that the overwhelming share of the problems we
see can be handled with a wire log and/or information about the headers.
But I have to prepare for a different kind of problems in http-async.
Bytecode fuddling is something I don't want to get into. I haven't
written assembler code since my diploma thesis, and bytecode manipulation
is close to that level :-) The advantage of log statements in the code is
that you don't have to put them there when problems show up. But given
the interactive nature of the support we're giving, that may indeed be
the better options.
I'm used to a different kind of troubleshooting at work. When I am sent
to a customer, they usually have a huge chunk of complex products
installed, and I need to track down something without knowing where
exactly the problem comes from. Playing with the bytecode is out of the
question, and even adding debug code to the systems is usually a problem.
So I enable logging of some components about which I am suspicious, and
hope that the developers didn't forget to put in the one log statement
that would help me.
> I think we should eat our own dog food. Besides, there is a lot of
> activity going out around commons [logging] these days. There are
> indications many sound concepts originating from UGLI will get
> incorporated into commons [logging]
OK, commons-logging it is. That's my preference too.
>>I intend to have another programming stint on http-async next weekend and
>>a few days of the week after. I hope for feedback on the logging questions
>>until then. If we come to a common understanding, and if time permits,
>>I would like to make http-async the pilot for all our components that are
>>going to use logging. But considering the effort it would take, I don't
>>want to rush ahead and implement something that will not be accepted as
>>the basis for other HTTP components.
Assuming that only http-client and http-async will use logging,
I'll forget about preprocessing and compiling different versions.
cheers,
Roland
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: The Logging Non-Problem
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Sat, 2006-03-04 at 20:08 +0100, Roland Weber wrote:
> Hello all,
>
Hello Roland,
Low level components such as http-core, http-nio, http-auth and, imo,
http-async must throw exceptions and never log messages either directly
to stdout or via a logging toolkit. We must not impose a specific
logging toolkit on the users of these components.
> I am in doubt what to do about logging. It is one of our project goals
> to minimize the dependencies on external libraries, including logging.
> http-core currently does that in a very straight-forward way: it does
> not log at all. Only the HttpService has some logging statements, which
> call methods that do nothing and have to be overridden in a derived
> class in order to generate output.
> I don't think this approach will last. Wire logs are our primary means
> to help people with strange server behavior, or to detect faults in the
> requests built by their applications. The non-wire logs are important
> to track problems in the HttpClient behavior.
There are no plans to remove wire and context logging from HttpClient
> I believe we'll need both
> for the HTTP components, too.
I disagree rather strongly
> I sure don't want to waste my time with
> guessing what the components do in absence of log files.
> Within http-core, we can implement some workarounds.
No need for workarounds. It is absolutely trivial to add wire logging on
top of http-core by injecting a thin wrapper around standard data
receiver and data transmitter.
> For example, we
> can use different implementations of connections or data reader/writer
> to get a wire log, and different implementations of HttpMessage and
> derived interfaces to log information about header updates. But that
> is second-rate logging, because the log statements are not where the
> action takes place. It also requires users to change their application
> code in order to generate log output. Or to use and reconfigure a
> Spring style mechanism. We can't tell them anymore to just add a few
> properties here or there to enable logging.
>
> I understand the desire to not have to configure commons-logging. But
> as a developer, I do have a strong desire to get an informative log if
> people expect me to help them.
There are other means to achieve the same net result, bytecode enhancers
for instance.
> And if the decision is between my time
> for analyzing their problems, or their time for installing a logging
> component, you can guess what I prefer :-)
Logging per ce does not solve any problem. I regularly spend quite a bit
of my time analyzing huge log files when trying to help people
troubleshoot their problems with HttpClient. I usually find 99% of the
context log entries completely useless. Usually only HTTP headers are of
any use. Moreover, 90% of problems are caused by cookie or
authentication headers. A tiny fraction of the time I have spent these
years on grepping through all those wire logs could have been spent on
writing a tutorial on how to enhance HttpClient bytecode to log just
specific type of information.
> Even if we can live without logging in http-core, I expect most people
> to use some other components as well, like http-auth or http-client.
Again, there are no plans to do anything about wire/context logging in
HttpClient. However, all low level components sitting beneath it should
communicate with the caller using standard java mechanisms such as
return parameters and exceptions. It should solely a job of HttpClient
to log certain type of exceptions (malformed cookies, authentication
retries, etc)
> The desire to not install a logging component will hardly be addressed
> if frequently used non-core components do require one. Are we going to
> have a discussion for each component whether it is complex enough to
> require log statements, or is used so frequently that it should not
> depend on logging?
>
> The best idea I've had so far to address both concerns - and I'm not
> claiming it is a good idea - is to compile two versions of each component.
> The default one with logging statements, and an alternative nolog version
> where all log statements and other dependencies on logging (imports)
> have been removed from the source.
> Technically, that requires preprocessing to generate the nolog version.
> I have found this blog entry about using Ant filters for preprocessing:
> http://weblogs.java.net/blog/schaefa/archive/2005/01/how_to_do_condi_1.html
> The charm of this idea is that the source code remains valid Java source
> and can always be compiled without preprocessing. Even the line numbers
> of the preprocessed source will match those of the original source, which
> is important for interpreting stack traces.
>
> A second logging question is which logging framework we should use. There
> is still a decision pending about whether to switch from commons-logging
> to Log4j UGLI: http://issues.apache.org/bugzilla/show_bug.cgi?id=32937
> I implicitly assume that we want to use the same logging framework for
> all HTTP components that require logging, so this should be discussed
> before I put logging into http-async. Currently, I am not convinced of
> the advantages of UGLI over commons-logging, but I haven't yet studied
> UGLI in detail.
>
I think we should eat our own dog food. Besides, there is a lot of
activity going out around commons [logging] these days. There are
indications many sound concepts originating from UGLI will get
incorporated into commons [logging]
>
> I intend to have another programming stint on http-async next weekend and
> a few days of the week after. I hope for feedback on the logging questions
> until then. If we come to a common understanding, and if time permits,
> I would like to make http-async the pilot for all our components that are
> going to use logging. But considering the effort it would take, I don't
> want to rush ahead and implement something that will not be accepted as
> the basis for other HTTP components.
>
Go for it
Cheers,
Oleg
> Please let me know what you think.
>
> cheers,
> Roland
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: The Logging Problem
Posted by Ortwin Glück <od...@odi.ch>.
Guys,
I really don't see the problem with the commons-logging dependency.
Dependencies to other libraries are a completely normal thing in modern
software engineering. And commons logging is not an exotic one but very
common. I simply don't want to make complicated workarounds to eliminate
the dependency. Neither do I want to live without logging.
Odi
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org