Re: [log4j-dev] Object rendering

[Ceki Gulcu <cg...@urbanet.ch>]

Hi Anders,

I apologize for not replying earlier. The questions you raise are at the 
heart of the debate. See my comments below.

At 23:39 08.01.2001 +0100, Anders Kristensen  wrote:
>Basically, IMHO, object rendering is, or ought to be, part of the
>existing layout mechanism for logging events.  There are two main parts
>to this
>
>   1. Object renderers exist per Appender (like Layouts) and are
>separately configurable.
>
>   2. Object rendering blends seamlessly with other layout decisions
>
>With the current Object rendering mechanism all objects belonging to the
>same class are rendered by the same renderer. I think that just like we
>have different layouts for different Appenders it is desirable to be
>able to render Objects differently depending on what Category they're
>generated on, or rather the Appender that's used. That's the first item
>above.
>
>The second item means that it's desirable to integrate object rendering
>with LoggingEvent rendering. In Tomcat one might want to log a request
>object. An admin should be able to extract two or more separate pieces
>of information from the request object and mix them up with "generic"
>conversion specifiers from PatternLayout (for example). The way this
>could work is to have a subclass of PatternLayout which recognizes
>additional conversion specifiers specific to Tomcat requests. The admin
>can then have conversion strings like, say, "%z [%d] %u%n" where maybe
>%z is the request method and %u is the request URL. We don't have that
>sort of flexibility with the current mechanism.

True. Currently, the message object that the user passes as an argument is 
transformed very early in the logging process without allowing subsequent 
log4j components to see the message object but just a message *string*. I 
am aware that this is not the smartest way to handle things. However, it 
provides a useful and simple abstraction:

Message objects are transformed to message strings according to a 
transformation rule (ObjectRenderer) specified by the user. This 
transformation is done independently of  any appenders or layouts.


In the alternative approach you propose, it would be  the layout that would 
transform the object. While this provides more degrees of freedom it 
requires more work on the part of layouts. More importantly, layouts and 
their patterns would be associated with a single object type. The 
FruitLayout would format fruits and there would be a need for another 
layout say the VegerableLayout to format vegetables. That would be 
unacceptable.

A better alternative is to allow multiple conversion patterns to be 
associated with a single Layout. For example, the user would specify:

"%d{ISO8601} %h  %t - %m%n"  ---> HttpServletRequest  (%h is the hostname 
of the requester)
"%b %color - %m%n"   ---> my.Company.Car (%b is the car brand and %color 
the car color)
......

This is clearly more complicated than the current solution although it is 
more flexible....

>Another issue is that of remote logging. The SocketAppender currently
>serializes logging events and then just regenerates events on the remote
>logging server, and so, depending on the local confiuration, might
>format and log on the logging server. I like that appraoch a lot and I
>think it would be better if, when Objects are logged, they would be
>serialized (if they're serializable that is), and sent as part of the
>LoggingEvent to the server and regenerated there, i.e. without having
>the generating VM format the Object.

This is another objection that is raised frequently, most notably by Peter 
Donald and also by Andrew Harris. Currently, only the string form of the 
message object is sent to the server. Strings are serializable which is not 
the case of all objects. Moreover, for serialization to succeed, the 
receiving (server) end has to possess a copy of the Serialized class. The 
RMI protocol for example could can  take of that detail...

Assuming we could send the object in serialized form to the other end, what 
advantage would that bring? Log4j is a logging library not a messaging 
middleware. At the end of the day, the server would still format the object 
and display it as a string. What difference does it make to transform at 
the client side rather than at the client? Your point of view would be 
greatly appreciated. Ceki

Re: [log4j-dev] Object rendering

[Anders Kristensen <ak...@dynamicsoft.com>]

Ceki Gulcu wrote:
> 
> Hi Anders,
> 
> I apologize for not replying earlier. The questions you raise are at the
> heart of the debate. See my comments below.
> 
> At 23:39 08.01.2001 +0100, Anders Kristensen  wrote:
> >Basically, IMHO, object rendering is, or ought to be, part of the
> >existing layout mechanism for logging events.  There are two main parts
> >to this
> >
> >   1. Object renderers exist per Appender (like Layouts) and are
> >separately configurable.
> >
> >   2. Object rendering blends seamlessly with other layout decisions
> >
> >With the current Object rendering mechanism all objects belonging to the
> >same class are rendered by the same renderer. I think that just like we
> >have different layouts for different Appenders it is desirable to be
> >able to render Objects differently depending on what Category they're
> >generated on, or rather the Appender that's used. That's the first item
> >above.
> >
> >The second item means that it's desirable to integrate object rendering
> >with LoggingEvent rendering. In Tomcat one might want to log a request
> >object. An admin should be able to extract two or more separate pieces
> >of information from the request object and mix them up with "generic"
> >conversion specifiers from PatternLayout (for example). The way this
> >could work is to have a subclass of PatternLayout which recognizes
> >additional conversion specifiers specific to Tomcat requests. The admin
> >can then have conversion strings like, say, "%z [%d] %u%n" where maybe
> >%z is the request method and %u is the request URL. We don't have that
> >sort of flexibility with the current mechanism.
> 
> True. Currently, the message object that the user passes as an argument is
> transformed very early in the logging process without allowing subsequent
> log4j components to see the message object but just a message *string*.

Exactly. That's the problem in a nutshell.

> I
> am aware that this is not the smartest way to handle things. However, it
> provides a useful and simple abstraction:
> 
> Message objects are transformed to message strings according to a
> transformation rule (ObjectRenderer) specified by the user. This
> transformation is done independently of  any appenders or layouts.
> 
> In the alternative approach you propose, it would be  the layout that would
> transform the object. While this provides more degrees of freedom it
> requires more work on the part of layouts. More importantly, layouts and
> their patterns would be associated with a single object type. The
> FruitLayout would format fruits and there would be a need for another
> layout say the VegerableLayout to format vegetables. That would be
> unacceptable.

I see what you're saying.

I think the solution, whatever it might be, will have to exist at two
levels so to speak. At the core log4j level LoggingEvent.message should
become an Object and Category should not render message Objects but just
pass LoggingEvents with "raw" Objects to backend Appenders. Then at
least people are not forced to live with whatever Object rendering
policy "standard" log4j Appenders and Layouts implement.

The next level up has to do with how Appenders and Layouts actually
handle those Object messages and it's less obvious what a good strategy
is there...

> 
> A better alternative is to allow multiple conversion patterns to be
> associated with a single Layout. For example, the user would specify:
> 
> "%d{ISO8601} %h  %t - %m%n"  ---> HttpServletRequest  (%h is the hostname
> of the requester)
> "%b %color - %m%n"   ---> my.Company.Car (%b is the car brand and %color
> the car color)
> ......

So this means that the appender in doAppend(event) would map
event.message.getClass() to a Layout and then use that for rendering?
Hmm, yes that might work.

Another option would be to have Layouts invoke the existing Object
rendering mechanism. So if you're using a PatternLayout with a %m
conversion, the Layout would actually do exactly what Category is
currently doing to transform the message Object into a String. Same for
the other Layouts.

I believe this is backwards compatible with the existing Object
rendering mechanism (for example, has the same simplicity) but in
addition allows custom rendering, e.g. new conversion specifiers to be
defined and configured on a per-Appender basis.

There is one comlication: Object rendering as currently defined depends
on access to the Categorys Hierarchy. This is not currently available to
Layouts. This is because LoggingEvents store the Category name rather
than the Category itself. The reason for this, I believe, was to
simplify serialization of LoggingEvents where we want the name to be
sent, and not the Category. It should be possible to solve this, though.
We can set Category in LoggingEvents (and add a getRenderer(Object msg)
or getRendererMap() to Category) and then add some code to LoggingEvent
which causes serialization to work exactly like it works now -- i.e.
serialize LoggingEvent.category as a String rather than by actually
serializing the Category instance which would not work. In fact this is
something which I think we should do regardless.

BTW there's a good discussion of serialization at
http://developer.java.sun.com/developer/TechTips/2000/tt0229.html

> 
> This is clearly more complicated than the current solution although it is
> more flexible....
> 
> >Another issue is that of remote logging. The SocketAppender currently
> >serializes logging events and then just regenerates events on the remote
> >logging server, and so, depending on the local confiuration, might
> >format and log on the logging server. I like that appraoch a lot and I
> >think it would be better if, when Objects are logged, they would be
> >serialized (if they're serializable that is), and sent as part of the
> >LoggingEvent to the server and regenerated there, i.e. without having
> >the generating VM format the Object.
> 
> This is another objection that is raised frequently, most notably by Peter
> Donald and also by Andrew Harris. Currently, only the string form of the
> message object is sent to the server. Strings are serializable which is not
> the case of all objects. Moreover, for serialization to succeed, the
> receiving (server) end has to possess a copy of the Serialized class. The
> RMI protocol for example could can  take of that detail...
> 
> Assuming we could send the object in serialized form to the other end, what
> advantage would that bring? Log4j is a logging library not a messaging
> middleware. At the end of the day, the server would still format the object
> and display it as a string. What difference does it make to transform at
> the client side rather than at the client? Your point of view would be
> greatly appreciated. Ceki

That's a valid question. I don't think one can say definitively that
formatting on the client side or one the server side is the "right"
thing to do. They have different strengths and weaknesses. I do think,
however, that it makes sense to try to be consistent. For better or
worse (mostly better I think) the SocketAppender serializes
LoggingEvents and sends them to the server where they're unserialized
and regenerated. Staying within that paradigm for Objects seem like the
more logical approach to me. More concretely I would say that the main
advantage of sending LoggingEvents and not strings is that multiple
clients can share a centralized logging server and an admin only has to
reconfigure in one place when something changes. Doing Object rendering
on clients to some extent negates that advantage.

Anyway, I think people should have the option of writing appenders that
do both -- send just strings or serialized objects. The main problem
with the current Object rendering as I see it is that Objects are
transformed before Appenders get a chance to see the original Object. By
insisting on transforming early we take away the option for people to
handle Object rendering in whatever way makes the most sense in their
application.

My 2 danske ører worth :)

--
Anders Kristensen