You are viewing a plain text version of this content. The canonical link for it is here.
Posted to log4j-dev@logging.apache.org by Curt Arnold <ca...@apache.org> on 2006/01/12 07:51:40 UTC
Experimental log4j formatter in sandbox
I committed a pass at external message formatting classes in the
sandbox. The code can be checked out using:
svn co http://svn.apache.org/repos/asf/logging/sandbox/log4j/
formatter formatter
and can be built using either Maven (JDK 1.5 only) or Ant. The Ant
build can produce both a JDK 1.5 compatible jar (build with ant jar
or mvn jar:jar) which supports varargs and the java.util.Formatter
formatter and a (hopefully) JDK 1.3 compatible jar (build with ant
jar-java2) which substitutes Object[] for Object... and omits support
for the java.util.Formatter. The formatter was built and tested with
log4j 1.2.13 (latest in Maven) and should work with earlier log4j
1.2.x (may throw MethodNotFound exceptions if trace() methods are
used with pre-TRACE enabled log4j's) and log4j 1.3.
log4j 1.3, SLF4J and NLOG4J provide methods that take a substitution
pattern and one or more parameters used to form the message where the
substitution is only performed if the logger threshold is satisfied.
For example:
logger.debug("The {} jumped over the moon {} times", "cow", new
Integer(5));
One of my concerns was introducing yet another pattern syntax when
Java class libraries already have two (java.text.MessageFormat and
java.util.Formatter). I had originally thought that the substitution
pattern format was a subset of that supported by
java.text.MessageFormat, however java.text.MessageFormat would
require parameter numbers within the braces (for example, "The {0}
jumped over the moon {1} times").
Instead of enshrining one pattern syntax, the external formatters
provide three different utility classes with identical methods that
each use a different formatter. The class names are short since a
previous concern with this approach was code bloat. All the classes
are (currently) in org.apache.log4j.formatter. They are:
LogF - uses java.util.Formatter (name should invoke fond memories of
printf)
LogMF - uses java.text.MessageFormat (adds M for Message)
LogSF - uses existing log4j 1.3 syntax (adds S for Simple)
The previous line could be written as:
LogF.debug(logger, "The %s jumped over the moon %d times", "cow", 5);
LogMF.debug(logger, "The {0} jumped over the moon {1} times", "cow", 5);
LogSF.debug(logger, "The {} jumped over the moon {} times", "cow", 5);
Each class offers identical sets of methods for each level (including
TRACE). Each method takes a Logger and a pattern string as the first
two arguments. There are methods with three parameters for Object,
each of the primitive types and either Object... (JDK 1.5) or Object
[] and a 4 and a 5 parameter method with either 2 or 3 Object
parameters. The 3 parameter methods with primitives should eliminate
most unnecessary object wrapping costs. If there is a need to have
two or more primitives, JDK 1.5 will autobox the primitives and use
either the 4, 5 or vararg methods as appropriate, on JDK 1.3, you
would need to explicitly box the primitives and use the 4, 5 or Object
[] methods.
// JDK 1.5 - calls LogSF.debug(Object, Object)
LogSF.debug(logger, "Iteration {} of {}", i, n);
// JDK 1.3 - also calls LogSF.debug(Object, Object)
LogSF.debug(logger, "Iteration {} of {}", new Integer(i), new Integer
(n));
Each of these may incur the cost of creating 2 new Integer objects
for the duration of the call even if the logger threshold is not
satisfied.
The primary design goal was to minimize the cost of ineffective log
requests by deferring the cost of formatting, parameter boxing and
array creation until after the isEnabledFor() is checked. The cost
of formatting, boxing and array creation is assumed to be small
compared to the append cost and there has been no attempt to optimize
those costs when the threshold is satisfied. No performance
evaluation has been attempted yet and the assumptions need to be
confirmed.
I assume the JVM will be able to effectively "in-line" the body of
the method calls so that:
LogMF.debug(logger, "The {0} jumped over the moon {1} times", "cow", 5);
results in code that is nearly indistinguishable in performance from:
if (logger.isDebugEnabled()) {
logger.debug(MessageFormat.format("The {0} jumped over the moon
{1} times", new Object[] { "cow", new Integer(5) }));
}
It is possible that the Java 5 JVM is smart enough to defer boxing
and array creation in the vararg logging methods resulting in no
performance benefit from the non-vararg methods. In that case, then
all of the non-vararg methods could be eliminated from LogF since it
only exists in the JDK 1.5. Probably would want to still support
them for the other formatters to preserve calling compatibility
between the JDK 1.3+ and the JDK 1.5 implementations of the formatters.
Unlike the log4j 1.3 and SLF4J implementations, the LogSF will
support more than 2 substitution parameters. The implementation is
different and may need to be tweaked to exactly reproduce the log4j
1.3 and SLF4J behavior. For example, LogSF doesn't attempt to
provide any escaping mechanism to allow "{}" to appear in a
message. I'm not sure if log4j 1.3 or SLF4J do.
The methods should only throw unchecked exceptions if logger is null
or if a substitution parameter's toString method throws an
exception. They should swallow exceptions due to bad patterns or
pattern/parameter type mismatches. In general, they will just output
the pattern without substitution if there is an exception.
I will attempt to gather some performance measures over the next few
days.
The Ant build by default attempts to locate log4j-1.2.13 and
junit-3.8.1 in the user's Maven2 repository. Use
ant -Dlog4j.jar=PATH_TO_LOG4J -Djunit.jar=PATH_TO_JUNIT
to specify different locations (or specify them in build.properties).
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-dev-help@logging.apache.org
Re: Experimental log4j formatter in sandbox
Posted by Endre Stølsvik <En...@Stolsvik.com>.
|
| One of my concerns was introducing yet another pattern syntax when Java class
| libraries already have two (java.text.MessageFormat and java.util.Formatter).
| I had originally thought that the substitution pattern format was a subset of
| that supported by java.text.MessageFormat,
same here..
| however java.text.MessageFormat
| would require parameter numbers within the braces (for example, "The {0}
| jumped over the moon {1} times").
..
The whole point of using explicit positions, is that in some languages,
the order of the two elements would be changed. So if you want localized
formatting/logging, the {} won't cut it.
I just made a set of methods for text-formatting in our portal-framework.
I think that MessageFormat is too complex, so I turned to log4j to look at
the approach there, and possibly loan some code. However, that won't cut
it either, since this approach doesn't take into account the fact about
languages' differences in word ordering. I ended up making a system that
combines the two: A "{}" chooses the /first available/ variable, while {x}
explicitly specifies one.
Thus, "{} {} {}" == "{0} {1} {2}" (most normal case), "{1} {} {}" == "{1}
{0} {2}" (some language that requires that subject and object is changed?)
and "{} {} {0}" == "ERROR".
Regards,
Endre.
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-dev-help@logging.apache.org
Re: Experimental log4j formatter in sandbox
Posted by Curt Arnold <ca...@apache.org>.
On Jan 12, 2006, at 9:47 AM, Ceki Gülcü wrote:
>
> What is wrong with enshrining a solution that is optimized for the
> task at hand by being simple, easy to use and CPU effective?
If you are providing "one true way" of doing something (in this case,
formatting messages for logging), then your definition of "task at
hand" should be comprehensive. The formatter in log4j 1.3 and SLF4J
is okay as long as you are substituting strings at specific places,
but is inadequate when, for example, you are using logging to track
the convergence of iterative calculations. In those cases, users
would have a very legitimate desire to avoid unnecessary boxing of
double and float values and would like to have the ability to control
width and precision of the numeric fields.
> What
> advantages are there for your approach which offers 3 distinct
> solutions without committing to any of them?
The LogF class is preferable in general as long as the developer is
willing limit their application to JDK 1.5 and higher. With it they
can use a pattern syntax that is similar to the C RTL printf and is
the same as the String.format() and Writer.format() methods added in
JDK 1.5.
The LogMF class would be useful for anyone who wants to maintain
compatibility with earlier JDK's, but still wants the features of
MessageFormat such as the ability to control width and precision of
numeric fields.
LogSF was placed there as a potential migration path for code that
used the log4j 1.3 pattern methods. It should be fairly easy to
write a regular expression that would change code that uses the log4j
pattern methods to use LogSF. I'm not suggesting at this point to
vote on removing those methods, but this implementation was designed
so that we could evaluate that option.
> By the way, an end-user
> can always implement the LogF or LogMF approach independently of what
> log4j 1.3 offers in agreement with the SLF4J API. Put differently, the
> LogF and LogMF code can always be implemented as a static utility
> class on top of log4j if need be.
That is exactly what the sandbox project is at the moment: a set of
utility classes that can be used with log4j. There should be an
advantage for the community to review and refine these
implementations and use them if they meet their needs. If someone
wants to create their own alternative formatters, there is nothing
magic about them.
> However, by being aligned with
> SLF4J, log4j offers a stable solution and freedom of choice to its
> end-users.
>
> Curt has expressed reservations regarding the SLF4J approach. Are
> there any other developers who are uneasy with the SLF4J approach? If
> my memory serves me correctly, Paul Smith expressed interest in the
> SLF4J approach. Are there any others?
On the slf4j-dev mailing list in August (http://
marc.theaimsgroup.com/?l=slf4j-dev&m=112327572703543&w=2), Joerg
Hohwiller wrote:
>
> I actually got on the list with a very specific issue and my focus
> lies
> on commons-logging, but anyways I want to point something out.
> You might find it strage that I, who proposed the need for additional
> methods in the Logger API, am saying this but:
> In my opinion a logger API should be very simple and focus on logging.
> What you are doing here is mixing logging with native language support
> (NLS, i18n or however you call it). In my oppinion this is
> misplaced in
> a Logging API (I mean all the formatting methods).
...
> This is all about "separation of concerns".
> But again you see I am one of these IoC container guyz and care very
> much seperation of components and decoupled logic - maybe too much :)
> Other people just want to have things done quickly and without
> using and
> understanding frameworks or stuff like this and they might like to
> type
> it like suggested above.
> Anyways if you make it this way: the syntax with "{}" for me looks
> like
> you are NOT using java.text.MessageFormat. Why is that? You do not
> need
> to reinvent the wheel here. But maybe I am missing something.
and in (http://marc.theaimsgroup.com/?l=slf4j-dev&m=112328325919649&w=2)
>
> MessageFormat is a standard so many people know its syntax. It is more
> powerfull that just doing the replacement of a variable. BTW It also
> solves some NLS issues.
End of quotes from Joerg Hohwiller
>
> The assumption is largely correct except that the formatting cost
> should not be prohibitive compared to the cost of appending. Given its
> simplicity, the SLF4J approach is designed from the ground up to
> ensure optimal performance for this particular case.
One of the things I'd like to collect in the performance analysis is
some quantification of the performance of java.text.MessageFormat.
I'd think that it would have to be pretty naively implemented to be
very much slower than org.slf4j.impl.MessageFormatter and maybe
earlier performance problems have been addressed in the current JDK's.
In http://www.bejug.org/confluenceBeJUG/download/attachments/3845/
Log4JSLF4J-CekiGulcu.pdf?version=1, one of your slides said:
Why not use java.text.MessageFormat?
• Performance, performance, performance.
• Parameterized messages exist solely to
augment performance, the extra
functionality offered by
j.t.MessageFis deemed
superfluous.
I have never seen numbers that quantify the relative cost of calling
java.text.MessageFormat.format() vs
org.slf4j.impl.MessageFormatter.format() relative to the cost of
something like overall append cost for NullAppender.
>
> The current log4j 1.3 implementation supports 1 or 2 parameters.
> The SLF4J implementation supports any number of parameters.
Sorry, must have looked at an old version of
org.slf4j.impl.MessageFormatter.
>
>> The implementation is
>> different and may need to be tweaked to exactly reproduce the log4j
>> 1.3 and SLF4J behavior. For example, LogSF doesn't attempt to
>> provide any escaping mechanism to allow "{}" to appear in a
>> message. I'm not sure if log4j 1.3 or SLF4J do.
>
> Both log4j 1.3 and SLF4J allow "{}" to be escaped. This is easy to
> verify by looking at org.slf4j.impl.MessageFormatter present in both
> SLF4J and log4j 1.3.
>
I was admitting that I did a quick and dirty on LogSF and would need
to go back and check the edge cases.
> See
> http://svn.slf4j.org/viewcvs/slf4j/trunk/src/java/org/slf4j/impl/
> MessageFormatter.java
> for more details.
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-dev-help@logging.apache.org
Re: Experimental log4j formatter in sandbox
Posted by Ceki Gülcü <ce...@qos.ch>.
At 07:51 AM 1/12/2006, Curt Arnold wrote:
>I committed a pass at external message formatting classes in the
>sandbox. The code can be checked out using:
>
>svn co http://svn.apache.org/repos/asf/logging/sandbox/log4j/ formatter
>formatter
>
>and can be built using either Maven (JDK 1.5 only) or Ant. The Ant
>build can produce both a JDK 1.5 compatible jar (build with ant jar
>or mvn jar:jar) which supports varargs and the java.util.Formatter
>formatter and a (hopefully) JDK 1.3 compatible jar (build with ant
>jar-java2) which substitutes Object[] for Object... and omits support
>for the java.util.Formatter. The formatter was built and tested with
>log4j 1.2.13 (latest in Maven) and should work with earlier log4j
>1.2.x (may throw MethodNotFound exceptions if trace() methods are
>used with pre-TRACE enabled log4j's) and log4j 1.3.
>
>log4j 1.3, SLF4J and NLOG4J provide methods that take a substitution
>pattern and one or more parameters used to form the message where the
>substitution is only performed if the logger threshold is satisfied.
>For example:
>
>logger.debug("The {} jumped over the moon {} times", "cow", new
>Integer(5));
>
>One of my concerns was introducing yet another pattern syntax when
>Java class libraries already have two (java.text.MessageFormat and
>java.util.Formatter). I had originally thought that the substitution
>pattern format was a subset of that supported by
>java.text.MessageFormat, however java.text.MessageFormat would
>require parameter numbers within the braces (for example, "The {0}
>jumped over the moon {1} times").
>
>Instead of enshrining one pattern syntax, the external formatters
>provide three different utility classes with identical methods that
>each use a different formatter. The class names are short since a
>previous concern with this approach was code bloat. All the classes
>are (currently) in org.apache.log4j.formatter. They are:
What is wrong with enshrining a solution that is optimized for the
task at hand by being simple, easy to use and CPU effective? What
advantages are there for your approach which offers 3 distinct
solutions without committing to any of them? By the way, an end-user
can always implement the LogF or LogMF approach independently of what
log4j 1.3 offers in agreement with the SLF4J API. Put differently, the
LogF and LogMF code can always be implemented as a static utility
class on top of log4j if need be. However, by being aligned with
SLF4J, log4j offers a stable solution and freedom of choice to its
end-users.
Curt has expressed reservations regarding the SLF4J approach. Are
there any other developers who are uneasy with the SLF4J approach? If
my memory serves me correctly, Paul Smith expressed interest in the
SLF4J approach. Are there any others?
>LogF - uses java.util.Formatter (name should invoke fond memories of
>printf)
>LogMF - uses java.text.MessageFormat (adds M for Message)
>LogSF - uses existing log4j 1.3 syntax (adds S for Simple)
>
>The previous line could be written as:
>
>LogF.debug(logger, "The %s jumped over the moon %d times", "cow", 5);
>
>LogMF.debug(logger, "The {0} jumped over the moon {1} times", "cow", 5);
>
>LogSF.debug(logger, "The {} jumped over the moon {} times", "cow", 5);
>
>The primary design goal was to minimize the cost of ineffective log
>requests by deferring the cost of formatting, parameter boxing and
>array creation until after the isEnabledFor() is checked. The cost
>of formatting, boxing and array creation is assumed to be small
>compared to the append cost and there has been no attempt to optimize
>those costs when the threshold is satisfied. No performance
>evaluation has been attempted yet and the assumptions need to be
>confirmed.
The assumption is largely correct except that the formatting cost
should not be prohibitive compared to the cost of appending. Given its
simplicity, the SLF4J approach is designed from the ground up to
ensure optimal performance for this particular case.
>I assume the JVM will be able to effectively "in-line" the body of
>the method calls so that:
>
>LogMF.debug(logger, "The {0} jumped over the moon {1} times", "cow", 5);
>
>results in code that is nearly indistinguishable in performance from:
>
>if (logger.isDebugEnabled()) {
> logger.debug(MessageFormat.format("The {0} jumped over the moon
>{1} times", new Object[] { "cow", new Integer(5) }));
>}
>
>It is possible that the Java 5 JVM is smart enough to defer boxing
>and array creation in the vararg logging methods resulting in no
>performance benefit from the non-vararg methods. In that case, then
>all of the non-vararg methods could be eliminated from LogF since it
>only exists in the JDK 1.5. Probably would want to still support
>them for the other formatters to preserve calling compatibility
>between the JDK 1.3+ and the JDK 1.5 implementations of the formatters.
>
>Unlike the log4j 1.3 and SLF4J implementations, the LogSF will
>support more than 2 substitution parameters.
The current log4j 1.3 implementation supports 1 or 2 parameters.
The SLF4J implementation supports any number of parameters.
>The implementation is
>different and may need to be tweaked to exactly reproduce the log4j
>1.3 and SLF4J behavior. For example, LogSF doesn't attempt to
>provide any escaping mechanism to allow "{}" to appear in a
>message. I'm not sure if log4j 1.3 or SLF4J do.
Both log4j 1.3 and SLF4J allow "{}" to be escaped. This is easy to
verify by looking at org.slf4j.impl.MessageFormatter present in both
SLF4J and log4j 1.3.
See
http://svn.slf4j.org/viewcvs/slf4j/trunk/src/java/org/slf4j/impl/MessageFormatter.java
for more details. Unfortunately, SLF4J support was recently removed. I
would like to see it restored. I can personally vouch that keeping
NLOG4J in sync with SLF4J is almost effortless, especially since the
SLF4J is now quite stable. Unless there is opposition, I'd like to
take the initiative to restore native SLF4J support in 1.3.
--
Ceki Gülcü
The complete log4j manual: http://www.qos.ch/log4j/
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-dev-help@logging.apache.org