Configurable error log format

[Stefan Fritsch <sf...@sfritsch.de>]

Hi,

I finally got around to finish the remaining bits of the configurable
error log format thing. The current version is at

http://people.apache.org/~sf/errorlog_format_v2.diff

I intend to commit it to trunk after I have written some docs and 2.3.7
is tagged. It duplicates some code from mod_log_config. If there are
enough people interested in unifying the log formaters for access and
error logs, we can do that work in trunk. Personally, I don't think
it's necessary for 2.4.

I have changed the configuration syntax with some of the suggestions
form the last thread. The log format is implicitly split into fields
at non-whitespace to whitespace transitions. A backslash before a space
will prevent this behaviour. Percent-space '% ' can be used as
zero-width field separator.

If a format item does not produce output, the whole field is omitted.
This behaviour can be changed with modifiers: "-" to log a "-" instead.
"+" to omit the whole line instead (useful for once per request
logging).

For normal log lines, there is now only one format that contains %M
for the actual log message. The separate prefix and suffix configuration
is gone.

For example, the configuration

ErrorLogFormat connection "[%{uc}t] [C:%{c}L] local\ %a remote\ %A"
ErrorLogFormat request "[%{uc}t] [R:%L] Request %k on C:%{c}L pid:%P tid:%T"
ErrorLogFormat request "[%{uc}t] [R:%L] UA:'%+{User-Agent}i'"
ErrorLogFormat request "[%{uc}t] [R:%L] Referer:'%+{Referer}i'"
ErrorLogFormat "[%{uc}t] [%-m:%-l] [R:%L] [C:%{C}L] %M"

gives 

[2010-08-18 20:10:11.135760] [C:CFn8+QwXPQo] local 127.0.0.1:43845 remote 127.0.0.1:8081
[2010-08-18 20:10:11.135783] [R:21j8+QwXPQo] Request 0 on C:CFn8+QwXPQo pid:11653
[2010-08-18 20:10:11.135788] [R:21j8+QwXPQo] UA:'curl/7.21.1 (i486-pc-linux-gnu) libcurl/7.21.1 OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.18 libssh2/1.2.6'
[2010-08-18 20:10:11.153439] [core:error] [R:21j8+QwXPQo] File does not exist: /usr/local/apache2/htdocs/xxx

Cheers,
Stefan

Re: Configurable error log format

[Stefan Fritsch <sf...@sfritsch.de>]

On Tuesday 07 September 2010, William A. Rowe Jr. wrote:
> > - there is no way yet to match the default format's behaviour of 
> > logging the source file info only for loglevel debug and higher
> 
> Just a thought; we have %nnn{}x syntax today, what about
> overloading the concept of %test codes with numeric 0-7 error
> levels?  I was thinking about allowing verbose levels, but
> obviously '%debug' would immediately trip up that theory :)

I don't have a better idea if we want to allow an arbitrary threshold. 
Or would it be enough to always fix the threshold at level debug? Then 
we could use a single special character, like %^{}x.

In any case I would like to keep the < > characters free in case we 
need some special handling for subrequests (that's maybe another todo 
item, but one that I think could wait until 2.4.x with x >= 1).

Re: Configurable error log format

["William A. Rowe Jr." <wr...@rowe-clan.net>]

On 9/5/2010 10:49 AM, Stefan Fritsch wrote:
> On Wednesday 18 August 2010, Stefan Fritsch wrote:
>> I intend to commit it to trunk after I have written some docs and
>> 2.3.7 is tagged. It duplicates some code from mod_log_config. If
>> there are enough people interested in unifying the log formaters
>> for access and error logs, we can do that work in trunk.
>> Personally, I don't think it's necessary for 2.4.
> 
> This took a bit longer than anticipated but it is now commited to 
> trunk. There are a still a few open issues but those can be resolved 
> while the code receives more testing in trunk:
> 
> - there is no way yet to match the default format's behaviour of 
> logging the source file info only for loglevel debug and higher

Just a thought; we have %nnn{}x syntax today, what about overloading the
concept of %test codes with numeric 0-7 error levels?  I was thinking
about allowing verbose levels, but obviously '%debug' would immediately
trip up that theory :)

Re: Configurable error log format

[Stefan Fritsch <sf...@sfritsch.de>]

On Wednesday 18 August 2010, Stefan Fritsch wrote:
> I intend to commit it to trunk after I have written some docs and
> 2.3.7 is tagged. It duplicates some code from mod_log_config. If
> there are enough people interested in unifying the log formaters
> for access and error logs, we can do that work in trunk.
> Personally, I don't think it's necessary for 2.4.

This took a bit longer than anticipated but it is now commited to 
trunk. There are a still a few open issues but those can be resolved 
while the code receives more testing in trunk:

- there is no way yet to match the default format's behaviour of 
logging the source file info only for loglevel debug and higher
- the syntax of the arguments %t (current time) should be more in line 
with mod_log_config's %t syntax
- there is some documentation in mod/core.xml but logs.xml still needs 
updating
- I am not sure that putting the module name before the loglevel in 
the default format is the best solution: [mpm_prefork:notice]
- it should be possible to use the unique ID as error log ID if 
mod_unique_id is loaded

Cheers,
Stefan