Google Summer of Code Applications Submitted for apr-build-system and apr-logging

[Walter Mundt <em...@spamcop.net>]

Thanks for the feedback on these applications.  I've refined them and 
submitted both via Google now.  Feel free to check the wiki page at 
http://wiki.apache.org/general/WalterMundt/SummerOfCodeProposals again 
for the update apps and please continue to provide any 
feedback/questions/ideas for improvement.

If you're signed up as an Apache mentor with Google, please also comment 
through their web application, as this allows me to submit updated 
versions of my applications for re-scoring, apparently even after the 
deadline for new applications.

Regards,
--
Walter Mundt
emage@spamcop.net

Re: Google Summer of Code Applications Submitted for apr-build-system and apr-logging

["William A. Rowe, Jr." <wr...@rowe-clan.net>]

Walter Mundt wrote:
> Henry Jen wrote:
> 
>> AFAIK, Windows event logging API supports Event Source and Unix syslog 
>> supports facility and a ident string. So I believe this should be 
>> implemented in a Platform-dependent way instead of being part of the 
>> message for a platform independent approach.
> 
> 
> True.  Some of this is added in my proposal on the wiki.  After some 
> further looking about on MSDN, however, I've noticed that the Windows 
> event sources must be individually predefined in the registry.  As such, 
> I've stuck with a set of them that correspond to the Unix syslog 
> facility constants for the API.  I may add a "prefix" option to the 
> constructors to correspond to the Unix "ident" string.

I strongly believe we should not go 'overboard' exploiting the full power of
the windows events.  By creating a 'matched set' to correspond to unix syslog
facilities, we have something reasonably generic, which can be filtered by
importance, and a common set that can be targetted by all.

If a windows build of an apr app has to do a little extra to take advantage of
these, then shucks, guess they have to do so.

But if they want the full power of windows event logging, by all means; let them
write a windows handler.  That's not part of our role in creating cross-platform
applications.

Bill

Re: Google Summer of Code Applications Submitted for apr-build-system and apr-logging

[Henry Jen <sl...@gmail.com>]

Curt Arnold wrote:

> On the logging proposal, I see a decent amount of discussion regarding 
> the Windows Event Log methods, but little regarding OutputDebugString or 
> TraceEvent which a native Windows app would use to output debug level 
> message or lower.  The Event Log methods are way too expensive for 
> diagnostic logging.  I'd assume a typical Windows configuration would do 
> something send info and higher messages to the Event Log and debug and 
> lower severity messages to OutputDebugString or TraceEvent.
> 

Good idea.

Cheers,
Henry

Re: Google Summer of Code Applications Submitted for apr-build-system and apr-logging

[Curt Arnold <ca...@apache.org>]

Couple of comments on both proposals.

First, I want you to be aware that log4cxx (http:// 
logging.apache.org) as a by-product can produce Visual Studio 6  
project files for apr and apr-util sufficient for its own use through  
its use of cpptasks (http://ant-contrib.sourceforge.net) and Ant  
(http://ant.apache.org).  If you get into that might save you some  
time trying to figure out the project file syntax.

On the logging proposal, I see a decent amount of discussion  
regarding the Windows Event Log methods, but little regarding  
OutputDebugString or TraceEvent which a native Windows app would use  
to output debug level message or lower.  The Event Log methods are  
way too expensive for diagnostic logging.  I'd assume a typical  
Windows configuration would do something send info and higher  
messages to the Event Log and debug and lower severity messages to  
OutputDebugString or TraceEvent.

I think the most desirable API for APR would mimic syslog but  
possibly add additional bitmasks for the logopt parameter to the  
apr_open_log method to correspond to different logging API's (like  
APR_LOG_DEBUGGER which would map to OutputDebugString on Win32 and  
NSLog on Mac OS/X).  A platform specific logging API (like  
APR_LOG_EVENT_LOG) could be ignored on platforms where it wasn't  
meaningful.

Re: Google Summer of Code Applications Submitted for apr-build-system and apr-logging

[Henry Jen <he...@ztune.net>]

Walter Mundt wrote:
> Henry Jen wrote:
>> As long as it allows multiple log to be opened and each can have it's 
>> own ident string, I am happy with that. :-)
> 
> Okay, I'll add something like this sometime soon unless someone objects.
> 
>>
>> I guess this depends on which level the apr_log is at.
>>
>> An application may have several components, and may need to use 
>> several facilities(the concept of facility here is more like the 
>> "ident" part, that's how I define the jxta API) to distinguish those 
>> components. Should they have different logs or a log support multiple 
>> facilities?
>>
>> If I add log in the apr/apr-util to aid debugging, I may have facility 
>> like thread/queue/ldap etc. But you can argue that this is not what 
>> log is for. :-P
>>
>> One facility per log works, just a little bit more to be done in the 
>> application. For example, when you want to filter the log, you got 
>> multiple logs to do.
>>
>> Anyway, that maybe beyond apr's scope for logging. But with those two 
>> features serve really well in the two projects I have been working on. 
>> Would be shame if I need to add another portable logging layer for 
>> this minimum feature set. :-)
> 
> I'm not sure really what you mean like this.  Please look at the changed 
> app on the wiki -- when *I* say "facility" what I mean is strictly "Unix 
> syslog facility"; you seem to mean something different.  So there is a 
> facility "mail", "user", "daemon", etc...and _that is all_.  See your 
> favorite 'man 3 syslog'.
> 

Understand, and you are right, I mean different by facility. So maybe I 
should not use the term "facility" to avoid confusion.

The sole purpose is to identify the source, so the "ident" string is 
what really matters.

> Beyond that, it seems to me that we end up right back in "optimal 
> cross-platform impl." territory.  The exception is that Windows event 
> sources can be almost-arbitrary strings (they must be regex keys, so no 
> \ or special chars.)...but see my previous comments on that.
> 
> Once again, I am absolutely fine with expanding the scope of the project 
> to make apr-logging a generally useful logging API.  Right now, I see it 
> more as a portability shim on top of which a full-fledged logging system 
> might be built outside of APR.
>

The way I see it is not much different from yours. I see it as a 
portable layer for full-fledge logging systems, sort of like apr_dbd for 
different databases.

apr-log define a minimum set of logging functionality, the underlying 
log system can do things like rotate log files, maintaining extra 
logging contexts, etc.

> One more consideration: I may add a single win32 specific 
> 'apr_log_win32_set_source' call; applications that want to use 
> apr-logging AND have their own "source" field in the Windows event 
> logger could add some conditionally-compiled calls to that function. 
> That would then disable the use of the syslog-emulation sources for that 
> log instance.

I wish we can somehow use ident string to register an event source on 
the fly in case that is not too much effort. Otherwise, a fixed set of 
source and embedded the ident string into the log message is good 
enough. Anyway, this is implementation detail to Windows. :-)

The progress so far looks really nice, good work.

Cheers,
Henry

Re: Google Summer of Code Applications Submitted for apr-build-system and apr-logging

[Walter Mundt <em...@spamcop.net>]

Henry Jen wrote:
> As long as it allows multiple log to be opened and each can have it's 
> own ident string, I am happy with that. :-)

Okay, I'll add something like this sometime soon unless someone objects.

> 
> I guess this depends on which level the apr_log is at.
> 
> An application may have several components, and may need to use several 
> facilities(the concept of facility here is more like the "ident" part, 
> that's how I define the jxta API) to distinguish those components. 
> Should they have different logs or a log support multiple facilities?
> 
> If I add log in the apr/apr-util to aid debugging, I may have facility 
> like thread/queue/ldap etc. But you can argue that this is not what log 
> is for. :-P
> 
> One facility per log works, just a little bit more to be done in the 
> application. For example, when you want to filter the log, you got 
> multiple logs to do.
> 
> Anyway, that maybe beyond apr's scope for logging. But with those two 
> features serve really well in the two projects I have been working on. 
> Would be shame if I need to add another portable logging layer for this 
> minimum feature set. :-)

I'm not sure really what you mean like this.  Please look at the changed 
app on the wiki -- when *I* say "facility" what I mean is strictly "Unix 
syslog facility"; you seem to mean something different.  So there is a 
facility "mail", "user", "daemon", etc...and _that is all_.  See your 
favorite 'man 3 syslog'.

Beyond that, it seems to me that we end up right back in "optimal 
cross-platform impl." territory.  The exception is that Windows event 
sources can be almost-arbitrary strings (they must be regex keys, so no 
\ or special chars.)...but see my previous comments on that.

Once again, I am absolutely fine with expanding the scope of the project 
to make apr-logging a generally useful logging API.  Right now, I see it 
more as a portability shim on top of which a full-fledged logging system 
might be built outside of APR.

One more consideration: I may add a single win32 specific 
'apr_log_win32_set_source' call; applications that want to use 
apr-logging AND have their own "source" field in the Windows event 
logger could add some conditionally-compiled calls to that function. 
That would then disable the use of the syslog-emulation sources for that 
log instance.

-Walter

Re: Google Summer of Code Applications Submitted for apr-build-system and apr-logging

[Henry Jen <he...@ztune.net>]

Walter Mundt wrote:
> Henry Jen wrote:
>> AFAIK, Windows event logging API supports Event Source and Unix syslog 
>> supports facility and a ident string. So I believe this should be 
>> implemented in a Platform-dependent way instead of being part of the 
>> message for a platform independent approach.
> 
> True.  Some of this is added in my proposal on the wiki.  After some 
> further looking about on MSDN, however, I've noticed that the Windows 
> event sources must be individually predefined in the registry.  As such, 
> I've stuck with a set of them that correspond to the Unix syslog 
> facility constants for the API.  I may add a "prefix" option to the 
> constructors to correspond to the Unix "ident" string.
> 

As long as it allows multiple log to be opened and each can have it's 
own ident string, I am happy with that. :-)

>> Sorry for pitching the jxta logging API again. The selector API is for 
>> this purpose so that you can choose what to be logged based on both 
>> facility and level. Which is basically same thing to the selector in 
>> syslog.conf man page.
> 
> Given the extremely limited number of facilities, and the fact that I 
> can't really see more than maybe two being used in a single application, 
> the value of being able to filter log messages by facility within an 
> application seems minimal to me.  I will happily listen to any arguments 
> to the contrary, however.  For the moment, I've left it out of the 
> design for simplicity's sake.
> 

I guess this depends on which level the apr_log is at.

An application may have several components, and may need to use several 
facilities(the concept of facility here is more like the "ident" part, 
that's how I define the jxta API) to distinguish those components. 
Should they have different logs or a log support multiple facilities?

If I add log in the apr/apr-util to aid debugging, I may have facility 
like thread/queue/ldap etc. But you can argue that this is not what log 
is for. :-P

One facility per log works, just a little bit more to be done in the 
application. For example, when you want to filter the log, you got 
multiple logs to do.

Anyway, that maybe beyond apr's scope for logging. But with those two 
features serve really well in the two projects I have been working on. 
Would be shame if I need to add another portable logging layer for this 
minimum feature set. :-)


>> Another thing is that, it would be much convenient to allow 
>> apr_snprintf style logging. What I am saying is
>>
>> apr_status_t apr_log_append(const char *cat, int level, const char 
>> *fmt, ...);
> 
> Done!  I can't believe I overlooked this.  In fact, such a function is 
> so useful that I've decided to see if I can get away with just calling 
> it apr_log(). ;)
> 
>> Another API might be useful is to close an log file early. That is,
>>
>> apr_status_t apr_log_close(apr_log_t *log);
> 
> Oops, definitely.  It's not on there, but I'll put it in with the next 
> round of changes; I'm assuming that you'll have a few more points to 
> bring up shortly.
> 

Not really. :-)

Cheers,
Henry

Re: Google Summer of Code Applications Submitted for apr-build-system and apr-logging

[Walter Mundt <em...@spamcop.net>]

Henry Jen wrote:
> AFAIK, Windows event logging API supports Event Source and Unix syslog 
> supports facility and a ident string. So I believe this should be 
> implemented in a Platform-dependent way instead of being part of the 
> message for a platform independent approach.

True.  Some of this is added in my proposal on the wiki.  After some 
further looking about on MSDN, however, I've noticed that the Windows 
event sources must be individually predefined in the registry.  As such, 
I've stuck with a set of them that correspond to the Unix syslog 
facility constants for the API.  I may add a "prefix" option to the 
constructors to correspond to the Unix "ident" string.

> Sorry for pitching the jxta logging API again. The selector API is for 
> this purpose so that you can choose what to be logged based on both 
> facility and level. Which is basically same thing to the selector in 
> syslog.conf man page.

Given the extremely limited number of facilities, and the fact that I 
can't really see more than maybe two being used in a single application, 
the value of being able to filter log messages by facility within an 
application seems minimal to me.  I will happily listen to any arguments 
to the contrary, however.  For the moment, I've left it out of the 
design for simplicity's sake.

> Another thing is that, it would be much convenient to allow apr_snprintf 
> style logging. What I am saying is
> 
> apr_status_t apr_log_append(const char *cat, int level, const char *fmt, 
> ...);

Done!  I can't believe I overlooked this.  In fact, such a function is 
so useful that I've decided to see if I can get away with just calling 
it apr_log(). ;)

> Another API might be useful is to close an log file early. That is,
> 
> apr_status_t apr_log_close(apr_log_t *log);

Oops, definitely.  It's not on there, but I'll put it in with the next 
round of changes; I'm assuming that you'll have a few more points to 
bring up shortly.

-Walter

Re: Google Summer of Code Applications Submitted for apr-build-system and apr-logging

[Henry Jen <he...@ztune.net>]

Walter Mundt wrote:
> Henry Jen wrote:
>> A couple comments:
>>
>> 1. How do you distinguish the event source? Even in a single 
>> application, multiple categories may be desired to classify the 
>> events. Not to mention different apps.
> 
> There is no facility to do this currently.  I deliberately decided to 
> leave that out for several reasons.  The most relevant for discussion on 
> this list is that I tried to make the API so that it provides the 
> minimal subset of logging functionality that must be implemented in a 
> platform-dependent way.  Since context-aware logging functions could 
> easily be built on top of this API without sacrificing any 
> functionality, I left them out.
> 

AFAIK, Windows event logging API supports Event Source and Unix syslog 
supports facility and a ident string. So I believe this should be 
implemented in a Platform-dependent way instead of being part of the 
message for a platform independent approach.

> If there is a consensus among APR developers that more functionality 
> (like this, but really anything useful that fits in the "has an optimal 
> platform-independent implementation" mold) should be included, I have no 
> problem adding it.
> 

These 2 things are really all I needed for a basic logging API.

>> 2. This may only be me, but I would like to have capability to turn 
>> off some logging capability at runtime. It's kind of different verbose 
>> levels. With that, you can have more verbose logs to help you 
>> debugging an application when needed while a minimum when things are 
>> running fine.
> 
> That's a very common usage scenario, and one that I allow via the level 
> parameters on the log constructors.  However, I neglected to add a 
> function for changing the logging level post-construction-time.  I'll 
> add a spec for one on the wiki and in any revised app if someone 
> comments through Google.
> 

Sorry for pitching the jxta logging API again. The selector API is for 
this purpose so that you can choose what to be logged based on both 
facility and level. Which is basically same thing to the selector in 
syslog.conf man page.

So add an apr_log_set_selector(apr_log_t *log, const char* str) to your 
proposal would do it. :-)

Another thing is that, it would be much convenient to allow apr_snprintf 
style logging. What I am saying is

apr_status_t apr_log_append(const char *cat, int level, const char *fmt, 
...);

Another API might be useful is to close an log file early. That is,

apr_status_t apr_log_close(apr_log_t *log);

Cheers,
Henry

Re: Google Summer of Code Applications Submitted for apr-build-system and apr-logging

[Walter Mundt <em...@spamcop.net>]

Henry Jen wrote:
> A couple comments:
> 
> 1. How do you distinguish the event source? Even in a single 
> application, multiple categories may be desired to classify the events. 
> Not to mention different apps.

There is no facility to do this currently.  I deliberately decided to 
leave that out for several reasons.  The most relevant for discussion on 
this list is that I tried to make the API so that it provides the 
minimal subset of logging functionality that must be implemented in a 
platform-dependent way.  Since context-aware logging functions could 
easily be built on top of this API without sacrificing any 
functionality, I left them out.

If there is a consensus among APR developers that more functionality 
(like this, but really anything useful that fits in the "has an optimal 
platform-independent implementation" mold) should be included, I have no 
problem adding it.

> 2. This may only be me, but I would like to have capability to turn off 
> some logging capability at runtime. It's kind of different verbose 
> levels. With that, you can have more verbose logs to help you debugging 
> an application when needed while a minimum when things are running fine.

That's a very common usage scenario, and one that I allow via the level 
parameters on the log constructors.  However, I neglected to add a 
function for changing the logging level post-construction-time.  I'll 
add a spec for one on the wiki and in any revised app if someone 
comments through Google.

-Walter

Re: Google Summer of Code Applications Submitted for apr-build-system and apr-logging

[Henry Jen <he...@ztune.net>]

Walter Mundt wrote:
> Thanks for the feedback on these applications.  I've refined them and 
> submitted both via Google now.  Feel free to check the wiki page at 
> http://wiki.apache.org/general/WalterMundt/SummerOfCodeProposals again 
> for the update apps and please continue to provide any 
> feedback/questions/ideas for improvement.
> 
> If you're signed up as an Apache mentor with Google, please also comment 
> through their web application, as this allows me to submit updated 
> versions of my applications for re-scoring, apparently even after the 
> deadline for new applications.
> 

I like the set_format part. :-)

A couple comments:

1. How do you distinguish the event source? Even in a single 
application, multiple categories may be desired to classify the events. 
Not to mention different apps.

2. This may only be me, but I would like to have capability to turn off 
some logging capability at runtime. It's kind of different verbose 
levels. With that, you can have more verbose logs to help you debugging 
an application when needed while a minimum when things are running fine.

Cheers,
Henry