You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Mladen Turk <mt...@apache.org> on 2009/02/04 17:09:32 UTC

new watchdog module

Hi,

I would like to commit this module to trunk.
It's a watchdog module that creates a worker threads
either in parent, child via an API, and it's not for
standalone use, but for other module usage like
mod_heartbeat, probably mod_jk and others that need
maintenance threads.

The reason is because couple of modules would benefit
from this functionality, and this adds a common API.

Usage is very simple
1. ap_watchdog_get_instance
2. set singleton mode (only one child will execute)
3. register a callback(s) with desired interval

Module can either use pre-created watchdog instance
or create it's own in special cases thus creating
additional thread(s)

For example heartbeat would be by an order of magnitude
simpler (I took some concepts from it)
Also I'd love to register the callbacks for balancer
so transferred method can normalize the load.

Comments?

Regards
-- 
^(TM)

Re: new watchdog module

Posted by Graham Leggett <mi...@sharp.fm>.
Ruediger Pluem wrote:

> This seems to be a very valid concern to me. Plus in the parent it runs with root
> privileges and we should minimize the code that runs with these privileges, even
> more so as an author of code that uses the watchdog may not really be aware that
> its code is running under root (in contrast to people who develop modules like
> mod_unixd and mod_privileges).
> 
> So we shouldn't run this in the parent process but only in the childs or fork a
> separate child (like mod_cgid does) that only runs the watchdog if a
> single-instance-non-locking watchdog is needed.

I don't think there is a one size fits all solution to this, I think we 
should offer modules an option to spawn a thread/process/whatever both 
before or after the drop privileges step, and let the module author 
decide which is most relevant to them.

Because it would be an explicit choice, and not an implicit one, there 
would be no confusion as to what user was running this code.

Regards,
Graham
--

Re: new watchdog module

Posted by Jim Jagielski <ji...@jaguNET.com>.
On Feb 5, 2009, at 2:36 AM, Plüm, Rüdiger, VF-Group wrote:
>>
>> The parent process watchdog is supposed to be used only for core
>> features that affect all children's and can be done without
>> requiring graceful restart.
>
> Why can these tasks not be performed by a dedicated child?
>
>> Making the module running as root is very easy and
>> average 'Joe the Developer' can already use such a feature
>> with or without this option.
>
> That was not my point. The current ways for a developer to do this
> IMHO point him more obviously to the fact that his code is running
> under root than the watchdog API would do.
>

We need a real watchdog/monitor hook that allows for something
useful to be done.


Re: new watchdog module

Posted by "Plüm, Rüdiger, VF-Group" <ru...@vodafone.com>.
 

> -----Ursprüngliche Nachricht-----
> Von: Mladen Turk
> Gesendet: Donnerstag, 5. Februar 2009 12:04
> An: dev@httpd.apache.org
> Betreff: Re: new watchdog module
> 
> Plüm, Rüdiger, VF-Group wrote:
> >  
> > 
> >> -----Ursprüngliche Nachricht-----
> >> Von: Mladen Turk  
> >> Gesendet: Donnerstag, 5. Februar 2009 06:36
> >> An: dev@httpd.apache.org
> >> Betreff: Re: new watchdog module
> >>
> >> Ruediger Pluem wrote:
> >>>>> I would like to commit this module to trunk.
> >>>>> It's a watchdog module that creates a worker threads
> >>>>> either in parent, child via an API, and it's not for
> >>>>> standalone use, but for other module usage like
> >>>>> mod_heartbeat, probably mod_jk and others that need
> >>>>> maintenance threads.
> >>>> It seems scary to me to have the MPM parent on Unix 
> >> forking children while a
> >>>> separate thread in that same process is running arbitrary 
> >> code.  Surprising
> >>>> problems can crop up when this arbitrary code is in 
> >> certain states (maybe a
> >>>> mutex gets inherited in held state by the new MPM child?)
> >>> This seems to be a very valid concern to me. Plus in the 
> >> parent it runs with root
> >>> privileges and we should minimize the code that runs with 
> >> these privileges, even
> >>> more so as an author of code that uses the watchdog may not 
> >> really be aware that
> >>> its code is running under root (in contrast to people who 
> >> develop modules like
> >>> mod_unixd and mod_privileges).
> >>>
> >> The parent process watchdog is supposed to be used only for core
> >> features that affect all children's and can be done without
> >> requiring graceful restart.
> > 
> > Why can these tasks not be performed by a dedicated child?
> >
> 
> The simplest one would be graceful restart.
> Child processing the requests would set some shared
> memory flag and parent watchdog thread could be
> used to monitor the shared memory field status,
> and signal the mpm to do a graceful restart.

Ok, I get your point, but I still don't like to extend the possibilies of
running in the parent process (especially during server runtime in contrast
to during server startup) for privileges reasons, for stability reasons
(think of the risk of such a provider crashing the parent) and for the
reasons Jeff mentioned.
If you create a chíld process and keep running as root you can avoid at
least Jeffs points and the stability point and you still can do graceful
restarts by sending the needed signal to the parent. So we would only
leave the privileges issue open.

> >> Making the module running as root is very easy and
> >> average 'Joe the Developer' can already use such a feature
> >> with or without this option.
> > 
> > That was not my point. The current ways for a developer to do this
> > IMHO point him more obviously to the fact that his code is running
> > under root than the watchdog API would do.
> >
> 
> Well, I don't under estimate module developers.

I don't do as well, but the problem is awareness. But possibly this
could be fixed by documentation.

Regards

Rüdiger

Re: new watchdog module

Posted by Mladen Turk <mt...@apache.org>.
Plüm, Rüdiger, VF-Group wrote:
>  
> 
>> -----Ursprüngliche Nachricht-----
>> Von: Mladen Turk  
>> Gesendet: Donnerstag, 5. Februar 2009 06:36
>> An: dev@httpd.apache.org
>> Betreff: Re: new watchdog module
>>
>> Ruediger Pluem wrote:
>>>>> I would like to commit this module to trunk.
>>>>> It's a watchdog module that creates a worker threads
>>>>> either in parent, child via an API, and it's not for
>>>>> standalone use, but for other module usage like
>>>>> mod_heartbeat, probably mod_jk and others that need
>>>>> maintenance threads.
>>>> It seems scary to me to have the MPM parent on Unix 
>> forking children while a
>>>> separate thread in that same process is running arbitrary 
>> code.  Surprising
>>>> problems can crop up when this arbitrary code is in 
>> certain states (maybe a
>>>> mutex gets inherited in held state by the new MPM child?)
>>> This seems to be a very valid concern to me. Plus in the 
>> parent it runs with root
>>> privileges and we should minimize the code that runs with 
>> these privileges, even
>>> more so as an author of code that uses the watchdog may not 
>> really be aware that
>>> its code is running under root (in contrast to people who 
>> develop modules like
>>> mod_unixd and mod_privileges).
>>>
>> The parent process watchdog is supposed to be used only for core
>> features that affect all children's and can be done without
>> requiring graceful restart.
> 
> Why can these tasks not be performed by a dedicated child?
>

The simplest one would be graceful restart.
Child processing the requests would set some shared
memory flag and parent watchdog thread could be
used to monitor the shared memory field status,
and signal the mpm to do a graceful restart.


>> Making the module running as root is very easy and
>> average 'Joe the Developer' can already use such a feature
>> with or without this option.
> 
> That was not my point. The current ways for a developer to do this
> IMHO point him more obviously to the fact that his code is running
> under root than the watchdog API would do.
>

Well, I don't under estimate module developers.
This is just an helper API, and it makes things more
robust at the end.


Regards
-- 
^(TM)

Re: new watchdog module

Posted by "Plüm, Rüdiger, VF-Group" <ru...@vodafone.com>.
 

> -----Ursprüngliche Nachricht-----
> Von: Mladen Turk  
> Gesendet: Donnerstag, 5. Februar 2009 06:36
> An: dev@httpd.apache.org
> Betreff: Re: new watchdog module
> 
> Ruediger Pluem wrote:
> > 
> >>>
> >>> I would like to commit this module to trunk.
> >>> It's a watchdog module that creates a worker threads
> >>> either in parent, child via an API, and it's not for
> >>> standalone use, but for other module usage like
> >>> mod_heartbeat, probably mod_jk and others that need
> >>> maintenance threads.
> >>
> >> It seems scary to me to have the MPM parent on Unix 
> forking children while a
> >> separate thread in that same process is running arbitrary 
> code.  Surprising
> >> problems can crop up when this arbitrary code is in 
> certain states (maybe a
> >> mutex gets inherited in held state by the new MPM child?)
> > 
> > This seems to be a very valid concern to me. Plus in the 
> parent it runs with root
> > privileges and we should minimize the code that runs with 
> these privileges, even
> > more so as an author of code that uses the watchdog may not 
> really be aware that
> > its code is running under root (in contrast to people who 
> develop modules like
> > mod_unixd and mod_privileges).
> >
> 
> The parent process watchdog is supposed to be used only for core
> features that affect all children's and can be done without
> requiring graceful restart.

Why can these tasks not be performed by a dedicated child?

> Making the module running as root is very easy and
> average 'Joe the Developer' can already use such a feature
> with or without this option.

That was not my point. The current ways for a developer to do this
IMHO point him more obviously to the fact that his code is running
under root than the watchdog API would do.

Besides Jeffs points remain valid apart from running with root or not.

Regards

Rüdiger

Re: new watchdog module

Posted by Jeff Trawick <tr...@gmail.com>.
On Fri, Feb 6, 2009 at 12:09 PM, Mladen Turk <mt...@apache.org> wrote:

> Jeff Trawick wrote:
>
>>
>> A long-standing design point for Apache on Unix is that we have a
>> single-threaded parent, because of all the problems which could conceivably
>> be caused by the MPM forking a threaded process.
>>
>
> OK, that explanation is fair enough.
> I'll remove the parent code then.
>
>
>> As you say, Joe the Developer (certainly the Joes that can get it running
>> reliably without rare crashes) doesn't need the help either.
>>
>> I think this means that nobody needs it ;)
>>
>>
> Well, here is the full code, just in case someone does
> since it's been already written ;)


In case it wasn't clear, I thought the scope of the discussion was the
ability to create a watchdog that promises to run in the Apache parent
process.  I'm +1 in concept to other watchdog capabilities as well as a
watchdog that might run in the parent on some non-Unix platforms ;)

Is it of no immediate use otherwise?

Re: new watchdog module

Posted by Mladen Turk <mt...@apache.org>.
Jeff Trawick wrote:
> 
> A long-standing design point for Apache on Unix is that we have a 
> single-threaded parent, because of all the problems which could 
> conceivably be caused by the MPM forking a threaded process.

OK, that explanation is fair enough.
I'll remove the parent code then.

> 
> As you say, Joe the Developer (certainly the Joes that can get it 
> running reliably without rare crashes) doesn't need the help either.
> 
> I think this means that nobody needs it ;)
> 

Well, here is the full code, just in case someone does
since it's been already written ;)

Regards
-- 
^(TM)

Re: new watchdog module

Posted by Jeff Trawick <tr...@gmail.com>.
On Thu, Feb 5, 2009 at 12:36 AM, Mladen Turk <mt...@apache.org> wrote:

> Ruediger Pluem wrote:
>
>>
>>
>>>> I would like to commit this module to trunk.
>>>> It's a watchdog module that creates a worker threads
>>>> either in parent, child via an API, and it's not for
>>>> standalone use, but for other module usage like
>>>> mod_heartbeat, probably mod_jk and others that need
>>>> maintenance threads.
>>>>
>>>
>>> It seems scary to me to have the MPM parent on Unix forking children
>>> while a
>>> separate thread in that same process is running arbitrary code.
>>>  Surprising
>>> problems can crop up when this arbitrary code is in certain states (maybe
>>> a
>>> mutex gets inherited in held state by the new MPM child?)
>>>
>>
>> This seems to be a very valid concern to me. Plus in the parent it runs
>> with root
>> privileges and we should minimize the code that runs with these
>> privileges, even
>> more so as an author of code that uses the watchdog may not really be
>> aware that
>> its code is running under root (in contrast to people who develop modules
>> like
>> mod_unixd and mod_privileges).
>>
>>
> The parent process watchdog is supposed to be used only for core
> features that affect all children's and can be done without
> requiring graceful restart.
> Making the module running as root is very easy and
> average 'Joe the Developer' can already use such a feature
> with or without this option.
>

A long-standing design point for Apache on Unix is that we have a
single-threaded parent, because of all the problems which could conceivably
be caused by the MPM forking a threaded process.  So *we* don't need that
support for running other threads in the parent.

As you say, Joe the Developer (certainly the Joes that can get it running
reliably without rare crashes) doesn't need the help either.

I think this means that nobody needs it ;)

Re: new watchdog module

Posted by Mladen Turk <mt...@apache.org>.
Ruediger Pluem wrote:
> 
>>>
>>> I would like to commit this module to trunk.
>>> It's a watchdog module that creates a worker threads
>>> either in parent, child via an API, and it's not for
>>> standalone use, but for other module usage like
>>> mod_heartbeat, probably mod_jk and others that need
>>> maintenance threads.
>>
>> It seems scary to me to have the MPM parent on Unix forking children while a
>> separate thread in that same process is running arbitrary code.  Surprising
>> problems can crop up when this arbitrary code is in certain states (maybe a
>> mutex gets inherited in held state by the new MPM child?)
> 
> This seems to be a very valid concern to me. Plus in the parent it runs with root
> privileges and we should minimize the code that runs with these privileges, even
> more so as an author of code that uses the watchdog may not really be aware that
> its code is running under root (in contrast to people who develop modules like
> mod_unixd and mod_privileges).
>

The parent process watchdog is supposed to be used only for core
features that affect all children's and can be done without
requiring graceful restart.
Making the module running as root is very easy and
average 'Joe the Developer' can already use such a feature
with or without this option.


Regards
-- 
^(TM)

Re: new watchdog module

Posted by Jeff Trawick <tr...@gmail.com>.
On Wed, Feb 4, 2009 at 5:02 PM, Ruediger Pluem <rp...@apache.org> wrote:

>
>
> On 02/04/2009 10:44 PM, Jeff Trawick wrote:
> > On Wed, Feb 4, 2009 at 11:09 AM, Mladen Turk <mt...@apache.org> wrote:
> >
> >> Hi,
> >>
> >> I would like to commit this module to trunk.
> >> It's a watchdog module that creates a worker threads
> >> either in parent, child via an API, and it's not for
> >> standalone use, but for other module usage like
> >> mod_heartbeat, probably mod_jk and others that need
> >> maintenance threads.
> >
> >
> > It seems scary to me to have the MPM parent on Unix forking children
> while a
> > separate thread in that same process is running arbitrary code.
>  Surprising
> > problems can crop up when this arbitrary code is in certain states (maybe
> a
> > mutex gets inherited in held state by the new MPM child?)
>
> This seems to be a very valid concern to me. Plus in the parent it runs
> with root
> privileges and we should minimize the code that runs with these privileges,
> even
> more so as an author of code that uses the watchdog may not really be aware
> that
> its code is running under root (in contrast to people who develop modules
> like
> mod_unixd and mod_privileges).
>
> So we shouldn't run this in the parent process but only in the childs or
> fork a
> separate child (like mod_cgid does) that only runs the watchdog if a
> single-instance-non-locking watchdog is needed.
>

Taking this just a little further: Replace the parent-process-monitor-thread
flavor with one that only promises to run outside of child processes serving
requests.  On Unix this will run in a separate child (potentially using one
child for all such monitor threads) but on Windows PERHAPS it is reasonable
to run in the Apache parent.

Re: new watchdog module

Posted by Ruediger Pluem <rp...@apache.org>.

On 02/04/2009 10:44 PM, Jeff Trawick wrote:
> On Wed, Feb 4, 2009 at 11:09 AM, Mladen Turk <mt...@apache.org> wrote:
> 
>> Hi,
>>
>> I would like to commit this module to trunk.
>> It's a watchdog module that creates a worker threads
>> either in parent, child via an API, and it's not for
>> standalone use, but for other module usage like
>> mod_heartbeat, probably mod_jk and others that need
>> maintenance threads.
> 
> 
> It seems scary to me to have the MPM parent on Unix forking children while a
> separate thread in that same process is running arbitrary code.  Surprising
> problems can crop up when this arbitrary code is in certain states (maybe a
> mutex gets inherited in held state by the new MPM child?)

This seems to be a very valid concern to me. Plus in the parent it runs with root
privileges and we should minimize the code that runs with these privileges, even
more so as an author of code that uses the watchdog may not really be aware that
its code is running under root (in contrast to people who develop modules like
mod_unixd and mod_privileges).

So we shouldn't run this in the parent process but only in the childs or fork a
separate child (like mod_cgid does) that only runs the watchdog if a
single-instance-non-locking watchdog is needed.


Regards

Rüdiger


Re: new watchdog module

Posted by Paul Querna <ch...@force-elite.com>.
Jeff Trawick wrote:
> On Wed, Feb 4, 2009 at 11:09 AM, Mladen Turk <mturk@apache.org 
> <ma...@apache.org>> wrote:
> 
>     Hi,
> 
>     I would like to commit this module to trunk.
>     It's a watchdog module that creates a worker threads
>     either in parent, child via an API, and it's not for
>     standalone use, but for other module usage like
>     mod_heartbeat, probably mod_jk and others that need
>     maintenance threads.
> 
> 
> It seems scary to me to have the MPM parent on Unix forking children 
> while a separate thread in that same process is running arbitrary code.  
> Surprising problems can crop up when this arbitrary code is in certain 
> states (maybe a mutex gets inherited in held state by the new MPM child?)
> 
> I once had a rather harsh quote on this topic from Ulrich Drepper taped 
> to my office wall amidst TCP/IP diagrams and definitions of C's break 
> and continue in very large print, but sadly it has gone missing. 
> 
> However, I just found another one which should suffice:
> 
> * Ulrich Drepper <drepper@redhat.com <ma...@redhat.com>> [030910 15:41]:
>> Aidan Van Dyk wrote:
>> 
>> > The particular problem I'm seeing is with syslog_lock.
> 
>> 
>> The standard doesn't guarantee that any functions but exec works after
>> fork in a multi-threaded program.
> 
> That doesn't match the profile of an MPM child process ;)
> 

^^ we have this problem on OSX too -- calling certain carbon APIs 
functions from apache, will cause it to abort() the child.

-Paul


Re: new watchdog module

Posted by Jeff Trawick <tr...@gmail.com>.
On Wed, Feb 4, 2009 at 11:09 AM, Mladen Turk <mt...@apache.org> wrote:

> Hi,
>
> I would like to commit this module to trunk.
> It's a watchdog module that creates a worker threads
> either in parent, child via an API, and it's not for
> standalone use, but for other module usage like
> mod_heartbeat, probably mod_jk and others that need
> maintenance threads.


It seems scary to me to have the MPM parent on Unix forking children while a
separate thread in that same process is running arbitrary code.  Surprising
problems can crop up when this arbitrary code is in certain states (maybe a
mutex gets inherited in held state by the new MPM child?)

I once had a rather harsh quote on this topic from Ulrich Drepper taped to
my office wall amidst TCP/IP diagrams and definitions of C's break and
continue in very large print, but sadly it has gone missing.

However, I just found another one which should suffice:

* Ulrich Drepper <dr...@redhat.com> [030910 15:41]:
> Aidan Van Dyk wrote:
>
> > The particular problem I'm seeing is with syslog_lock.
>
> The standard doesn't guarantee that any functions but exec works after
> fork in a multi-threaded program.

That doesn't match the profile of an MPM child process ;)

Re: new watchdog module

Posted by Brian McCallister <br...@skife.org>.
+1 to the concept, I haven't looked at code yet. I would use this,
more or less, immediately in fact.

-Brian

On Wed, Feb 4, 2009 at 8:09 AM, Mladen Turk <mt...@apache.org> wrote:
> Hi,
>
> I would like to commit this module to trunk.
> It's a watchdog module that creates a worker threads
> either in parent, child via an API, and it's not for
> standalone use, but for other module usage like
> mod_heartbeat, probably mod_jk and others that need
> maintenance threads.
>
> The reason is because couple of modules would benefit
> from this functionality, and this adds a common API.
>
> Usage is very simple
> 1. ap_watchdog_get_instance
> 2. set singleton mode (only one child will execute)
> 3. register a callback(s) with desired interval
>
> Module can either use pre-created watchdog instance
> or create it's own in special cases thus creating
> additional thread(s)
>
> For example heartbeat would be by an order of magnitude
> simpler (I took some concepts from it)
> Also I'd love to register the callbacks for balancer
> so transferred method can normalize the load.
>
> Comments?
>
> Regards
> --
> ^(TM)
>

Re: new watchdog module

Posted by Mladen Turk <mt...@apache.org>.
Paul Querna wrote:
>>
>> Comments?
> 
> I would rather make this kind of thing (scheduled tasks) part of the 
> MPMs, as they would be able to just use one of their exists task threads 
> to handle a scheduled task.
> 
> The Simple MPM has the ability to do this.
>

Scheduled task are just one of the problems.
It executes inside each child process, but that isn't
what we need in all cases.
Like with heartbeat it must allow execution in a
single child process only as well, and continue
on next when the first one dies.

Think that duplicating the code inside each mpm
would only make things needlessly complex.

And even if inside mpm, because the task worker
thread that executes different callbacks at different
intervals has to be detached all the time, there's no
difference between using existing or creating a
new thread thought.
Well the thread could be used for request processing
instead doing timer step resolution sleep,
but in that case there is no guarantee one will have
a free one when needed.

Regards
-- 
^(TM)

Re: new watchdog module

Posted by Paul Querna <ch...@force-elite.com>.
Nick Kew wrote:
> On Wed, 04 Feb 2009 10:38:44 -0800
> Paul Querna <ch...@force-elite.com> wrote:
> 
>> Mladen Turk wrote:
>>> I would like to commit this module to trunk.
> 
> Tending to +1 in principle, subject to think-time.
> 
>>> For example heartbeat would be by an order of magnitude
>>> simpler (I took some concepts from it)
> 
> That looks like a good example on which to elaborate to
> convince me and perhaps others of the merits of your module.
> 
>> I would rather make this kind of thing (scheduled tasks) part of the 
>> MPMs, as they would be able to just use one of their exists task
>> threads to handle a scheduled task.
> 
> Disagree.  You might export hooks, but let's have proper modularity
> in the implementation.

providers, death to hooks :(

Re: new watchdog module

Posted by Nick Kew <ni...@webthing.com>.
On Wed, 04 Feb 2009 10:38:44 -0800
Paul Querna <ch...@force-elite.com> wrote:

> Mladen Turk wrote:
> > I would like to commit this module to trunk.

Tending to +1 in principle, subject to think-time.

> > For example heartbeat would be by an order of magnitude
> > simpler (I took some concepts from it)

That looks like a good example on which to elaborate to
convince me and perhaps others of the merits of your module.

> I would rather make this kind of thing (scheduled tasks) part of the 
> MPMs, as they would be able to just use one of their exists task
> threads to handle a scheduled task.

Disagree.  You might export hooks, but let's have proper modularity
in the implementation.

> The Simple MPM has the ability to do this.

Examples?

-- 
Nick Kew

Re: new watchdog module

Posted by Paul Querna <ch...@force-elite.com>.
Mladen Turk wrote:
> Hi,
> 
> I would like to commit this module to trunk.
> It's a watchdog module that creates a worker threads
> either in parent, child via an API, and it's not for
> standalone use, but for other module usage like
> mod_heartbeat, probably mod_jk and others that need
> maintenance threads.
> 
> The reason is because couple of modules would benefit
> from this functionality, and this adds a common API.
> 
> Usage is very simple
> 1. ap_watchdog_get_instance
> 2. set singleton mode (only one child will execute)
> 3. register a callback(s) with desired interval
> 
> Module can either use pre-created watchdog instance
> or create it's own in special cases thus creating
> additional thread(s)
> 
> For example heartbeat would be by an order of magnitude
> simpler (I took some concepts from it)
> Also I'd love to register the callbacks for balancer
> so transferred method can normalize the load.
> 
> Comments?

I would rather make this kind of thing (scheduled tasks) part of the 
MPMs, as they would be able to just use one of their exists task threads 
to handle a scheduled task.

The Simple MPM has the ability to do this.

-Paul

Re: new watchdog module

Posted by Graham Leggett <mi...@sharp.fm>.
Mladen Turk wrote:

> I would like to commit this module to trunk.
> It's a watchdog module that creates a worker threads
> either in parent, child via an API, and it's not for
> standalone use, but for other module usage like
> mod_heartbeat, probably mod_jk and others that need
> maintenance threads.
> 
> The reason is because couple of modules would benefit
> from this functionality, and this adds a common API.
> 
> Usage is very simple
> 1. ap_watchdog_get_instance
> 2. set singleton mode (only one child will execute)
> 3. register a callback(s) with desired interval
> 
> Module can either use pre-created watchdog instance
> or create it's own in special cases thus creating
> additional thread(s)
> 
> For example heartbeat would be by an order of magnitude
> simpler (I took some concepts from it)
> Also I'd love to register the callbacks for balancer
> so transferred method can normalize the load.
> 
> Comments?

A huge +1.

The availability of this API will help mod_cache and friends to keep the 
  cache cleaned without expecting the admin to jump through hoops, and 
mod_session_dbd needs this to clean up expired sessions from time to time.

Regards,
Graham
--