Re: Shambhala Modules Musings [network wizard advice sought at bottom]

["Robert S. Thau" <rs...@ai.mit.edu>]

   From: Randy Terbush <ra...@zyzzyva.com>
   Reply-To: new-httpd@hyperreal.com

   After looking at this a bit closer this afternoon, I am beginning to
   see some pretty incredible abuses of this module model.

   Consider a CGI server...

   You could create a separate module for each of your CGI applications.
   Based on the AddType, you would call that particular module.

So far, not abusive at all... the only thing to beware of is that the
server itself is a rather less forgiving environment for anyone's code
than an external CGI process --- file descriptors it fails to close
and memory it fails to allocate all leak, etc.  I think that for most
scripts, at most sites, it probably isn't worth the hassle.  But if
you want the option, it's there.

   You could
   even have a "standalone" CGI server listening on another port for
   handler requests from the main HTTP server.  No more startup delays
   for the various CGI programs.

On the other hand, if you're going to turn the scripts into
full-fledged internet servers in their own right, maybe turning them
into modules is an alternative to consider ;-).

   We're going to have to write a perl
   bootstrapper to load in modules. :-)  

Ha!  Bet you thought I wouldn't take that seriously!

What you're actually asking for is the ability to write modules in
Perl, java, or the other embedded language of your choice.  The major
barrier to that in Shambhala as it stands is the handling of timeouts:
when a request times out or gets a SIGPIPE, the signal handler in
http_main.c does a longjmp() straight back to child_main(), which
frees everything in the per-transaction resource pool and asks for
another request.

The problem here is that everything which *isn't* tied into the
resource pool machinery in alloc.c is a potential leak.  The code in
my development sandbox makes this a little less restrictive by
allowing just about anything to be registered with alloc.c as a
cleanup function, but you still can't get free of the resource pools
completely.  What makes that a problem for embedded languages is that
files opened, memory allocated, etc., by the embedded language's
native primitives won't be tied in to any Shambhala resource pool, and
will be potential leaks.

The easiest way to fix this is to change the way that timeouts are
handled.  Here's one idea --- instead of doing an immediate longjmp(),
just put the connection in an "aborted" state where the server will
refuse to do any more I/O on it.  (The easiest thing would be to just
close the file descriptor).

In fact, if I can safely assume that no sensible Unix-oid OS will
attempt to restart I/O on a socket when a SIGALRM handler returns,
this would allow me to ditch the longjmp() from the handling of client
aborts.  If that assumption is good, I'm going to try this (next
weekend at the latest).

That leaves the following requirements on a language to be embedded
into Shambhala:

1) it should be reentrant --- that is, it should be possible for code
   in, say, Perl to call the Shambhala C code, and to have that code
   in turn invoke more Perl code.  If not, you lose sub_requests.

2) Ideally, it should be thread-safe.

There probably aren't any essential conflicts between that and the
Perl5 embedding support (though it's a little hard to tell, given that
the documentation for embedded Perl5 consists, in its entirety, of the
sentence "Look at perl_main.c, and do something like that.").  At any
rate, it doesn't seem infeasable, although you'd need someone
knowledgable in Perl internals to do a proper job.

rst