Forking off lengthy processes

[Jeremy Howard <jh...@fastmail.fm>]

I've just read the Guide's advice on fork and system calls, but it
doesn't really seem to offer a resolution to the problems it raises.

I'm trying to set up an SMS gateway for my webmail users. The gateway
just uses LWP to contact a web->SMS provider and sends a POST request
(following some redirects in the process). This can be a lengthy process,
so rather than have my mod_perl process waiting, I'd like to pass on the
request to another process. Here are the potential solutions I can think
of and their problems:
* fork(): Memory hog, since it copies the entire Apache process (is this
right--does it share the memory for all the modules etc...?)
* system(): Slow--would have to start up a new Perl process
* Create a simple preforking server and have mod_perl just pass a request
to it through a socket: Seems like overkill.

The most hopeful possibility seems to be fork(), but the Guide says to
avoid it. Isn't almost all the memory shared though?

The other place I'd like to avoid holding up my mod_perl processes is in
waiting for file uploads to finish (which is a common situation as people
upload attachments for their emails). Is there a any way to do this...
perhaps by using some kind of 'gateway' server?

-- 
  Jeremy Howard
  jhoward@fastmail.fm
  FastMail--Sign up at http://fastmail.fm

Re: Forking off lengthy processes

[Barrie Slaymaker <ba...@slaysys.com>]

Jeremy Howard wrote:
> 
> * fork(): Memory hog, since it copies the entire Apache process (is this
> right--does it share the memory for all the modules etc...?)

Assuming you're on a modern Unixish system, it shouldn't be bad at all if
you don't change too many variables and if the child doesn't often live long
enough for the parent to change too many (due to the memory sharing you list
below).  A bit of testing might be in order to see how hard it affects you.

> * system(): Slow--would have to start up a new Perl process

system() does a fork/exec under the hood.

> * Create a simple preforking server and have mod_perl just pass a request
> to it through a socket: Seems like overkill.
> 
> The most hopeful possibility seems to be fork(), but the Guide says to
> avoid it. Isn't almost all the memory shared though?

Yup.

> The other place I'd like to avoid holding up my mod_perl processes is in
> waiting for file uploads to finish (which is a common situation as people
> upload attachments for their emails). Is there a any way to do this...
> perhaps by using some kind of 'gateway' server?

The most often wished-for approach is for a http cache to accept the whole
request before passing it through to the main httpd, but I'm not aware of
any that do that.

- Barrie

Re: Forking off lengthy processes

[Robin Berjon <ro...@knowscape.com>]

At 01:19 10/07/2000 +0000, Jeremy Howard wrote:
>I've just read the Guide's advice on fork and system calls, but it
>doesn't really seem to offer a resolution to the problems it raises.
>
>I'm trying to set up an SMS gateway for my webmail users. The gateway
>just uses LWP to contact a web->SMS provider and sends a POST request
>(following some redirects in the process). This can be a lengthy process,
>so rather than have my mod_perl process waiting, I'd like to pass on the
>request to another process. Here are the potential solutions I can think
>of and their problems:
>* fork(): Memory hog, since it copies the entire Apache process (is this
>right--does it share the memory for all the modules etc...?)
>* system(): Slow--would have to start up a new Perl process
>* Create a simple preforking server and have mod_perl just pass a request
>to it through a socket: Seems like overkill.

It may only overkill if you have to write it in it's entirety just for that
:) There's plenty of example code and modules to build one such tool
easily. If you don't need all this to be done in real time you might want
to try a simple cron based system. Write your requests to a database, and
have them be sent every now and then.



-- robin b.
Suicidal twin kills sister by mistake!