You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by "Jim Morrison [Mailing-Lists]" <ju...@mediaisotope.com> on 2003/12/05 16:09:29 UTC
Jumping out of Apache/mod_perl..
Hello,
Not sure if this isn't slightly OT, but wonder if someone can help..
I have a piece of one of my mod_perl apps that potentially takes quite a
while to complete. What I would like to do is to get Apache/mod_perl to
start a second process which would do the work, return a page to the
user, and get on with the next thing..
( The page would have a meta-refresh and could be updated by the
separate process so the user would know how things are progressing... )
I've been trying to read up on fork, etc. but it appears that even if I
fork, I have to wait about to clear up after my kids have passed on -
which still leaves me with an Apache/mod_perl process hanging around...
Is there a simple solution? Am I missing something?
Any help or pointing in the right direction would be much appreciated!
Thanks,
Jim
--
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
Re: Jumping out of Apache/mod_perl..
Posted by Christian Wetzig <mo...@wetzig.de>.
Jim Morrison [Mailing-Lists] wrote:
> I have a piece of one of my mod_perl apps that potentially takes quite a
> while to complete. What I would like to do is to get Apache/mod_perl to
> start a second process which would do the work, return a page to the
> user, and get on with the next thing..
I just did the following yesterday (for generating some PDFs and email
them to the user after finishing). So far it seems to work for me
(Slackware 9, Apache 2.0.48, mod_perl 1.99.11, Perl 5.8.0). It's adopted
from:
http://perl.apache.org/docs/1.0/guide/performance.html#Forking_and_Executing_Subprocesses_from_mod_perl
regards,
Christian
--------------------------------------------------
#!/usr/bin/perl -w
use strict;
use POSIX 'setsid';
# parent code goes here
$SIG{CHLD} = 'IGNORE';
defined (my $kid = fork) or die "Cannot fork: $!\n";
if ($kid) {
print "Parent $$ has finished, kid's PID: $kid\n";
}
else {
chdir '/' or die "Can't chdir to /: $!";
open STDIN, '/dev/null' or die "Can't read /dev/null: $!";
open STDOUT, '>/dev/null' or die "Can't write to /dev/null: $!";
# comment this out, if you want to keep apache error log, works for me
open STDERR, '>/tmp/log' or die "Can't write to /tmp/log: $!";
setsid or die "Can't start a new session: $!";
my $oldfh = select STDERR;
local $| = 1;
select $oldfh;
warn "started background task foo from $0\n";
# do something time-consuming
sleep 1, warn "$_\n" for 1..20;
warn "finished background task foo from $0\n";
CORE::exit(0); # terminate the process
}
# other parent code
--
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
Re: Jumping out of Apache/mod_perl..
Posted by Ken Corey <ke...@kencorey.com>.
> On Fri, 2003-12-05 at 15:09, Jim Morrison [Mailing-Lists] wrote:
> > I have a piece of one of my mod_perl apps that potentially takes quite a
> > while to complete. What I would like to do is to get Apache/mod_perl to
> > start a second process which would do the work, return a page to the
> > user, and get on with the next thing..
When I run into something like this, I usually end up using a database
as a buffer. You know, the initializing perl CGI creates a row in the
database iwth enough information to start the operation, and then
returns a 'success' message.
It seems a bit heavyweight at first, but it gives you lots of reporting
flexibility.
There was a daemon that would wake up every so often and ping the
database to see if there were any new rows to process. As it would
process them it would update the database.
To check on the status, I had another CGI that could query the database
to see progress and return it in a web page.
-Ken
--
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
RE: Jumping out of Apache/mod_perl..
Posted by "Jim Morrison [Mailing-Lists]" <ju...@mediaisotope.com>.
Thank you all -
- sorry, I ask the question and then I've been away for a couple of
days - blush..
The process is usually from half to 3 minutes - so the cleanup handler
seems like a great idea - then, there are instances where it's much much
longer, but in these cases it's probably easier to write a little script
to do the job and stay away from apache - the code should know roughly
how long it's going to take when it starts, because it can look at how
long it took last time.. I think I will go with the cleanup handler as
my first attempt, it's unlikely more than a couple of processes will be
busy with this job at anyone time, and there are 50 sitting there so.. .
And then have a go at forking for the longer jobs.
Thank you all :-)
J.
> -----Original Message-----
> From: Cees Hek [mailto:cees@sitesuite.com.au]
> Sent: 06 December 2003 05:36
> To: Stas Bekman
> Cc: junk@mediaisotope.com; modperl@perl.apache.org
> Subject: Re: Jumping out of Apache/mod_perl..
>
>
> Stas Bekman wrote:
> > Cees Hek wrote:
> >
> >> Using fork is expensive. It will use up as much memory as the
> >> current
> >> child is already using, so you might as well just use the current
> >> child to finished the processing.
> >
> > That will work only if you care to wait for all processes to finish
> > untill you can restart apache. forking and detaching the
> process allows
> > you shutdown your httpd servers without affecting your long
> term processes.
>
> I guess it depends on how long the process is going to take.
> I should
> have mentioned that I would only use this for something that
> is going to
> run for a few minutes. Long enough for a user to get anoyed and be
> tempted to press stop or refresh in the browser.
>
> If it is truely a long term process, then it will definately
> be better
> to fork (or to use the other suggestion of a daemon process
> looking for
> new work in a specified location)
>
> Cheers,
>
> Cees
>
>
> --
> Reporting bugs: http://perl.apache.org/bugs/
> Mail list info: http://perl.apache.org/maillist/modperl.html
>
> ---
> Incoming mail is certified Virus Free.
> Checked by AVG anti-virus system (http://www.grisoft.com).
> Version: 6.0.515 / Virus Database: 313 - Release Date: 01/09/2003
>
>
--
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
Re: Jumping out of Apache/mod_perl..
Posted by Cees Hek <ce...@sitesuite.com.au>.
Stas Bekman wrote:
> Cees Hek wrote:
>
>> Using fork is expensive. It will use up as much memory as the current
>> child is already using, so you might as well just use the current
>> child to finished the processing.
>
> That will work only if you care to wait for all processes to finish
> untill you can restart apache. forking and detaching the process allows
> you shutdown your httpd servers without affecting your long term processes.
I guess it depends on how long the process is going to take. I should
have mentioned that I would only use this for something that is going to
run for a few minutes. Long enough for a user to get anoyed and be
tempted to press stop or refresh in the browser.
If it is truely a long term process, then it will definately be better
to fork (or to use the other suggestion of a daemon process looking for
new work in a specified location)
Cheers,
Cees
--
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
Re: Jumping out of Apache/mod_perl..
Posted by Stas Bekman <st...@stason.org>.
Cees Hek wrote:
> Jim Morrison [Mailing-Lists] wrote:
>
>> Hello,
>>
>> Not sure if this isn't slightly OT, but wonder if someone can help..
>
>
> It seems on topic to me...
>
>> I have a piece of one of my mod_perl apps that potentially takes quite a
>> while to complete. What I would like to do is to get Apache/mod_perl to
>> start a second process which would do the work, return a page to the
>> user, and get on with the next thing..
>
>
> Using fork is expensive. It will use up as much memory as the current
> child is already using, so you might as well just use the current child
> to finished the processing.
That will work only if you care to wait for all processes to finish untill you
can restart apache. forking and detaching the process allows you shutdown your
httpd servers without affecting your long term processes.
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
--
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
Re: Jumping out of Apache/mod_perl..
Posted by "Charles C. Fu" <pe...@web-i18n.net>.
In <3F...@sitesuite.com.au> on 05 Dec 2003,
Cees Hek <ce...@sitesuite.com.au> wrote:
> Using fork is expensive. It will use up as much memory as the
> current child is already using, so you might as well just use the
> current child to finished the processing.
In most OSes, fork just sets the memory to copy-on-write, makes
another process table slot, .... It doesn't actually copy the memory,
so the memory cost is small.
In some cases (most notably Windows using Cygwin w/o copy-on-write
semantics), fork is emulated by copy the parent's memory. In this
case, the memory cost certainly is significant.
-ccwf
--
Charles C. Fu ,--
Founder ___ __ __. . ,-/--
Web i18n, LLC (_,(_,|/|/ /
www.web-i18n.net ----'
--
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
Re: Jumping out of Apache/mod_perl..
Posted by Cees Hek <ce...@sitesuite.com.au>.
Jim Morrison [Mailing-Lists] wrote:
> Hello,
>
> Not sure if this isn't slightly OT, but wonder if someone can help..
It seems on topic to me...
> I have a piece of one of my mod_perl apps that potentially takes quite a
> while to complete. What I would like to do is to get Apache/mod_perl to
> start a second process which would do the work, return a page to the
> user, and get on with the next thing..
Using fork is expensive. It will use up as much memory as the current
child is already using, so you might as well just use the current child
to finished the processing.
What I usually do in this situation is to register a cleanup handler
that does the actual processing. This means the child is locked up
until the processing is complete, but the client is free to continue
browsing since the cleanup phase runs after the request phase has
already finished.
Just add something like the following to your code.
$request->register_cleanup(sub { # Do your heavy processing here });
If you are going to have a lot of clients using this page
simultaneously, then you will need to make sure that all your child
processes aren't taken up by these long running programs. Otherwise any
new requests will block until a child process is avaliable.
Along with the other suggestions that have already been made, you should
have some options to look into now...
Cheers,
Cees
--
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
Re: Jumping out of Apache/mod_perl..
Posted by Larry Leszczynski <la...@furph.com>.
On Fri, 5 Dec 2003, Jim Morrison [Mailing-Lists] wrote:
> I have a piece of one of my mod_perl apps that potentially takes quite a
> while to complete. What I would like to do is to get Apache/mod_perl to
> start a second process which would do the work, return a page to the
> user, and get on with the next thing..
>
> ( The page would have a meta-refresh and could be updated by the
> separate process so the user would know how things are progressing... )
Not specific to mod_perl, but I like Randal's scheme for doing this kind
of thing (the example in the article is a CGI script but could just as
easily be a mod_perl handler):
http://marc.theaimsgroup.com/?l=apache-modperl&m=104538145031668&w=2
Larry Leszczynski
larryl@furph.com
--
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html