You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by "Mendonce, Kiran (STSD)" <ki...@hp.com> on 2006/06/19 21:05:54 UTC
Question on multi-process CGID
Hi,
We had a scenario where the worker MPM was not performing as expected.
The bottleneck was identified as a single CGI daemon not being able to
cope with the volume of CGI requests coming in. So I made some changes
to convert the single process CGI daemon to multi-process. On multiple
CPU machines, there is a marginal increase in performance but not to the
extent that we expected. Any ideas on how I can improve performance ?
Mod_perl and mod_fastcgi are not acceptable solutions since it's a pure
CGI environment we are dealing with.
Any help would be greatly appreciated.
Thanks,
Kiran
RE: Question on multi-process CGID
Posted by "Mendonce, Kiran (STSD)" <ki...@hp.com>.
The problem with that would be HP does not package mod_fastcgi and
customers are always reluctant to change their existing scripts.
We tried the multi CGID approach and on multiple CPU machines, there was
an improvement in performance.
Regards,
Kiran
-----Original Message-----
From: Ian Holsman [mailto:lists@holsman.net]
Sent: Wednesday, June 21, 2006 4:54 AM
To: dev@httpd.apache.org
Subject: Re: Question on multi-process CGID
tackling this another way.
how hard would it be to use something like mod_fastcgi of instead of the
standard CGI interface?
On 21/06/2006, at 8:00 AM, Paul Querna wrote:
> Mendonce, Kiran (STSD) wrote:
>>
>> We tried using mod_cgi with worker. And its very slow. So that's not
>> an option we have. Currently we have only worker MPM supported on
>> HP-UX which is why I tried the multiple cgid approach.
>
> Ah. Now it makes sense. My experiences with this have only been on
> Linux, FreeBSD and Solaris. It is very much a possibility that fork()
> on HP-UX really sucks.
>
>> And the solution that was discussed then was also to provide multiple
>> CGI daemons. Will this be a feature that will be available sometime ?
>
> I don't think anyone is currently working on it. Feel Free to post
> patches :)
>
>> Also, I'd like to know if any benchmaking results for CGI has been
>> published with comparisons between the different Apache MPMs.
>
> Never done it on HP-UX. On Linux 2.6 the experimental Event MPM is
> fastest, then the Worker MPM and then the Prefork MPM.
>
> -Paul
>
Re: Question on multi-process CGID
Posted by Ian Holsman <li...@holsman.net>.
tackling this another way.
how hard would it be to use something like mod_fastcgi of instead of
the standard CGI interface?
On 21/06/2006, at 8:00 AM, Paul Querna wrote:
> Mendonce, Kiran (STSD) wrote:
>>
>> We tried using mod_cgi with worker. And its very slow. So that's
>> not an
>> option we have. Currently we have only worker MPM supported on HP-UX
>> which is why I tried the multiple cgid approach.
>
> Ah. Now it makes sense. My experiences with this have only been on
> Linux, FreeBSD and Solaris. It is very much a possibility that fork()
> on HP-UX really sucks.
>
>> And the solution that was discussed then was also to provide multiple
>> CGI daemons. Will this be a feature that will be available sometime ?
>
> I don't think anyone is currently working on it. Feel Free to post
> patches :)
>
>> Also, I'd like to know if any benchmaking results for CGI has been
>> published with comparisons between the different Apache MPMs.
>
> Never done it on HP-UX. On Linux 2.6 the experimental Event MPM is
> fastest, then the Worker MPM and then the Prefork MPM.
>
> -Paul
>
Re: Question on multi-process CGID
Posted by Paul Querna <ch...@force-elite.com>.
Mendonce, Kiran (STSD) wrote:
>
> We tried using mod_cgi with worker. And its very slow. So that's not an
> option we have. Currently we have only worker MPM supported on HP-UX
> which is why I tried the multiple cgid approach.
Ah. Now it makes sense. My experiences with this have only been on
Linux, FreeBSD and Solaris. It is very much a possibility that fork()
on HP-UX really sucks.
> And the solution that was discussed then was also to provide multiple
> CGI daemons. Will this be a feature that will be available sometime ?
I don't think anyone is currently working on it. Feel Free to post
patches :)
> Also, I'd like to know if any benchmaking results for CGI has been
> published with comparisons between the different Apache MPMs.
Never done it on HP-UX. On Linux 2.6 the experimental Event MPM is
fastest, then the Worker MPM and then the Prefork MPM.
-Paul
RE: Question on multi-process CGID
Posted by "Mendonce, Kiran (STSD)" <ki...@hp.com>.
We tried using mod_cgi with worker. And its very slow. So that's not an
option we have. Currently we have only worker MPM supported on HP-UX
which is why I tried the multiple cgid approach.
I went through some of the archived discussions to see if anybody had
experienced something similar. There was one similar discussion. So
theres at least one other person who experienced the same limitation.
Here's the URL :
http://mail-archives.apache.org/mod_mbox/httpd-dev/200303.mbox/%3c12B31D
D1A712D611920F0008C70D393B03CD685C@w12tcxu04.tv.bbc.co.uk%3e
And the solution that was discussed then was also to provide multiple
CGI daemons. Will this be a feature that will be available sometime ?
Also, I'd like to know if any benchmaking results for CGI has been
published with comparisons between the different Apache MPMs.
Thanks,
Kiran
-----Original Message-----
From: Paul Querna [mailto:chip@force-elite.com]
Sent: Tuesday, June 20, 2006 2:18 PM
To: dev@httpd.apache.org
Subject: Re: Question on multi-process CGID
Mendonce, Kiran (STSD) wrote:
> I am looking into the probable bottlenecks.
>
> Agreed that the worker MPM has its advantages. But for a customer who
> is being asked to move to Apache 2.0, we are falling short on the
> performance and that makes it hard to sell. Since worker + mod_cgid
> was supposed to improve performance, how is it that the benchmarking
> numbers fall short ?
It falls short on the CGI benchmarks, because of how mod_cgid works --
it has to communicate across a pipe. In the real world Ive never seen
that as a bottleneck. You can use mod_cgi with the Worker MPM, it might
or might not be safe, depending on your operating system.
The worker MPM will generally use less ram (less ram = more concurrency
is possible), be faster for static files, etc.
-Paul
Re: Question on multi-process CGID
Posted by Paul Querna <ch...@force-elite.com>.
Mendonce, Kiran (STSD) wrote:
> I am looking into the probable bottlenecks.
>
> Agreed that the worker MPM has its advantages. But for a customer who is
> being asked to move to Apache 2.0, we are falling short on the
> performance and that makes it hard to sell. Since worker + mod_cgid was
> supposed to improve performance, how is it that the benchmarking numbers
> fall short ?
It falls short on the CGI benchmarks, because of how mod_cgid works --
it has to communicate across a pipe. In the real world Ive never seen
that as a bottleneck. You can use mod_cgi with the Worker MPM, it might
or might not be safe, depending on your operating system.
The worker MPM will generally use less ram (less ram = more concurrency
is possible), be faster for static files, etc.
-Paul
> -----Original Message-----
> From: Paul Querna [mailto:chip@force-elite.com]
> Sent: Tuesday, June 20, 2006 1:44 PM
> To: dev@httpd.apache.org
> Subject: Re: Question on multi-process CGID
>
> Mendonce, Kiran (STSD) wrote:
>>
>>> It depends on where the real bottleneck is.
>>>
>>> Most of the time, if you are unable to cope with the volume of
>>> incoming
>> CGI requests, its because your CGIs themselves are slow to start.
>>> For example, if your CGIs are coded in Perl, just starting them can
>> take a long time, which is independent of mod_cgi(d).
>>> If your CGIs are written in C, then the startup time is minimal.
>> No, the CGIs are not in perl. They are either shell scripts or C
>> programs. Strangely, Apache 1.3 performs better than Apache 2.0.xx
>> with the worker MPM when the CGIs are C programs or shell scripts. We
>> did the benchmarking with the ab utility. Anyone has similar
>> experiences ? And any idea why that should be the case ?
>
> I believe you really need to look at identifying the bottleneck. I have
> a hard time believing the speed of fork()+execv() is truly the cause of
> your problems.
>
> Last time I tested 2.x, C-programs for me were about 1-2% slower with
> the Worker MPM, compared to the prefork MPM+mod_cgi.
>
> I don't consider it significant, because of the other advantages to the
> worker MPM.
RE: Question on multi-process CGID
Posted by "Mendonce, Kiran (STSD)" <ki...@hp.com>.
I am looking into the probable bottlenecks.
Agreed that the worker MPM has its advantages. But for a customer who is
being asked to move to Apache 2.0, we are falling short on the
performance and that makes it hard to sell. Since worker + mod_cgid was
supposed to improve performance, how is it that the benchmarking numbers
fall short ?
Regards,
Kiran
-----Original Message-----
From: Paul Querna [mailto:chip@force-elite.com]
Sent: Tuesday, June 20, 2006 1:44 PM
To: dev@httpd.apache.org
Subject: Re: Question on multi-process CGID
Mendonce, Kiran (STSD) wrote:
>
>> It depends on where the real bottleneck is.
>>
>> Most of the time, if you are unable to cope with the volume of
>> incoming
> CGI requests, its because your CGIs themselves are slow to start.
>> For example, if your CGIs are coded in Perl, just starting them can
> take a long time, which is independent of mod_cgi(d).
>> If your CGIs are written in C, then the startup time is minimal.
>
> No, the CGIs are not in perl. They are either shell scripts or C
> programs. Strangely, Apache 1.3 performs better than Apache 2.0.xx
> with the worker MPM when the CGIs are C programs or shell scripts. We
> did the benchmarking with the ab utility. Anyone has similar
> experiences ? And any idea why that should be the case ?
I believe you really need to look at identifying the bottleneck. I have
a hard time believing the speed of fork()+execv() is truly the cause of
your problems.
Last time I tested 2.x, C-programs for me were about 1-2% slower with
the Worker MPM, compared to the prefork MPM+mod_cgi.
I don't consider it significant, because of the other advantages to the
worker MPM.
Re: Question on multi-process CGID
Posted by Paul Querna <ch...@force-elite.com>.
Mendonce, Kiran (STSD) wrote:
>
>> It depends on where the real bottleneck is.
>>
>> Most of the time, if you are unable to cope with the volume of incoming
> CGI requests, its because your CGIs themselves are slow to start.
>> For example, if your CGIs are coded in Perl, just starting them can
> take a long time, which is independent of mod_cgi(d).
>> If your CGIs are written in C, then the startup time is minimal.
>
> No, the CGIs are not in perl. They are either shell scripts or C
> programs. Strangely, Apache 1.3 performs better than Apache 2.0.xx with
> the worker MPM when the CGIs are C programs or shell scripts. We did the
> benchmarking with the ab utility. Anyone has similar experiences ? And
> any idea why that should be the case ?
I believe you really need to look at identifying the bottleneck. I have
a hard time believing the speed of fork()+execv() is truly the cause of
your problems.
Last time I tested 2.x, C-programs for me were about 1-2% slower with
the Worker MPM, compared to the prefork MPM+mod_cgi.
I don't consider it significant, because of the other advantages to the
worker MPM.
RE: Question on multi-process CGID
Posted by "Mendonce, Kiran (STSD)" <ki...@hp.com>.
>It depends on where the real bottleneck is.
>
>Most of the time, if you are unable to cope with the volume of incoming
CGI requests, its because your CGIs themselves are slow to start.
>
>For example, if your CGIs are coded in Perl, just starting them can
take a long time, which is independent of mod_cgi(d).
>
>If your CGIs are written in C, then the startup time is minimal.
No, the CGIs are not in perl. They are either shell scripts or C
programs. Strangely, Apache 1.3 performs better than Apache 2.0.xx with
the worker MPM when the CGIs are C programs or shell scripts. We did the
benchmarking with the ab utility. Anyone has similar experiences ? And
any idea why that should be the case ?
Thanks,
Kiran
Re: Question on multi-process CGID
Posted by Paul Querna <ch...@force-elite.com>.
Mendonce, Kiran (STSD) wrote:
> Hi,
>
> We had a scenario where the worker MPM was not performing as expected.
> The bottleneck was identified as a single CGI daemon not being able to
> cope with the volume of CGI requests coming in. So I made some changes
> to convert the single process CGI daemon to multi-process. On multiple
> CPU machines, there is a marginal increase in performance but not to the
> extent that we expected. Any ideas on how I can improve performance ?
> Mod_perl and mod_fastcgi are not acceptable solutions since it's a pure
> CGI environment we are dealing with.
It depends on where the real bottleneck is.
Most of the time, if you are unable to cope with the volume of incoming
CGI requests, its because your CGIs themselves are slow to start.
For example, if your CGIs are coded in Perl, just starting them can take
a long time, which is independent of mod_cgi(d).
If your CGIs are written in C, then the startup time is minimal.
-Paul