You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Igor Tatarinov <ta...@prairie.NoDak.edu> on 1998/01/26 23:15:17 UTC
CGI output caching
Hi folks,
Sorry I've already asked this question but didn't get any answer. Is the
idea absolutely dumb or what?
What I am thinking of is caching of CGI output. I know that it's tough and
not always useful (I hear you saying "Don't cache our counter, you moron!).
But everytime I visit imdb (hi Rob!) I think that it may worth it. Just
think of eliminating the need to start a new perl process and execute it.
Of course, in some situations it might be a bad idea. The main problem is
probably that the length of the output stream is unknown.
Yeah, it may be tougher than I think and isn't it what proxies are for?
anyway your comments are welcome
igor
Re: CGI output caching
Posted by Ed Korthof <ed...@organic.com>.
Another way to achieve this effect would be to serve static HTML which is
generated from CGIs or other scripts when appropriate. That way you can
change the pages when appropriate, but they'll be served up w/o additional
processing.
Using .htaccess files and mod_expires, if you know when the files will
expire, you could do this quite effectively.
It's kind of a pain figuring out all the stuff you'd need to do, though.
Stuff like mod_perl, FastCGI, or mod_jserv probably makes more sense...
-- Ed Korthof | Web Server Engineer --
-- ed@organic.com | Organic Online, Inc --
-- (415) 278-5676 | Fax: (415) 284-6891 --
On Mon, 26 Jan 1998, Igor Tatarinov wrote:
> I am talking about caching the output from my server's CGI scripts.
> Yes, I know that proxies can do that but the problem is that when people
> from different parts of the world (diff ISPs) request data they may
> be using different proxies. So, it is more efficient to install a
> proxy at the Web server site not at the user sites.
>
> But all this doesn't matter if what Michael said is true: each time a
> CGI script is called (even with the same parameters) the output will be
> different, e.g., there will be a timestamp inserted somewhere.
>
> Is it really so (I have no industrial experience)?
> if yes most CGI scripts are non-cacheable (expire immediately).
>
> If no there is still space for improvement since producing the output
> may be costly, e.g. a DB lookup.
>
> igor
Re: CGI output caching
Posted by Igor Tatarinov <ta...@prairie.NoDak.edu>.
I am talking about caching the output from my server's CGI scripts.
Yes, I know that proxies can do that but the problem is that when people
from different parts of the world (diff ISPs) request data they may
be using different proxies. So, it is more efficient to install a
proxy at the Web server site not at the user sites.
But all this doesn't matter if what Michael said is true: each time a
CGI script is called (even with the same parameters) the output will be
different, e.g., there will be a timestamp inserted somewhere.
Is it really so (I have no industrial experience)?
if yes most CGI scripts are non-cacheable (expire immediately).
If no there is still space for improvement since producing the output
may be costly, e.g. a DB lookup.
igor
Rodent of Unusual Size wrote:
>
> Igor Tatarinov wrote:
> >
> > What I am thinking of is caching of CGI output. I know that it's tough and
> > not always useful (I hear you saying "Don't cache our counter, you moron!).
>
> Are you talking about having control over whether the output from
> *your* scripts is cached, or caching that from some remote site?
> For the former, you should be able to do what you want by just having
> your script emit the correct response header fields; for the latter..
> well, don't cache anything that says it shouldn't be cached. Under
> any circumstances.
>
> I guess I'm missing the context here.
>
> #ken P-)}
--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Igor Tatarinov, graduate student, Computer Science Dept, NDSU
e-mail: tatarino@prairie.nodak.edu or itat@acm.org
http://www.cs.ndsu.nodak.edu/~tatarino
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Re: CGI output caching
Posted by Rodent of Unusual Size <Ke...@Golux.Com>.
Igor Tatarinov wrote:
>
> What I am thinking of is caching of CGI output. I know that it's tough and
> not always useful (I hear you saying "Don't cache our counter, you moron!).
Are you talking about having control over whether the output from
*your* scripts is cached, or caching that from some remote site?
For the former, you should be able to do what you want by just having
your script emit the correct response header fields; for the latter..
well, don't cache anything that says it shouldn't be cached. Under
any circumstances.
I guess I'm missing the context here.
#ken P-)}
Re: CGI output caching
Posted by Michael Douglass <mi...@texas.net>.
On Mon, Jan 26, 1998 at 04:15:17PM -0600, Igor Tatarinov said:
> But everytime I visit imdb (hi Rob!) I think that it may worth it. Just
> think of eliminating the need to start a new perl process and execute it.
You should look for mod_perl as I believe that the imdb is using mod_perl.
With mod_perl you aren't starting up a new perl process every time; in
fact you never start a perl process. Furthermore, you only end up compiling
each perl program into memory once for each child--unless the perl script
changes, at which time mod_perl reparses it. :)
This may be what you want. But as far as I can tell, there are no CGI
scripts that I have ever written that are static. The very nature that
I write CGI scripts is to let me build dynamic-html.
--
Michael Douglass
Texas Networking, Inc.
<tnet admin> anyway, I'm off, perl code is making me [a] crosseyed toady
Re: CGI output caching (bad idea)
Posted by Igor Tatarinov <ta...@prairie.NoDak.edu>.
Well, guys, you convinced me:
CGI output caching is generally a bad idea :(
What I was thinking about is indeed should be done on a lower layer.
Thanks!
igor
Marc Slemko wrote:
>
> You can't normally do CGI caching in the web server because you don't know
> what the output is based on.
>
> Now, if you have a CGI and you know its output is based on the
> query string, for example, then you may be able to safely cache that if
> you decide that the data it is getting will not change too often, etc.
>
> Normally the better solution is to change the CGI into some other method
> (eg. module, mod_perl. mod_fastcgi, etc.) so you remove the CGI overhead
> and then cache at some lower layer if you are pulling it from a database,
> etc. The times when you can usefully cache CGI output are very limited
> and require extensive manual configuration.
Re: CGI output caching
Posted by Marc Slemko <ma...@worldgate.com>.
You can't normally do CGI caching in the web server because you don't know
what the output is based on.
Now, if you have a CGI and you know its output is based on the
query string, for example, then you may be able to safely cache that if
you decide that the data it is getting will not change too often, etc.
Normally the better solution is to change the CGI into some other method
(eg. module, mod_perl. mod_fastcgi, etc.) so you remove the CGI overhead
and then cache at some lower layer if you are pulling it from a database,
etc. The times when you can usefully cache CGI output are very limited
and require extensive manual configuration.
On Mon, 26 Jan 1998, Igor Tatarinov wrote:
> Hi folks,
>
> Sorry I've already asked this question but didn't get any answer. Is the
> idea absolutely dumb or what?
>
> What I am thinking of is caching of CGI output. I know that it's tough and
> not always useful (I hear you saying "Don't cache our counter, you moron!).
>
> But everytime I visit imdb (hi Rob!) I think that it may worth it. Just
> think of eliminating the need to start a new perl process and execute it.
>
> Of course, in some situations it might be a bad idea. The main problem is
> probably that the length of the output stream is unknown.
>
> Yeah, it may be tougher than I think and isn't it what proxies are for?
>
> anyway your comments are welcome
> igor
>
Re: CGI output caching
Posted by Rob Hartill <ro...@imdb.com>.
On Mon, 26 Jan 1998, Igor Tatarinov wrote:
> Hi folks,
>
> Sorry I've already asked this question but didn't get any answer. Is the
> idea absolutely dumb or what?
>
> What I am thinking of is caching of CGI output. I know that it's tough and
> not always useful (I hear you saying "Don't cache our counter, you moron!).
>
> But everytime I visit imdb (hi Rob!) I think that it may worth it. Just
> think of eliminating the need to start a new perl process and execute it.
As others have suggested, eliminating CGI will be a big step in the
right direction. If you still need more speedups and have output that
is stable enough to benefit from caching then you can look at adding that
later.
Note that even with cached data, you can still change it on-the-fly
with SSI. All my cached pages go through SSI filtering to present
different views depending on how and when accessed.
--
Rob Hartill Internet Movie Database (Ltd)
http://www.moviedatabase.com/ .. a site for sore eyes.