You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Philip Mak <pm...@aaanime.net> on 2000/12/23 03:08:48 UTC
Dynamic content that is static
Hi everyone,
I have been going over the modperl tuning guide and the suggestions that
people on this list sent me earlier. I've reduced MaxClients to 33 (each
httpd process takes up 3-4% of my memory, so that's how much I can fit
without swapping) so if the web server overloads again, at least it won't
take the machine down with it.
Running a non-modperl apache that proxies to a modperl apache doesn't seem
like it would help much because the vast majority of pages served require
modperl.
I realized something, though: Although the pages on my site are
dynamically generated, they are really static. Their content doesn't
change unless I change the files on the website. (For example,
http://www.animewallpapers.com/wallpapers/ccs.htm depends on header.asp,
footer.asp, series.dat and index.inc. If none of those files change, the
content of ccs.htm remains the same.)
So, it would probably be more efficient if I had a /src directory and a
/html directory. The /src directory could contain my modperl files and a
Makefile that knows the dependencies; when I type "make", it will evaluate
the modperl files and parse them into plain HTML files in the /html
directory.
Does anyone have any suggestions on how to implement this? Is there an
existing tool for doing this? How can I evaluate modperl/Apache::ASP files
from the command line?
Thanks,
-Philip Mak (pmak@aaanime.net)
Re: Dynamic content that is static
Posted by John Michael <jo...@acadiacom.net>.
I may be out of my realm here. I use mostly perl for everything and have
done similar things. Create a directory tree with the source files. In the
source files use something like %%INCLUDE_HEADER%% for each part of the page
that changes and have the script use flat text files for the build. Have
the script traverse the tree of the source files writing the output to the
html directory. Whenver you update the flat text files, just run the script
from the command line or write it to run from the web with a password.
Mason does something similar I believe.
You can even have the script write in the %%INCLUDES%% dynamically if you
take in the input and assign it like so.
$$Var = $value; instead of $input{'key'} = $value;
Then do the substitutions like so.
foreach (@variables){
$template_txt =~ s/%%$_%%/$$_/gi;
}
Works great for me. You can then make any change to the source.html page
and the flat text file without having to change the script.
John Michael
Not a mod perl solution, but it will work.
----- Original Message -----
From: "Dave Rolsky" <au...@urth.org>
To: "Dave Seidel" <da...@superluminal.com>
Cc: <mo...@apache.org>
Sent: Friday, December 22, 2000 9:28 PM
Subject: Re: Dynamic content that is static
> On 22 Dec 2000, Dave Seidel wrote:
>
> > I don't know if either Mason or Embperl offer static compilation, but
Mason has
> > caching and I believe that Embperl is getting caching. AxKit is also
very
> > cool, and caches.
>
> Using Mason to generate a set of HTML pages would not be too terribly
> difficult.
>
> If someone is interested in doing this and needs some guidance I'd be
> happy to help. It would be nice to include an offline site builder with
> Mason (or as a separate project).
>
> -dave
>
> /*==================
> www.urth.org
> We await the New Sun
> ==================*/
>
Re: Dynamic content that is static
Posted by Dave Rolsky <au...@urth.org>.
On 22 Dec 2000, Dave Seidel wrote:
> I don't know if either Mason or Embperl offer static compilation, but Mason has
> caching and I believe that Embperl is getting caching. AxKit is also very
> cool, and caches.
Using Mason to generate a set of HTML pages would not be too terribly
difficult.
If someone is interested in doing this and needs some guidance I'd be
happy to help. It would be nice to include an offline site builder with
Mason (or as a separate project).
-dave
/*==================
www.urth.org
We await the New Sun
==================*/
Re: Dynamic content that is static
Posted by Dave Seidel <da...@superluminal.com>.
> I realized something, though: Although the pages on my site are
> dynamically generated, they are really static. Their content doesn't
> change unless I change the files on the website. (For example,
> http://www.animewallpapers.com/wallpapers/ccs.htm depends on header.asp,
> footer.asp, series.dat and index.inc. If none of those files change, the
> content of ccs.htm remains the same.)
You might want to consider using a tool other than mod_perl in this situation.
There are preprocessor/compiler-type such as htmlpp or WML (both written in
Perl), or you can build the pages in PHP (e.g.) and compile them into static
pages with the command-line version. I've used both htmlpp and PHP, and both
work well, and I drive them both with make.
I don't know if either Mason or Embperl offer static compilation, but Mason has
caching and I believe that Embperl is getting caching. AxKit is also very
cool, and caches.
--
Dave Seidel
dave@superluminal.com
Re: Dynamic content that is static
Posted by barries <ba...@slaysys.com>.
On Fri, Dec 22, 2000 at 09:51:55PM -0500, brian d foy wrote:
>
> however, i have been talking to a few people about something like a
> mod_makefile. :)
I've used this approach succesfully on a lower volume site where the it
was taking lots of time to build the final HTML but the data sources
didn't change much. <plug>I have a module (Slay::Maker) I use for
exactly this purpose that takes a "makefile" written in Perl and I use
that to rebuild the pages, and if no page needs to be rebuilt, I can 304
the result</plug>. A mod_makefile would be even nicer, being written in
C.
If you're looking for Perlish makes, Nick Ing-Simmons also has a
Make.pm, there's a Makepp project out there, and <plug>I have an unreleased
but releasable Make.pm that supports most of the GNU constructs</plug>.
Putting squid or something in front of a heavily trafficed site (and
remembering to flush all or part of it's cache when you change the back
end would definitely help) and using a makefile approach on the backend
to avoid reading & writing lots of rarely updated data every page view
should both help.
- Barrie
RE: Dynamic content that is static
Posted by Christian Gilmore <cg...@tivoli.com>.
You might want to take a look at Strudel. It is a project people from my
last job were working on: http://www.research.att.com/~mff/strudel/.
Regards,
Christian
> -----Original Message-----
> From: Philip Mak [mailto:pmak@aaanime.net]
> Sent: Friday, December 22, 2000 8:09 PM
> To: modperl@apache.org
> Subject: Dynamic content that is static
>
>
> Hi everyone,
>
> I have been going over the modperl tuning guide and the
> suggestions that
> people on this list sent me earlier. I've reduced MaxClients
> to 33 (each
> httpd process takes up 3-4% of my memory, so that's how much I can fit
> without swapping) so if the web server overloads again, at
> least it won't
> take the machine down with it.
>
> Running a non-modperl apache that proxies to a modperl apache
> doesn't seem
> like it would help much because the vast majority of pages
> served require
> modperl.
>
> I realized something, though: Although the pages on my site are
> dynamically generated, they are really static. Their content doesn't
> change unless I change the files on the website. (For example,
> http://www.animewallpapers.com/wallpapers/ccs.htm depends on
> header.asp,
> footer.asp, series.dat and index.inc. If none of those files
> change, the
> content of ccs.htm remains the same.)
>
> So, it would probably be more efficient if I had a /src
> directory and a
> /html directory. The /src directory could contain my modperl
> files and a
> Makefile that knows the dependencies; when I type "make", it
> will evaluate
> the modperl files and parse them into plain HTML files in the /html
> directory.
>
> Does anyone have any suggestions on how to implement this? Is there an
> existing tool for doing this? How can I evaluate
> modperl/Apache::ASP files
> from the command line?
>
> Thanks,
>
> -Philip Mak (pmak@aaanime.net)
>
>
>
>
Re: Dynamic content that is static
Posted by brian d foy <co...@panix.com>.
On Fri, 22 Dec 2000, Philip Mak wrote:
> So, it would probably be more efficient if I had a /src directory and a
> /html directory. The /src directory could contain my modperl files and a
> Makefile that knows the dependencies; when I type "make", it will evaluate
> the modperl files and parse them into plain HTML files in the /html
> directory.
i've had great success with squid in http accelerator mode. we squeezed a
factor of 100 in speed with just that. :)
however, i have been talking to a few people about something like a
mod_makefile. :)
--
brian d foy <co...@panix.com>
Perl hacker for hire
Re: [OT] Where to download Sablotron for AxKit
Posted by Matt Sergeant <ma...@sergeant.org>.
On Sat, 23 Dec 2000, Philip Mak wrote:
> This is off-topic, but I am having problems downloading Sablotron from its
> website (Sablotron is a component that AxKit requires).
>
> On http://www.gingerall.com/charlie-bin/get/webGA/act/download.act the
> link for "Sablotron 0.50 - sources" and "Sablotron 0.50 - Linux
> binary" redirects to download.gingerall.cz, which is an unknown host.
>
> The Sablotron mailing list also appears to be busted, having an unknown
> host problem.
>
> Since several people mentioned AxKit on this list, I thought someone here
> might know about Sablotron. Do you know where I can download it from? I
> haven't been able to find any mirrors for it.
I'm having the exact same problem here. Mail me direct if you want me to
send you 0.44 which I have on my hard drive.
--
<Matt/>
/|| ** Director and CTO **
//|| ** AxKit.com Ltd ** ** XML Application Serving **
// || ** http://axkit.org ** ** XSLT, XPathScript, XSP **
// \\| // ** Personal Web Site: http://sergeant.org/ **
\\//
//\\
// \\
[OT] Where to download Sablotron for AxKit
Posted by Philip Mak <pm...@aaanime.net>.
This is off-topic, but I am having problems downloading Sablotron from its
website (Sablotron is a component that AxKit requires).
On http://www.gingerall.com/charlie-bin/get/webGA/act/download.act the
link for "Sablotron 0.50 - sources" and "Sablotron 0.50 - Linux
binary" redirects to download.gingerall.cz, which is an unknown host.
The Sablotron mailing list also appears to be busted, having an unknown
host problem.
Since several people mentioned AxKit on this list, I thought someone here
might know about Sablotron. Do you know where I can download it from? I
haven't been able to find any mirrors for it.
Thanks,
-Philip Mak (pmak@aaanime.net)
Re: Dynamic content that is static
Posted by Matt Sergeant <ma...@sergeant.org>.
On Fri, 22 Dec 2000, Philip Mak wrote:
> I realized something, though: Although the pages on my site are
> dynamically generated, they are really static. Their content doesn't
> change unless I change the files on the website. (For example,
> http://www.animewallpapers.com/wallpapers/ccs.htm depends on header.asp,
> footer.asp, series.dat and index.inc. If none of those files change, the
> content of ccs.htm remains the same.)
Thats just the sort of layout AxKit is great for. Its basically how Take23
works - even though it looks like a dynamically generated site, its not.
Everything just comes from static files, and when those files change,
AxKit recompiles the pages and caches the results (with some
intelligence associated with the parameters used to determine if this is
the same view of that URL).
--
<Matt/>
/|| ** Director and CTO **
//|| ** AxKit.com Ltd ** ** XML Application Serving **
// || ** http://axkit.org ** ** XSLT, XPathScript, XSP **
// \\| // ** Personal Web Site: http://sergeant.org/ **
\\//
//\\
// \\
Re: Dynamic content that is static
Posted by Ask Bjoern Hansen <as...@valueclick.com>.
On Fri, 22 Dec 2000, Philip Mak wrote:
> > > Running a non-modperl apache that proxies to a modperl apache doesn't seem
> > > like it would help much because the vast majority of pages served require
> > > modperl.
> >
> > Not necessarily.
> >
> > You can use mod_proxy to cache the dynamically generated pages on the
> > lightweight apache.
>
> I thought about this... but I'm not sure how I would tell the lightweight
> Apache to refresh its cache when a file gets changed. I suppose I could
> graceful restart it, but the other webmasters of the site do not have root
> access. (Or is there another way? Is it possible to teach Apache or Squid
> that ccs.htm depends on header.asp, footer.asp, series.dat and index.inc?)
I don't know with Apache::ASP, but it can probably be done. With
HTMl::Embperl it would be pretty trivial.
You could probably get quite a gain with having a squid or a
mod_proxy process in front. Both because they would slurp up the
data and feed it to slow clients and because if you could cache the
documents for just a minute or so it might save quite some hits on
the backend.
> Also, does this mess up the REMOTE_HOST variable, or is Apache smart
> enough to replace that with X-Forwarded-For when the forwarded traffic is
> being sent from a local priviledged process?
Have a look at ftp://ftp.netcetera.dk/pub/apache/mod_proxy_add_forward.c
- ask
--
ask bjoern hansen - <http://ask.netcetera.dk/>
more than 70M impressions per day, <http://valueclick.com>
Re: Dynamic content that is static
Posted by Edward Moon <em...@mooned.org>.
You should check out the documentation on mod_proxy to see what it's
capable of: <http://httpd.apache.org/docs/mod/mod_proxy.html>
You can specify expiration values and be assured that cached files older
than expiry will be deleted.
So, for example, if you know that your content gets updated every 48 hours
you can specify 'CacheMaxExpire 48' and force the proxy server to
retrieve a new copy every 48 hours.
You can also set headers within a dynamic document that specifies an
expiration time. Check out the link in my previous e-mail for more info.
On Fri, 22 Dec 2000, Philip Mak wrote:
> I thought about this... but I'm not sure how I would tell the lightweight
> Apache to refresh its cache when a file gets changed. I suppose I could
> graceful restart it, but the other webmasters of the site do not have root
> access. (Or is there another way? Is it possible to teach Apache or Squid
> that ccs.htm depends on header.asp, footer.asp, series.dat and index.inc?)
>
> Also, does this mess up the REMOTE_HOST variable, or is Apache smart
> enough to replace that with X-Forwarded-For when the forwarded traffic is
> being sent from a local priviledged process?
>
> -Philip Mak (pmak@aaanime.net)
>
Re: Dynamic content that is static
Posted by Philip Mak <pm...@aaanime.net>.
On Fri, 22 Dec 2000, Edward Moon wrote:
> > Running a non-modperl apache that proxies to a modperl apache doesn't seem
> > like it would help much because the vast majority of pages served require
> > modperl.
>
> Not necessarily.
>
> You can use mod_proxy to cache the dynamically generated pages on the
> lightweight apache.
I thought about this... but I'm not sure how I would tell the lightweight
Apache to refresh its cache when a file gets changed. I suppose I could
graceful restart it, but the other webmasters of the site do not have root
access. (Or is there another way? Is it possible to teach Apache or Squid
that ccs.htm depends on header.asp, footer.asp, series.dat and index.inc?)
Also, does this mess up the REMOTE_HOST variable, or is Apache smart
enough to replace that with X-Forwarded-For when the forwarded traffic is
being sent from a local priviledged process?
-Philip Mak (pmak@aaanime.net)
Re: Dynamic content that is static
Posted by Edward Moon <em...@mooned.org>.
Not necessarily.
You can use mod_proxy to cache the dynamically generated pages on the
lightweight apache.
Check out <http://perl.apache.org/guide/strategy.html#Apache_s_mod_proxy>
for details on what headers you'll need to set for caching to work.
On Fri, 22 Dec 2000, Philip Mak wrote:
> Hi everyone,
>
> I have been going over the modperl tuning guide and the suggestions that
> people on this list sent me earlier. I've reduced MaxClients to 33 (each
> httpd process takes up 3-4% of my memory, so that's how much I can fit
> without swapping) so if the web server overloads again, at least it won't
> take the machine down with it.
>
> Running a non-modperl apache that proxies to a modperl apache doesn't seem
> like it would help much because the vast majority of pages served require
> modperl.
>
Re: Dynamic content that is static
Posted by "G.W. Haywood" <ge...@www.jubileegroup.co.uk>.
Hi there,
On Fri, 22 Dec 2000, Philip Mak wrote:
> I realized something, though: Although the pages on my site are
> dynamically generated, they are really static.
You're not alone.
> Does anyone have any suggestions on how to implement this? Is there an
> existing tool for doing this? How can I evaluate modperl/Apache::ASP files
> from the command line?
You could use 'lynx -source URI >filename'.
73,
Ged.