[users@httpd] Filesystem types for mod_proxy cache in reverse proxy

[Neil Gunton <ne...@nilspace.com>]

Hi all,

I am using Apache 2.2.9 (built from source) on Debian Lenny to run a 
fairly large community LAMP (Perl, MySQL) site. I use the proxy and 
cache of Apache to improve site performance - I have a front end proxy 
build and a back-end mod_perl build. I have been using this setup for 
years successfully, but most of that time was using Apache 1.3, with 
mod_access and mod_deflate from Igor Sysoev. Since moving to Apache 2.2, 
I am using the stock caching.

I am finding that the cache directories that mod_cache builds are very 
large, and take a long time to traverse under ext2. There is currently 
about 10 GB under the cache according to du, and it took 162 minutes 
just to tell me that. I'm using three levels of directory. Htcacheclean 
also takes a long time to process this if I run it from cron nightly, 
during which time I see a huge spike in iowait on the server. If I run 
htcacheclean in daemon mode, using the -n (nice) option, then it doesn't 
seem to be able to keep up, the cache just creeps up in size.

So what I'm observing is that at least part of the problem appears to be 
that the directory structure is just very, very big and takes a long 
time to traverse, even for basic system functions like du.

This leads to my main question, which is this: Would a different 
filesystem, perhaps reiserfs, be better for the cache? I have never used 
reiser before, but from reputation it seems to be designed for handling 
many small files efficiently. I wonder if it would be any easier for my 
system to traverse the directory and maintain the cache if it was under 
reiser rather than ext.

Has anybody any experience that would confirm or deny this theory? 
Anybody have any advice for making cache management a little more, well, 
manageable?

I have a quad core server (AMD Opteron 265), with four 10k SCSI drives 
set up in RAID0 (yeah I know it's risky, but everything is backed up 
immediately via mysql replication, and I need the space and performance).

Thanks!

Neil

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org