sluggish servers

[Rob Hartill <ha...@ooo.lanl.gov>]

I just traced a problem with the server at Cardiff to NFS.

I guess lots of people offload cron jobs onto other local machines to
keep the load on the web server low, well, if those jobs need to write
to the disk on the web server, then this can be more trouble that it's
worth.

The movie db on the Cardiff machine was keeping a big cache of CGI output,
and another local machine was flushing it every 6 hours. The simple job
of deleting files over NFS can suck the life out of a web server.. Ours
was responding immediately, but pausing 5-10 seconds to send output...
Why ?, a file delete needs to update a dorectory file which hold the 
directory info...  that file can get pretty big, so if you are 
removing lots of files in rapid succession, the NFS overhead hits hard.

Even after the cache flush script had finished, the delays stayed for
at least 15 minutes (at which point I good tired of waiting and rebooted)..
The server responds and sends output immediately once more.


...just a lesson for you all, and something for the www documentation I
think.


rob
--
http://nqcd.lanl.gov/~hartill/