You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Matthew Tice <mj...@gmail.com> on 2008/12/15 17:11:14 UTC
htcacheclean ignoring -l (limit)?
I'm running apache 2.2.9 on several nodes that serve static content only.
I'm running into a problem where the disk cache is filling up (and
subsequently problems serving the files). I've the following configuration:
# /etc/fstab
tmpfs /www/cache tmpfs auto,rw,size=1200M,nr_inodes=1M 0 0
# ps aux | grep htcacheclean
/opt/apache2/bin/htcacheclean -d10 -t -p/www/cache -l750M
# httpd.conf
LoadModule cache_module modules/mod_cache.so
LoadModule disk_cache_module modules/mod_disk_cache.so
LoadModule mem_cache_module modules/mod_mem_cache.so
##################
## Cache Config ##
##################
CacheRoot /www/cache
# The default expire time for items in seconds:
CacheDefaultExpire 14400
CacheIgnoreCacheControl On
CacheEnable disk /img
CacheEnable disk /images
CacheEnable disk /aimages
CacheEnable disk /css
CacheEnable disk /js
###################
## DCache config ##
###################
# Maximum size of disk to use for caching in kbytes:
CacheDirLength 6
CacheDirLevels 3
# Maximum size of object to disk cache in bytes:
CacheMaxFileSize 100000
and the appropriate <Directory> directives.
So I've setup the disk_cache on a tmpfs and I'm running htcacheclean every
10 minutes to trim the cache down to 750M. If I run a df -h on that
filesystem I see that I've passed that 750M mark:
Filesystem Size Used Avail Use% Mounted on
tmpfs 1.2G 926M 275M 78% /www/cache
I'm not sure what I'm missing here. The only thing I can think of is that
htcacheclean will only clean down to the limit if there is content that is
expired and needs to be removed (all the content is below the 4 hour expire
time).
Any insights would be appreciated.
Matt
Re: htcacheclean ignoring -l (limit)?
Posted by Matthew Tice <mj...@gmail.com>.
>
> Okay, I tried that. Here are the results:
[snip]
Alright, I've figured it out. It's somewhat counter-intuitive (at least for
me). It looks like the -l (limit) doesn't take into consideration the size
of the directory.
Example:
# /opt/apache2/bin/htcacheclean -v -t -p/www/cache -l300M
Statistics:
size limit 300.0M
total size was 399.9M, total size now 299.9M
total entries was 27344, total entries now 20483
# df -h /www/cache/
Filesystem Size Used Avail Use% Mounted on
tmpfs 1.2G 415M 786M 35% /www/cache
But if I just add up the .header and .data files:
# count=0; for i in `find /www/cache -type f -ls | awk '{print $7}'`; do let
"count=$i + $count";done; echo $count
314566786
Convert bytes into megabytes:
# echo "scale=1;314566786/1048576" | bc
299.9
So I just need to keep playing with my -l (limit) and -d (daemonize
interval) so I can avoid hitting the ceiling.
Thanks for the help Morgan and Dan.
Matt
Re: htcacheclean ignoring -l (limit)?
Posted by Matthew Tice <mj...@gmail.com>.
>
> Try setting your cache size to some rediculously low (say 5K) mark. See if
> htclean does anything. Also, does root have read/write permissions on
> /www/cache? try chmod -R 777 /www/cache
>
>
Okay, I tried that. Here are the results:
# df -h /www/cache
Filesystem Size Used Avail Use% Mounted on
tmpfs 1.2G 369M 832M 31% /www/cache
# /opt/apache2/bin/htcacheclean -v -t -p/www/cache -l5k
Statistics:
size limit 5.0K
total size was 275.9M, total size now 0.0K
total entries was 18602, total entries now 0
# df -h /www/cache
Filesystem Size Used Avail Use% Mounted on
tmpfs 1.2G 75M 1.1G 7% /www/cache
Re: htcacheclean ignoring -l (limit)?
Posted by Morgan Gangwere <0....@gmail.com>.
Matthew Tice wrote:
> Statistics:
> size limit 750.0M
> total size was 705.3M, total size now 705.3M
> total entries was 105415, total entries now 105415
>
> # df -h /www/cache
> Filesystem Size Used Avail Use% Mounted on
> tmpfs 1.2G 1.2G 69M 95% /www/cache
>
Try setting your cache size to some rediculously low (say 5K) mark. See
if htclean does anything. Also, does root have read/write permissions on
/www/cache? try chmod -R 777 /www/cache
--
Morgan Gangwere
Unknown Software
http://sonof.bandit.name/
Member, INCOSE [ incose.org ] PACA [ paca.org ] and NMUG [ nmug.net ]
FAIL: Cannot wind the coils on the Super High Power Extemely Large
Hadron Collider! (fail code 0xd34df00d). please use the /extrafunds hook
to avoid this next start.
Find me on FreeNode! irc.freenode.net ~indrora on ##linux
Re: htcacheclean ignoring -l (limit)?
Posted by Matthew Tice <mj...@gmail.com>.
Exactly what have you done to start the daemon? have you nohup'd it? are
you using an INIT script?
>
> It appears that you'd have to run
> nohup htcacheclean -d5 -r -l700M -i
> or some such
>
> There IS an option that you can do. you can do this:
> add to your crontab a command to change out your cache. it could go in
> cron.hourly if you needed.
>
I'm starting the application with the -d daemonize option. Even if I start
it with -v I get the same inaccurate outcome:
# /opt/apache2/bin/htcacheclean -v -t -p/www/cache -l750M
Statistics:
size limit 750.0M
total size was 705.3M, total size now 705.3M
total entries was 105415, total entries now 105415
# df -h /www/cache
Filesystem Size Used Avail Use% Mounted on
tmpfs 1.2G 1.2G 69M 95% /www/cache
Re: htcacheclean ignoring -l (limit)?
Posted by Morgan Gangwere <0....@gmail.com>.
Matthew Tice wrote:
>
>
> On Tue, Dec 16, 2008 at 10:09 AM, Morgan Gangwere <0.fractalus@gmail.com
> <ma...@gmail.com>> wrote:
>
> [snip]
>
> Could be that something is taking up more space and its not being
> caught. Either that or the -l says the maximum size for 1 file. Have
> you RTFM'd?
>
>
> Oh yeah, I've read the man page,
> http://httpd.apache.org/docs/2.2/programs/htcacheclean.html, I even
> tried looking at the source. According to the man page:
> -l Specify limit as the total disk cache size limit
> I've specified -l with G,M, and bytes (thinking there was a
> miscalculation) - no dice. I've run through several times thinking
> maybe there is some kind of percentage of -l that is being reached; that
> doesn't seem to be the case.
>
> Matt
Exactly what have you done to start the daemon? have you nohup'd it? are
you using an INIT script?
It appears that you'd have to run
nohup htcacheclean -d5 -r -l700M -i
or some such
There IS an option that you can do. you can do this:
add to your crontab a command to change out your cache. it could go in
cron.hourly if you needed.
--
Morgan Gangwere
Unknown Software
http://sonof.bandit.name/
Member, INCOSE [ incose.org ] PACA [ paca.org ] and NMUG [ nmug.net ]
FAIL: Cannot wind the coils on the Super High Power Extemely Large
Hadron Collider! (fail code 0xd34df00d). please use the /extrafunds hook
to avoid this next start.
Find me on FreeNode! irc.freenode.net ~indrora on ##linux
Re: htcacheclean ignoring -l (limit)?
Posted by Matthew Tice <mj...@gmail.com>.
On Tue, Dec 16, 2008 at 2:01 PM, Dan Poirier <po...@pobox.com> wrote:
> Have you tried -t? It looks like htcacheclean just counts the sizes of
> the files, but if you have a lot of empty directories, that might be
> taking up some space too. -t looks like it would clean that up.
>
I thought so too - I tried -t but I still have the same problem.
Matt
Re: htcacheclean ignoring -l (limit)?
Posted by Dan Poirier <po...@pobox.com>.
Have you tried -t? It looks like htcacheclean just counts the sizes of
the files, but if you have a lot of empty directories, that might be
taking up some space too. -t looks like it would clean that up.
--
Dan Poirier <po...@pobox.com>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: htcacheclean ignoring -l (limit)?
Posted by Matthew Tice <mj...@gmail.com>.
On Tue, Dec 16, 2008 at 10:09 AM, Morgan Gangwere <0....@gmail.com>wrote:
> [snip]
>
> Could be that something is taking up more space and its not being caught.
> Either that or the -l says the maximum size for 1 file. Have you RTFM'd?
>
>
Oh yeah, I've read the man page,
http://httpd.apache.org/docs/2.2/programs/htcacheclean.html, I even tried
looking at the source. According to the man page:
-l Specify limit as the total disk cache size limit
I've specified -l with G,M, and bytes (thinking there was a miscalculation)
- no dice. I've run through several times thinking maybe there is some kind
of percentage of -l that is being reached; that doesn't seem to be the case.
Matt
Re: htcacheclean ignoring -l (limit)?
Posted by Morgan Gangwere <0....@gmail.com>.
[snip]
Could be that something is taking up more space and its not being
caught. Either that or the -l says the maximum size for 1 file. Have you
RTFM'd?
--
Morgan Gangwere
Unknown Software
http://sonof.bandit.name/
Member, INCOSE [ incose.org ] PACA [ paca.org ] and NMUG [ nmug.net ]
[indrora@boxly ~]#_
Find me on FreeNode! irc.freenode.net ~indrora on ##linux