htcacheclean and expired cache entries

[Rob Landrito <rl...@gmail.com>]

Greetings,

I noticed that htcacheclean won't clean expired entries unless the
specified size limit is reached.  Is there any specific reason for this ?
 I'm looking into freeing up some inodes on some of my servers where the
total cache size hasn't reached my specified limit.  Would it be worthwhile
to add a switch to enable cleaning of expired entries even if the size
limit isn't reached ?

Re: htcacheclean and expired cache entries

[Rob Landrito <rl...@gmail.com>]

Thanks for the response.  You're right, the -L option would be ideal.  We
probably won't be going to 2.4 for some time after release so I'll see if I
can't backport htcacheclean to 2.2.

Cheers

2011/12/2 Graham Leggett <mi...@sharp.fm>

> On 02 Dec 2011, at 8:52 PM, Rob Landrito wrote:
>
> > I noticed that htcacheclean won't clean expired entries unless the
> specified size limit is reached.
>
> That's by design.
>
> >  Is there any specific reason for this ?  I'm looking into freeing up
> some inodes on some of my servers where the total cache size hasn't reached
> my specified limit.  Would it be worthwhile to add a switch to enable
> cleaning of expired entries even if the size limit isn't reached ?
>
>
> The short answer is that you're probably looking for the -L option in the
> (soon to be released) httpd v2.4 to limit to the given number of inodes, as
> described here:
>
> http://httpd.apache.org/docs/trunk/programs/htcacheclean.html#options
>
> You probably want to use both the -l and -L options together to keep files
> safely within limits on disk.
>
> To answer your question, creating content has a cost associated with it,
> you want to keep content around for as long as humanly possible. Unlike
> typical caching systems that have a dual state "it's cached / it's
> deleted", HTTP caching offers a three state system: "it's fresh / it's
> stale / it's deleted".
>
> Initially, when cached, content is typically fresh for a "freshness
> lifetime", and when it gets old enough, the content goes into the second
> state, that of being "stale". Using the "conditional request" mechanism,
> the backend server is given the opportunity to confirm whether the stale
> content is actually still fresh, and in many cases this is so. The "stale"
> content is then updated to be "fresh" again. Eventually, the cache becomes
> full, so htcacheclean kicks in and reduces the cache size until it fits
> inside the given limits, but no more.
>
> Apart from the obvious benefit of not having to regenerate content from
> scratch when it goes stale, having stale content available allows you to do
> some useful things, like serve stale content to a thundering herd while the
> stale content is being freshened, or returning stale content when the
> backend returned an error. None of this is possible if you strip the cache
> of stale data.
>
> Regards,
> Graham
> --
>
>


-- 
:wq!

Re: htcacheclean and expired cache entries

[Graham Leggett <mi...@sharp.fm>]

On 02 Dec 2011, at 8:52 PM, Rob Landrito wrote:

> I noticed that htcacheclean won't clean expired entries unless the specified size limit is reached.

That's by design.

>  Is there any specific reason for this ?  I'm looking into freeing up some inodes on some of my servers where the total cache size hasn't reached my specified limit.  Would it be worthwhile to add a switch to enable cleaning of expired entries even if the size limit isn't reached ?


The short answer is that you're probably looking for the -L option in the (soon to be released) httpd v2.4 to limit to the given number of inodes, as described here:

http://httpd.apache.org/docs/trunk/programs/htcacheclean.html#options

You probably want to use both the -l and -L options together to keep files safely within limits on disk.

To answer your question, creating content has a cost associated with it, you want to keep content around for as long as humanly possible. Unlike typical caching systems that have a dual state "it's cached / it's deleted", HTTP caching offers a three state system: "it's fresh / it's stale / it's deleted".

Initially, when cached, content is typically fresh for a "freshness lifetime", and when it gets old enough, the content goes into the second state, that of being "stale". Using the "conditional request" mechanism, the backend server is given the opportunity to confirm whether the stale content is actually still fresh, and in many cases this is so. The "stale" content is then updated to be "fresh" again. Eventually, the cache becomes full, so htcacheclean kicks in and reduces the cache size until it fits inside the given limits, but no more.

Apart from the obvious benefit of not having to regenerate content from scratch when it goes stale, having stale content available allows you to do some useful things, like serve stale content to a thundering herd while the stale content is being freshened, or returning stale content when the backend returned an error. None of this is possible if you strip the cache of stale data.

Regards,
Graham
--