ATS RAM scaling with disk cache size

[Tomasz Kuzemko <to...@kuzemko.net>]

Hello,
I have setup ATS with a disk cache of 2x 9TB raw disks and I see a 
constant resident memory usage of 21GB of RAM right after starting. 
Initially I thought this was the automatic RAM store using all this 
space, because according to documentation it should use around 1MB per 
1GB of disk storage, so this sums to around 18GB of RAM. But also 
according to the docs I should be able to put a hard limit on the RAM 
cache with this option:

CONFIG proxy.config.cache.ram_cache.size INT -1

but no matter what value I set here, it doesn't affect the initial 
memory usage. Is this by design? If yes, so it means that my maximum 
disk store size is limited by RAM size?

I'm using ATS version 3.2.0.

-- 
Best regards,
Tomasz Kuzemko
tomasz@kuzemko.net

Re: ATS RAM scaling with disk cache size

[Leif Hedstrom <zw...@apache.org>]

On 11/30/12 8:01 AM, Leif Hedstrom wrote:
>
> Note that 10 is a very small number of bytes per object. It's a key 
> benefit of the ATS cache vs e.g. Squid cache (which consumes 60 or so 
> bytes, last I checked). Nginx doesn't have this, since it just creates a 
> file on disk (but imagine having 1 billion i-nodes on the disk, good times 
> doing a full fsck on that ;).


Not to go into details, but obviously the kernel consumes memory for this 
type of disk cache (the i-node cache).

-- leif

Re: ATS RAM scaling with disk cache size

[Leif Hedstrom <zw...@apache.org>]

On 11/30/12 4:07 AM, Tomasz Kuzemko wrote:
> Hello,
> I have setup ATS with a disk cache of 2x 9TB raw disks and I see a 
> constant resident memory usage of 21GB of RAM right after starting. 
> Initially I thought this was the automatic RAM store using all this space, 
> because according to documentation it should use around 1MB per 1GB of 
> disk storage, so this sums to around 18GB of RAM. But also according to 
> the docs I should be able to put a hard limit on the RAM cache with this 
> option:
>
> CONFIG proxy.config.cache.ram_cache.size INT -1
>
> but no matter what value I set here, it doesn't affect the initial memory 
> usage. Is this by design? If yes, so it means that my maximum disk store 
> size is limited by RAM size?

Seems about right. Every disk cache object consumes 10 bytes of RAM, this is 
a "directory entry" (necessary for fast lookups). The default config 
allocates one directory entry for every 8000 bytes of storage. So, quick 
math tells me 2*9TiB / 8000  = 20.9GB.

Your config can hold roughly 1B HTTP objects in the cache (I varies 
depending on number of alternates etc.). If that is much more than you 
expect (typically because your average object size is much > than 16KB) then 
you can change this configuration in records.config. Doubling it to 16000 
would reduce memory consumption by 1/2 (so, 10.5GB) and so on and so forth.

Note that 10 is a very small number of bytes per object. It's a key benefit 
of the ATS cache vs e.g. Squid cache (which consumes 60 or so bytes, last I 
checked). Nginx doesn't have this, since it just creates a file on disk (but 
imagine having 1 billion i-nodes on the disk, good times doing a full fsck 
on that ;).

I hope that helps.

-- Leif