KeysCached and sstable

[Paul Prescod <pa...@prescod.net>]

The inline docs say:

       ~ The optional KeysCached attribute specifies
       ~ the number of keys per sstable whose locations we keep in
       ~ memory in "mostly LRU" order.

There are a few confusing bits in that sentence.

 1. Why is "keys per sstable" rather than "keys per column family". If
I have 7 SSTable files and I set KeysCached to 10000, will I have
70000 keys cached? If so, why? What is the logical relationship here?

 2. What makes the algorithm "mostly LRU" rather than just LRU?

 3. Is it accurate the say that the goal of the Key Cache is to avoid
looking through a bunch off SSTable's Bloom Filters? (how big do the
bloom filters grow to...too much to be cached themselves?)

I'd like to document the detail.

 Paul Prescod

Re: KeysCached and sstable

[Jonathan Ellis <jb...@gmail.com>]

On Wed, Apr 14, 2010 at 10:23 AM, Paul Prescod <pa...@prescod.net> wrote:
> The inline docs say:
>
>       ~ The optional KeysCached attribute specifies
>       ~ the number of keys per sstable whose locations we keep in
>       ~ memory in "mostly LRU" order.
>
> There are a few confusing bits in that sentence.
>
>  1. Why is "keys per sstable" rather than "keys per column family". If
> I have 7 SSTable files and I set KeysCached to 10000, will I have
> 70000 keys cached? If so, why? What is the logical relationship here?

This is out of date, it's per CF now.

>  2. What makes the algorithm "mostly LRU" rather than just LRU?

it's called second-chance eviction, discussed at
http://code.google.com/p/concurrentlinkedhashmap/wiki/ProductionVersion

>  3. Is it accurate the say that the goal of the Key Cache is to avoid
> looking through a bunch off SSTable's Bloom Filters?

No.  it's to avoid deserializing the rows from the sstables and merging them.

-Jonathan