LRUCache issues

[Emmanuel Lécharny <el...@gmail.com>]

Hi guys,

so yesterday, I thought I have fixed the issue we have where the 
server-integ tests are blocked waiting for some slot to bee freed from 
the LRUCache.

Sadly, this is not the case, and while trying to cut the release another 
time, I get blocked again.

I slept on this problem and had come to the conclusion that the eviction 
algorithm we use is just not acceptable : we simply can't wait for some 
other thread to free a slot. That would make it possible for a user 
application to just kill the server if it forgets to close all the 
cursors it has created.

In no case we should allow such thing to happen. Increasing the cache 
size is not an alternative either.

All in all, this is just a cache, supposed to speed the server, not 
supposed to freeze it.

We need to change the eviction strategy to something that simply select 
the LRU element from the cache, even f it means we may have way more 
disk access.

-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com

Re: LRUCache issues

[Selcuk AYA <ay...@gmail.com>]

On Tue, May 8, 2012 at 10:55 PM, Emmanuel Lécharny <el...@gmail.com> wrote:
> Hi guys,
>
> so yesterday, I thought I have fixed the issue we have where the
> server-integ tests are blocked waiting for some slot to bee freed from the
> LRUCache.
>
> Sadly, this is not the case, and while trying to cut the release another
> time, I get blocked again.
>
> I slept on this problem and had come to the conclusion that the eviction
> algorithm we use is just not acceptable : we simply can't wait for some
> other thread to free a slot. That would make it possible for a user
> application to just kill the server if it forgets to close all the cursors
> it has created.

I can think of some ways to make the problem easier, however I cannot
think of anyway where the snapshot LRU cache or transactions would
work in the case cursors are not closed. For example we can make the
LRU cache to allocate cache element on demand rather than use a cache
element from the fixed size slots for storing previous versions of
btree pages but that would lead to out of memory after some time if
cursors are not closed. We can write the previous versions of pages to
somewhere on disk but again if cursors are not closed, this space will
keep growing.

>
> In no case we should allow such thing to happen. Increasing the cache size
> is not an alternative either.
>
> All in all, this is just a cache, supposed to speed the server, not supposed
> to freeze it.
>
> We need to change the eviction strategy to something that simply select the
> LRU element from the cache, even f it means we may have way more disk
> access.
>
> --
> Regards,
> Cordialement,
> Emmanuel Lécharny
> www.iktek.com
>

thanks
Selcuk