RE: Using group within region causing out of memory problem

[Aaron Smuts <aa...@wisc.edu>]

I'm ignoring the problem with the groups for the moment.  Try your test
without using the groups at all.  

What was your test code like for JPROBE?  The indexed disk cache keeps
the key and the location of the item in the file in memory.  This is
kept in the disk element descriptor, which is very small.  Yes, if you
will have an infinite number of items, it will take up an infinite
amount of space.  This isn't a memory leak though.  

Items are removed from the list of keys for the disk when they are
removed.  This should not be a problem in most applications, since there
is no point in caching unique data and the size of the key is very
small.  You should be able to have millions and millions of items
without memory problems.  I can't understand how anyone would ever have
this many items in the cache.  

I guess if you were putting something like session related data in the
cache and none of the sessions were cleaned up, then after months it
could be a problem if you have as many users as, say, yahoo.
  
There is another, much slower b-tree auxiliary that doesn't keep many
keys in memory.  

We may want to implement a disk cleanup system.  It could check to see
if elements have expired, etc.  I've been thinking about how to have
periodic defragmentation too.  Right now it just defrags on shutdown. 

What does your test code look like?  What are you planning on caching?
Can you give some rough idea of what you are using JCS for?

Cheers,

Aaron




> -----Original Message-----
> From: Amy Wang [mailto:amy.wang@intersperse.com]
> Sent: Thursday, May 29, 2003 7:08 PM
> To: 'turbine-jcs-user@jakarta.apache.org'; 'turbine-jcs-
> dev@jakarta.apache.org'
> Subject: Using group within region causing out of memory problem
> 
> Hi, All,
> 
> I just wrote some tests against JCS performance.  Here is some
interesting
> thing I discovered.
> If I put cached object to group using putInGroup(key, groupName,
value),
> it
> will result in memory leak problem. Below is the memory report from
> JProble:
> Before it ran out of memory, the groupId, groupAttrName, Integer(don't
> know
> what it is used for) and IndexedDiskElementDescriptor used up most of
the
> heap.
> 
> Class
> Memory usage
> org.apache.jcs.engine.control.group.GroupId
> 1,697,328 (21.0%)
> org.apache.jcs.engine.control.group.GroupAttrName
> 1,697,328 (21.0%)
> java.lang.Integer
> 1,115,024 (13.8%)
> org.apache.jcs.auxiliary.disk.indexed.IndexedDiskElementDescriptor
> 1,624,560
> (20.1%)
> 
> If I put the object directly to a region without using any group, the
> performance is getting better but still, IndexedDiskElementDescriptor
is
> the
> top one object consuming the heap. Why the
IndexedDiskElementDescriptor
> need
> to keep the serialized form of the cached object. Shouldn't the cached
> object suppose to be written out to the auxiliary disk and be removed
from
> the memory? Without using the grouping, the number of objects in JCS
is 6
> times more than using the grouping.
> 
> With this memory issue, it is really intimidating us from using JCS.
Any
> explanation or answer will be greatly appreciated.
> 
> Best regards,
> 
> Amy
> 
> 
> 
> 
> 
> 
>