You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jcs-users@jakarta.apache.org by Vladimir Olenin <VO...@cihi.ca> on 2006/08/26 00:42:32 UTC
configuration question
Hi.
I need to configure JCS in the following manner:
- need to load 100K entries into the cache
- NONE of the entries are allowed to be expired (ie, ALL must be
eternal)
- have to keep in memory not more than 1000 entries at a time
- the rest entries should be swapped to disk automatically
What is the default setting that would work the best? The current
configuration I use is below and it seem to work. I can't quite figure
out the meaning of 'MaxPurgatorySize' property. From the docs I
understood that it's just the size of the 'disk purge' queue. But the
tests show that if it's a positive number, then the entries start to be
'expired' from the cache (eg, if set to 1000 it seems like only a small
fraction of 100K entries remains in cache and some of the entries put in
the cache become missing for sure, since jcs.get returns null).
Thanks!.
===========================================
jcs.default = DC_SMALL_FOOTPRINT
jcs.default.cacheattributes =
org.apache.jcs.engine.CompositeCacheAttributes
jcs.default.cacheattributes.MaxObjects = 1000
jcs.default.cacheattributes.MemoryCacheName =
org.apache.jcs.engine.memory.lru.LRUMemoryCache
jcs.auxiliary.DC_SMALL_FOOTPRINT=org.apache.jcs.auxiliary.disk.indexed.I
ndexedDiskCacheFactory
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes=org.apache.jcs.auxiliary.dis
k.indexed.IndexedDiskCacheAttributes
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.DiskPath=c:/Projects/Grouper
/temp/jcs
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.maxKeySize=-1
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.MaxPurgatorySize=-1
RE: configuration question
Posted by Vladimir Olenin <VO...@cihi.ca>.
... Think I found it. There is a bug in the documentation
(http://jakarta.apache.org/jcs/RegionProperties.html)
Instead of 'DiskUsagePattern' the property name should read
'DiskUsagePatternName'. The first one does accept short number, while
the second one maps string value to the short number. So, the line in
config file should read:
jcs.default.cacheattributes.DiskUsagePatternName = UPDATE
Thanks. It seems to work now. I wonder if 'UPDATE' policy is set, does
'MaxPurgatorySize' property has any effect then? What's the lifecycle of
cache element with UPDATE policy? Does it go just between 'memory' and
'dc'? (if not found in memory, load from disk and put in memory cache,
expiring some entries from memory on the way if necessary)?
-----Original Message-----
From: Vladimir Olenin [mailto:VOlenin@cihi.ca]
Sent: Monday, August 28, 2006 9:15 AM
To: JCS Users List
Subject: RE: configuration question
Trying to do this as per docs, with line
jcs.default.cacheattributes.DiskUsagePattern = UPDATE
But getting this in the output log:
[WARN] Failed to set property diskUsagePattern to value "UPDATE".
Conversion to type [short] failed.
I think 'UPDATE' is a final short variable somewhere in the code.... But
there is no referral to it in the guides. What should it be?
-----Original Message-----
From: Aaron Smuts [mailto:asmuts@yahoo.com]
Sent: Friday, August 25, 2006 8:21 PM
To: JCS Users List
Subject: Re: configuration question
Purgatory is a queue on which items on the way to disk can still be
accessed. Setting a limit on purgatory allows you to prevent the queue
from getting too large. If you set a purgatory to 1000 and then quickly
add 100,000 items faster than they can be written to disk, then the
queue will certainly hit 1000. When the max purgatory size is reached
the oldest are dropped.
Purgatory is basically a bounded queue with a map on top. There is a
bit more description here:
http://jakarta.apache.org/jcs/IndexedDiskAuxCache.html
You should probably run with the cacheattributes properly called
DiskUsagePatternName=UPDATE. This will cause the items to go to disk
when they are added to the cache instead of when they fall out of the
Memory cache.
If you are going to flood the cache with items, then don't set a
purgatory limit, or set it to the number you will be adding if it is
known.
Aaron
--- Vladimir Olenin <VO...@cihi.ca> wrote:
> Hi.
>
> I need to configure JCS in the following manner:
>
> - need to load 100K entries into the cache
> - NONE of the entries are allowed to be expired (ie, ALL must be
> eternal)
> - have to keep in memory not more than 1000 entries at a time
> - the rest entries should be swapped to disk automatically
>
> What is the default setting that would work the best? The current
> configuration I use is below and it seem to work. I can't quite figure
> out the meaning of 'MaxPurgatorySize' property. From the docs I
> understood that it's just the size of the 'disk purge' queue. But the
> tests show that if it's a positive number, then the entries start to
> be 'expired' from the cache (eg, if set to 1000 it seems like only a
> small fraction of 100K entries remains in cache and some of the
> entries put in the cache become missing for sure, since jcs.get
> returns null).
>
> Thanks!.
>
> ===========================================
> jcs.default = DC_SMALL_FOOTPRINT
> jcs.default.cacheattributes =
> org.apache.jcs.engine.CompositeCacheAttributes
> jcs.default.cacheattributes.MaxObjects = 1000
> jcs.default.cacheattributes.MemoryCacheName =
> org.apache.jcs.engine.memory.lru.LRUMemoryCache
>
>
jcs.auxiliary.DC_SMALL_FOOTPRINT=org.apache.jcs.auxiliary.disk.indexed.I
> ndexedDiskCacheFactory
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes=org.apache.jcs.auxiliary.dis
> k.indexed.IndexedDiskCacheAttributes
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.DiskPath=c:/Projects/Grouper
> /temp/jcs
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.maxKeySize=-1
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.MaxPurgatorySize=-1
>
---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-users-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-users-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-users-help@jakarta.apache.org
RE: configuration question
Posted by Vladimir Olenin <VO...@cihi.ca>.
Hi, Aaron.
Browsed through the source code, ran some tests, still the entries are
NOT evicted only if Purgatory size is negative (ie, when HashMap is used
as purgatory implementation, NOT LRUMapJcs; ie purgatory is unbounded).
I have the following configuration now:
======
# DEFAULT CACHE REGION
jcs.default = DC
jcs.default.cacheattributes=org.apache.jcs.engine.CompositeCacheAttribut
es
jcs.default.cacheattributes.MaxObjects = 1000
jcs.default.cacheattributes.MemoryCacheName=org.apache.jcs.engine.memory
.lru.LRUMemoryCache
jcs.default.cacheattributes.DiskUsagePatternName = UPDATE
# AVAILABLE AUXILIARY CACHES
jcs.auxiliary.DC=org.apache.jcs.auxiliary.disk.indexed.IndexedDiskCacheF
actory
jcs.auxiliary.DC.attributes=org.apache.jcs.auxiliary.disk.indexed.Indexe
dDiskCacheAttributes
jcs.auxiliary.DC.attributes.DiskPath=./temp
jcs.auxiliary.DC.attributes.maxKeySize=-1
jcs.auxiliary.DC.attributes.MaxPurgatorySize=100
jcs.auxiliary.DC.attributes.EventQueueType=POOLED
jcs.auxiliary.DC.attributes.EventQueuePoolName=disk_cache_event_queue
# Disk Cache Event Queue Pool
thread_pool.disk_cache_event_queue.useBoundary=false
thread_pool.disk_cache_event_queue.maximumPoolSize=5
thread_pool.disk_cache_event_queue.minimumPoolSize=1
thread_pool.disk_cache_event_queue.keepAliveTime=5000
thread_pool.disk_cache_event_queue.startUpSize=1
thread_pool.disk_cache_event_queue.whenBlockedPolicy=WAIT
================
And with this one the entries are obviously evicted from purgatory
before they even have a chance to be written to disk (when I'm putting
the 'cache writing' thread to sleep for even 1ms after each put, ALL
entries are purged to disk as expected). Would have to give up on this
one I guess, unless you have some suggestions...
-----Original Message-----
From: Aaron Smuts [mailto:asmuts@yahoo.com]
Sent: Monday, August 28, 2006 6:03 PM
To: JCS Users List
Subject: RE: configuration question
Yes, you should be able to setup a queue that will block or try to run
in the current thread. Configure it to use a pooled type of queue. Set
that queue with a when blocked policy of BLOCK or RUN. Configure the
purgatory size to -1 or to higher than the max buffer size of the queue.
This way the queue will effectively manage the purgatory size. (This
does make the disk cache potentially blocking. I've never tried this
but it shoudl work.)
/** abort when queue is full and max threads is reached. */
public static final String POLICY_ABORT = "ABORT";
/** block when queue is full and max threads is reached. */
public static final String POLICY_BLOCK = "BLOCK";
/** run in current thread when queue is full and max threads is
reached. */
public static final String POLICY_RUN = "RUN";
/** wait when queue is full and max threads is reached. */
public static final String POLICY_WAIT = "WAIT";
/** discard oldest when queue is full and max threads is reached. */
public static final String POLICY_DISCARDOLDEST = "DISCARDOLDEST";
There are some details a the bottom of this page:
http://jakarta.apache.org/jcs/IndexedDiskAuxCache.html
Also, you can call getStats to see how many items are in purgatory etc.
[YOUR FILE]
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.EventQueueType=POOLED
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.EventQueuePoolName=disk_cach
e_event_queue
##############################################################
################## OPTIONAL THREAD POOL CONFIGURATION ########
# Disk Cache Event Queue Pool
thread_pool.disk_cache_event_queue.useBoundary=true
thread_pool.disk_cache_event_queue.boundarySize=50000
thread_pool.disk_cache_event_queue.maximumPoolSize=15
thread_pool.disk_cache_event_queue.minimumPoolSize=1
thread_pool.disk_cache_event_queue.keepAliveTime=3500
thread_pool.disk_cache_event_queue.startUpSize=1
thread_pool.disk_cache_event_queue.whenBlockedPolicy=RUN
--- Vladimir Olenin <VO...@cihi.ca> wrote:
> Still having problems with 'Purgatory' settings...
>
> Questions:
>
> 1) Is the purgatory object pool ever being 'shrinked' (when all data
> is already written to disk)
> 2) if so, does JCS do it automatically or there are some specific
> settings? (couldn't find anything in the docs)
> 3) Is there a way to 'ask' Purgatory to 'block' any subsequent 'put'
> requests until the current 'bounded buffer' (with
> MaxPurgatorySize) is
> dumped to the disk and space in the purgatory becomes available again
> (I realized it's quite against the rational why the Purgotory was
> introduced in the first place, but read on)
>
>
> The thing I need to do is this:
>
> 1) I need to put, lets say, 100K entries into the cache, and keep
> _ONLY_ 1K of them readily available in the memory. The rest should be
> dynamically load from the disk when required (replacing least
> frequently used entries in memory)
> 2) the cache is written into only ONCE, at the application startup
> 3) the cache is then accessed (read-only) extensively during the
> application life cycle
>
>
> The problem I'm having is this:
>
> 1) with current settings (see below) everything works as expected,
> with only one exception:
> 2) the memory consumption with 'unbounded' Purgatory size
> (MaxPurgatorySize=-1, correct?) is at least 150M higher than a bounded
> one (the values I tried ranged from 100 to 1000), even after all data
> is loaded into cache. In my case 100M higher means twice as much (300M
> vs
> 150M)
> 3) while it's unknown right now whether the memory would eventually be
> freed-up, I suspect it might not
> 4) with bounded purgatory (MaxPurgatorySize = 1000), some entries got
> evicted from the cache before they were written to the disk.
>
>
> Below are my current settings:
>
> #----------------------
> jcs.default = DC_SMALL_FOOTPRINT
> jcs.default.cacheattributes =
> org.apache.jcs.engine.CompositeCacheAttributes
> jcs.default.cacheattributes.MaxObjects = 1000
> jcs.default.cacheattributes.MemoryCacheName =
> org.apache.jcs.engine.memory.lru.LRUMemoryCache
> jcs.default.cacheattributes.DiskUsagePatternName = UPDATE
>
>
jcs.auxiliary.DC_SMALL_FOOTPRINT=org.apache.jcs.auxiliary.disk.indexed.I
> ndexedDiskCacheFactory
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes=org.apache.jcs.auxiliary.dis
> k.indexed.IndexedDiskCacheAttributes
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.maxKeySize=-1
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.MaxPurgatorySize=-1
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.DiskPath=jcs
> #----------------------
>
> Thanks!
>
> -----Original Message-----
> From: Aaron Smuts [mailto:asmuts@yahoo.com]
> Sent: Monday, August 28, 2006 9:33 AM
> To: JCS Users List
> Subject: RE: configuration question
>
>
>
> --- Vladimir Olenin <VO...@cihi.ca> wrote:
>
> > Thanks. It seems to work now. I wonder if 'UPDATE'
> > policy is set, does
> > 'MaxPurgatorySize' property has any effect then?
>
> Yes. In JCS mythology, purgatory is simply a queue on the way to
> disk, where the items can be accessed.
> It's a map on top of a queue. All items that go to disk, go to the
> queue. You can set a limit on the queue size regardless of the disk
> usage pattern.
>
>
>
> > What's the lifecycle of
> > cache element with UPDATE policy? Does it go just
> between 'memory' and
>
> > 'dc'? (if not found in memory, load from disk and
> put in memory cache,
>
> > expiring some entries from memory on the way if
> necessary)?
>
> When the item is put into the cache for the first time it goes to disk
> as well as memory (if the max memory size is greater than 0). If the
> item is later now in memory and is retrieved from disk, it will go
> back into memory, but it should will still be on disk.
> If the item is
> evicted from memory, because, let's say, it falls off the LRU, then it
> will NOT be put to disk
> again. This prevents thrashing. . . .
>
> (I want to make another pattern called PRIMARY where the items just go
> to disk until they are accessed yet again . . .)
>
> >
> > Thanks for quick answers, Aaron.
> >
> > Vlad
> >
> > -----Original Message-----
> > From: Aaron Smuts [mailto:asmuts@yahoo.com]
> > Sent: Monday, August 28, 2006 9:21 AM
> > To: JCS Users List
> > Subject: RE: configuration question
> >
> >
> >
> > --- Vladimir Olenin <VO...@cihi.ca> wrote:
> > Sorry, the docs are wrong. DiskUsagePattern is
> the property but you
> > have to set it with a "name". Use
> "DiskUsagePatternName=UPDATE"
> >
> >
> > > Trying to do this as per docs, with line
> > >
> > > jcs.default.cacheattributes.DiskUsagePattern =
> > UPDATE
> > >
> > > But getting this in the output log:
> > >
> > > [WARN] Failed to set property diskUsagePattern
> to
> > value "UPDATE".
> > > Conversion to type [short] failed.
> > >
> > >
> > > I think 'UPDATE' is a final short variable
> > somewhere in the code....
> > > But there is no referral to it in the guides.
> What
> > should it be?
> > >
> > > -----Original Message-----
> > > From: Aaron Smuts [mailto:asmuts@yahoo.com]
> > > Sent: Friday, August 25, 2006 8:21 PM
> > > To: JCS Users List
> > > Subject: Re: configuration question
> > >
> > > Purgatory is a queue on which items on the way
> to
> > disk can still be
> > > accessed. Setting a limit on purgatory allows
> you
> > to prevent the
> > > queue from getting too large. If you set a
> > purgatory to 1000 and then
> >
> > > quickly add 100,000 items faster than they can
> be
> > written to disk,
> > > then the queue will certainly hit 1000. When
> the
> > max purgatory size
> > > is reached the oldest are dropped.
> > >
> > > Purgatory is basically a bounded queue with a
> map
> > on top. There is a
> > > bit more description here:
>
=== message truncated ===
---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-users-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-users-help@jakarta.apache.org
RE: configuration question
Posted by Aaron Smuts <as...@yahoo.com>.
The pooled queue is a wrapper around the util
concurrent pooled executor. You can read more here:
http://gee.cs.oswego.edu/dl/classes/EDU/oswego/cs/dl/util/concurrent/PooledExecutor.html
Aaron
--- Vladimir Olenin <VO...@cihi.ca> wrote:
> This thread pool is a bit confusing.... What's the
> difference between
> 'WAIT' and 'BLOCK'? The default one I guess is
> 'POLICY_DISCARDOLDEST'...
> Will 'RUN' policy avoid discarding the entries?
> Where can I find a bit
> more detailed explanation on thread pool
> properties?... Started to
> browse through JCS source code, but I'm a bit short
> on time at this
> point, so some quick link would be very appriciated!
>
> Vlad
>
> -----Original Message-----
> From: Aaron Smuts [mailto:asmuts@yahoo.com]
> Sent: Monday, August 28, 2006 6:03 PM
> To: JCS Users List
> Subject: RE: configuration question
>
> Yes, you should be able to setup a queue that will
> block or try to run
> in the current thread. Configure it to use a pooled
> type of queue. Set
> that queue with a when blocked policy of BLOCK or
> RUN. Configure the
> purgatory size to -1 or to higher than the max
> buffer size of the queue.
> This way the queue will effectively manage the
> purgatory size. (This
> does make the disk cache potentially blocking. I've
> never tried this
> but it shoudl work.)
>
> /** abort when queue is full and max threads is
> reached. */
> public static final String POLICY_ABORT =
> "ABORT";
>
> /** block when queue is full and max threads is
> reached. */
> public static final String POLICY_BLOCK =
> "BLOCK";
>
> /** run in current thread when queue is full and
> max threads is
> reached. */
> public static final String POLICY_RUN = "RUN";
>
> /** wait when queue is full and max threads is
> reached. */
> public static final String POLICY_WAIT = "WAIT";
>
> /** discard oldest when queue is full and max
> threads is reached. */
> public static final String POLICY_DISCARDOLDEST
> = "DISCARDOLDEST";
>
> There are some details a the bottom of this page:
>
http://jakarta.apache.org/jcs/IndexedDiskAuxCache.html
>
> Also, you can call getStats to see how many items
> are in purgatory etc.
>
>
> [YOUR FILE]
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.EventQueueType=POOLED
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.EventQueuePoolName=disk_cach
> e_event_queue
>
>
##############################################################
> ################## OPTIONAL THREAD POOL
> CONFIGURATION ########
>
> # Disk Cache Event Queue Pool
> thread_pool.disk_cache_event_queue.useBoundary=true
>
thread_pool.disk_cache_event_queue.boundarySize=50000
>
thread_pool.disk_cache_event_queue.maximumPoolSize=15
> thread_pool.disk_cache_event_queue.minimumPoolSize=1
>
thread_pool.disk_cache_event_queue.keepAliveTime=3500
> thread_pool.disk_cache_event_queue.startUpSize=1
>
thread_pool.disk_cache_event_queue.whenBlockedPolicy=RUN
>
>
>
> --- Vladimir Olenin <VO...@cihi.ca> wrote:
>
> > Still having problems with 'Purgatory' settings...
> >
> > Questions:
> >
> > 1) Is the purgatory object pool ever being
> 'shrinked' (when all data
> > is already written to disk)
> > 2) if so, does JCS do it automatically or there
> are some specific
> > settings? (couldn't find anything in the docs)
> > 3) Is there a way to 'ask' Purgatory to 'block'
> any subsequent 'put'
> > requests until the current 'bounded buffer' (with
> > MaxPurgatorySize) is
> > dumped to the disk and space in the purgatory
> becomes available again
> > (I realized it's quite against the rational why
> the Purgotory was
> > introduced in the first place, but read on)
> >
> >
> > The thing I need to do is this:
> >
> > 1) I need to put, lets say, 100K entries into the
> cache, and keep
> > _ONLY_ 1K of them readily available in the memory.
> The rest should be
> > dynamically load from the disk when required
> (replacing least
> > frequently used entries in memory)
> > 2) the cache is written into only ONCE, at the
> application startup
> > 3) the cache is then accessed (read-only)
> extensively during the
> > application life cycle
> >
> >
> > The problem I'm having is this:
> >
> > 1) with current settings (see below) everything
> works as expected,
> > with only one exception:
> > 2) the memory consumption with 'unbounded'
> Purgatory size
> > (MaxPurgatorySize=-1, correct?) is at least 150M
> higher than a bounded
>
> > one (the values I tried ranged from 100 to 1000),
> even after all data
> > is loaded into cache. In my case 100M higher means
> twice as much (300M
>
> > vs
> > 150M)
> > 3) while it's unknown right now whether the memory
> would eventually be
>
> > freed-up, I suspect it might not
> > 4) with bounded purgatory (MaxPurgatorySize =
> 1000), some entries got
> > evicted from the cache before they were written to
> the disk.
> >
> >
> > Below are my current settings:
> >
> > #----------------------
> > jcs.default = DC_SMALL_FOOTPRINT
> > jcs.default.cacheattributes =
> > org.apache.jcs.engine.CompositeCacheAttributes
> > jcs.default.cacheattributes.MaxObjects = 1000
> > jcs.default.cacheattributes.MemoryCacheName =
> > org.apache.jcs.engine.memory.lru.LRUMemoryCache
> > jcs.default.cacheattributes.DiskUsagePatternName =
> UPDATE
> >
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT=org.apache.jcs.auxiliary.disk.indexed.I
> > ndexedDiskCacheFactory
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes=org.apache.jcs.auxiliary.dis
> > k.indexed.IndexedDiskCacheAttributes
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.maxKeySize=-1
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.MaxPurgatorySize=-1
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.DiskPath=jcs
> > #----------------------
> >
> > Thanks!
> >
> > -----Original Message-----
> > From: Aaron Smuts [mailto:asmuts@yahoo.com]
> > Sent: Monday, August 28, 2006 9:33 AM
> > To: JCS Users List
> > Subject: RE: configuration question
> >
> >
> >
> > --- Vladimir Olenin <VO...@cihi.ca> wrote:
> >
> > > Thanks. It seems to work now. I wonder if
> 'UPDATE'
> > > policy is set, does
> > > 'MaxPurgatorySize' property has any effect then?
> >
> > Yes. In JCS mythology, purgatory is simply a
> queue on the way to
> > disk, where the items can be accessed.
> > It's a map on top of a queue. All items that go
> to disk, go to the
> > queue. You can set a limit on the queue size
> regardless of the disk
>
=== message truncated ===
---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-users-help@jakarta.apache.org
RE: configuration question
Posted by Vladimir Olenin <VO...@cihi.ca>.
This thread pool is a bit confusing.... What's the difference between
'WAIT' and 'BLOCK'? The default one I guess is 'POLICY_DISCARDOLDEST'...
Will 'RUN' policy avoid discarding the entries? Where can I find a bit
more detailed explanation on thread pool properties?... Started to
browse through JCS source code, but I'm a bit short on time at this
point, so some quick link would be very appriciated!
Vlad
-----Original Message-----
From: Aaron Smuts [mailto:asmuts@yahoo.com]
Sent: Monday, August 28, 2006 6:03 PM
To: JCS Users List
Subject: RE: configuration question
Yes, you should be able to setup a queue that will block or try to run
in the current thread. Configure it to use a pooled type of queue. Set
that queue with a when blocked policy of BLOCK or RUN. Configure the
purgatory size to -1 or to higher than the max buffer size of the queue.
This way the queue will effectively manage the purgatory size. (This
does make the disk cache potentially blocking. I've never tried this
but it shoudl work.)
/** abort when queue is full and max threads is reached. */
public static final String POLICY_ABORT = "ABORT";
/** block when queue is full and max threads is reached. */
public static final String POLICY_BLOCK = "BLOCK";
/** run in current thread when queue is full and max threads is
reached. */
public static final String POLICY_RUN = "RUN";
/** wait when queue is full and max threads is reached. */
public static final String POLICY_WAIT = "WAIT";
/** discard oldest when queue is full and max threads is reached. */
public static final String POLICY_DISCARDOLDEST = "DISCARDOLDEST";
There are some details a the bottom of this page:
http://jakarta.apache.org/jcs/IndexedDiskAuxCache.html
Also, you can call getStats to see how many items are in purgatory etc.
[YOUR FILE]
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.EventQueueType=POOLED
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.EventQueuePoolName=disk_cach
e_event_queue
##############################################################
################## OPTIONAL THREAD POOL CONFIGURATION ########
# Disk Cache Event Queue Pool
thread_pool.disk_cache_event_queue.useBoundary=true
thread_pool.disk_cache_event_queue.boundarySize=50000
thread_pool.disk_cache_event_queue.maximumPoolSize=15
thread_pool.disk_cache_event_queue.minimumPoolSize=1
thread_pool.disk_cache_event_queue.keepAliveTime=3500
thread_pool.disk_cache_event_queue.startUpSize=1
thread_pool.disk_cache_event_queue.whenBlockedPolicy=RUN
--- Vladimir Olenin <VO...@cihi.ca> wrote:
> Still having problems with 'Purgatory' settings...
>
> Questions:
>
> 1) Is the purgatory object pool ever being 'shrinked' (when all data
> is already written to disk)
> 2) if so, does JCS do it automatically or there are some specific
> settings? (couldn't find anything in the docs)
> 3) Is there a way to 'ask' Purgatory to 'block' any subsequent 'put'
> requests until the current 'bounded buffer' (with
> MaxPurgatorySize) is
> dumped to the disk and space in the purgatory becomes available again
> (I realized it's quite against the rational why the Purgotory was
> introduced in the first place, but read on)
>
>
> The thing I need to do is this:
>
> 1) I need to put, lets say, 100K entries into the cache, and keep
> _ONLY_ 1K of them readily available in the memory. The rest should be
> dynamically load from the disk when required (replacing least
> frequently used entries in memory)
> 2) the cache is written into only ONCE, at the application startup
> 3) the cache is then accessed (read-only) extensively during the
> application life cycle
>
>
> The problem I'm having is this:
>
> 1) with current settings (see below) everything works as expected,
> with only one exception:
> 2) the memory consumption with 'unbounded' Purgatory size
> (MaxPurgatorySize=-1, correct?) is at least 150M higher than a bounded
> one (the values I tried ranged from 100 to 1000), even after all data
> is loaded into cache. In my case 100M higher means twice as much (300M
> vs
> 150M)
> 3) while it's unknown right now whether the memory would eventually be
> freed-up, I suspect it might not
> 4) with bounded purgatory (MaxPurgatorySize = 1000), some entries got
> evicted from the cache before they were written to the disk.
>
>
> Below are my current settings:
>
> #----------------------
> jcs.default = DC_SMALL_FOOTPRINT
> jcs.default.cacheattributes =
> org.apache.jcs.engine.CompositeCacheAttributes
> jcs.default.cacheattributes.MaxObjects = 1000
> jcs.default.cacheattributes.MemoryCacheName =
> org.apache.jcs.engine.memory.lru.LRUMemoryCache
> jcs.default.cacheattributes.DiskUsagePatternName = UPDATE
>
>
jcs.auxiliary.DC_SMALL_FOOTPRINT=org.apache.jcs.auxiliary.disk.indexed.I
> ndexedDiskCacheFactory
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes=org.apache.jcs.auxiliary.dis
> k.indexed.IndexedDiskCacheAttributes
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.maxKeySize=-1
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.MaxPurgatorySize=-1
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.DiskPath=jcs
> #----------------------
>
> Thanks!
>
> -----Original Message-----
> From: Aaron Smuts [mailto:asmuts@yahoo.com]
> Sent: Monday, August 28, 2006 9:33 AM
> To: JCS Users List
> Subject: RE: configuration question
>
>
>
> --- Vladimir Olenin <VO...@cihi.ca> wrote:
>
> > Thanks. It seems to work now. I wonder if 'UPDATE'
> > policy is set, does
> > 'MaxPurgatorySize' property has any effect then?
>
> Yes. In JCS mythology, purgatory is simply a queue on the way to
> disk, where the items can be accessed.
> It's a map on top of a queue. All items that go to disk, go to the
> queue. You can set a limit on the queue size regardless of the disk
> usage pattern.
>
>
>
> > What's the lifecycle of
> > cache element with UPDATE policy? Does it go just
> between 'memory' and
>
> > 'dc'? (if not found in memory, load from disk and
> put in memory cache,
>
> > expiring some entries from memory on the way if
> necessary)?
>
> When the item is put into the cache for the first time it goes to disk
> as well as memory (if the max memory size is greater than 0). If the
> item is later now in memory and is retrieved from disk, it will go
> back into memory, but it should will still be on disk.
> If the item is
> evicted from memory, because, let's say, it falls off the LRU, then it
> will NOT be put to disk
> again. This prevents thrashing. . . .
>
> (I want to make another pattern called PRIMARY where the items just go
> to disk until they are accessed yet again . . .)
>
> >
> > Thanks for quick answers, Aaron.
> >
> > Vlad
> >
> > -----Original Message-----
> > From: Aaron Smuts [mailto:asmuts@yahoo.com]
> > Sent: Monday, August 28, 2006 9:21 AM
> > To: JCS Users List
> > Subject: RE: configuration question
> >
> >
> >
> > --- Vladimir Olenin <VO...@cihi.ca> wrote:
> > Sorry, the docs are wrong. DiskUsagePattern is
> the property but you
> > have to set it with a "name". Use
> "DiskUsagePatternName=UPDATE"
> >
> >
> > > Trying to do this as per docs, with line
> > >
> > > jcs.default.cacheattributes.DiskUsagePattern =
> > UPDATE
> > >
> > > But getting this in the output log:
> > >
> > > [WARN] Failed to set property diskUsagePattern
> to
> > value "UPDATE".
> > > Conversion to type [short] failed.
> > >
> > >
> > > I think 'UPDATE' is a final short variable
> > somewhere in the code....
> > > But there is no referral to it in the guides.
> What
> > should it be?
> > >
> > > -----Original Message-----
> > > From: Aaron Smuts [mailto:asmuts@yahoo.com]
> > > Sent: Friday, August 25, 2006 8:21 PM
> > > To: JCS Users List
> > > Subject: Re: configuration question
> > >
> > > Purgatory is a queue on which items on the way
> to
> > disk can still be
> > > accessed. Setting a limit on purgatory allows
> you
> > to prevent the
> > > queue from getting too large. If you set a
> > purgatory to 1000 and then
> >
> > > quickly add 100,000 items faster than they can
> be
> > written to disk,
> > > then the queue will certainly hit 1000. When
> the
> > max purgatory size
> > > is reached the oldest are dropped.
> > >
> > > Purgatory is basically a bounded queue with a
> map
> > on top. There is a
> > > bit more description here:
>
=== message truncated ===
---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-users-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-users-help@jakarta.apache.org
RE: configuration question
Posted by Aaron Smuts <as...@yahoo.com>.
Yes, you should be able to setup a queue that will
block or try to run in the current thread. Configure
it to use a pooled type of queue. Set that queue with
a when blocked policy of BLOCK or RUN. Configure the
purgatory size to -1 or to higher than the max buffer
size of the queue. This way the queue will
effectively manage the purgatory size. (This does
make the disk cache potentially blocking. I've never
tried this but it shoudl work.)
/** abort when queue is full and max threads is
reached. */
public static final String POLICY_ABORT = "ABORT";
/** block when queue is full and max threads is
reached. */
public static final String POLICY_BLOCK = "BLOCK";
/** run in current thread when queue is full and
max threads is reached. */
public static final String POLICY_RUN = "RUN";
/** wait when queue is full and max threads is
reached. */
public static final String POLICY_WAIT = "WAIT";
/** discard oldest when queue is full and max
threads is reached. */
public static final String POLICY_DISCARDOLDEST =
"DISCARDOLDEST";
There are some details a the bottom of this page:
http://jakarta.apache.org/jcs/IndexedDiskAuxCache.html
Also, you can call getStats to see how many items are
in purgatory etc.
[YOUR FILE]
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.EventQueueType=POOLED
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.EventQueuePoolName=disk_cache_event_queue
##############################################################
################## OPTIONAL THREAD POOL CONFIGURATION
########
# Disk Cache Event Queue Pool
thread_pool.disk_cache_event_queue.useBoundary=true
thread_pool.disk_cache_event_queue.boundarySize=50000
thread_pool.disk_cache_event_queue.maximumPoolSize=15
thread_pool.disk_cache_event_queue.minimumPoolSize=1
thread_pool.disk_cache_event_queue.keepAliveTime=3500
thread_pool.disk_cache_event_queue.startUpSize=1
thread_pool.disk_cache_event_queue.whenBlockedPolicy=RUN
--- Vladimir Olenin <VO...@cihi.ca> wrote:
> Still having problems with 'Purgatory' settings...
>
> Questions:
>
> 1) Is the purgatory object pool ever being
> 'shrinked' (when all data is
> already written to disk)
> 2) if so, does JCS do it automatically or there are
> some specific
> settings? (couldn't find anything in the docs)
> 3) Is there a way to 'ask' Purgatory to 'block' any
> subsequent 'put'
> requests until the current 'bounded buffer' (with
> MaxPurgatorySize) is
> dumped to the disk and space in the purgatory
> becomes available again (I
> realized it's quite against the rational why the
> Purgotory was
> introduced in the first place, but read on)
>
>
> The thing I need to do is this:
>
> 1) I need to put, lets say, 100K entries into the
> cache, and keep _ONLY_
> 1K of them readily available in the memory. The rest
> should be
> dynamically load from the disk when required
> (replacing least frequently
> used entries in memory)
> 2) the cache is written into only ONCE, at the
> application startup
> 3) the cache is then accessed (read-only)
> extensively during the
> application life cycle
>
>
> The problem I'm having is this:
>
> 1) with current settings (see below) everything
> works as expected, with
> only one exception:
> 2) the memory consumption with 'unbounded' Purgatory
> size
> (MaxPurgatorySize=-1, correct?) is at least 150M
> higher than a bounded
> one (the values I tried ranged from 100 to 1000),
> even after all data is
> loaded into cache. In my case 100M higher means
> twice as much (300M vs
> 150M)
> 3) while it's unknown right now whether the memory
> would eventually be
> freed-up, I suspect it might not
> 4) with bounded purgatory (MaxPurgatorySize = 1000),
> some entries got
> evicted from the cache before they were written to
> the disk.
>
>
> Below are my current settings:
>
> #----------------------
> jcs.default = DC_SMALL_FOOTPRINT
> jcs.default.cacheattributes =
> org.apache.jcs.engine.CompositeCacheAttributes
> jcs.default.cacheattributes.MaxObjects = 1000
> jcs.default.cacheattributes.MemoryCacheName =
> org.apache.jcs.engine.memory.lru.LRUMemoryCache
> jcs.default.cacheattributes.DiskUsagePatternName =
> UPDATE
>
>
jcs.auxiliary.DC_SMALL_FOOTPRINT=org.apache.jcs.auxiliary.disk.indexed.I
> ndexedDiskCacheFactory
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes=org.apache.jcs.auxiliary.dis
> k.indexed.IndexedDiskCacheAttributes
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.maxKeySize=-1
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.MaxPurgatorySize=-1
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.DiskPath=jcs
> #----------------------
>
> Thanks!
>
> -----Original Message-----
> From: Aaron Smuts [mailto:asmuts@yahoo.com]
> Sent: Monday, August 28, 2006 9:33 AM
> To: JCS Users List
> Subject: RE: configuration question
>
>
>
> --- Vladimir Olenin <VO...@cihi.ca> wrote:
>
> > Thanks. It seems to work now. I wonder if 'UPDATE'
> > policy is set, does
> > 'MaxPurgatorySize' property has any effect then?
>
> Yes. In JCS mythology, purgatory is simply a queue
> on the way to disk,
> where the items can be accessed.
> It's a map on top of a queue. All items that go to
> disk, go to the
> queue. You can set a limit on the queue size
> regardless of the disk
> usage pattern.
>
>
>
> > What's the lifecycle of
> > cache element with UPDATE policy? Does it go just
> between 'memory' and
>
> > 'dc'? (if not found in memory, load from disk and
> put in memory cache,
>
> > expiring some entries from memory on the way if
> necessary)?
>
> When the item is put into the cache for the first
> time it goes to disk
> as well as memory (if the max memory size is greater
> than 0). If the
> item is later now in memory and is retrieved from
> disk, it will go back
> into memory, but it should will still be on disk.
> If the item is
> evicted from memory, because, let's say, it falls
> off the LRU, then it
> will NOT be put to disk
> again. This prevents thrashing. . . .
>
> (I want to make another pattern called PRIMARY where
> the items just go
> to disk until they are accessed yet again . . .)
>
> >
> > Thanks for quick answers, Aaron.
> >
> > Vlad
> >
> > -----Original Message-----
> > From: Aaron Smuts [mailto:asmuts@yahoo.com]
> > Sent: Monday, August 28, 2006 9:21 AM
> > To: JCS Users List
> > Subject: RE: configuration question
> >
> >
> >
> > --- Vladimir Olenin <VO...@cihi.ca> wrote:
> > Sorry, the docs are wrong. DiskUsagePattern is
> the property but you
> > have to set it with a "name". Use
> "DiskUsagePatternName=UPDATE"
> >
> >
> > > Trying to do this as per docs, with line
> > >
> > > jcs.default.cacheattributes.DiskUsagePattern =
> > UPDATE
> > >
> > > But getting this in the output log:
> > >
> > > [WARN] Failed to set property diskUsagePattern
> to
> > value "UPDATE".
> > > Conversion to type [short] failed.
> > >
> > >
> > > I think 'UPDATE' is a final short variable
> > somewhere in the code....
> > > But there is no referral to it in the guides.
> What
> > should it be?
> > >
> > > -----Original Message-----
> > > From: Aaron Smuts [mailto:asmuts@yahoo.com]
> > > Sent: Friday, August 25, 2006 8:21 PM
> > > To: JCS Users List
> > > Subject: Re: configuration question
> > >
> > > Purgatory is a queue on which items on the way
> to
> > disk can still be
> > > accessed. Setting a limit on purgatory allows
> you
> > to prevent the
> > > queue from getting too large. If you set a
> > purgatory to 1000 and then
> >
> > > quickly add 100,000 items faster than they can
> be
> > written to disk,
> > > then the queue will certainly hit 1000. When
> the
> > max purgatory size
> > > is reached the oldest are dropped.
> > >
> > > Purgatory is basically a bounded queue with a
> map
> > on top. There is a
> > > bit more description here:
>
=== message truncated ===
---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-users-help@jakarta.apache.org
RE: configuration question
Posted by Vladimir Olenin <VO...@cihi.ca>.
Still having problems with 'Purgatory' settings...
Questions:
1) Is the purgatory object pool ever being 'shrinked' (when all data is
already written to disk)
2) if so, does JCS do it automatically or there are some specific
settings? (couldn't find anything in the docs)
3) Is there a way to 'ask' Purgatory to 'block' any subsequent 'put'
requests until the current 'bounded buffer' (with MaxPurgatorySize) is
dumped to the disk and space in the purgatory becomes available again (I
realized it's quite against the rational why the Purgotory was
introduced in the first place, but read on)
The thing I need to do is this:
1) I need to put, lets say, 100K entries into the cache, and keep _ONLY_
1K of them readily available in the memory. The rest should be
dynamically load from the disk when required (replacing least frequently
used entries in memory)
2) the cache is written into only ONCE, at the application startup
3) the cache is then accessed (read-only) extensively during the
application life cycle
The problem I'm having is this:
1) with current settings (see below) everything works as expected, with
only one exception:
2) the memory consumption with 'unbounded' Purgatory size
(MaxPurgatorySize=-1, correct?) is at least 150M higher than a bounded
one (the values I tried ranged from 100 to 1000), even after all data is
loaded into cache. In my case 100M higher means twice as much (300M vs
150M)
3) while it's unknown right now whether the memory would eventually be
freed-up, I suspect it might not
4) with bounded purgatory (MaxPurgatorySize = 1000), some entries got
evicted from the cache before they were written to the disk.
Below are my current settings:
#----------------------
jcs.default = DC_SMALL_FOOTPRINT
jcs.default.cacheattributes =
org.apache.jcs.engine.CompositeCacheAttributes
jcs.default.cacheattributes.MaxObjects = 1000
jcs.default.cacheattributes.MemoryCacheName =
org.apache.jcs.engine.memory.lru.LRUMemoryCache
jcs.default.cacheattributes.DiskUsagePatternName = UPDATE
jcs.auxiliary.DC_SMALL_FOOTPRINT=org.apache.jcs.auxiliary.disk.indexed.I
ndexedDiskCacheFactory
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes=org.apache.jcs.auxiliary.dis
k.indexed.IndexedDiskCacheAttributes
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.maxKeySize=-1
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.MaxPurgatorySize=-1
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.DiskPath=jcs
#----------------------
Thanks!
-----Original Message-----
From: Aaron Smuts [mailto:asmuts@yahoo.com]
Sent: Monday, August 28, 2006 9:33 AM
To: JCS Users List
Subject: RE: configuration question
--- Vladimir Olenin <VO...@cihi.ca> wrote:
> Thanks. It seems to work now. I wonder if 'UPDATE'
> policy is set, does
> 'MaxPurgatorySize' property has any effect then?
Yes. In JCS mythology, purgatory is simply a queue on the way to disk,
where the items can be accessed.
It's a map on top of a queue. All items that go to disk, go to the
queue. You can set a limit on the queue size regardless of the disk
usage pattern.
> What's the lifecycle of
> cache element with UPDATE policy? Does it go just between 'memory' and
> 'dc'? (if not found in memory, load from disk and put in memory cache,
> expiring some entries from memory on the way if necessary)?
When the item is put into the cache for the first time it goes to disk
as well as memory (if the max memory size is greater than 0). If the
item is later now in memory and is retrieved from disk, it will go back
into memory, but it should will still be on disk. If the item is
evicted from memory, because, let's say, it falls off the LRU, then it
will NOT be put to disk
again. This prevents thrashing. . . .
(I want to make another pattern called PRIMARY where the items just go
to disk until they are accessed yet again . . .)
>
> Thanks for quick answers, Aaron.
>
> Vlad
>
> -----Original Message-----
> From: Aaron Smuts [mailto:asmuts@yahoo.com]
> Sent: Monday, August 28, 2006 9:21 AM
> To: JCS Users List
> Subject: RE: configuration question
>
>
>
> --- Vladimir Olenin <VO...@cihi.ca> wrote:
> Sorry, the docs are wrong. DiskUsagePattern is the property but you
> have to set it with a "name". Use "DiskUsagePatternName=UPDATE"
>
>
> > Trying to do this as per docs, with line
> >
> > jcs.default.cacheattributes.DiskUsagePattern =
> UPDATE
> >
> > But getting this in the output log:
> >
> > [WARN] Failed to set property diskUsagePattern to
> value "UPDATE".
> > Conversion to type [short] failed.
> >
> >
> > I think 'UPDATE' is a final short variable
> somewhere in the code....
> > But there is no referral to it in the guides. What
> should it be?
> >
> > -----Original Message-----
> > From: Aaron Smuts [mailto:asmuts@yahoo.com]
> > Sent: Friday, August 25, 2006 8:21 PM
> > To: JCS Users List
> > Subject: Re: configuration question
> >
> > Purgatory is a queue on which items on the way to
> disk can still be
> > accessed. Setting a limit on purgatory allows you
> to prevent the
> > queue from getting too large. If you set a
> purgatory to 1000 and then
>
> > quickly add 100,000 items faster than they can be
> written to disk,
> > then the queue will certainly hit 1000. When the
> max purgatory size
> > is reached the oldest are dropped.
> >
> > Purgatory is basically a bounded queue with a map
> on top. There is a
> > bit more description here:
> >
> >
>
http://jakarta.apache.org/jcs/IndexedDiskAuxCache.html
> >
> > You should probably run with the cacheattributes
> properly called
> > DiskUsagePatternName=UPDATE. This will cause the
> items to go to disk
> > when they are added to the cache instead of when
> they fall out of the
> > Memory cache.
> >
> > If you are going to flood the cache with items,
> then don't set a
> > purgatory limit, or set it to the number you will
> be adding if it is
> > known.
> >
> > Aaron
> >
> > --- Vladimir Olenin <VO...@cihi.ca> wrote:
> >
> > > Hi.
> > >
> > > I need to configure JCS in the following manner:
> > >
> > > - need to load 100K entries into the cache
> > > - NONE of the entries are allowed to be expired
> > (ie, ALL must be
> > > eternal)
> > > - have to keep in memory not more than 1000
> > entries at a time
> > > - the rest entries should be swapped to disk
> > automatically
> > >
> > > What is the default setting that would work the
> > best? The current
> > > configuration I use is below and it seem to
> work.
> > I can't quite figure
> >
> > > out the meaning of 'MaxPurgatorySize' property.
> > From the docs I
> > > understood that it's just the size of the 'disk
> > purge' queue. But the
> > > tests show that if it's a positive number, then
> > the entries start to
> > > be 'expired' from the cache (eg, if set to 1000
> it
> > seems like only a
> > > small fraction of 100K entries remains in cache
> > and some of the
> > > entries put in the cache become missing for
> sure,
> > since jcs.get
> > > returns null).
> > >
> > > Thanks!.
> > >
> > > ===========================================
> > > jcs.default = DC_SMALL_FOOTPRINT
> > > jcs.default.cacheattributes =
> > > org.apache.jcs.engine.CompositeCacheAttributes
> > > jcs.default.cacheattributes.MaxObjects = 1000
> > > jcs.default.cacheattributes.MemoryCacheName =
> > > org.apache.jcs.engine.memory.lru.LRUMemoryCache
> > >
> > >
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT=org.apache.jcs.auxiliary.disk.indexed.I
> > > ndexedDiskCacheFactory
> > >
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes=org.apache.jcs.auxiliary.dis
> > > k.indexed.IndexedDiskCacheAttributes
> > >
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.DiskPath=c:/Projects/Grouper
> > > /temp/jcs
> > >
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.maxKeySize=-1
> > >
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.MaxPurgatorySize=-1
> > >
> >
> >
> >
>
---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> > jcs-users-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail:
> > jcs-users-help@jakarta.apache.org
> >
> >
> >
>
---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> > jcs-users-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail:
> > jcs-users-help@jakarta.apache.org
> >
> >
>
>
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> jcs-users-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> jcs-users-help@jakarta.apache.org
>
>
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> jcs-users-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> jcs-users-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-users-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-users-help@jakarta.apache.org
RE: configuration question
Posted by Vladimir Olenin <VO...@cihi.ca>.
Ok. Seems clear now. Got a bit confused, since in my case the usage
pattern is just a DC backed hash map, which is initialized (written
into) only once, on the app startup, and later on accessed (read from)
numerous times from the app. The amount of data need to be cached is
significant and does not fit into JVM comfortably, hence the need for
DC.
Will play with configurations now. So far the runtime results seem
encouraging.
-----Original Message-----
From: Aaron Smuts [mailto:asmuts@yahoo.com]
Sent: Monday, August 28, 2006 9:33 AM
To: JCS Users List
Subject: RE: configuration question
--- Vladimir Olenin <VO...@cihi.ca> wrote:
> Thanks. It seems to work now. I wonder if 'UPDATE'
> policy is set, does
> 'MaxPurgatorySize' property has any effect then?
Yes. In JCS mythology, purgatory is simply a queue on the way to disk,
where the items can be accessed.
It's a map on top of a queue. All items that go to disk, go to the
queue. You can set a limit on the queue size regardless of the disk
usage pattern.
> What's the lifecycle of
> cache element with UPDATE policy? Does it go just between 'memory' and
> 'dc'? (if not found in memory, load from disk and put in memory cache,
> expiring some entries from memory on the way if necessary)?
When the item is put into the cache for the first time it goes to disk
as well as memory (if the max memory size is greater than 0). If the
item is later now in memory and is retrieved from disk, it will go back
into memory, but it should will still be on disk. If the item is
evicted from memory, because, let's say, it falls off the LRU, then it
will NOT be put to disk
again. This prevents thrashing. . . .
(I want to make another pattern called PRIMARY where the items just go
to disk until they are accessed yet again . . .)
>
> Thanks for quick answers, Aaron.
>
> Vlad
>
> -----Original Message-----
> From: Aaron Smuts [mailto:asmuts@yahoo.com]
> Sent: Monday, August 28, 2006 9:21 AM
> To: JCS Users List
> Subject: RE: configuration question
>
>
>
> --- Vladimir Olenin <VO...@cihi.ca> wrote:
> Sorry, the docs are wrong. DiskUsagePattern is the property but you
> have to set it with a "name". Use "DiskUsagePatternName=UPDATE"
>
>
> > Trying to do this as per docs, with line
> >
> > jcs.default.cacheattributes.DiskUsagePattern =
> UPDATE
> >
> > But getting this in the output log:
> >
> > [WARN] Failed to set property diskUsagePattern to
> value "UPDATE".
> > Conversion to type [short] failed.
> >
> >
> > I think 'UPDATE' is a final short variable
> somewhere in the code....
> > But there is no referral to it in the guides. What
> should it be?
> >
> > -----Original Message-----
> > From: Aaron Smuts [mailto:asmuts@yahoo.com]
> > Sent: Friday, August 25, 2006 8:21 PM
> > To: JCS Users List
> > Subject: Re: configuration question
> >
> > Purgatory is a queue on which items on the way to
> disk can still be
> > accessed. Setting a limit on purgatory allows you
> to prevent the
> > queue from getting too large. If you set a
> purgatory to 1000 and then
>
> > quickly add 100,000 items faster than they can be
> written to disk,
> > then the queue will certainly hit 1000. When the
> max purgatory size
> > is reached the oldest are dropped.
> >
> > Purgatory is basically a bounded queue with a map
> on top. There is a
> > bit more description here:
> >
> >
>
http://jakarta.apache.org/jcs/IndexedDiskAuxCache.html
> >
> > You should probably run with the cacheattributes
> properly called
> > DiskUsagePatternName=UPDATE. This will cause the
> items to go to disk
> > when they are added to the cache instead of when
> they fall out of the
> > Memory cache.
> >
> > If you are going to flood the cache with items,
> then don't set a
> > purgatory limit, or set it to the number you will
> be adding if it is
> > known.
> >
> > Aaron
> >
> > --- Vladimir Olenin <VO...@cihi.ca> wrote:
> >
> > > Hi.
> > >
> > > I need to configure JCS in the following manner:
> > >
> > > - need to load 100K entries into the cache
> > > - NONE of the entries are allowed to be expired
> > (ie, ALL must be
> > > eternal)
> > > - have to keep in memory not more than 1000
> > entries at a time
> > > - the rest entries should be swapped to disk
> > automatically
> > >
> > > What is the default setting that would work the
> > best? The current
> > > configuration I use is below and it seem to
> work.
> > I can't quite figure
> >
> > > out the meaning of 'MaxPurgatorySize' property.
> > From the docs I
> > > understood that it's just the size of the 'disk
> > purge' queue. But the
> > > tests show that if it's a positive number, then
> > the entries start to
> > > be 'expired' from the cache (eg, if set to 1000
> it
> > seems like only a
> > > small fraction of 100K entries remains in cache
> > and some of the
> > > entries put in the cache become missing for
> sure,
> > since jcs.get
> > > returns null).
> > >
> > > Thanks!.
> > >
> > > ===========================================
> > > jcs.default = DC_SMALL_FOOTPRINT
> > > jcs.default.cacheattributes =
> > > org.apache.jcs.engine.CompositeCacheAttributes
> > > jcs.default.cacheattributes.MaxObjects = 1000
> > > jcs.default.cacheattributes.MemoryCacheName =
> > > org.apache.jcs.engine.memory.lru.LRUMemoryCache
> > >
> > >
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT=org.apache.jcs.auxiliary.disk.indexed.I
> > > ndexedDiskCacheFactory
> > >
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes=org.apache.jcs.auxiliary.dis
> > > k.indexed.IndexedDiskCacheAttributes
> > >
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.DiskPath=c:/Projects/Grouper
> > > /temp/jcs
> > >
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.maxKeySize=-1
> > >
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.MaxPurgatorySize=-1
> > >
> >
> >
> >
>
---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> > jcs-users-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail:
> > jcs-users-help@jakarta.apache.org
> >
> >
> >
>
---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> > jcs-users-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail:
> > jcs-users-help@jakarta.apache.org
> >
> >
>
>
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> jcs-users-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> jcs-users-help@jakarta.apache.org
>
>
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> jcs-users-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> jcs-users-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-users-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-users-help@jakarta.apache.org
RE: configuration question
Posted by Aaron Smuts <as...@yahoo.com>.
--- Vladimir Olenin <VO...@cihi.ca> wrote:
> Thanks. It seems to work now. I wonder if 'UPDATE'
> policy is set, does
> 'MaxPurgatorySize' property has any effect then?
Yes. In JCS mythology, purgatory is simply a queue on
the way to disk, where the items can be accessed.
It's a map on top of a queue. All items that go to
disk, go to the queue. You can set a limit on the
queue size regardless of the disk usage pattern.
> What's the lifecycle of
> cache element with UPDATE policy? Does it go just
> between 'memory' and
> 'dc'? (if not found in memory, load from disk and
> put in memory cache,
> expiring some entries from memory on the way if
> necessary)?
When the item is put into the cache for the first time
it goes to disk as well as memory (if the max memory
size is greater than 0). If the item is later now in
memory and is retrieved from disk, it will go back
into memory, but it should will still be on disk. If
the item is evicted from memory, because, let's say,
it falls off the LRU, then it will NOT be put to disk
again. This prevents thrashing. . . .
(I want to make another pattern called PRIMARY where
the items just go to disk until they are accessed yet
again . . .)
>
> Thanks for quick answers, Aaron.
>
> Vlad
>
> -----Original Message-----
> From: Aaron Smuts [mailto:asmuts@yahoo.com]
> Sent: Monday, August 28, 2006 9:21 AM
> To: JCS Users List
> Subject: RE: configuration question
>
>
>
> --- Vladimir Olenin <VO...@cihi.ca> wrote:
> Sorry, the docs are wrong. DiskUsagePattern is the
> property but you
> have to set it with a "name". Use
> "DiskUsagePatternName=UPDATE"
>
>
> > Trying to do this as per docs, with line
> >
> > jcs.default.cacheattributes.DiskUsagePattern =
> UPDATE
> >
> > But getting this in the output log:
> >
> > [WARN] Failed to set property diskUsagePattern to
> value "UPDATE".
> > Conversion to type [short] failed.
> >
> >
> > I think 'UPDATE' is a final short variable
> somewhere in the code....
> > But there is no referral to it in the guides. What
> should it be?
> >
> > -----Original Message-----
> > From: Aaron Smuts [mailto:asmuts@yahoo.com]
> > Sent: Friday, August 25, 2006 8:21 PM
> > To: JCS Users List
> > Subject: Re: configuration question
> >
> > Purgatory is a queue on which items on the way to
> disk can still be
> > accessed. Setting a limit on purgatory allows you
> to prevent the
> > queue from getting too large. If you set a
> purgatory to 1000 and then
>
> > quickly add 100,000 items faster than they can be
> written to disk,
> > then the queue will certainly hit 1000. When the
> max purgatory size
> > is reached the oldest are dropped.
> >
> > Purgatory is basically a bounded queue with a map
> on top. There is a
> > bit more description here:
> >
> >
>
http://jakarta.apache.org/jcs/IndexedDiskAuxCache.html
> >
> > You should probably run with the cacheattributes
> properly called
> > DiskUsagePatternName=UPDATE. This will cause the
> items to go to disk
> > when they are added to the cache instead of when
> they fall out of the
> > Memory cache.
> >
> > If you are going to flood the cache with items,
> then don't set a
> > purgatory limit, or set it to the number you will
> be adding if it is
> > known.
> >
> > Aaron
> >
> > --- Vladimir Olenin <VO...@cihi.ca> wrote:
> >
> > > Hi.
> > >
> > > I need to configure JCS in the following manner:
> > >
> > > - need to load 100K entries into the cache
> > > - NONE of the entries are allowed to be expired
> > (ie, ALL must be
> > > eternal)
> > > - have to keep in memory not more than 1000
> > entries at a time
> > > - the rest entries should be swapped to disk
> > automatically
> > >
> > > What is the default setting that would work the
> > best? The current
> > > configuration I use is below and it seem to
> work.
> > I can't quite figure
> >
> > > out the meaning of 'MaxPurgatorySize' property.
> > From the docs I
> > > understood that it's just the size of the 'disk
> > purge' queue. But the
> > > tests show that if it's a positive number, then
> > the entries start to
> > > be 'expired' from the cache (eg, if set to 1000
> it
> > seems like only a
> > > small fraction of 100K entries remains in cache
> > and some of the
> > > entries put in the cache become missing for
> sure,
> > since jcs.get
> > > returns null).
> > >
> > > Thanks!.
> > >
> > > ===========================================
> > > jcs.default = DC_SMALL_FOOTPRINT
> > > jcs.default.cacheattributes =
> > > org.apache.jcs.engine.CompositeCacheAttributes
> > > jcs.default.cacheattributes.MaxObjects = 1000
> > > jcs.default.cacheattributes.MemoryCacheName =
> > > org.apache.jcs.engine.memory.lru.LRUMemoryCache
> > >
> > >
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT=org.apache.jcs.auxiliary.disk.indexed.I
> > > ndexedDiskCacheFactory
> > >
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes=org.apache.jcs.auxiliary.dis
> > > k.indexed.IndexedDiskCacheAttributes
> > >
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.DiskPath=c:/Projects/Grouper
> > > /temp/jcs
> > >
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.maxKeySize=-1
> > >
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.MaxPurgatorySize=-1
> > >
> >
> >
> >
>
---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> > jcs-users-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail:
> > jcs-users-help@jakarta.apache.org
> >
> >
> >
>
---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> > jcs-users-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail:
> > jcs-users-help@jakarta.apache.org
> >
> >
>
>
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> jcs-users-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> jcs-users-help@jakarta.apache.org
>
>
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> jcs-users-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> jcs-users-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-users-help@jakarta.apache.org
RE: configuration question
Posted by Vladimir Olenin <VO...@cihi.ca>.
Thanks. It seems to work now. I wonder if 'UPDATE' policy is set, does
'MaxPurgatorySize' property has any effect then? What's the lifecycle of
cache element with UPDATE policy? Does it go just between 'memory' and
'dc'? (if not found in memory, load from disk and put in memory cache,
expiring some entries from memory on the way if necessary)?
Thanks for quick answers, Aaron.
Vlad
-----Original Message-----
From: Aaron Smuts [mailto:asmuts@yahoo.com]
Sent: Monday, August 28, 2006 9:21 AM
To: JCS Users List
Subject: RE: configuration question
--- Vladimir Olenin <VO...@cihi.ca> wrote:
Sorry, the docs are wrong. DiskUsagePattern is the property but you
have to set it with a "name". Use "DiskUsagePatternName=UPDATE"
> Trying to do this as per docs, with line
>
> jcs.default.cacheattributes.DiskUsagePattern = UPDATE
>
> But getting this in the output log:
>
> [WARN] Failed to set property diskUsagePattern to value "UPDATE".
> Conversion to type [short] failed.
>
>
> I think 'UPDATE' is a final short variable somewhere in the code....
> But there is no referral to it in the guides. What should it be?
>
> -----Original Message-----
> From: Aaron Smuts [mailto:asmuts@yahoo.com]
> Sent: Friday, August 25, 2006 8:21 PM
> To: JCS Users List
> Subject: Re: configuration question
>
> Purgatory is a queue on which items on the way to disk can still be
> accessed. Setting a limit on purgatory allows you to prevent the
> queue from getting too large. If you set a purgatory to 1000 and then
> quickly add 100,000 items faster than they can be written to disk,
> then the queue will certainly hit 1000. When the max purgatory size
> is reached the oldest are dropped.
>
> Purgatory is basically a bounded queue with a map on top. There is a
> bit more description here:
>
>
http://jakarta.apache.org/jcs/IndexedDiskAuxCache.html
>
> You should probably run with the cacheattributes properly called
> DiskUsagePatternName=UPDATE. This will cause the items to go to disk
> when they are added to the cache instead of when they fall out of the
> Memory cache.
>
> If you are going to flood the cache with items, then don't set a
> purgatory limit, or set it to the number you will be adding if it is
> known.
>
> Aaron
>
> --- Vladimir Olenin <VO...@cihi.ca> wrote:
>
> > Hi.
> >
> > I need to configure JCS in the following manner:
> >
> > - need to load 100K entries into the cache
> > - NONE of the entries are allowed to be expired
> (ie, ALL must be
> > eternal)
> > - have to keep in memory not more than 1000
> entries at a time
> > - the rest entries should be swapped to disk
> automatically
> >
> > What is the default setting that would work the
> best? The current
> > configuration I use is below and it seem to work.
> I can't quite figure
>
> > out the meaning of 'MaxPurgatorySize' property.
> From the docs I
> > understood that it's just the size of the 'disk
> purge' queue. But the
> > tests show that if it's a positive number, then
> the entries start to
> > be 'expired' from the cache (eg, if set to 1000 it
> seems like only a
> > small fraction of 100K entries remains in cache
> and some of the
> > entries put in the cache become missing for sure,
> since jcs.get
> > returns null).
> >
> > Thanks!.
> >
> > ===========================================
> > jcs.default = DC_SMALL_FOOTPRINT
> > jcs.default.cacheattributes =
> > org.apache.jcs.engine.CompositeCacheAttributes
> > jcs.default.cacheattributes.MaxObjects = 1000
> > jcs.default.cacheattributes.MemoryCacheName =
> > org.apache.jcs.engine.memory.lru.LRUMemoryCache
> >
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT=org.apache.jcs.auxiliary.disk.indexed.I
> > ndexedDiskCacheFactory
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes=org.apache.jcs.auxiliary.dis
> > k.indexed.IndexedDiskCacheAttributes
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.DiskPath=c:/Projects/Grouper
> > /temp/jcs
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.maxKeySize=-1
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.MaxPurgatorySize=-1
> >
>
>
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> jcs-users-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> jcs-users-help@jakarta.apache.org
>
>
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> jcs-users-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> jcs-users-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-users-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-users-help@jakarta.apache.org
RE: configuration question
Posted by Aaron Smuts <as...@yahoo.com>.
--- Vladimir Olenin <VO...@cihi.ca> wrote:
Sorry, the docs are wrong. DiskUsagePattern is the
property but you have to set it with a "name". Use
"DiskUsagePatternName=UPDATE"
> Trying to do this as per docs, with line
>
> jcs.default.cacheattributes.DiskUsagePattern =
> UPDATE
>
> But getting this in the output log:
>
> [WARN] Failed to set property diskUsagePattern to
> value "UPDATE".
> Conversion to type [short] failed.
>
>
> I think 'UPDATE' is a final short variable somewhere
> in the code.... But
> there is no referral to it in the guides. What
> should it be?
>
> -----Original Message-----
> From: Aaron Smuts [mailto:asmuts@yahoo.com]
> Sent: Friday, August 25, 2006 8:21 PM
> To: JCS Users List
> Subject: Re: configuration question
>
> Purgatory is a queue on which items on the way to
> disk can still be
> accessed. Setting a limit on purgatory allows you
> to prevent the queue
> from getting too large. If you set a purgatory to
> 1000 and then quickly
> add 100,000 items faster than they can be written to
> disk, then the
> queue will certainly hit 1000. When the max
> purgatory size is reached
> the oldest are dropped.
>
> Purgatory is basically a bounded queue with a map on
> top. There is a
> bit more description here:
>
>
http://jakarta.apache.org/jcs/IndexedDiskAuxCache.html
>
> You should probably run with the cacheattributes
> properly called
> DiskUsagePatternName=UPDATE. This will cause the
> items to go to disk
> when they are added to the cache instead of when
> they fall out of the
> Memory cache.
>
> If you are going to flood the cache with items, then
> don't set a
> purgatory limit, or set it to the number you will be
> adding if it is
> known.
>
> Aaron
>
> --- Vladimir Olenin <VO...@cihi.ca> wrote:
>
> > Hi.
> >
> > I need to configure JCS in the following manner:
> >
> > - need to load 100K entries into the cache
> > - NONE of the entries are allowed to be expired
> (ie, ALL must be
> > eternal)
> > - have to keep in memory not more than 1000
> entries at a time
> > - the rest entries should be swapped to disk
> automatically
> >
> > What is the default setting that would work the
> best? The current
> > configuration I use is below and it seem to work.
> I can't quite figure
>
> > out the meaning of 'MaxPurgatorySize' property.
> From the docs I
> > understood that it's just the size of the 'disk
> purge' queue. But the
> > tests show that if it's a positive number, then
> the entries start to
> > be 'expired' from the cache (eg, if set to 1000 it
> seems like only a
> > small fraction of 100K entries remains in cache
> and some of the
> > entries put in the cache become missing for sure,
> since jcs.get
> > returns null).
> >
> > Thanks!.
> >
> > ===========================================
> > jcs.default = DC_SMALL_FOOTPRINT
> > jcs.default.cacheattributes =
> > org.apache.jcs.engine.CompositeCacheAttributes
> > jcs.default.cacheattributes.MaxObjects = 1000
> > jcs.default.cacheattributes.MemoryCacheName =
> > org.apache.jcs.engine.memory.lru.LRUMemoryCache
> >
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT=org.apache.jcs.auxiliary.disk.indexed.I
> > ndexedDiskCacheFactory
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes=org.apache.jcs.auxiliary.dis
> > k.indexed.IndexedDiskCacheAttributes
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.DiskPath=c:/Projects/Grouper
> > /temp/jcs
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.maxKeySize=-1
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.MaxPurgatorySize=-1
> >
>
>
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> jcs-users-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> jcs-users-help@jakarta.apache.org
>
>
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> jcs-users-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> jcs-users-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-users-help@jakarta.apache.org
RE: configuration question
Posted by Vladimir Olenin <VO...@cihi.ca>.
Trying to do this as per docs, with line
jcs.default.cacheattributes.DiskUsagePattern = UPDATE
But getting this in the output log:
[WARN] Failed to set property diskUsagePattern to value "UPDATE".
Conversion to type [short] failed.
I think 'UPDATE' is a final short variable somewhere in the code.... But
there is no referral to it in the guides. What should it be?
-----Original Message-----
From: Aaron Smuts [mailto:asmuts@yahoo.com]
Sent: Friday, August 25, 2006 8:21 PM
To: JCS Users List
Subject: Re: configuration question
Purgatory is a queue on which items on the way to disk can still be
accessed. Setting a limit on purgatory allows you to prevent the queue
from getting too large. If you set a purgatory to 1000 and then quickly
add 100,000 items faster than they can be written to disk, then the
queue will certainly hit 1000. When the max purgatory size is reached
the oldest are dropped.
Purgatory is basically a bounded queue with a map on top. There is a
bit more description here:
http://jakarta.apache.org/jcs/IndexedDiskAuxCache.html
You should probably run with the cacheattributes properly called
DiskUsagePatternName=UPDATE. This will cause the items to go to disk
when they are added to the cache instead of when they fall out of the
Memory cache.
If you are going to flood the cache with items, then don't set a
purgatory limit, or set it to the number you will be adding if it is
known.
Aaron
--- Vladimir Olenin <VO...@cihi.ca> wrote:
> Hi.
>
> I need to configure JCS in the following manner:
>
> - need to load 100K entries into the cache
> - NONE of the entries are allowed to be expired (ie, ALL must be
> eternal)
> - have to keep in memory not more than 1000 entries at a time
> - the rest entries should be swapped to disk automatically
>
> What is the default setting that would work the best? The current
> configuration I use is below and it seem to work. I can't quite figure
> out the meaning of 'MaxPurgatorySize' property. From the docs I
> understood that it's just the size of the 'disk purge' queue. But the
> tests show that if it's a positive number, then the entries start to
> be 'expired' from the cache (eg, if set to 1000 it seems like only a
> small fraction of 100K entries remains in cache and some of the
> entries put in the cache become missing for sure, since jcs.get
> returns null).
>
> Thanks!.
>
> ===========================================
> jcs.default = DC_SMALL_FOOTPRINT
> jcs.default.cacheattributes =
> org.apache.jcs.engine.CompositeCacheAttributes
> jcs.default.cacheattributes.MaxObjects = 1000
> jcs.default.cacheattributes.MemoryCacheName =
> org.apache.jcs.engine.memory.lru.LRUMemoryCache
>
>
jcs.auxiliary.DC_SMALL_FOOTPRINT=org.apache.jcs.auxiliary.disk.indexed.I
> ndexedDiskCacheFactory
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes=org.apache.jcs.auxiliary.dis
> k.indexed.IndexedDiskCacheAttributes
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.DiskPath=c:/Projects/Grouper
> /temp/jcs
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.maxKeySize=-1
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.MaxPurgatorySize=-1
>
---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-users-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-users-help@jakarta.apache.org
Re: configuration question
Posted by Aaron Smuts <as...@yahoo.com>.
Purgatory is a queue on which items on the way to disk
can still be accessed. Setting a limit on purgatory
allows you to prevent the queue from getting too
large. If you set a purgatory to 1000 and then
quickly add 100,000 items faster than they can be
written to disk, then the queue will certainly hit
1000. When the max purgatory size is reached the
oldest are dropped.
Purgatory is basically a bounded queue with a map on
top. There is a bit more description here:
http://jakarta.apache.org/jcs/IndexedDiskAuxCache.html
You should probably run with the cacheattributes
properly called DiskUsagePatternName=UPDATE. This
will cause the items to go to disk when they are added
to the cache instead of when they fall out of the
Memory cache.
If you are going to flood the cache with items, then
don't set a purgatory limit, or set it to the number
you will be adding if it is known.
Aaron
--- Vladimir Olenin <VO...@cihi.ca> wrote:
> Hi.
>
> I need to configure JCS in the following manner:
>
> - need to load 100K entries into the cache
> - NONE of the entries are allowed to be expired (ie,
> ALL must be
> eternal)
> - have to keep in memory not more than 1000 entries
> at a time
> - the rest entries should be swapped to disk
> automatically
>
> What is the default setting that would work the
> best? The current
> configuration I use is below and it seem to work. I
> can't quite figure
> out the meaning of 'MaxPurgatorySize' property. From
> the docs I
> understood that it's just the size of the 'disk
> purge' queue. But the
> tests show that if it's a positive number, then the
> entries start to be
> 'expired' from the cache (eg, if set to 1000 it
> seems like only a small
> fraction of 100K entries remains in cache and some
> of the entries put in
> the cache become missing for sure, since jcs.get
> returns null).
>
> Thanks!.
>
> ===========================================
> jcs.default = DC_SMALL_FOOTPRINT
> jcs.default.cacheattributes =
> org.apache.jcs.engine.CompositeCacheAttributes
> jcs.default.cacheattributes.MaxObjects = 1000
> jcs.default.cacheattributes.MemoryCacheName =
> org.apache.jcs.engine.memory.lru.LRUMemoryCache
>
>
jcs.auxiliary.DC_SMALL_FOOTPRINT=org.apache.jcs.auxiliary.disk.indexed.I
> ndexedDiskCacheFactory
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes=org.apache.jcs.auxiliary.dis
> k.indexed.IndexedDiskCacheAttributes
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.DiskPath=c:/Projects/Grouper
> /temp/jcs
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.maxKeySize=-1
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.MaxPurgatorySize=-1
>
---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-users-help@jakarta.apache.org