You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jcs-dev@jakarta.apache.org by as...@apache.org on 2005/01/12 02:22:36 UTC
cvs commit: jakarta-turbine-jcs/src/test-conf TestDiskCacheCon.ccf
asmuts 2005/01/11 17:22:36
Modified: xdocs BasicJCSConfiguration.xml IndexedDiskAuxCache.xml
UsingJCSBasicWeb.xml
src/conf log4j.properties
src/java/org/apache/jcs/auxiliary/disk LRUMapJCS.java
src/java/org/apache/jcs/engine/memory/shrinking
ShrinkerThread.java
src/test-conf TestDiskCacheCon.ccf
Added: tempbuild jcs-1.2-dev.jar
Log:
Optional, configurable thread pools can be set for all auxiliaries. By default
the single queue is used. (Will document later.)
Shrinkers for all regions now share one thread and are executed by a Doug Lea
Clock Daemon.
The disk cache now has more configuration options exposed. You can set the purgatory size,
recycle bin size, etc. Alos, you can set a bound on the purgatory. This solves the possibility
or an ever growing purgatory with a bounded queue.
Uped version to 1.2-dev to mark the changes.
Updated some documentation. I need to rebuild the site. I published the new
disk cache config page already.
The disk cache options are described on the web site.
jcs.auxiliary.DC.attributes.MaxPurgatorySize=10000
jcs.auxiliary.DC.attributes.MaxKeySize=10000
jcs.auxiliary.DC.attributes.MaxRecycleBinSize=5000
jcs.auxiliary.DC.attributes.OptimizeAtRemoveCount=300000
Revision Changes Path
1.3 +1 -21 jakarta-turbine-jcs/xdocs/BasicJCSConfiguration.xml
Index: BasicJCSConfiguration.xml
===================================================================
RCS file: /home/cvs/jakarta-turbine-jcs/xdocs/BasicJCSConfiguration.xml,v
retrieving revision 1.2
retrieving revision 1.3
diff -u -r1.2 -r1.3
--- BasicJCSConfiguration.xml 14 Jul 2004 03:51:35 -0000 1.2
+++ BasicJCSConfiguration.xml 12 Jan 2005 01:22:35 -0000 1.3
@@ -61,19 +61,7 @@
</p>
<source><![CDATA[
jcs.default=DC
- ]]></source>
- <p>
- It is a good idea to specify the
- <code>system.GroupIdCache</code>, so add this to the file:
- </p>
- <source><![CDATA[
-jcs.system.groupIdCache=DC
-jcs.system.groupIdCache.cacheattributes=
- org.apache.jcs.engine.CompositeCacheAttributes
-jcs.system.groupIdCache.cacheattributes.MaxObjects=10000
-jcs.system.groupIdCache.cacheattributes.MemoryCacheName=
- org.apache.jcs.engine.memory.lru.LRUMemoryCache
- ]]></source>
+ ]]></source>
<p>
If you want to predefine a specific region, say called
<code>testCache1</code>, then add these lines:
@@ -140,14 +128,6 @@
org.apache.jcs.engine.CompositeCacheAttributes
jcs.default.cacheattributes.MaxObjects=1000
jcs.default.cacheattributes.MemoryCacheName=
- org.apache.jcs.engine.memory.lru.LRUMemoryCache
-
-# System CACHE REGION
-jcs.system.groupIdCache=DC,LTCP
-jcs.system.groupIdCache.cacheattributes=
- org.apache.jcs.engine.CompositeCacheAttributes
-jcs.system.groupIdCache.cacheattributes.MaxObjects=10000
-jcs.system.groupIdCache.cacheattributes.MemoryCacheName=
org.apache.jcs.engine.memory.lru.LRUMemoryCache
# PRE-DEFINED CACHE REGIONS
1.4 +82 -4 jakarta-turbine-jcs/xdocs/IndexedDiskAuxCache.xml
Index: IndexedDiskAuxCache.xml
===================================================================
RCS file: /home/cvs/jakarta-turbine-jcs/xdocs/IndexedDiskAuxCache.xml,v
retrieving revision 1.3
retrieving revision 1.4
diff -u -r1.3 -r1.4
--- IndexedDiskAuxCache.xml 14 Jul 2004 04:56:24 -0000 1.3
+++ IndexedDiskAuxCache.xml 12 Jan 2005 01:22:35 -0000 1.4
@@ -63,7 +63,6 @@
</p>
</subsection>
-
<subsection name="Configuration">
<p>
The simple configuration and is done in the auxiliary
@@ -73,8 +72,10 @@
"DiskPath" directory.
</p>
<p>
- The Disk indexes are equipped with an LRU storage limit. The maximum
- number of keys is configured by the maxKeySize parameter.
+ The Disk indexes are equipped with an LRU storage limit. The maximum
+ number of keys is configured by the maxKeySize parameter. If the
+ maximum key size is less than 0, no limit will be placed on the
+ number of keys. By default, the max key size is 5000.
</p>
<source><![CDATA[
jcs.auxiliary.DC=
@@ -82,9 +83,86 @@
jcs.auxiliary.DC.attributes=
org.apache.jcs.auxiliary.disk.indexed.IndexedDiskCacheAttributes
jcs.auxiliary.DC.attributes.DiskPath=g:\dev\jakarta-turbine-stratum\raf
-jcs.auxiliary.DC.attributes.maxKeySize=100000
+jcs.auxiliary.DC.attributes.MaxKeySize=100000
+ ]]></source>
+ </subsection>
+
+ <subsection name="Additional Configuration Options">
+ <p>
+ The indexed disk cache provides some additional configuration options.
+ </p>
+ <p>
+ The purgatory size of the Disk cache is equipped with an LRU storage limit.
+ The maximum number of elements allowed in purgatory is configured by the
+ MaxPurgatorySize parameter. By default, the max purgatory size is 5000.
+ </p>
+ <p>
+ Initial testing indicates that the disk cache performs better when the
+ key and purgatory sizes are limited.
+ </p>
+ <source><![CDATA[
+jcs.auxiliary.DC.attributes.MaxPurgatorySize=10000
+ ]]></source>
+ <p>
+ Slots in the data file become empty when items are removed from the disk cache.
+ The indexed disk cache keeps track of empty slots in the data file, so they can
+ be reused. The slot locations are stored in a sorted preferential array --
+ the recycle bin. The smallest items are removed from the recycle bin when it
+ reaches the specified limit. The MaxRecycleBinSize cannot be larger than
+ the MaxKeySize. If the MaxKeySize is less than 0, the recycle bin will default
+ to 5000.
+ </p>
+ <source><![CDATA[
+jcs.auxiliary.DC.attributes.MaxRecycleBinSize=10000
+ ]]></source>
+ <p>
+ The Disk cache can be configured to defragment the data file at runtime. Since
+ defragmentation is only necessary if items have been removed, the deframentation
+ interval is determined by the number of removes. Currently there is no way
+ to schedule defragmentation to run at a set time. If you set the OptimizeAtRemoveCount
+ to -1, no optimizations of the data file will occur until shutdown. By default
+ the value is -1.
+ </p>
+ <source><![CDATA[
+jcs.auxiliary.DC.attributes.OptimizeAtRemoveCount=30000
]]></source>
</subsection>
+
+ <subsection name="A Complete Configuration Example">
+ <p>
+ In this sample cache.ccf file, I configured the cache to use a disk cache,
+ called DC, by default. Also, I explicitly set a cache region called myRegion1
+ to use DC. I specified custom settings for all of the Indexed Disk Cache
+ configuration parameters.
+ </p>
+ <source><![CDATA[
+##############################################################
+##### Default Region Configuration
+jcs.default=DC
+jcs.default.cacheattributes=org.apache.jcs.engine.CompositeCacheAttributes
+jcs.default.cacheattributes.MaxObjects=100
+jcs.default.cacheattributes.MemoryCacheName=org.apache.jcs.engine.memory.lru.LRUMemoryCache
+
+##############################################################
+##### CACHE REGIONS
+jcs.region.myRegion1=DC
+jcs.region.myRegion1.cacheattributes=org.apache.jcs.engine.CompositeCacheAttributes
+jcs.region.myRegion1.cacheattributes.MaxObjects=1000
+jcs.region.myRegion1.cacheattributes.MemoryCacheName=org.apache.jcs.engine.memory.lru.LRUMemoryCache
+
+##############################################################
+##### AUXILIARY CACHES
+# Indexed Disk Cache
+jcs.auxiliary.DC=org.apache.jcs.auxiliary.disk.indexed.IndexedDiskCacheFactory
+jcs.auxiliary.DC.attributes=org.apache.jcs.auxiliary.disk.indexed.IndexedDiskCacheAttributes
+jcs.auxiliary.DC.attributes.DiskPath=target/test-sandbox/indexed-disk-cache
+jcs.auxiliary.DC.attributes.MaxPurgatorySize=10000
+jcs.auxiliary.DC.attributes.MaxKeySize=10000
+jcs.auxiliary.DC.attributes.OptimizeAtRemoveCount=300000
+jcs.auxiliary.DC.attributes.MaxRecycleBinSize=7500
+ ]]></source>
+ </subsection>
+
<subsection name="TODO">
<p>
The Indexed Disk Auxiliary Cache will eventually be equiped
1.4 +4 -9 jakarta-turbine-jcs/xdocs/UsingJCSBasicWeb.xml
Index: UsingJCSBasicWeb.xml
===================================================================
RCS file: /home/cvs/jakarta-turbine-jcs/xdocs/UsingJCSBasicWeb.xml,v
retrieving revision 1.3
retrieving revision 1.4
diff -u -r1.3 -r1.4
--- UsingJCSBasicWeb.xml 16 Nov 2004 07:33:29 -0000 1.3
+++ UsingJCSBasicWeb.xml 12 Jan 2005 01:22:35 -0000 1.4
@@ -360,15 +360,6 @@
jcs.default.elementattributes.IsRemote=true
jcs.default.elementattributes.IsLateral=true
-# SYSTEM CACHE
-
-# should be defined for the storage of group attribute list
-jcs.system.groupIdCache=DC,RFailover
-jcs.system.groupIdCache.cacheattributes=
- org.apache.jcs.engine.CompositeCacheAttributes
-jcs.system.groupIdCache.cacheattributes.MaxObjects=10000
-jcs.system.groupIdCache.cacheattributes.MemoryCacheName=
- org.apache.jcs.engine.memory.lru.LRUMemoryCache
# CACHE REGIONS AVAILABLE
@@ -394,6 +385,10 @@
jcs.auxiliary.DC.attributes=
org.apache.jcs.auxiliary.disk.indexed.IndexedDiskCacheAttributes
jcs.auxiliary.DC.attributes.DiskPath=/usr/opt/bookstore/raf
+jcs.auxiliary.DC.attributes.MaxPurgatorySize=10000
+jcs.auxiliary.DC.attributes.MaxKeySize=10000
+jcs.auxiliary.DC.attributes.OptimizeAtRemoveCount=300000
+jcs.auxiliary.DC.attributes.MaxRecycleBinSize=7500
# Remote RMI Cache set up to failover
jcs.auxiliary.RFailover=
1.1 jakarta-turbine-jcs/tempbuild/jcs-1.2-dev.jar
<<Binary file>>
1.7 +1 -1 jakarta-turbine-jcs/src/conf/log4j.properties
Index: log4j.properties
===================================================================
RCS file: /home/cvs/jakarta-turbine-jcs/src/conf/log4j.properties,v
retrieving revision 1.6
retrieving revision 1.7
diff -u -r1.6 -r1.7
--- log4j.properties 6 Jan 2005 04:44:28 -0000 1.6
+++ log4j.properties 12 Jan 2005 01:22:36 -0000 1.7
@@ -28,7 +28,7 @@
#log4j.category.org.apache.jcs.acess=WARN,WF
#log4j.category.org.apache.jcs.engine.control=WARN,WF
-#log4j.logger.org.apache.jcs.engine.memory.shrinking=DEBUG
+log4j.logger.org.apache.jcs.engine.memory.shrinking=DEBUG
#log4j.logger.org.apache.jcs.auxiliary.disk=DEBUG
log4j.logger.org.apache.jcs.auxiliary.disk.indexed=INFO
1.2 +1 -1 jakarta-turbine-jcs/src/java/org/apache/jcs/auxiliary/disk/LRUMapJCS.java
Index: LRUMapJCS.java
===================================================================
RCS file: /home/cvs/jakarta-turbine-jcs/src/java/org/apache/jcs/auxiliary/disk/LRUMapJCS.java,v
retrieving revision 1.1
retrieving revision 1.2
diff -u -r1.1 -r1.2
--- LRUMapJCS.java 10 May 2004 01:15:26 -0000 1.1
+++ LRUMapJCS.java 12 Jan 2005 01:22:36 -0000 1.2
@@ -2,7 +2,7 @@
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
-import org.apache.commons.collections.LRUMap;
+import org.apache.commons.collections.map.LRUMap;
/**
* Extension of LRUMap for logging of removals.
1.16 +10 -41 jakarta-turbine-jcs/src/java/org/apache/jcs/engine/memory/shrinking/ShrinkerThread.java
Index: ShrinkerThread.java
===================================================================
RCS file: /home/cvs/jakarta-turbine-jcs/src/java/org/apache/jcs/engine/memory/shrinking/ShrinkerThread.java,v
retrieving revision 1.15
retrieving revision 1.16
diff -u -r1.15 -r1.16
--- ShrinkerThread.java 13 Jul 2004 02:48:30 -0000 1.15
+++ ShrinkerThread.java 12 Jan 2005 01:22:36 -0000 1.16
@@ -40,16 +40,13 @@
*
* @version $Id$
*/
-public class ShrinkerThread extends Thread
+public class ShrinkerThread implements Runnable
{
private final static Log log = LogFactory.getLog( ShrinkerThread.class );
/** The MemoryCache instance which this shrinker is watching */
private final MemoryCache cache;
- /** The time to sleep between shrink runs */
- private final long shrinkerInterval;
-
/** Maximum memory idle time for the whole cache */
private final long maxMemoryIdleTime;
@@ -57,9 +54,6 @@
private int maxSpoolPerRun;
private boolean spoolLimit = false;
- /** Flag that indicates if the thread is still alive */
- boolean alive = true;
-
/**
* Constructor for the ShrinkerThread object.
*
@@ -71,9 +65,6 @@
this.cache = cache;
- this.shrinkerInterval =
- cache.getCacheAttributes().getShrinkerIntervalSeconds() * 1000;
-
long maxMemoryIdleTimeSeconds =
cache.getCacheAttributes().getMaxMemoryIdleTimeSeconds();
@@ -96,39 +87,11 @@
}
/**
- * Graceful shutdown after this round of processing.
- */
- public void kill()
- {
- if ( log.isInfoEnabled() )
- {
- log.info( "Killing Shrinker for: "
- + this.cache.getCompositeCache().getCacheName() );
- }
- alive = false;
- }
-
- /**
* Main processing method for the ShrinkerThread object
*/
public void run()
{
- while ( alive )
- {
-
- shrink();
-
- try
- {
- sleep( shrinkerInterval );
- }
- catch ( InterruptedException ie )
- {
- // Continue until killed ( alive == false )
- }
- }
-
- return;
+ shrink();
}
/**
@@ -257,12 +220,18 @@
log.debug( "spoolCount = '" + spoolCount + "'; " +
"maxSpoolPerRun = '" + maxSpoolPerRun + "'");
}
+
+ // stop processing if limit has been reached.
+ if ( spoolLimit && (spoolCount >= this.maxSpoolPerRun ) )
+ {
+ keys = null;
+ return;
+ }
}
}
}
keys = null;
- //System.gc();
}
catch ( Throwable t )
{
1.4 +6 -9 jakarta-turbine-jcs/src/test-conf/TestDiskCacheCon.ccf
Index: TestDiskCacheCon.ccf
===================================================================
RCS file: /home/cvs/jakarta-turbine-jcs/src/test-conf/TestDiskCacheCon.ccf,v
retrieving revision 1.3
retrieving revision 1.4
diff -u -r1.3 -r1.4
--- TestDiskCacheCon.ccf 7 Jan 2005 22:29:17 -0000 1.3
+++ TestDiskCacheCon.ccf 12 Jan 2005 01:22:36 -0000 1.4
@@ -6,11 +6,6 @@
jcs.default.cacheattributes.MaxObjects=100
jcs.default.cacheattributes.MemoryCacheName=org.apache.jcs.engine.memory.lru.LRUMemoryCache
-# SYSTEM GROUP ID CACHE
-jcs.system.groupIdCache=indexedDiskCache
-jcs.system.groupIdCache.cacheattributes=org.apache.jcs.engine.CompositeCacheAttributes
-jcs.system.groupIdCache.cacheattributes.MaxObjects=10000
-jcs.system.groupIdCache.cacheattributes.MemoryCacheName=org.apache.jcs.engine.memory.lru.LRUMemoryCache
##### CACHE REGIONS FOR TEST
@@ -42,9 +37,11 @@
jcs.auxiliary.indexedDiskCache=org.apache.jcs.auxiliary.disk.indexed.IndexedDiskCacheFactory
jcs.auxiliary.indexedDiskCache.attributes=org.apache.jcs.auxiliary.disk.indexed.IndexedDiskCacheAttributes
jcs.auxiliary.indexedDiskCache.attributes.DiskPath=target/test-sandbox/indexed-disk-cache-conc
-jcs.auxiliary.indexedDiskCache.attributes.maxKeySize=10000
-jcs.auxiliary.indexedDiskCache.attributes.optimizeAtRemoveCount=3000000
-jcs.auxiliary.indexedDiskCache.attributes.EventQueueType=POOLED
+jcs.auxiliary.indexedDiskCache.attributes.MaxPurgatorySize=10000
+jcs.auxiliary.indexedDiskCache.attributes.MaxKeySize=10000
+jcs.auxiliary.indexedDiskCache.attributes.MaxRecycleBinSize=5000
+jcs.auxiliary.indexedDiskCache.attributes.OptimizeAtRemoveCount=300000
+jcs.auxiliary.indexedDiskCache.attributes.EventQueueType=SINGLE
jcs.auxiliary.indexedDiskCache.attributes.EventQueuePoolName=disk_cache_event_queue
@@ -72,7 +69,7 @@
thread_pool.disk_cache_event_queue.useBoundary=false
thread_pool.disk_cache_event_queue.minimumPoolSize=2
thread_pool.disk_cache_event_queue.keepAliveTime=3500
-thread_pool.disk_cache_event_queue.startUpSize=1
+thread_pool.disk_cache_event_queue.startUpSize=10
# Remote cache client thread pool config
thread_pool.remote_cache_client.boundarySize=75
---------------------------------------------------------------------
To unsubscribe, e-mail: turbine-jcs-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: turbine-jcs-dev-help@jakarta.apache.org