You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@turbine.apache.org by Aaron Smuts <aa...@verizon.net> on 2002/02/17 00:41:19 UTC
JCS -- run time config / adding auxiliaries
I want to allow the cache to be programmatically configured. Right now,
if a cache isn't configured in the cache.ccf, it will take the defaults.
It would be nice if under certain conditions a program that wanted to
add an auxiliary to a region could do so.
The easiest and cleanest way to do this would be to allow programs to
get a list of auxiliaries that were in the props file and add or remove
these from the particular cache region.
Anything more would be tricky. I also wouldn't want people to be
hardcoding configuration in their programs. This is what jcache
proposes. It is easier for them since they have a fixed set of
auxiliaries (disk and lateral), but JCS has an array of possibilities
and they may not play well together. You wouldn't want two disk caches
and you might have trouble running a remote and a lateral for the same
region. This shouldn't cause any looping, but the behavior could be
different. The remote coud be configured to remove upon put and the
later to put on put. This might get funny and would be wasteful.
One thing that might be nice would be auxiliary failover switching. It
would be cool to say use this auxiliary and if it goes into an error
mode, start using this one. An xml config could handle this. The
current configuration could do this in the list of auxiliaries.
DC(JISP),RC(LC) -- use the indexed disk cache and iff it fails use the
JISP. Use the remote cache and if it fails us the lateral cache . . .
This wouldn't be so hard.
Zombie mode you can detect this and switch. This might involve an
internal wrapper over the auxiliaries that are added to the cache.
(Maybe this should be in the notes file?)
Aaron
> -----Original Message-----
> From: jtaylor@apache.org [mailto:jtaylor@apache.org]
> Sent: Saturday, February 16, 2002 5:00 PM
> To: jakarta-turbine-stratum-cvs@apache.org
> Subject: cvs commit: jakarta-turbine-stratum/src/test-conf
> TestDiskCache.ccf
>
> jtaylor 02/02/16 13:59:33
>
> Modified: src/java/org/apache/stratum/jcs JCS.java
> src/java/org/apache/stratum/jcs/engine/control
> CompositeCacheManager.java
> Added: src/test/org/apache/stratum/jcs TestDiskCache.java
> src/test-conf TestDiskCache.ccf
> Log:
> Added a unit test which causes objects to be pushed to the disk
cache.
> This
> required the addition of a means to set the cache configuration
file.
> This
> should be pushed up to the CacheAccess class (or some other means to
> change
> the config file provided).
>
> Revision Changes Path
> 1.6 +21 -2 jakarta-turbine-
> stratum/src/java/org/apache/stratum/jcs/JCS.java
>
> Index: JCS.java
> ===================================================================
> RCS file: /home/cvs/jakarta-turbine-
> stratum/src/java/org/apache/stratum/jcs/JCS.java,v
> retrieving revision 1.5
> retrieving revision 1.6
> diff -u -r1.5 -r1.6
> --- JCS.java 16 Feb 2002 02:37:16 -0000 1.5
> +++ JCS.java 16 Feb 2002 21:59:33 -0000 1.6
> @@ -75,10 +75,12 @@
> * @author <a href="mailto:asmuts@yahoo.com">Aaron Smuts</a>
> * @author <a href="mailto:jtaylor@apache.org">James Taylor</a>
> * @created February 13, 2002
> - * @version $Id: JCS.java,v 1.5 2002/02/16 02:37:16 jtaylor Exp $
> + * @version $Id: JCS.java,v 1.6 2002/02/16 21:59:33 jtaylor Exp $
> */
> public class JCS extends GroupCacheAccess
> {
> + private static String configFilename = null;
> +
> private static GroupCacheManager cacheMgr;
>
> /**
> @@ -137,9 +139,26 @@
> {
> if ( cacheMgr == null )
> {
> - cacheMgr =
GroupCacheManagerFactory.getInstance();
> + if ( configFilename == null )
> + {
> + cacheMgr =
> GroupCacheManagerFactory.getInstance();
> + }
> + else
> + {
> + cacheMgr = GroupCacheManagerFactory
> + .getInstance( configFilename );
> + }
> }
> }
> }
> + }
> +
> + /**
> + * Set the filename that the cache manager will be initialized
> with. Only
> + * matters before the instance is initialized.
> + */
> + public static void setConfigFilename( String configFilename )
> + {
> + JCS.configFilename = configFilename;
> }
> }
>
>
>
> 1.7 +1 -1 jakarta-turbine-
>
stratum/src/java/org/apache/stratum/jcs/engine/control/CompositeCacheMan
ag
> er.java
>
> Index: CompositeCacheManager.java
> ===================================================================
> RCS file: /home/cvs/jakarta-turbine-
>
stratum/src/java/org/apache/stratum/jcs/engine/control/CompositeCacheMan
ag
> er.java,v
> retrieving revision 1.6
> retrieving revision 1.7
> diff -u -r1.6 -r1.7
> --- CompositeCacheManager.java 16 Feb 2002 17:58:12 -0000
1.6
> +++ CompositeCacheManager.java 16 Feb 2002 21:59:33 -0000
1.7
> @@ -104,7 +104,7 @@
> {
> is.close();
> }
> - catch ( IOException ignore )
> + catch ( Exception ignore )
> {
> // Ignored
> }
>
>
>
> 1.1 jakarta-turbine-
> stratum/src/test/org/apache/stratum/jcs/TestDiskCache.java
>
> Index: TestDiskCache.java
> ===================================================================
> package org.apache.stratum.jcs;
>
> /*
> *
====================================================================
> * The Apache Software License, Version 1.1
> *
> * Copyright (c) 2001-2002 The Apache Software Foundation. All
rights
> * reserved.
> *
> * Redistribution and use in source and binary forms, with or
without
> * modification, are permitted provided that the following
conditions
> * are met:
> *
> * 1. Redistributions of source code must retain the above copyright
> * notice, this list of conditions and the following disclaimer.
> *
> * 2. Redistributions in binary form must reproduce the above
copyright
> * notice, this list of conditions and the following disclaimer in
> * the documentation and/or other materials provided with the
> * distribution.
> *
> * 3. The end-user documentation included with the redistribution,
> * if any, must include the following acknowledgment:
> * "This product includes software developed by the
> * Apache Software Foundation (http://www.apache.org/)."
> * Alternately, this acknowledgment may appear in the software
itself,
> * if and wherever such third-party acknowledgments normally appear.
> *
> * 4. The names "Apache" and "Apache Software Foundation" and
> * "Apache Turbine" must not be used to endorse or promote products
> * derived from this software without prior written permission. For
> * written permission, please contact apache@apache.org.
> *
> * 5. Products derived from this software may not be called
"Apache",
> * "Apache Turbine", nor may "Apache" appear in their name, without
> * prior written permission of the Apache Software Foundation.
> *
> * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESSED OR IMPLIED
> * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
> * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
> * DISCLAIMED. IN NO EVENT SHALL THE APACHE SOFTWARE FOUNDATION OR
> * ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
> * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
> * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
> * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
AND
> * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY,
> * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
OUT
> * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY
OF
> * SUCH DAMAGE.
> *
====================================================================
> *
> * This software consists of voluntary contributions made by many
> * individuals on behalf of the Apache Software Foundation. For
more
> * information on the Apache Software Foundation, please see
> * <http://www.apache.org/>.
> */
>
> import junit.framework.Test;
> import junit.framework.TestCase;
> import junit.framework.TestSuite;
>
> import org.apache.stratum.jcs.JCS;
> import
org.apache.stratum.jcs.engine.behavior.ICompositeCacheAttributes;
>
> /**
> * Test which excercises the disk cache.
> *
> * @author <a href="mailto:asmuts@yahoo.com">Aaron Smuts</a>
> */
> public class TestDiskCache extends TestCase
> {
> /**
> * Number of items to cache, twice the configured maxObjects for
the
> * memory cache
> */
> private static int items = 200;
>
> /**
> * Constructor for the TestSimpleLoad object
> */
> public TestDiskCache( String testName )
> {
> super( testName );
> }
>
> /**
> * Main method passes this test to the text test runner.
> */
> public static void main( String args[] )
> {
> String[] testCaseName = {TestDiskCache.class.getName()};
> junit.textui.TestRunner.main( testCaseName );
> }
>
> /**
> * A unit test suite for JUnit
> *
> * @return The test suite
> */
> public static Test suite()
> {
> return new TestSuite( TestDiskCache.class );
> }
>
> /**
> * Adds items to cache, gets them, and removes them. The item
count
> is more
> * than
> *
> * @exception Exception Description of the Exception
> */
> public void testDiskCache()
> throws Exception
> {
> JCS.setConfigFilename( "/TestDiskCache.ccf" );
>
> JCS jcs = JCS.getInstance( "testCache" );
>
> // Add items to cache
>
> for ( int i = 0; i <= items; i++ )
> {
> jcs.put( i + ":key", "data" + i );
> }
>
> // Test that all items are in cache
>
> for ( int i = 0; i <= items; i++ )
> {
> String value = ( String ) jcs.get( i + ":key" );
>
> this.assertEquals( "data" + i, value );
> }
>
> // Remove all the items
>
> for ( int i = 0; i <= items; i++ )
> {
> jcs.destroy( i + ":key" );
> }
>
> // Verify removal
>
> for ( int i = 0; i <= items; i++ )
> {
> assertNull( jcs.get( i + ":key" ) );
> }
> }
> }
>
>
>
> 1.1 jakarta-turbine-stratum/src/test-
> conf/TestDiskCache.ccf
>
> Index: TestDiskCache.ccf
> ===================================================================
> # Cache configuration for the 'TestDiskCache' test. The memory cache
has
> a
> # a maximum of 100 objects, so objects should get pushed into the
disk
> cache
>
> jcs.default=DC
>
>
jcs.default.cacheattributes=org.apache.stratum.jcs.engine.CompositeCache
At
> tributes
> jcs.default.cacheattributes.MaxObjects=1000
>
>
jcs.default.cacheattributes.MemoryCacheName=org.apache.stratum.jcs.engin
e.
> memory.lru.LRUMemoryCache
>
> # SYSTEM GROUP ID CACHE
> jcs.system.groupIdCache=DC
>
>
jcs.system.groupIdCache.cacheattributes=org.apache.stratum.jcs.engine.Co
mp
> ositeCacheAttributes
> jcs.system.groupIdCache.cacheattributes.MaxObjects=10000
>
>
jcs.system.groupIdCache.cacheattributes.MemoryCacheName=org.apache.strat
um
> .jcs.engine.memory.lru.LRUMemoryCache
>
>
> # CACHE REGIONS AVAILABLE
> jcs.region.testCache=DC
>
>
jcs.region.testCache.cacheattributes=org.apache.stratum.jcs.engine.Compo
si
> teCacheAttributes
> jcs.region.testCache.cacheattributes.MaxObjects=100
>
>
jcs.region.testCache.cacheattributes.MemoryCacheName=org.apache.stratum.
jc
> s.engine.memory.lru.LRUMemoryCache
>
> # AUXILIARY CACHES AVAILABLE
>
> # Primary Disk Cache-- faster than the rest because of memory key
> storage
>
jcs.auxiliary.DC=org.apache.stratum.jcs.auxiliary.disk.DiskCacheFactory
>
>
jcs.auxiliary.DC.attributes=org.apache.stratum.jcs.auxiliary.disk.DiskCa
ch
> eAttributes
> jcs.auxiliary.DC.attributes.DiskPath=bin/test/disk-cache
>
>
>
>
> --
> To unsubscribe, e-mail: <mailto:turbine-dev-
> unsubscribe@jakarta.apache.org>
> For additional commands, e-mail: <mailto:turbine-dev-
> help@jakarta.apache.org>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: JCS -- run time config / adding auxiliaries
Posted by Aaron Smuts <aa...@verizon.net>.
> -----Original Message-----
> From: James Taylor [mailto:jtaylor@4lane.com]
> Sent: Saturday, February 16, 2002 10:51 PM
> To: turbine-dev@jakarta.apache.org
> Subject: Re: JCS -- run time config / adding auxiliaries
>
> Okay, let me see if I have a handle on the configuration needs. It
seems
> that there are currently two sets of objects which need to be
> configured. First, the regions which are available in the cache.
Second,
> the auxiliaries which are available to those regions. Every region and
> auxiliary within the cache is identified by a unique ID.
>
> For a region users need to be able to configure
>
> 1. The id that identifies the region.
> 2. The list of auxiliaries that should be used for the region,
> possibly with a failover (or sequence of failovers) for
each
> auxilliary.
Later, auxiliaries like the remote cache server should have failover
routes too. . . .
> 3. The memory cache which should be used for the region.
> 4. A set of cache attributes for the region. Currently the
> configuration allows the user to define the class used to
> contain the cache attributes, and the set of attributes to
> populate it with.
>
> For an auxiliary one needs to be able to configure
>
> 1. The id that identifies the auxiliary.
> 2. The class to use for the auxiliary.
> 3. The attributes object to use for configuring the auxiliary.
> 4. The actual attributes.
>
Yep. Yep.
>
> Two things immediately occur to me. The first is that the
specification
> of the class to use for attributes seems unnecessary, since there is
> exactly one attributes class for each auxiliary, we can simplify
> configuration by determining it at runtime.
Ya. This makes sense. Would a naming convention do it? The current
method was an easy way to implement it. Every factory takes an
IAuxiliaryAttributes class. It was easy to just define the class in the
props file. I'm not sure why anyone would want to change it unless they
wanted to make one giant configuration class, but this would be a
bleeding monster.
It is harder to compiler enforce a naming convention than an interface,
otherwise it sounds good and a bit less confusing.
Second, that something like
> the mapper might be an easy way to cleanup the configuration using
XML.
> However the mapper probably doesn't yet offer enough flexibility.
>
When it does . .
> For regions, can the attributes class also be eliminated? In what
> situations would one use something other than
CompositeCacheAttributes?
> I suppose it is configurable to allow more flexibility, but perhaps
you
> could elaborate on this area Aaron? I noticed the comment in
> CacheAttributes says that it should probably be removed. Would it
limit
> flexibility for the Composite Cache ( Group Cache ? ) to be the only
> option? Would seem to simplify things quite a bit conceptually.
>
CompositeCacheAttributes should be renamed CacheAttrbitues and the
original CacheAttributes, if it isn't used should be scrapped.
"Region" and "cache" are synonymous. Everything could be called
"Region" but this is confusing. I really don't like that a Logger in
log4j is called a category. It is much less confusing for me when it
was called a Logger.
> > I want to allow the cache to be programmatically configured. Right
now,
> > if a cache isn't configured in the cache.ccf, it will take the
defaults.
> > It would be nice if under certain conditions a program that wanted
to
> > add an auxiliary to a region could do so.
>
> I agree that that would be cool. If we created a standard public
> interface for configuring the above things, then one could add regions
> or auxiliaries to the cache, and assign auxiliaries to regions at
> runtime. This would seem to be a good idea. The configuration
mechanism
> could use the same interfaces. It would be great if you could read a
> configuration file at any time to add new modules to the cache.
Hmmn. Not a bad idea. This would involve some deregistration to remote
servers though. That should pretty much already be there, just need to
call the methods. (maybe)
>
> > The easiest and cleanest way to do this would be to allow programs
to
> > get a list of auxiliaries that were in the props file and add or
remove
> > these from the particular cache region.
>
> Exactly. Really we need to divorce things from the props file at all.
At
> any given time we have a list of regions and auxiliaries in the cache,
> which can be associated. Either programmatic configuration or reading
of
> another config file can be done at any time to add to those lists, or
> add more associations.
>
It is nice to be able to define auxiliaries in the props file that are
not currently used, so they can be added later. You don't know if they
work until they are used. Right now the auxiliaries are only discovered
when they are used.
> > Anything more would be tricky.
>
> I don't know what more people would need, this should allow enough
> flexibility.
>
> > I also wouldn't want people to be
> > hardcoding configuration in their programs. This is what jcache
> > proposes. It is easier for them since they have a fixed set of
> > auxiliaries (disk and lateral), but JCS has an array of
possibilities
> > and they may not play well together. You wouldn't want two disk
caches
> > and you might have trouble running a remote and a lateral for the
same
> > region. This shouldn't cause any looping, but the behavior could be
> > different. The remote coud be configured to remove upon put and the
> > later to put on put. This might get funny and would be wasteful.
>
> Certainly there are bad configurations, but is it worth worrying too
> much about preventing users from having two disk caches?
I'm not preventing it. They can screw it up if they want. I'm mainly
worried about people hard coding configuration parameters in their
programs. These kinds of things are configuration dependent. Regions
could be defined in a program and what auxiliaries they use, yes.
Things like memory size and especially auxiliaries will change when an
environment changes. The auxiliary list should definitely be defined in
the props file. You couldn't move from dev, to qa, to production very
easily otherwise . . .
This would
> probably be better accomplished with documentation and examples. I'm
> inclined the best bet is to provide the simplest interface which
offers
> a nice amount of flexibility.
>
Yep.
> > One thing that might be nice would be auxiliary failover switching.
It
> > would be cool to say use this auxiliary and if it goes into an error
> > mode, start using this one. An xml config could handle this. The
> > current configuration could do this in the list of auxiliaries.
> > DC(JISP),RC(LC) -- use the indexed disk cache and iff it fails use
the
> > JISP. Use the remote cache and if it fails us the lateral cache . .
.
> > This wouldn't be so hard.
> >
> > Zombie mode you can detect this and switch. This might involve an
> > internal wrapper over the auxiliaries that are added to the cache.
>
> These all seem like good ideas to me as we move forward. I like the
idea
> of being able to have more flexibility in configuring the cache. While
> I'm not sure what the exact nature of the config files should be (I
like
> XML since there is a bit of hierarchy in the configuration) separating
> the configuration of the cache from parsing of the config file, and
> making the interfaces for the former public, seems like a good way to
> accomplish what we need
>
This could turn into a huge job.
For now to keep it simple we could:
1. Rename the CompositeCacheAttributes
2. Load up the auxiliary list in the base CacheManager.
3. Expose methods to get a list of auxiliary names and types. Perhaps
this should be in a new class retrieved from the JCS access class.
4. Expose methods to set the auxiliary names for a region --
attachAuxiliaries( String [] names ). All current auxiliaries could be
removed if they are not in the list. We many need to warn that this
could be dangerous if the cache is currently very busy.(?)
The attachAuxiliaries method could then lookup the auxiliary factory by
the name factory configuration and then get an instance.
5. Make sure that part of the IAuxiliaryCache interface is
deregistration or detach (better). I think destroy is currently used,
but this kills the auxiliary in shutdown (which needs some work).
One other thing: The memory cache uses the CompositeCacheAttributes
(soon to be CacheAttributes) since the memory size needs to be
configurable and there is no way to access the configuration classes of
the auxiliaries. We might also want some methods to get to
setAuxiliaryAttribute methods of the auxiliary caches, but this could
come later.
Aaron
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: JCS -- run time config / adding auxiliaries
Posted by James Taylor <jt...@4lane.com>.
Okay, let me see if I have a handle on the configuration needs. It seems
that there are currently two sets of objects which need to be
configured. First, the regions which are available in the cache. Second,
the auxiliaries which are available to those regions. Every region and
auxiliary within the cache is identified by a unique ID.
For a region users need to be able to configure
1. The id that identifies the region.
2. The list of auxiliaries that should be used for the region,
possibly with a failover (or sequence of failovers) for each
auxilliary.
3. The memory cache which should be used for the region.
4. A set of cache attributes for the region. Currently the
configuration allows the user to define the class used to
contain the cache attributes, and the set of attributes to
populate it with.
For an auxiliary one needs to be able to configure
1. The id that identifies the auxiliary.
2. The class to use for the auxiliary.
3. The attributes object to use for configuring the auxiliary.
4. The actual attributes.
Two things immediately occur to me. The first is that the specification
of the class to use for attributes seems unnecessary, since there is
exactly one attributes class for each auxiliary, we can simplify
configuration by determining it at runtime. Second, that something like
the mapper might be an easy way to cleanup the configuration using XML.
However the mapper probably doesn't yet offer enough flexibility.
For regions, can the attributes class also be eliminated? In what
situations would one use something other than CompositeCacheAttributes?
I suppose it is configurable to allow more flexibility, but perhaps you
could elaborate on this area Aaron? I noticed the comment in
CacheAttributes says that it should probably be removed. Would it limit
flexibility for the Composite Cache ( Group Cache ? ) to be the only
option? Would seem to simplify things quite a bit conceptually.
> I want to allow the cache to be programmatically configured. Right now,
> if a cache isn't configured in the cache.ccf, it will take the defaults.
> It would be nice if under certain conditions a program that wanted to
> add an auxiliary to a region could do so.
I agree that that would be cool. If we created a standard public
interface for configuring the above things, then one could add regions
or auxiliaries to the cache, and assign auxiliaries to regions at
runtime. This would seem to be a good idea. The configuration mechanism
could use the same interfaces. It would be great if you could read a
configuration file at any time to add new modules to the cache.
> The easiest and cleanest way to do this would be to allow programs to
> get a list of auxiliaries that were in the props file and add or remove
> these from the particular cache region.
Exactly. Really we need to divorce things from the props file at all. At
any given time we have a list of regions and auxiliaries in the cache,
which can be associated. Either programmatic configuration or reading of
another config file can be done at any time to add to those lists, or
add more associations.
> Anything more would be tricky.
I don't know what more people would need, this should allow enough
flexibility.
> I also wouldn't want people to be
> hardcoding configuration in their programs. This is what jcache
> proposes. It is easier for them since they have a fixed set of
> auxiliaries (disk and lateral), but JCS has an array of possibilities
> and they may not play well together. You wouldn't want two disk caches
> and you might have trouble running a remote and a lateral for the same
> region. This shouldn't cause any looping, but the behavior could be
> different. The remote coud be configured to remove upon put and the
> later to put on put. This might get funny and would be wasteful.
Certainly there are bad configurations, but is it worth worrying too
much about preventing users from having two disk caches? This would
probably be better accomplished with documentation and examples. I'm
inclined the best bet is to provide the simplest interface which offers
a nice amount of flexibility.
> One thing that might be nice would be auxiliary failover switching. It
> would be cool to say use this auxiliary and if it goes into an error
> mode, start using this one. An xml config could handle this. The
> current configuration could do this in the list of auxiliaries.
> DC(JISP),RC(LC) -- use the indexed disk cache and iff it fails use the
> JISP. Use the remote cache and if it fails us the lateral cache . . .
> This wouldn't be so hard.
>
> Zombie mode you can detect this and switch. This might involve an
> internal wrapper over the auxiliaries that are added to the cache.
These all seem like good ideas to me as we move forward. I like the idea
of being able to have more flexibility in configuring the cache. While
I'm not sure what the exact nature of the config files should be (I like
XML since there is a bit of hierarchy in the configuration) separating
the configuration of the cache from parsing of the config file, and
making the interfaces for the former public, seems like a good way to
accomplish what we need
Any thoughts?
-- jt
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>