You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Aaron Smuts <AS...@therealm.com> on 2002/01/17 21:40:10 UTC
RE: [simplestore] enhancements (was: [simplestore] inital che
ck in)
> -----Original Message-----
> From: Juozas Baliuka [mailto:baliuka@mwm.lt]
> Sent: Thursday, January 17, 2002 8:17 AM
> To: Jakarta Commons Developers List
> Subject: RE: [simplestore] enhancements (was: [simplestore] inital check
> in)
>
>
> >Maybe I understand you wrong, but look at this:
> >In the Cocoon project we have 2 stores a) MRUMemoryStore for quick access
> >and b) JispFilesystemStore for -well- swapping. Why? Because we
> store/cache
> >generated data (xml -> xslt -> html or fop or ...) in the cache.
> Generating
> >this data is very expensive.
>
>
>
> Ok, but it is not memory management, it is optimization for generation.
It is reference management.
> Storing cache to disk has this
> disadvantage :
> if(memory.isLow()){
> // if method "isLow()" correct your cache is in disk, OS did this.
> // else if method "isLow()" is incorrect next has no meaning.
> store.flushToDisk();
>
> }
> void flushToDisk(){
>
> save(memory);// if method "isLow()" was correct you loaded cache from
> disk to memory here , OS did this.
>
> }
> You must find alternative way to Optimize generation, not memory
> management.
>
> I use this way to optimize generation :
>
> Generate content directly to stable storage if don't have it on memory and
> in disk:
>
> SomeStream getContent(Something id){
>
> Content c = find(id);
> if( c != null )return c.openStream();
>
> Content c = generateToStorageFirst(id);
> //cache to memory if possible, if your generation is very expensive and
> your going to copy it to Socket at last,
> // generating content to disk seems very fast :)
>
> return c.openStream();
>
> }
>
> //Optimization is in find
>
> Content find( Something id){
>
> Content c = findInMemoryCache(id);
>
> if (c != null )return c.openStream();
>
> return findInSomeStorage();
>
>
> }
>
> Content generateToStorageFirst(Something id){
>
> Content c = generate(id);
> if(!c.isVeryLage()){
> cache(c);
> }
> return c;
> }
> // constant for the same content;
> bolean isVeryLarge(){
>
> // if very lage, it can be faster to generate, large memory block
> allocation is very expensive.
>
> return lenght > ( totalVirtulMemorySizeForJVMis4GBon32bitOS/
> ConstantCanBeTested ) *
> CanBeTestedAppDependantConstant;
>
>
> }
>
> It is very possible to design it better , but this must be more clear read
> here.
> You never need method "isLow()" it has no meaning, because it is
> impossible
> to implement.
> if application says "low" it does not mean it is true for JVM, if JVM
> says "low" it does not mean it is true for OS.
> if OS never says this for JAVA application. If you have two applications
> on
> your machine "isLow" for OS has no
> meaning in your application, It will not help for OS to move Virtual
> Memory, or application and OS will do it both.
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> --
> To unsubscribe, e-mail: <mailto:commons-dev-
> unsubscribe@jakarta.apache.org>
> For additional commands, e-mail: <mailto:commons-dev-
> help@jakarta.apache.org>
RE: [simplestore] enhancements (was: [simplestore] inital
check in)
Posted by Juozas Baliuka <ba...@mwm.lt>.
Hi,
simplestore is not the best optimization for projects like Cocoon.
You don't need it here. You don't need JISP and StoreJanitor.
It kind of abstraction, but forget it if you need performance and want to
save resources.
There are a lot of better ways to solve this problem.
1. Intercept request for HTTP Headers "Send IF Not Modified".
You can cache some content on client side.
2. Don't use single file for cache. Map URL to file and store "large"
content in file OS will do all this MRU stuf and cache for file system.
3. Don't write content to file if it is "small" store it in memory,
you will have statistics on next request, and here you can decide what
"large", "small","old","new","fast","slow".
4. Don't use any background cleanup, delegate all resource management
stuff for OS,JVM,GC where possible.
I don't know Cocoons code, it is possible there are more solutions and
believe you will find them.
I believe we will make simplestore to be a good and abstact, but it is not
solution for all cases, it can become
too "abstact" and it will be not "simple". It too simple for store and too
complex for optimizations.
You can forward my emails for Cocoon developers, I believe they will
understand me and this will help for you.
Cocoon seems very good framework for me, but I am not going to use it in
any of my projects at this time, there are a lot of work to do.
At 11:44 PM 1/17/2002 +0100, you wrote:
>People,
>
> >From: Aaron Smuts [mailto:ASmuts@therealm.com]
> >
> >> -----Original Message-----
> >> From: Juozas Baliuka [mailto:baliuka@mwm.lt]
> >> Sent: Thursday, January 17, 2002 8:17 AM
> >> To: Jakarta Commons Developers List
> >> Subject: RE: [simplestore] enhancements (was: [simplestore] inital check
> >> in)
> >>
> >>
> >> >Maybe I understand you wrong, but look at this:
> >> >In the Cocoon project we have 2 stores a) MRUMemoryStore for quick access
> >> >and b) JispFilesystemStore for -well- swapping. Why? Because we
> >> store/cache
> >> >generated data (xml -> xslt -> html or fop or ...) in the cache.
> >> Generating
> >> >this data is very expensive.
> >>
> >>
> >>
> >> Ok, but it is not memory management, it is optimization for generation.
> >
> >It is reference management.
>
>I think we have definition problem here. I don't wanna re-build the memory
>management of the JVM. I just wanna provide some _small_, _quick_ and _ease_
>to plug components where people can store -well- *things*. Like in the Cocoon
>project generated HTML or PDF files to spare System resources, or a
>Application
>Server which pools Objects, or whatever. Object in and throw Object out.
>The memory management of the JVM is doing the rest.
>
>We have to consider how we can use the JVM memory management and the GC in an
>most perfect way. Solution a) was the StoreJanitor which derefences
>Objects and
>forces the GC after that or solution b) WeakReferences which seem to be a very
>elegant solution. I'm open for everything!
>
>That's the picture I have. What's yours?
>
> Gerhard
>
>----------------------------
>You can't fall off the floor
>----------------------------
>
>
>--
>To unsubscribe, e-mail: <ma...@jakarta.apache.org>
>For additional commands, e-mail: <ma...@jakarta.apache.org>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: [simplestore] enhancements (was: [simplestore] inital check in)
Posted by Gerhard Froehlich <g-...@gmx.de>.
People,
>From: Aaron Smuts [mailto:ASmuts@therealm.com]
>
>> -----Original Message-----
>> From: Juozas Baliuka [mailto:baliuka@mwm.lt]
>> Sent: Thursday, January 17, 2002 8:17 AM
>> To: Jakarta Commons Developers List
>> Subject: RE: [simplestore] enhancements (was: [simplestore] inital check
>> in)
>>
>>
>> >Maybe I understand you wrong, but look at this:
>> >In the Cocoon project we have 2 stores a) MRUMemoryStore for quick access
>> >and b) JispFilesystemStore for -well- swapping. Why? Because we
>> store/cache
>> >generated data (xml -> xslt -> html or fop or ...) in the cache.
>> Generating
>> >this data is very expensive.
>>
>>
>>
>> Ok, but it is not memory management, it is optimization for generation.
>
>It is reference management.
I think we have definition problem here. I don't wanna re-build the memory
management of the JVM. I just wanna provide some _small_, _quick_ and _ease_
to plug components where people can store -well- *things*. Like in the Cocoon
project generated HTML or PDF files to spare System resources, or a Application
Server which pools Objects, or whatever. Object in and throw Object out.
The memory management of the JVM is doing the rest.
We have to consider how we can use the JVM memory management and the GC in an
most perfect way. Solution a) was the StoreJanitor which derefences Objects and
forces the GC after that or solution b) WeakReferences which seem to be a very
elegant solution. I'm open for everything!
That's the picture I have. What's yours?
Gerhard
----------------------------
You can't fall off the floor
----------------------------
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>