You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cocoon.apache.org by Berin Loritsch <bl...@apache.org> on 2001/12/10 19:56:36 UTC
[RT] Unification of Source/Resource Management (was Re: Adding Resource Monitor to Generators)
For those of you who want to make XIncludeTransformer and CIncludeTransformer
cacheable, we have to address some major issues. One of which is that the
base Source interface does not permit cacheability. Also, we want to make
the Cache invisible to the rest of the system. Therefore, I propose some
organizational changes:
1) All Sources must be made a Resource to allow notifications if the source
has changed. The best option would be to merge the concepts. I personally
like the name Resource as it does not cognitively restrict you to read
only resources.
2) The URLFactory must be removed, and it's functionality merged into SourceHandler.
The SourceHandler must allow the mapping of protocols to the Resource types.
The new Component should be called a ResourceManager.
3) The ResourceManager is responsible for obtaining the handle to the resource
in question. It makes sure that the resource returned is current, and
manages the Resource cache entries.
4) All accesses to resources go through the ResourceManager.
The interface should be something like this:
interface ResourceManager {
getResource(String uri) throws IOException;
getResource(String uri, boolean create) throws IOException;
}
It is *intentionally* simple, and does not supply "base" URIs
because the configuration should handle what the "base" URI is.
Notice that there is no Environment object (which is decidedly
Cocoon centric and muddies the concept of what it should do).
Also notice that the second method has the boolean "create",
that way we can create a resource to write for bi-directional
pipelines.
COCOON IMPLEMENTATION DETAILS
-----------------------------
The ResourceManager should be made Contextualizable so that it can
get the needed reference to the HttpContext object. It should be
made Configurable so that it can understand how to map protocols to
Resource classes. It should work with the Cache system and the
Monitor system.
If the Resource type is Cacheable, then the ResourceManager will
take care of the Cache plumbing. IOW, it will returned the cached
Resource (if available), or create the new reference and add it to
the cache. Also, if the Resource is used in the caching system,
it is added to the Monitor, and the Cache system is passed as the
PropertyChangedListener--either that, or a different object that
manages the interaction between the two is used.
Confused yet? Good.
STORE UNIFICATION
-----------------
Currently, we have three implementations of Store, and only one
is used. To make things worse, there are three roles to which
the one implementation is referenced: Store, Stream-Cache, and
Event-Cache. I hope you realize that with 3 different roles
in the system, there are at least 3 different instances--even
if they are declared ThreadSafe! That means you have three
instances of MRUMemoryStore all pointing to the same repository!
There are bound to be resource conflicts. This must be fixed.
In order to remedy this situation, we have to refine exactly
what the store contracts are. We used to have two ideas:
Persistent Store, and Volitile Store. The implementations
were FilesystemStore and MemoryStore respectively. However,
with the flexibility of the MRUMemoryStore, this is no longer
necessary. We simply only need the role of "Store".
Let us decide here and now: Do we need to distinguish between
Persistent Store and Volitile Store? I do not think this is
necessary any longer. There is absolutely no reason to
differentiate between Event and Stream storeage with the
Store interface--when both implementations are the same.
Am I making sense? Hopefully we can clean up some cruft that
we have already accumulated.
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [RT] Unification of Source/Resource Management (was Re: Adding Resource Monitor to Generators)
Posted by Berin Loritsch <bl...@apache.org>.
Stefano Mazzocchi wrote:
> Thanks for pointing this out!
>
> Berin Loritsch wrote:
<snip/>
>>Also notice that the second method has the boolean "create",
>>that way we can create a resource to write for bi-directional
>>pipelines.
>>
>
> Hmmm, don't find myself resonating this this. Can you elaborate further?
The problem occurs if you want to open a resource that is not there.
For instance, if we want to write some information to the filesystem
in a file named "foo.xml", we would be thrown an exception if it was
not there.
This is a problem if we are actually creating the resource for the
first time!
>>Let us decide here and now: Do we need to distinguish between
>>Persistent Store and Volitile Store?
>>
>
> No, I don't think so. It's not hard to come up with heuristics that take
> care of this and adapt to the current caching needs (see that RT for all
> the algorithms that implement that!)
>
>
>>I do not think this is
>>necessary any longer. There is absolutely no reason to
>>differentiate between Event and Stream storeage with the
>>Store interface--when both implementations are the same.
>>
>>Am I making sense? Hopefully we can clean up some cruft that
>>we have already accumulated.
>>
>
> Definately +1, I'm very happy to are bringing this further.
:)
I am back on a Cocoon centered project, so I am happy to begin contributing
again (beyond just Avalon maintenance.
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
RE: [RT] Unification of Source/Resource Management (was Re: Adding Resource Monitor to Generators)
Posted by Carsten Ziegeler <cz...@s-und-n.de>.
Carsten Ziegeler wrote:
>
> Hm, some weeks ago we all agreed to move the Source components to Avalon
> as they are not Cocoon specific but usable for every project.
> I started with this move weeks ago, but never had time to finish it....
> You will find all this in the scratchpad of excalibur.
> The interface are based on Cocoon, but cleaned up! The missing
> functionality
> is the link to the monitoring.
> I really don't want to have monitoring for every Source, I think, this is
> simply overkill.
And how will you monitor such protocols as the xml:db protocol or a cvs
protocol? So these are really two different area: describing/obtaining
resources
and monitoring them.
> So I could imagine something like a Monitorable interface for a Source
> object
> with a getResource() method.
> I agree, that Resource is a better name than Source, but there are already
> many getResource() method throughout the JDK used, so using a new name
> shows immediately what's going on.
>
> So I think, we should first finish the work on the Avalon side - this can
> either be my proposal above or something completly different - and then
> use this in Cocoon.
>
Carsten
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
RE: [RT] Unification of Source/Resource Management (was Re: Adding Resource Monitor to Generators)
Posted by Carsten Ziegeler <cz...@s-und-n.de>.
Stefano Mazzocchi wrote:
>
> Thanks for pointing this out!
>
> Berin Loritsch wrote:
> >
> > For those of you who want to make XIncludeTransformer and
> CIncludeTransformer
> > cacheable, we have to address some major issues. One of which
> is that the
> > base Source interface does not permit cacheability. Also, we
> want to make
> > the Cache invisible to the rest of the system. Therefore, I
> propose some
> > organizational changes:
> >
> > 1) All Sources must be made a Resource to allow notifications
> if the source
> > has changed. The best option would be to merge the
> concepts. I personally
> > like the name Resource as it does not cognitively restrict
> you to read
> > only resources.
>
> Totally agreed.
>
Hm, some weeks ago we all agreed to move the Source components to Avalon
as they are not Cocoon specific but usable for every project.
I started with this move weeks ago, but never had time to finish it....
You will find all this in the scratchpad of excalibur.
The interface are based on Cocoon, but cleaned up! The missing functionality
is the link to the monitoring.
I really don't want to have monitoring for every Source, I think, this is
simply overkill.
So I could imagine something like a Monitorable interface for a Source
object
with a getResource() method.
I agree, that Resource is a better name than Source, but there are already
many getResource() method throughout the JDK used, so using a new name
shows immediately what's going on.
So I think, we should first finish the work on the Avalon side - this can
either be my proposal above or something completly different - and then
use this in Cocoon.
> > 2) The URLFactory must be removed, and it's functionality
> merged into SourceHandler.
> > The SourceHandler must allow the mapping of protocols to
> the Resource types.
> > The new Component should be called a ResourceManager.
>
> Good choice.
>
Done in the excalibur scratchpad code (although the name differs)
> > 3) The ResourceManager is responsible for obtaining the handle
> to the resource
> > in question. It makes sure that the resource returned is
> current, and
> > manages the Resource cache entries.
> >
> > 4) All accesses to resources go through the ResourceManager.
>
> Sounds good.
>
Yupp, but I call it SourceResolver (hey, this is just a teaser).
> > The interface should be something like this:
> >
> > interface ResourceManager {
> > getResource(String uri) throws IOException;
> >
> > getResource(String uri, boolean create) throws IOException;
> > }
> >
> > It is *intentionally* simple, and does not supply "base" URIs
> > because the configuration should handle what the "base" URI is.
> > Notice that there is no Environment object (which is decidedly
> > Cocoon centric and muddies the concept of what it should do).
>
> good.
>
Yupp, Environment is of course removed in the scratchpad code.
But, you might need a base URI. For example in Cocoon you have
more than one sitemap. Now if you only have one ResourceManager
(SourceResolver, Trashcan or whatever) it needs to know to what
sitemap the resource has to be resolved. So here you have a base
URI (the locating of the actual sitemap) and a (perhaps) relative
source definition.
Carsten
> > Also notice that the second method has the boolean "create",
> > that way we can create a resource to write for bi-directional
> > pipelines.
>
> Hmmm, don't find myself resonating this this. Can you elaborate further?
>
> > COCOON IMPLEMENTATION DETAILS
> > -----------------------------
> >
> > The ResourceManager should be made Contextualizable so that it can
> > get the needed reference to the HttpContext object.
>
> absolutely
>
> > It should be
> > made Configurable so that it can understand how to map protocols to
> > Resource classes. It should work with the Cache system and the
> > Monitor system.
>
> correct
>
> > If the Resource type is Cacheable, then the ResourceManager will
> > take care of the Cache plumbing. IOW, it will returned the cached
> > Resource (if available), or create the new reference and add it to
> > the cache. Also, if the Resource is used in the caching system,
> > it is added to the Monitor, and the Cache system is passed as the
> > PropertyChangedListener--either that, or a different object that
> > manages the interaction between the two is used.
> >
> > Confused yet? Good.
> >
> > STORE UNIFICATION
> > -----------------
> >
> > Currently, we have three implementations of Store, and only one
> > is used. To make things worse, there are three roles to which
> > the one implementation is referenced: Store, Stream-Cache, and
> > Event-Cache. I hope you realize that with 3 different roles
> > in the system, there are at least 3 different instances--even
> > if they are declared ThreadSafe! That means you have three
> > instances of MRUMemoryStore all pointing to the same repository!
> >
> > There are bound to be resource conflicts. This must be fixed.
>
> Totally agreed.
>
> > In order to remedy this situation, we have to refine exactly
> > what the store contracts are. We used to have two ideas:
> > Persistent Store, and Volitile Store. The implementations
> > were FilesystemStore and MemoryStore respectively. However,
> > with the flexibility of the MRUMemoryStore, this is no longer
> > necessary. We simply only need the role of "Store".
>
> This is what I've been proposing since my very first RT about caching
> (which largerly when unimplemented, but I still have them in my todo
> list!). A 'good' store implementation should know how to manage its
> resources in such a way that optimizes use and overall performance.
>
> having two store behaviors overlaps concerns.
>
> > Let us decide here and now: Do we need to distinguish between
> > Persistent Store and Volitile Store?
>
> No, I don't think so. It's not hard to come up with heuristics that take
> care of this and adapt to the current caching needs (see that RT for all
> the algorithms that implement that!)
>
> > I do not think this is
> > necessary any longer. There is absolutely no reason to
> > differentiate between Event and Stream storeage with the
> > Store interface--when both implementations are the same.
> >
> > Am I making sense? Hopefully we can clean up some cruft that
> > we have already accumulated.
>
> Definately +1, I'm very happy to are bringing this further.
>
> --
> Stefano Mazzocchi One must still have chaos in oneself to be
> able to give birth to a dancing star.
> <st...@apache.org> Friedrich Nietzsche
> --------------------------------------------------------------------
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
> For additional commands, email: cocoon-dev-help@xml.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [RT] Unification of Source/Resource Management (was Re: Adding
Resource Monitor to Generators)
Posted by Stefano Mazzocchi <st...@apache.org>.
Thanks for pointing this out!
Berin Loritsch wrote:
>
> For those of you who want to make XIncludeTransformer and CIncludeTransformer
> cacheable, we have to address some major issues. One of which is that the
> base Source interface does not permit cacheability. Also, we want to make
> the Cache invisible to the rest of the system. Therefore, I propose some
> organizational changes:
>
> 1) All Sources must be made a Resource to allow notifications if the source
> has changed. The best option would be to merge the concepts. I personally
> like the name Resource as it does not cognitively restrict you to read
> only resources.
Totally agreed.
> 2) The URLFactory must be removed, and it's functionality merged into SourceHandler.
> The SourceHandler must allow the mapping of protocols to the Resource types.
> The new Component should be called a ResourceManager.
Good choice.
> 3) The ResourceManager is responsible for obtaining the handle to the resource
> in question. It makes sure that the resource returned is current, and
> manages the Resource cache entries.
>
> 4) All accesses to resources go through the ResourceManager.
Sounds good.
> The interface should be something like this:
>
> interface ResourceManager {
> getResource(String uri) throws IOException;
>
> getResource(String uri, boolean create) throws IOException;
> }
>
> It is *intentionally* simple, and does not supply "base" URIs
> because the configuration should handle what the "base" URI is.
> Notice that there is no Environment object (which is decidedly
> Cocoon centric and muddies the concept of what it should do).
good.
> Also notice that the second method has the boolean "create",
> that way we can create a resource to write for bi-directional
> pipelines.
Hmmm, don't find myself resonating this this. Can you elaborate further?
> COCOON IMPLEMENTATION DETAILS
> -----------------------------
>
> The ResourceManager should be made Contextualizable so that it can
> get the needed reference to the HttpContext object.
absolutely
> It should be
> made Configurable so that it can understand how to map protocols to
> Resource classes. It should work with the Cache system and the
> Monitor system.
correct
> If the Resource type is Cacheable, then the ResourceManager will
> take care of the Cache plumbing. IOW, it will returned the cached
> Resource (if available), or create the new reference and add it to
> the cache. Also, if the Resource is used in the caching system,
> it is added to the Monitor, and the Cache system is passed as the
> PropertyChangedListener--either that, or a different object that
> manages the interaction between the two is used.
>
> Confused yet? Good.
>
> STORE UNIFICATION
> -----------------
>
> Currently, we have three implementations of Store, and only one
> is used. To make things worse, there are three roles to which
> the one implementation is referenced: Store, Stream-Cache, and
> Event-Cache. I hope you realize that with 3 different roles
> in the system, there are at least 3 different instances--even
> if they are declared ThreadSafe! That means you have three
> instances of MRUMemoryStore all pointing to the same repository!
>
> There are bound to be resource conflicts. This must be fixed.
Totally agreed.
> In order to remedy this situation, we have to refine exactly
> what the store contracts are. We used to have two ideas:
> Persistent Store, and Volitile Store. The implementations
> were FilesystemStore and MemoryStore respectively. However,
> with the flexibility of the MRUMemoryStore, this is no longer
> necessary. We simply only need the role of "Store".
This is what I've been proposing since my very first RT about caching
(which largerly when unimplemented, but I still have them in my todo
list!). A 'good' store implementation should know how to manage its
resources in such a way that optimizes use and overall performance.
having two store behaviors overlaps concerns.
> Let us decide here and now: Do we need to distinguish between
> Persistent Store and Volitile Store?
No, I don't think so. It's not hard to come up with heuristics that take
care of this and adapt to the current caching needs (see that RT for all
the algorithms that implement that!)
> I do not think this is
> necessary any longer. There is absolutely no reason to
> differentiate between Event and Stream storeage with the
> Store interface--when both implementations are the same.
>
> Am I making sense? Hopefully we can clean up some cruft that
> we have already accumulated.
Definately +1, I'm very happy to are bringing this further.
--
Stefano Mazzocchi One must still have chaos in oneself to be
able to give birth to a dancing star.
<st...@apache.org> Friedrich Nietzsche
--------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [RT] Unification of Source/Resource Management (was Re: Adding Resource Monitor to Generators)
Posted by Maciek Kaminski <ma...@tiger.com.pl>.
On 11 Dec 2001, at 15:41, Sylvain Wallez wrote:
>
> I'm not really aware of the issues in IncludeTransformers. Could you
> please elaborate more ?
>
See: http://nagoya.apache.org/bugzilla/show_bug.cgi?id=4641
Maciek Kaminski
maciejka@tiger.com.pl
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [RT] Unification of Source/Resource Management (was Re: Adding
Resource Monitor to Generators)
Posted by Sylvain Wallez <sy...@anyware-tech.com>.
Berin Loritsch a écrit :
>
> For those of you who want to make XIncludeTransformer and CIncludeTransformer
> cacheable, we have to address some major issues. One of which is that the
> base Source interface does not permit cacheability. Also, we want to make
> the Cache invisible to the rest of the system. Therefore, I propose some
> organizational changes:
I'm not really aware of the issues in IncludeTransformers. Could you
please elaborate more ?
> 1) All Sources must be made a Resource to allow notifications if the source
> has changed. The best option would be to merge the concepts. I personally
> like the name Resource as it does not cognitively restrict you to read
> only resources.
+1 to merge both as long as it doesn't enforce the use or ActiveMonitor.
> 2) The URLFactory must be removed, and it's functionality merged into SourceHandler.
> The SourceHandler must allow the mapping of protocols to the Resource types.
> The new Component should be called a ResourceManager.
SourceHandler methods have an Environment parameter which limits its use
to the context of a sitemap. This is mainly to resolve cocoon: URLs
relative to the current sitemap.
We could duplicate the methods without the Environment parameter and let
the implementations decide whether they allow no environment or not. For
example, CocoonSourceFactory could allow no environment for cocoon://
(absolute URLs) and disallow it for cocoon:/ (relative).
> 3) The ResourceManager is responsible for obtaining the handle to the resource
> in question. It makes sure that the resource returned is current, and
> manages the Resource cache entries.
>
> 4) All accesses to resources go through the ResourceManager.
>
> The interface should be something like this:
>
> interface ResourceManager {
> getResource(String uri) throws IOException;
>
> getResource(String uri, boolean create) throws IOException;
> }
>
> It is *intentionally* simple, and does not supply "base" URIs
> because the configuration should handle what the "base" URI is.
> Notice that there is no Environment object (which is decidedly
> Cocoon centric and muddies the concept of what it should do).
> Also notice that the second method has the boolean "create",
> that way we can create a resource to write for bi-directional
> pipelines.
If Environment is removed, how can we resolve relative cocoon: URLs ?
> COCOON IMPLEMENTATION DETAILS
> -----------------------------
>
> The ResourceManager should be made Contextualizable so that it can
> get the needed reference to the HttpContext object. It should be
> made Configurable so that it can understand how to map protocols to
> Resource classes. It should work with the Cache system and the
> Monitor system.
Why is the HttpContext object needed ? What about CommandLineEnvironment
?
> If the Resource type is Cacheable, then the ResourceManager will
> take care of the Cache plumbing. IOW, it will returned the cached
> Resource (if available), or create the new reference and add it to
> the cache. Also, if the Resource is used in the caching system,
> it is added to the Monitor, and the Cache system is passed as the
> PropertyChangedListener--either that, or a different object that
> manages the interaction between the two is used.
>
> Confused yet? Good.
Not that much, but after yesterday's discussion, I had a hard night full
of thoughts about all this stuff and even if I now understand the exact
purpose of ActiveMonitor, I'm still afraid of both the size of the
resource Map and the cost of resource scanning on a large site just for
discarding cache entries.
Is it really worth it ? Can't we let entries sleep in the cache until
they are actually used (and checked) or until the MRU throws them out ?
What about the delayed call to getLastModified() I proposed yesterday ?
This could be handled by a DelayedCacheValidity which delays calls to
the filesystem while still enforcing a limited age of the lastModified
information. I'll try to set up a concrete example of that in the coming
days (free time is scarce).
Another thought : the scan period cannot IMO be one and unique for all
resources. There are some resources which can/should be checked more
frequently (e.g. file:) than others (e.g. http:). This configuration
could be part of the definition of the various source implementations.
How can the ActiveMonitor handle this ?
Now let me clearly make the point : ActiveMonitor is a nice idea. But
IMO it should only be used when event listeners do a job that
functionnaly has to be done quickly after the resource changes at the
price of periodic scanning. And I'm not sure "automagic" purge of the
cache fits into this category.
Sylvain.
> STORE UNIFICATION
> -----------------
>
> Currently, we have three implementations of Store, and only one
> is used. To make things worse, there are three roles to which
> the one implementation is referenced: Store, Stream-Cache, and
> Event-Cache. I hope you realize that with 3 different roles
> in the system, there are at least 3 different instances--even
> if they are declared ThreadSafe! That means you have three
> instances of MRUMemoryStore all pointing to the same repository!
>
> There are bound to be resource conflicts. This must be fixed.
>
> In order to remedy this situation, we have to refine exactly
> what the store contracts are. We used to have two ideas:
> Persistent Store, and Volitile Store. The implementations
> were FilesystemStore and MemoryStore respectively. However,
> with the flexibility of the MRUMemoryStore, this is no longer
> necessary. We simply only need the role of "Store".
>
> Let us decide here and now: Do we need to distinguish between
> Persistent Store and Volitile Store? I do not think this is
> necessary any longer. There is absolutely no reason to
> differentiate between Event and Stream storeage with the
> Store interface--when both implementations are the same.
>
> Am I making sense? Hopefully we can clean up some cruft that
> we have already accumulated.
>
> --
>
> "They that give up essential liberty to obtain a little temporary safety
> deserve neither liberty nor safety."
> - Benjamin Franklin
>
--
Sylvain Wallez
Anyware Technologies - http://www.anyware-tech.com
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
RE: [RT] Unification of Source/Resource Management (was Re: Adding Resource Monitor to Generators)
Posted by Gerhard Froehlich <g-...@gmx.de>.
>From: Berin Loritsch [mailto:bloritsch@apache.org]
>
>
>Gerhard Froehlich wrote:
>
>>>From: Berin Loritsch [mailto:bloritsch@apache.org]
>>>
>>
>> </snip>
>>
>>>COCOON IMPLEMENTATION DETAILS
>>>-----------------------------
>>>
>>>The ResourceManager should be made Contextualizable so that it can
>>>get the needed reference to the HttpContext object. It should be
>>>made Configurable so that it can understand how to map protocols to
>>>Resource classes. It should work with the Cache system and the
>>>Monitor system.
>>>
>>>If the Resource type is Cacheable, then the ResourceManager will
>>>take care of the Cache plumbing. IOW, it will returned the cached
>>>Resource (if available), or create the new reference and add it to
>>>the cache. Also, if the Resource is used in the caching system,
>>>it is added to the Monitor, and the Cache system is passed as the
>>>PropertyChangedListener--either that, or a different object that
>>>manages the interaction between the two is used.
>>>
>>
>> Why so modestly. I thought the Monitor can be used by every Resource,
>> independent if it's Cachable or not. Like the Configuration file(s), etc.
>> How about a interface which marks that Resource Monitorable
>> (bad name, but you know what I mean), then ResourceManager could decide
>> if he puts the Resource to the Monitor or not...
>
>
>Think in terms of WHAT needs to be done and WHO needs to do it. Let's
>say for instance, that the configuration file changed (cocoon.xconf).
>It is the CocoonServlet's responsibility to create a new Cocoon instance
>with the new configuration file, dispose of the old one, and instantiate
>the new one. Cocoon is not Reconfigurable, so it should never reconfigure
>itself. This is known as the antipattern Subversion of Control.
>
>Also, a resource only needs to be monitored as long as it is in use.
>ProgramGenerator needs to know if any of it's source files have been
>altered so that it can recompile the resource and reload it in the
>classloader. The Cache system only needs to track something as long
>as it is in the cache.
>
>At all other times, the file can be changed a million times, but if
>it has never been needed why do we need to consume resources observing
>it? That would end up with too many change events being fired with
>noone listening.
>
Ok this is my 6th try to write an intelligent contribution to this
issue ;-).
My main problem is I don't really understand how do you want do integrate
this ResourceManagemement this central in the existing Pipline architecture
and how it would be accessable for non Pipline relevant Resources.
I think you can't integrate a PropertyChangeListener this easy into the
Piplines. I think this PropertyChangeListener listener should be nearer
at the Resource. Because when you append it on the Generators for i.e.
we ran into the same problem as I with the ProgramGeneratorImpl.
Errr...Maybe I think to complicated here.
Gerhard
"Three o'clock is always too late or too early for
anything you want to do.
(Jean-Paul Sartre)"
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [RT] Unification of Source/Resource Management (was Re: Adding Resource Monitor to Generators)
Posted by Berin Loritsch <bl...@apache.org>.
Gerhard Froehlich wrote:
>>From: Berin Loritsch [mailto:bloritsch@apache.org]
>>
>
> </snip>
>
>>COCOON IMPLEMENTATION DETAILS
>>-----------------------------
>>
>>The ResourceManager should be made Contextualizable so that it can
>>get the needed reference to the HttpContext object. It should be
>>made Configurable so that it can understand how to map protocols to
>>Resource classes. It should work with the Cache system and the
>>Monitor system.
>>
>>If the Resource type is Cacheable, then the ResourceManager will
>>take care of the Cache plumbing. IOW, it will returned the cached
>>Resource (if available), or create the new reference and add it to
>>the cache. Also, if the Resource is used in the caching system,
>>it is added to the Monitor, and the Cache system is passed as the
>>PropertyChangedListener--either that, or a different object that
>>manages the interaction between the two is used.
>>
>
> Why so modestly. I thought the Monitor can be used by every Resource,
> independent if it's Cachable or not. Like the Configuration file(s), etc.
> How about a interface which marks that Resource Monitorable
> (bad name, but you know what I mean), then ResourceManager could decide
> if he puts the Resource to the Monitor or not...
Think in terms of WHAT needs to be done and WHO needs to do it. Let's
say for instance, that the configuration file changed (cocoon.xconf).
It is the CocoonServlet's responsibility to create a new Cocoon instance
with the new configuration file, dispose of the old one, and instantiate
the new one. Cocoon is not Reconfigurable, so it should never reconfigure
itself. This is known as the antipattern Subversion of Control.
Also, a resource only needs to be monitored as long as it is in use.
ProgramGenerator needs to know if any of it's source files have been
altered so that it can recompile the resource and reload it in the
classloader. The Cache system only needs to track something as long
as it is in the cache.
At all other times, the file can be changed a million times, but if
it has never been needed why do we need to consume resources observing
it? That would end up with too many change events being fired with
noone listening.
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
RE: [RT] Unification of Source/Resource Management (was Re: Adding Resource Monitor to Generators)
Posted by Gerhard Froehlich <g-...@gmx.de>.
>From: Berin Loritsch [mailto:bloritsch@apache.org]
</snip>
>COCOON IMPLEMENTATION DETAILS
>-----------------------------
>
>The ResourceManager should be made Contextualizable so that it can
>get the needed reference to the HttpContext object. It should be
>made Configurable so that it can understand how to map protocols to
>Resource classes. It should work with the Cache system and the
>Monitor system.
>
>If the Resource type is Cacheable, then the ResourceManager will
>take care of the Cache plumbing. IOW, it will returned the cached
>Resource (if available), or create the new reference and add it to
>the cache. Also, if the Resource is used in the caching system,
>it is added to the Monitor, and the Cache system is passed as the
>PropertyChangedListener--either that, or a different object that
>manages the interaction between the two is used.
Why so modestly. I thought the Monitor can be used by every Resource,
independent if it's Cachable or not. Like the Configuration file(s), etc.
How about a interface which marks that Resource Monitorable
(bad name, but you know what I mean), then ResourceManager could decide
if he puts the Resource to the Monitor or not...
Just shy thoughts.
>Confused yet? Good.
</snip>
Gerhard
--------------------------------------------------------------
By making things absolutely clear, people will become confused
-------------------------------------------------------------->
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
RE: [RT] Unification of Source/Resource Management (was Re: Adding Resource Monitor to Generators)
Posted by Gerhard Froehlich <g-...@gmx.de>.
Berin,
some latly (inline) comments,
>From: Berin Loritsch [mailto:bloritsch@apache.org]
>
>
>For those of you who want to make XIncludeTransformer and CIncludeTransformer
>cacheable, we have to address some major issues. One of which is that the
>base Source interface does not permit cacheability. Also, we want to make
>the Cache invisible to the rest of the system. Therefore, I propose some
>organizational changes:
>
>1) All Sources must be made a Resource to allow notifications if the source
> has changed. The best option would be to merge the concepts. I personally
> like the name Resource as it does not cognitively restrict you to read
> only resources.
Extended from org.apache.avalon.excalibur.monitor.Resource, I guess.
>2) The URLFactory must be removed, and it's functionality merged into SourceHandler.
> The SourceHandler must allow the mapping of protocols to the Resource types.
> The new Component should be called a ResourceManager.
>
>3) The ResourceManager is responsible for obtaining the handle to the resource
> in question. It makes sure that the resource returned is current, and
> manages the Resource cache entries.
>
>4) All accesses to resources go through the ResourceManager.
>
That makes all sense to me, but I see problems with the different protocols. Of
what kind of Resources do you speak? Remember your "conundrum"...
>
>The interface should be something like this:
>
>interface ResourceManager {
> getResource(String uri) throws IOException;
>
> getResource(String uri, boolean create) throws IOException;
>}
>
>It is *intentionally* simple, and does not supply "base" URIs
>because the configuration should handle what the "base" URI is.
>Notice that there is no Environment object (which is decidedly
>Cocoon centric and muddies the concept of what it should do).
>Also notice that the second method has the boolean "create",
>that way we can create a resource to write for bi-directional
>pipelines.
>
>COCOON IMPLEMENTATION DETAILS
>-----------------------------
>
>The ResourceManager should be made Contextualizable so that it can
>get the needed reference to the HttpContext object. It should be
>made Configurable so that it can understand how to map protocols to
>Resource classes. It should work with the Cache system and the
>Monitor system.
>
>If the Resource type is Cacheable, then the ResourceManager will
>take care of the Cache plumbing. IOW, it will returned the cached
>Resource (if available), or create the new reference and add it to
>the cache. Also, if the Resource is used in the caching system,
>it is added to the Monitor, and the Cache system is passed as the
>PropertyChangedListener--either that, or a different object that
>manages the interaction between the two is used.
>
>Confused yet? Good.
Yeah, therefore I will stop know working on the Resource Monitor
in the current design. I see a lawine coming....
>STORE UNIFICATION
>-----------------
>
>Currently, we have three implementations of Store, and only one
>is used. To make things worse, there are three roles to which
>the one implementation is referenced: Store, Stream-Cache, and
>Event-Cache. I hope you realize that with 3 different roles
>in the system, there are at least 3 different instances--even
>if they are declared ThreadSafe! That means you have three
>instances of MRUMemoryStore all pointing to the same repository!
>
>There are bound to be resource conflicts. This must be fixed.
I mentioned this more then one time in this list, but somehow
I don't get an ear here.
>In order to remedy this situation, we have to refine exactly
>what the store contracts are. We used to have two ideas:
>Persistent Store, and Volitile Store. The implementations
>were FilesystemStore and MemoryStore respectively. However,
>with the flexibility of the MRUMemoryStore, this is no longer
>necessary. We simply only need the role of "Store".
(Be careful the FilesystemStore is used over the whole Cocoon
core. His major role was to serialize for i.e the compiled
Sitemap to the Filesystem. But this things cut be covered
up by a other component)
>Let us decide here and now: Do we need to distinguish between
>Persistent Store and Volitile Store? I do not think this is
>necessary any longer. There is absolutely no reason to
>differentiate between Event and Stream storeage with the
>Store interface--when both implementations are the same.
Well, for me it doesn't make any sense to have 3 complete Stores on
runtime. Neither for the Event-Store, Stream-Store and Store (which
is really dubious). They seems to me the same! The only argument
therefor was the flexibility to have different Stores for different
Piplines. But somehow that's like repeating himself.
The existing way as the MRUMemoryStore holds objects in
a Persistent state is far away of being optimal. When you
kick the FilesystemStore we have to change this implementation,
either. But that would be good!
For some days I made a RT to improve this with something more
sexy. See the Thread "[RT] Jisp as Store Persistence" for
more information.
May picture of a kool Store system would be:
- A fast memory store for current requested objects (like the
MRUMemoryStore as now). Independent to the Caching implementation,
of course.
- A fast way to write/read objects to the filesystem to
hold them in a persitence way. Maybe like the Jisp implementation
in an indexed data file for fast access!
>Am I making sense? Hopefully we can clean up some cruft that
>we have already accumulated.
Yeah would be cool
Gerhard
--------------------------
Hey! It compiles! Ship it!
--------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org