You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cocoon.apache.org by Unico Hommes <un...@hippo.nl> on 2004/03/06 22:35:58 UTC
Re: cvs commit: cocoon-2.1/src/blocks/scratchpad/java/org/apache/cocoon/components/source/impl
CachingSource.java AsyncCachingSource.java RefresherImpl.java
haul@apache.org wrote:
> haul 2004/03/06 13:00:39
>
> Modified: src/blocks/scratchpad/java/org/apache/cocoon/components/source/impl
> CachingSource.java AsyncCachingSource.java
> RefresherImpl.java
> Log:
> Add log statements for swallowed exceptions
> Extract some methods in RefresherImpl
> Extract some constants in RefresherImpl
> (IOW just moving things around)
>
> Revision Changes Path
> 1.6 +18 -4 cocoon-2.1/src/blocks/scratchpad/java/org/apache/cocoon/components/source/impl/CachingSource.java
>
>
:-( I had so many changes on my local FS for these classes. I totally
reworked their implemenatations. Not your fault though. I should have
worked more incrementally and checked them in sooner. I'll find a way to
merge it all.
Just to give you an idea of what I am doing here is a quick summary:
Basically I want to move all communication with the Cache into the
Refresher. I am renaming RefresherImpl to DelayRefresher and adding a
Refresher implementation that is externally triggered by events. I am
changing the location protocol string as discussed earlier. Probably can
merge CachingSource and AsynchCachingSource.
I also changed the way DelayRefresher persists its entry configurations
to instead of doing a manual xml serialization of the data just dump the
whole Map of entries into a persistent store.
By the way I think you are the one to ask Christian: I noticed there is
no scheduler.addPeriodicJob(.. Object job, ..) method. Is there a reason
for this or can I add one? So that the Refresher can add itself and its
update targets directly.
Cheers,
Unico
Re: cvs commit: cocoon-2.1/src/blocks/scratchpad/java/org/apache/cocoon/components/source/impl
CachingSource.java AsyncCachingSource.java RefresherImpl.java
Posted by Christian Haul <ha...@informatik.tu-darmstadt.de>.
Unico Hommes wrote:
>
> Christian Haul wrote:
>
>> Unico Hommes wrote:
>> However, I'm not quite sure if this is the way to go. I'm currently
>> thinking of intercepting and wrapping sources at the selector level.
>>
>> With the current setup, we need to add the configuration to the URL
>> in all places it is used. By wrapping a protocol with a cached version
>> we wouldn't be able to distinguish eg access to a local services from
>> a remote service (in terms of network hops).
>>
>> So what if we were to intercept those sources at the selector based
>> on eg wildcard matching on URL? This would be "cross-cutting" and it
>> would be easy to provide more complex caching parameters for a wrapped
>> source.
>>
>
> For the life of me I can't figure out what exactly it is you are saying.
> Care to elaborate? Perhaps give an example?
Say, you would like to access you webdav always asynchronously as well
as remote webservices. Local services, however, should be accessed
directly.
Solution 1 is to go through your sitemap and wrap your URLs accordingly.
Solution 2 is to say, for "http://*.localsite.com/*" use plain http, for
everything else use "acached://600@{0}" (or a better syntax that doesn't
encode everything into the URL).
To achieve this, one could replace the component selector holding the
source factories and always return a wrapper. Only when the real URL
is available (eg absolutize() or get() is called), the decision is made
which factory to use.
>> Esp. with a event based cache, how would you specify the event to
>> invalidate the source from within the URL?
>>
>
> I was thinking to use querystring parameter to pass it in and default to
> the wrapped Source's system id (getURI). This is what I am using right
> now for WebDAV sources and it works perfectly.
Ah well. I have Documentum DQL as URL and would need to have something
else for a key, like the document ID or a folder name.
>>
>>> By the way I think you are the one to ask Christian: I noticed there
>>> is no scheduler.addPeriodicJob(.. Object job, ..) method. Is there a
>>> reason for this or can I add one? So that the Refresher can add
>>> itself and its update targets directly.
>>
>>
>>
>> Although I'd like to help I haven't touched the Cron block before. But
>> speaking of addPeriodicJob() it would be great to have a way to say
>> "do this right now AND repeat every 10 minutes". The current
>> implementation
>> provides only for a "do this every 10 minutes and start in 10 minutes
>> for the first time". IIUC this limitation does not stem from the Qartz
>> API but from what the Cocoon JobScheduler provides.
>>
>
> Yeah it does. There already exists a plain addJob method with the
> signature I am talking about.
But AFAIK there's either "do now", "do at cron spec" (like mondays at
12am), or "do every x minutes starting in x minutes". Of course, what
I would like to have is just a combination of addPeriodicJob() and
fireJob() ....
>> Speaking of random ideas around the cached source, it might be nice to
>> provide a URL for the content when it's not yet available. But then this
>> might add too much features to a supposed to be simple concept.
>>
>
> I was also thinking of an implementation of the update mechanism where
> objects don't get invalidated if they cannot be regenerated (eg. due to
> WebDAV server offline, network problems, etc.) but get rescheduled instead.
Makes sense.
Chris.
Re: cvs commit: cocoon-2.1/src/blocks/scratchpad/java/org/apache/cocoon/components/source/impl
CachingSource.java AsyncCachingSource.java RefresherImpl.java
Posted by Unico Hommes <un...@hippo.nl>.
Christian Haul wrote:
> Unico Hommes wrote:
>
>> :-( I had so many changes on my local FS for these classes. I totally
>> reworked their implemenatations. Not your fault though. I should have
>> worked more incrementally and checked them in sooner. I'll find a way
>> to merge it all.
>
>
> Well, I wouldn't mind if you just dump my changes. As the message said,
> I was just moving things around for documentation purposes, really.
> (Some prefer comments, some prefer speaking names for rather small methods)
>
>> Just to give you an idea of what I am doing here is a quick summary:
>>
>> Basically I want to move all communication with the Cache into the
>> Refresher. I am renaming RefresherImpl to DelayRefresher and adding a
>> Refresher implementation that is externally triggered by events. I am
>> changing the location protocol string as discussed earlier. Probably
>> can merge CachingSource and AsynchCachingSource.
>
>
> This sound really cool and actually, I was planning for something
> similar :-)
>
> However, I'm not quite sure if this is the way to go. I'm currently
> thinking of intercepting and wrapping sources at the selector level.
>
> With the current setup, we need to add the configuration to the URL
> in all places it is used. By wrapping a protocol with a cached version
> we wouldn't be able to distinguish eg access to a local services from
> a remote service (in terms of network hops).
>
> So what if we were to intercept those sources at the selector based
> on eg wildcard matching on URL? This would be "cross-cutting" and it
> would be easy to provide more complex caching parameters for a wrapped
> source.
>
For the life of me I can't figure out what exactly it is you are saying.
Care to elaborate? Perhaps give an example?
> Esp. with a event based cache, how would you specify the event to
> invalidate the source from within the URL?
>
I was thinking to use querystring parameter to pass it in and default to
the wrapped Source's system id (getURI). This is what I am using right
now for WebDAV sources and it works perfectly.
>> I also changed the way DelayRefresher persists its entry
>> configurations to instead of doing a manual xml serialization of the
>> data just dump the whole Map of entries into a persistent store.
>
>
> Cool.
>
>> By the way I think you are the one to ask Christian: I noticed there
>> is no scheduler.addPeriodicJob(.. Object job, ..) method. Is there a
>> reason for this or can I add one? So that the Refresher can add itself
>> and its update targets directly.
>
>
> Although I'd like to help I haven't touched the Cron block before. But
> speaking of addPeriodicJob() it would be great to have a way to say "do
> this right now AND repeat every 10 minutes". The current implementation
> provides only for a "do this every 10 minutes and start in 10 minutes
> for the first time". IIUC this limitation does not stem from the Qartz
> API but from what the Cocoon JobScheduler provides.
>
Yeah it does. There already exists a plain addJob method with the
signature I am talking about.
> Speaking of random ideas around the cached source, it might be nice to
> provide a URL for the content when it's not yet available. But then this
> might add too much features to a supposed to be simple concept.
>
I was also thinking of an implementation of the update mechanism where
objects don't get invalidated if they cannot be regenerated (eg. due to
WebDAV server offline, network problems, etc.) but get rescheduled instead.
> Sorry for messing your stuff up. I'm very excited to see what you have
> done.
>
No problem. I expect to check in some of my work later this week.
--
Unico
Re: cvs commit: cocoon-2.1/src/blocks/scratchpad/java/org/apache/cocoon/components/source/impl
CachingSource.java AsyncCachingSource.java RefresherImpl.java
Posted by Christian Haul <ha...@informatik.tu-darmstadt.de>.
Unico Hommes wrote:
> :-( I had so many changes on my local FS for these classes. I totally
> reworked their implemenatations. Not your fault though. I should have
> worked more incrementally and checked them in sooner. I'll find a way to
> merge it all.
Well, I wouldn't mind if you just dump my changes. As the message said,
I was just moving things around for documentation purposes, really.
(Some prefer comments, some prefer speaking names for rather small methods)
> Just to give you an idea of what I am doing here is a quick summary:
>
> Basically I want to move all communication with the Cache into the
> Refresher. I am renaming RefresherImpl to DelayRefresher and adding a
> Refresher implementation that is externally triggered by events. I am
> changing the location protocol string as discussed earlier. Probably can
> merge CachingSource and AsynchCachingSource.
This sound really cool and actually, I was planning for something
similar :-)
However, I'm not quite sure if this is the way to go. I'm currently
thinking of intercepting and wrapping sources at the selector level.
With the current setup, we need to add the configuration to the URL
in all places it is used. By wrapping a protocol with a cached version
we wouldn't be able to distinguish eg access to a local services from
a remote service (in terms of network hops).
So what if we were to intercept those sources at the selector based
on eg wildcard matching on URL? This would be "cross-cutting" and it
would be easy to provide more complex caching parameters for a wrapped
source.
Esp. with a event based cache, how would you specify the event to
invalidate the source from within the URL?
> I also changed the way DelayRefresher persists its entry configurations
> to instead of doing a manual xml serialization of the data just dump the
> whole Map of entries into a persistent store.
Cool.
> By the way I think you are the one to ask Christian: I noticed there is
> no scheduler.addPeriodicJob(.. Object job, ..) method. Is there a reason
> for this or can I add one? So that the Refresher can add itself and its
> update targets directly.
Although I'd like to help I haven't touched the Cron block before. But
speaking of addPeriodicJob() it would be great to have a way to say "do
this right now AND repeat every 10 minutes". The current implementation
provides only for a "do this every 10 minutes and start in 10 minutes
for the first time". IIUC this limitation does not stem from the Qartz
API but from what the Cocoon JobScheduler provides.
Speaking of random ideas around the cached source, it might be nice to
provide a URL for the content when it's not yet available. But then this
might add too much features to a supposed to be simple concept.
Sorry for messing your stuff up. I'm very excited to see what you have done.
Chris.
(who's currently too busy to read this list every day :-(
Re: cvs commit: cocoon-2.1/src/blocks/scratchpad/java/org/apache/cocoon/components/sou
rce/implCachingSource.java AsyncCachingSource.java RefresherImpl.java
Posted by Unico Hommes <un...@hippo.nl>.
Hi Antonio :-)
Integration of JCS as an excalibur Store service is going pretty well.
Solved most of the remaining issues I think.
There's one quite fundamental issue on the JCS side that still needs to
be addressed though. There seems to be a problem flushing the memory
cache onto disk during shutdown. This effectively means that a lot of
stuff won't be recoverable between server runs. Obviously this MUST be
fixed before considering JCS as a caching system for cocoon, let alone
the default one.
Going by what the guy that helped me on the JCS users mailinglist said
this is a known issue but no big deal. They just need to put in a simple
fix.
--
Unico
Antonio Gallardo wrote:
> Unico:
>
> Is near the day, when we will have JCS as the default Cache system in Cocoon?
>
> I cannot wait. :-)
>
> Best Regards,
>
> Antonio Gallardo
Re: cvs commit:
cocoon-2.1/src/blocks/scratchpad/java/org/apache/cocoon/components/sou
rce/implCachingSource.java AsyncCachingSource.java RefresherImpl.java
Posted by Antonio Gallardo <ag...@agssa.net>.
Unico:
Is near the day, when we will have JCS as the default Cache system in Cocoon?
I cannot wait. :-)
Best Regards,
Antonio Gallardo