Accessing cache validities from flow

[Stefano Mazzocchi <st...@apache.org>]

After lots of thinking, I came to the conclusion that the only way of 
doing serious caching is using the "inverted cache" approach, which is 
prototyped in the "eventcache" block.

My problem is that linotype is too slow for the load that my blog 
generates and runs at 98% of server CPU. The reason for this is that 
very little caching is performed. I could pregenerate the pages and let 
them served static, but that would prevent me from playing with 
run-time data like the referring request headers.

Now, the core of linotype is the following pipeline:

  1) scans the directories that hold the news
  2) includes them
  3) filters them

as you can easily see, the approach is totally wrong: filtering should 
happen *before* the inclusion... the problem is that since the file 
system is such a stupid repository, metadata is included in the files, 
so I have to include them before I can filter them.

The use of a serious repository and a serious query language on top of 
the metadata will solve that issue and allow to scale a few orders of 
magnitue more in terms of requests, which will keep me happy for a 
while.

But still, thinking a little bigger than my own blog, the ideal 
solution would be to avoid querying entirely if there is no need to.

Now, the way the event cache works is like this:

  a) a cache validity is generated
  b) pipeline is executed
  c) result is stored in the cache

then the pipeline is never called, until an event is triggered 
externally (from an avalon component) that invalidates that particular 
cache entity.

Now, the problem is: if I am not the one who generates the validity (in 
linotype, it's the directory generator), how can I invalidate it? how 
can I have access to it?

I'm starting to think that the cocoon cache needs a serious redesign to 
deal with these issues... and this scares me :-( but it's *waay* too 
important to let go.

Thoughts?

--
Stefano.

Re: Accessing cache validities from flow

[Upayavira <uv...@upaya.co.uk>]

Sylvain Wallez wrote:

> Stefano Mazzocchi wrote:
>
>>
>> On 16 Dec 2003, at 14:02, bernhard huber wrote:
>>
>>> hi,
>>> <snip/>
>>>
>>>>
>>>> Now, the way the event cache works is like this:
>>>>
>>>>   a) a cache validity is generated
>>>>   b) pipeline is executed
>>>>   c) result is stored in the cache
>>>>
>>>> then the pipeline is never called, until an event is triggered 
>>>> externally (from an avalon component) that invalidates that 
>>>> particular cache entity.
>>>
>>>
>>> Some experiences I had using some sort of simple Servlet Cache 
>>> Filter using caching by sessionid: The session is not touched as 
>>> long the cache entry is valid, the session gets expired due to this 
>>> caching. But perhaps that's just an issue of the servlet engine, or 
>>> the Servlet CachFilter issue,
>>>
>>> Your sentence ..the pipeline is never called, just reminded me of 
>>> the that situation, and of the danger of pruning to optimistically.
>>
>>
>>
>> Thru my JSR 170 work, I've been exposed to what Day Software does 
>> with their Communique CMS.
>>
>> What they do is very simple architecturally yet extremely elegant and 
>> effective.
>>
>> They don't use the file system. Never. They store everything in a 
>> repository. Consider it a virtual file system with observable hooks 
>> for now (it's much more than that but it's not important for this 
>> discussion).
>>
>> Whenever a resource is generated by the publishing layer, this layer 
>> instantiates a sort of "reading transaction" so that the repository 
>> can keep track of all the dependencies of that particular resource.
>>
>> Note that they have libraries that, for example, generate images out 
>> of markup (sort-of Batik serializer style) so those dependencies 
>> might be quite big (I heard up to 100 files for a single resource).
>>
>> When a resource is modified into the repository, the tree of 
>> dependencies is crawled "backwards" and all resources that depend on 
>> it gets invalidated. Invalidation gets all the way up to an Apache 
>> module.
>>
>> This allows Communique to handle *extreme* load (they run Sony Style 
>> with just two boxes for fault tollerance and simple load balancing 
>> and that site generates tens of millions of requests per day, with 
>> huge peaks at break times). Note that communique is a 100% pure java 
>> servlet and the repository is all java again and runs in the same 
>> JVM: no database at all, no networking overhead.
>>
>> How do that do that? well, first thing is that most requests are 
>> handled directly by the web server... the servlet engine is called 
>> only when the resource needs to be regenerated.
>>
>> This leaves the machines almost doing nothing all day (if you run 
>> stuff from mod_cache, you can fill a T1 with a 486) and ready to go 
>> when a new resource has to be generated.
>>
>> Now, the drawbacks:
>>
>>  1) if you are *not* in control of your data environment, the above 
>> system doesn't work... unless you have synchronous polling on the 
>> datasources... which is not any better than the caching system we have.
>>
>>  2) the caching strategy is centralized. I'm not sure if components 
>> can have their own, but for sure it's a pain. [note: they don't have 
>> a pipelined rendering layer, just a one stage, template driven, 
>> approach]
>>
>> Communique is a publishing system on steroids, so I hear that writing 
>> an entire web application with Communique is probably harder than 
>> using a simple webapp framework.
>>
>> Cocoon wants to do both things and do them well, with as less effort 
>> and code as possible.
>>
>> Cocoon cannot has a predefined global caching strategy, it doesn't 
>> make sense. But it *does* make sense to have a pipeline-granular 
>> caching strategy, with the ability to modify it at the component level.
>>
>> We have this already, we just need to polish it up a little and find 
>> out what is *really* useful and how things can be made more usable.
>>
>> Today, modifying the caching strategy at the component level is black 
>> magic: nobody does. I'm scared about it myself, so I can't even 
>> imagine users trying to do this themselves.
>>
>> The off-the-shelf pipeline caches have some "magic" associated to 
>> it.... they are black boxes, basically, nobody really knows when 
>> something is caching or not.... it's hard to tell, hard to visualize, 
>> hard to control, hard to tune and hard to modify.
>>
>> This makes the whole thing much less powerful than it really is.
>>
>> You know how much I care about caching, but there is still a lot of 
>> work to do... expecially now that new "inverted" scenarios of use are 
>> going to appear on the horizon with observable repositories.
>
>
>
> We're talking about validities, but before checking a validity, we 
> first have to obtain it through the cache key.
>
> In the current Cocoon architecture, keys of cache entries are built 
> with abitrary data defined by each of the individual pipeline 
> components. The result of this is that we can have several different 
> cached responses for a single request definition (URI + headers).
>
> The big benefit of this approach is that many variations can be cached 
> (depending on night/day, local weather, whatever), but the main 
> disadvantage is that the pipeline *must* be built for every request in 
> order to compute the cache key, even if the response is served from 
> the cache afterwards.
>
> A solution would be to have another pipeline implementation that uses 
> a different strategy to build cache keys. What comes to mind is that 
> instead of returning abitrary values for key, components could return 
> some matching criteria on request metadata. The pipeline could then 
> organize the cache entries by URIs, each URI having a list of cached 
> responses along with the matching criteria.

It has just occurred to me how cool this would be for the CLI. Doing 
this would make it possible to identify whether or not _any_ effort 
should be expended upon generating a page, rather than the current 
system which involves actually getting a value from the cache before you 
decide to discard it.

The approach you describe above could result in a truly significant 
speed improvement for offline site creation.

Regards,

Upayavira, who is actually starting to use the CLI/bean on a real site 
for the first time!

> This approach would reduce the possible cached variations for a given 
> request, but would allow to find cached content (and its validity) 
> without incuring the cost of building the pipeline.
>
> What do you think?
>
> Sylvain
>

Re: Accessing cache validities from flow

[Stefano Mazzocchi <st...@apache.org>]

On 18 Dec 2003, at 05:27, Sylvain Wallez wrote:

> We're talking about validities, but before checking a validity, we 
> first have to obtain it through the cache key.

> In the current Cocoon architecture, keys of cache entries are built 
> with abitrary data defined by each of the individual pipeline 
> components. The result of this is that we can have several different 
> cached responses for a single request definition (URI + headers).

Yes.

> The big benefit of this approach is that many variations can be cached 
> (depending on night/day, local weather, whatever), but the main 
> disadvantage is that the pipeline *must* be built for every request in 
> order to compute the cache key, even if the response is served from 
> the cache afterwards.

Very true.

> A solution would be to have another pipeline implementation that uses 
> a different strategy to build cache keys. What comes to mind is that 
> instead of returning abitrary values for key, components could return 
> some matching criteria on request metadata. The pipeline could then 
> organize the cache entries by URIs, each URI having a list of cached 
> responses along with the matching criteria.


> This approach would reduce the possible cached variations for a given 
> request, but would allow to find cached content (and its validity) 
> without incuring the cost of building the pipeline.
>
> What do you think?

Hmmm, sounds very interestings, can you elaborate more on this?

--
Stefano.

Re: Accessing cache validities from flow

[Sylvain Wallez <sy...@apache.org>]

Stefano Mazzocchi wrote:

>
> On 16 Dec 2003, at 14:02, bernhard huber wrote:
>
>> hi,
>> <snip/>
>>
>>>
>>> Now, the way the event cache works is like this:
>>>
>>>   a) a cache validity is generated
>>>   b) pipeline is executed
>>>   c) result is stored in the cache
>>>
>>> then the pipeline is never called, until an event is triggered 
>>> externally (from an avalon component) that invalidates that 
>>> particular cache entity.
>>
>> Some experiences I had using some sort of simple Servlet Cache Filter 
>> using caching by sessionid: The session is not touched as long the 
>> cache entry is valid, the session gets expired due to this caching. 
>> But perhaps that's just an issue of the servlet engine, or the 
>> Servlet CachFilter issue,
>>
>> Your sentence ..the pipeline is never called, just reminded me of the 
>> that situation, and of the danger of pruning to optimistically.
>
>
> Thru my JSR 170 work, I've been exposed to what Day Software does with 
> their Communique CMS.
>
> What they do is very simple architecturally yet extremely elegant and 
> effective.
>
> They don't use the file system. Never. They store everything in a 
> repository. Consider it a virtual file system with observable hooks 
> for now (it's much more than that but it's not important for this 
> discussion).
>
> Whenever a resource is generated by the publishing layer, this layer 
> instantiates a sort of "reading transaction" so that the repository 
> can keep track of all the dependencies of that particular resource.
>
> Note that they have libraries that, for example, generate images out 
> of markup (sort-of Batik serializer style) so those dependencies might 
> be quite big (I heard up to 100 files for a single resource).
>
> When a resource is modified into the repository, the tree of 
> dependencies is crawled "backwards" and all resources that depend on 
> it gets invalidated. Invalidation gets all the way up to an Apache 
> module.
>
> This allows Communique to handle *extreme* load (they run Sony Style 
> with just two boxes for fault tollerance and simple load balancing and 
> that site generates tens of millions of requests per day, with huge 
> peaks at break times). Note that communique is a 100% pure java 
> servlet and the repository is all java again and runs in the same JVM: 
> no database at all, no networking overhead.
>
> How do that do that? well, first thing is that most requests are 
> handled directly by the web server... the servlet engine is called 
> only when the resource needs to be regenerated.
>
> This leaves the machines almost doing nothing all day (if you run 
> stuff from mod_cache, you can fill a T1 with a 486) and ready to go 
> when a new resource has to be generated.
>
> Now, the drawbacks:
>
>  1) if you are *not* in control of your data environment, the above 
> system doesn't work... unless you have synchronous polling on the 
> datasources... which is not any better than the caching system we have.
>
>  2) the caching strategy is centralized. I'm not sure if components 
> can have their own, but for sure it's a pain. [note: they don't have a 
> pipelined rendering layer, just a one stage, template driven, approach]
>
> Communique is a publishing system on steroids, so I hear that writing 
> an entire web application with Communique is probably harder than 
> using a simple webapp framework.
>
> Cocoon wants to do both things and do them well, with as less effort 
> and code as possible.
>
> Cocoon cannot has a predefined global caching strategy, it doesn't 
> make sense. But it *does* make sense to have a pipeline-granular 
> caching strategy, with the ability to modify it at the component level.
>
> We have this already, we just need to polish it up a little and find 
> out what is *really* useful and how things can be made more usable.
>
> Today, modifying the caching strategy at the component level is black 
> magic: nobody does. I'm scared about it myself, so I can't even 
> imagine users trying to do this themselves.
>
> The off-the-shelf pipeline caches have some "magic" associated to 
> it.... they are black boxes, basically, nobody really knows when 
> something is caching or not.... it's hard to tell, hard to visualize, 
> hard to control, hard to tune and hard to modify.
>
> This makes the whole thing much less powerful than it really is.
>
> You know how much I care about caching, but there is still a lot of 
> work to do... expecially now that new "inverted" scenarios of use are 
> going to appear on the horizon with observable repositories.


We're talking about validities, but before checking a validity, we first 
have to obtain it through the cache key.

In the current Cocoon architecture, keys of cache entries are built with 
abitrary data defined by each of the individual pipeline components. The 
result of this is that we can have several different cached responses 
for a single request definition (URI + headers).

The big benefit of this approach is that many variations can be cached 
(depending on night/day, local weather, whatever), but the main 
disadvantage is that the pipeline *must* be built for every request in 
order to compute the cache key, even if the response is served from the 
cache afterwards.

A solution would be to have another pipeline implementation that uses a 
different strategy to build cache keys. What comes to mind is that 
instead of returning abitrary values for key, components could return 
some matching criteria on request metadata. The pipeline could then 
organize the cache entries by URIs, each URI having a list of cached 
responses along with the matching criteria.

This approach would reduce the possible cached variations for a given 
request, but would allow to find cached content (and its validity) 
without incuring the cost of building the pipeline.

What do you think?

Sylvain

-- 
Sylvain Wallez                                  Anyware Technologies
http://www.apache.org/~sylvain           http://www.anyware-tech.com
{ XML, Java, Cocoon, OpenSource }*{ Training, Consulting, Projects }
Orixo, the opensource XML business alliance  -  http://www.orixo.com

Re: Accessing cache validities from flow

[Stefano Mazzocchi <st...@apache.org>]

On 16 Dec 2003, at 14:02, bernhard huber wrote:

> hi,
> <snip/>
>>
>> Now, the way the event cache works is like this:
>>
>>   a) a cache validity is generated
>>   b) pipeline is executed
>>   c) result is stored in the cache
>>
>> then the pipeline is never called, until an event is triggered
>> externally (from an avalon component) that invalidates that particular
>> cache entity.
> Some experiences I had using some sort of simple Servlet Cache Filter 
> using
> caching by sessionid:
> The session is not touched as long the cache entry is valid, the 
> session
> gets
> expired due to this caching.
> But perhaps that's just an issue of the servlet engine, or the Servlet
> CachFilter issue,
>
> Your sentence ..the pipeline is never called, just reminded me of the 
> that
> situation,
> and of the danger of pruning to optimistically.

Thru my JSR 170 work, I've been exposed to what Day Software does with 
their Communique CMS.

What they do is very simple architecturally yet extremely elegant and 
effective.

They don't use the file system. Never. They store everything in a 
repository. Consider it a virtual file system with observable hooks for 
now (it's much more than that but it's not important for this 
discussion).

Whenever a resource is generated by the publishing layer, this layer 
instantiates a sort of "reading transaction" so that the repository can 
keep track of all the dependencies of that particular resource.

Note that they have libraries that, for example, generate images out of 
markup (sort-of Batik serializer style) so those dependencies might be 
quite big (I heard up to 100 files for a single resource).

When a resource is modified into the repository, the tree of 
dependencies is crawled "backwards" and all resources that depend on it 
gets invalidated. Invalidation gets all the way up to an Apache module.

This allows Communique to handle *extreme* load (they run Sony Style 
with just two boxes for fault tollerance and simple load balancing and 
that site generates tens of millions of requests per day, with huge 
peaks at break times). Note that communique is a 100% pure java servlet 
and the repository is all java again and runs in the same JVM: no 
database at all, no networking overhead.

How do that do that? well, first thing is that most requests are 
handled directly by the web server... the servlet engine is called only 
when the resource needs to be regenerated.

This leaves the machines almost doing nothing all day (if you run stuff 
from mod_cache, you can fill a T1 with a 486) and ready to go when a 
new resource has to be generated.

Now, the drawbacks:

  1) if you are *not* in control of your data environment, the above 
system doesn't work... unless you have synchronous polling on the 
datasources... which is not any better than the caching system we have.

  2) the caching strategy is centralized. I'm not sure if components can 
have their own, but for sure it's a pain. [note: they don't have a 
pipelined rendering layer, just a one stage, template driven, approach]

Communique is a publishing system on steroids, so I hear that writing 
an entire web application with Communique is probably harder than using 
a simple webapp framework.

Cocoon wants to do both things and do them well, with as less effort 
and code as possible.

Cocoon cannot has a predefined global caching strategy, it doesn't make 
sense. But it *does* make sense to have a pipeline-granular caching 
strategy, with the ability to modify it at the component level.

We have this already, we just need to polish it up a little and find 
out what is *really* useful and how things can be made more usable.

Today, modifying the caching strategy at the component level is black 
magic: nobody does. I'm scared about it myself, so I can't even imagine 
users trying to do this themselves.

The off-the-shelf pipeline caches have some "magic" associated to 
it.... they are black boxes, basically, nobody really knows when 
something is caching or not.... it's hard to tell, hard to visualize, 
hard to control, hard to tune and hard to modify.

This makes the whole thing much less powerful than it really is.

You know how much I care about caching, but there is still a lot of 
work to do... expecially now that new "inverted" scenarios of use are 
going to appear on the horizon with observable repositories.

--
Stefano.

Re: Accessing cache validities from flow

[bernhard huber <be...@gmx.at>]

hi,
<snip/>
> 
> Now, the way the event cache works is like this:
> 
>   a) a cache validity is generated
>   b) pipeline is executed
>   c) result is stored in the cache
> 
> then the pipeline is never called, until an event is triggered 
> externally (from an avalon component) that invalidates that particular 
> cache entity.
Some experiences I had using some sort of simple Servlet Cache Filter using
caching by sessionid:
The session is not touched as long the cache entry is valid, the session
gets
expired due to this caching.
But perhaps that's just an issue of the servlet engine, or the Servlet
CachFilter issue,

Your sentence ..the pipeline is never called, just reminded me of the that
situation,
and of the danger of pruning to optimistically.

regards bernhard

-- 
+++ GMX - die erste Adresse für Mail, Message, More +++
Neu: Preissenkung für MMS und FreeMMS! http://www.gmx.net

Re: Accessing cache validities from flow

[Christian Haul <ha...@informatik.tu-darmstadt.de>]

Stefano Mazzocchi wrote:
> 
> Now, the way the event cache works is like this:
> 
>  a) a cache validity is generated
>  b) pipeline is executed
>  c) result is stored in the cache
> 
> then the pipeline is never called, until an event is triggered 
> externally (from an avalon component) that invalidates that particular 
> cache entity.
> 
> Now, the problem is: if I am not the one who generates the validity (in 
> linotype, it's the directory generator), how can I invalidate it? how 
> can I have access to it?

Not sure if I understand correctly,  so bear with me if this doesn't
answer the question at hand.

The cache component that is event aware can easily be obtained through 
the component manager. The event name could be set explicitly or 
determined by convention. In the event cache block there is a 
EventCacheGenerator that can wrap any other generator to make it produce 
compliant validities.

The CacheEventAction does exactly that to invalidate a cache.

Of course you could do the same directly from flow.

The next level is the JMS block. Here, a JMS listener component does the 
actual invalidating. Hence, any JMS publisher can cause a cache to 
invalidate. A simpler way is of course to do it through a URL that 
causes a cache to invalidate.

Still, there is a problem to solve: Which is the right point for the 
decision to use the cache? Will the pipeline be assembled and all 
components have their setup method called or is the validity checked 
first? Should actions and selectors execute? (probably yes because a 
different set of components could make up the pipeline) Should setup() 
be called? (probably no)

	Chris.

Re: Accessing cache validities from flow

[Christian Haul <ha...@informatik.tu-darmstadt.de>]

Geoff Howard wrote:

> Now, one problem I never solved related to the file system specifically 
> was how to do what I thought of as "wild card" events.  In a file 
> system, if your validity is built on /dir/to/your/files/myFile.xml and 
> you delete the entire /dir/to directory tree how do you fire that event? 

When doing the DB sample, a similar thought occurred to me. The solution
I favoured was to notify the change of the individual file AND the
directory.

So, for your example above, a notification for /dir/to would be fired
(since only a directory was deleted) but before, listeners register
for /dir/to/your/files/myFile.xml, /dir/to/your/files, /dir/to/your,
/dir/to, and /dir. Sure, when I was thinking about it, I had only row
and table to worry about :-)

	Chris.

Re: Accessing cache validities from flow

[Geoff Howard <co...@leverageweb.com>]

Stefano Mazzocchi wrote:

I also am long on interest, short on time right now but I'll chime in below.

...


> the problem is that since the file 
> system is such a stupid repository, 

OT: I have looked into experimenting with the filesystem event 
notification built into some operating systems (windows and linux both I 
think) to trigger file system "events".  If anyone good with native code 
and JNI wants to play, it'd be a cool experiment.  Non-portable 
unforunately.

...

> Now, the way the event cache works is like this:
> 
>  a) a cache validity is generated
>  b) pipeline is executed
>  c) result is stored in the cache
> 
> then the pipeline is never called, until an event is triggered 
> externally (from an avalon component) that invalidates that particular 
> cache entity.
> 
> Now, the problem is: if I am not the one who generates the validity (in 
> linotype, it's the directory generator), how can I invalidate it? how 
> can I have access to it?

I'm not sure I'm correctly interpreting the problem as you state it.  If 
directory generator were made "event aware" either by using the validity 
delegating generator Unico mentioned or by some new pipeline-level 
pluggable "Cache Strategy" (which I've wanted to look into) that would 
be a sort of known contract.  Currently the two EventValidity impls are 
brain dead and so the method for creating an event is vague because I 
wasn't sure if anything more than this would ever be needed.  One could 
imagine a "FileSystemEventValidity" which uses some internal logic for 
constructing the proper validity.  Does this not get at the problem?

Now, one problem I never solved related to the file system specifically 
was how to do what I thought of as "wild card" events.  In a file 
system, if your validity is built on /dir/to/your/files/myFile.xml and 
you delete the entire /dir/to directory tree how do you fire that event? 
  With the current design, you'd have to either fire off an individual 
event for each file under the deleted tree, or come up with some magic 
for handling an event like /dir/to/*.  Because the event lookups are 
handled by Hash based collections, which rely on hashCode() and equals() 
  you'd have to examine every event in turn to see if it matched the 
wild card.  I'm sure there is some brilliant solution to this but it 
hasn't occurred to me yet. ;)

> I'm starting to think that the cocoon cache needs a serious redesign to 
> deal with these issues... and this scares me :-( but it's *waay* too 
> important to let go.

Well, spill the beans - what do you have in mind?

Geoff