making pipelines wait for each other

[Max Pfingsthorn <m....@hippo.nl>]

Hello!

I am not sure what has been done about this so far, but we are still experiencing some not-so-great behavior from the CachingPipeline:
We have some pipelines which take a long time to complete processing, like generating a homepage from many different sources. We see that if a second request for exactly the same pipeline comes in before the first request is done, it takes again as long because the first pipeline did not put its result in the cache yet and it is recomputed. This is quite obvious, but hurts our performance a lot.

I was thinking that we might implement some thread synchronization by sharing some objects and calling wait() and notify() on them.

First I thought putting an empty CachedResponse into the Cache when we start generating the data. Any pipeline trying to access that CachedResponse will notice that it is empty and call wait() on it. As soon as the first pipeline is done saving its content to the cache, it calls notify(). Then I remembered that the Cache puts these on disk, and I am not sure what happens to the locks when an object is serialized...
So, what about doing the same with a store instead? We push an empty response into a store, preferrably a transient, in-memory one without size limits, and do the same as I outlined above.

WDYT? Would that solve it? Is there a better way?

Best regards,

Max Pfingsthorn

Hippo  

Oosteinde 11
1017WT Amsterdam
The Netherlands
Tel  +31 (0)20 5224466
-------------------------------------------------------------
m.pfingsthorn@hippo.nl / www.hippo.nl
-------------------------------------------------------------

Re: making pipelines wait for each other

[Bertrand Delacretaz <bd...@apache.org>]

Le 6 févr. 06, à 10:44, Max Pfingsthorn a écrit :

> ...We see that if a second request for exactly the same pipeline comes 
> in before the first request is done, it takes again as long because 
> the first pipeline did not put its result in the cache yet and it is 
> recomputed. This is quite obvious, but hurts our performance a lot...

FYI, a while ago I had some performance issues and I was wondering what 
happens if you're using a front-end cache (mod_cache, Squid or 
something): do these caches optimize simultaneous requests by making 
all but one wait until the first response is generated?

In the end my issues where much more stupid than that (some pages were 
not stored in the front-end cache due to incorrect headers), so I 
didn't analyze in detail and don't have the answer.

 From your analysis it sounds like what you're seeing is really 
Cocoon-specific, but I thought I'd share my questioning, in case you're 
using a front-end cache as well.

-Bertrand

Re: making pipelines wait for each other

[Jean-Baptiste Quenot <jb...@apache.org>]

* Max Pfingsthorn:

> We see  that if a second  request for exactly the  same pipeline
> comes in  before the first  request is  done, it takes  again as
> long because  the first pipeline did  not put its result  in the
> cache yet and it is recomputed.

Same here.

> First I thought  putting an empty CachedResponse  into the Cache
> when we start generating the data. Any pipeline trying to access
> that CachedResponse will notice that it is empty and call wait()
> on it.

I won't  comment on the  *way* to  do it, but  I'd be glad  if you
could do it, because we need that feature.

Thanks a lot,
-- 
Jean-Baptiste Quenot
http://caraldi.com/jbq/

Re: making pipelines wait for each other

[Carsten Ziegeler <cz...@apache.org>]

Max Pfingsthorn wrote:
> Hello!
> 
> I am not sure what has been done about this so far, but we are still experiencing some not-so-great behavior from the CachingPipeline:
> We have some pipelines which take a long time to complete processing, like generating a homepage from many different sources. We see that if a second request for exactly the same pipeline comes in before the first request is done, it takes again as long because the first pipeline did not put its result in the cache yet and it is recomputed. This is quite obvious, but hurts our performance a lot.
> 
> I was thinking that we might implement some thread synchronization by sharing some objects and calling wait() and notify() on them.
> 
> First I thought putting an empty CachedResponse into the Cache when we start generating the data. Any pipeline trying to access that CachedResponse will notice that it is empty and call wait() on it. As soon as the first pipeline is done saving its content to the cache, it calls notify(). Then I remembered that the Cache puts these on disk, and I am not sure what happens to the locks when an object is serialized...
> So, what about doing the same with a store instead? We push an empty response into a store, preferrably a transient, in-memory one without size limits, and do the same as I outlined above.
> 
> WDYT? Would that solve it? Is there a better way?
> 
I think this topic has been discussed several times, so you might want
to search through the archives to see what others did say about it.

We usually solve this problem by pre-caching, so before the first real
user can invoke the pipeline, we already invoked it (using cron etc.)
and therefore the content is always cached.

Carsten

-- 
Carsten Ziegeler - Open Source Group, S&N AG
http://www.s-und-n.de
http://www.osoco.org/weblogs/rael/