include transformer explained [Fwd: RE: Deprecation of CInclude transformer]

[Jörn Nettingsmeier <ne...@apache.org>]

here's a little gem about the include transformer posted to cocoon-dev 
by hippo's ard schrijvers:

-------- Original Message --------
Subject: RE: Deprecation of CInclude transformer
Date: Wed, 21 Nov 2007 11:06:01 +0100
From: Ard Schrijvers <a....@hippo.nl>
Reply-To: dev@cocoon.apache.org
To: <de...@cocoon.apache.org>
References: <47...@tuffmail.com> 
<47...@dslextreme.com> <47...@tuffmail.com>

Hello,

> Grzegorz Kossakowski wrote:
> Could you elaborate on the known differences between CInclude 
> and Include transformers? Was it discussed somewhere?

I am not sure if it was ever discussed somewhere, but I can elaborate a
little on it ( though it has been a while, so I might be a little off at
some places... though understanding of this IncludeTransformer gives you
a very good insight in Cocoon's caching mechanism. ).

First of all, I think there are different useages of both transformers,
where I must admit I never used the CInclude ( I know you can also POST
with it,  append parameters, etc etc ). With the includeTransformer, all
you do is including a src which can use any of the defined
source-factories in you cocoon.xconf, for example

<include:include src="cocoon://some/pipeline"/> or
<include:include src="file://some/fs/file"/>

We at Hippo *always* used the IncludeTransformer, because it is way
better cacheable (and blistering fast ones included....but ofcourse,
there is a very subtle catch which the author didn't realize i
think...or just did not tell  :-) .), as opposed to the CInclude
transformer. AFAIU, the IncludeTransformer can only be cached by
defining some expires. Clearly, you cannot really know how to set this,
or if you fetch an external http source, just give it some heuristic
value.

Now, the IncludeTransformer adds the validity object of the included
sources to the validity object of the calling pipeline (pfffffff....).
Let's have an example:

someXML.xml :

<doc>
	<include:include src="cocoon://fetchSnippet"/>
</doc>

<map:match pattern="main">
	<map:generate src="someXML.xml"/>
	<map:transform type="include"/>
	<map:serialize/>
</map:match

<map:match pattern="fetchSnippet">
	<map:generate src="included.xml"/>
	<map:serialize/>
</map:match>

included.xml : <includethis/>

So, when calling /main, I will get

<doc>
	<includethis>	
</doc>

and the validity object of the "fetchSnippet" pipeline is added to the
validity of the "main" pipeline. So, the second call for /main, does the
setup for the "main" pipeline, and computes a cachekey....and finds a
hit in the cache with a valid cached object, thus returns the cached
result. NOTE: the second pipeline is never used in the second call. Not
even the setup!

Suppose I change now the second matcher the src from included.xml to
included2.xml, and call /main again....Well, my cachekey of the first
pipeline is unchanged, as well as the validity object (the validity
object is with respect to included.xml which did *not* change). So, I
still get the same cached result. Now, when I open "included.xml",
change something and save, then the validity obj of the first pipe is
invalid, so the result is re-computed, fetching included2.xml!!

Also, if the second pipeline /fetchSnippet start with a map:select
first, the things are getting even complexer (the little catch)....if
somebody wants to know about it I can elaborate.

Anyway, why use the IncludeTransformer if you have this 'strange'
behavior....Well, it is just way much faster than the
CIncludeTransformer, so if you have pipelines with 10-20 includes (which
we have in some projects) I want a cached result withing a couplt of ms.
Do realize, that setting up a pipeline, compute the compound cachekey of
a generators and transformers, check the cache, check the validity
object, do take time, which you do not have with the includetransformer
(it only returns the validity obj). OTOH, it really only makes sense to
use it when you have a lot of traffic, or when pages have to load fast.

Hope things are a little clear. Sorry I cannot say to much about the
CIncludeTransformer...I just do not know enough about it. I just know it
is much slower (I just looked at the code and see it uses a
cachingSession with an expires...i do not like it that much i think,
though it can be used for much more than the IncludeTransformer)

Regards Ard

> 
> BTW. I forgot to say that I would like to deprecate CInclude 
> in 2.2 only.
> 
> --
> Grzegorz Kossakowski
> 



-- 
Jörn Nettingsmeier

"One of my most productive days was throwing away 1000 lines of code."
   - Ken Thompson.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
For additional commands, e-mail: dev-help@lenya.apache.org