You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cocoon.apache.org by Sylvain Wallez <sy...@anyware-tech.com> on 2002/11/21 11:05:30 UTC
[RT] Using pipeline as sitemap components (long)
The discussions around Stefano's "Cocoon blocks version 1.1" showed the
need for pipelines to provide not only resources, but also services,
identified by their URI.
This document defines this concept of "pipeline service", which, as we
will see, consists in using pipelines as sitemap components (generator,
transformer and serializer). It is separated from the blocks design
document since pipeline services can be used without blocks, even if
they will be mostly useful in that context.
What is a pipeline ?
--------------------
The concept of pipeline, a central part of the Cocoon architecture, is a
chain of components handling XML documents as SAX events. By "handling",
we mean 3 different things :
- generate : at the start of the chain, produce an initial document and
feed the next component in the chain with the result.
- transform : take the content produced by preceding components in the
chain (either a generator or another transformer), transform it and feed
the next component in the chain with the result.
- serialize : take the content produced by the preceding component in
the chain (either a generator or a transformer), and convert this XML
stream to a binary stream.
These 3 concepts are represented using only 2 interfaces, XMLProducer
and XMLConsumer :
- a generator is an XMLProducer,
- a transformer is an XMLConsumer _and_ an XMLProducer,
- a serializer is an XMLConsumer.
The "cocoon:" protocol
----------------------
Up to now, we've considered pipelines as a "final" concept. This means
that a pipeline has to be considered as a whole : it handles a request
and answers by the result of it's execution.
Well, in fact, we "nearly" considered it as final. Consider the
"cocoon:" protocol that is so useful. What happens if we write the
following :
<map:match pattern="first-uri">
<map:generate type="file" src="cocoon://other-uri"/>
<map:transform src="foo.xslt"/>
<map:serialize/>
</map:match>
We're simply using another pipeline as the starting point of the current
one. We have used a pipeline as the generator of another one.
Most often, the "other-uri" builds a pipeline that is terminated by a
<map:serialize type="xml"/> because we want it to produce xml for the
calling generator. But this serializer is a fake : you can put any
serializer you like, it doesn't matter. What happens under the hood is
that the SAX events produced by the component immediately preceding the
serializer are used as the output of the generator in the calling pipeline.
So in the above example, when requesting "first-uri", we actually chain
the generators and transformers of "other-uri" to the transformers and
serializer of "first-uri".
Pipelines as generators
-----------------------
This leads to a first conclusion : using a pipeline as a generator means
using the SAX events produced by the last XMLProducer of that pipeline,
i.e. the last transformer or the generator if there are no transformers.
Since we've used a pipeline as a genererator, let's introduce a new
generator for this purpose, instead of using the "file" one, which fools
us in thinking we use a full pipeline when it actually strips out the
serializer :
<map:match src="first-uri">
<map:generate type="pipeline" src="/other-uri"/>
<map:transform src="foo.xslt"/>
<map:serialize/>
</map:match/>
I don't see a need for a new sitemap element such as "map:call-pipeline"
or "map:generate-from-pipeline". What we want is to generate and initial
content in the current pipeline, and for this we just use a particular
implementation of a generator, as we already do for files, XSP, etc.
Pipelines as serializers
------------------------
We've seen how to use a pipeline as the generator of another one, let's
consider now the other end of the chain : using a pipeline as a serializer.
Let's suppose have defined a pipeline that gets an XML document in the
xdoc DTD and formats it to PDF. This can be for example :
<map:match pattern="doc2pdf">
<map:generate src="an_xdoc.xml"/>
<map:transform src="doc2fo.xslt"/>
<map:serialize type="fo2pdf"/>
</map:match>
The interesting part here isn't the initial document, but the chaining
of a stylesheet that produces an xsl:fo version of its input and the FOP
serializer. This is the typical example of what is called a "service" in
the current block specification.
Now how do we reuse this in other pipelines ? Yes, we can define a
<map:resource>. But this resource will be available only in the current
sitemap, and not in other sitemaps nor blocks.
What actually means "reusing" this ? This means producing a xdoc
document and _serializing_ it to PDF. We don't actually care if there is
a serializer to PDF that directly accepts xdocs or if there are one or
more transformations before serializing.
This leads to a second conclusion : using a pipeline as a serializer
means sending the SAX events of the calling pipeline to the first
XMLConsumer of the called pipeline.
How do we use this ? Well, just as for the generator, let's define a new
"pipeline" serializer :
<map:generate src="another_xdoc.xml"/>
<map:serialize type="pipeline" src="doc2pdf"/>
Note : the "src" attribute doesn't currently exist on <map:serialize>,
but it seems the more natural and consistent way to name the called
pipeline. Wether this translates to implementing SitemapModelComponent
or not is another story.
Pipelines as transformers
-------------------------
And here comes the last use of a pipeline : as a transformer. Let's
consider the following :
<map:match pattern="a_page">
<map:generate src="an_xdoc.xml"/>
<map:transform type="i18n"/>
<map:transform src="xdoc2html.xsl"/>
<map:transform src="htmlskin.xsl"/>
<map:serialize type="html"/>
</map:match>
The 3 transformers define a transformation service that takes an xdoc as
input and produces some skinned html. To achieve reusability, we would
like to have a "xdoc2skinnedHtml" transformer. We can write this like
the following :
<map:match pattern="a_page">
<map:generate src="an_xdoc.xml"/>
<map:transform type="pipeline" src="xdoc2skinnedHtml"/>
<map:serialize type="html"/>
</map:match>
and
<map:match pattern="xdoc2skinnedHtml">
<map:generate type="dont_care"/>
<map:transform type="i18n"/>
<map:transform type="xdoc2html.xsl"/>
<map:transform type="htmlskin.xsl"/>
<map:serialize type="dont_care"/>
</map:match>
This leads to a third conclusion : using a pipeline as a transformer
means feeding the SAX events of the calling pipeline to the first
transformer of the called pipeline, and sending the output of the last
transformer of the called pipeline to the next XMLConsumer of the
calling pipeline.
Note : if there are no transformers in the called pipeline (i.e. it's
only a generator and a serializer), the "pipeline" transformer does
nothing and only copies its input to its output.
Relation to blocks
------------------
Up to now, we made no mention of blocks. The "src" attribute of the new
"pipeline" sitemap components is an URI that is considered as what
follows the first "/" in the "cocoon:" protocol :
- "/pipeline-uri" is resolved by calling the root sitemap,
- "pipeline-uri" is resolved by calling the current sitemap.
We can now introduce blocks :
- "block:foo:pipeline-uri" is resolved by calling the "foo" block.
So if we consider the transformer example above, and move the
"xdoc2skinnedHtml" pipeline to a "skin" block, our sitemap becomes :
<map:match pattern="a_page">
<map:generate src="an_xdoc.xml"/>
<map:transform type="pipeline" src="block:skin:xdoc2skinnedHtml"/>
<map:serialize/>
</map:match/>
Questions and answers
---------------------
Q: What about caching when we call a pipeline ?
A: This should integrate smoothly : the cache key and validity of the
"pipeline" generator, transformer and serializer are the composition of
cache keys and validities of the used components of the called pipeline.
--o--
Q: Doesn't this deprecate the use of the "cocoon:" protocol ?
A: No. The only notation that may be deprecated is <map:generate
type="file" src="cocoon://xxx"/> that can now be written <map:generate
type="pipeline" src="/xxx"/>. Other uses of the "cocoon" protocol keep
their usefulness.
--o--
Q: I want do define a pipeline that will be used only as a
transformation service. Why must I write a <map:generate> and a
<map:serialize> in its definition ?
A: Because the sitemap, as a pipeline building language, must be able to
determine the start of a pipeline and its end, even if not all its
components are used. Like opening and closing braces in Java, the
generator begins the pipeline definition and the serializer ends it.
Ok. Thanks for reading so far. What are your thoughts about this ? If we
agree on it, I'll update the Cocoon blocks document so that block
services are shown as "pipeline" sitemap components.
Sylvain
--
Sylvain Wallez Anyware Technologies
http://www.apache.org/~sylvain http://www.anyware-tech.com
{ XML, Java, Cocoon, OpenSource }*{ Training, Consulting, Projects }
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [RT] Using pipeline as sitemap components (long)
Posted by Sylvain Wallez <sy...@anyware-tech.com>.
Stefano Mazzocchi wrote:
> Sylvain Wallez wrote:
>
>> We can already can choose the view for a "cocoon:" resource by means
>> of the standard "cocoon-view" parameter. So I would avoid adding a
>> new attribute and simply write
>>
>> <map:generate type="cocoon"
>> src="/whatever/resource?cocoon-view=content"/>
>
>
> I much rather see an explicit way to indicate the view. It removes
> dependencies on how the view is encoded into the URI (which might
> change in the future once the 'view' concept is used by non-cocoon
> frameworks)
What about
<map:generate type="cocoon" src="/whatever/resource">
<map:parameter name="view" value="content"/>
</map:generate>
> Moreover, this is another reason to have a specific sitemap semantics
> for explicit pipeline calls.
Once again, I don't consider this to be a new concept on the caller side
: the caller just wants a generator, transformer or serializer. This is
only on the callee part that things are new, since some parts of the
called pipeline aren't used.
Sylvain
--
Sylvain Wallez Anyware Technologies
http://www.apache.org/~sylvain http://www.anyware-tech.com
{ XML, Java, Cocoon, OpenSource }*{ Training, Consulting, Projects }
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [RT] Using pipeline as sitemap components (long)
Posted by Stefano Mazzocchi <st...@apache.org>.
Sylvain Wallez wrote:
> We can already can choose the view for a "cocoon:" resource by means of
> the standard "cocoon-view" parameter. So I would avoid adding a new
> attribute and simply write
>
> <map:generate type="cocoon" src="/whatever/resource?cocoon-view=content"/>
I much rather see an explicit way to indicate the view. It removes
dependencies on how the view is encoded into the URI (which might change
in the future once the 'view' concept is used by non-cocoon frameworks)
Moreover, this is another reason to have a specific sitemap semantics
for explicit pipeline calls.
--
Stefano Mazzocchi <st...@apache.org>
--------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [RT] Using pipeline as sitemap components (long)
Posted by Sylvain Wallez <sy...@anyware-tech.com>.
Stefano Mazzocchi wrote:
> Sylvain Wallez wrote:
>
>> The discussions around Stefano's "Cocoon blocks version 1.1" showed
>> the need for pipelines to provide not only resources, but also
>> services, identified by their URI.
>>
>> This document defines this concept of "pipeline service", which, as
>> we will see, consists in using pipelines as sitemap components
>> (generator, transformer and serializer). It is separated from the
>> blocks design document since pipeline services can be used without
>> blocks, even if they will be mostly useful in that context.
>
>
> Thanks for writing this Sylvain, it's very helpful.
>
> [snip]
>
> In short, what you are describing is a model for pipeline extension:
> one pipeline 'extends' another pipeline by calling it and cocoon
> transparently adapts it by removing the parts which aren't useful.
>
> I do see the problem of cocoon transparently removing pipeline
> components, but it feels more like OOP inheritance where the system
> transparently ignores the overloaded method.
>
> Yes, the parallell is a little streched, but I think that the fact
> that nobody ever complained about the fact that serializers are
> ignored on cocoon: subpipeline calls makes me think that the concept
> is not so unfriendly.
I'm not sure we can say that a pipeline "extends" another one, as this
happens at a particular part of the calling pipeline, and we could even
have a pipeline composed of several other pipelines. This is more
related to defining a function, but even then the parallel is stretched.
This should be more considered as defining composite sitemap components
: for example, the "cocoon" (better name than "pipeline") generator is
composed of a generator and a set of transformers.
> Moreover, the fact of allowing *complete* pipelines is also
> tremendously helpful for debugging purposes and for block-reuse:
> instead of having passive pipeline fragments, we'll have the ability
> to write a pipeline, test it and then extend it with another.
>
> Note that a pipeline might still be required to be called directly.
>
> For example:
>
> <match pattern="*.xml">
> <generate type="file" src="{1}.xml"/>
> <serialize type="xml"/>
> </match>
>
> <match pattern="*.html">
> <generate type="pipeline" src="{1}.xml"/>
> <transform src="xml2html.xslt"/>
> <serialize type="html"/>
> </match>
>
> <match pattern="*.pdf">
> <generate type="pipeline" src="{1}.xml"/>
> <transform src="xml2fo.xslt"/>
> <serialize type="fo2pdf"/>
> </match>
>
> The nice thing would the above is that if the generation of the *.xml
> resource is changed, this is automatically inherited by all the
> pipelines that build on it.
Good example : this shows that a pipeline can be used, depending on the
context, either as a complete pipeline, or as a service (a generator in
the above sample).
> Note that one might also want to connect to "views" of those
> pipeline... so for example why don't we add the ability to specify
> which "view" of the pipeline we should start from?
>
> <map:generate type="pipeline" src="/whatever/resource" view="content"/>
>
> What do you think think?
Thinking twice is certainly better than once ;-)
We can already can choose the view for a "cocoon:" resource by means of
the standard "cocoon-view" parameter. So I would avoid adding a new
attribute and simply write
<map:generate type="cocoon" src="/whatever/resource?cocoon-view=content"/>
Sylvain
--
Sylvain Wallez Anyware Technologies
http://www.apache.org/~sylvain http://www.anyware-tech.com
{ XML, Java, Cocoon, OpenSource }*{ Training, Consulting, Projects }
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [RT] Using pipeline as sitemap components (long)
Posted by Vadim Gritsenko <va...@verizon.net>.
Sylvain Wallez wrote:
> Vadim Gritsenko wrote:
>
...
>> I see two issues to be resolved before Sylvain's proposal can be
>> implemented.
>>
>> First one is naming issue. It's very confusing (to me, and I bet to
>> users too) when ProcessingPipeline object built by sitemap engine
>> after sitemap execution is referenced in the sitemap simply as
>> "pipeline" - it can be very easiliy mistaken for map:pipeline section
>> (the whole different beast). Thus, I suggest we use "cocoon" type
>> instead of "pipeline" type. The reason is: because this name is more
>> similar to "cocoon" protocol which does very similar thing: it lets
>> you access ProcessingPipeline object, augment it, and use it.
>
>
>
> Yep. I now use "cocoon" as you suggested in an earlier post.
Actually, it was vice versa: that's was the first post, second I wrote
later to show how your proposal could be misinterpreted ;-)
And may be to help clear up some misunderstanding.
>> Second issue is that clear understanding of how this new components
>> relate to existing sitemap constructs such as views and resources.
>> Its mainly documentation issue I think. On the surface, "pipeline as
>> serializer" can be mistaken with a view, and "pipeline as
>> transformer" is somewhat similar to broken resources implementation
>> in 2.1.
>
>
>
> Resources are internal only to a sitemap, so this proposal doesn't
> affect the way they're handled.
My only point here is that I feel lots of users will ask questions. Some
kind of doco will be needed.
> BTW what's broken about resources in 2.1 ?
:)
AFAIR, as of now, they do not enforce completeness of the pipeline.
Officially, resource *must* be the complete pipeline, from generator to
serializer. In practice, in 2.1, you can call resource and complete it
later with transformer or two and serializer. I think, this practice
shall be terminated (== bug fixed) when we implement <transform
type="cocoon">.
> As for views, we already have defined how view relate to the "cocoon:"
> protocol. So I would use the same behaviour, which uses the well-known
> "cocoon-view" request parameter.
I mean here, that also some doco explaining what's <serialize
type="cocoon"> is and what's the difference with view (view is also
kinda serialize ;) is required.
> <snip/>
>
>> Can we do not use "pipeline" word for these components? That's the
>> major concern I have.
>
>
>
> As I said previously, I now use the "cocoon" word, which is best
> suited, as you suggested.
We are on the same page now. With terms and understanding. Which is a
good thing.
Vadim
> Sylvain
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [RT] Using pipeline as sitemap components (long)
Posted by Sylvain Wallez <sy...@anyware-tech.com>.
Vadim Gritsenko wrote:
> Stefano Mazzocchi wrote:
>
>> Sylvain Wallez wrote:
>>
>>> The discussions around Stefano's "Cocoon blocks version 1.1" showed
>>> the need for pipelines to provide not only resources, but also
>>> services, identified by their URI.
>>>
>>> This document defines this concept of "pipeline service", which, as
>>> we will see, consists in using pipelines as sitemap components
>>> (generator, transformer and serializer). It is separated from the
>>> blocks design document since pipeline services can be used without
>>> blocks, even if they will be mostly useful in that context.
>>
>>
>>
>> Thanks for writing this Sylvain, it's very helpful.
>>
>> [snip]
>>
>> In short, what you are describing is a model for pipeline extension:
>> one pipeline 'extends' another pipeline by calling it and cocoon
>> transparently adapts it by removing the parts which aren't useful.
>>
>> I do see the problem of cocoon transparently removing pipeline
>> components, but it feels more like OOP inheritance where the system
>> transparently ignores the overloaded method.
>>
>> Yes, the parallell is a little streched, but I think that the fact
>> that nobody ever complained about the fact that serializers are
>> ignored on cocoon: subpipeline calls makes me think that the concept
>> is not so unfriendly.
>>
>> Moreover, the fact of allowing *complete* pipelines is also
>> tremendously helpful for debugging purposes and for block-reuse:
>> instead of having passive pipeline fragments, we'll have the ability
>> to write a pipeline, test it and then extend it with another.
>>
>> Note that a pipeline might still be required to be called directly.
>
>
>
> I see two issues to be resolved before Sylvain's proposal can be
> implemented.
>
> First one is naming issue. It's very confusing (to me, and I bet to
> users too) when ProcessingPipeline object built by sitemap engine
> after sitemap execution is referenced in the sitemap simply as
> "pipeline" - it can be very easiliy mistaken for map:pipeline section
> (the whole different beast). Thus, I suggest we use "cocoon" type
> instead of "pipeline" type. The reason is: because this name is more
> similar to "cocoon" protocol which does very similar thing: it lets
> you access ProcessingPipeline object, augment it, and use it.
Yep. I now use "cocoon" as you suggested in an earlier post.
> Second issue is that clear understanding of how this new components
> relate to existing sitemap constructs such as views and resources. Its
> mainly documentation issue I think. On the surface, "pipeline as
> serializer" can be mistaken with a view, and "pipeline as transformer"
> is somewhat similar to broken resources implementation in 2.1.
Resources are internal only to a sitemap, so this proposal doesn't
affect the way they're handled. BTW what's broken about resources in 2.1 ?
As for views, we already have defined how view relate to the "cocoon:"
protocol. So I would use the same behaviour, which uses the well-known
"cocoon-view" request parameter.
<snip/>
> Can we do not use "pipeline" word for these components? That's the
> major concern I have.
As I said previously, I now use the "cocoon" word, which is best suited,
as you suggested.
Sylvain
--
Sylvain Wallez Anyware Technologies
http://www.apache.org/~sylvain http://www.anyware-tech.com
{ XML, Java, Cocoon, OpenSource }*{ Training, Consulting, Projects }
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [RT] Using pipeline as sitemap components (long)
Posted by Vadim Gritsenko <va...@verizon.net>.
Stefano Mazzocchi wrote:
> Sylvain Wallez wrote:
>
>> The discussions around Stefano's "Cocoon blocks version 1.1" showed
>> the need for pipelines to provide not only resources, but also
>> services, identified by their URI.
>>
>> This document defines this concept of "pipeline service", which, as
>> we will see, consists in using pipelines as sitemap components
>> (generator, transformer and serializer). It is separated from the
>> blocks design document since pipeline services can be used without
>> blocks, even if they will be mostly useful in that context.
>
>
> Thanks for writing this Sylvain, it's very helpful.
>
> [snip]
>
> In short, what you are describing is a model for pipeline extension:
> one pipeline 'extends' another pipeline by calling it and cocoon
> transparently adapts it by removing the parts which aren't useful.
>
> I do see the problem of cocoon transparently removing pipeline
> components, but it feels more like OOP inheritance where the system
> transparently ignores the overloaded method.
>
> Yes, the parallell is a little streched, but I think that the fact
> that nobody ever complained about the fact that serializers are
> ignored on cocoon: subpipeline calls makes me think that the concept
> is not so unfriendly.
>
> Moreover, the fact of allowing *complete* pipelines is also
> tremendously helpful for debugging purposes and for block-reuse:
> instead of having passive pipeline fragments, we'll have the ability
> to write a pipeline, test it and then extend it with another.
>
> Note that a pipeline might still be required to be called directly.
I see two issues to be resolved before Sylvain's proposal can be
implemented.
First one is naming issue. It's very confusing (to me, and I bet to
users too) when ProcessingPipeline object built by sitemap engine after
sitemap execution is referenced in the sitemap simply as "pipeline" - it
can be very easiliy mistaken for map:pipeline section (the whole
different beast). Thus, I suggest we use "cocoon" type instead of
"pipeline" type. The reason is: because this name is more similar to
"cocoon" protocol which does very similar thing: it lets you access
ProcessingPipeline object, augment it, and use it.
Second issue is that clear understanding of how this new components
relate to existing sitemap constructs such as views and resources. Its
mainly documentation issue I think. On the surface, "pipeline as
serializer" can be mistaken with a view, and "pipeline as transformer"
is somewhat similar to broken resources implementation in 2.1.
> For example:
>
> <match pattern="*.xml">
> <generate type="file" src="{1}.xml"/>
> <serialize type="xml"/>
> </match>
> <match pattern="*.html">
> <generate type="pipeline" src="{1}.xml"/>
> <transform src="xml2html.xslt"/>
> <serialize type="html"/>
> </match>
As it was already mentioned (in previous mail), this is equivalent to
combination of file generator with cocoon protocol. But, I do not see
any issues adding special "cocoon" type generator:
<generate type="cocoon" src="{1}.xml"/>
<serialize type="xml"/>
And its implementation could differ a lot from file generator/cocoon
source pair...
...
> The nice thing would the above is that if the generation of the *.xml
> resource is changed, this is automatically inherited by all the
> pipelines that build on it.
>
> Note that one might also want to connect to "views" of those
> pipeline... so for example why don't we add the ability to specify
> which "view" of the pipeline we should start from?
>
> <map:generate type="pipeline" src="/whatever/resource" view="content"/>
>
> What do you think think?
Can we do not use "pipeline" word for these components? That's the major
concern I have.
Vadim
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [RT] Using pipeline as sitemap components (long)
Posted by Stefano Mazzocchi <st...@apache.org>.
Sylvain Wallez wrote:
> The discussions around Stefano's "Cocoon blocks version 1.1" showed the
> need for pipelines to provide not only resources, but also services,
> identified by their URI.
>
> This document defines this concept of "pipeline service", which, as we
> will see, consists in using pipelines as sitemap components (generator,
> transformer and serializer). It is separated from the blocks design
> document since pipeline services can be used without blocks, even if
> they will be mostly useful in that context.
Thanks for writing this Sylvain, it's very helpful.
[snip]
In short, what you are describing is a model for pipeline extension: one
pipeline 'extends' another pipeline by calling it and cocoon
transparently adapts it by removing the parts which aren't useful.
I do see the problem of cocoon transparently removing pipeline
components, but it feels more like OOP inheritance where the system
transparently ignores the overloaded method.
Yes, the parallell is a little streched, but I think that the fact that
nobody ever complained about the fact that serializers are ignored on
cocoon: subpipeline calls makes me think that the concept is not so
unfriendly.
Moreover, the fact of allowing *complete* pipelines is also tremendously
helpful for debugging purposes and for block-reuse: instead of having
passive pipeline fragments, we'll have the ability to write a pipeline,
test it and then extend it with another.
Note that a pipeline might still be required to be called directly.
For example:
<match pattern="*.xml">
<generate type="file" src="{1}.xml"/>
<serialize type="xml"/>
</match>
<match pattern="*.html">
<generate type="pipeline" src="{1}.xml"/>
<transform src="xml2html.xslt"/>
<serialize type="html"/>
</match>
<match pattern="*.pdf">
<generate type="pipeline" src="{1}.xml"/>
<transform src="xml2fo.xslt"/>
<serialize type="fo2pdf"/>
</match>
The nice thing would the above is that if the generation of the *.xml
resource is changed, this is automatically inherited by all the
pipelines that build on it.
Note that one might also want to connect to "views" of those pipeline...
so for example why don't we add the ability to specify which "view" of
the pipeline we should start from?
<map:generate type="pipeline" src="/whatever/resource" view="content"/>
What do you think think?
--
Stefano Mazzocchi <st...@apache.org>
--------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [RT] Using pipeline as sitemap components (long)
Posted by Sylvain Wallez <sy...@anyware-tech.com>.
Vadim Gritsenko wrote:
> Christian Haul wrote:
> ...
>
>> A last thought: would it be necessary to have those pipelines match a
>> pattern or would it be sufficiant to have them names?
>
>
> After reading it second time... And seems that Christian also has
> concern about syntax... There are two ways to understand word
> "pipeline" and Sylvain's proposal.
Vadim, you point out exactly the misunderstanding in which Christian and
Gianugo seem to have fallen. My proposal is the first one below : "A la
Cocoon source". The "src" attribute is effectively the URI handled by a
sitemap, just as the one we use for the "cocoon:" protocol.
The second way ("A la Text Substitution") is based on the use in the
names of <map:pipeline>, which doesn't currently exist. IMO, the
decomposition of a sitemap in pipelines is an internal matter, and
should not used to define the public contract of a sitemap or block.
> First Way: A-la Cocoon Source
> --------------------------------
<snip/>
> Second Way: A-la Text Substitution
> ------------------------------------
<snip/>
> Results are the same, but please note the differences.
>
> Implementation via the first way is better named as "cocoon"-type
> components, second way can be named as "pipeline"-type components.
Good point. Using "cocoon" as the component type certainly helps to
understand that the "src" is a sitemap URI. We shall use it from now on.
Sylvain
--
Sylvain Wallez Anyware Technologies
http://www.apache.org/~sylvain http://www.anyware-tech.com
{ XML, Java, Cocoon, OpenSource }*{ Training, Consulting, Projects }
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [RT] Using pipeline as sitemap components (long)
Posted by Sylvain Wallez <sy...@anyware-tech.com>.
Carsten Ziegeler wrote:
>I'm sorry - I really value the effort, Sylvain has put into
>writing this RT, but I'm currently against forcing this new concept
>of "pipeline services" into the existing features by adding
>some special components with a very special behaviour.
>
>The cocoon protocol is a *protocol*, so it works like any other
>protocol (http, ftp etc). A protocol is usually not forced
>to deliver a strict content type (like xml). If you
>see http://something/here you don't know what content and in
>which format this is delivered.
>And actually the same is true for the cocoon: protocol. You
>can have a
><map:match pattern="test">
> <map:read src="bla"/>
></map:match>
>and use cocoon:/test returning the byte stream of the reader.
>No XML in here, no SAX events etc.
>The other way also works:
><map:match pattern="test"
> <map:read src="cocoon:/some-pipeline"/>
></map:match>
>
>Let's not discuss if these examples are very usefull. But these
>are examples of usual protocol usage and they show the way you
>usually handle protocols.
>
>It is correct, that internally in most cases the serializer
>of a pipeline is ignored, when the cocoon protocol is used.
>But this is only because of performance.
>
>You can totally confuse new cocoon users by telling them
>that the serializer does not have to be the xml serializer.
>Because they don't know the internal things of Cocoon which
>we know. And it's not obvious for them.
>
I don't agree with the above : the first thing that a Cocoon user has to
know are the concepts of generator, transformer and serializer. And all
documents explaining these concepts explain the usage of SAX events (or
at least XML documents) to communicate between components and the fact
that the serializer translates SAX events to a byte stream.
The concept of SAX-event pipeline is *the* fundamental concept of
Cocoon. A such, a user not knowing this concept will understand about
nothing to what Cocoon is and what it can do.
But let's not argue about this, which isn't the real problem here.
>So, defining a service with a generator and a serializer where
>both are ignored is absolutely not intuitiv and will scare
>many.
>And I can already hear the complains :"What? This should be the
>new great concept and I always have to specify a generator
>and serializer and then they are not used. Now, that's simple."
>
>Let's be honest - do you know any framework etc. where you
>have to specify things which are not used to define this
>service? It's like implementing a method where you have to
>add some statement at the beginning and at the end and then
>later on they are ignored.
>
Mmmh... what about languages like Eiffel where each method can have
preconditions and postconditions (or the new "assert" in JDK 1.4) ?
Depending on the runtime environment, these blocks may or may not be called.
>I think we should not mix things: the concept we want and
>how to implement it. We should first focus on what we want
>and than see how it fits into our architecture. Let's not
>talk at this stage about SAX events etc.
>
>So, my perception is that we want to have services that
>can be used as a generator, as a transformer and as a
>serializer.
>
>The first question is: how can I call these services
>and the second one is how can one define a service?
>
>We saw some different examples on this and I personally like
>a new syntax best to call a service instead of defining
>a special component (generator, transformer, serializer).
>As Stefano mentioned, we did this for content aggregation
>as well.
>
>
Content aggregation is a different thing, since we want to *merge*
different sources, which isn't possible with a language (the sitemap)
that defines a linear pipeline.
Note also that Cocoon provides another way of aggregating content
through the X/Cinclude transformers, which are nothing but regular
sitemap components.
>And I think if we have a special syntax for calling a service
>it's nearly natural to have a special concept for defining
>the service as well.
>
Yep. But once again, there is no new concept on the caller part. I admit
the syntax is somewhat stretched on the callee and that we may want a
particular syntax there, but certainly not on the caller part.
So what if we introduce some new statements to define incomplete
pipelines to be used as pipeline services ? For example, a serialization
service could be defined using :
<map:begin-service/>
<map:transform src="xdoc2fo.xsl"/>
<map:serialize type="fo2pdf"/>
and a transformation service could be defined using :
<map:begin-service/>
<map:transform src="foo.xsl"/>
<map:end-service/>
That is <map:begin-service> replaces a <map:generator> that would be
ignored and has to be used for transformation or serialization services.
Similarily, <map:end-service> replaces a <map:serialize> that would be
ignored and has to be used for generation or transformation services.
>And one interesting thought: I could imagine that as soon
>as people can call a cocoon pipeline as a transformation
>service they will want to call a web services doing the same
>as well, so something like this comes into mind:
><generate src="hallo.xml"/>
><call-pipeline src="http://server/transformation"/>
><serialize>
>
Why should a remote web service be called with a "call-pipeline", which
is a Cocoon concept ? Here again, we just want transformation. So this
can be written :
<generate src="hallo.xml"/>
<transform type="soap" src="http://server/transformation"/>
<serialize/>
>Ok, ignore this last comment - I only had to show how usefull
>web services can be - even for Cocoon :)
>
This comment is really useful to show that there's no need for a new
<call-pipeline> ;-)
Sylvain
--
Sylvain Wallez Anyware Technologies
http://www.apache.org/~sylvain http://www.anyware-tech.com
{ XML, Java, Cocoon, OpenSource }*{ Training, Consulting, Projects }
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
RE: [RT] Using pipeline as sitemap components (long)
Posted by Carsten Ziegeler <cz...@s-und-n.de>.
I'm sorry - I really value the effort, Sylvain has put into
writing this RT, but I'm currently against forcing this new concept
of "pipeline services" into the existing features by adding
some special components with a very special behaviour.
The cocoon protocol is a *protocol*, so it works like any other
protocol (http, ftp etc). A protocol is usually not forced
to deliver a strict content type (like xml). If you
see http://something/here you don't know what content and in
which format this is delivered.
And actually the same is true for the cocoon: protocol. You
can have a
<map:match pattern="test">
<map:read src="bla"/>
</map:match>
and use cocoon:/test returning the byte stream of the reader.
No XML in here, no SAX events etc.
The other way also works:
<map:match pattern="test"
<map:read src="cocoon:/some-pipeline"/>
</map:match>
Let's not discuss if these examples are very usefull. But these
are examples of usual protocol usage and they show the way you
usually handle protocols.
It is correct, that internally in most cases the serializer
of a pipeline is ignored, when the cocoon protocol is used.
But this is only because of performance.
You can totally confuse new cocoon users by telling them
that the serializer does not have to be the xml serializer.
Because they don't know the internal things of Cocoon which
we know. And it's not obvious for them.
So, defining a service with a generator and a serializer where
both are ignored is absolutely not intuitiv and will scare
many.
And I can already hear the complains :"What? This should be the
new great concept and I always have to specify a generator
and serializer and then they are not used. Now, that's simple."
Let's be honest - do you know any framework etc. where you
have to specify things which are not used to define this
service? It's like implementing a method where you have to
add some statement at the beginning and at the end and then
later on they are ignored.
I think we should not mix things: the concept we want and
how to implement it. We should first focus on what we want
and than see how it fits into our architecture. Let's not
talk at this stage about SAX events etc.
So, my perception is that we want to have services that
can be used as a generator, as a transformer and as a
serializer.
The first question is: how can I call these services
and the second one is how can one define a service?
We saw some different examples on this and I personally like
a new syntax best to call a service instead of defining
a special component (generator, transformer, serializer).
As Stefano mentioned, we did this for content aggregation
as well.
And I think if we have a special syntax for calling a service
it's nearly natural to have a special concept for defining
the service as well.
And one interesting thought: I could imagine that as soon
as people can call a cocoon pipeline as a transformation
service they will want to call a web services doing the same
as well, so something like this comes into mind:
<generate src="hallo.xml"/>
<call-pipeline src="http://server/transformation"/>
<serialize>
Ok, ignore this last comment - I only had to show how usefull
web services can be - even for Cocoon :)
Carsten
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [RT] Using pipeline as sitemap components (long)
Posted by Stefano Mazzocchi <st...@apache.org>.
Vadim Gritsenko wrote:
> Christian Haul wrote:
>
> ...
>
>> A last thought: would it be necessary to have those pipelines match a
>> pattern or would it be sufficiant to have them names?
>>
>>
>
> After reading it second time... And seems that Christian also has
> concern about syntax... There are two ways to understand word "pipeline"
> and Sylvain's proposal.
>
>
> First Way: A-la Cocoon Source
> --------------------------------
>
> Figurally speaking: ProcessingPipeline which "extends" another
> ProcessingPipeline invokes sitemap (as cocoon source usually do), builds
> ProcessingPipeline object, and "inserts" it in place of own
> "pipeline"-typed component. Note: When sitemap is processed, all
> <pipeline> tags are scanned until complete ProcessingPipeline is built.
>
> Example sitemap:
> <pipeline>
> <match pattern="*.xml">
> <generate type="file" src="docs/{1}.xml"/>
> <serialize type="xml"/>
> </match>
> <match pattern="*.jpg">
> <read mime-type src="images/{1}.jpg"/>
> </match>
> </pipeline>
>
> <pipeline>
> <match pattern="*.html">
> <generate type="pipeline" src="{1}.xml"/>
> <transform src="xml2html.xslt"/>
> <serialize type="html"/>
> </match>
> </pipeline>
>
> Pipeline built after execution of "test.html" request:
> <generate type="file" src="docs/test.xml"/>
> <transform src="xml2html.xslt"/>
> <serialize type="html"/>
>
>
>
> Second Way: A-la Text Substitution
> ------------------------------------
>
> Figurally speaking: <pipeline> which "extends" another <pipeline> finds
> named <pipeline> tag and inserts the whole its content in place of
> "pipeline"-typed component. Note: only named <pipeline> is used, all
> other pipeline tags are ignored.
>
> Example sitemap (differs from the above!):
> <pipeline name="mypipeline">
> <match pattern="*.html">
> <generate type="file" src="docs/{1}.xml"/>
> <serialize type="xml"/>
> </match>
> <match pattern="*.jpg">
> <read mime-type src="images/{1}.jpg"/>
> </match>
> </pipeline>
>
> <pipeline>
> <match pattern="*.html">
> <generate type="pipeline" src="mypipeline"/>
> <transform src="xml2html.xslt"/>
> <serialize type="html"/>
> </match>
> </pipeline>
>
> After expansion is made (and serializer is removed):
> <pipeline>
> <match pattern="*.html">
> <match pattern="*.html">
> <generate type="file" src="docs/{1}.xml"/>
> </match>
> <match pattern="*.jpg">
> <read mime-type src="images/{1}.jpg"/>
> </match>
> <transform src="xml2html.xslt"/>
> <serialize type="html"/>
> </match>
> </pipeline>
>
> Pipeline built after execution of "test.html" request:
> <generate type="file" src="docs/test.xml"/>
> <transform src="xml2html.xslt"/>
> <serialize type="html"/>
>
>
> Results are the same, but please note the differences.
>
> Implementation via the first way is better named as "cocoon"-type
> components, second way can be named as "pipeline"-type components.
Very interesting reasoning, Vadim. I do see your point.
I don't think you are proposing that we move on the second approach, but
in case others wonder, I see the first design model much more desirable
than the second. Here is why:
1) named pipelines remind me of named xslt templates. I think everybody
here experienced the massive abuse named templates, since they look much
more 'procedural' to people.
It is very common to see <xsl:for-each> and then <xsl:call-template>,
while the very same thing could have been accomplished with a simple
declarative template match.
The sitemap was built to be highly declerative. This was a problem when
we didn't have a nice way to include procedural flow logic, but now that
we have the flowscript, we must make all possible efforts to keep the
sitemap semantics the more declarative as possible.
2) named pipeline will increase the chance of name collisions, unless a
serious effort for pipeline identification uniqueness is applied.
Note that a pipeline unique identifier will likely become a URI, thus
falling back on the first case.
3) having named pipelines allows different pipelines to match the same
URI space and reside in the same sitemap. That might create confusion.
- o -
This said, I really don't like the concept of having something that is
so important as pipeline inheritance being applied at the same semantic
level of components that can be removed from the system.
I mean: we managed to have <aggregate> (which is nothing but a
generator) built into the sitemap semantics, but we fail to have
something radically important as pipeline inheritance as a 'regular'
component.
The perceived difference between something like
<generate type="pipeline" src=".."/>
<transform .../>
<serialize .../>
and (I'm just making up a tag name, this is not mandatory at all)
<delegate-to-pipeline src="{1}.xml"/>
<transform .../>
<serialize .../>
or even
<generate .../>
<call-pipeline src="{1}.xml"/>
<transform .../>
<serialize .../>
is *very* different.
We must provide our users with "incremental learning paths". The value
of <map:aggregate> compared to <map:generate type="aggregator"> is that
users perceive the value of having such a concept built into the sitemap
and they can plan ahead to make an effort to learn the concept or,
simply, to put it on a side and learn it when they have time.
Flattening everythig at component level just because it is convenient
implementation-wise is, IMO, a bad design pattern for development of
usable technology.
What do you people think?
--
Stefano Mazzocchi <st...@apache.org>
--------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [RT] Using pipeline as sitemap components (long)
Posted by Vadim Gritsenko <va...@verizon.net>.
Christian Haul wrote:
...
>A last thought: would it be necessary to have those pipelines match a
>pattern or would it be sufficiant to have them names?
>
>
After reading it second time... And seems that Christian also has
concern about syntax... There are two ways to understand word "pipeline"
and Sylvain's proposal.
First Way: A-la Cocoon Source
--------------------------------
Figurally speaking: ProcessingPipeline which "extends"
another ProcessingPipeline invokes sitemap (as cocoon source usually
do), builds ProcessingPipeline object, and "inserts" it in place of own
"pipeline"-typed component. Note: When sitemap is processed, all
<pipeline> tags are scanned until complete ProcessingPipeline is built.
Example sitemap:
<pipeline>
<match pattern="*.xml">
<generate type="file" src="docs/{1}.xml"/>
<serialize type="xml"/>
</match>
<match pattern="*.jpg">
<read mime-type src="images/{1}.jpg"/>
</match>
</pipeline>
<pipeline>
<match pattern="*.html">
<generate type="pipeline" src="{1}.xml"/>
<transform src="xml2html.xslt"/>
<serialize type="html"/>
</match>
</pipeline>
Pipeline built after execution of "test.html" request:
<generate type="file" src="docs/test.xml"/>
<transform src="xml2html.xslt"/>
<serialize type="html"/>
Second Way: A-la Text Substitution
------------------------------------
Figurally speaking: <pipeline> which "extends" another <pipeline> finds
named <pipeline> tag and inserts the whole its content in place of
"pipeline"-typed component. Note: only named <pipeline> is used, all
other pipeline tags are ignored.
Example sitemap (differs from the above!):
<pipeline name="mypipeline">
<match pattern="*.html">
<generate type="file" src="docs/{1}.xml"/>
<serialize type="xml"/>
</match>
<match pattern="*.jpg">
<read mime-type src="images/{1}.jpg"/>
</match>
</pipeline>
<pipeline>
<match pattern="*.html">
<generate type="pipeline" src="mypipeline"/>
<transform src="xml2html.xslt"/>
<serialize type="html"/>
</match>
</pipeline>
After expansion is made (and serializer is removed):
<pipeline>
<match pattern="*.html">
<match pattern="*.html">
<generate type="file" src="docs/{1}.xml"/>
</match>
<match pattern="*.jpg">
<read mime-type src="images/{1}.jpg"/>
</match>
<transform src="xml2html.xslt"/>
<serialize type="html"/>
</match>
</pipeline>
Pipeline built after execution of "test.html" request:
<generate type="file" src="docs/test.xml"/>
<transform src="xml2html.xslt"/>
<serialize type="html"/>
Results are the same, but please note the differences.
Implementation via the first way is better named as "cocoon"-type
components, second way can be named as "pipeline"-type components.
Vadim
> Chris.
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [RT] Using pipeline as sitemap components (long)
Posted by Bertrand Delacretaz <bd...@codeconsult.ch>.
On Tuesday 26 November 2002 14:55, Christian Haul wrote:
>. ..what about e.g.
>
> <map:pipeline type="transformer" name="super_skin">
> <!-- ... -->
> </map:pipeline>
For someone like me who doesn't know a lot about sitemap and pipeline
internals, this looks very logical and understandable.
Logical extensions of this for other types of incomplete pipelines could
then be:
<map:pipeline type="source" name="xmlized-google">
<map:generate...>
<map:transform...>
<!-- no serializer, this is a "source only" pipeline -->
</map:pipeline>
(but source is not a good name, already used)
<map:pipeline type="sink" name="fo2rtf">
<!-- no generator, this is a "sink only" pipeline -->
<map:transform src="fix-fo-for-jfor.xsl"/>
<map:serialize type="rtf"/>
</map:pipeline>
Not sure if this is already covered by other mechanisms though, but it looks
logical for defining partial pipelines.
Alternatively they could just be called "pipe" which IRL is a part of a
"pipeline":
<map:pipe type="transformer">
-Bertrand
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [RT] Using pipeline as sitemap components (long)
Posted by Christian Haul <ha...@dvs1.informatik.tu-darmstadt.de>.
On 22.Nov.2002 -- 11:43 AM, Christian Haul wrote:
> On 21.Nov.2002 -- 11:05 AM, Sylvain Wallez wrote:
> >
> > Q: I want do define a pipeline that will be used only as a
> > transformation service. Why must I write a <map:generate> and a
> > <map:serialize> in its definition ?
> >
> > A: Because the sitemap, as a pipeline building language, must be able to
> > determine the start of a pipeline and its end, even if not all its
> > components are used. Like opening and closing braces in Java, the
> > generator begins the pipeline definition and the serializer ends it.
>
> We have the notion of pipelines in the sitemap. Today they are only
> used to get different error handlers. Maybe we should rethink this and
> use the map:pipeline as boundaries.
>
> Another idea would be to place those pipelines acting as generator,
> transformer, serializer in the same part as we do with other
> components of that kind. Problem would be that they would need to use
> other components just introduced there.
>
> A last thought: would it be necessary to have those pipelines match a
> pattern or would it be sufficiant to have them names?
Looks like my reply was too concise to be understandable.
OK, next try:
Sylvain explained, that
A) a generator is a component that produces SAX events
B) a transformer is a component that consumes SAX events
and produces SAX events
C) a serializer is a component that consumes SAX events
and produces a binary stream
Now, I'd like to add
D) a reader is a component that produces a binary stream
A "pipeline" could be any of these (according to the suggestion).
To be more precise: Currently, a pipeline is only D). In conjunction
with the cocoon: protocol, we can have A) as well. A resource is a C)
while we can actually use it as B) as well because of a "bug" in 2.1
This comparison is a bit unfair as a resource for example is only
visible in the current sitemap.
But a pipeline is more: it is also a contract for the external URI
space.
Now, if we step back a little and look at the heart of things, why
would a pipeline need to be referenced by anything else than its name,
and why is a pipeline acting as a transformer placed anywhere else
than the map:transformers section?
This is probably a bit too radical and faces implementation
problems. OTOH, what about e.g.
<map:pipeline type="transformer" name="super_skin">
<!-- ... -->
</map:pipeline>
in the "traditional" map:pipelines section. This could be turned into
a plain Avalon component filling the "transformer" role using the
short hand "super_skin".
Wait -- now, what about the external URI space? That would of course
need the present notation of pipeline. Without a type attribute, that
is. To use one of the new transformers, nothing chances:
<map:pipeline>
<map:match pattern="*">
<map:generate type="file" src="{1}.xml"/>
<map:transform type="super_skin"/>
<map:serialize type="xhtml"/>
</map:match>
</map:pipeline>
I imagine that this could be done with only some tweaking of the
current implementation.
It would however, change the way we currently use the
map:pipelines/map:pipeline section in the sitemap. But there seems to
be little use of it anyways.
So, what about blocks? Blocks would need to provide a way to let other
block access components declared within. Interestingly enough, this
gets easier as we have to deal with fewer different concepts that need
to be visible outside: components, classes, URIs
Oh yes, textual replacement wouldn't do for this.
Does this relate to XSL named templates? No, I don't think it is
related in any way.
So, what do you think?
Chris.
--
C h r i s t i a n H a u l
haul@informatik.tu-darmstadt.de
fingerprint: 99B0 1D9D 7919 644A 4837 7D73 FEF9 6856 335A 9E08
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [RT] Using pipeline as sitemap components (long)
Posted by Christian Haul <ha...@dvs1.informatik.tu-darmstadt.de>.
On 21.Nov.2002 -- 11:05 AM, Sylvain Wallez wrote:
> This document defines this concept of "pipeline service", which, as we
> will see, consists in using pipelines as sitemap components (generator,
> transformer and serializer). It is separated from the blocks design
> document since pipeline services can be used without blocks, even if
> they will be mostly useful in that context.
This sounds very natural to me. And as most of it is already possible
with a somewhat strange syntax I have mostly concerns with the
proposed new syntax.
>
> Q: I want do define a pipeline that will be used only as a
> transformation service. Why must I write a <map:generate> and a
> <map:serialize> in its definition ?
>
> A: Because the sitemap, as a pipeline building language, must be able to
> determine the start of a pipeline and its end, even if not all its
> components are used. Like opening and closing braces in Java, the
> generator begins the pipeline definition and the serializer ends it.
We have the notion of pipelines in the sitemap. Today they are only
used to get different error handlers. Maybe we should rethink this and
use the map:pipeline as boundaries.
Another idea would be to place those pipelines acting as generator,
transformer, serializer in the same part as we do with other
components of that kind. Problem would be that they would need to use
other components just introduced there.
A last thought: would it be necessary to have those pipelines match a
pattern or would it be sufficiant to have them names?
Chris.
--
C h r i s t i a n H a u l
haul@informatik.tu-darmstadt.de
fingerprint: 99B0 1D9D 7919 644A 4837 7D73 FEF9 6856 335A 9E08
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [RT] Using pipeline as sitemap components (long)
Posted by Sylvain Wallez <sy...@anyware-tech.com>.
Gianugo Rabellino wrote:
>> This document defines this concept of "pipeline service", which, as we
>> will see, consists in using pipelines as sitemap components (generator,
>> transformer and serializer). It is separated from the blocks design
>> document since pipeline services can be used without blocks, even if
>> they will be mostly useful in that context.
>
>
>
> Interesting read, Sylvain (BTW: you should have told us that some guys
> from your company were at Comdex :-) We met them yesterday at the
> Cocoon BOF and I would have been most interested to meet them there...
> anyway, kudos for the great job of putting Cocoon in such a small
> device, definitely impressing).
Actually, I didn't even knew ApacheCon and Comdex where so close to each
other :-/
I'm glad you liked this "Cocoon in unusual places" (see my Gent
presentation for more on this)
> I have a some problems, though, with the whole concept. First of all I
> hear FS bells all around, but I still have to focus more on the topic.
> The secon problem is about extending pipeline syntax in such a radical
> way: I think that this deserves a lot of thoughts about what might
> come out from this change.
>
> The third and last one has to do with the specific syntax. What you
> are labeling as a type="pipeline" attribute is actually the result of
> a matcher (i.e. a "<map:match src="whatever">" tag). This might really
> be confusing. But then again I can't think of a name that might really
> make sense: using <map:transform match="whatever"> is even worse, so I
> don't really see a solution here.
I think there is a misunderstanding here, related to the word
"pipeline". The "src" attribute of these new components is a sitemap
URI, just like in the "cocoon:" protocol. See my answer to Vadim for
more about this.
> What I see here is actually expressing in an explicit way the fact
> that when you are calling a resource via the cocoon protocol the
> serializer is infact ignored and the two pipelines are infact somehow
> "merged"... but I need to have more use scenarios where extending this
> might turn out useful without falling into FS.
I easily admit that this set of new components can be considered FS in a
single sitemap, in which the use of <map:resource> should be preferred
to reuse sitemap snippets (but then using a different naming scheme).
> Can you shed some more light on that, with special regard to the whole
> block concept?
See my answer to Stefano's block version 1.1 [1] : blocks define their
contracts by means of URIs handled by their respective sitemaps, but we
need blocks to provide not only resources (don't like much this name to
which I prefer "content"), but also services.
And these services, such as translating xdoc to html, producing PDF,
etc. cannot be described simply by URIs producing content, but by URIs
providing services such as generation, transformation and serialization.
Hence these new components.
Sylvain
[1] http://marc.theaimsgroup.com/?l=xml-cocoon-dev&m=103627852716993&w=2
--
Sylvain Wallez Anyware Technologies
http://www.apache.org/~sylvain http://www.anyware-tech.com
{ XML, Java, Cocoon, OpenSource }*{ Training, Consulting, Projects }
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [RT] Using pipeline as sitemap components (long)
Posted by Gianugo Rabellino <gi...@apache.org>.
> This document defines this concept of "pipeline service", which, as we
> will see, consists in using pipelines as sitemap components (generator,
> transformer and serializer). It is separated from the blocks design
> document since pipeline services can be used without blocks, even if
> they will be mostly useful in that context.
Interesting read, Sylvain (BTW: you should have told us that some guys
from your company were at Comdex :-) We met them yesterday at the Cocoon
BOF and I would have been most interested to meet them there... anyway,
kudos for the great job of putting Cocoon in such a small device,
definitely impressing).
I have a some problems, though, with the whole concept. First of all I
hear FS bells all around, but I still have to focus more on the topic.
The secon problem is about extending pipeline syntax in such a radical
way: I think that this deserves a lot of thoughts about what might come
out from this change.
The third and last one has to do with the specific syntax. What you are
labeling as a type="pipeline" attribute is actually the result of a
matcher (i.e. a "<map:match src="whatever">" tag). This might really be
confusing. But then again I can't think of a name that might really make
sense: using <map:transform match="whatever"> is even worse, so I don't
really see a solution here.
What I see here is actually expressing in an explicit way the fact that
when you are calling a resource via the cocoon protocol the serializer
is infact ignored and the two pipelines are infact somehow "merged"...
but I need to have more use scenarios where extending this might turn
out useful without falling into FS. Can you shed some more light on
that, with special regard to the whole block concept?
Ciao,
--
Gianugo Rabellino
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [RT] Using pipeline as sitemap components (long)
Posted by Sylvain Wallez <sy...@anyware-tech.com>.
John Morrison wrote:
>>From: Colin Paul Adams [mailto:colin@colina.demon.co.uk]
>>
>>
>>>>>>>"Sylvain" == Sylvain Wallez
>>>>>>>
>>>>>>>
>><sy...@anyware-tech.com> writes:
>>
>> Sylvain> <map:generate src="another_xdoc.xml"/> <map:serialize
>> Sylvain> type="pipeline" src="doc2pdf"/>
>>
>> Sylvain> Note : the "src" attribute doesn't currently exist on
>> Sylvain> <map:serialize>, but it seems the more natural and
>> Sylvain> consistent way to name the called pipeline. Wether this
>> Sylvain> translates to implementing SitemapModelComponent or not
>> Sylvain> is another story.
>>
>>It seems highly unnatural to me, as it is a drain rather than a
>>source.
>>How about drain="doc2pdf"?
>>
>>
>
>In engineering the opposite of source is "sink" :)
>
And what about "dest" for destination ?
Sylvain
--
Sylvain Wallez Anyware Technologies
http://www.apache.org/~sylvain http://www.anyware-tech.com
{ XML, Java, Cocoon, OpenSource }*{ Training, Consulting, Projects }
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
RE: [RT] Using pipeline as sitemap components (long)
Posted by John Morrison <jo...@ntlworld.com>.
> From: Colin Paul Adams [mailto:colin@colina.demon.co.uk]
> >>>>> "Sylvain" == Sylvain Wallez
> <sy...@anyware-tech.com> writes:
>
> Sylvain> <map:generate src="another_xdoc.xml"/> <map:serialize
> Sylvain> type="pipeline" src="doc2pdf"/>
>
> Sylvain> Note : the "src" attribute doesn't currently exist on
> Sylvain> <map:serialize>, but it seems the more natural and
> Sylvain> consistent way to name the called pipeline. Wether this
> Sylvain> translates to implementing SitemapModelComponent or not
> Sylvain> is another story.
>
> It seems highly unnatural to me, as it is a drain rather than a
> source.
> How about drain="doc2pdf"?
In engineering the opposite of source is "sink" :)
J.
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [RT] Using pipeline as sitemap components (long)
Posted by Colin Paul Adams <co...@colina.demon.co.uk>.
>>>>> "Sylvain" == Sylvain Wallez <sy...@anyware-tech.com> writes:
Sylvain> <map:generate src="another_xdoc.xml"/> <map:serialize
Sylvain> type="pipeline" src="doc2pdf"/>
Sylvain> Note : the "src" attribute doesn't currently exist on
Sylvain> <map:serialize>, but it seems the more natural and
Sylvain> consistent way to name the called pipeline. Wether this
Sylvain> translates to implementing SitemapModelComponent or not
Sylvain> is another story.
It seems highly unnatural to me, as it is a drain rather than a
source.
How about drain="doc2pdf"?
--
Colin Paul Adams
Preston Lancashire
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [RT] Using pipeline as sitemap components (long)
Posted by Daniel Fagerstrom <da...@nada.kth.se>.
Carsten Ziegeler wrote:
> Thanks Sylvain for this RT on a missing piece for the blocks concept.
I also find the concepts proposed by Sylvain useful and important and
will discuss below how they can be realized by extending the
cocoon:-protocol to be writable as well as readable.
> Sylvain Wallez wrote:
>
>><snip/>
>>
>>
>>Pipelines as generators
>>-----------------------
>>
>>This leads to a first conclusion : using a pipeline as a generator means
>>using the SAX events produced by the last XMLProducer of that pipeline,
>>i.e. the last transformer or the generator if there are no transformers.
>>
>
> This is only the technical implementations because of performance. The
> first implementation of the cocoon:-protocol actually required the xml
> serializer as the result was serialized and then parsed again - for
> performance reason we made it possible to use the SAX stream directly
> from the last XMLProducer.
> In fact, that the serializer does not have to be the xml serializer
> can be considered as a bug or weakness.
IMO this interpretation of what the cocoon:-protocol does is very
important. We have two possible interpretation about what the
cocoon:-protocol does: 1. It generates sax events from the last
XMLProducer in the pipeline or 2. it generates octets from the
serializer in the pipeline (which for the intended use cases is supposed
to form an xml-document).
I find the first interpretation problematic. Consider the case when have
a pipeline where the serializer is of pipeline type (i.e. it uses the
transformers and the serializer from another pipeline), and we use it
from the cocoon:-protocol in the generator in another pipeline. What
would it mean to take away the serializer in this case? Skipping the
pipeline serializer or skipping the serializer in the used pipeline?
With the second interpretation, i.e. the cocoon:-protocol always makes the
OutputStream from the serializer available, the example in the last
paragraph should have an obvious interpretation. As the cocoon:-protocol
is a protocol and such are normally supposed make an octet stream
available (an InputStream), it seem more natural that it should deliver
the octet stream from the serializer than an sax stream from the
pipeline step before the serializer.
From performance reasons a SAX stream should of course be available
from the cocoon:-protocol if the serializer is of xml type. As a minor
technical note: it would probably be more natural to let the
xml-serializer (and possibly the proposed pipeline serializer) implement
XMLProducer (or finding some other way to make sax events available from
some of the serializers).
If we use a pipeline ending with e.g. a pdf-serializer from the
cocoon:-protocol this should mean that it actually produces a pdf octet
stream and that it would be an error to use it as source for e.g. the
file generator. If we consider it FS to let the cocoon:-protocol
generating anything else than xml, it should be considered an error to
use a non xml producing serializer from the cocoon:-protocol.
<snip/>
>>Pipelines as serializers
>>------------------------
>>
>><snip/>
>>
>>How do we use this ? Well, just as for the generator, let's define a new
>>"pipeline" serializer :
>>
>> <map:generate src="another_xdoc.xml"/>
>> <map:serialize type="pipeline" src="doc2pdf"/>
>>
>>Note : the "src" attribute doesn't currently exist on <map:serialize>,
>>but it seems the more natural and consistent way to name the called
>>pipeline. Wether this translates to implementing SitemapModelComponent
>>or not is another story.
>>
>
> Ok, I would call this "src" but something different, but that doesn't
> play a role for the concept itself.
> What I don't like is that a complete pipeline is called and there
> the generator is ignored. This would confuse everyone, I guess.
I agree, just ignoring the generator would be confusing. There seem to
be two options: creating a new sitemap construction for pipelines
without generator and or letting the generator do something meaningful.
The first option has already been discussed in several comments, here I
would like to see what happens if we try to follow the later path.
If we continue along the way hinted by Carstens comment about the
cocoon:-protocol above, a pipeline as a serializer could be thought
about as an writable protocol and as such making an writable octet
stream (an OutputStream) available (as it is used as a serializer it is
of course also supposed to generate output). We would thus like to make
the cocoon:-protocol writable as well. Of course the writable
cocoon:-protocol should make an content handler for sax input available
when appropriate, but this is only for optimization purposes.
Here we immediately runs into the problem that generators not are the
inverse to serializers. To get into more technical detail: a serializer
implements SitemapOutputComponent which contains the method
setOutputStream. The OutputStream is set by the context, in ordinary
servlet usage it is set to the OutputStream of the HttpResponse object.
This is analogue to unix pipelines where pipeline components writes to
standard output and the context can redirect the standard output.
Generators however do not implement anything corresponding
"SitemapInputComponent"-interface, instead of getting their InputStream
from the context, they are responsible for finding they are responsible
for finding their input, (typically by applying a SourceResolver on the
content of the src attribute). If we continue the unix analogue a
generator would correspond to a command that ignore the standard input
and instead requires a file name in the parameter list.
If we, as a thought experiment, would like generators to be the
"inverse" to serializers they should implement a
"SitemapInputComponent"-interface which would contain a setInputStream
method or possibly a setInputSource method it should probably also
contain a getContentHandler for efficiency reasons when the input
happens to be xml. The input would then be set from the src attribute
when existent and from the "standard input" set from the context
otherwise. In servlet use, I guess that standard input would correspond
to the InputStream from the request object, when used as a writable
cocoon:-protocol, standard input would be set to whatever writes into
the protocol.
Now I am of course completely aware that would be a bad idea to change
the Generator interface instead one could maybe let the FileGenerator
and the proposed PipelineGenerator (and possibly some other), implement
some "WritableGenerator"-interface containing the methods listed above.
Then the context, e.g. the writable cocoon:-protocol or the servlet
could set the input for a WritableGenerator and ignore it for ordinary
an ordinary Generator.
>>Pipelines as transformers
>>-------------------------
<snip/>
We can use the above described writable cocoon:-protocol to reuse
pipelines as transformers as well, here we must of course require booth
input and output to be xml. We could have something like:
<map:transform type="pipeline" src="cocoon:foo"/>
and
<map:transform type="pipeline" src="block:bar:foo"/>
or maybe a more general construct that also can use SOAP services or WebDAV:
<map:transform type="protocol" src="http://bar.org/foo"/>
-------------
What do you think?
Daniel Fagerstrom
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [RT] Using pipeline as sitemap components (long)
Posted by Konstantin Piroumian <kp...@apache.org>.
From: "Carsten Ziegeler" <cz...@s-und-n.de>
> Thanks Sylvain for this RT on a missing piece for the blocks concept.
> More inline:
>
> Sylvain Wallez wrote:
> >
> > <snip/>
[...]
> >
> > Pipelines as transformers
> > -------------------------
> >
> > <snip/>
> >
> > <map:match pattern="a_page">
> > <map:generate src="an_xdoc.xml"/>
> > <map:transform type="pipeline" src="xdoc2skinnedHtml"/>
> > <map:serialize type="html"/>
> > </map:match>
> >
> > and
> >
> > <map:match pattern="xdoc2skinnedHtml">
> > <map:generate type="dont_care"/>
> > <map:transform type="i18n"/>
> > <map:transform type="xdoc2html.xsl"/>
> > <map:transform type="htmlskin.xsl"/>
> > <map:serialize type="dont_care"/>
> > </map:match>
> >
> Again, this looks a little bit strange that generator and serializer
> have to be specified but are actually ignored. I can already hear
> thousands of complains from users telling: "Why do I have to declare
> them when they are not used?".
>
> I haven't thought much about this, but I think if we want to define
> services which can be called, than we should have a special syntax for
> this, to make explicitly clear that services are defined and not
> pipelines.
Why not <map:resource />? Current implementation of resources is very
similar to a callable service, but has some, IMO, artificial limitations. By
removing those limitations it'll be possible to declare resources as
services and then <map:call /> them where needed.
>
> I personally would like to add new concepts for new things rather
> than to try to press them into existing concepts where they do not
> fit 100%.
The current sitemap language is already too complex. Adding more new things
to it would make it even worse. Why not extend the current concepts to fit
the needs rather than inventing new ones?
>
> It's a pitty that we didn't had time to discuss this face-to-face,
> because that's a lot easier as I learned from the blocks concept...
For me, it's much easier to read English than comprehend the speach. But
it's really pity that I wasn't able to attend to GetTogether. Hopefully,
next year...
Konstantin
>
> Carsten
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
> For additional commands, email: cocoon-dev-help@xml.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [RT] Using pipeline as sitemap components (long)
Posted by Sylvain Wallez <sy...@anyware-tech.com>.
Carsten Ziegeler wrote:
>Thanks Sylvain for this RT on a missing piece for the blocks concept.
>More inline:
>
>Sylvain Wallez wrote:
>
<snip/>
>>I don't see a need for a new sitemap element such as "map:call-pipeline"
>>or "map:generate-from-pipeline". What we want is to generate and initial
>>content in the current pipeline, and for this we just use a particular
>>implementation of a generator, as we already do for files, XSP, etc.
>>
>
>I don't see the need for this special generator as the current components
>(file generator - which has not the best name) do it already. Another
>component would imho more confuse than help.
>
This generator is there for consistency with the transformer and
serializer : a whole set is defined. Also, its implementation will be
different and maybe allow better performance and/or caching than the
current source (maybe not with the SourceValidity introduced in 2.1).
>And what do you want to do with map:aggregate?
>
We keep the "cocoon:" protocol for this. But I always have been
frustrated that map:part only accepts a simple "src", and would love to
be able to put sitemap statements in it. But that's another story...
>>Pipelines as serializers
>>------------------------
>>
>><snip/>
>>
>>How do we use this ? Well, just as for the generator, let's define a new
>>"pipeline" serializer :
>>
>> <map:generate src="another_xdoc.xml"/>
>> <map:serialize type="pipeline" src="doc2pdf"/>
>>
>>Note : the "src" attribute doesn't currently exist on <map:serialize>,
>>but it seems the more natural and consistent way to name the called
>>pipeline. Wether this translates to implementing SitemapModelComponent
>>or not is another story.
>>
>>
>>
>Ok, I would call this "src" but something different, but that doesn't
>play a role for the concept itself.
>What I don't like is that a complete pipeline is called and there
>the generator is ignored. This would confuse everyone, I guess.
>
</snip>
>Again, this looks a little bit strange that generator and serializer
>have to be specified but are actually ignored. I can already hear
>thousands of complains from users telling: "Why do I have to declare
>them when they are not used?".
>
I thought of defining some "not-used" generator and serializer, if you
really want to constrain a particular use of a pipeline. These would for
example simply throw an exception explaining that this pipeline is not
used as expected.
>I haven't thought much about this, but I think if we want to define
>services which can be called, than we should have a special syntax for
>this, to make explicitly clear that services are defined and not
>pipelines.
>
>I personally would like to add new concepts for new things rather
>than to try to press them into existing concepts where they do not
>fit 100%.
>
Mmmh... actually, the concept is new only on the called block/sitemap
part. On the caller side, we just want a
generator/transformer/serializer. Also, please refer to Stefano's post
where he explains the usefulness of complete pipelines for testing and
reuse.
>It's a pitty that we didn't had time to discuss this face-to-face,
>because that's a lot easier as I learned from the blocks concept...
>
Yep. But we discussed a lot of other topics that were also interesting.
Next year, we should manage to have more time for BOFs or in-depth
design sessions.
Sylvain
--
Sylvain Wallez Anyware Technologies
http://www.apache.org/~sylvain http://www.anyware-tech.com
{ XML, Java, Cocoon, OpenSource }*{ Training, Consulting, Projects }
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
RE: [RT] Using pipeline as sitemap components (long)
Posted by Carsten Ziegeler <cz...@s-und-n.de>.
Thanks Sylvain for this RT on a missing piece for the blocks concept.
More inline:
Sylvain Wallez wrote:
>
> <snip/>
>
>
> Pipelines as generators
> -----------------------
>
> This leads to a first conclusion : using a pipeline as a generator means
> using the SAX events produced by the last XMLProducer of that pipeline,
> i.e. the last transformer or the generator if there are no transformers.
>
This is only the technical implementations because of performance. The
first implementation of the cocoon: protocol actually required the xml
serializer as the result was serialized and then parsed again - for
performance reason we made it possible to use the SAX stream directly
from the last XMLProducer.
In fact, that the serializer does not have to be the xml serializer
can be considered as a bug or weakness.
> Since we've used a pipeline as a genererator, let's introduce a new
> generator for this purpose, instead of using the "file" one, which fools
> us in thinking we use a full pipeline when it actually strips out the
> serializer :
>
> <map:match src="first-uri">
> <map:generate type="pipeline" src="/other-uri"/>
> <map:transform src="foo.xslt"/>
> <map:serialize/>
> </map:match/>
>
> I don't see a need for a new sitemap element such as "map:call-pipeline"
> or "map:generate-from-pipeline". What we want is to generate and initial
> content in the current pipeline, and for this we just use a particular
> implementation of a generator, as we already do for files, XSP, etc.
>
I don't see the need for this special generator as the current components
(file generator - which has not the best name) do it already. Another
component would imho more confuse than help. And what do you want to do
with map:aggregate?
>
> Pipelines as serializers
> ------------------------
>
> <snip/>
>
> How do we use this ? Well, just as for the generator, let's define a new
> "pipeline" serializer :
>
> <map:generate src="another_xdoc.xml"/>
> <map:serialize type="pipeline" src="doc2pdf"/>
>
> Note : the "src" attribute doesn't currently exist on <map:serialize>,
> but it seems the more natural and consistent way to name the called
> pipeline. Wether this translates to implementing SitemapModelComponent
> or not is another story.
>
Ok, I would call this "src" but something different, but that doesn't
play a role for the concept itself.
What I don't like is that a complete pipeline is called and there
the generator is ignored. This would confuse everyone, I guess.
>
> Pipelines as transformers
> -------------------------
>
> <snip/>
>
> <map:match pattern="a_page">
> <map:generate src="an_xdoc.xml"/>
> <map:transform type="pipeline" src="xdoc2skinnedHtml"/>
> <map:serialize type="html"/>
> </map:match>
>
> and
>
> <map:match pattern="xdoc2skinnedHtml">
> <map:generate type="dont_care"/>
> <map:transform type="i18n"/>
> <map:transform type="xdoc2html.xsl"/>
> <map:transform type="htmlskin.xsl"/>
> <map:serialize type="dont_care"/>
> </map:match>
>
Again, this looks a little bit strange that generator and serializer
have to be specified but are actually ignored. I can already hear
thousands of complains from users telling: "Why do I have to declare
them when they are not used?".
I haven't thought much about this, but I think if we want to define
services which can be called, than we should have a special syntax for
this, to make explicitly clear that services are defined and not
pipelines.
I personally would like to add new concepts for new things rather
than to try to press them into existing concepts where they do not
fit 100%.
It's a pitty that we didn't had time to discuss this face-to-face,
because that's a lot easier as I learned from the blocks concept...
Carsten
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org