You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cocoon.apache.org by Nils Kaiser <Ni...@gmx.de> on 2006/03/13 18:48:33 UTC
Is Cocoon the right choice?
Hello!
Well, I am an happy cocoon user but I am working on a new project and not
sure if cocoon is the right choice for it. The goal of the thing is to be
able to extract information from existing web sites and to transform it and
publish it to different channels. I already have built a project with cocoon
and from the basic idea it covers pretty much with what coocoon is.
The bad thing is that we have some complex requirements for this project and
I am not sure how these can be solved using existing cocoon technology. Here
what we need to be able to do:
1) Logical decisions in the pipeline depending on pipeline content... Yes I
know this is not in the idea of the current cocoon pipeline. But seems to be
pretty important in this project. On a higher view it might be the necessity
to run transformers only if a certain condition refering to the pipeline
applies.
2) As we are working with a lot of information coming from different pages
and 'following' links to get this information, we need a more dynamical
approach to get content. Example:
User requests our URL
- GET content of page 1
- Page 1 has a link to page 2 (which is not known before)
- GET content of page 2
- Transform content of page 1 and page 2
We cant be sure how many links we have to follow and we have to do some
transformations after the request so this can't be a single component... The
problem I see is not accessing and transforming the content. It is the fact
of "restarting" the pipeline dynamically (depending on the content of page
1).
I am not sure how this is possible with cocoon, especially because I have
only little knowledge of flow. Is it the answer??
I also know that it is possible to adress those issues with some tricks: for
example using a special transformer followed by a CIncludeTransformer to
call the same pipeline using cocoon:// for implementing pipeline "restart",
or having a set of DOM transformers which make decisions and write things to
the session to simulate conditional behavior of the pipeline.
I am very open to innovative solutions like having an own pipeline
implementation or a DOM based framework which can adress content based
decisions better.
I am just afraid we might end with a kind of monster doing some ugly things
to force cocoon to do thing it is not supposed to ;)
Thx for your comments,
Nils
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org
Re: Is Cocoon the right choice?
Posted by Ralph Goers <Ra...@dslextreme.com>.
Sorry, wrong name. It is the XMLFileModule.
http://cocoon.apache.org/2.1/apidocs/org/apache/cocoon/components/modules/input/XMLFileModule.html.
This message provides some insight
http://www.nabble.com/RE:-XMLFileModule-for-a-dynamic-file.-p355608.html.
However, before you use this please read
http://issues.apache.org/jira/browse/COCOON-1574?page=all.
Ralph
Kamal Bhatt wrote:
>
> Ralph Goers wrote:
>
>> Reinhard Poetz wrote:
>>
>>>
>>> What you're asking for is a long-time no-no of Cocoon which we call
>>> "dynamic pipelines". Currently there is no pipeline implementation
>>> that would allow this.
>>
>> I hesitate to mention this, but this isn't strictly true. We are
>> currenly using the XMLInputModule to dynamically generate the src url
>> for a transformer. This is,
>
> Where can I find more information about this? I cannot seem to find
> any api or documentation on XMLInputModule.
>
> Thanks
>
>> in effect, dynamically choosing which pipeline to use. I hesitate to
>> mention this because the XMLInputModule has severe caching problems
>> that need to be addressed.
>>
>> Ralph
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
>> For additional commands, e-mail: users-help@cocoon.apache.org
>>
>>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org
Re: Is Cocoon the right choice?
Posted by Kamal Bhatt <kb...@tt.com.au>.
Ralph Goers wrote:
> Reinhard Poetz wrote:
>
>>
>> What you're asking for is a long-time no-no of Cocoon which we call
>> "dynamic pipelines". Currently there is no pipeline implementation
>> that would allow this.
>
> I hesitate to mention this, but this isn't strictly true. We are
> currenly using the XMLInputModule to dynamically generate the src url
> for a transformer. This is,
Where can I find more information about this? I cannot seem to find any
api or documentation on XMLInputModule.
Thanks
> in effect, dynamically choosing which pipeline to use. I hesitate to
> mention this because the XMLInputModule has severe caching problems
> that need to be addressed.
>
> Ralph
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
> For additional commands, e-mail: users-help@cocoon.apache.org
>
>
--
Kamal Bhatt
Tourism Technology Pty Ltd
T: +61 (02) 9956 7300
F: +61 (02) 9956 7411
Level 10, 1 Pacific Highway, North Sydney, NSW 2060
http://www.tt.com.au
--
Disclaimer: This email is confidential and may contain privileged information for the sole use of the person or business to which it is addressed. If you are not the intended recipient, please notify the sender by return e-mail or phone as you must not view, disseminate, distribute or copy this email without our consent. We do not accept any liability in connection with any computer virus, data corruption, incompleteness, or unauthorised amendment of this email. It is the sole responsibility of the receiver to scan for viruses before opening.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org
Re: Is Cocoon the right choice?
Posted by Ralph Goers <Ra...@dslextreme.com>.
Reinhard Poetz wrote:
>
> What you're asking for is a long-time no-no of Cocoon which we call
> "dynamic pipelines". Currently there is no pipeline implementation
> that would allow this.
I hesitate to mention this, but this isn't strictly true. We are
currenly using the XMLInputModule to dynamically generate the src url
for a transformer. This is, in effect, dynamically choosing which
pipeline to use. I hesitate to mention this because the XMLInputModule
has severe caching problems that need to be addressed.
Ralph
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org
XSL transforms based on xpath expression (was: Re: Is Cocoon the
right choice?)
Posted by Sandor Spruit <sa...@cs.uu.nl>.
Nils Kaiser wrote:
> Reinhard Poetz schrieb:
>> Nils Kaiser wrote:
>>
>> What you're asking for is a long-time no-no of Cocoon which we call
>> "dynamic pipelines". Currently there is no pipeline implementation
>> that would allow this. That's the bad news. The good news is that
>> the ice is breaking :-) I understand that this doesn't help you very
>> much ATM, Cocoon gets massivly refactored which will make it much
>> simpler to plugin your own processor implementations. I don't know in
>> which timeframes you're thinking, but if you can wait some more
>> weeks, you should get some solid ground to start from to contribute
>> to a dynamic pipelines implementation (which maybe doesn't need to be
>> XML based ...)
>>
> Well thats very good to know! I'll follow the developments on the dev
> list. Is this cocoon 3.0 or supposed to be part of 2.2?
>
> The concrete usecase in this project is that we want to apply
> XSL-Transforms based on xpath expressions.
This usecase has been discussed before by yours truely and others.
We ended-up creating a separate pipeline to generate the XSLT,
i.e. treat a stylesheet as just another XML document that needs to
be generated. You can then insert an xpath expression into your
XSLT wherever you like. It's not ideal, but it works quite well.
HTH,
Sandor
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org
Re: Is Cocoon the right choice?
Posted by Reinhard Poetz <re...@apache.org>.
Nils Kaiser wrote:
> Reinhard Poetz schrieb:
>
>> Nils Kaiser wrote:
>>
>> What you're asking for is a long-time no-no of Cocoon which we call
>> "dynamic pipelines". Currently there is no pipeline implementation
>> that would allow this. That's the bad news. The good news is that
>> the ice is breaking :-) I understand that this doesn't help you very
>> much ATM, Cocoon gets massivly refactored which will make it much
>> simpler to plugin your own processor implementations. I don't know in
>> which timeframes you're thinking, but if you can wait some more weeks,
>> you should get some solid ground to start from to contribute to a
>> dynamic pipelines implementation (which maybe doesn't need to be XML
>> based ...)
>>
> Well thats very good to know! I'll follow the developments on the dev
> list. Is this cocoon 3.0 or supposed to be part of 2.2?
The next major Cocoon version hasn't got a version number yet. I was speaking of
the code in SVN trunk ;-)
> The concrete usecase in this project is that we want to apply
> XSL-Transforms based on xpath expressions. Imagine 5 XSL Transforms with
> a condition for each of them. At the moment I don't see any other
> solution than having a Transformer which builds a DOM and runs the
> conditions and the XSLs... This would also be better in term of
> performance, because I can feed the DOM directly to the transformer. The
> disadvantage is that I cannot use other cocoon components as I move the
> XSL transformations from the sitemap context to my own transformer...
>
> The alternative would be an action building a DOM and evaluating an
> XPath. This would allow me to use other cocoon components but I am
> afraid of performance problems due to frequent DOM building and
> writing... Has someone experienced something similar? How bad do DOM
> based transformers degrade performance in general??
A DOM based implementation is far from being ideal as you have to create DOM
documents for every stage. Cocoon 1 was DOM based and a lot of the users had
huge memory problems.
On the dev-list some of our developers where talking about pull-pipelines.
You're more than welcome to join dev@cocoon if you want to share your ideas with
us :-)
>> Your second requirement sounds to be easier to be implemented based on
>> the current codebase, but I'm not sure if I understand it completly. I
>> assume that you were talking about a pipeline like this:
>>
>> <map:match pattern="aggregator/**">
>> <map:generate type="html" src="http://{1}"/>
>> <map:transform src="filter-all-links.xslt"/>
>> <map:transform type="cinclude"/>
>> <map:serialize/>
>> </map:match>
>>
> Yes thats what I'm talking about. I think we only will need to follow
> one link. The problem is more that
> we'll have multiple 'exit points' at different steps of the pipeline.
> For example, we might be doing somthing after the cinclude transformer
> in your example. When the pipeline gets called by the cinclude, I have
> to take care not to run transformations twice. The is a consequence of
> the fact to have a subcall instead of a real restart.
>
> <map:match pattern="content/**">
> <map:generate type="html" src="http://{1}"/>
> <map:transform src="process_or_cinclude.xsl"/>
> <map:transform type="cinclude"/>
> <map:transform src="process_or_cinclude2.xsl"/>
> <map:transform type="cinclude"/>
> <map:transform src="makeitnice.xsl"/>
> <map:serialize/>
> </map:match>
>
> So if the pipeline gets called by the first cinclude, I will run
> makeitnice.xsl (and process_or_include2.xsl) twice. Well I could take
> care of this by using views... is that an idea?
or adding some selectors that check if some parameter is set by the caller could
solve your problem.
--
Reinhard Pötz Independent Consultant, Trainer & (IT)-Coach
{Software Engineering, Open Source, Web Applications, Apache Cocoon}
web(log): http://www.poetz.cc
--------------------------------------------------------------------
___________________________________________________________
Telefonate ohne weitere Kosten vom PC zum PC: http://messenger.yahoo.de
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org
Re: Is Cocoon the right choice?
Posted by Nils Kaiser <Ni...@gmx.de>.
Reinhard Poetz schrieb:
> Nils Kaiser wrote:
>
> What you're asking for is a long-time no-no of Cocoon which we call
> "dynamic pipelines". Currently there is no pipeline implementation
> that would allow this. That's the bad news. The good news is that
> the ice is breaking :-) I understand that this doesn't help you very
> much ATM, Cocoon gets massivly refactored which will make it much
> simpler to plugin your own processor implementations. I don't know in
> which timeframes you're thinking, but if you can wait some more weeks,
> you should get some solid ground to start from to contribute to a
> dynamic pipelines implementation (which maybe doesn't need to be XML
> based ...)
>
Well thats very good to know! I'll follow the developments on the dev
list. Is this cocoon 3.0 or supposed to be part of 2.2?
The concrete usecase in this project is that we want to apply
XSL-Transforms based on xpath expressions. Imagine 5 XSL Transforms with
a condition for each of them. At the moment I don't see any other
solution than having a Transformer which builds a DOM and runs the
conditions and the XSLs... This would also be better in term of
performance, because I can feed the DOM directly to the transformer. The
disadvantage is that I cannot use other cocoon components as I move the
XSL transformations from the sitemap context to my own transformer...
The alternative would be an action building a DOM and evaluating an
XPath. This would allow me to use other cocoon components but I am
afraid of performance problems due to frequent DOM building and
writing... Has someone experienced something similar? How bad do DOM
based transformers degrade performance in general??
>
> Your second requirement sounds to be easier to be implemented based on
> the current codebase, but I'm not sure if I understand it completly. I
> assume that you were talking about a pipeline like this:
>
> <map:match pattern="aggregator/**">
> <map:generate type="html" src="http://{1}"/>
> <map:transform src="filter-all-links.xslt"/>
> <map:transform type="cinclude"/>
> <map:serialize/>
> </map:match>
>
Yes thats what I'm talking about. I think we only will need to follow
one link. The problem is more that
we'll have multiple 'exit points' at different steps of the pipeline.
For example, we might be doing somthing after the cinclude transformer
in your example. When the pipeline gets called by the cinclude, I have
to take care not to run transformations twice. The is a consequence of
the fact to have a subcall instead of a real restart.
<map:match pattern="content/**">
<map:generate type="html" src="http://{1}"/>
<map:transform src="process_or_cinclude.xsl"/>
<map:transform type="cinclude"/>
<map:transform src="process_or_cinclude2.xsl"/>
<map:transform type="cinclude"/>
<map:transform src="makeitnice.xsl"/>
<map:serialize/>
</map:match>
So if the pipeline gets called by the first cinclude, I will run
makeitnice.xsl (and process_or_include2.xsl) twice. Well I could take
care of this by using views... is that an idea?
Thx for helping!
Nils
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org
Re: Is Cocoon the right choice?
Posted by Reinhard Poetz <re...@apache.org>.
Nils Kaiser wrote:
> Hello!
>
> Well, I am an happy cocoon user but I am working on a new project and not
> sure if cocoon is the right choice for it. The goal of the thing is to be
> able to extract information from existing web sites and to transform it and
> publish it to different channels. I already have built a project with cocoon
> and from the basic idea it covers pretty much with what coocoon is.
>
> The bad thing is that we have some complex requirements for this project and
> I am not sure how these can be solved using existing cocoon technology. Here
> what we need to be able to do:
>
> 1) Logical decisions in the pipeline depending on pipeline content... Yes I
> know this is not in the idea of the current cocoon pipeline. But seems to be
> pretty important in this project. On a higher view it might be the necessity
> to run transformers only if a certain condition refering to the pipeline
> applies.
>
> 2) As we are working with a lot of information coming from different pages
> and 'following' links to get this information, we need a more dynamical
> approach to get content. Example:
>
> User requests our URL
> - GET content of page 1
> - Page 1 has a link to page 2 (which is not known before)
> - GET content of page 2
> - Transform content of page 1 and page 2
>
> We cant be sure how many links we have to follow and we have to do some
> transformations after the request so this can't be a single component... The
> problem I see is not accessing and transforming the content. It is the fact
> of "restarting" the pipeline dynamically (depending on the content of page
> 1).
>
> I am not sure how this is possible with cocoon, especially because I have
> only little knowledge of flow. Is it the answer??
>
> I also know that it is possible to adress those issues with some tricks: for
> example using a special transformer followed by a CIncludeTransformer to
> call the same pipeline using cocoon:// for implementing pipeline "restart",
> or having a set of DOM transformers which make decisions and write things to
> the session to simulate conditional behavior of the pipeline.
>
> I am very open to innovative solutions like having an own pipeline
> implementation or a DOM based framework which can adress content based
> decisions better.
>
> I am just afraid we might end with a kind of monster doing some ugly things
> to force cocoon to do thing it is not supposed to ;)
What you're asking for is a long-time no-no of Cocoon which we call "dynamic
pipelines". Currently there is no pipeline implementation that would allow this.
That's the bad news. The good news is that the ice is breaking :-) I
understand that this doesn't help you very much ATM, Cocoon gets massivly
refactored which will make it much simpler to plugin your own processor
implementations. I don't know in which timeframes you're thinking, but if you
can wait some more weeks, you should get some solid ground to start from to
contribute to a dynamic pipelines implementation (which maybe doesn't need to be
XML based ...)
Having said this, if you have time and the needs to work on Cocoon itself and
want to get involved in desicions, it is the *perfekt* point of time now for
this as nothing is carved in stone :-)
Your second requirement sounds to be easier to be implemented based on the
current codebase, but I'm not sure if I understand it completly. I assume that
you were talking about a pipeline like this:
<map:match pattern="aggregator/**">
<map:generate type="html" src="http://{1}"/>
<map:transform src="filter-all-links.xslt"/>
<map:transform type="cinclude"/>
<map:serialize/>
</map:match>
The filter-all-links.xslt generates cinclude elements, that recursivly call the
aggregator/** pipeline. You'd also have to implement the decision there if you
want to follow a link or not. If this involved some complex desicions, you would
have to implement some custom transformer that can make this decision as
writting Java is for some usecases easier than writing XSLT - in particular if
you have to call some kind of business logc. If you have to follow all links (=
no decision), you will have to make sure that you don't create an infinite loop
but I guess you have already thought of this ;-)
One final thought: I don't know where all those pages that you want to aggregate
are located but you could run into some serious performance problems. It would
be a good idea to think of caching right from the beginning.
--
Reinhard Pötz Independent Consultant, Trainer & (IT)-Coach
{Software Engineering, Open Source, Web Applications, Apache Cocoon}
web(log): http://www.poetz.cc
--------------------------------------------------------------------
___________________________________________________________
Telefonate ohne weitere Kosten vom PC zum PC: http://messenger.yahoo.de
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org
Re: Is Cocoon the right choice?
Posted by Martynas Jusevicius <ma...@gmail.com>.
For me Cocoon also looked like a perfect choice at first, and sitemap
with URL-matching as a perfect idea. But when a simple static sitemap
is becoming dynamic and involves logic or actions, it gets nasty. Some
flow control, a concept which I don't like at all, some JavaScript
gets involved, dummyn aggregations and transform results, dozens of
(never used) modules.You might find yourself trying to work around
sitemap's restrictions while building quite a simple thing and ending
up with strange and unreliable mix of technologies, some of them being
Cocoon-only. At this point, you might think: I would already have done
that with servlets or just PHP.
I would suggest that you check out XQuery. With an extension of
servlet-like functionality it became my favorite tool for web
development. Take a look at XQuery Server Pages or eXist XML DB.
Martynas
On 3/13/06, Nils Kaiser <Ni...@gmx.de> wrote:
> Hello!
>
> Well, I am an happy cocoon user but I am working on a new project and not
> sure if cocoon is the right choice for it. The goal of the thing is to be
> able to extract information from existing web sites and to transform it and
> publish it to different channels. I already have built a project with cocoon
> and from the basic idea it covers pretty much with what coocoon is.
>
> The bad thing is that we have some complex requirements for this project and
> I am not sure how these can be solved using existing cocoon technology. Here
> what we need to be able to do:
>
> 1) Logical decisions in the pipeline depending on pipeline content... Yes I
> know this is not in the idea of the current cocoon pipeline. But seems to be
> pretty important in this project. On a higher view it might be the necessity
> to run transformers only if a certain condition refering to the pipeline
> applies.
>
> 2) As we are working with a lot of information coming from different pages
> and 'following' links to get this information, we need a more dynamical
> approach to get content. Example:
>
> User requests our URL
> - GET content of page 1
> - Page 1 has a link to page 2 (which is not known before)
> - GET content of page 2
> - Transform content of page 1 and page 2
>
> We cant be sure how many links we have to follow and we have to do some
> transformations after the request so this can't be a single component... The
> problem I see is not accessing and transforming the content. It is the fact
> of "restarting" the pipeline dynamically (depending on the content of page
> 1).
>
> I am not sure how this is possible with cocoon, especially because I have
> only little knowledge of flow. Is it the answer??
>
> I also know that it is possible to adress those issues with some tricks: for
> example using a special transformer followed by a CIncludeTransformer to
> call the same pipeline using cocoon:// for implementing pipeline "restart",
> or having a set of DOM transformers which make decisions and write things to
> the session to simulate conditional behavior of the pipeline.
>
> I am very open to innovative solutions like having an own pipeline
> implementation or a DOM based framework which can adress content based
> decisions better.
>
> I am just afraid we might end with a kind of monster doing some ugly things
> to force cocoon to do thing it is not supposed to ;)
>
> Thx for your comments,
>
> Nils
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
> For additional commands, e-mail: users-help@cocoon.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org