You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cocoon.apache.org by Reinhard Poetz <re...@apache.org> on 2008/03/26 14:14:37 UTC
Layered software designs
Carsten Ziegeler wrote:
> The question is now if we need support for caching in the low level apis
> or if it is possible to have a layered approach - which would make the
> entry barrier much easier.
Yes, this layered approach is what I'm aiming for. All the reactions in this
thread make me think that everybody, who has commented on this mailing list so
far (except Carsten), believes that we want to throw away good things that have
profed to be useful in many situations.
Rest assured, that's not the case. Carsten and I only want to break up this
all-or-nothing situation that we (still) have now.
What I want to see is a concise pipeline API that comes with only little
overhead in terms of its learning curve and its dependencies on third-party
software. Usually this means that we stick with standard APIs as much as
possible - and I think this rule applies for our situation too.
This means that the user of the API only needs to learn as little as possible.
When he wants more, we offer additional modules that help him. Since he has a
concrete need, the motiviation to learn something new is much higher than when
he has to learn everything right from the beginning.
If you want to learn how this whole concept *might* apply for a next generation
Cocoon, have a look at Steven's and my "Exploring Corona" mail from last week
(http://marc.info/?l=xml-cocoon-dev&m=120611990603725&w=2).
The idea of Corona is having a concise core that doesn't have any dependencies
on a particular component container (Spring, OSGi, etc.), source resolving
mechanisms or environment (http, java only, etc.) or even the type of the
components (XML-SAX event stream, XML-Stax event stream, binary streams, etc.)
that are linked together in a pipeline.
A simple scenario could be:
Pipeline API + java.net.URL + XML-SAX components
A more advanced scenario could consist of
Pipeline API + Sourceresolve + XML-SAX components + Sitemap Engine
or maybe you need the full stack that corresponds to Cocoon Core 2.2 - here you are:
Pipeline API + Sourceresolve + HTTP-enabled + Sitemap Engine + Spring
XML-SAX
componnents
This layered approach makes Cocoon easily embeddable in any Java application and
Cocoon's learning curve becomes more gradual.
Is such a situation only appealing to Carsten, Steven and me?
--
Reinhard Pötz Managing Director, {Indoqa} GmbH
http://www.indoqa.com/en/people/reinhard.poetz/
Member of the Apache Software Foundation
Apache Cocoon Committer, PMC member, PMC Chair reinhard@apache.org
_________________________________________________________________________
Re: Layered software designs
Posted by Carsten Ziegeler <cz...@apache.org>.
David Crossley wrote:
> Reinhard Poetz wrote:
>> David Crossley wrote:
>>> Reinhard Poetz wrote:
>>>> A simple scenario could be:
>>>>
>>>> Pipeline API + java.net.URL + XML-SAX components
>>>>
>>>> A more advanced scenario could consist of
>>>>
>>>> Pipeline API + Sourceresolve + XML-SAX components + Sitemap Engine
>>> Is "sourceresolve" where the Apache XML Commons Resolver
>>> is hooked up?
>> no, I was talking about the Excalibur Sourceresolve component.
>
> Yes, i know. However that might involve the Catalog Entity Resolver too.
> This configuration used to be done in Cocoon, then we
> moved it to Excalibur so that it would be more widely used.
And now we have a better version back in Cocoon for 2.2 :)
Carsten
--
Carsten Ziegeler
cziegeler@apache.org
Re: Layered software designs
Posted by David Crossley <cr...@apache.org>.
Reinhard Poetz wrote:
> David Crossley wrote:
> >Reinhard Poetz wrote:
> >>A simple scenario could be:
> >>
> >> Pipeline API + java.net.URL + XML-SAX components
> >>
> >>A more advanced scenario could consist of
> >>
> >> Pipeline API + Sourceresolve + XML-SAX components + Sitemap Engine
> >
> >Is "sourceresolve" where the Apache XML Commons Resolver
> >is hooked up?
>
> no, I was talking about the Excalibur Sourceresolve component.
Yes, i know. However that might involve the Catalog Entity Resolver too.
This configuration used to be done in Cocoon, then we
moved it to Excalibur so that it would be more widely used.
> >I would be concerned if our base offering enabled
> >mis-use of net resources, e.g. processing an xml file
> >which declares a DTD, causes an extra network trip if
> >we don't have the xml entity resolver.
>
> Resolving XML entities is important and we will definitly offer solutions
> for it in the future too.
>
> Corona, Steven's and my proposal of a Cocoon reimplementation, is about
> doing things differently so that Cocoon becomes easily embeddable and
> reuseable from within as many environments as possible. We think that a
> "layered design" is the key to achive this goal. When you put all layers
> together, the result (= a web application framework) should be nearly[1] as
> powerful as that what we have today.
>
> For that purpose Steven and I have also started to reimplement existing
> concepts instead of doing everything from the scratch. First, we believe
> that many of the existing concepts are good as they are and second, this
> makes it easier for others to chime in because if you see Corona as a black
> box, it should (more or less) provide the same results as 2.x.
>
> HTH
Yes it does.
-David
> [1] The only exceptions are things that we want to remove, e.g. sub
> sitemaps etc. -> see the Micro-Cocoon discussion:
> http://marc.info/?l=xml-cocoon-dev&m=119903256406947&w=2
>
> --
> Reinhard P?tz Managing Director, {Indoqa} GmbH
> http://www.indoqa.com/en/people/reinhard.poetz/
>
> Member of the Apache Software Foundation
> Apache Cocoon Committer, PMC member, PMC Chair reinhard@apache.org
> _________________________________________________________________________
Re: Layered software designs
Posted by Reinhard Poetz <re...@apache.org>.
David Crossley wrote:
> Reinhard Poetz wrote:
>> A simple scenario could be:
>>
>> Pipeline API + java.net.URL + XML-SAX components
>>
>> A more advanced scenario could consist of
>>
>> Pipeline API + Sourceresolve + XML-SAX components + Sitemap Engine
>
> Is "sourceresolve" where the Apache XML Commons Resolver
> is hooked up?
no, I was talking about the Excalibur Sourceresolve component.
> I would be concerned if our base offering enabled
> mis-use of net resources, e.g. processing an xml file
> which declares a DTD, causes an extra network trip if
> we don't have the xml entity resolver.
Resolving XML entities is important and we will definitly offer solutions for it
in the future too.
Corona, Steven's and my proposal of a Cocoon reimplementation, is about doing
things differently so that Cocoon becomes easily embeddable and reuseable from
within as many environments as possible. We think that a "layered design" is the
key to achive this goal. When you put all layers together, the result (= a web
application framework) should be nearly[1] as powerful as that what we have today.
For that purpose Steven and I have also started to reimplement existing concepts
instead of doing everything from the scratch. First, we believe that many of the
existing concepts are good as they are and second, this makes it easier for
others to chime in because if you see Corona as a black box, it should (more or
less) provide the same results as 2.x.
HTH
[1] The only exceptions are things that we want to remove, e.g. sub sitemaps
etc. -> see the Micro-Cocoon discussion:
http://marc.info/?l=xml-cocoon-dev&m=119903256406947&w=2
--
Reinhard Pötz Managing Director, {Indoqa} GmbH
http://www.indoqa.com/en/people/reinhard.poetz/
Member of the Apache Software Foundation
Apache Cocoon Committer, PMC member, PMC Chair reinhard@apache.org
_________________________________________________________________________
Re: Layered software designs
Posted by David Crossley <cr...@apache.org>.
Reinhard Poetz wrote:
>
> A simple scenario could be:
>
> Pipeline API + java.net.URL + XML-SAX components
>
> A more advanced scenario could consist of
>
> Pipeline API + Sourceresolve + XML-SAX components + Sitemap Engine
Is "sourceresolve" where the Apache XML Commons Resolver
is hooked up?
I would be concerned if our base offering enabled
mis-use of net resources, e.g. processing an xml file
which declares a DTD, causes an extra network trip if
we don't have the xml entity resolver.
-David
Re: Layered software designs
Posted by Reinhard Poetz <re...@apache.org>.
Rainer Pruy wrote:
>
> Reinhard Poetz schrieb:
>> Dev at weitling wrote:
>>> But (maybe I have missed some mails) how do you want to make this
>>> Pipeline API? E.g. a SAX-based pipeline is something different than image
>>> data running through several filters. How do you want to prevent the use
>>> of a SAX-events generating component together with an image data
>>> transformer? What about something like it's used in clipboards: each
>>> component offers a list of importable and exportable formats?
>> See http://marc.info/?l=xml-cocoon-dev&m=120611990603725&w=2 Do the
>> explanations that this mail answer your question?
>>
>
> They answer a number of questions.
>
> Nevertheless, the question of connection "type" is still open.
>
> Ok, the answer given is "it is up to the components to ensure compatibility".
>
>
> However, what does this mean practically? Will there be different
> "Interfaces" for different type of data to e processed. (SAX events will use
> SAX interfaces, image data will use others)
yes, that's the idea (and IIRC it is already implemented this way Corona).
--
Reinhard Pötz Managing Director, {Indoqa} GmbH
http://www.indoqa.com/en/people/reinhard.poetz/
Member of the Apache Software Foundation
Apache Cocoon Committer, PMC member, PMC Chair reinhard@apache.org
_________________________________________________________________________
Re: Layered software designs
Posted by Rainer Pruy <Ra...@Acrys.COM>.
Reinhard Poetz schrieb:
> Dev at weitling wrote:
>> But (maybe I have missed some mails) how do you want to make this
>> Pipeline API?
>> E.g. a SAX-based pipeline is something different than image data
>> running through several filters. How do you want to prevent the use of
>> a SAX-events generating component together with an image data
>> transformer? What about something like it's used in clipboards: each
>> component offers a list of importable and exportable formats?
>
> See http://marc.info/?l=xml-cocoon-dev&m=120611990603725&w=2
> Do the explanations that this mail answer your question?
>
They answer a number of questions.
Nevertheless, the question of connection "type" is still open.
Ok, the answer given is "it is up to the components to ensure compatibility".
However, what does this mean practically?
Will there be different "Interfaces" for different type of data to e processed.
(SAX events will use SAX interfaces, image data will use others)
Or will there be some "tagging" where a component be asked for a list of "types" it will be able to process?
(An image "resizer" will state "PNG JPEG and GIF" as being acceptable inputs for consumer side and "PNG" the type being produced, or
something along?) And compatibility will be assumed if a linked pair of producer and consumer share a common "tag" string?
Thus answering the "type" question might effect back to the API definitions...
Rainer
Re: Layered software designs
Posted by Dev at weitling <de...@weitling.net>.
Reinhard Poetz wrote:
> Dev at weitling wrote:
>> But (maybe I have missed some mails) how do you want to make this
>> Pipeline API?
>> E.g. a SAX-based pipeline is something different than image data
>> running through several filters. How do you want to prevent the use
>> of a SAX-events generating component together with an image data
>> transformer? What about something like it's used in clipboards: each
>> component offers a list of importable and exportable formats?
>
> See http://marc.info/?l=xml-cocoon-dev&m=120611990603725&w=2
> Do the explanations that this mail answer your question?
Quote from the text:
"We hope that this gives you enough hints in order to understand Corona.
We are
very interested in further discussions. So fire at will!"
My Answer: Great work!
And yes, it answered my questions :-)
Re: Layered software designs
Posted by Reinhard Poetz <re...@apache.org>.
Dev at weitling wrote:
> But (maybe I have missed some mails) how do you want to make this
> Pipeline API?
> E.g. a SAX-based pipeline is something different than image data running
> through several filters. How do you want to prevent the use of a
> SAX-events generating component together with an image data transformer?
> What about something like it's used in clipboards: each component offers
> a list of importable and exportable formats?
See http://marc.info/?l=xml-cocoon-dev&m=120611990603725&w=2
Do the explanations that this mail answer your question?
--
Reinhard Pötz Managing Director, {Indoqa} GmbH
http://www.indoqa.com/en/people/reinhard.poetz/
Member of the Apache Software Foundation
Apache Cocoon Committer, PMC member, PMC Chair reinhard@apache.org
_________________________________________________________________________
Re: Layered software designs
Posted by Dev at weitling <de...@weitling.net>.
Reinhard Poetz wrote:
> Carsten Ziegeler wrote:
>> The question is now if we need support for caching in the low level
>> apis or if it is possible to have a layered approach - which would
>> make the entry barrier much easier.
>
> Yes, this layered approach is what I'm aiming for. All the reactions
> in this thread make me think that everybody, who has commented on this
> mailing list so far (except Carsten), believes that we want to throw
> away good things that have profed to be useful in many situations.
> Rest assured, that's not the case. Carsten and I only want to break up
> this all-or-nothing situation that we (still) have now.
>
> What I want to see is a concise pipeline API that comes with only
> little overhead in terms of its learning curve and its dependencies on
> third-party software. Usually this means that we stick with standard
> APIs as much as possible - and I think this rule applies for our
> situation too.
>
> This means that the user of the API only needs to learn as little as
> possible. When he wants more, we offer additional modules that help
> him. Since he has a concrete need, the motiviation to learn something
> new is much higher than when he has to learn everything right from the
> beginning.
>
> If you want to learn how this whole concept *might* apply for a next
> generation Cocoon, have a look at Steven's and my "Exploring Corona"
> mail from last week
> (http://marc.info/?l=xml-cocoon-dev&m=120611990603725&w=2).
>
> The idea of Corona is having a concise core that doesn't have any
> dependencies on a particular component container (Spring, OSGi, etc.),
> source resolving mechanisms or environment (http, java only, etc.) or
> even the type of the components (XML-SAX event stream, XML-Stax event
> stream, binary streams, etc.) that are linked together in a pipeline.
>
> A simple scenario could be:
>
> Pipeline API + java.net.URL + XML-SAX components
>
>
> A more advanced scenario could consist of
>
> Pipeline API + Sourceresolve + XML-SAX components + Sitemap
> Engine
>
>
> or maybe you need the full stack that corresponds to Cocoon Core 2.2 -
> here you are:
>
> Pipeline API + Sourceresolve + HTTP-enabled + Sitemap Engine
> + Spring
> XML-SAX
> componnents
>
>
> This layered approach makes Cocoon easily embeddable in any Java
> application and Cocoon's learning curve becomes more gradual.
>
> Is such a situation only appealing to Carsten, Steven and me?
Your mail makes things a little bit more clear.
But (maybe I have missed some mails) how do you want to make this
Pipeline API?
E.g. a SAX-based pipeline is something different than image data running
through several filters. How do you want to prevent the use of a
SAX-events generating component together with an image data transformer?
What about something like it's used in clipboards: each component offers
a list of importable and exportable formats?
Just my 2 Räppli :-)
Florian
Re: Layered software designs
Posted by Reinhard Poetz <re...@apache.org>.
Steven Dolg wrote:
>
>
> Joerg Heinicke schrieb:
>> On 26.03.2008 09:14, Reinhard Poetz wrote:
>>
>>> What I want to see is a concise pipeline API that comes with only
>>> little overhead in terms of its learning curve and its dependencies
>>> on third-party software. Usually this means that we stick with
>>> standard APIs as much as possible - and I think this rule applies for
>>> our situation too.
>>
>> See, one thing that I just don't get (and already asked) is how the
>> pipeline API has anything to do with uri resolving. For me the latter
>> (using java.net or source resolve) is an implementation detail. Our
>> current pipeline interface [1] has no relationship to uri resolving.
>> It only has a reference to SourceValidity because of caching [2].
>>
>> If all this discussion is about removing this method (and the related
>> getKeyForEventPipeline()) to get rid of this dependency I'm ok with
>> it. The caching concern could be implemented in a separate Cacheable
>> interface which should then also be decoupled from uri resolving
>> (which seems to point to Peter's approach [3]).
>>
>> Joerg
>>
>> [1]
>> http://cocoon.apache.org/2.1/apidocs/org/apache/cocoon/components/pipeline/ProcessingPipeline.html
>>
>> [2]
>> http://cocoon.apache.org/2.1/apidocs/org/apache/cocoon/components/pipeline/ProcessingPipeline.html#getValidityForEventPipeline()
>>
>> [3] http://marc.info/?l=xml-cocoon-dev&m=120654005017297&w=4
>>
> Just as a general observation:
> I'm starting to believe that I do not understand (anymore) what this is
> all about.
> We're jumping from high-level concepts ("caching", "layering") straight
> down to the lowest level ("it's just method a in class B") and then
> right back.
> We're arguing that a certain feature is already existing and working,
> while talking about a rewrite-experiment that definitely does not have
> this feature.
>
>
> But back to caching:
> Caching appears to be incredibly important. Even to the point where "no
> caching" means "not acceptable".
> On the other hand, when trying to find out what is really necessary and
> wanted, there isn't much left. Suddenly it's "just an implementation
> detail", "not really important", "makes no difference for the user".
>
> Forgive me for being blunt, but all this appears to me like "I need that
> feature. I do not care how it is implemented or how it works. I just
> have to have it."
> I did not expect this kind of discussion on a dev list. (I even have a
> hard time accepting this from a paying customer).
>
> But since it's just a not very important implementation detail, I added
> a (simple) caching approach to Corona. (I hope to get a patch ready today)
> Perhaps this is all just too abstract and far fetched without any common
> basis (iow. code).
Steven,
one of the greatest strengths of community-driven opensource development (I
don't talk about projects like Spring, Eclipse or many other company-driven
projects but about those projects here at the Apache Software Foundation) is the
diversity of the people who participate. At the same time it is also one of its
weaknesses because things need time to grow.
In addition, we people here have different development and language skills and,
don't forget that we both have already had many discussions about what a
reimplementation is about and how we intend to implement this layered design.
I'm happy that people are catching up and participate in the discussion on
different levels. I'm sure that we will become more focused in the future - and
I think that you are right that code will help. At least people will finally
come up with their requirements in more detail ("I need caching" is too
general!) finally so that terms like "layered design" and "caching" become more
conrete.
--
Reinhard Pötz Managing Director, {Indoqa} GmbH
http://www.indoqa.com/en/people/reinhard.poetz/
Member of the Apache Software Foundation
Apache Cocoon Committer, PMC member, PMC Chair reinhard@apache.org
_________________________________________________________________________
Re: Layered software designs
Posted by Reinhard Poetz <re...@apache.org>.
Rainer Pruy wrote:
> Hi, I was off the net for some time and while catching up this discussion I
> also got the feeling of being somehow lost a bit in the different aspects of
> the discussions....
>
>> From what I see the starting point was
> the (technical) question of how to get rid of a unwanted dependency.
>> From that it changed to the conceptual question of how to provide flexible
>> support for arbitrary URI strings.
> (out-of-the-box support for "standard" protocols and an easy extensibility
> for more complex needs)
>
> If I did get it right, the main difference between the two approaches in
> discussion (URL vs. SourceResolver) is support for caching.
>
> It was noted (and I fully agree) that adding caching later on will require
> support from the lower level. (e.g. caching a file will need info about
> modification times. caching a resource accessed via http might need access to
> the expires header for example. This is not a place for talking about cache
> *implementations* or what kind of caching is suited for what layer, it is
> just a question of what information will any kind of caching require from
> lower levels and will the intended implementation provide such info).
>
> The standard URL implementation does not provide related methods (for
> accessing cache control data). On the other hand URL has the benefit of being
> plain standard and familiar to a large community of developers.
>
> This to answer the initial question of "can we drop SourceResolver support",
> we must answer the question "Can URL be extended to support "cachable"
> implementations of protocols?" and if true "Is it possible to override
> standard (non-cachable) implementations of protocols with cachable ones?"
>
> If both question can be answered with YES, then it will be possible to
> implement the uri interpreting layer using plain Java URL and to drop use of
> source resolver.
Thanks for sharing your thoughts. I think we have to get our hands dirty now to
figure out all the details.
> This will raise questions of semantic differences remaining. E.g. was the
> default protocol implementation used for "path" values without protocol
> different with different locations (e.g. per component implementation?)
> Probably, it will then be necessary to add some glue code to keep semantics,
> or point out differences for migration guidelines.
The question is also, *where* we will introduce this change (if we do it at
all). Corona doesn't impose any constraints in terms of backwards-compatibility,
whereas we can't drop important contracts in 2.2.
--
Reinhard Pötz Managing Director, {Indoqa} GmbH
http://www.indoqa.com/en/people/reinhard.poetz/
Member of the Apache Software Foundation
Apache Cocoon Committer, PMC member, PMC Chair reinhard@apache.org
_________________________________________________________________________
Re: Layered software designs
Posted by Rainer Pruy <Ra...@Acrys.COM>.
Hi,
I was off the net for some time
and while catching up this discussion I also got the feeling of being somehow lost a bit in the different aspects of the discussions....
>From what I see the starting point was
the (technical) question of how to get rid of a unwanted dependency.
>From that it changed to the conceptual question of how to provide flexible support for arbitrary URI strings.
(out-of-the-box support for "standard" protocols and an easy extensibility for more complex needs)
If I did get it right, the main difference between the two approaches in discussion (URL vs. SourceResolver)
is support for caching.
It was noted (and I fully agree) that adding caching later on will require support from the lower level.
(e.g. caching a file will need info about modification times. caching a resource accessed via http might need access to the expires
header for example. This is not a place for talking about cache *implementations* or what kind of caching is suited for what layer, it
is just a question of what information will any kind of caching require from lower levels and will the intended implementation provide
such info).
The standard URL implementation does not provide related methods (for accessing cache control data).
On the other hand URL has the benefit of being plain standard and familiar to a large community of developers.
This to answer the initial question of "can we drop SourceResolver support",
we must answer the question
"Can URL be extended to support "cachable" implementations of protocols?"
and if true "Is it possible to override standard (non-cachable) implementations of protocols with cachable ones?"
If both question can be answered with YES, then it will be possible to implement the uri interpreting layer using plain Java URL
and to drop use of source resolver.
This will raise questions of semantic differences remaining.
E.g. was the default protocol implementation used for "path" values without protocol different with different locations (e.g. per
component implementation?) Probably, it will then be necessary to add some glue code to keep semantics, or point out differences for
migration guidelines.
Rainer
Joerg Heinicke schrieb:
> On 27.03.2008 02:25, Steven Dolg wrote:
>
>>>> What I want to see is a concise pipeline API that comes with only
>>>> little overhead in terms of its learning curve and its dependencies
>>>> on third-party software. Usually this means that we stick with
>>>> standard APIs as much as possible - and I think this rule applies
>>>> for our situation too.
>>>
>>> See, one thing that I just don't get (and already asked) is how the
>>> pipeline API has anything to do with uri resolving. For me the latter
>>> (using java.net or source resolve) is an implementation detail. Our
>>> current pipeline interface [1] has no relationship to uri resolving.
>>> It only has a reference to SourceValidity because of caching [2].
>>>
>>> If all this discussion is about removing this method (and the related
>>> getKeyForEventPipeline()) to get rid of this dependency I'm ok with
>>> it. The caching concern could be implemented in a separate Cacheable
>>> interface which should then also be decoupled from uri resolving
>>> (which seems to point to Peter's approach [3]).
>>>
>> Just as a general observation:
>> I'm starting to believe that I do not understand (anymore) what this
>> is all about.
>> We're jumping from high-level concepts ("caching", "layering")
>> straight down to the lowest level ("it's just method a in class B")
>> and then right back.
>
> I have had the same problem from the beginning [1] and expressed it in
> my question above again. What is this discussion about - if uri
> resolving has nothing to do with the pipeline API? And what do we win
> when replacing source resolve with java.net except that we have one less
> dependency - when nobody really gets in contact with uri resolving
> anyway? I have only received half an answer to only the second question
> (standard API). Only then I started to look at the relationship between
> uri resolving and pipeline API and found only this one reference to the
> source resolve package.
>
> Joerg
>
> [1] http://marc.info/?l=xml-cocoon-dev&m=120649777119480&w=4
--
Rainer Pruy
Geschäftsführer
Acrys Consult GmbH & Co. KG
Untermainkai 29-30, D-60329 Frankfurt
Tel: +49-69-244506-0 - Fax: +49-69-244506-50
Web: http://www.acrys.com - Email: office@acrys.com
Handelsregister: Frankfurt am Main, HRA 31151
Re: Layered software designs
Posted by Joerg Heinicke <jo...@gmx.de>.
On 27.03.2008 02:25, Steven Dolg wrote:
>>> What I want to see is a concise pipeline API that comes with only
>>> little overhead in terms of its learning curve and its dependencies
>>> on third-party software. Usually this means that we stick with
>>> standard APIs as much as possible - and I think this rule applies for
>>> our situation too.
>>
>> See, one thing that I just don't get (and already asked) is how the
>> pipeline API has anything to do with uri resolving. For me the latter
>> (using java.net or source resolve) is an implementation detail. Our
>> current pipeline interface [1] has no relationship to uri resolving.
>> It only has a reference to SourceValidity because of caching [2].
>>
>> If all this discussion is about removing this method (and the related
>> getKeyForEventPipeline()) to get rid of this dependency I'm ok with
>> it. The caching concern could be implemented in a separate Cacheable
>> interface which should then also be decoupled from uri resolving
>> (which seems to point to Peter's approach [3]).
>>
> Just as a general observation:
> I'm starting to believe that I do not understand (anymore) what this is
> all about.
> We're jumping from high-level concepts ("caching", "layering") straight
> down to the lowest level ("it's just method a in class B") and then
> right back.
I have had the same problem from the beginning [1] and expressed it in
my question above again. What is this discussion about - if uri
resolving has nothing to do with the pipeline API? And what do we win
when replacing source resolve with java.net except that we have one less
dependency - when nobody really gets in contact with uri resolving
anyway? I have only received half an answer to only the second question
(standard API). Only then I started to look at the relationship between
uri resolving and pipeline API and found only this one reference to the
source resolve package.
Joerg
[1] http://marc.info/?l=xml-cocoon-dev&m=120649777119480&w=4
Re: Layered software designs
Posted by Steven Dolg <st...@gmx.at>.
Joerg Heinicke schrieb:
> On 26.03.2008 09:14, Reinhard Poetz wrote:
>
>> What I want to see is a concise pipeline API that comes with only
>> little overhead in terms of its learning curve and its dependencies
>> on third-party software. Usually this means that we stick with
>> standard APIs as much as possible - and I think this rule applies for
>> our situation too.
>
> See, one thing that I just don't get (and already asked) is how the
> pipeline API has anything to do with uri resolving. For me the latter
> (using java.net or source resolve) is an implementation detail. Our
> current pipeline interface [1] has no relationship to uri resolving.
> It only has a reference to SourceValidity because of caching [2].
>
> If all this discussion is about removing this method (and the related
> getKeyForEventPipeline()) to get rid of this dependency I'm ok with
> it. The caching concern could be implemented in a separate Cacheable
> interface which should then also be decoupled from uri resolving
> (which seems to point to Peter's approach [3]).
>
> Joerg
>
> [1]
> http://cocoon.apache.org/2.1/apidocs/org/apache/cocoon/components/pipeline/ProcessingPipeline.html
>
> [2]
> http://cocoon.apache.org/2.1/apidocs/org/apache/cocoon/components/pipeline/ProcessingPipeline.html#getValidityForEventPipeline()
>
> [3] http://marc.info/?l=xml-cocoon-dev&m=120654005017297&w=4
>
Just as a general observation:
I'm starting to believe that I do not understand (anymore) what this is
all about.
We're jumping from high-level concepts ("caching", "layering") straight
down to the lowest level ("it's just method a in class B") and then
right back.
We're arguing that a certain feature is already existing and working,
while talking about a rewrite-experiment that definitely does not have
this feature.
But back to caching:
Caching appears to be incredibly important. Even to the point where "no
caching" means "not acceptable".
On the other hand, when trying to find out what is really necessary and
wanted, there isn't much left. Suddenly it's "just an implementation
detail", "not really important", "makes no difference for the user".
Forgive me for being blunt, but all this appears to me like "I need that
feature. I do not care how it is implemented or how it works. I just
have to have it."
I did not expect this kind of discussion on a dev list. (I even have a
hard time accepting this from a paying customer).
But since it's just a not very important implementation detail, I added
a (simple) caching approach to Corona. (I hope to get a patch ready today)
Perhaps this is all just too abstract and far fetched without any common
basis (iow. code).
Steven
Re: Layered software designs
Posted by Joerg Heinicke <jo...@gmx.de>.
On 26.03.2008 09:14, Reinhard Poetz wrote:
> What I want to see is a concise pipeline API that comes with only little
> overhead in terms of its learning curve and its dependencies on
> third-party software. Usually this means that we stick with standard
> APIs as much as possible - and I think this rule applies for our
> situation too.
See, one thing that I just don't get (and already asked) is how the
pipeline API has anything to do with uri resolving. For me the latter
(using java.net or source resolve) is an implementation detail. Our
current pipeline interface [1] has no relationship to uri resolving. It
only has a reference to SourceValidity because of caching [2].
If all this discussion is about removing this method (and the related
getKeyForEventPipeline()) to get rid of this dependency I'm ok with it.
The caching concern could be implemented in a separate Cacheable
interface which should then also be decoupled from uri resolving (which
seems to point to Peter's approach [3]).
Joerg
[1]
http://cocoon.apache.org/2.1/apidocs/org/apache/cocoon/components/pipeline/ProcessingPipeline.html
[2]
http://cocoon.apache.org/2.1/apidocs/org/apache/cocoon/components/pipeline/ProcessingPipeline.html#getValidityForEventPipeline()
[3] http://marc.info/?l=xml-cocoon-dev&m=120654005017297&w=4
Re: Layered software designs
Posted by Reinhard Poetz <re...@apache.org>.
Grzegorz Kossakowski wrote:
> Ralph Goers pisze:
>>
>> Reinhard Poetz wrote:
>>> Pipeline API + java.net.URL + XML-SAX components
>>>
>>>
>>> A more advanced scenario could consist of
>>>
>>> Pipeline API + Sourceresolve + XML-SAX components + Sitemap Engine
>>>
>>>
>>> or maybe you need the full stack that corresponds to Cocoon Core 2.2 -
>>> here you are:
>>>
>>> Pipeline API + Sourceresolve + HTTP-enabled + Sitemap Engine +
>>> Spring XML-SAX componnents
>>>
>>>
>>> This layered approach makes Cocoon easily embeddable in any Java
>>> application and Cocoon's learning curve becomes more gradual.
>>>
>>> Is such a situation only appealing to Carsten, Steven and me?
>> Appealing? yes. Actually implementable in Java so that it isn;t even more
>> complicated than what we have? I don't know.
>
> +1.
>
> I've got my hands dirty with URL stuff from Java and I fail to see how we can
> avoid using Source interface, especially when caching comes in.
>
> The issue is not about lack of the will but more about lack of ability, at
> least at this stage. Examine current
> Source/SourcaFactory/CocoonSourceResolver usage patterns to see what I mean.
Why do you want to give up even before you start? Because things _seem_ to be
difficult?
At the current stage we haven't even defined what we expect from caching. Please
let's first put all our requirements on the table.
My (and that of many others) *main requirement is a layered software design*
that makes Cocoon easily useable and embeddable in many different environments.
Given, caching is important, but compared to that it is of subordinate relevance.
Maybe we have to go back to the old Source/SourceResolver solution because
everything is impossible as some of us think, but let's decide this latter when
we are sure that there is no better alternative.
--
Reinhard Pötz Managing Director, {Indoqa} GmbH
http://www.indoqa.com/en/people/reinhard.poetz/
Member of the Apache Software Foundation
Apache Cocoon Committer, PMC member, PMC Chair reinhard@apache.org
_________________________________________________________________________
Re: Layered software designs
Posted by Grzegorz Kossakowski <gr...@tuffmail.com>.
Ralph Goers pisze:
>
>
> Reinhard Poetz wrote:
>>
>> Pipeline API + java.net.URL + XML-SAX components
>>
>>
>> A more advanced scenario could consist of
>>
>> Pipeline API + Sourceresolve + XML-SAX components + Sitemap
>> Engine
>>
>>
>> or maybe you need the full stack that corresponds to Cocoon Core 2.2 -
>> here you are:
>>
>> Pipeline API + Sourceresolve + HTTP-enabled + Sitemap Engine
>> + Spring
>> XML-SAX
>> componnents
>>
>>
>> This layered approach makes Cocoon easily embeddable in any Java
>> application and Cocoon's learning curve becomes more gradual.
>>
>> Is such a situation only appealing to Carsten, Steven and me?
> Appealing? yes. Actually implementable in Java so that it isn;t even
> more complicated than what we have? I don't know.
+1.
I've got my hands dirty with URL stuff from Java and I fail to see how we can avoid using Source
interface, especially when caching comes in.
The issue is not about lack of the will but more about lack of ability, at least at this stage.
Examine current Source/SourcaFactory/CocoonSourceResolver usage patterns to see what I mean.
--
Grzegorz
Re: Layered software designs
Posted by Reinhard Poetz <re...@apache.org>.
Yes, using Cocoon pipelines in Ant is one of my long-time favorits ;-)
Reinhard
Bruce Atherton wrote:
> I'm with you, too.
>
> Just as an example, I think it might be useful to use Corona as the
> basis for a new Ant task called Pipeline. That task could do any number
> text transformations to a set of files as part of a build process. Here,
> caching is a non-issue for the most part since the point of using a
> pipeline would be to process each file only once. Rather than either
> java.net.URL or SourceResolve, we'd probably want to feed the pipeline
> based on Ant resources[1], just as the existing XSLT task is fed.
>
> It may be that SourceResolver and an AntResourceSource is the best way
> to solve the problem, but on a cursory glance it sure looks like it is
> difficult to separate the banana from the gorilla[2] that is Avalon.
>
> [1] http://ant.apache.org/manual/CoreTypes/resources.html
> [2] http://www.ddj.com/architect/184408251
>
> Torsten Curdt wrote:
>> On Mar 26, 2008, at 14:44, Ralph Goers wrote:
>>> Reinhard Poetz wrote:
>>>>
>>>> Pipeline API + java.net.URL + XML-SAX components
>>>>
>>>>
>>>> A more advanced scenario could consist of
>>>>
>>>> Pipeline API + Sourceresolve + XML-SAX components + Sitemap
>>>> Engine
>>>>
>>>>
>>>> or maybe you need the full stack that corresponds to Cocoon Core 2.2
>>>> - here you are:
>>>>
>>>> Pipeline API + Sourceresolve + HTTP-enabled + Sitemap Engine
>>>> + Spring
>>>> XML-SAX
>>>> componnents
>>>>
>>>>
>>>> This layered approach makes Cocoon easily embeddable in any Java
>>>> application and Cocoon's learning curve becomes more gradual.
>>>>
>>>> Is such a situation only appealing to Carsten, Steven and me?
>>
>> Just lurking but I am with you guys.
>>
>>>>
>>> Appealing? yes. Actually implementable in Java so that it isn;t even
>>> more complicated than what we have? I don't know.
>>
>>
>> IMO this would simplify a lot as it separates concerns and the inner
>> guts can be used in other projects without the pain of dependencies we
>> have right now. People have been asking for this for years. I really
>> think think this is the right direction.
--
Reinhard Pötz Managing Director, {Indoqa} GmbH
http://www.indoqa.com/en/people/reinhard.poetz/
Member of the Apache Software Foundation
Apache Cocoon Committer, PMC member, PMC Chair reinhard@apache.org
_________________________________________________________________________
Re: Layered software designs
Posted by Bruce Atherton <br...@callenish.com>.
I'm with you, too.
Just as an example, I think it might be useful to use Corona as the
basis for a new Ant task called Pipeline. That task could do any number
text transformations to a set of files as part of a build process. Here,
caching is a non-issue for the most part since the point of using a
pipeline would be to process each file only once. Rather than either
java.net.URL or SourceResolve, we'd probably want to feed the pipeline
based on Ant resources[1], just as the existing XSLT task is fed.
It may be that SourceResolver and an AntResourceSource is the best way
to solve the problem, but on a cursory glance it sure looks like it is
difficult to separate the banana from the gorilla[2] that is Avalon.
[1] http://ant.apache.org/manual/CoreTypes/resources.html
[2] http://www.ddj.com/architect/184408251
Torsten Curdt wrote:
> On Mar 26, 2008, at 14:44, Ralph Goers wrote:
>> Reinhard Poetz wrote:
>>>
>>> Pipeline API + java.net.URL + XML-SAX components
>>>
>>>
>>> A more advanced scenario could consist of
>>>
>>> Pipeline API + Sourceresolve + XML-SAX components + Sitemap
>>> Engine
>>>
>>>
>>> or maybe you need the full stack that corresponds to Cocoon Core 2.2
>>> - here you are:
>>>
>>> Pipeline API + Sourceresolve + HTTP-enabled + Sitemap Engine
>>> + Spring
>>> XML-SAX
>>> componnents
>>>
>>>
>>> This layered approach makes Cocoon easily embeddable in any Java
>>> application and Cocoon's learning curve becomes more gradual.
>>>
>>> Is such a situation only appealing to Carsten, Steven and me?
>
> Just lurking but I am with you guys.
>
>>>
>> Appealing? yes. Actually implementable in Java so that it isn;t even
>> more complicated than what we have? I don't know.
>
>
> IMO this would simplify a lot as it separates concerns and the inner
> guts can be used in other projects without the pain of dependencies we
> have right now. People have been asking for this for years. I really
> think think this is the right direction.
>
> cheers
> --
> Torsten
Re: Layered software designs
Posted by Torsten Curdt <tc...@apache.org>.
On Mar 26, 2008, at 14:44, Ralph Goers wrote:
> Reinhard Poetz wrote:
>>
>> Pipeline API + java.net.URL + XML-SAX components
>>
>>
>> A more advanced scenario could consist of
>>
>> Pipeline API + Sourceresolve + XML-SAX components + Sitemap
>> Engine
>>
>>
>> or maybe you need the full stack that corresponds to Cocoon Core
>> 2.2 - here you are:
>>
>> Pipeline API + Sourceresolve + HTTP-enabled + Sitemap
>> Engine + Spring
>> XML-SAX
>> componnents
>>
>>
>> This layered approach makes Cocoon easily embeddable in any Java
>> application and Cocoon's learning curve becomes more gradual.
>>
>> Is such a situation only appealing to Carsten, Steven and me?
Just lurking but I am with you guys.
>>
> Appealing? yes. Actually implementable in Java so that it isn;t
> even more complicated than what we have? I don't know.
IMO this would simplify a lot as it separates concerns and the inner
guts can be used in other projects without the pain of dependencies we
have right now. People have been asking for this for years. I really
think think this is the right direction.
cheers
--
Torsten
Re: Layered software designs
Posted by Ralph Goers <Ra...@dslextreme.com>.
Steven Dolg wrote:
>
>
> Ralph Goers schrieb:
>>
>> Appealing? yes. Actually implementable in Java so that it isn;t even
>> more complicated than what we have? I don't know.
>>
>
> Just curious - do you have doubts, that this is achievable
> specifically with Java, or generally with any language?
Taking 5 or 6 concepts (interfaces?) and trying to tie them together can
get quite messy in Java. C++ could use multiple inheritence. While that
would work it also brings along all the problems all the problems
multiple inheritence introduces. I'm sure there are other languages
that might tie these concepts better, but off hand I can't think of one.
I'm also sure there are some interesting ways to do this in Java.
Whether they end up being complicated (either to maintain or use)
remains to be seen.
I'm all for letting whoever wants to move forward on this and see what
they come up with. But I won't be willing to accept it until it supports
caching.
Ralph
Re: Layered software designs
Posted by Steven Dolg <st...@gmx.at>.
Ralph Goers schrieb:
>
>
> Reinhard Poetz wrote:
>>
>> Pipeline API + java.net.URL + XML-SAX components
>>
>>
>> A more advanced scenario could consist of
>>
>> Pipeline API + Sourceresolve + XML-SAX components + Sitemap
>> Engine
>>
>>
>> or maybe you need the full stack that corresponds to Cocoon Core 2.2
>> - here you are:
>>
>> Pipeline API + Sourceresolve + HTTP-enabled + Sitemap Engine
>> + Spring
>> XML-SAX
>> componnents
>>
>>
>> This layered approach makes Cocoon easily embeddable in any Java
>> application and Cocoon's learning curve becomes more gradual.
>>
>> Is such a situation only appealing to Carsten, Steven and me?
> Appealing? yes. Actually implementable in Java so that it isn;t even
> more complicated than what we have? I don't know.
>
Just curious - do you have doubts, that this is achievable specifically
with Java, or generally with any language?
Re: Layered software designs
Posted by Ralph Goers <Ra...@dslextreme.com>.
Reinhard Poetz wrote:
>
> Pipeline API + java.net.URL + XML-SAX components
>
>
> A more advanced scenario could consist of
>
> Pipeline API + Sourceresolve + XML-SAX components + Sitemap
> Engine
>
>
> or maybe you need the full stack that corresponds to Cocoon Core 2.2 -
> here you are:
>
> Pipeline API + Sourceresolve + HTTP-enabled + Sitemap Engine
> + Spring
> XML-SAX
> componnents
>
>
> This layered approach makes Cocoon easily embeddable in any Java
> application and Cocoon's learning curve becomes more gradual.
>
> Is such a situation only appealing to Carsten, Steven and me?
Appealing? yes. Actually implementable in Java so that it isn;t even
more complicated than what we have? I don't know.
Re: Layered software designs
Posted by Reinhard Poetz <re...@apache.org>.
Rainer Pruy wrote:
> Reinhard Poetz schrieb:
>> The idea of Corona is having a concise core that doesn't have any
>> dependencies on a particular component container (Spring, OSGi, etc.),
>> source resolving mechanisms or environment (http, java only, etc.) or even
>> the type of the components (XML-SAX event stream, XML-Stax event stream,
>> binary streams, etc.) that are linked together in a pipeline.
>>
>> A simple scenario could be:
>>
>> Pipeline API + java.net.URL + XML-SAX components
>>
>>
>> A more advanced scenario could consist of
>>
>> Pipeline API + Sourceresolve + XML-SAX components + Sitemap Engine
>>
>>
>> or maybe you need the full stack that corresponds to Cocoon Core 2.2 - here
>> you are:
>>
>> Pipeline API + Sourceresolve + HTTP-enabled + Sitemap Engine +
>> Spring XML-SAX componnents
>>
>>
>> This layered approach makes Cocoon easily embeddable in any Java
>> application and Cocoon's learning curve becomes more gradual.
>>
>> Is such a situation only appealing to Carsten, Steven and me?
>>
>
> It's appealing to me also.
>
> However, I'm not sure I did get the "layers" correctly.
Note: All my answers are related to Corona. Here we go:
> I did see:
>
> Pipeline API: responsible for composing different components, introduces the
> notion of Producer/Consumer and first and last component
the pipeline api will be enough to execute pipelines too.
> Sitemap API: responsible for executing pipelines
no, the sitemap is a builder that builds pipelines and is put on top of the
pipeline API.
> Spring: responsible for setting up layers and identifying implementations to
> "wildcard" functionality (not really a layer for itself?)
Corona has a SpringComponentProvider which is used by the sitemap to lookup its
components. Spring manages the components but is completly hidden behind the
ComponentProvider interface. This makes it very easy to use alternative
containers (e.g. OSGi).
> URI API (had no better term at hand): responsible for interpreting protocol
> strings for resource access. Currently two implementations in dicussion: URL
> and SourceResolver
yes. And as Sylvain and you pointed out, we maybe don't even need the
Sourceresolver anymore.
> Pipeline Components? really a layer or just implementations of Pipeline API
Steven came up with the idea of having components that belong to different
layers. The first type is useable in every environment. This could be a
SAX-based FileGenerator or a SAX-based XSLTTransformer. There is no need for any
ties to the ServletAPI. Then there might be a second group of components that
are based on e.g. the Stax-API.
A third group of components has relations to the ServletAPI in order to produce
useful results.
In short, the idea is that the pipeline API is completly independant from any
environments. It can "only" connect components that belong together.
> Can we line out the intended or existing "layers"? I do feel this would help
> focussing the discussion.
I hope that my explanations above are already helpful. Steven and I will work on
Corona next week again and we hope that we can show how this "layered design"
can be applied in code.
> We then can have a track reflecting actual layers, a track exploring
> interaction among layers (e.g caching, configuration...), and a track
> pondering implementation aspects.
--
Reinhard Pötz Managing Director, {Indoqa} GmbH
http://www.indoqa.com/en/people/reinhard.poetz/
Member of the Apache Software Foundation
Apache Cocoon Committer, PMC member, PMC Chair reinhard@apache.org
_________________________________________________________________________
Re: Layered software designs
Posted by Rainer Pruy <Ra...@Acrys.COM>.
Reinhard Poetz schrieb:
> Carsten Ziegeler wrote:
>> The question is now if we need support for caching in the low level
>> apis or if it is possible to have a layered approach - which would
>> make the entry barrier much easier.
>
> Yes, this layered approach is what I'm aiming for. All the reactions in
> this thread make me think that everybody, who has commented on this
> mailing list so far (except Carsten), believes that we want to throw
> away good things that have profed to be useful in many situations.
> Rest assured, that's not the case. Carsten and I only want to break up
> this all-or-nothing situation that we (still) have now.
>
> What I want to see is a concise pipeline API that comes with only little
> overhead in terms of its learning curve and its dependencies on
> third-party software. Usually this means that we stick with standard
> APIs as much as possible - and I think this rule applies for our
> situation too.
>
> This means that the user of the API only needs to learn as little as
> possible. When he wants more, we offer additional modules that help him.
> Since he has a concrete need, the motiviation to learn something new is
> much higher than when he has to learn everything right from the beginning.
>
> If you want to learn how this whole concept *might* apply for a next
> generation Cocoon, have a look at Steven's and my "Exploring Corona"
> mail from last week
> (http://marc.info/?l=xml-cocoon-dev&m=120611990603725&w=2).
>
> The idea of Corona is having a concise core that doesn't have any
> dependencies on a particular component container (Spring, OSGi, etc.),
> source resolving mechanisms or environment (http, java only, etc.) or
> even the type of the components (XML-SAX event stream, XML-Stax event
> stream, binary streams, etc.) that are linked together in a pipeline.
>
> A simple scenario could be:
>
> Pipeline API + java.net.URL + XML-SAX components
>
>
> A more advanced scenario could consist of
>
> Pipeline API + Sourceresolve + XML-SAX components + Sitemap Engine
>
>
> or maybe you need the full stack that corresponds to Cocoon Core 2.2 -
> here you are:
>
> Pipeline API + Sourceresolve + HTTP-enabled + Sitemap Engine +
> Spring
> XML-SAX
> componnents
>
>
> This layered approach makes Cocoon easily embeddable in any Java
> application and Cocoon's learning curve becomes more gradual.
>
> Is such a situation only appealing to Carsten, Steven and me?
>
It's appealing to me also.
However, I'm not sure I did get the "layers" correctly.
I did see:
Pipeline API: responsible for composing different components, introduces the notion of Producer/Consumer and first and last component
Sitemap API: responsible for executing pipelines
Spring: responsible for setting up layers and identifying implementations to "wildcard" functionality
(not really a layer for itself?)
URI API (had no better term at hand): responsible for interpreting protocol strings for resource access.
Currently two implementations in dicussion: URL and SourceResolver
Pipeline Components? really a layer or just implementations of Pipeline API
Can we line out the intended or existing "layers"?
I do feel this would help focussing the discussion.
We then can have a track reflecting actual layers, a track exploring interaction among layers (e.g caching, configuration...), and a
track pondering implementation aspects.
Rainer
Re: Layered software designs
Posted by Reinhard Poetz <re...@apache.org>.
Vadim Gritsenko wrote:
> On Mar 26, 2008, at 9:14 AM, Reinhard Poetz wrote:
>> A simple scenario could be:
>>
>> Pipeline API + java.net.URL + XML-SAX components
>>
>>
>> A more advanced scenario could consist of
>>
>> Pipeline API + Sourceresolve + XML-SAX components + Sitemap Engine
>
> Do you really need both URL and SourceResolver? As Sylvain described it,
> with java.net.URL we should be able to implement replacement for all
> features of existing SourceResolver. I just don't think it is worth it,
> if in the end we are going to have a mix of two paradigms.
I had written this before I read Sylvain's mail. Even better, if we don't need
the SourceResolver at all.
--
Reinhard Pötz Managing Director, {Indoqa} GmbH
http://www.indoqa.com/en/people/reinhard.poetz/
Member of the Apache Software Foundation
Apache Cocoon Committer, PMC member, PMC Chair reinhard@apache.org
_________________________________________________________________________
Re: Layered software designs
Posted by Vadim Gritsenko <va...@reverycodes.com>.
On Mar 26, 2008, at 9:14 AM, Reinhard Poetz wrote:
> A simple scenario could be:
>
> Pipeline API + java.net.URL + XML-SAX components
>
>
> A more advanced scenario could consist of
>
> Pipeline API + Sourceresolve + XML-SAX components + Sitemap
> Engine
Do you really need both URL and SourceResolver? As Sylvain described
it, with java.net.URL we should be able to implement replacement for
all features of existing SourceResolver. I just don't think it is
worth it, if in the end we are going to have a mix of two paradigms.
Vadim