You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cocoon.apache.org by Stefano Mazzocchi <st...@apache.org> on 2002/01/11 21:10:43 UTC
Matchers vs. Selectors [was Re: Retuning Sitemap Design]
Judson Lester wrote:
<snip/>
> The clearest dilemma I can see is that <select><match test=A><match
> test=B></select> implies that if A matches, and B matches, two clauses occur,
> but <select><when test=A><when test=B></select> implies that if A matches, B
> won't be tested. Is this FS or useful power?
I think you touched the nerve of the discussion:
- matchers distill the behavior of boolean 'or'
- selectors distill the behavior of boolean 'and'
Berin suggested that URI-matching performed with 'and'-type boolean
logic is instrinsically faster since it is algorithmically less complex.
I think he does have a point here!
<snip/>
> > Moreover, matchers 'match a request' while selectors 'select paths'.
> > They clearly separate concerns! Their functionality and implementations
> > partially overlap because both require 'conditional operations', but one
> > is declarative (matchers) and one is procedural (selectors).
>
> Again, this may be a FS, but I'm dubious about how 'natural' a seperation
> those concerns are. Matchers match anything, and on what other basis do
> selectors 'select paths' that on the request? I'm concerned about pipelines
> becoming unnecesarily convoluted because a single concern has been seperated.
Good concern.
And given the fact that this confrontation between matchers and
selectors seems to appear once and a while, I think we have some design
work to do.
Ok, let's do it.
- o -
The goals of the indirect sitemap components (those who don't generate
data that goes directly inside the SAX stream) are to provide support
for the other direct sitemap components (those who do).
Right at the beginning of the sitemap design, it was clear that the
functionality of 'routing' the request thru the different components was
absolutely required in order to allow Cocoon2's sitemap to match
Cocoon1's reactor in functionality. Otherwise, forking friction would
have been developped.
At the same time, the reactor (which appeared as a very good idea when I
proposed it) is sort of a dynamic-pipeline composer. We knew that
dynamic pipelines provided several limitations:
- where impossible to 'validate' at load-time
- where impossible to 'pre-compile' and optimize
- where harder to pool
- where less cache friendly
and more important:
- where much more painful to debug!
Just like a real pipeline requires 'valves' to route its flow, we
decided to introduce 'support components' that could do the same and
decided to make them 'components' to allow the 'routing logic' to be
pluggable.
One first objection Berin posed is that being web browsers driven by
URIs, 'uri-based routing' is the most used logic and should be
explicitly expressed in the sitemap design, just like we do for
aggregation.
---------------------------------------------------
Question: should URI-based routing logic have a more explict place in
the sitemap?
---------------------------------------------------
I think it's a very good question, even if I disagree on the intentions
to ease URI-space administration for system administrators.
At the same time (and this was the reason *not* to do it since the
beginning!), the fact that URI-based routing is not more important than
any other routing logic suggests that other routing logics can be
implemented and used with the same ease!
Look at Apache: URI-based routing is piece of cake (well, not really,
but easy enough, just add your 'Alias' and you're done) while
Non-URI-based routing is DEADLY HARD! or at the very least, is perceived
as such since mod_rewrite is black art! (and deadly plagued with
unbalanced FS!)
The cocoon sitemap was *designed* to remove that unbalanced perception
and make it possible for people to start designing their request space
*without* starting from the URI-space.
I still consider it valuable to provide this 'routing-logic equivalence'
right from the sitemap semantics.
Berin also suggested that placing this URI-based logic into the sitemap
should allow for performance improvements. While I think this is mostly
an implementation detail, I think he is right noting that matchers and
selectors have different algorithmical behaviors and sometimes,
URI-based routing should use selectors to improve efficiency.
In fact, like Judson brilliantly suggested above, selectors perform real
'routing' (choose one route into n) while matchers perform conditional
deviations that go back to the main flow.
Using tubes and valves that would yield:
+->[ ]--+
| |
| |
matchers -->*-------+--->
->[ ]-->
/
selectors -->*-->[ ]-->
\
->[ ]-->
[having studied opto-electronics I've seen all sort of things with
optical pipelines, ie. fibers]
As you can evidently see, topologically, the two are *not* equivalent:
you can't use one to make the other.
-------------
Question: can one component perform both?
-------------
the good-old if/else semantics does matchers
<if test="">
...
</if>
as well as selectors
<if test="">
...
<else>
<if test="">
...
</if>
</else>
</if>
----------
Question: *should* they be merged?
----------
Well, this is the key question.
The above syntax is too verbose and ugly for my personal taste.
Moreover, the <if> tag provides a strong negative appeal for people
approaching an XML markup language.
Let's list the differences:
matchers:
- or-like behavior
- return map of tokens
- don't provide a fall-back mechanism
selectors:
- and-like behavior
- do not return anything
- provide a fall-back mechanism
Let's now outline the reasons for this differences:
1) matchers don't provide a fall-back mechanism because they can't:
every matcher is executed anyway.
2) if selectors and matchers can be mixed with no restriction, there is
no need to have a single component that performs both or and and-type
boolean action.
IMO, the real key question becomes
------------
question: should selectors return maps of tokens?
------------
In that case, we achieve two things:
1) the interfaces Matcher and Selector are unified, thus removing the
perception that they are just doing the same thing. In fact, from the
topological views, the * (star) performs the routing process and it
doesn't make any sense to have two different ones. Expecially if they
end up being the cut/paste clone of one another.
2) and-type boolean processing can be performed even for URI-matching,
thus providing an algorithmical way to increase sitemap interpretation.
3) back-compatibility at the sitemap level can achieved
4) we don't use ugly markup code.
Comments?
--
Stefano Mazzocchi One must still have chaos in oneself to be
able to give birth to a dancing star.
<st...@apache.org> Friedrich Nietzsche
--------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: Matchers vs. Selectors [was Re: Retuning Sitemap Design]
Posted by Stefano Mazzocchi <st...@apache.org>.
Peter Royal wrote:
>
> On Tuesday 15 January 2002 03:23 pm, Berin Loritsch wrote:
> > Are there some things that can't be unrolled? Sure, but a great many
> > things can. In many installations, a URI will represent a collection of
> > resources on the harddrive that the sitemap then assembles into the final
> > resource.
> >
> > With your pattern given above, we might have something like this:
> >
> > <map:match pattern="movies/*/*/*">
> > <map:generate src="docs/movies/{1}/{2}/{3}.xml"/>
> > <map:transform type="i18n" src="{1}"/>
> > <map:transform src="site2xhtml.xsl"/>
> > <map:serialize.>
> > </map:match>
>
> When I saw Stefano's example I instantly thought he was demonstrating a
> resource that would be generated dynamically, and thus unrolling would do no
> good. (Just to show that two people can look at the same thing and get
> something completely different)
>
> While many Cocoon users will be starting from static files on a filesystem, I
> would imagine that an equal number would be dealing with pure dynamic
> resources that would not benefit from the unrolling.
>
> I do think that your idea is excellent for filesystem-based resources.
> Perhaps the ability to plug in different types of pipeline's would be the
> solution?
>
> <map:pipeline type="filesystem-unrolling"/>
I think there is no need for this: a special selector will just do it.
--
Stefano Mazzocchi One must still have chaos in oneself to be
able to give birth to a dancing star.
<st...@apache.org> Friedrich Nietzsche
--------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: Matchers vs. Selectors [was Re: Retuning Sitemap Design]
Posted by Peter Royal <pr...@managingpartners.com>.
On Tuesday 15 January 2002 03:23 pm, Berin Loritsch wrote:
> Are there some things that can't be unrolled? Sure, but a great many
> things can. In many installations, a URI will represent a collection of
> resources on the harddrive that the sitemap then assembles into the final
> resource.
>
> With your pattern given above, we might have something like this:
>
> <map:match pattern="movies/*/*/*">
> <map:generate src="docs/movies/{1}/{2}/{3}.xml"/>
> <map:transform type="i18n" src="{1}"/>
> <map:transform src="site2xhtml.xsl"/>
> <map:serialize.>
> </map:match>
When I saw Stefano's example I instantly thought he was demonstrating a
resource that would be generated dynamically, and thus unrolling would do no
good. (Just to show that two people can look at the same thing and get
something completely different)
While many Cocoon users will be starting from static files on a filesystem, I
would imagine that an equal number would be dealing with pure dynamic
resources that would not benefit from the unrolling.
I do think that your idea is excellent for filesystem-based resources.
Perhaps the ability to plug in different types of pipeline's would be the
solution?
<map:pipeline type="filesystem-unrolling"/>
-pete
--
peter royal -> proyal@managingpartners.com
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: Matchers vs. Selectors [was Re: Retuning Sitemap Design]
Posted by Stefano Mazzocchi <st...@apache.org>.
Berin Loritsch wrote:
>
> Stefano Mazzocchi wrote:
>
> > Berin Loritsch wrote:
> >
> >>My approach assumes that the regexp is used in the unrolling process. I.e.
> >>when the sitemap is set up, the sitemap checks for files that match the
> >>patterns in the "src" attributes like this:
> >>
> >>docs/
> >> store-1001/
> >> section1/index.html
> >> section2/index.html
> >> store-2056/
> >> section1/index.html
> >> section2/index.html
> >>
> >>Such a directory structure will unroll to the following lookup values in
> >>the hashmap:
> >>
> >>"store-1001/section-1/index.html"
> >>"store-1001/section-2/index.html"
> >>"store-2056/section-1/index.html"
> >>"store-2056/section-2/index.html"
> >>
> >>So that during the runtime of the system, the request is easily matched
> >>with a simple get on a hashmap.
> >>
> >>This also provides another layer of validation: No uri is added that
> >>cannot be resolved--offering a quick 404 detection.
> >>
> >
> > ????
> >
> > The above it's possible *only* if you assume that Cocoon already knows
> > all the URIs that it will serve, but this is *almost never* the case,
> > unless you go back to the URI-2-filesystem type of matching that is
> > *exactly* what I want to stay away from.
>
> Notice the key word "unrolling". The resources are dynamically unrolled
> when the sitemap is created. It is possible to figure out the actual
> URIs from the combination of pattern and "src" attribute looking at the
> filesystem.
Yes, but this is an extremly particular case and you can't know this for
sure (even if you use the 'file' generator, it could be a URI on an
external machine...)
> > Think of a wildcard matching of
> >
> > movies/*/*/*
> >
> > where
> >
> > {1} is a country
> > {2} is a city
> > {3} is a movie-title
> >
> > how in hell are you going to 'unroll' this?
> >
> > I'm very puzzled.
>
> You are missing the key pieces. You need three pieces of information in
> order to unroll a URI.
>
> 1) the pattern
> 2) the src attribute on the generator/transformer(s)
> 3) the resource location
>
> Are there some things that can't be unrolled? Sure, but a great many
> things can. In many installations, a URI will represent a collection of
> resources on the harddrive that the sitemap then assembles into the final
> resource.
>
> With your pattern given above, we might have something like this:
>
> <map:match pattern="movies/*/*/*">
> <map:generate src="docs/movies/{1}/{2}/{3}.xml"/>
> <map:transform type="i18n" src="{1}"/>
> <map:transform src="site2xhtml.xsl"/>
> <map:serialize.>
> </map:match>
>
> We now have two of the three parts.
>
> The unrolling algorithm would perform the following steps:
>
> 1) look in ${context}/docs/movies and get a set of all directories
> (since we have another level)
> 2) look in each directory in the set for another subdirectory
> and refine the set to include all matches so far--
> discarding false matches from the previous step.
> 3) look in each directory in the new set for all "*.xml" files
> refining the set to include all definite matches.
>
> That approach is the monolithic "all in advance" algorithm.
> An alternative algorithm--one that is not so time consuming, and
> one that is immune from resources that are not on the drive is
> the incremental approach.
>
> The sitemap always checks the unrolled URIs first. If the URI
> is not in there, it tries to resolve it the hard way. When a
> match is found, the requesting URI is added to the unrolled
> URI map.
>
> This approach is preferred, as it favors real matches and speeds
> the process of matching those resources. Bad URIs will always
> take longer to process, but that is ok--we want to favor good
> uris.
How do you update this? thread in the background?
How does it work if the stuff is stored in an XML DB? or in a CVS? or in
a RDBMS? or on another HTTP machine?
How long will it take for a cocoon to 'start'?
Don't get me wrong, I do see some value in your approach, but this is a
very special case and must be dealt with as such... we can't turn this
behavior into sitemap semantics or it will work ok (even if with very
high latencies) for file systems and very bad for anything else.
--
Stefano Mazzocchi One must still have chaos in oneself to be
able to give birth to a dancing star.
<st...@apache.org> Friedrich Nietzsche
--------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: Matchers vs. Selectors [was Re: Retuning Sitemap Design]
Posted by Berin Loritsch <bl...@apache.org>.
Stefano Mazzocchi wrote:
> Berin Loritsch wrote:
>
>>My approach assumes that the regexp is used in the unrolling process. I.e.
>>when the sitemap is set up, the sitemap checks for files that match the
>>patterns in the "src" attributes like this:
>>
>>docs/
>> store-1001/
>> section1/index.html
>> section2/index.html
>> store-2056/
>> section1/index.html
>> section2/index.html
>>
>>Such a directory structure will unroll to the following lookup values in
>>the hashmap:
>>
>>"store-1001/section-1/index.html"
>>"store-1001/section-2/index.html"
>>"store-2056/section-1/index.html"
>>"store-2056/section-2/index.html"
>>
>>So that during the runtime of the system, the request is easily matched
>>with a simple get on a hashmap.
>>
>>This also provides another layer of validation: No uri is added that
>>cannot be resolved--offering a quick 404 detection.
>>
>
> ????
>
> The above it's possible *only* if you assume that Cocoon already knows
> all the URIs that it will serve, but this is *almost never* the case,
> unless you go back to the URI-2-filesystem type of matching that is
> *exactly* what I want to stay away from.
Notice the key word "unrolling". The resources are dynamically unrolled
when the sitemap is created. It is possible to figure out the actual
URIs from the combination of pattern and "src" attribute looking at the
filesystem.
> Think of a wildcard matching of
>
> movies/*/*/*
>
> where
>
> {1} is a country
> {2} is a city
> {3} is a movie-title
>
> how in hell are you going to 'unroll' this?
>
> I'm very puzzled.
You are missing the key pieces. You need three pieces of information in
order to unroll a URI.
1) the pattern
2) the src attribute on the generator/transformer(s)
3) the resource location
Are there some things that can't be unrolled? Sure, but a great many
things can. In many installations, a URI will represent a collection of
resources on the harddrive that the sitemap then assembles into the final
resource.
With your pattern given above, we might have something like this:
<map:match pattern="movies/*/*/*">
<map:generate src="docs/movies/{1}/{2}/{3}.xml"/>
<map:transform type="i18n" src="{1}"/>
<map:transform src="site2xhtml.xsl"/>
<map:serialize.>
</map:match>
We now have two of the three parts.
The unrolling algorithm would perform the following steps:
1) look in ${context}/docs/movies and get a set of all directories
(since we have another level)
2) look in each directory in the set for another subdirectory
and refine the set to include all matches so far--
discarding false matches from the previous step.
3) look in each directory in the new set for all "*.xml" files
refining the set to include all definite matches.
That approach is the monolithic "all in advance" algorithm.
An alternative algorithm--one that is not so time consuming, and
one that is immune from resources that are not on the drive is
the incremental approach.
The sitemap always checks the unrolled URIs first. If the URI
is not in there, it tries to resolve it the hard way. When a
match is found, the requesting URI is added to the unrolled
URI map.
This approach is preferred, as it favors real matches and speeds
the process of matching those resources. Bad URIs will always
take longer to process, but that is ok--we want to favor good
uris.
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: Matchers vs. Selectors [was Re: Retuning Sitemap Design]
Posted by Stefano Mazzocchi <st...@apache.org>.
Berin Loritsch wrote:
>
> Stefano Mazzocchi wrote:
>
> >
> > So, for example (keeping syntax back compatible)
> >
> > <select type="uri">
> > <when test="store-([0-9]*)\/section-([0-9]*)\/index\.xhtml"
> > type="regexp>
> > <generator src="docs/store-{1}/section{2}/index.html" />
> > <transformer/>
> > <serializer/>
> > </when>
> > <when test="store-*/compliance.pdf" type="wildcard">
> > <generator src="docs/compliance-report.xsp">
> > <parameter name="storenbr" value="{1}"/>
> > </generator>
> > <transformer/>
> > <serializer/>
> > </when>
> > <otherwise>
> > <generator src="{1}"> // what does the 'otherwise' part return?
> > <transformer/>
> > <serializer/>
> > </otherwise>
> > </select>
> >
> > but the question becomes:
> >
> > if we can't implement hash-based jumping based on regexps, the above has
> > no effect on performance since you have to perform the regexp matching
> > sequentially anyway, and hitting a serializer has the effect of exiting
> > the pipeline anyway, even on matchers.
> >
> > So, really, what benefits are we getting?
>
> My approach assumes that the regexp is used in the unrolling process. I.e.
> when the sitemap is set up, the sitemap checks for files that match the
> patterns in the "src" attributes like this:
>
> docs/
> store-1001/
> section1/index.html
> section2/index.html
> store-2056/
> section1/index.html
> section2/index.html
>
> Such a directory structure will unroll to the following lookup values in
> the hashmap:
>
> "store-1001/section-1/index.html"
> "store-1001/section-2/index.html"
> "store-2056/section-1/index.html"
> "store-2056/section-2/index.html"
>
> So that during the runtime of the system, the request is easily matched
> with a simple get on a hashmap.
>
> This also provides another layer of validation: No uri is added that
> cannot be resolved--offering a quick 404 detection.
????
The above it's possible *only* if you assume that Cocoon already knows
all the URIs that it will serve, but this is *almost never* the case,
unless you go back to the URI-2-filesystem type of matching that is
*exactly* what I want to stay away from.
Think of a wildcard matching of
movies/*/*/*
where
{1} is a country
{2} is a city
{3} is a movie-title
how in hell are you going to 'unroll' this?
I'm very puzzled.
--
Stefano Mazzocchi One must still have chaos in oneself to be
able to give birth to a dancing star.
<st...@apache.org> Friedrich Nietzsche
--------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: Matchers vs. Selectors [was Re: Retuning Sitemap Design]
Posted by Berin Loritsch <bl...@apache.org>.
Stefano Mazzocchi wrote:
>
> So, for example (keeping syntax back compatible)
>
> <select type="uri">
> <when test="store-([0-9]*)\/section-([0-9]*)\/index\.xhtml"
> type="regexp>
> <generator src="docs/store-{1}/section{2}/index.html" />
> <transformer/>
> <serializer/>
> </when>
> <when test="store-*/compliance.pdf" type="wildcard">
> <generator src="docs/compliance-report.xsp">
> <parameter name="storenbr" value="{1}"/>
> </generator>
> <transformer/>
> <serializer/>
> </when>
> <otherwise>
> <generator src="{1}"> // what does the 'otherwise' part return?
> <transformer/>
> <serializer/>
> </otherwise>
> </select>
>
> but the question becomes:
>
> if we can't implement hash-based jumping based on regexps, the above has
> no effect on performance since you have to perform the regexp matching
> sequentially anyway, and hitting a serializer has the effect of exiting
> the pipeline anyway, even on matchers.
>
> So, really, what benefits are we getting?
My approach assumes that the regexp is used in the unrolling process. I.e.
when the sitemap is set up, the sitemap checks for files that match the
patterns in the "src" attributes like this:
docs/
store-1001/
section1/index.html
section2/index.html
store-2056/
section1/index.html
section2/index.html
Such a directory structure will unroll to the following lookup values in
the hashmap:
"store-1001/section-1/index.html"
"store-1001/section-2/index.html"
"store-2056/section-1/index.html"
"store-2056/section-2/index.html"
So that during the runtime of the system, the request is easily matched
with a simple get on a hashmap.
This also provides another layer of validation: No uri is added that
cannot be resolved--offering a quick 404 detection.
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: Matchers vs. Selectors [was Re: Retuning Sitemap Design]
Posted by Stefano Mazzocchi <st...@apache.org>.
Berin Loritsch wrote:
>
> Stefano Mazzocchi wrote:
>
> > Judson Lester wrote:
> >
> > <snip/>
> >
> >>The clearest dilemma I can see is that <select><match test=A><match
> >>test=B></select> implies that if A matches, and B matches, two clauses occur,
> >>but <select><when test=A><when test=B></select> implies that if A matches, B
> >>won't be tested. Is this FS or useful power?
> >>
> >
> > I think you touched the nerve of the discussion:
> >
> > - matchers distill the behavior of boolean 'or'
> > - selectors distill the behavior of boolean 'and'
> >
> > Berin suggested that URI-matching performed with 'and'-type boolean
> > logic is instrinsically faster since it is algorithmically less complex.
> >
> > I think he does have a point here!
>
> I think that my point also includes the fact that URI is the primary match
> for any site. When we go to a site, we expect that a specific URI returns
> a specific resource. This is a contract we never want to break.
absolutely.
> However, if I request "http://jarkarta.apache.org/developing-with-avalon"
> the resource can be one of a number of different representations of the
> same resource. For instance, if Adobe Acrobat Reader had a browser in it,
> Cocoon could detect it as the browser and return the PDF view of the
> resource. If I have Mozilla 0.9.7, I could get my markup optimized for
> spec compliant browsers. The basic underlying contract is that the URI
> directly relates to a resource.
as the name implies.
> In that case, using a simple HashMap to match unrolled URIs to pipelines
> that have only one resource per pipeline we effectively speed our access
> to the desired resource.
The problem is that if you require a pipeline per resource, your
administration costs grow linear with the URI-space size. and we
*DO*NOT* want this, no matter what speed improvement you get.
[btw, I think it's possible to perform time-constant hash-based jumps
even on regexps, but didn't think about the algorithm yet]
> Currently the critical path to access the pipelines is way too long. A
> while ago, I posted a sequence diagram that shows just how complex a single
> request is. It has only gotten more complex. The link to the diagram is
> http://jakarta.apache.org/~bloritsch/CocoonServlet_service.png and is
> quite large.
Yes, I was perfectly aware of the fact that the intrinsic linear-flow of
the sitemap could potentially be an architectural bottleneck, this is
why we introduced wildcard based matching to reduce the pipeline
complexity.
<snip/>
> > Just like a real pipeline requires 'valves' to route its flow, we
> > decided to introduce 'support components' that could do the same and
> > decided to make them 'components' to allow the 'routing logic' to be
> > pluggable.
>
> :/
>
> The more I look at it, the concept of matcher/selector is merely program
> flow directives. Granting them full Component status seems wasteful.
I disagree. We just barely started to scratch the surface of what
matchers can do. I'm sure people will find very creative ways to add new
matching logics (CC/PP for example, or load, or time of the day to
perform color-shading sunset-like stylesheets, you name it)
The current problem is that Cocoon doesn't allow you to drop your Cocoon
Web Applications on it just like we deploy WARs on Tomcat.
*this* is the key. Once we have that, people will find very creative
ways of using these conditional components and will love the ability to
reuse them all over their stuff with a simple line added to their
sitemap.
> In fact, such flow control devices are bound to the particular Sitemap
> implementation.
See above: the problem is not in the sitemap design, but in Cocoon's
lack of easy deploying of web applications on top. (which is something I
want to fix ASAP!)
> For instance, the difference between a RegexpURIMatcher
> and the WildcardURIMatcher is the pattern used for matching. In fact,
> you could have one matcher that performs the test with a "protocol"
> to select the pattern type. For example:
> "re:store-([0-9]*)/section-([0-9]*)/index.html" and
> "wildcard:store-*/section-*/index.html" are roughly equivalent matches
> (the regular expression is more specific).
Yes, but I think it's useful syntax sugar since
<match type="regexp" pattern="...">
is much more readable and understandable than
<match pattern="re:...">
> The idea of the matcher and selector is best applied by unrolling the URIs.
> The process of unrolling a URI can be demand based, or static. If it is
> demand based, the URI is processed through the matchers as it is received,
> and it is added to the "good" map or the "bad" map. If unrolling is statically
> done, the Sitemap expands the matches and examines the harddrive for the
> files that the match applies to.
>
> More a bit later.
>
> > In fact, like Judson brilliantly suggested above, selectors perform real
> > 'routing' (choose one route into n) while matchers perform conditional
> > deviations that go back to the main flow.
> >
> > Using tubes and valves that would yield:
> >
> > +->[ ]--+
> > | |
> > | |
> > matchers -->*-------+--->
> >
> >
> > ->[ ]-->
> > /
> > selectors -->*-->[ ]-->
> > \
> > ->[ ]-->
> >
> > [having studied opto-electronics I've seen all sort of things with
> > optical pipelines, ie. fibers]
> >
> > As you can evidently see, topologically, the two are *not* equivalent:
> > you can't use one to make the other.
> >
> > -------------
> > Question: can one component perform both?
> > -------------
>
> It is not an easy thing to decide. For instance, the Action and the Matcher
> could both be merged quite easily as they have similar semantics, though
> differing focus. Consider the API for an Action:
>
> interface Action
> {
> Map act(Redirector rd, Resolver rs, Map objectMap, String source, Parameters p);
> }
>
> It's at least close. A matcher is basically a simplified version of this.
Yes, good point.
> In essence the redirector and resolver are removed from a Matcher, and a matcher
> has the ability to precompile the pattern.
>
> Both of them return a Map to define substitution variables.
>
> Contrast this with a Selector which returns a boolean.
Hmmm, what do other think of this?
> > the good-old if/else semantics does matchers
> >
> > <if test="">
> > ...
> > </if>
> >
> > as well as selectors
> >
> > <if test="">
> > ...
> > <else>
> > <if test="">
> > ...
> > </if>
> > </else>
> > </if>
>
> Yuck. I think this why we had the Matcher/Selector idea to begin with.
Exactly!
> > ----------
> > Question: *should* they be merged?
> > ----------
> >
> > Well, this is the key question.
> >
> > The above syntax is too verbose and ugly for my personal taste.
> > Moreover, the <if> tag provides a strong negative appeal for people
> > approaching an XML markup language.
>
> I agree.
>
> > ------------
> > question: should selectors return maps of tokens?
> > ------------
> >
> > In that case, we achieve two things:
> >
> > 1) the interfaces Matcher and Selector are unified, thus removing the
> > perception that they are just doing the same thing. In fact, from the
> > topological views, the * (star) performs the routing process and it
> > doesn't make any sense to have two different ones. Expecially if they
> > end up being the cut/paste clone of one another.
> >
> > 2) and-type boolean processing can be performed even for URI-matching,
> > thus providing an algorithmical way to increase sitemap interpretation.
> >
> > 3) back-compatibility at the sitemap level can achieved
> >
> > 4) we don't use ugly markup code.
>
> The returned map should have a static lookup object, that way unrolled
> selectors would store the possible answers in a lookup with a callback
> for the logic that is performed for that case. For instance:
>
> Map results = selector.select( "test", objectMap, params );
> Command callback = (Command) commandMap.get( results.get( Selector.RETURN_OBJECT ) );
> callback.execute( results, objectMap, params, pipeline );
>
> It simplifies alot of implementation details, and the sitemap semantics
> for the improved selector would look like this:
>
> <selector type="regexp-uri">
> <case test="store-([0-9]*)\/section-([0-9]*)\/index\.xhtml">
> <generator uri="docs/store-{1}/section{2}/index.html" />
> <transformer/>
> <serializer/>
> </case>
> <case test="store-([0-9]*)\/compliance\.pdf">
> <generator uri="docs/compliance-report.xsp">
> <parameter name="storenbr" value="{1}"/>
> </generator>
> <transformer/>
> <serializer/>
> </case>
> </selector>
>
> This I think is not only more compact than several matchers, but
> the connotation is that every test is mutually exclusive.
I agree with you on the fact that sometimes the 'fall-thru' behavior of
matchers is misleading.
And I agree with you that even selectors should provide the ability to
return tokens.
So, for example (keeping syntax back compatible)
<select type="uri">
<when test="store-([0-9]*)\/section-([0-9]*)\/index\.xhtml"
type="regexp>
<generator src="docs/store-{1}/section{2}/index.html" />
<transformer/>
<serializer/>
</when>
<when test="store-*/compliance.pdf" type="wildcard">
<generator src="docs/compliance-report.xsp">
<parameter name="storenbr" value="{1}"/>
</generator>
<transformer/>
<serializer/>
</when>
<otherwise>
<generator src="{1}"> // what does the 'otherwise' part return?
<transformer/>
<serializer/>
</otherwise>
</select>
but the question becomes:
if we can't implement hash-based jumping based on regexps, the above has
no effect on performance since you have to perform the regexp matching
sequentially anyway, and hitting a serializer has the effect of exiting
the pipeline anyway, even on matchers.
So, really, what benefits are we getting?
--
Stefano Mazzocchi One must still have chaos in oneself to be
able to give birth to a dancing star.
<st...@apache.org> Friedrich Nietzsche
--------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: Matchers vs. Selectors [was Re: Retuning Sitemap Design]
Posted by Berin Loritsch <bl...@apache.org>.
Stefano Mazzocchi wrote:
> Judson Lester wrote:
>
> <snip/>
>
>>The clearest dilemma I can see is that <select><match test=A><match
>>test=B></select> implies that if A matches, and B matches, two clauses occur,
>>but <select><when test=A><when test=B></select> implies that if A matches, B
>>won't be tested. Is this FS or useful power?
>>
>
> I think you touched the nerve of the discussion:
>
> - matchers distill the behavior of boolean 'or'
> - selectors distill the behavior of boolean 'and'
>
> Berin suggested that URI-matching performed with 'and'-type boolean
> logic is instrinsically faster since it is algorithmically less complex.
>
> I think he does have a point here!
I think that my point also includes the fact that URI is the primary match
for any site. When we go to a site, we expect that a specific URI returns
a specific resource. This is a contract we never want to break.
However, if I request "http://jarkarta.apache.org/developing-with-avalon"
the resource can be one of a number of different representations of the
same resource. For instance, if Adobe Acrobat Reader had a browser in it,
Cocoon could detect it as the browser and return the PDF view of the
resource. If I have Mozilla 0.9.7, I could get my markup optimized for
spec compliant browsers. The basic underlying contract is that the URI
directly relates to a resource.
In that case, using a simple HashMap to match unrolled URIs to pipelines
that have only one resource per pipeline we effectively speed our access
to the desired resource.
Currently the critical path to access the pipelines is way too long. A
while ago, I posted a sequence diagram that shows just how complex a single
request is. It has only gotten more complex. The link to the diagram is
http://jakarta.apache.org/~bloritsch/CocoonServlet_service.png and is
quite large.
Getting back to the point of direct URI to pipeline matching: if we require
any part of the resource to be dynamic (such as representation), we can
perform the selection of transformers based on other request aspects such
as client. What will not change for the pipeline is the generator.
> - o -
>
> The goals of the indirect sitemap components (those who don't generate
> data that goes directly inside the SAX stream) are to provide support
> for the other direct sitemap components (those who do).
>
> Right at the beginning of the sitemap design, it was clear that the
> functionality of 'routing' the request thru the different components was
> absolutely required in order to allow Cocoon2's sitemap to match
> Cocoon1's reactor in functionality. Otherwise, forking friction would
> have been developped.
>
> At the same time, the reactor (which appeared as a very good idea when I
> proposed it) is sort of a dynamic-pipeline composer. We knew that
> dynamic pipelines provided several limitations:
>
> - where impossible to 'validate' at load-time
> - where impossible to 'pre-compile' and optimize
> - where harder to pool
> - where less cache friendly
>
> and more important:
>
> - where much more painful to debug!
We have only marginally addressed these issues in our implementation.
However, we are closer to where we belong. Most of the resources
I deal with (i.e. URI targets) are static pipelines--even though the
content is somewhat dynamic (tied to database). This will be the case
about 70% of the time or more. However, themed sites do have a
dynamic pipeline. Usually the point of dynacism in the pipeline is
directly related to one session attribute.
We also have to come to grips with what appears to be a dynamic pipeline,
but is really only a choice between two static pipelines. This is the
authentication and authorization issue. In this case, the priority of
the request/session attributes comes above the URI.
> Just like a real pipeline requires 'valves' to route its flow, we
> decided to introduce 'support components' that could do the same and
> decided to make them 'components' to allow the 'routing logic' to be
> pluggable.
:/
The more I look at it, the concept of matcher/selector is merely program
flow directives. Granting them full Component status seems wasteful.
In fact, such flow control devices are bound to the particular Sitemap
implementation. For instance, the difference between a RegexpURIMatcher
and the WildcardURIMatcher is the pattern used for matching. In fact,
you could have one matcher that performs the test with a "protocol"
to select the pattern type. For example:
"re:store-([0-9]*)/section-([0-9]*)/index.html" and
"wildcard:store-*/section-*/index.html" are roughly equivalent matches
(the regular expression is more specific).
The idea of the matcher and selector is best applied by unrolling the URIs.
The process of unrolling a URI can be demand based, or static. If it is
demand based, the URI is processed through the matchers as it is received,
and it is added to the "good" map or the "bad" map. If unrolling is statically
done, the Sitemap expands the matches and examines the harddrive for the
files that the match applies to.
More a bit later.
> In fact, like Judson brilliantly suggested above, selectors perform real
> 'routing' (choose one route into n) while matchers perform conditional
> deviations that go back to the main flow.
>
> Using tubes and valves that would yield:
>
> +->[ ]--+
> | |
> | |
> matchers -->*-------+--->
>
>
> ->[ ]-->
> /
> selectors -->*-->[ ]-->
> \
> ->[ ]-->
>
> [having studied opto-electronics I've seen all sort of things with
> optical pipelines, ie. fibers]
>
> As you can evidently see, topologically, the two are *not* equivalent:
> you can't use one to make the other.
>
> -------------
> Question: can one component perform both?
> -------------
It is not an easy thing to decide. For instance, the Action and the Matcher
could both be merged quite easily as they have similar semantics, though
differing focus. Consider the API for an Action:
interface Action
{
Map act(Redirector rd, Resolver rs, Map objectMap, String source, Parameters p);
}
It's at least close. A matcher is basically a simplified version of this.
In essence the redirector and resolver are removed from a Matcher, and a matcher
has the ability to precompile the pattern.
Both of them return a Map to define substitution variables.
Contrast this with a Selector which returns a boolean.
> the good-old if/else semantics does matchers
>
> <if test="">
> ...
> </if>
>
> as well as selectors
>
> <if test="">
> ...
> <else>
> <if test="">
> ...
> </if>
> </else>
> </if>
Yuck. I think this why we had the Matcher/Selector idea to begin with.
> ----------
> Question: *should* they be merged?
> ----------
>
> Well, this is the key question.
>
> The above syntax is too verbose and ugly for my personal taste.
> Moreover, the <if> tag provides a strong negative appeal for people
> approaching an XML markup language.
I agree.
> ------------
> question: should selectors return maps of tokens?
> ------------
>
> In that case, we achieve two things:
>
> 1) the interfaces Matcher and Selector are unified, thus removing the
> perception that they are just doing the same thing. In fact, from the
> topological views, the * (star) performs the routing process and it
> doesn't make any sense to have two different ones. Expecially if they
> end up being the cut/paste clone of one another.
>
> 2) and-type boolean processing can be performed even for URI-matching,
> thus providing an algorithmical way to increase sitemap interpretation.
>
> 3) back-compatibility at the sitemap level can achieved
>
> 4) we don't use ugly markup code.
The returned map should have a static lookup object, that way unrolled
selectors would store the possible answers in a lookup with a callback
for the logic that is performed for that case. For instance:
Map results = selector.select( "test", objectMap, params );
Command callback = (Command) commandMap.get( results.get( Selector.RETURN_OBJECT ) );
callback.execute( results, objectMap, params, pipeline );
It simplifies alot of implementation details, and the sitemap semantics
for the improved selector would look like this:
<selector type="regexp-uri">
<case test="store-([0-9]*)\/section-([0-9]*)\/index\.xhtml">
<generator uri="docs/store-{1}/section{2}/index.html" />
<transformer/>
<serializer/>
</case>
<case test="store-([0-9]*)\/compliance\.pdf">
<generator uri="docs/compliance-report.xsp">
<parameter name="storenbr" value="{1}"/>
</generator>
<transformer/>
<serializer/>
</case>
</selector>
This I think is not only more compact than several matchers, but
the connotation is that every test is mutually exclusive.
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org