You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cocoon.apache.org by Jeremy Quinn <je...@media.demon.co.uk> on 2003/04/28 16:16:34 UTC
[RT] Link Rewriting
Hi All
Welcome to my first RT!
Sorry to start on a negative note, but IMHO, the current scheme for
LinkRewriting just does not make sense for dynamic sites.
What the current scheme allows is for authors to use a simple set of
declarators for defining links, which are then translated into actual
URLs, (and in the samples) mapped directly to filespace (etc.).
What I believe a LinkRewriting infrastructure should offer is rather
different.
A URL is a CONTRACT.
Cocoon's sitemap is excellent at disconnecting any direct relationship
between URL and resource ID/Path (SystemID). Allowing the
re-implementation of storage schema independently of URL contract.
What I feel makes more sense is this:
URL == permanent contract == authoring link != SystemID
What this means is, authors should write links using a static absolute
URL, the same one as the public contract for that particular piece of
information.
When that URL is accessed, it should be mapped to a SystemID, allowing
independent re-implementation of the storage layer.
If this is to be handled by an input module accessing a linkmap (before
Generation, rather than during Transformation), but requires that
dynamic sitemap URL fragments can be passed to input modules (as they
cannot currently).
see: <http://marc.theaimsgroup.com/?l=xml-cocoon-dev&m=104895916400307>
The transformation stage is then merely one of relativising author's
URLs.
Most of what I have described here, is how most people use Cocoon.
What is new here is the use of a LinkMap at the Generation stage to
de-couple URL from SystemID in a totally arbitrary way. A version of
the LinkMap idea that makes sense for Dynamic sites. This is what
requires changes to the way we are able to use input modules, as input
modules would provide a much cleaner path to handle this rather than
Transforming a generated LinkMap into CInclude tags to get the content.
Does this make any sense to anybody?
Thanks for reading this far.
regards Jeremy
Re: [RT] Link Rewriting
Posted by Jeremy Quinn <je...@media.demon.co.uk>.
On Monday, April 28, 2003, at 03:30 PM, Nicola Ken Barozzi wrote:
>
> Jeremy Quinn wrote, On 28/04/2003 16.16:
> ...
>> Most of what I have described here, is how most people use Cocoon.
>> What is new here is the use of a LinkMap at the Generation stage to
>> de-couple URL from SystemID in a totally arbitrary way. A version of
>> the LinkMap idea that makes sense for Dynamic sites. This is what
>> requires changes to the way we are able to use input modules, as
>> input modules would provide a much cleaner path to handle this rather
>> than Transforming a generated LinkMap into CInclude tags to get the
>> content.
>> Does this make any sense to anybody?
>
> Sure it does, although IMHO it's more suited to forrest-dev than
> cocoon-dev ATM.
Agreed
> What you describe here is what I calla resourcemap, that is a
> descriptor that creates a mapping between a conceptual source space
> and the actual source locations.
>
> If you search the forrest-dev mailing list for resourcemap, you should
> find plenty of RTs, even quite recent.
OK, this does not surprise me ;)
> A brief explanation:
>
> There are different URI spaces:
>
> 1 - the actual sources
> 2 - a conceptual aggregated source space
By this you mean something like a mixture of sources, like some from
file, some from XMLDB, or mixed URLs like this:
http://www.iniva.org/library/123456
+-------------+-------+------+
| | |
virtual host file SQL ID
> 3 - the actual URIs
> 4 - a conceptual URI space
>
> The current link rewriting is used to make users be able to use 4 and
> obtain 3.
Which is logical for Forrest
> What you (and I) feel the need of too, is to have 2 map to 1, so that
> I can easily swap content sources and keep the sitemap as-is.
Particularly when the URL space and System ID space are arbitrarily
different.
regards Jeremy
------------------------------------------------------------------------
"Objective reality is a synthetic construct, dealing with a
hypothetical universalization of a multitude of subjective realities."
Philip K Dick - "The Electric Ant"
Re: [RT] Link Rewriting
Posted by Nicola Ken Barozzi <ni...@apache.org>.
Jeremy Quinn wrote, On 28/04/2003 16.16:
...
> Most of what I have described here, is how most people use Cocoon.
> What is new here is the use of a LinkMap at the Generation stage to
> de-couple URL from SystemID in a totally arbitrary way. A version of the
> LinkMap idea that makes sense for Dynamic sites. This is what requires
> changes to the way we are able to use input modules, as input modules
> would provide a much cleaner path to handle this rather than
> Transforming a generated LinkMap into CInclude tags to get the content.
>
> Does this make any sense to anybody?
Sure it does, although IMHO it's more suited to forrest-dev than
cocoon-dev ATM.
What you describe here is what I calla resourcemap, that is a descriptor
that creates a mapping between a conceptual source space and the actual
source locations.
If you search the forrest-dev mailing list for resourcemap, you should
find plenty of RTs, even quite recent.
A brief explanation:
There are different URI spaces:
1 - the actual sources
2 - a conceptual aggregated source space
3 - the actual URIs
4 - a conceptual URI space
The current link rewriting is used to make users be able to use 4 and
obtain 3. What you (and I) feel the need of too, is to have 2 map to 1,
so that I can easily swap content sources and keep the sitemap as-is.
--
Nicola Ken Barozzi nicolaken@apache.org
- verba volant, scripta manent -
(discussions get forgotten, just code remains)
---------------------------------------------------------------------
Re: [RT] Link Rewriting
Posted by Nicola Ken Barozzi <ni...@apache.org>.
Jeff Turner wrote, On 29/04/2003 16.19:
> (detaching cocoon-dev)
Ouch! ;-)
> On Tue, Apr 29, 2003 at 02:37:09PM +0200, Nicola Ken Barozzi wrote:
...
>>Then we come out with a new version of the sitemap, and the user has to
>>again manually change the new sitemap...
>
> Oh yes..
>
>>It's the same thing I have discussed with the news section, etc. If you
>>want to use sitemaps as reusable blocks, you have to separate the source
>>resoilving and the processing blocks. The above does both.
>
> Is "the above" the suggestion to look up sources in an XML file?
I mean the conceptual thing, not implementation.
>>>>I had committed some time back a refactoring of the sitemap that did
>>>>just that, but you strongly asked to revert it, so... ;-P
>>>
>>><innocent> Me? </innocent> I don't remember it.. was this the
>>>resource-exists-athon patch? Anyway..
>>
>>Yes, it was that ;-P I centralized obtaining the source in a single
>>place ;-)
>
> I thought it was about getting rid of xdocs/.
Not really, but it doesn't matter, as now we are doing it, and much
better :-)
>>>>So, as I think you also would agree, the important thing is that one
>>>>modularizes the sitemap so that some pieces can remain as-is, and other
>>>>parts can be plugged in/expanded separately in each layer:
>>>>
>>>>0 - source resolving (for us it's fixed on the filesystem)
>>>>1 - conversion to common source
>>>>2 - processing
>>>>3 - presentation (output)
>>>>
>>>>What I'd like now is to add level 0 for our sitemap if possible.
>>>
>>>In Forrest, 0 and 1 are both handled in forrest.xmap, which defines the
>>>VFS on which everything else builds.
>>
>>Yes, as I'm trying to say, I have understood that making a new design
>>for this is not needed as proper layering of sitemaps can do it.
>>So I would propose that we have a sources.xmap and a forrest.xmap.
>>
>>The sources.xmap would do the mapping that now is done by the
>>forrest.properties.
>
>
> I was thinking we could do that by defining the project structure in XML,
> like Centipede's layout.xml used to:
>
> <dir path="/">
> <dir id="content" path="src/documentation/content/">
> <dir id="xdocs" path="xdocs/"/>
> </dir>
> <dir id="resources" path="resources/">
> <dir id="images" path="images/"/>
> </dir>
> </dir>
>
> And then using an InputModule to make this available to the sitemap:
>
> <map:match pattern="images/**.*">
> <map:read src="{layout:resources.images}/{1}.{2}" mime-type="{2}"/>
> </map:match>
>
> The layout XML would also be available to XSLTs via something like
> document('cocoon:/layout.xml').
>
> Isn't this XML more or less what a resourcemap is?
Well, you have put in nice xml what I still had as only a concept...
Hey, I mad you do work for me! <the-force>Yayyyyy!</the-force> ;-)
Ok, I'm joking... actually that was the first idea:
On 14/10/2002 17.56 Nicola Ken Barozzi wrote:
<resourcemap>
<mount space="/javadocs" resource="file://../../tools/javadocs">
<mount space="/" resource="file://.">
</resourcemap>
<resourcemap>
<mount space="/javadocs"
resource="file://../../tools/javadocs">
<mount space="/code/sourcedocs"
resource="myprotocol:/path/to/docs">
<mount space="/"
resource="file://.">
</resourcemap>
But as you know I'm not able to look near even if I have to wear
glasses, so I came up with (conceptual)
<resourcemap>
<match resource="/javadocs/*"
src="file://tools/javadocs/{1}.html">
<match resource="/"
src="file://.">
</resourcemap>
Opps, it's looking more and more like a sitemap snippet ;-)
I though I sent a mail on this but somehow it didn't go through o I
can't find it.
Anyway, we can simply expose to the users a sitemap that exposes these
resources, that can be gotten from the pipelines as cocoon://
Example resources.xmap:
<map:sitemap xmlns:map="http://apache.org/cocoon/sitemap/1.0">
<map:components>...</map:components>
<map:pipelines>
<map:pipeline>
<!-- main resoure match -->
<map:match pattern="forrest-resource/**.dtdx.xml">
<!-- ***** users can change from here... ***** -->
<!-- Include external newsfeeds -->
<map:match pattern="content/xdocs/sitenews.xml">
<map:generate type="html" src="http://mysire/rss">
<map:serialize type="xml"/>
</map:match>
<!-- #project.content-dir -->
<map:match pattern="content/**">
<map:generate src="content/{1}">
<map:serialize type="xml"/>
</map:match>
<!-- #project.xdocs-dir -->
<map:match pattern="content/xdocs/**.xml">
<map:generate src="content/xdocs/{1}.xml">
<map:serialize type="xml"/>
</map:match>
<!-- etc -->
<!-- ************** ... till here ************** -->
</map:match>
</map:pipeline>
</map:pipelines>
</map:sitemap>
--
Nicola Ken Barozzi nicolaken@apache.org
- verba volant, scripta manent -
(discussions get forgotten, just code remains)
---------------------------------------------------------------------
Re: [RT] Link Rewriting
Posted by Jeff Turner <je...@apache.org>.
(detaching cocoon-dev)
On Tue, Apr 29, 2003 at 02:37:09PM +0200, Nicola Ken Barozzi wrote:
> Jeff Turner wrote, On 29/04/2003 14.19:
> >On Tue, Apr 29, 2003 at 12:09:22PM +0200, Nicola Ken Barozzi wrote:
...
> Ok, so you give away the sitemap, and the user wants to use it to show
> his wiki. So he goes in the sitemap and changes
>
> <map:generate type="txt2xml"
> src="http://wiki.cocoondev.org/pages/{1}.txt"/>
>
> to
>
> <map:generate type="txt2xml"
> src="http://mywiki.org/pages/{1}.txt"/>
>
> Then we come out with a new version of the sitemap, and the user has to
> again manually change the new sitemap...
Oh yes..
> It's the same thing I have discussed with the news section, etc. If you
> want to use sitemaps as reusable blocks, you have to separate the source
> resoilving and the processing blocks. The above does both.
Is "the above" the suggestion to look up sources in an XML file?
> >>I had committed some time back a refactoring of the sitemap that did
> >>just that, but you strongly asked to revert it, so... ;-P
> >
> ><innocent> Me? </innocent> I don't remember it.. was this the
> >resource-exists-athon patch? Anyway..
>
> Yes, it was that ;-P I centralized obtaining the source in a single
> place ;-)
I thought it was about getting rid of xdocs/.
> >>So, as I think you also would agree, the important thing is that one
> >>modularizes the sitemap so that some pieces can remain as-is, and other
> >>parts can be plugged in/expanded separately in each layer:
> >>
> >>0 - source resolving (for us it's fixed on the filesystem)
> >>1 - conversion to common source
> >>2 - processing
> >>3 - presentation (output)
> >>
> >>What I'd like now is to add level 0 for our sitemap if possible.
> >
> >In Forrest, 0 and 1 are both handled in forrest.xmap, which defines the
> >VFS on which everything else builds.
>
> Yes, as I'm trying to say, I have understood that making a new design
> for this is not needed as proper layering of sitemaps can do it.
> So I would propose that we have a sources.xmap and a forrest.xmap.
>
> The sources.xmap would do the mapping that now is done by the
> forrest.properties.
I was thinking we could do that by defining the project structure in XML,
like Centipede's layout.xml used to:
<dir path="/">
<dir id="content" path="src/documentation/content/">
<dir id="xdocs" path="xdocs/"/>
</dir>
<dir id="resources" path="resources/">
<dir id="images" path="images/"/>
</dir>
</dir>
And then using an InputModule to make this available to the sitemap:
<map:match pattern="images/**.*">
<map:read src="{layout:resources.images}/{1}.{2}" mime-type="{2}"/>
</map:match>
The layout XML would also be available to XSLTs via something like
document('cocoon:/layout.xml').
Isn't this XML more or less what a resourcemap is?
--Jeff
> --
> Nicola Ken Barozzi nicolaken@apache.org
> - verba volant, scripta manent -
> (discussions get forgotten, just code remains)
> ---------------------------------------------------------------------
>
Re: [RT] Link Rewriting
Posted by Nicola Ken Barozzi <ni...@apache.org>.
Jeff Turner wrote, On 29/04/2003 14.19:
> On Tue, Apr 29, 2003 at 12:09:22PM +0200, Nicola Ken Barozzi wrote:
>
>>Jeff Turner wrote, On 29/04/2003 8.50:
>>...
...
>>It's about SoC. One part defines the source mapping, the other how to
>>process it. I think that he is talking about a scenario where the user
>>does not add new types, but can easily configure just the new places
>>where to find the sources.
>
>
> Yes, which can be achieved nicely with a sitemap. For instance, we could
> map the whole Wiki source into a Cocoon site with one matcher:
>
> <map:match pattern="wiki/*.xml">
> <map:generate type="txt2xml" src="http://wiki.cocoondev.org/pages/{1}.txt"/>
> <map:transform type="lexer" src="resources/grammars/wiki.xlex"/>
> <map:transform type="parser" src="resources/grammars/wiki.xgrm"/>
> <map:transform src="resources/stylesheets/wiki2document.xsl">
> <map:serialize type="xml"/>
> </map:match>
>
> (for people not on forrest-dev, there's a demo of Cocoon Wiki integration
> at http://cvs.apache.org/~jefft/forrest/samples/wikirenderer-site/)
Ok, so you give away the sitemap, and the user wants to use it to show
his wiki. So he goes in the sitemap and changes
<map:generate type="txt2xml"
src="http://wiki.cocoondev.org/pages/{1}.txt"/>
to
<map:generate type="txt2xml"
src="http://mywiki.org/pages/{1}.txt"/>
Then we come out with a new version of the sitemap, and the user has to
again manually change the new sitemap...
It's the same thing I have discussed with the news section, etc. If you
want to use sitemaps as reusable blocks, you have to separate the source
resoilving and the processing blocks. The above does both.
>>I had committed some time back a refactoring of the sitemap that did
>>just that, but you strongly asked to revert it, so... ;-P
>
> <innocent> Me? </innocent> I don't remember it.. was this the
> resource-exists-athon patch? Anyway..
Yes, it was that ;-P I centralized obtaining the source in a single
place ;-)
>>So, as I think you also would agree, the important thing is that one
>>modularizes the sitemap so that some pieces can remain as-is, and other
>>parts can be plugged in/expanded separately in each layer:
>>
>> 0 - source resolving (for us it's fixed on the filesystem)
>> 1 - conversion to common source
>> 2 - processing
>> 3 - presentation (output)
>>
>>What I'd like now is to add level 0 for our sitemap if possible.
>
> In Forrest, 0 and 1 are both handled in forrest.xmap, which defines the
> VFS on which everything else builds.
Yes, as I'm trying to say, I have understood that making a new design
for this is not needed as proper layering of sitemaps can do it.
So I would propose that we have a sources.xmap and a forrest.xmap.
The sources.xmap would do the mapping that now is done by the
forrest.properties.
> Guess we should take this over to
> forrest-dev..
Done :-)
--
Nicola Ken Barozzi nicolaken@apache.org
- verba volant, scripta manent -
(discussions get forgotten, just code remains)
---------------------------------------------------------------------
Re: [RT] Link Rewriting
Posted by Nicola Ken Barozzi <ni...@apache.org>.
Jeff Turner wrote, On 29/04/2003 14.19:
> On Tue, Apr 29, 2003 at 12:09:22PM +0200, Nicola Ken Barozzi wrote:
>
>>Jeff Turner wrote, On 29/04/2003 8.50:
>>...
...
>>It's about SoC. One part defines the source mapping, the other how to
>>process it. I think that he is talking about a scenario where the user
>>does not add new types, but can easily configure just the new places
>>where to find the sources.
>
>
> Yes, which can be achieved nicely with a sitemap. For instance, we could
> map the whole Wiki source into a Cocoon site with one matcher:
>
> <map:match pattern="wiki/*.xml">
> <map:generate type="txt2xml" src="http://wiki.cocoondev.org/pages/{1}.txt"/>
> <map:transform type="lexer" src="resources/grammars/wiki.xlex"/>
> <map:transform type="parser" src="resources/grammars/wiki.xgrm"/>
> <map:transform src="resources/stylesheets/wiki2document.xsl">
> <map:serialize type="xml"/>
> </map:match>
>
> (for people not on forrest-dev, there's a demo of Cocoon Wiki integration
> at http://cvs.apache.org/~jefft/forrest/samples/wikirenderer-site/)
Ok, so you give away the sitemap, and the user wants to use it to show
his wiki. So he goes in the sitemap and changes
<map:generate type="txt2xml"
src="http://wiki.cocoondev.org/pages/{1}.txt"/>
to
<map:generate type="txt2xml"
src="http://mywiki.org/pages/{1}.txt"/>
Then we come out with a new version of the sitemap, and the user has to
again manually change the new sitemap...
It's the same thing I have discussed with the news section, etc. If you
want to use sitemaps as reusable blocks, you have to separate the source
resoilving and the processing blocks. The above does both.
>>I had committed some time back a refactoring of the sitemap that did
>>just that, but you strongly asked to revert it, so... ;-P
>
> <innocent> Me? </innocent> I don't remember it.. was this the
> resource-exists-athon patch? Anyway..
Yes, it was that ;-P I centralized obtaining the source in a single
place ;-)
>>So, as I think you also would agree, the important thing is that one
>>modularizes the sitemap so that some pieces can remain as-is, and other
>>parts can be plugged in/expanded separately in each layer:
>>
>> 0 - source resolving (for us it's fixed on the filesystem)
>> 1 - conversion to common source
>> 2 - processing
>> 3 - presentation (output)
>>
>>What I'd like now is to add level 0 for our sitemap if possible.
>
> In Forrest, 0 and 1 are both handled in forrest.xmap, which defines the
> VFS on which everything else builds.
Yes, as I'm trying to say, I have understood that making a new design
for this is not needed as proper layering of sitemaps can do it.
So I would propose that we have a sources.xmap and a forrest.xmap.
The sources.xmap would do the mapping that now is done by the
forrest.properties.
> Guess we should take this over to
> forrest-dev..
Done :-)
--
Nicola Ken Barozzi nicolaken@apache.org
- verba volant, scripta manent -
(discussions get forgotten, just code remains)
---------------------------------------------------------------------
Re: [RT] Link Rewriting
Posted by Jeff Turner <je...@apache.org>.
On Tue, Apr 29, 2003 at 12:09:22PM +0200, Nicola Ken Barozzi wrote:
>
> Jeff Turner wrote, On 29/04/2003 8.50:
> ...
> >Now let me play devils advocate..
> >
> ...
> >What's
> >the difference between editing a sitemap.xmap, and editing a linkmap.xml
> >file? Both just map declarators to SystemIds.
> >
> >In CVS Forrest, we've organized the sitemap into functional layers:
> >
> >LAYER 1 | (each format or subdir handler in its own sub-sitemap)
> >*.xml |
> > various | docv11 faq howto docbook community/* ....
> > xml types | \ | | | /
> >-------------------------------------------------------------------------
> > DOCUMENT-V11 INTERMEDIATE FORMAT
> >-------------------------------------------------------------------------
> >LAYER 2 | / | \
> > Intermediate | **body-*.xml **menu-*.xml **tab-*.xml
> > HTML formats | \ | /
> >-------------------------------------------------------------------------
> >LAYER 3 | \|/ \|/
> > Output | *.html *.pdf
> > formats |
> >-------------------------------------------------------------------------
> >
> ...
> >In effect, this defines a virtual filesystem of XML sources, completely
> >abstracted from the real filesystem.
>
> Exactly.
>
> ...
> >So what I'm suggesting is that if your users are sufficiently
> >sitemap-savvy, and your sitemap is sufficiently modular, one can have all
> >the benefits of your proposed declarator -> XML linkmapping system. In
> >fact it's much more powerful; how could a linkmap file handle a RSS file,
> >for example? You'd need to tweak the sitemap to do the rss2document.xsl
> >conversion.
>
> It's about SoC. One part defines the source mapping, the other how to
> process it. I think that he is talking about a scenario where the user
> does not add new types, but can easily configure just the new places
> where to find the sources.
Yes, which can be achieved nicely with a sitemap. For instance, we could
map the whole Wiki source into a Cocoon site with one matcher:
<map:match pattern="wiki/*.xml">
<map:generate type="txt2xml" src="http://wiki.cocoondev.org/pages/{1}.txt"/>
<map:transform type="lexer" src="resources/grammars/wiki.xlex"/>
<map:transform type="parser" src="resources/grammars/wiki.xgrm"/>
<map:transform src="resources/stylesheets/wiki2document.xsl">
<map:serialize type="xml"/>
</map:match>
(for people not on forrest-dev, there's a demo of Cocoon Wiki integration
at http://cvs.apache.org/~jefft/forrest/samples/wikirenderer-site/)
> I had committed some time back a refactoring of the sitemap that did
> just that, but you strongly asked to revert it, so... ;-P
<innocent> Me? </innocent> I don't remember it.. was this the
resource-exists-athon patch? Anyway..
>
> So, as I think you also would agree, the important thing is that one
> modularizes the sitemap so that some pieces can remain as-is, and other
> parts can be plugged in/expanded separately in each layer:
>
> 0 - source resolving (for us it's fixed on the filesystem)
> 1 - conversion to common source
> 2 - processing
> 3 - presentation (output)
>
> What I'd like now is to add level 0 for our sitemap if possible.
In Forrest, 0 and 1 are both handled in forrest.xmap, which defines the
VFS on which everything else builds. Guess we should take this over to
forrest-dev..
--Jeff
> --
> Nicola Ken Barozzi nicolaken@apache.org
> - verba volant, scripta manent -
> (discussions get forgotten, just code remains)
> ---------------------------------------------------------------------
>
Re: [RT] Link Rewriting
Posted by Nicola Ken Barozzi <ni...@apache.org>.
Jeff Turner wrote, On 29/04/2003 8.50:
...
> Now let me play devils advocate..
>
...
> What's
> the difference between editing a sitemap.xmap, and editing a linkmap.xml
> file? Both just map declarators to SystemIds.
>
> In CVS Forrest, we've organized the sitemap into functional layers:
>
> LAYER 1 | (each format or subdir handler in its own sub-sitemap)
> *.xml |
> various | docv11 faq howto docbook community/* ....
> xml types | \ | | | /
> -------------------------------------------------------------------------
> DOCUMENT-V11 INTERMEDIATE FORMAT
> -------------------------------------------------------------------------
> LAYER 2 | / | \
> Intermediate | **body-*.xml **menu-*.xml **tab-*.xml
> HTML formats | \ | /
> -------------------------------------------------------------------------
> LAYER 3 | \|/ \|/
> Output | *.html *.pdf
> formats |
> -------------------------------------------------------------------------
>
...
> In effect, this defines a virtual filesystem of XML sources, completely
> abstracted from the real filesystem.
Exactly.
...
> So what I'm suggesting is that if your users are sufficiently
> sitemap-savvy, and your sitemap is sufficiently modular, one can have all
> the benefits of your proposed declarator -> XML linkmapping system. In
> fact it's much more powerful; how could a linkmap file handle a RSS file,
> for example? You'd need to tweak the sitemap to do the rss2document.xsl
> conversion.
It's about SoC. One part defines the source mapping, the other how to
process it. I think that he is talking about a scenario where the user
does not add new types, but can easily configure just the new places
where to find the sources.
I had committed some time back a refactoring of the sitemap that did
just that, but you strongly asked to revert it, so... ;-P
So, as I think you also would agree, the important thing is that one
modularizes the sitemap so that some pieces can remain as-is, and other
parts can be plugged in/expanded separately in each layer:
0 - source resolving (for us it's fixed on the filesystem)
1 - conversion to common source
2 - processing
3 - presentation (output)
What I'd like now is to add level 0 for our sitemap if possible.
--
Nicola Ken Barozzi nicolaken@apache.org
- verba volant, scripta manent -
(discussions get forgotten, just code remains)
---------------------------------------------------------------------
Re: [RT] Link Rewriting
Posted by Jeremy Quinn <je...@media.demon.co.uk>.
On Tuesday, April 29, 2003, at 07:50 AM, Jeff Turner wrote:
> On Mon, Apr 28, 2003 at 03:16:34PM +0100, Jeremy Quinn wrote:
>> Hi All
>>
>> Welcome to my first RT!
>>
>> Sorry to start on a negative note, but IMHO, the current scheme for
>> LinkRewriting just does not make sense for dynamic sites.
>>
>> What the current scheme allows is for authors to use a simple set of
>> declarators for defining links, which are then translated into actual
>> URLs, (and in the samples) mapped directly to filespace (etc.).
>
> Yes. <link href="site:foo">, where 'foo' is your declarator, and the
> link is translated into the actual URL <a href="foo.html">.
>
> So LinkRewriterTransformer does the mapping:
>
> declarator -> *.html, *.pdf etc (URL)
>
> However, that was only half of the original Linkmap vision :) I think
> you're asking for the other half, which is declarator -> SystemID
> (*.xml).
Ah Ha!
> See the original linkmap RT:
>
> http://marc.theaimsgroup.com/?l=forrest-dev&m=103444028129281&w=2
OK, this is interesting!
> The declarator -> SystemID mapping in that RT was declared in a
> site.xml
> file:
>
> <site dir="./content/xdocs">
> <index file="index.xml"/>
> <dreams file="dreams.xml"/>
> <faq file="faq.xml">
> <how_can_I_help xpath="/faqs/faq/question[@id='how_can_I_help']">
> <building_own_website
> xpath="/faqs/faq/question[@id='own_website']"/>
> ....
> </site>
>
> That is functionally equivalent to the XML in your post quoted below.
Yes, I think so.
> In the linkmap RT, I envisioned a Source that accepts a 'declarator'
> as a
> @src, and emits XML from that declarator's SystemID. Your idea of
> using
> input modules with dynamic inputs, {site:{1}}, is a nicer
> implementation
> than what I had in mind.
>
>> What I believe a LinkRewriting infrastructure should offer is rather
>> different.
>>
>> A URL is a CONTRACT.
>>
>> Cocoon's sitemap is excellent at disconnecting any direct relationship
>> between URL and resource ID/Path (SystemID). Allowing the
>> re-implementation of storage schema independently of URL contract.
>>
>> What I feel makes more sense is this:
>>
>> URL == permanent contract == authoring link != SystemID
>>
>> What this means is, authors should write links using a static absolute
>> URL, the same one as the public contract for that particular piece of
>> information.
>>
>> When that URL is accessed, it should be mapped to a SystemID, allowing
>> independent re-implementation of the storage layer.
>
> So we have URLs everywhere, and in the sitemap we do a lookup on a
> linkmap to determine the actual source? This kind of thing:
>
> <map:match pattern="*.html">
> <map:generate src="content/xdocs/{linkmap:{1}}"/>
> ...
> </map:match>
Yes, using nested input modules to modify the XPath expression to
handle the different LinkMap formats people would want. For instance, I
prefer to use some thing close to XLink.
>
>> If this is to be handled by an input module accessing a linkmap
>> (before
>> Generation, rather than during Transformation), but requires that
>> dynamic sitemap URL fragments can be passed to input modules (as they
>> cannot currently).
>>
>> see:
>> <http://marc.theaimsgroup.com/?l=xml-cocoon-dev&m=104895916400307>
>>
>> The transformation stage is then merely one of relativising author's
>> URLs.
>
> But that isn't the only use of link rewriting :) The original idea was
> that <link href="site:index"> could be translated into:
>
> - <a href="index.html"> if linked to from a *.html file
> - <a href="index.pdf"> if linked to from a *.pdf file
> - <a href="#index"> if linked to from an aggregate HTML
> - <fo:basic-link internal-destination="index"> if linked to from an
> aggregate PDF
> - ....
>
> This is the value of the descriptor -> URL mapping.
Yes, this is not something I had dealt with yet.
Very nice!
>
> Actually, Forrest doesn't yet support this neat stuff yet, but that was
> the original idea and it will be implemented as soon as enough users
> complain about broken 'site:' links in their PDFs :o)
>
> Anyway, point is, there's two sites to 'linkmapping', and both are
> valuable.
>
>> Most of what I have described here, is how most people use Cocoon.
>> What is new here is the use of a LinkMap at the Generation stage to
>> de-couple URL from SystemID in a totally arbitrary way. A version of
>> the LinkMap idea that makes sense for Dynamic sites. This is what
>> requires changes to the way we are able to use input modules, as input
>> modules would provide a much cleaner path to handle this rather than
>> Transforming a generated LinkMap into CInclude tags to get the
>> content.
>>
>>
>> Does this make any sense to anybody?
>
> Perfectly :)
>
> Now let me play devils advocate..
>
> If we assume the user knows how to edit the sitemap, is there any real
> benefit in this extra level of declarator -> *.xml indirection? What's
> the difference between editing a sitemap.xmap, and editing a
> linkmap.xml
> file? Both just map declarators to SystemIds.
I think Nicola Ken said it better than I could in his previous reply
....
Many thanks for the feedback
regards Jeremy
Re: [RT] Link Rewriting
Posted by Jeff Turner <je...@apache.org>.
On Mon, Apr 28, 2003 at 03:16:34PM +0100, Jeremy Quinn wrote:
> Hi All
>
> Welcome to my first RT!
>
> Sorry to start on a negative note, but IMHO, the current scheme for
> LinkRewriting just does not make sense for dynamic sites.
>
> What the current scheme allows is for authors to use a simple set of
> declarators for defining links, which are then translated into actual
> URLs, (and in the samples) mapped directly to filespace (etc.).
Yes. <link href="site:foo">, where 'foo' is your declarator, and the
link is translated into the actual URL <a href="foo.html">.
So LinkRewriterTransformer does the mapping:
declarator -> *.html, *.pdf etc (URL)
However, that was only half of the original Linkmap vision :) I think
you're asking for the other half, which is declarator -> SystemID
(*.xml).
See the original linkmap RT:
http://marc.theaimsgroup.com/?l=forrest-dev&m=103444028129281&w=2
The declarator -> SystemID mapping in that RT was declared in a site.xml
file:
<site dir="./content/xdocs">
<index file="index.xml"/>
<dreams file="dreams.xml"/>
<faq file="faq.xml">
<how_can_I_help xpath="/faqs/faq/question[@id='how_can_I_help']">
<building_own_website xpath="/faqs/faq/question[@id='own_website']"/>
....
</site>
That is functionally equivalent to the XML in your post quoted below.
In the linkmap RT, I envisioned a Source that accepts a 'declarator' as a
@src, and emits XML from that declarator's SystemID. Your idea of using
input modules with dynamic inputs, {site:{1}}, is a nicer implementation
than what I had in mind.
> What I believe a LinkRewriting infrastructure should offer is rather
> different.
>
> A URL is a CONTRACT.
>
> Cocoon's sitemap is excellent at disconnecting any direct relationship
> between URL and resource ID/Path (SystemID). Allowing the
> re-implementation of storage schema independently of URL contract.
>
> What I feel makes more sense is this:
>
> URL == permanent contract == authoring link != SystemID
>
> What this means is, authors should write links using a static absolute
> URL, the same one as the public contract for that particular piece of
> information.
>
> When that URL is accessed, it should be mapped to a SystemID, allowing
> independent re-implementation of the storage layer.
So we have URLs everywhere, and in the sitemap we do a lookup on a
linkmap to determine the actual source? This kind of thing:
<map:match pattern="*.html">
<map:generate src="content/xdocs/{linkmap:{1}}"/>
...
</map:match>
> If this is to be handled by an input module accessing a linkmap (before
> Generation, rather than during Transformation), but requires that
> dynamic sitemap URL fragments can be passed to input modules (as they
> cannot currently).
>
> see: <http://marc.theaimsgroup.com/?l=xml-cocoon-dev&m=104895916400307>
>
> The transformation stage is then merely one of relativising author's
> URLs.
But that isn't the only use of link rewriting :) The original idea was
that <link href="site:index"> could be translated into:
- <a href="index.html"> if linked to from a *.html file
- <a href="index.pdf"> if linked to from a *.pdf file
- <a href="#index"> if linked to from an aggregate HTML
- <fo:basic-link internal-destination="index"> if linked to from an
aggregate PDF
- ....
This is the value of the descriptor -> URL mapping.
Actually, Forrest doesn't yet support this neat stuff yet, but that was
the original idea and it will be implemented as soon as enough users
complain about broken 'site:' links in their PDFs :o)
Anyway, point is, there's two sites to 'linkmapping', and both are
valuable.
> Most of what I have described here, is how most people use Cocoon.
> What is new here is the use of a LinkMap at the Generation stage to
> de-couple URL from SystemID in a totally arbitrary way. A version of
> the LinkMap idea that makes sense for Dynamic sites. This is what
> requires changes to the way we are able to use input modules, as input
> modules would provide a much cleaner path to handle this rather than
> Transforming a generated LinkMap into CInclude tags to get the content.
>
>
> Does this make any sense to anybody?
Perfectly :)
Now let me play devils advocate..
If we assume the user knows how to edit the sitemap, is there any real
benefit in this extra level of declarator -> *.xml indirection? What's
the difference between editing a sitemap.xmap, and editing a linkmap.xml
file? Both just map declarators to SystemIds.
In CVS Forrest, we've organized the sitemap into functional layers:
LAYER 1 | (each format or subdir handler in its own sub-sitemap)
*.xml |
various | docv11 faq howto docbook community/* ....
xml types | \ | | | /
-------------------------------------------------------------------------
DOCUMENT-V11 INTERMEDIATE FORMAT
-------------------------------------------------------------------------
LAYER 2 | / | \
Intermediate | **body-*.xml **menu-*.xml **tab-*.xml
HTML formats | \ | /
-------------------------------------------------------------------------
LAYER 3 | \|/ \|/
Output | *.html *.pdf
formats |
-------------------------------------------------------------------------
The lowest layer (in forrest.xmap) is a bunch of *.xml matchers, which
provides the raw doc-v11 XML source. For example, the faq.xml matcher
reads content/xdocs/faq.xml, transforms it with faq2document.xsl and
serves the content.
In effect, this defines a virtual filesystem of XML sources, completely
abstracted from the real filesystem. For instance, to integrate RSS into
a Forrest site, one need define a single *.xml matcher that reads RSS
from somewhere and does a rss2document.xsl transformation. The upper
layers don't care where they get their XML from, so you magically get a
HTML and PDF from the RSS:
http://xml.apache.org/forrest/forrest-issues.html
So what I'm suggesting is that if your users are sufficiently
sitemap-savvy, and your sitemap is sufficiently modular, one can have all
the benefits of your proposed declarator -> XML linkmapping system. In
fact it's much more powerful; how could a linkmap file handle a RSS file,
for example? You'd need to tweak the sitemap to do the rss2document.xsl
conversion.
--Jeff
>
> regards Jeremy
>
Re: [RT] Link Rewriting
Posted by Stefano Mazzocchi <st...@apache.org>.
on 4/29/03 2:25 PM Jeremy Quinn wrote:
> On Tuesday, April 29, 2003, at 11:02 AM, Nicola Ken Barozzi wrote:
>
>
>>Stefano Mazzocchi wrote, On 28/04/2003 22.06:
>>
>>>on 4/28/03 9:16 AM Jeremy Quinn wrote:
>>
>>...
>>
>>>>When that URL is accessed, it should be mapped to a SystemID,
>>>>allowing independent re-implementation of the storage layer.
>>>
>>>I don't see this. doesn't the sitemap give you enough decoupling
>>>capabilities already?
>>
>>How it's generally used, it simply matches urls to sources. This means
>>that if I need to change all sources to a different store, or part of
>>it, I have this "aspect" intermixed in all matches.
>
>
> Spot on.
Look at what I do in linotype: one pipeline generates the xml file,
another one does the transformation and they are glued using cocoon:
protocol.
That allows concerns to be separate and the sitemap to be minimized from
a cut/paste point of view.
--
Stefano.
Re: [RT] Link Rewriting
Posted by Jeremy Quinn <je...@media.demon.co.uk>.
On Tuesday, April 29, 2003, at 11:02 AM, Nicola Ken Barozzi wrote:
>
> Stefano Mazzocchi wrote, On 28/04/2003 22.06:
>> on 4/28/03 9:16 AM Jeremy Quinn wrote:
> ...
>>> When that URL is accessed, it should be mapped to a SystemID,
>>> allowing independent re-implementation of the storage layer.
>> I don't see this. doesn't the sitemap give you enough decoupling
>> capabilities already?
>
> How it's generally used, it simply matches urls to sources. This means
> that if I need to change all sources to a different store, or part of
> it, I have this "aspect" intermixed in all matches.
Spot on.
On Tuesday, April 29, 2003, at 11:09 AM, Nicola Ken Barozzi wrote:
>
> Jeff Turner wrote, On 29/04/2003 8.50:
[snip]
> ...
>> So what I'm suggesting is that if your users are sufficiently
>> sitemap-savvy, and your sitemap is sufficiently modular, one can have
>> all
>> the benefits of your proposed declarator -> XML linkmapping system.
>> In
>> fact it's much more powerful; how could a linkmap file handle a RSS
>> file,
>> for example? You'd need to tweak the sitemap to do the
>> rss2document.xsl
>> conversion.
>
> It's about SoC. One part defines the source mapping, the other how to
> process it. I think that he is talking about a scenario where the user
> does not add new types, but can easily configure just the new places
> where to find the sources.
"Right again Robin" - Batman
[snip]
When the burden of mapping becomes to great to maintain in a sitemap
(or set of sitemaps) techniques of (pre-generation) URL mapping become
really useful.
Thanks for all the feedback !!
regards Jeremy
------------------------------------------------------------------------
"Objective reality is a synthetic construct, dealing with a
hypothetical universalization of a multitude of subjective realities."
Philip K Dick - "The Electric Ant"
Re: [RT] Link Rewriting
Posted by Nicola Ken Barozzi <ni...@apache.org>.
Stefano Mazzocchi wrote, On 28/04/2003 22.06:
> on 4/28/03 9:16 AM Jeremy Quinn wrote:
...
>>When that URL is accessed, it should be mapped to a SystemID, allowing
>>independent re-implementation of the storage layer.
>
> I don't see this. doesn't the sitemap give you enough decoupling
> capabilities already?
How it's generally used, it simply matches urls to sources. This means
that if I need to change all sources to a different store, or part of
it, I have this "aspect" intermixed in all matches.
The separation of this aspect can be done by making a pipeline do the
mapping, and having the generator ask for a cocoon://get/source/for/xxx
url that gets you the data using that pipeline.
IIUC he is asking that this mechanism be also available via modules as a
Cocoon 'plugin', and factored out of Cocoon. This is what we need for
Forrest, where the pipeline is "reusable", and should not be touched.
Probably mounting this pipeline would solve the problem, but I don't
know how it scales as performance.
--
Nicola Ken Barozzi nicolaken@apache.org
- verba volant, scripta manent -
(discussions get forgotten, just code remains)
---------------------------------------------------------------------
Re: [RT] Link Rewriting
Posted by Jeremy Quinn <je...@media.demon.co.uk>.
On Monday, April 28, 2003, at 09:06 PM, Stefano Mazzocchi wrote:
> on 4/28/03 9:16 AM Jeremy Quinn wrote:
>
>> Hi All
>>
>> Welcome to my first RT!
>
> Happy to see this. :-)
It has got an interesting set of responses too :)
>
>> Sorry to start on a negative note, but IMHO, the current scheme for
>> LinkRewriting just does not make sense for dynamic sites.
>
> Sorry, but what is the current scheme for link rewriting? I'm probably
> missing something.
see: http://localhost:8888/samples/linkrewriter/welcome
>> What the current scheme allows is for authors to use a simple set of
>> declarators for defining links, which are then translated into actual
>> URLs, (and in the samples) mapped directly to filespace (etc.).
>
> Uh? what do you mean?
Since the current incarnation of LinkreWritingTransformer came from the
needs of Forrest (I believe), and Forrest is designed to output a
static site, the mapping is for the convenience of Authors .... writing
an easier-to-remember link, that gets mapped to a published URL.
When you apply LinkRewriting to a dynamic situation, the mapping you
want to do changes in orientation.
>
>> What I believe a LinkRewriting infrastructure should offer is rather
>> different.
>>
>> A URL is a CONTRACT.
>>
>> Cocoon's sitemap is excellent at disconnecting any direct relationship
>> between URL and resource ID/Path (SystemID). Allowing the
>> re-implementation of storage schema independently of URL contract.
>>
>> What I feel makes more sense is this:
>>
>> URL == permanent contract == authoring link != SystemID
>
> Wait a second.
>
> URI -> uniform resource identifiers = permanent contract that
> identifies uniquely the resource in a uniform manner.
>
> URL -> uniform resource locator = address used to locate the resource.
>
> Cocoon makes it easy to allow your URIs to be URLs.
Yeah, sorry, I never really grokked the difference between them ;)
>
> The problem with the above is that URLs and URIs are easy to
> discriminate if they are absolute. Not so if they are relative.
>
>> What this means is, authors should write links using a static absolute
>> URL, the same one as the public contract for that particular piece of
>> information.
>
> Yes, but this makes relocability hard.
Umm, not so I have noticed ....
>
>> When that URL is accessed, it should be mapped to a SystemID, allowing
>> independent re-implementation of the storage layer.
>
> I don't see this. doesn't the sitemap give you enough decoupling
> capabilities already?
It provides excellent facilities, but depends on how different they are.
In most cases, yes .... I managed to do the whole inIVA site in a
pretty simple SiteMap, with various Sources and a legacy URL space to
contend with.
The KISS site (kiss-ccr.dyndns.org) however is completely mapped by a
LinkMap, because I wanted to allow SystemIDs to be completely different
from URLs.
Why? One reason was because I often want the same info in different
contexts.
This means there are 2 different URLs within different hierarchal
relationships, but coming from the same source, without special
handling in the SiteMap.
The KISS LinkMap:
is hierarchal
- it provides the basis of the site menu child/parent relationships
maps URIs to SystemIds
- disassociates URL from Source
is an Extended XLink with:
Locators - both internal and external
Arcs - aliases referring to another Locator
Resources - names and static links
>> Does this make any sense to anybody?
>
> Yes and no.
>
> can you make an explicit example?
Lets say you wanted to serve the Source of different Author's documents
from their own folders, with file names of their choosing, while retain
control over a unified URL Space, without lots of rules embedded in the
Sitemap.
<linkmap>
<loc xl:label="home" xl:href="users/stef/domus-03.xml"
xl:title="Stefano looks after the home page">
<name>Home Page</name>
<loc xl:label="about" xl:href="users/jerm/About the Site v2.xml"
xl:title="Jeremy looks after the about page">
<name>About the Site</name>
<loc xl:label="who"
xl:href="users/nico/informazioni_sulla_squadra.xml"
xl:title="Nicola Ken writes about the team">
<name>Who writes the Site</name>
<loc xl:label="about/stefano" xl:href="users/nico/stefano.xml"
xl:title="Nicola Ken writes about Stefano">
<name>About Stefano</name>
</loc>
<loc xl:label="about/nicola" xl:href="users/nico/sono.xml"
xl:title="Nicola Ken on himself">
<name>About Stefano</name>
</loc>
<loc xl:label="about/jeremy" xl:href="http://jerm.dyndns.org/cv.xml"
xl:title="Jeremy is elsewhere">
<name>About Jeremy</name>
</loc>
</loc>
<loc xl:label="how" xl:href="users/stef/cocoon-2.1dev-06.xml"
xl:title="Stefano writes about Cocoon">
<name>How does the Site work</name>
<ref xl:href="http://cocoon.apache.org"
xl:title="A static child link in the menu">
<name>Cocoon Home</name>
</ref>
</loc>
</loc>
<arc xl:label="home/auth" xl:to="about/stefano"/>
</loc>
</linkmap>
This is similar to what I do on the KISS site.
I hope it's a bit more clear.
This type of mapping is useful when the URL space, Site hierarchy and
Source space are not in any way similar.
regards Jeremy
------------------------------------------------------------------------
"Objective reality is a synthetic construct, dealing with a
hypothetical universalization of a multitude of subjective realities."
Philip K Dick - "The Electric Ant"
Re: [RT] Link Rewriting
Posted by Stefano Mazzocchi <st...@apache.org>.
on 4/28/03 9:16 AM Jeremy Quinn wrote:
> Hi All
>
> Welcome to my first RT!
Happy to see this. :-)
> Sorry to start on a negative note, but IMHO, the current scheme for
> LinkRewriting just does not make sense for dynamic sites.
Sorry, but what is the current scheme for link rewriting? I'm probably
missing something.
> What the current scheme allows is for authors to use a simple set of
> declarators for defining links, which are then translated into actual
> URLs, (and in the samples) mapped directly to filespace (etc.).
Uh? what do you mean?
> What I believe a LinkRewriting infrastructure should offer is rather
> different.
>
> A URL is a CONTRACT.
>
> Cocoon's sitemap is excellent at disconnecting any direct relationship
> between URL and resource ID/Path (SystemID). Allowing the
> re-implementation of storage schema independently of URL contract.
>
> What I feel makes more sense is this:
>
> URL == permanent contract == authoring link != SystemID
Wait a second.
URI -> uniform resource identifiers = permanent contract that
identifies uniquely the resource in a uniform manner.
URL -> uniform resource locator = address used to locate the resource.
Cocoon makes it easy to allow your URIs to be URLs.
The problem with the above is that URLs and URIs are easy to
discriminate if they are absolute. Not so if they are relative.
> What this means is, authors should write links using a static absolute
> URL, the same one as the public contract for that particular piece of
> information.
Yes, but this makes relocability hard.
> When that URL is accessed, it should be mapped to a SystemID, allowing
> independent re-implementation of the storage layer.
I don't see this. doesn't the sitemap give you enough decoupling
capabilities already?
> If this is to be handled by an input module accessing a linkmap (before
> Generation, rather than during Transformation), but requires that
> dynamic sitemap URL fragments can be passed to input modules (as they
> cannot currently).
>
> see: <http://marc.theaimsgroup.com/?l=xml-cocoon-dev&m=104895916400307>
>
> The transformation stage is then merely one of relativising author's
> URLs.
>
> Most of what I have described here, is how most people use Cocoon.
> What is new here is the use of a LinkMap at the Generation stage to
> de-couple URL from SystemID in a totally arbitrary way. A version of
> the LinkMap idea that makes sense for Dynamic sites. This is what
> requires changes to the way we are able to use input modules, as input
> modules would provide a much cleaner path to handle this rather than
> Transforming a generated LinkMap into CInclude tags to get the content.
>
>
> Does this make any sense to anybody?
Yes and no.
can you make an explicit example?
--
Stefano.
Re: [RT] Link Rewriting
Posted by Nicola Ken Barozzi <ni...@apache.org>.
Jeremy Quinn wrote, On 28/04/2003 16.16:
...
> Most of what I have described here, is how most people use Cocoon.
> What is new here is the use of a LinkMap at the Generation stage to
> de-couple URL from SystemID in a totally arbitrary way. A version of the
> LinkMap idea that makes sense for Dynamic sites. This is what requires
> changes to the way we are able to use input modules, as input modules
> would provide a much cleaner path to handle this rather than
> Transforming a generated LinkMap into CInclude tags to get the content.
>
> Does this make any sense to anybody?
Sure it does, although IMHO it's more suited to forrest-dev than
cocoon-dev ATM.
What you describe here is what I calla resourcemap, that is a descriptor
that creates a mapping between a conceptual source space and the actual
source locations.
If you search the forrest-dev mailing list for resourcemap, you should
find plenty of RTs, even quite recent.
A brief explanation:
There are different URI spaces:
1 - the actual sources
2 - a conceptual aggregated source space
3 - the actual URIs
4 - a conceptual URI space
The current link rewriting is used to make users be able to use 4 and
obtain 3. What you (and I) feel the need of too, is to have 2 map to 1,
so that I can easily swap content sources and keep the sitemap as-is.
--
Nicola Ken Barozzi nicolaken@apache.org
- verba volant, scripta manent -
(discussions get forgotten, just code remains)
---------------------------------------------------------------------