Auto generating site.xml

[Ross Gardler <rg...@apache.org>]

Ferdinand Soethe wrote:

[moved to the Forrest dev list since this has now turned to issues 
regarding the development of Forrest -

Background for Forrest devs: this mail is part of a thread between 
myself and Ferdinand discussing our ApacheeCon presentation and features 
we may implement to illustrate some of our points]

> - I extend the OOplugin to convert that glossary markup into a link
>   (<a href="">...</a>) where the URL follows the rules you tell me are
>   required for your glossary plugin.
> 
> - you make the glossary-plugin work in Forrest and Burrokeet.
> 
> we are done :-)

+1

Time allowing I will see if I can make the Daisy editor and/or the 
htmlArea editor work like this too.

> Say I have a dir structure like
> 
>         xdocs/
>            files ...
>            about/
>            work/
>            demo/
>              mywiki/
>                index.wiki
>                install.wiki
>                wikidirs/
>                  howtoUseDirs.wiki
>                  sortingDirs.wiki
>                wikisyntax/
>                  howtoWriteAWikiFile.wiki
>                  
>                
>  and I only wanted to apply the automatic generation of
>  site entries to the mywiki/ subdir,
> 
>  - could we have a plugin that will
>    parse the dir structure from mywiki/ downwards?
>    Yes, I know we can with cocoon directory transformer
> 
>  - can we generate site structure from the dir structure and the
>    filenames?
>    Yes
> 
>  - How can we make the plugin insert the resulting site structure into
>    the manually edited structure in site.xml.
> 
>    Could I for example have a special uri so that a site element like
> 
>         <mywiki label="My Wiki"
>         href="automatedDirectory/demo/mywiki"/>
> 
>    = would automatically generate the site struture?
>      Yes as this is similar to S5 using a prefix to the url to trigger
>      a pipeline!

[update for devs: this would be an internal plugin we would build to 
illustrate the extension of core internal functionality]

No, this is not the same as the s5 plugin. In the s5 plugin it is the 
request that triggers the processing by the s5 plugin. In this case 
there is no request generated when the site.xml file is processed.

[OT: The s5 plugin will be changing the way it does this because it is 
bad design to have the URL indicate the format of the file being served. 
Instead of 's5slides/presentation.html' it will be 'presentation.s5' or 
presentation.s5.html]

I see two ways of doing what you want.

We could change the XSL that builds the site.xml file so that it looks 
for something like:

<myWiki label="MyWiki" type="auto"/>

This would then make a request for something like:

demo/mywiki/site.auto.xml

which would then provide the site.xml snippet.

Alternatively (and more simply), we could do:

<xi:include src="demo/mywiki/site.auto.xml"/>

>    = would insert the inserted structure in place of the
>      <mywiki>-element into the final sitemap?
>      That should be possible but may be I'm too optmistic here.

I think the above solutions will do the job just fine.

> My reasons behind this proposal:
> 
> You and I talked about the usefulness of such a plugin in general.
> Our main problem was that parsing the complete dir structure of a big
> site might take to long. And - may I add - perhaps people would want
> to use anvanced techniques for some of the site but allow for easy
> maintenance for others.
> 
> By being able to have it process only part of the tree, we solve both
> problems.
> 
> Let me know if this is too much work, otherwise I'll add some more
> implementation details.

It look pretty easy on the surface, only experimentation (or a keen eye 
from one of the other devs) can now show otherwise. It's on my list.

Ross

Re: [RT] businessHelper (was Re: Auto generating site.xml)

[Ross Gardler <rg...@apache.org>]

Thorsten Scherler wrote:
> Actually it sounds that parts of the businessHelper-plugin can be found
> in the auto generating site.xml proposal. 
> 
> The goal of the businessHelper-concept is to offer a framework to access
> different businessServices in a generic way. The businessHelper would be
> responsible to connect to a businessService (e.g. a wiki) and deliver
> the presentation model.

Isn't that the job of an input plugin? Like we did with the RSS feed to 
be included within another contract...

<forrest:contract name="feeder">
    <forrest:properties contract="feeder">
      <forrest:property name="feeder" nugget="get.nugget.feeder">
        <url>/feeds/somefeed.xml</url>
      </forrest:property>
    </forrest:properties>
</forrest:contract>

What is it that makes a businessHelper different from a contract that 
gets its content from a plugin in this way?

> I did some experiments with the inx-plugin I am developing. It is pretty
> generic so far, but still I need to capsule more the customer specific
> business logic, discuss the patent issue on legal@a.o and using more
> flexible triggers to release it to forrest. ATM the inx plugin contains
> both view/businessHelper what means that in general for each format a
> specific businessHelper is expected (like viewHelper.{format}) 

I'm sorry, I've read and re-read the rest of this thread many times. But 
I just don't get it. I suspect it is because you are referring to 
something I can't see and to tools I am not familiar with. I'll 
highlight why I am so lost in my comments, perhaps, when you have time, 
you can re-write this with more background and a use case appropriate to 
Forrest.

> raw.user.data - custom doctype that is coming from a previous stage that
> contains the user data that have to be transformed into indesign.
> Actually this is coming from an editor which defines the dtd to be used.

So this is the raw data from the src, say an RSS feed.

> data.model - this is the result of the transformation (rud2model.xsl) of
> r.u.d into something more generic regarding the containing models (the
> actual data).  A small snippet may show the format:

Sorry, no idea what this means. You are transforming the raw data into 
what? I'm afraid the XML snippet provided sheds no light on it for me 
since there is no explanation of what the tags and attributes are 
intended to mean.

> data.view - this is the result of the transformation (rud2view.xsl) of
> r.u.d into something more generic regarding the presentation of the
> data. A small snippet may show the format (which basicly is a generated
> forrest:view):

Nope, still lost I'm afraid. The XML snippet here shows a set of hooks 
and contracts, I get that part. It also seems to refer to some of the 
stuff in the previous section, but since I don't get that either...

> With the view and the model I then build the inx output. This is done by
> a generic model+view2inx.xsl. This stylesheets connects to the inx
> specific helper classes (here I still have some customer business logic)
> to do the transformation. 

Nope, since I don't know what 	 is this doesn't help either I'm afraid.

> Like you see I do not using the xdocs intermediate format but a custom
> doctype as input. That means for another doctype one have to write
> newDocType2model(/view).xsl.

So is this good or bad? It sounds bad, but I've not followed this far so 
maybe its good ???

> Now looking again into the original mail I can imagine that you can use
> this concept to achieve what you want. The mayor difference between my
> approach and the one you are working on is that I have the following
> processing:
> {request}.inx-> viewDispatcher -> {request}.fv + {request}.model
> ->{response}.inx

This just looks really complicated to do something that is essentially 
really simple (automate the generation of site.xml based on a filtered 
directory listing). What is wrong with my original proposal to just 
write a plugin that generates the necessary site.xml snippet using the 
existing directory generator?

> ...but with this concept you can actually generate only parts of the
> site.xml. The model would contain the dir structure that you want to
> export as plain data and the view will define the hierarchy. Then you
> only need a final transformation model+view2site.xml.xsl. 

This advantage seems to be exactly what I said in the last paragraph. Am 
I missing something in my confusion?

Ross

[RT] businessHelper (was Re: Auto generating site.xml)

[Thorsten Scherler <th...@apache.org>]

Actually it sounds that parts of the businessHelper-plugin can be found
in the auto generating site.xml proposal. 

The goal of the businessHelper-concept is to offer a framework to access
different businessServices in a generic way. The businessHelper would be
responsible to connect to a businessService (e.g. a wiki) and deliver
the presentation model. It is the counter part of the
viewHelper-concept. Now this concept have to be extend a wee bit for the
generation of the site.xml.

I did some experiments with the inx-plugin I am developing. It is pretty
generic so far, but still I need to capsule more the customer specific
business logic, discuss the patent issue on legal@a.o and using more
flexible triggers to release it to forrest. ATM the inx plugin contains
both view/businessHelper what means that in general for each format a
specific businessHelper is expected (like viewHelper.{format}) 

Anyway I am doing the following:

raw.user.data - custom doctype that is coming from a previous stage that
contains the user data that have to be transformed into indesign.
Actually this is coming from an editor which defines the dtd to be used.

data.model - this is the result of the transformation (rud2model.xsl) of
r.u.d into something more generic regarding the containing models (the
actual data).  A small snippet may show the format:
<?xml version="1.0" encoding="ISO-8859-1"?>
  <forrest:presentationModel 
    xmlns:forrest="http://apache.org/forrest/templates/1.0"
name="MottoTest" 
    required="true" state="new" title="Mailing1" lastPage="8">
    <!--currentPage ->1-->
    <forrest:model name="cover" newId="111" vertical="1" horizontal="1" 
      page="1">
      <!--cover-image*img-->
      <forrest:property newId="111" name="cover-image">
        forrest-slogan.jpg</forrest:property>
      <!--cover-text*txt-->
      <forrest:property newId="111" name="cover-text">Forrest example
for inx-generation </forrest:property>
      <!--cover-logo*img-->
      <forrest:property newId="111" name="cover-logo">
forrest.png</forrest:property>
    </forrest:model>
</forrest:presentationModel>

data.view - this is the result of the transformation (rud2view.xsl) of
r.u.d into something more generic regarding the presentation of the
data. A small snippet may show the format (which basicly is a generated
forrest:view):
<?xml version="1.0" encoding="ISO-8859-1"?>
  <forrest:views
xmlns:forrest="http://apache.org/forrest/templates/1.0" 
    xmlns:logic="http://apache.org/forrest/logic/1.0">
    <forrest:view format="inx">
      <forrest:hook name="sprd" count="1">
        <!--newId param debug -> 0-->
        <forrest:contract type="static" name="background-right"/>
        <forrest:hook name="page" count="1">
          <!--newId param debug -> 0-->
          <forrest:contract name="cover-image" type="img" newId="111"/>
          <!--newId param debug -> 0-->
          <forrest:contract name="cover-text" type="txt" newId="111"/>
          <!--newId param debug -> 0-->
          <forrest:contract name="cover-logo" type="img" newId="111"/>
        </forrest:hook>
      </forrest:hook>
   </forrest:view>
</forrest:views>

With the view and the model I then build the inx output. This is done by
a generic model+view2inx.xsl. This stylesheets connects to the inx
specific helper classes (here I still have some customer business logic)
to do the transformation. 

Like you see I do not using the xdocs intermediate format but a custom
doctype as input. That means for another doctype one have to write
newDocType2model(/view).xsl.

Now looking again into the original mail I can imagine that you can use
this concept to achieve what you want. The mayor difference between my
approach and the one you are working on is that I have the following
processing:
{request}.inx-> viewDispatcher -> {request}.fv + {request}.model
->{response}.inx

I actually bypass the site.xml because I am using this plugin not in
forrest but in lenya where we do not have site.xml and using a
viewDispatcher (basically a custom sitemap). 

...but with this concept you can actually generate only parts of the
site.xml. The model would contain the dir structure that you want to
export as plain data and the view will define the hierarchy. Then you
only need a final transformation model+view2site.xml.xsl. 

WDYT?

salu2

On Fri, 2005-05-20 at 09:51 +0100, Ross Gardler wrote:
> Ferdinand Soethe wrote:
> 
> [moved to the Forrest dev list since this has now turned to issues 
> regarding the development of Forrest -
> 
> Background for Forrest devs: this mail is part of a thread between 
> myself and Ferdinand discussing our ApacheeCon presentation and features 
> we may implement to illustrate some of our points]
> 
> > - I extend the OOplugin to convert that glossary markup into a link
> >   (<a href="">...</a>) where the URL follows the rules you tell me are
> >   required for your glossary plugin.
> > 
> > - you make the glossary-plugin work in Forrest and Burrokeet.
> > 
> > we are done :-)
> 
> +1
> 
> Time allowing I will see if I can make the Daisy editor and/or the 
> htmlArea editor work like this too.
> 
> > Say I have a dir structure like
> > 
> >         xdocs/
> >            files ...
> >            about/
> >            work/
> >            demo/
> >              mywiki/
> >                index.wiki
> >                install.wiki
> >                wikidirs/
> >                  howtoUseDirs.wiki
> >                  sortingDirs.wiki
> >                wikisyntax/
> >                  howtoWriteAWikiFile.wiki
> >                  
> >                
> >  and I only wanted to apply the automatic generation of
> >  site entries to the mywiki/ subdir,
> > 
> >  - could we have a plugin that will
> >    parse the dir structure from mywiki/ downwards?
> >    Yes, I know we can with cocoon directory transformer
> > 
> >  - can we generate site structure from the dir structure and the
> >    filenames?
> >    Yes
> > 
> >  - How can we make the plugin insert the resulting site structure into
> >    the manually edited structure in site.xml.
> > 
> >    Could I for example have a special uri so that a site element like
> > 
> >         <mywiki label="My Wiki"
> >         href="automatedDirectory/demo/mywiki"/>
> > 
> >    = would automatically generate the site struture?
> >      Yes as this is similar to S5 using a prefix to the url to trigger
> >      a pipeline!
> 
> [update for devs: this would be an internal plugin we would build to 
> illustrate the extension of core internal functionality]
> 
> No, this is not the same as the s5 plugin. In the s5 plugin it is the 
> request that triggers the processing by the s5 plugin. In this case 
> there is no request generated when the site.xml file is processed.
> 
> [OT: The s5 plugin will be changing the way it does this because it is 
> bad design to have the URL indicate the format of the file being served. 
> Instead of 's5slides/presentation.html' it will be 'presentation.s5' or 
> presentation.s5.html]
> 
> I see two ways of doing what you want.
> 
> We could change the XSL that builds the site.xml file so that it looks 
> for something like:
> 
> <myWiki label="MyWiki" type="auto"/>
> 
> This would then make a request for something like:
> 
> demo/mywiki/site.auto.xml
> 
> which would then provide the site.xml snippet.
> 
> Alternatively (and more simply), we could do:
> 
> <xi:include src="demo/mywiki/site.auto.xml"/>
> 
> >    = would insert the inserted structure in place of the
> >      <mywiki>-element into the final sitemap?
> >      That should be possible but may be I'm too optmistic here.
> 
> I think the above solutions will do the job just fine.
> 
> > My reasons behind this proposal:
> > 
> > You and I talked about the usefulness of such a plugin in general.
> > Our main problem was that parsing the complete dir structure of a big
> > site might take to long. And - may I add - perhaps people would want
> > to use anvanced techniques for some of the site but allow for easy
> > maintenance for others.
> > 
> > By being able to have it process only part of the tree, we solve both
> > problems.
> > 
> > Let me know if this is too much work, otherwise I'll add some more
> > implementation details.
> 
> It look pretty easy on the surface, only experimentation (or a keen eye 
> from one of the other devs) can now show otherwise. It's on my list.
> 
> Ross
-- 
thorsten

"Together we stand, divided we fall!" 
Hey you (Pink Floyd)

Re: Auto generating site.xml

[Ferdinand Soethe <sa...@soethe.net>]

Ross Gardler wrote:

> We could change the XSL that builds the site.xml file so that it looks
> for something like:

> <myWiki label="MyWiki" type="auto"/>
> This would then make a request for something like:
> demo/mywiki/site.auto.xml
> which would then provide the site.xml snippet.

> Alternatively (and more simply), we could do:
> <xi:include src="demo/mywiki/site.auto.xml"/>

I realize that the xinclude statement would be much easier to
implement but I prefer solution number one because it remains closer
to the concept of site.xml and this easier to document and maintain in
the long run.

Or building on your first solution, why not have something like

<myWiki label="MyWiki" type="include" href="demo/mywiki/site.auto.xml"/>

That way it would be up to the user to have the sub-site-structure
generated by our plugin, other plugins or even be third part software
(point to a static file).

--
Ferdinand Soethe