You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@forrest.apache.org by Ross Gardler <rg...@apache.org> on 2004/10/29 03:30:33 UTC
[FIXED] Re: [HEADS-UP] forrest site is broken in SVN Head
Ross Gardler wrote:
> Ross Gardler wrote:
>
>> Hmmmm... Forrest run works fine, but forrest site is broken.
>>
>> I *think* I broke this with the plugin stuff. I'm looking into it now.
>
>
Please do an svn up, this is now fixed.
The fix has broken the IMSManifest plugin but it is very late here and I
need to go to bed, since (as far as I am aware) I am the only person
using that plugin this shouldn't impact anyone. I'll create a more
flexible fix when I'm not as tired.
(sorry if anyone lost time over this - I know I saw something on the
lists regarding such a bug, I'll hunt it down and mail them now)
Ross
Re: Solving matching problems caused by plugins
Posted by David Crossley <cr...@apache.org>.
David Crossley wrote:
> Ross Gardler wrote:
>
> Perhaps it was just the insertion point of the plugins
> sitemaps that caused those matches to start being missed.
> Or perhaps it *must* take full control and we just need
> to proceed with implementation of all of those as plugins.
Ah, i see that later in the day you fixed both
those issues by moving the mount point.
Ross, here is a big thankyou for the brilliant work
and the effort that you have put in.
Now don't neglect that family furniture.
--
David Crossley
Re: Solving matching problems caused by plugins
Posted by David Crossley <cr...@apache.org>.
Ross Gardler wrote:
> David Crossley wrote:
> >
> > The Cocoon cli.xconf is how extra abilities are configured
> > for our Cocoon Command-Line Interface CLI, e.g. patterns
> > to be ignored like "apidocs" and extra docs to be processed
> > like "mirrors.html". I presume that it is read by Cocoon
> > at startup.
> >
> > For some unknown reason, each project cli.xconf started
> > being ignored and Forrest's core default cli.xconf is used.
> > 'forrest echo-properties' shows the problem. The value
> > from forrest.properties is not being used. Again that
> > could be a pointer to some new wider bug.
>
> I'm not familiar with cli stuff from cocoon. What I do know is that
> moving the sitemap plugin to just before the final "catchall" match of
> "**.xml" in forrest.xmap fixed the problem with mirrors.html indicated
> above.
>
> Can you (or someone else) look to see if there is still a problem with
> the loading of cli.xconf. I don't think that the position of the plugin
> sitemap mount would affect this.
It all works again now. I reported that earlier in this thread.
The mirrors.html is added by the cli.xconf
Yeah i didn't think the position of the plugin sitemap
would affect it either, but it did seem to.
--
David Crossley
Re: Solving matching problems caused by plugins
Posted by Ross Gardler <rg...@apache.org>.
David Crossley wrote:
> Ross Gardler wrote:
>
<snip/>
> Perhaps it was just the insertion point of the plugins
> sitemaps that caused those matches to start being missed.
> Or perhaps it *must* take full control and we just need
> to proceed with implementation of all of those as plugins.
I am confident it is the insertion point. I did some experimentation
last night with lots of different types of interaction. I will put my
findings together as a proposal next week sometime. For now, the current
insertion point is fine for source formats.
>>>>------ from 'forrest' on the core docs ...
>>>><link message="...content/xdocs/mirrors.xml
>>>>(No such file or directory)" uri="mirrors.html">
>>>>------
>>>>
>>>>... that one indicates that the project cli.xconf
>>>>is no longer being found, and echo properties
>>>>confirms that.
>>
>>Where would this normally be processed, I can't seem to find it.
>
>
> The Cocoon cli.xconf is how extra abilities are configured
> for our Cocoon Command-Line Interface CLI, e.g. patterns
> to be ignored like "apidocs" and extra docs to be processed
> like "mirrors.html". I presume that it is read by Cocoon
> at startup.
>
> For some unknown reason, each project cli.xconf started
> being ignored and Forrest's core default cli.xconf is used.
> 'forrest echo-properties' shows the problem. The value
> from forrest.properties is not being used. Again that
> could be a pointer to some new wider bug.
I'm not familiar with cli stuff from cocoon. What I do know is that
moving the sitemap plugin to just before the final "catchall" match of
"**.xml" in forrest.xmap fixed the problem with mirrors.html indicated
above.
Can you (or someone else) look to see if there is still a problem with
the loading of cli.xconf. I don't think that the position of the plugin
sitemap mount would affect this.
Ross
Re: Solving matching problems caused by plugins
Posted by David Crossley <cr...@apache.org>.
Ross Gardler wrote:
> Ross Gardler wrote:
> > David Crossley wrote:
> >>
> >> It mostly works now, but building our own docs and building
> >> the test site shows some other related new bug. I can only
> >> describe the symptoms at this stage ...
> >>
> >> ------ from 'forrest seed site' ...
> >> <link message="...content/xdocs/samples/ihtml-sample.xml
> >> (No such file or directory)" uri="samples/ihtml-sample.html">
> >>
> >> <link message="...content/xdocs/samples/wiki-sample.xml
> >> (No such file or directory)" uri="samples/wiki-sample.html">
> >> ------
> >>
> >> ... those two indicate that the special filetype tests
> >> in forrest.xmap at line 234 are not being triggered.
>
> These should really be extracted to a plugin which will solve the
> problem. I think that the only thing that should be left in Forrest is
> the handling of our intermediate format (this will be plural when we
> move to XHTML2).
>
> Dues anyone have a problem with that?
I agree with that. Your plan, later in this thread,
sounds good.
My concern here was that the addition of the new
plugins ability yesterday, caused these two bugs.
They might be a pointer to something sinister.
Perhaps it was just the insertion point of the plugins
sitemaps that caused those matches to start being missed.
Or perhaps it *must* take full control and we just need
to proceed with implementation of all of those as plugins.
> >> ------ from 'forrest' on the core docs ...
> >> <link message="...content/xdocs/mirrors.xml
> >> (No such file or directory)" uri="mirrors.html">
> >> ------
> >>
> >> ... that one indicates that the project cli.xconf
> >> is no longer being found, and echo properties
> >> confirms that.
>
> Where would this normally be processed, I can't seem to find it.
The Cocoon cli.xconf is how extra abilities are configured
for our Cocoon Command-Line Interface CLI, e.g. patterns
to be ignored like "apidocs" and extra docs to be processed
like "mirrors.html". I presume that it is read by Cocoon
at startup.
For some unknown reason, each project cli.xconf started
being ignored and Forrest's core default cli.xconf is used.
'forrest echo-properties' shows the problem. The value
from forrest.properties is not being used. Again that
could be a pointer to some new wider bug.
--
David Crossley
Re: Solving matching problems caused by plugins
Posted by Ross Gardler <rg...@apache.org>.
David Crossley wrote:
> Ross Gardler wrote:
>
>>Dave Brondsema wrote:
>>
>>>Ross Gardler wrote:
>>>I think that's the right idea. Lots of stuff can be moved to be a
>>>plugin. For example, changes/todo.html as a plugin would make that
>>>functionality optional and so people can use those reserveds names.
>>>
>>>I haven't tried yet.. does the plugin location allow for input formats
>>>and output formats?
>>
>>I'm starting to think that we need a more flexible plugin system that
>>would allow insertion at different points in the generation process.
>>
>>For example, IMSManifest is currently broken because of having to move
>>the plugin sitemap mount to *after* the generation of things like
>>site.xml, faq.xml etc.
>>
>>Your question about input and output formats is another point, output
>>formats would need to be mounted in a different location from the input
>>formats (which is all I have worked with to date).
>>
>>I was thinking of moving to a system where plugins can provide three
>>sitemaps:
>>
>>preprocessing (comes before *everything* in core - such sitemaps would
>>have to be very careful to only match what they need, i.e. IMSManifest
>>would match abs-linkmap, site.xml. tabs.xml etc.)
>
>
> Already the project sitemap.xmap must be careful to
> only match what they need.
>
> So where does their project sitemap fit into this?
Yes, and I suspect that this will start to cause us real problems. As
Dave Bronsema suggested earlier in this thread, we need to create a
contract for plugins. I think this same contract can be used for the
project sitemap. In the meantime we'll leave project sitemap where it is
since we will learn more about the correct handling through working with
the plugins.
>>input formats (these would be placed where the current mount is, i.e.
>>just before the default **.xml match in forrest.xmap)
>
>
> Would a project sitemap still be able to match those
> if they needed to?
Yes, the project sitemap is (and will always be) mounted before the
plugins. In other words, the order of processing precedence would be
project -> plugins -> forrest.
There is the question of which plugin gets preference since there could
be conflicts. Right now I have done two things to try and tackle this:
1) only the plugins required for the current project will be mounted,
regardless of what plugins are on the local HD
2) plugins will be processed in the order they appear in the
project.requried.plugins property
I believe this will be sufficient for the time being. For example, it
will handle the overlap between the current simplified-docbook plugin
and the proposed full-docbook plugin.
> [snip]
>
>>>We will also have to document a "plugin contract" of what pipeline parts
>>>will be made available and supported. I think we have need to work
>>>through some more real examples of plugins before we fully understand
>>>their power and limitations. Then we can best write the contract.
>>
>>I agree. I will, over the coming weeks move all none core functionality
>>from core into a plugin. That should give us a fair number of use cases
>>to at least draft the contract from.
>>
>>The question at the moment is where to *stop* moving stuff into plugins.
>
>
> Bit-by-bit until we can understand the core sitemaps again.
> If we start to duplicate functionality in each plugin, then
> it will become apparent where to stop.
OK sounds wise. I'll keep going until someone holds up a stop sign (I
suspect I may miss it myself - extracting plugin code is quick reward
stuff - much better than building flat pack furnitre ;-))
Ross
Re: Solving matching problems caused by plugins
Posted by David Crossley <cr...@apache.org>.
Ross Gardler wrote:
> Dave Brondsema wrote:
> > Ross Gardler wrote:
> > I think that's the right idea. Lots of stuff can be moved to be a
> > plugin. For example, changes/todo.html as a plugin would make that
> > functionality optional and so people can use those reserveds names.
> >
> > I haven't tried yet.. does the plugin location allow for input formats
> > and output formats?
>
> I'm starting to think that we need a more flexible plugin system that
> would allow insertion at different points in the generation process.
>
> For example, IMSManifest is currently broken because of having to move
> the plugin sitemap mount to *after* the generation of things like
> site.xml, faq.xml etc.
>
> Your question about input and output formats is another point, output
> formats would need to be mounted in a different location from the input
> formats (which is all I have worked with to date).
>
> I was thinking of moving to a system where plugins can provide three
> sitemaps:
>
> preprocessing (comes before *everything* in core - such sitemaps would
> have to be very careful to only match what they need, i.e. IMSManifest
> would match abs-linkmap, site.xml. tabs.xml etc.)
Already the project sitemap.xmap must be careful to
only match what they need.
So where does their project sitemap fit into this?
> input formats (these would be placed where the current mount is, i.e.
> just before the default **.xml match in forrest.xmap)
Would a project sitemap still be able to match those
if they needed to?
[snip]
> > We will also have to document a "plugin contract" of what pipeline parts
> > will be made available and supported. I think we have need to work
> > through some more real examples of plugins before we fully understand
> > their power and limitations. Then we can best write the contract.
>
> I agree. I will, over the coming weeks move all none core functionality
> from core into a plugin. That should give us a fair number of use cases
> to at least draft the contract from.
>
> The question at the moment is where to *stop* moving stuff into plugins.
Bit-by-bit until we can understand the core sitemaps again.
If we start to duplicate functionality in each plugin, then
it will become apparent where to stop.
> I'm starting to think that Forrest core should follow the Eclipse model
> where the isn't any real functionality in core, just a solid framework
> onto which to build the required functionality.
That sounds sensible.
> Our releases would then become (for example)
How we do releases and what they contain is something
that we need to carefully consider. Let us first get our
plugin framework settled a bit.
> - Forrest for Software Projects (core + site.xml + HTML + PDF + JavaDoc)
>
> - Forrest for Learning Objects (core + IMSManifest + OpenOffice + Latex
> +PDF)
>
> - Forrest for Technical Authors (core + site.xml + docbook + PDF)
>
> Etc.
>
> I may be getting a little ahead of myself though ;-)
No, i think that you are correct to look ahead.
--
David Crossley
Re: Solving matching problems caused by plugins
Posted by Ross Gardler <rg...@apache.org>.
Dave Brondsema wrote:
> Ross Gardler wrote:
> I think that's the right idea. Lots of stuff can be moved to be a
> plugin. For example, changes/todo.html as a plugin would make that
> functionality optional and so people can use those reserveds names.
>
> I haven't tried yet.. does the plugin location allow for input formats
> and output formats?
I'm starting to think that we need a more flexible plugin system that
would allow insertion at different points in the generation process.
For example, IMSManifest is currently broken because of having to move
the plugin sitemap mount to *after* the generation of things like
site.xml, faq.xml etc.
Your question about input and output formats is another point, output
formats would need to be mounted in a different location from the input
formats (which is all I have worked with to date).
I was thinking of moving to a system where plugins can provide three
sitemaps:
preprocessing (comes before *everything* in core - such sitemaps would
have to be very careful to only match what they need, i.e. IMSManifest
would match abs-linkmap, site.xml. tabs.xml etc.)
input formats (these would be placed where the current mount is, i.e.
just before the default **.xml match in forrest.xmap)
output formats (not sure where yet, as I haven't thought through a use
case, I suspect Sean's Docbook plugin will be the first)
> Is it in a location that wholesite could be in a
> plugin?
No idea. To be honest, I've not looked since this is currently broken
anyway.
> We will also have to document a "plugin contract" of what pipeline parts
> will be made available and supported. I think we have need to work
> through some more real examples of plugins before we fully understand
> their power and limitations. Then we can best write the contract.
I agree. I will, over the coming weeks move all none core functionality
from core into a plugin. That should give us a fair number of use cases
to at least draft the contract from.
The question at the moment is where to *stop* moving stuff into plugins.
I'm starting to think that Forrest core should follow the Eclipse model
where the isn't any real functionality in core, just a solid framework
onto which to build the required functionality. Our releases would then
become (for example)
- Forrest for Software Projects (core + site.xml + HTML + PDF + JavaDoc)
- Forrest for Learning Objects (core + IMSManifest + OpenOffice + Latex
+PDF)
- Forrest for Technical Authors (core + site.xml + docbook + PDF)
Etc.
I may be getting a little ahead of myself though ;-)
Ross
Re: Solving matching problems caused by plugins
Posted by Dave Brondsema <da...@brondsema.net>.
Ross Gardler wrote:
> Ross Gardler wrote:
>
>> David Crossley wrote:
>>
>
>>> It mostly works now, but building our own docs and building
>>> the test site shows some other related new bug. I can only
>>> describe the symptoms at this stage ...
>>
>>
>>
>>>
>>> ------ from 'forrest seed site' ...
>>> <link message="...content/xdocs/samples/ihtml-sample.xml
>>> (No such file or directory)" uri="samples/ihtml-sample.html">
>>>
>>> <link message="...content/xdocs/samples/wiki-sample.xml
>>> (No such file or directory)" uri="samples/wiki-sample.html">
>>> ------
>>>
>>> ... those two indicate that the special filetype tests
>>> in forrest.xmap at line 234 are not being triggered.
>
>
> These should really be extracted to a plugin which will solve the
> problem. I think that the only thing that should be left in Forrest is
> the handling of our intermediate format (this will be plural when we
> move to XHTML2).
>
> Dues anyone have a problem with that?
>
I think that's the right idea. Lots of stuff can be moved to be a
plugin. For example, changes/todo.html as a plugin would make that
functionality optional and so people can use those reserveds names.
I haven't tried yet.. does the plugin location allow for input formats
and output formats? Is it in a location that wholesite could be in a
plugin?
We will also have to document a "plugin contract" of what pipeline parts
will be made available and supported. I think we have need to work
through some more real examples of plugins before we fully understand
their power and limitations. Then we can best write the contract.
--
Dave Brondsema : dave@brondsema.net
http://www.splike.com : programming
http://csx.calvin.edu : student org
http://www.brondsema.net : personal
Solving matching problems caused by plugins (was Re: [FIXED] Re:
[HEADS-UP] forrest site is broken in SVN Head)
Posted by Ross Gardler <rg...@apache.org>.
Ross Gardler wrote:
> David Crossley wrote:
>
>> It mostly works now, but building our own docs and building
>> the test site shows some other related new bug. I can only
>> describe the symptoms at this stage ...
>
>
>>
>> ------ from 'forrest seed site' ...
>> <link message="...content/xdocs/samples/ihtml-sample.xml
>> (No such file or directory)" uri="samples/ihtml-sample.html">
>>
>> <link message="...content/xdocs/samples/wiki-sample.xml
>> (No such file or directory)" uri="samples/wiki-sample.html">
>> ------
>>
>> ... those two indicate that the special filetype tests
>> in forrest.xmap at line 234 are not being triggered.
These should really be extracted to a plugin which will solve the
problem. I think that the only thing that should be left in Forrest is
the handling of our intermediate format (this will be plural when we
move to XHTML2).
Dues anyone have a problem with that?
>> ------ from 'forrest' on the core docs ...
>> <link message="...content/xdocs/mirrors.xml
>> (No such file or directory)" uri="mirrors.html">
>> ------
>>
>> ... that one indicates that the project cli.xconf
>> is no longer being found, and echo properties
>> confirms that.
Where would this normally be processed, I can't seem to find it.
Ross
Re: [FIXED] Re: [HEADS-UP] forrest site is broken in SVN Head
Posted by Ross Gardler <rg...@apache.org>.
David Crossley wrote:
> Ross Gardler wrote:
>
>>Ross Gardler wrote:
>>
>>>Ross Gardler wrote:
>>>
>>>
>>>>Hmmmm... Forrest run works fine, but forrest site is broken.
>>>>
>>>>I *think* I broke this with the plugin stuff. I'm looking into it now.
>>
>>Please do an svn up, this is now fixed.
>>
>
> [snip]
>
>>(sorry if anyone lost time over this - I know I saw something on the
>>lists regarding such a bug, I'll hunt it down and mail them now)
>
>
> It mostly works now, but building our own docs and building
> the test site shows some other related new bug. I can only
> describe the symptoms at this stage ...
Thanks, I'll look into that later today (have to build some furniture
first ;-)
Ross
>
> ------ from 'forrest seed site' ...
> <link message="...content/xdocs/samples/ihtml-sample.xml
> (No such file or directory)" uri="samples/ihtml-sample.html">
>
> <link message="...content/xdocs/samples/wiki-sample.xml
> (No such file or directory)" uri="samples/wiki-sample.html">
> ------
>
> ... those two indicate that the special filetype tests
> in forrest.xmap at line 234 are not being triggered.
>
> ------ from 'forrest' on the core docs ...
> <link message="...content/xdocs/mirrors.xml
> (No such file or directory)" uri="mirrors.html">
> ------
>
> ... that one indicates that the project cli.xconf
> is no longer being found, and echo properties
> confirms that.
>
Re: [FIXED] Re: [HEADS-UP] forrest site is broken in SVN Head
Posted by David Crossley <cr...@apache.org>.
Ross Gardler wrote:
> Ross Gardler wrote:
> > Ross Gardler wrote:
> >
> >> Hmmmm... Forrest run works fine, but forrest site is broken.
> >>
> >> I *think* I broke this with the plugin stuff. I'm looking into it now.
>
> Please do an svn up, this is now fixed.
>
[snip]
>
> (sorry if anyone lost time over this - I know I saw something on the
> lists regarding such a bug, I'll hunt it down and mail them now)
It mostly works now, but building our own docs and building
the test site shows some other related new bug. I can only
describe the symptoms at this stage ...
------ from 'forrest seed site' ...
<link message="...content/xdocs/samples/ihtml-sample.xml
(No such file or directory)" uri="samples/ihtml-sample.html">
<link message="...content/xdocs/samples/wiki-sample.xml
(No such file or directory)" uri="samples/wiki-sample.html">
------
... those two indicate that the special filetype tests
in forrest.xmap at line 234 are not being triggered.
------ from 'forrest' on the core docs ...
<link message="...content/xdocs/mirrors.xml
(No such file or directory)" uri="mirrors.html">
------
... that one indicates that the project cli.xconf
is no longer being found, and echo properties
confirms that.
--
David Crossley