[RT] CLI and Forrestbot (was Re: New Forrestbot in the works (Re: forrestbot configuration))

[Upayavira <uv...@upaya.co.uk>]

Jeff Turner wrote:

>On Tue, Sep 30, 2003 at 02:52:30PM -0400, Dave Brondsema wrote:
>  
>
>>I'm trying to set configuration files for forrestbot to generate several
>>sites.  However, it seems that the <prepare> section of my conf.xml file
>>has to be set with the same values as I already have in
>>forrest.properties.  Why can't forrestbot just use forrest.properties?
>>
>>Moreover, how to I specify forrest.maxmemory using the
>>forrestbot?  I have it set in forrest.properties, but I get an out of
>>memory error when I run forrestbot.
>>    
>>
>
>The current Forrestbot implementation (XSLT generating Ant) is inflexible
>and convoluted.  Over the last few days I've been rewriting the
>Forrestbot using Ant 1.6's <import> facility.  There's 'get-src', 'site',
>'deploy' and 'notify' targets in a generic script, and each project then
>has a script which overrides key properties.  The targets themselves can
>also be overridden.
>
>It's worked wonderfully well.  *Much* cleaner and more flexible.  Ant's
><import> is a killer feature.
>
>As for your specific question, the generic script <import>'s
>forrest.build.xml, so it does use forrest.properties.
>
>I'll commit my stuff tonight.
>  
>
Jeff,

Is there a reason why you use Ant for this sort of thing, as opposed to 
a more pure Cocoon CLI/bean approach? (other than because the CLI/bean 
isn't good enough yet).

To my mind, this is the sort of thing that I see the CLI/bean doing 
eventually. It can already send pages directly to a modifiable source, 
so with an SCPSource you could remove the final 'copy' stage. There is 
already a CVSSource which can be used to read out of a repository 
(although that would require a change to your sitemap, which is not 
preferable). The cli.xconf can now handle multiple sites (using <uris> 
elements). Then, once the CLI/bean can do all this, it can either be run 
from the command line via cron, or potentially be called from within a 
web interface (run as an Avalon component via the flow).

Can you see the CLI/bean taking over this role at some point in the 
future, or do you think there is some intrinsic role for Ant to play here?

Just musing.

Re: [RT] CLI and Forrestbot (was Re: New Forrestbot in the works (Re: forrestbot configuration))

[Jeff Turner <je...@apache.org>]

On Wed, Oct 01, 2003 at 11:29:29AM +0100, Upayavira wrote:
> Jeff Turner wrote:
> 
> >On Tue, Sep 30, 2003 at 02:52:30PM -0400, Dave Brondsema wrote:
> > 
> >
> >>I'm trying to set configuration files for forrestbot to generate several
> >>sites.  However, it seems that the <prepare> section of my conf.xml file
> >>has to be set with the same values as I already have in
> >>forrest.properties.  Why can't forrestbot just use forrest.properties?
> >>
> >>Moreover, how to I specify forrest.maxmemory using the
> >>forrestbot?  I have it set in forrest.properties, but I get an out of
> >>memory error when I run forrestbot.
> >>   
> >>
> >
> >The current Forrestbot implementation (XSLT generating Ant) is inflexible
> >and convoluted.  Over the last few days I've been rewriting the
> >Forrestbot using Ant 1.6's <import> facility.  There's 'get-src', 'site',
> >'deploy' and 'notify' targets in a generic script, and each project then
> >has a script which overrides key properties.  The targets themselves can
> >also be overridden.
> >
> >It's worked wonderfully well.  *Much* cleaner and more flexible.  Ant's
> ><import> is a killer feature.
> >
> >As for your specific question, the generic script <import>'s
> >forrest.build.xml, so it does use forrest.properties.
> >
> >I'll commit my stuff tonight.
> > 
> >
> Jeff,
> 
> Is there a reason why you use Ant for this sort of thing, as opposed to 
> a more pure Cocoon CLI/bean approach? (other than because the CLI/bean 
> isn't good enough yet).
> 
> To my mind, this is the sort of thing that I see the CLI/bean doing 
> eventually. It can already send pages directly to a modifiable source, 
> so with an SCPSource you could remove the final 'copy' stage. There is 
> already a CVSSource which can be used to read out of a repository 
> (although that would require a change to your sitemap, which is not 
> preferable). The cli.xconf can now handle multiple sites (using <uris> 
> elements). Then, once the CLI/bean can do all this, it can either be run 
> from the command line via cron, or potentially be called from within a 
> web interface (run as an Avalon component via the flow).
> 
> Can you see the CLI/bean taking over this role at some point in the 
> future, or do you think there is some intrinsic role for Ant to play here?

Well where does the CLI's job begin and end?  Its contract seems pretty
clear to me; it wants a directory of content and cocoon.xconf file, and
spits out a bunch of content.

But there are jobs to do before and after that.

Before:

 - Collate or generate content to be included in the site.  For Cocoon
   this means copying lib/jars.xml to src/documentation/xdocs/.  It might
   mean invoking Javadoc, or invoking Maven targets to generate all those
   pointless stats people love.  All far too general for the CLI to do.

 - More mundanely, obtain a project's static XML from somewhere (CVS,
   scp, ftp).  You're right, this could be done, slowly (opening a ssh
   connection for each file I imagine), with a CVSSource.

After:

 - Notify someone of the render status (via email, <echo>, sms etc).

 - Publish the results (local copy, scp, ftp).


Most of that seems more suited to Ant than the Cocoon CLI.


--Jeff


> Just musing.
> 
> 
> 

Re: [RT] CLI and Forrestbot (was Re: New Forrestbot in the works

[Upayavira <uv...@upaya.co.uk>]

Nicola Ken,

Long time no speak! Hello again!

> Upayavira wrote:
>
> ...
>
>> Jeff,
>>
>> Is there a reason why you use Ant for this sort of thing, as opposed 
>> to a more pure Cocoon CLI/bean approach? (other than because the 
>> CLI/bean isn't good enough yet).
>
>
> Jeff is correctly making the current system better. It's not a "final" 
> thing, as nothing is final, so we should eventually discuss options soon.
>
>> To my mind, this is the sort of thing that I see the CLI/bean doing 
>> eventually. It can already send pages directly to a modifiable 
>> source, so with an SCPSource you could remove the final 'copy' stage. 
>> There is already a CVSSource which can be used to read out of a 
>> repository (although that would require a change to your sitemap, 
>> which is not preferable). The cli.xconf can now handle multiple sites 
>> (using <uris> elements). Then, once the CLI/bean can do all this, it 
>> can either be run from the command line via cron, or potentially be 
>> called from within a web interface (run as an Avalon component via 
>> the flow).
>>
>> Can you see the CLI/bean taking over this role at some point in the 
>> future, or do you think there is some intrinsic role for Ant to play 
>> here?
>
>
> I didn't want to come out with this discussion now, but I see Forrest 
> running *live* using CVSSource, doing multiple sites and playing nice 
> with an Apache proxy in front, for added speed. In this way Cocoon 
> works as a "live CLI", doing slow transformations at specified expire 
> times and driven by Apache.
>
> Of course, if this works, it would be trivial to make the CLI work on 
> this. But it's still some time off, and the Forrestbot is being used 
> and already relied upon. We need to do the changes being discussed, 
> clean the build and source copying, and then have time to see what 
> will be missing from a CLI solution you highlight and fill in the gaps 
> (for example the webapp).
>
> ATM I think that we will still need some sort of Ant frontend, and of 
> course the webapp, but I hope that many things can be done easier by 
> Cocoon.

Thanks. You've answered my question. I wasn't thinking of anything 
immediately, and I'm sure Jeff's improvements are needed.

The background to my question was the fact that, to-date I've been 
wondering what on earth this forrestbot thing is. It was with Jeff's 
post that the penny dropped. I then started wondering if some (or all) 
of that functionality could be taken over by the CLI/bean. And you're 
saying that this could be so, which has answered my question.

I realise that any transition from a stable working system to a newer 
(better?) one needs to be managed well, so I wasn't proposing to do 
anything quickly. It just helps to guide my planning of what I want to 
do next on the cli/bean, and tells me that my ideas for a bean that can 
be used with cron or as a webapp and that can collect data from 
whereever and deliver it wherever could actually be useful to people. 
That always helps!

Regards, Upayavira

Re: [RT] CLI and Forrestbot (was Re: New Forrestbot in the works (Re: forrestbot configuration))

[Nicola Ken Barozzi <ni...@apache.org>]

Upayavira wrote:

...
> Jeff,
> 
> Is there a reason why you use Ant for this sort of thing, as opposed to 
> a more pure Cocoon CLI/bean approach? (other than because the CLI/bean 
> isn't good enough yet).

Jeff is correctly making the current system better. It's not a "final" 
thing, as nothing is final, so we should eventually discuss options soon.

> To my mind, this is the sort of thing that I see the CLI/bean doing 
> eventually. It can already send pages directly to a modifiable source, 
> so with an SCPSource you could remove the final 'copy' stage. There is 
> already a CVSSource which can be used to read out of a repository 
> (although that would require a change to your sitemap, which is not 
> preferable). The cli.xconf can now handle multiple sites (using <uris> 
> elements). Then, once the CLI/bean can do all this, it can either be run 
> from the command line via cron, or potentially be called from within a 
> web interface (run as an Avalon component via the flow).
> 
> Can you see the CLI/bean taking over this role at some point in the 
> future, or do you think there is some intrinsic role for Ant to play here?

I didn't want to come out with this discussion now, but I see Forrest 
running *live* using CVSSource, doing multiple sites and playing nice 
with an Apache proxy in front, for added speed. In this way Cocoon works 
as a "live CLI", doing slow transformations at specified expire times 
and driven by Apache.

Of course, if this works, it would be trivial to make the CLI work on 
this. But it's still some time off, and the Forrestbot is being used and 
already relied upon. We need to do the changes being discussed, clean 
the build and source copying, and then have time to see what will be 
missing from a CLI solution you highlight and fill in the gaps (for 
example the webapp).

ATM I think that we will still need some sort of Ant frontend, and of 
course the webapp, but I hope that many things can be done easier by Cocoon.

-- 
Nicola Ken Barozzi                   nicolaken@apache.org
             - verba volant, scripta manent -
    (discussions get forgotten, just code remains)
---------------------------------------------------------------------