You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@forrest.apache.org by br...@apache.org on 2003/12/04 22:58:03 UTC
cvs commit: xml-forrest status.xml
brondsem 2003/12/04 13:58:03
Modified: src/documentation/content/xdocs forrestbot.xml site.xml
. status.xml
Added: src/documentation/content/xdocs forrestbot-intro-old.xml
forrestbot-old.xml
Removed: src/documentation/content/xdocs forrestbot-intro.xml
Log:
<action dev="DB" type="add" context="docs">
Began documentation for new forrestbot; moved stable forrestbot docs
</action>
Revision Changes Path
1.14 +101 -413 xml-forrest/src/documentation/content/xdocs/forrestbot.xml
Index: forrestbot.xml
===================================================================
RCS file: /home/cvs/xml-forrest/src/documentation/content/xdocs/forrestbot.xml,v
retrieving revision 1.13
retrieving revision 1.14
diff -u -r1.13 -r1.14
--- forrestbot.xml 29 Sep 2003 07:27:48 -0000 1.13
+++ forrestbot.xml 4 Dec 2003 21:58:02 -0000 1.14
@@ -1,426 +1,114 @@
-<?xml version="1.0"?>
-<!DOCTYPE document PUBLIC "-//APACHE//DTD Documentation V1.2//EN" "http://apache.org/forrest/dtd/document-v12.dtd">
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE document PUBLIC "-//APACHE//DTD Documentation V1.2//EN"
+"document-v12.dtd">
<document>
<header>
- <title>The ForrestBot</title>
+ <title>Forrestbot</title>
+
<authors>
- <person name="Marc Portier" email="mpo@outerthought.org"/>
+ <person email="brondsem@apache.org" name="Dave Brondsema" />
</authors>
- <abstract>This document explains how the Forrest project can be used to
- centrally build the documentation for different projects. It also explains
- which pieces in the Forrest distribution form this
- functionality, and how they could be extended in the future.</abstract>
</header>
+
<body>
+ <warning>This documentation applies to the forrestbot in CVS, not the
+ forrestbot included with Forrest 0.5.1 or any previous release.
+ Documentation for that forrestbot is <link href="site:forrestbot-old">here</link>.
+ This page is incomplete and may have errors.</warning>
+
<section>
- <title>Foreword</title>
- <p>This document explains how the Forrest project can be used to
- centrally build the documentation for different projects.
- It also explains which pieces in the Forrest distribution form this
- functionality, and how they could be extended in the future.</p>
- <p>See some related documents:</p>
- <ul>
- <li><link href="site:forrest-contract">Our Contract</link>
- - explains the rules with which your project should comply
- </li>
- <li><link href="site:forrestbot-intro">Forrestbot introduction</link>
- - concise overview of Forrestbot
- </li>
- </ul>
- </section>
- <section>
- <title>Using the Forrestbot</title>
- <section>
- <title>Setting up the centralized service</title>
- <p>The goal is to publish a static website that can be updated
- in near-real-time to accomodate newly available content. Forrestbot
- can run on one machine, get content from another machine, and publish
- to yet another machine.</p>
- <p>Running the bot is as easy as calling the following in your shell.
- (executed in the root dir of the distribution.)</p>
- <source>forrestbot -Dbot.config=myOwn.conf.xml -Dtemplate.echo=[true|false]</source>
- <p>It is quite typical to perform this from a shell script that gets
- scheduled for execution on a regular basis.</p>
- </section>
- <section>
- <title>Describing your project to the bot</title>
- <fixme author="mpo">There is not yet a DTD for this configuration
- file.</fixme>
- <p>The actions the bot is going to take are defined in the XML
- file pointed at with the <code>bot.config</code> System
- property. (if not specified with <code>-Dbot.config</code>
- then the default config from the
- distribution is taken: <code>./forrestbot.conf.xml</code>) </p>
- <p>This configuration file has one (required, but optionally empty) child-element for
- setting cross-project defaults, followed by a list of
- project-elements that hold the configuration for the various projects
- that the bot is going to work on.</p>
- <source>
-<![CDATA[<?xml version="1.0"?>
-<forrest-config>
- <defaults>
- ...<!-- different fallback values for various workstages -->
- </defaults>
-
- <project name="project-name-1">
- ...<!-- different subelements configure the various workstages -->
- </project>
-
- <project name="project-name-2">
- ...<!-- different subelements configure the various workstages -->
- </project>
-</forrest-config>
-]]>
- </source>
- <p>During the bot-run, each project entry will start a parallel and
- independent process consisting of a number of so called 'workstages'.</p>
- <p>These workstages are listed in the
- <code>./src/resources/forrestbot/stages.conf.xml</code> file and their actual
- execution is handled by the various template tasks in the
- <code>./resources/forrestbot/ant/templates.build.xml</code> file. </p>
- <p>Currently the various workstages are:</p>
- <ul>
- <dl>
- <dt>prepare</dt>
- <dd> Make a clean Forrest-defined Cocoon context environment.</dd>
- </dl>
- <dl>
- <dt>get-src</dt>
- <dd>Slide in the project specific document sources in that Cocoon
- context environment.</dd>
- </dl>
- <dl>
- <dt>generate</dt>
- <dd> Actually generate the site using Cocoon</dd>
- </dl>
- <dl>
- <dt>deploy</dt>
- <dd>Publish the newly created site</dd>
- </dl>
- <dl>
- <dt>cleanup</dt>
- <dd><No real actions yet></dd>
- </dl>
- </ul>
- <p>Inside the project element the configuration allows to set some
- parameters that get consumed in the template targets. This workstage dependant
- configuration always takes the following layout: an element with the name of
- one of the tasks, and an optional type-attribute with nested elements that hold
- parameter names, with required name attribute holding the actual parameter
- value. Example:</p>
- <source><project name="xml-forrest">
- <prepare>
- <skin name="forrest-site"/>
- </prepare>
-
- <get-src type="cvs">
- <host name="cvs.apache.org"/>
- <root name="/home/cvspublic"/>
- <user name="anoncvs"/>
- <passwd name="anoncvs"/>
- <module name="xml-forrest"/>
- <content-dir name="src/documentation"/>
- <project-dir name=""/>
- </get-src>
-
- <generate>
- <debuglevel name="ERROR" />
- </generate>
-
- <deploy type="scp">
- <host name="krysalis.sourceforge.net"/>
- <root name="/home/groups/k/kr/krysalis/htdocs"/>
- <user name="forrestbot"/>
- <dir name="forrest"/>
- </deploy>
- </project></source>
- <note>Only the workstages that need configuration settings have an
- entry here. At all times there will be a workstage template called for each
- stage of the stages.conf.xml (even if it is not explicitly mentioned inside the
- <project> element.) </note>
- <p>Explained in detail, this will</p>
- <ul>
- <li>SET names properties (workstage(.@type)?.*) to appropriate values
- (*/@name)</li>
- <ul>
- <dl>
- <dt>prepare.skin</dt>
- <dd>forrest-site</dd>
- </dl>
- <dl>
- <dt>get-src.cvs.host</dt>
- <dd>cvs.apache.org</dd>
- </dl>
- <dl>
- <dt>get-src.cvs.root</dt>
- <dd>/home/cvspublic</dd>
- </dl>
- <dl>
- <dt>get-src.cvs.user</dt>
- <dd>anoncvs</dd>
- </dl>
- <dl>
- <dt>get-src.cvs.passwd</dt>
- <dd>anoncvs</dd>
- </dl>
- <dl>
- <dt>get-src.cvs.module</dt>
- <dd>xml-forrest</dd>
- </dl>
- <dl>
- <dt>get-src.cvs.content-dir</dt>
- <dd>src/documentation</dd>
- </dl><dl>
- <dt>get-src.cvs.project-dir</dt>
- <dd></dd>
- </dl>
- <dl>
- <dt>generate.debuglevel</dt>
- <dd>ERROR</dd>
- </dl>
- <dl>
- <dt>deploy.scp.host</dt>
- <dd>krysalis.sourceforge.net</dd>
- </dl>
- <dl>
- <dt>deploy.scp.root</dt>
- <dd>/home/groups/k/kr/krysalis/htdocs</dd>
- </dl>
- <dl>
- <dt>deploy.scp.user</dt>
- <dd>forrestbot</dd>
- </dl>
- <dl>
- <dt>deploy.scp.dir</dt>
- <dd>forrest</dd>
- </dl>
- </ul>
- <li>and CALL the following templates
- (template.workstage(.@type)?):</li>
- <ul>
- <li>template.prepare</li>
- <li>template.get-src.cvs</li>
- <li>template.generate</li>
- <li>template.deploy.scp</li>
- <li>template.cleanup</li>
- </ul>
- </ul>
- <p>What each of these templates is going to do with the parameters can
- be found in the templates.build.xml itself (at this stage) and is put out to
- the console if the bot is run with the <code>-Dtemplate.echo=true</code>
- flag</p>
- <section>
- <title>Setting bot-wide defaults</title>
- <p>The <code><defaults></code> section in the
- <code><forrest-config></code> allows for one-time setting of parameters
- you want to reuse for different projects. The syntax (and semantic value) of
- the nested elements is identical to the use inside the
- <code><project></code> tags.</p>
- </section>
- </section>
- <note>Once you understand all of this you might decide NOT to run your
- own centralized service but rather ask your project to be taken up in the
- centralized service of the project team itself: just send a mail to
- forrest-dev <link href="site:mail-lists">mail list</link>
- defining the required parameters to be taken up in the
- process.</note>
- <section>
- <title>Checking logs and getting them by mail</title>
- <p>For each distinct project the Forrest bot is maintaining a separate
- working log of the different workstages being executed. These logs can be
- opened and viewed in the ./build/bot directory</p>
- <p>In addition to the @name attribute the <project> tag also can
- contain a @sendlogto attribute that must be holding a (comma separated list
- off) email address(es) where the project specific log needs to be sent to after
- all workstages have been completed (or as soon as building failed)</p>
- </section>
+ <title>Using Forrestbot</title>
+
+ <p>Simply execute:</p>
+
+ <source>forrest -f mybuildfile.xml</source>
+
+ <p>The next section explains how to create your buildfile.</p>
</section>
+
<section>
- <title>Current supported workstages and types.</title>
- <section>
- <title>
- <code><prepare></code>
- </title>
- <p>No specific type versions. (don't use type-attribute)</p>
- <p>Template arguments:</p>
- <ul>
- <li><code><skin></code> @name holds the name of the skin to use.</li>
- </ul>
- </section>
- <section>
- <title>
- <code><get-src ...></code>
- </title>
- <section>
- <title>
- <code><get-src type="cvs"></code>
- </title>
- <p>Template arguments:</p>
- <ul>
- <li><code><host></code> @name holds the ipaddress or dns name of the cvs serving host</li>
- <li><code><root></code> @name holds the cvs-root directory on that host</li>
- <li><code><user></code> @name holds the username to use on the cvs repository </li>
- <li><code><passwd></code> @name holds the password to use on the cvs repository</li>
- <li><code><module></code> @name holds the module name that holds the </li>
- <li><code><dir></code> @name holds the relative path to the {docroot} directory. (This is the dir that is holding the ./content/xdocs as specified in t. </li>
- </ul>
- </section>
- <section>
- <title>
- <code><get-src type="local-copy"></code>
- </title>
- <p>Template arguments:</p>
- <ul>
- <li><code><content-dir/></code> @name holds the path pointing to the {docroot} directory. (See the <link href="site:forrest-contract">contract</link> to understand what should be there.)</li>
- <li><code><project-dir/></code> @name holds the path pointing to the {projecthome} directory where the xml project descriptors reside.</li>
- </ul>
- </section>
- </section>
- <section>
- <title>
- <code><generate></code>
- </title>
- <p>No specific type versions. (don't use type-attribute)</p>
- <p>Template arguments:</p>
- <ul>
- <li><code><debuglevel></code> @name holds the threshold-level for showing debug statements in the cocoon generation process.</li>
- </ul>
- </section>
- <section>
- <title>
- <code><deploy ...></code>
- </title>
- <section>
- <title>
- <code><deploy type="scp"></code>
- </title>
- <p>Template arguments:</p>
- <ul>
- <li><code><host></code> @name holds the remote host where the file will be copied to.</li>
- <li><code><user></code> @name holds the username to be used when logging onto the remote host.</li>
- <li><code><root></code> @name holds the prefix part of where the content needs to be published.</li>
- <li><code><dir></code> @name holds the suffix part of where the content needs to be published.</li>
- </ul>
- <warning>Using this approach requires that the public key of the user executing the bot target is present in the authorized key file (<code>~/.ssh/authorized_keys2</code>) of the remote user (on the remote host).</warning>
- <note>Might be interesting to understand that the full process actually is bundling all files to copy in a *.tar.gz that is un-tar.gzipped on the remote host using ssh. </note>
- </section>
- <section>
- <title>
- <code><deploy type="local-copy"></code>
- </title>
- <p>Template arguments:</p>
- <ul>
- <li><code><destination></code> @name holds the path where the generated content needs to be copied to.</li>
- </ul>
- </section>
- <section>
- <title>
- <code><deploy type="ftp"></code>
- </title>
- <p>Template arguments:</p>
- <ul>
- <li><code><host></code> @name holds the hostname of the ftp server to publish to.</li>
- <li><code><user></code> @name holds the username to use for loging onto the ftp server.</li>
- <li><code><passwd></code> @name holds the password to use.</li>
- <li><code><destination></code> @name holds the path to the directory on the remote host where the generated content needs to be published. The process is using the <code>ftp cd</code> command to get there. This means this directory has to exist.</li>
- </ul>
- </section>
- <warning>Currently the ant distribution that is included in forrest cvs is not offering the required NetComponents.jar to actually support this type of deployment. If you want to use it anyway you should get the required jar from <link href="http://www.savarese.org/oro/downloads/index.html#NetComponents">http://www.savarese.org/oro/downloads</link> and drop that into your <code>{forrest-sandbox}/tools/antipede/lib</code> directory. </warning>
- </section>
- <section>
- <title>
- <code><cleanup></code>
- </title>
- <p>No specific type versions. (don't use type-attribute)</p>
- <p>Template arguments: None.</p>
- </section>
+ <title>Creating a buildfile</title>
+
+ <p>A sample buildfile:</p>
+
+ <source><project name="myproject">
+ <property name="mail.host" value="smtp.myhost.com"/>
+ <property name="mail.to" value="me@myhost.com"/>
+ <property name="administrator" value="Your Name &lt;me@myhost.com>"/>
+ <property name="user" value="anoncvs"/>
+ <property name="password" value="anoncvs"/>
+ <property name="root" value="/home/cvspublic"/>
+ <property name="host" value="cvs.myhost.com"/>
+ <property name="module" value="myproject"/>
+
+ <target name="notify" depends="local.notify, email.notify"/>
+ <import file="${env.FORREST_HOME}/../../../scratchpad/forrestbot2/core/forrestbot.xml"/>
+</project>
+
+</source>
+
+ <p>First, set properties needed by the workstages you are going to use.
+ Here, we set properties that will be used by notify.email and getsrc.cvs.
+ Next, specify what implementations will be used by each workstage. By
+ default, 'local' will be used.</p>
+
+ <table>
+ <tr>
+ <th>Workstage</th>
+
+ <th>Implementations</th>
+ </tr>
+
+ <tr>
+ <td>getsrc</td>
+
+ <td><ul><li>local</li><li>cvs</li></ul></td>
+ </tr>
+
+ <tr>
+ <td>site</td>
+
+ <td><em>forrest itself</em></td>
+ </tr>
+
+ <tr>
+ <td>deploy</td>
+
+ <td><ul><li>local</li></ul></td>
+ </tr>
+
+ <tr>
+ <td>notify</td>
+
+ <td><ul><li>local</li><li>email</li></ul></td>
+ </tr>
+ </table>
+
+ <p>If you want to do more advanced processing for your project, you can
+ override the 'main' target, which by default is <code><target
+ name="main" depends="getsrc, site, deploy, notify"/></code>,
+ create your own implementation of a workstage, or use any other ant tasks
+ to do additional work.</p>
</section>
+
<section>
- <title>ForrestBot design</title>
- <p>Most of us will just like things to work, and will be happy enough
- just using it, maybe even letting the forrest-dev
- <link href="site:mail-lists">mail list</link> know if we could improve
- this or the other. Some however might be drawn to the
- <link href="http://www.howstuffworks.com/">how stuff works</link> of things.
- For them is this section as a start in the rest of their own pursuit.</p>
- <section>
- <title>The Ant play</title>
- <p>All actions and targets related to the actual execution of the bot's
- work is separated over 3 distinct build files:</p>
- <ul>
- <dl>
- <dt>[main] <code>./build.xml</code></dt>
- <dd>the starting point where generation and preparation is initiated.
- </dd>
- </dl>
- <dl>
- <dt>[generated] <code>./build/bot/work.build.xml</code></dt>
- <dd>the one that reflects the configured tasks in the
- forrestbot.conf.xml</dd>
- </dl>
- <dl>
- <dt>[templates]
- <code>./src/resources/forrestbot/ant/templates.build.xml</code></dt>
- <dd>listing the different targets that actually depict the supported
- workstages (and types of)</dd>
- </dl>
- </ul>
- <p>The complete process runs as follows:</p>
- <ol>
- <li>'bot' target inside main build.xml depends on a number of targets
- that</li>
- <ol>
- <li>'bot.init': initialize the environment</li>
- <li>'bot.conf2build': use XSLT stylesheets to convert the
- forrestbot.conf.xml into the work.build.xml (and a properties file for the
- default-section)</li>
- <li>'bot.run': delegates to the 'work' target of the generated
- work.build.xml</li>
- </ol>
- <li>'work' target inside generated work.build.xml will then start in
- parallel (so failure of the one would not influence the other) the different
- project specific work-processes. </li>
- <li>each of these project runs (generated 'work.[projectname]' targets)
- depends on the sequential execution of the different workstage actions
- (generated '[workstage].[projectname(.[type])?]' targets) </li>
- <li>In those specific targets the pattern is as follows:</li>
- <ol>
- <li>set the project specific parameters for this project and
- workstage</li>
- <li>load the default parameter values for the ones that aren't set
- yet</li>
- <li>call the actual template.[workstage(.[type]?)]</li>
- </ol>
- </ol>
- </section>
- <section>
- <title>Adding features to the bot - The meta template</title>
- <p>Given the mechanism, the actual things the bot <strong>can</strong>
- do boil down to the template targets it can call. So extending the bot means:
- adding templates-targets.</p>
- <p>If you want to add your own tempales to
- <code>template.build.xml</code>, take in account the following guidelines:</p>
- <ul>
- <li> in the future there will be a DTD for forrestbot.conf.xml that
- should be updated probably after you added stuff to the
- <code>templates.build.xml</code>. Backwards-incompatible changes need a formal
- vote.</li>
- <li>build an echo target for each template target you make: </li>
- </ul>
- <source>
-<![CDATA[
-<target name="echo.workstage-name.optional-type" if="template.echo">
- ... here you put echo ant-tasks that somewhat tell what you will
- ... do with all the params you expect
-</target>
-
-<target name="template.workstage-name.optional-type"
- depends="shared.set-props, echo.workstage-name.optional-type">
- ... here you put whatever ant-tasks that actually do it
-</target>
- ]]>
- </source>
- </section>
+ <title>Forrestbot design</title>
+
+ <p>Forrest and forrestbot use ant buildfiles extensively. Ant 1.6's
+ import task is used to import multiple buildfiles into a single build. The
+ following is the flow of control when running forrestbot:</p>
+
+ <ul>
+ <li>Your buildfile<ul><li>forrestbot.xml<ul><li>workstage buildfiles</li><li>forrest.build.xml</li></ul></li></ul></li>
+ </ul>
+
+ <p>The workstage buildfiles set up the properties and files so that the
+ main forrest buildfile (forrest.build.xml) will run. After it is run,
+ other workstages buildfiles can implement reporting, deployment, or other
+ post-build activities.</p>
+
+ <p>Your buildfile can specify which workstages you want to use, set
+ properties for them, and do any additional pre- and post-processing.</p>
</section>
</body>
-</document>
+</document>
\ No newline at end of file
1.34 +3 -2 xml-forrest/src/documentation/content/xdocs/site.xml
Index: site.xml
===================================================================
RCS file: /home/cvs/xml-forrest/src/documentation/content/xdocs/site.xml,v
retrieving revision 1.33
retrieving revision 1.34
diff -u -r1.33 -r1.34
--- site.xml 22 Nov 2003 05:55:35 -0000 1.33
+++ site.xml 4 Dec 2003 21:58:02 -0000 1.34
@@ -43,8 +43,9 @@
<dtd-docs label="DTD documentation" href="dtd-docs.html"/>
<cap label="Sourcetype Action" href="cap.html"/>
<forrestbar label="The ForrestBar" href="forrestbar.html"/>
- <forrestbot label="The Forrestbot" href="forrestbot.html"/>
- <forrestbot-intro href="forrestbot-intro.html"/>
+ <forrestbot label="Forrestbot (CVS)" href="forrestbot.html"/>
+ <forrestbot-old label="Forrestbot (stable)" href="forrestbot-old.html"/>
+ <forrestbot-intro-old href="forrestbot-intro-old.html"/>
<upgrading_05 label="Upgrading to 0.5" href="upgrading_05.html"/>
</documentation>
1.1 xml-forrest/src/documentation/content/xdocs/forrestbot-intro-old.xml
Index: forrestbot-intro-old.xml
===================================================================
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE document PUBLIC "-//APACHE//DTD Documentation V1.2//EN" "http://apache.org/forrest/dtd/document-v12.dtd">
<document>
<header>
<title>Forrestbot introduction</title>
<subtitle>Getting started with using Forrestbot</subtitle>
<version>0.2</version>
<authors>
<person name="David Crossley" email="crossley@apache.org"/>
</authors>
</header>
<body>
<warning>This documentation applies to the forrestbot included with Forrest 0.5.1 or previous releases.
The next version of forrest will include a new forrestbot, which has some documentation <link href="site:forrestbot">here</link>.
</warning>
<section>
<title>Goal</title>
<p>
You have your own project and you want to use Forrest to build the project
documentation.
</p>
</section>
<section>
<title>Introduction</title>
<p>
The <strong>ForrestBot</strong> will automatically build and publish your
documentation on a regular basis. The versatile ForrestBot can retrieve
source XML instances from various type of repository (e.g. local filesystem,
local CVS, remote CVS), generate the documents, and copy the result to a
local or remote location (optionally via secure methods). It can be run
from the command-line as part of your documentation edit-build-review cycle,
it can be called from cron to automate your website update, it can operate
as a centralised docs-build service for a set of remote projects, it can do
tasks in parallel, and do even more.
See some websites that are automatically built with ForrestBot at
<link href="http://forrestbot.cocoondev.org/">http://forrestbot.cocoondev.org/</link>
</p>
<p>
This document provides a concise overview of one particular scenario. See
<link href="site:forrestbot">The ForrestBot</link> for details and understand
<link href="site:forrest-contract">Our Contract</link>.
We assume that you have built and configured Forrest as described in
<link href="your-project.html">Using Forrest</link>.
</p>
<p>
Please send your feedback to <code>forrest-dev</code> - the ForrestBot
will be enhanced to meet diverse needs.
</p>
<p>
(Building the documentation for the actual Forrest project is a separate
issue. Use
<code>./build.sh site</code> ... that will provide overview
documentation about how Forrest operates and how you can help to improve it.)
</p>
</section>
<section>
<title>Establish project source directory</title>
<p>The directory that contains your XML instances.</p>
<ul>
<li><code>PROJECT_HOME = /home/you/yourproject</code></li>
<li><code>SRC_DOC = $PROJECT_HOME/src/documentation</code></li>
</ul>
<p>
Your XML instance documents are at <code>$SRC_DOC/content/xdocs</code>
</p>
</section>
<section>
<title>Establish project configuration</title>
<p>See the sample configurations at
<code>xml-forrest/src/resources/forrestbot/samples/</code>
copy one and edit it to replace the <code><![CDATA[<project>]]></code>
definitions with our own, i.e.
</p>
<source><![CDATA[
<project name="yourProject-local">
<prepare>
<skin name="forrest-site"/>
</prepare>
<get-src type="local-copy">
<project-dir name="/home/you/yourproject" />
<content-dir name="/home/you/yourproject/src/documentation"/>
</get-src>
<deploy type="local-copy">
<destination name="/var/www/html/yourproject"/>
</deploy>
</project>
]]></source>
</section>
<section>
<title>Run ForrestBot</title>
<source><![CDATA[
[you@locahost]$ cd $PROJECT_HOME
[you@locahost]$ forrestbot -Dbot.config=<name>
... where <name> is the pathname to your configuration file.
]]></source>
<p>There are also convenience scripts for automation.
See
<code>xml-forrest/src/resources/forrestbot/scripts/README.txt</code>
</p>
</section>
<section>
<title>Next Steps</title>
<p>Some enhancements to your ForrestBot would be ...</p>
<ul>
<li>Add your own skin as described in
<link href="your-project.html">Using Forrest</link>.
</li>
<li>Configure another <code><![CDATA[<project>]]></code> to conduct a secure
copy to your public website.
</li>
<li>Instruct ForrestBot to send you email about the activities.
</li>
<li>Manage your XML instances with CVS and instruct ForrestBot to get
the source.
</li>
<li>Use the "Forrestbot web interface" (website staging application).
<code>xml-forrest/src/resources/forrestbot/webapp/WEB-INF/README.txt</code>
</li>
</ul>
</section>
<section>
<title>Other methods</title>
<p>See the <link href="site:your-project">Using Forrest</link> document.</p>
<p>Forrest can also run as a dynamic webapp.</p>
</section>
</body>
</document>
1.1 xml-forrest/src/documentation/content/xdocs/forrestbot-old.xml
Index: forrestbot-old.xml
===================================================================
<?xml version="1.0"?>
<!DOCTYPE document PUBLIC "-//APACHE//DTD Documentation V1.2//EN" "http://apache.org/forrest/dtd/document-v12.dtd">
<document>
<header>
<title>The ForrestBot</title>
<authors>
<person name="Marc Portier" email="mpo@outerthought.org"/>
</authors>
<abstract>This document explains how the Forrest project can be used to
centrally build the documentation for different projects. It also explains
which pieces in the Forrest distribution form this
functionality, and how they could be extended in the future.</abstract>
</header>
<body>
<warning>This documentation applies to the forrestbot included with Forrest 0.5.1 and previous releases.
The next version of forrest will include a new forrestbot, which has some documentation <link href="site:forrestbot">here</link>.
</warning>
<section>
<title>Foreword</title>
<p>This document explains how the Forrest project can be used to
centrally build the documentation for different projects.
It also explains which pieces in the Forrest distribution form this
functionality, and how they could be extended in the future.</p>
<p>See some related documents:</p>
<ul>
<li><link href="site:forrest-contract">Our Contract</link>
- explains the rules with which your project should comply
</li>
<li><link href="site:forrestbot-intro-old">Forrestbot introduction</link>
- concise overview of Forrestbot
</li>
</ul>
</section>
<section>
<title>Using the Forrestbot</title>
<section>
<title>Setting up the centralized service</title>
<p>The goal is to publish a static website that can be updated
in near-real-time to accomodate newly available content. Forrestbot
can run on one machine, get content from another machine, and publish
to yet another machine.</p>
<p>Running the bot is as easy as calling the following in your shell.
(executed in the root dir of the distribution.)</p>
<source>forrestbot -Dbot.config=myOwn.conf.xml -Dtemplate.echo=[true|false]</source>
<p>It is quite typical to perform this from a shell script that gets
scheduled for execution on a regular basis.</p>
</section>
<section>
<title>Describing your project to the bot</title>
<fixme author="mpo">There is not yet a DTD for this configuration
file.</fixme>
<p>The actions the bot is going to take are defined in the XML
file pointed at with the <code>bot.config</code> System
property. (if not specified with <code>-Dbot.config</code>
then the default config from the
distribution is taken: <code>./forrestbot.conf.xml</code>) </p>
<p>This configuration file has one (required, but optionally empty) child-element for
setting cross-project defaults, followed by a list of
project-elements that hold the configuration for the various projects
that the bot is going to work on.</p>
<source>
<![CDATA[<?xml version="1.0"?>
<forrest-config>
<defaults>
...<!-- different fallback values for various workstages -->
</defaults>
<project name="project-name-1">
...<!-- different subelements configure the various workstages -->
</project>
<project name="project-name-2">
...<!-- different subelements configure the various workstages -->
</project>
</forrest-config>
]]>
</source>
<p>During the bot-run, each project entry will start a parallel and
independent process consisting of a number of so called 'workstages'.</p>
<p>These workstages are listed in the
<code>./src/resources/forrestbot/stages.conf.xml</code> file and their actual
execution is handled by the various template tasks in the
<code>./resources/forrestbot/ant/templates.build.xml</code> file. </p>
<p>Currently the various workstages are:</p>
<ul>
<dl>
<dt>prepare</dt>
<dd> Make a clean Forrest-defined Cocoon context environment.</dd>
</dl>
<dl>
<dt>get-src</dt>
<dd>Slide in the project specific document sources in that Cocoon
context environment.</dd>
</dl>
<dl>
<dt>generate</dt>
<dd> Actually generate the site using Cocoon</dd>
</dl>
<dl>
<dt>deploy</dt>
<dd>Publish the newly created site</dd>
</dl>
<dl>
<dt>cleanup</dt>
<dd><No real actions yet></dd>
</dl>
</ul>
<p>Inside the project element the configuration allows to set some
parameters that get consumed in the template targets. This workstage dependant
configuration always takes the following layout: an element with the name of
one of the tasks, and an optional type-attribute with nested elements that hold
parameter names, with required name attribute holding the actual parameter
value. Example:</p>
<source><project name="xml-forrest">
<prepare>
<skin name="forrest-site"/>
</prepare>
<get-src type="cvs">
<host name="cvs.apache.org"/>
<root name="/home/cvspublic"/>
<user name="anoncvs"/>
<passwd name="anoncvs"/>
<module name="xml-forrest"/>
<content-dir name="src/documentation"/>
<project-dir name=""/>
</get-src>
<generate>
<debuglevel name="ERROR" />
</generate>
<deploy type="scp">
<host name="krysalis.sourceforge.net"/>
<root name="/home/groups/k/kr/krysalis/htdocs"/>
<user name="forrestbot"/>
<dir name="forrest"/>
</deploy>
</project></source>
<note>Only the workstages that need configuration settings have an
entry here. At all times there will be a workstage template called for each
stage of the stages.conf.xml (even if it is not explicitly mentioned inside the
<project> element.) </note>
<p>Explained in detail, this will</p>
<ul>
<li>SET names properties (workstage(.@type)?.*) to appropriate values
(*/@name)</li>
<ul>
<dl>
<dt>prepare.skin</dt>
<dd>forrest-site</dd>
</dl>
<dl>
<dt>get-src.cvs.host</dt>
<dd>cvs.apache.org</dd>
</dl>
<dl>
<dt>get-src.cvs.root</dt>
<dd>/home/cvspublic</dd>
</dl>
<dl>
<dt>get-src.cvs.user</dt>
<dd>anoncvs</dd>
</dl>
<dl>
<dt>get-src.cvs.passwd</dt>
<dd>anoncvs</dd>
</dl>
<dl>
<dt>get-src.cvs.module</dt>
<dd>xml-forrest</dd>
</dl>
<dl>
<dt>get-src.cvs.content-dir</dt>
<dd>src/documentation</dd>
</dl><dl>
<dt>get-src.cvs.project-dir</dt>
<dd></dd>
</dl>
<dl>
<dt>generate.debuglevel</dt>
<dd>ERROR</dd>
</dl>
<dl>
<dt>deploy.scp.host</dt>
<dd>krysalis.sourceforge.net</dd>
</dl>
<dl>
<dt>deploy.scp.root</dt>
<dd>/home/groups/k/kr/krysalis/htdocs</dd>
</dl>
<dl>
<dt>deploy.scp.user</dt>
<dd>forrestbot</dd>
</dl>
<dl>
<dt>deploy.scp.dir</dt>
<dd>forrest</dd>
</dl>
</ul>
<li>and CALL the following templates
(template.workstage(.@type)?):</li>
<ul>
<li>template.prepare</li>
<li>template.get-src.cvs</li>
<li>template.generate</li>
<li>template.deploy.scp</li>
<li>template.cleanup</li>
</ul>
</ul>
<p>What each of these templates is going to do with the parameters can
be found in the templates.build.xml itself (at this stage) and is put out to
the console if the bot is run with the <code>-Dtemplate.echo=true</code>
flag</p>
<section>
<title>Setting bot-wide defaults</title>
<p>The <code><defaults></code> section in the
<code><forrest-config></code> allows for one-time setting of parameters
you want to reuse for different projects. The syntax (and semantic value) of
the nested elements is identical to the use inside the
<code><project></code> tags.</p>
</section>
</section>
<note>Once you understand all of this you might decide NOT to run your
own centralized service but rather ask your project to be taken up in the
centralized service of the project team itself: just send a mail to
forrest-dev <link href="site:mail-lists">mail list</link>
defining the required parameters to be taken up in the
process.</note>
<section>
<title>Checking logs and getting them by mail</title>
<p>For each distinct project the Forrest bot is maintaining a separate
working log of the different workstages being executed. These logs can be
opened and viewed in the ./build/bot directory</p>
<p>In addition to the @name attribute the <project> tag also can
contain a @sendlogto attribute that must be holding a (comma separated list
off) email address(es) where the project specific log needs to be sent to after
all workstages have been completed (or as soon as building failed)</p>
</section>
</section>
<section>
<title>Current supported workstages and types.</title>
<section>
<title>
<code><prepare></code>
</title>
<p>No specific type versions. (don't use type-attribute)</p>
<p>Template arguments:</p>
<ul>
<li><code><skin></code> @name holds the name of the skin to use.</li>
</ul>
</section>
<section>
<title>
<code><get-src ...></code>
</title>
<section>
<title>
<code><get-src type="cvs"></code>
</title>
<p>Template arguments:</p>
<ul>
<li><code><host></code> @name holds the ipaddress or dns name of the cvs serving host</li>
<li><code><root></code> @name holds the cvs-root directory on that host</li>
<li><code><user></code> @name holds the username to use on the cvs repository </li>
<li><code><passwd></code> @name holds the password to use on the cvs repository</li>
<li><code><module></code> @name holds the module name that holds the </li>
<li><code><dir></code> @name holds the relative path to the {docroot} directory. (This is the dir that is holding the ./content/xdocs as specified in t. </li>
</ul>
</section>
<section>
<title>
<code><get-src type="local-copy"></code>
</title>
<p>Template arguments:</p>
<ul>
<li><code><content-dir/></code> @name holds the path pointing to the {docroot} directory. (See the <link href="site:forrest-contract">contract</link> to understand what should be there.)</li>
<li><code><project-dir/></code> @name holds the path pointing to the {projecthome} directory where the xml project descriptors reside.</li>
</ul>
</section>
</section>
<section>
<title>
<code><generate></code>
</title>
<p>No specific type versions. (don't use type-attribute)</p>
<p>Template arguments:</p>
<ul>
<li><code><debuglevel></code> @name holds the threshold-level for showing debug statements in the cocoon generation process.</li>
</ul>
</section>
<section>
<title>
<code><deploy ...></code>
</title>
<section>
<title>
<code><deploy type="scp"></code>
</title>
<p>Template arguments:</p>
<ul>
<li><code><host></code> @name holds the remote host where the file will be copied to.</li>
<li><code><user></code> @name holds the username to be used when logging onto the remote host.</li>
<li><code><root></code> @name holds the prefix part of where the content needs to be published.</li>
<li><code><dir></code> @name holds the suffix part of where the content needs to be published.</li>
</ul>
<warning>Using this approach requires that the public key of the user executing the bot target is present in the authorized key file (<code>~/.ssh/authorized_keys2</code>) of the remote user (on the remote host).</warning>
<note>Might be interesting to understand that the full process actually is bundling all files to copy in a *.tar.gz that is un-tar.gzipped on the remote host using ssh. </note>
</section>
<section>
<title>
<code><deploy type="local-copy"></code>
</title>
<p>Template arguments:</p>
<ul>
<li><code><destination></code> @name holds the path where the generated content needs to be copied to.</li>
</ul>
</section>
<section>
<title>
<code><deploy type="ftp"></code>
</title>
<p>Template arguments:</p>
<ul>
<li><code><host></code> @name holds the hostname of the ftp server to publish to.</li>
<li><code><user></code> @name holds the username to use for loging onto the ftp server.</li>
<li><code><passwd></code> @name holds the password to use.</li>
<li><code><destination></code> @name holds the path to the directory on the remote host where the generated content needs to be published. The process is using the <code>ftp cd</code> command to get there. This means this directory has to exist.</li>
</ul>
</section>
<warning>Currently the ant distribution that is included in forrest cvs is not offering the required NetComponents.jar to actually support this type of deployment. If you want to use it anyway you should get the required jar from <link href="http://www.savarese.org/oro/downloads/index.html#NetComponents">http://www.savarese.org/oro/downloads</link> and drop that into your <code>{forrest-sandbox}/tools/antipede/lib</code> directory. </warning>
</section>
<section>
<title>
<code><cleanup></code>
</title>
<p>No specific type versions. (don't use type-attribute)</p>
<p>Template arguments: None.</p>
</section>
</section>
<section>
<title>ForrestBot design</title>
<p>Most of us will just like things to work, and will be happy enough
just using it, maybe even letting the forrest-dev
<link href="site:mail-lists">mail list</link> know if we could improve
this or the other. Some however might be drawn to the
<link href="http://www.howstuffworks.com/">how stuff works</link> of things.
For them is this section as a start in the rest of their own pursuit.</p>
<section>
<title>The Ant play</title>
<p>All actions and targets related to the actual execution of the bot's
work is separated over 3 distinct build files:</p>
<ul>
<dl>
<dt>[main] <code>./build.xml</code></dt>
<dd>the starting point where generation and preparation is initiated.
</dd>
</dl>
<dl>
<dt>[generated] <code>./build/bot/work.build.xml</code></dt>
<dd>the one that reflects the configured tasks in the
forrestbot.conf.xml</dd>
</dl>
<dl>
<dt>[templates]
<code>./src/resources/forrestbot/ant/templates.build.xml</code></dt>
<dd>listing the different targets that actually depict the supported
workstages (and types of)</dd>
</dl>
</ul>
<p>The complete process runs as follows:</p>
<ol>
<li>'bot' target inside main build.xml depends on a number of targets
that</li>
<ol>
<li>'bot.init': initialize the environment</li>
<li>'bot.conf2build': use XSLT stylesheets to convert the
forrestbot.conf.xml into the work.build.xml (and a properties file for the
default-section)</li>
<li>'bot.run': delegates to the 'work' target of the generated
work.build.xml</li>
</ol>
<li>'work' target inside generated work.build.xml will then start in
parallel (so failure of the one would not influence the other) the different
project specific work-processes. </li>
<li>each of these project runs (generated 'work.[projectname]' targets)
depends on the sequential execution of the different workstage actions
(generated '[workstage].[projectname(.[type])?]' targets) </li>
<li>In those specific targets the pattern is as follows:</li>
<ol>
<li>set the project specific parameters for this project and
workstage</li>
<li>load the default parameter values for the ones that aren't set
yet</li>
<li>call the actual template.[workstage(.[type]?)]</li>
</ol>
</ol>
</section>
<section>
<title>Adding features to the bot - The meta template</title>
<p>Given the mechanism, the actual things the bot <strong>can</strong>
do boil down to the template targets it can call. So extending the bot means:
adding templates-targets.</p>
<p>If you want to add your own tempales to
<code>template.build.xml</code>, take in account the following guidelines:</p>
<ul>
<li> in the future there will be a DTD for forrestbot.conf.xml that
should be updated probably after you added stuff to the
<code>templates.build.xml</code>. Backwards-incompatible changes need a formal
vote.</li>
<li>build an echo target for each template target you make: </li>
</ul>
<source>
<![CDATA[
<target name="echo.workstage-name.optional-type" if="template.echo">
... here you put echo ant-tasks that somewhat tell what you will
... do with all the params you expect
</target>
<target name="template.workstage-name.optional-type"
depends="shared.set-props, echo.workstage-name.optional-type">
... here you put whatever ant-tasks that actually do it
</target>
]]>
</source>
</section>
</section>
</body>
</document>
1.257 +6 -3 xml-forrest/status.xml
Index: status.xml
===================================================================
RCS file: /home/cvs/xml-forrest/status.xml,v
retrieving revision 1.256
retrieving revision 1.257
diff -u -r1.256 -r1.257
--- status.xml 29 Nov 2003 15:38:59 -0000 1.256
+++ status.xml 4 Dec 2003 21:58:03 -0000 1.257
@@ -15,6 +15,7 @@
<person name="Marc Portier" email="mpo@apache.org" id="MP" />
<person name="Jeff Turner" email="jefft@apache.org" id="JT" />
<person name="Juan Jose Pablos" email="cheche@apache.org" id="JJP" />
+ <person name="Dave Brondsema" email="brondsem@apache.org" id="DB" />
<person name="Volunteer needed" email="forrest-dev@xml.apache.org" id="open"/>
</developers>
@@ -25,6 +26,9 @@
<changes>
<release version="0.6-dev" date="unreleased">
+ <action dev="DB" type="add" context="docs">
+ Began documentation for new forrestbot; moved stable forrestbot docs
+ </action>
<action dev="NKB" type="fix" context="skins" fixes-bug="FOR-87,FOR-88"
due-to="Ross Gardler" due-to-email="ross@saafe.org" >
Now skins that are edited in place have their non-traversable
@@ -717,8 +721,7 @@
href="site:linking">Menus and Linking</link> for more info.
</action>
<action dev="DC" type="update" context="forrestbot">
- Updated some documentation, especially
- <link href="forrestbot-intro.html">Forrestbot Introduction</link>
+ Updated some documentation, especially Forrestbot Introduction
</action>
<action dev="NKB" type="fix" context="shbat">
Now files present in the content dir are correctly linked and