Documentation generation

[Robert Greig <ro...@gmail.com>]

Hi all,

I think it is perhaps worth reconsidering how we generate our
documentation. I think it would be good to have an approach that
supported the following:

1) storing the documentation "source" in svn, with the code. That
means it can be versioned and diffed easily

2) ability to generate at high quality HTML and PDF versions of the
documentation

The Hibernate and Spring projects use docbook as a format and IMO
their documentation is excellent.

One other option I have heard mentioned is Forrest
(http://forrest.apache.org) but I don't have any experience of that.
Can anyone share any opinions of it?

RG

Re: Documentation generation

[Martin Ritchie <ri...@apache.org>]

On 11/09/2007, Jonathan Robie <jo...@redhat.com> wrote:
> Robert Godfrey wrote:
> > +1 to the whole idea, but particularly Martin's comment that we need to
> > *create* documentation first.
> >
> > Can we agree on using docbook as a format, and forrest for site generation?
> >
>
> Personally, I'm not sure that I have enough of an overview of what we
> want to accomplish. What kinds of documentation will we have, and how
> many pages total? Do we expect to be maintaining it for several versions
> at a time?

Yes 0_8/0_9 (M2) and 0_10+ (M3)

> > I would like to see what a forrest generated site would look like... having
> > something to review will also expose us to how little documentation we
> > really have, I think.
> >
> I imagine Forrest is quite flexible in the formats it can generate.
> Personally, I'd like to start with a list of what documentation we want
> to have, our plan for maintaining multiple versions of the documentation
> as we go, the formats we want to produce, etc. and then pick the tool.
>
> I'm new here, perhaps you already have that planned out. But to me, it's
> usually a bad idea to pick the tool first and then figure out what you
> want to do with it. If we don't know yet, I'd keep it simple, using
> DocBook and simple ant scripts to get some documentation done to start
> with, and keeping our options open for the big picture to come.

>From what I see forrest site generation is as easy as

$forrest site

Like maven you put your docs in a standard location and it picks it all up.

> > IMHO one of the key requirements for making this project a success will be
> > the quality of our documentation.  Right now I see the lack of documentation
> > as a clear barrier to adoption of Qpid.
> >
>
> I agree.

+1
This has to be our first focus. We can see what the generating tools
give us be it forrest, ant scripts or a.n. other tool at a later date.

> Jonathan
>


-- 
Martin Ritchie

Re: Documentation generation

[Jonathan Robie <jo...@redhat.com>]

Robert Godfrey wrote:
> On 11/09/2007, Jonathan Robie <jo...@redhat.com> wrote:
>   
>> OK. Is there anyone to do a prototype using Forrest?
>>     
>
> Between myself,  Martin, Rupert and Aidan we'll try to get some prototype
> work done...
>
> My main concern is that all documentation be available in formats like
>   
>> DocBook, and not tied to any particular system. I don't care that much
>> which system we use, as long as all data is available in useful formats
>> so that it can easily be repurposed outside of any system we use.
>>     
> Absolutely - I don't want us to be tied to any system... the docs should all
> formats and locations that make sense without the use of a tool...  I see
> using forrest more as being a jumpstart than anything else...
>   

Then I'm fine with this plan. And I do like the way these 
Forrest-generated sites seem to look.

Jonathan

Re: Documentation generation

[Robert Godfrey <ro...@gmail.com>]

On 11/09/2007, Jonathan Robie <jo...@redhat.com> wrote:
>
> Robert Godfrey wrote:
> > I would like us to use forrest to get started with generating site
> documents
> > which include docs from multiple source (e.g. doxygen and javadoc as
> well as
> > docbook files in the svn repository)...  If this doesn;t work out then
> we
> > can change to an ant based system or whatever.  I'm not religious about
> the
> > tooling... just want us to get started with some sort of framework in
> place.
> >
>
> OK. Is there anyone to do a prototype using Forrest?



Between myself,  Martin, Rupert and Aidan we'll try to get some prototype
work done...

My main concern is that all documentation be available in formats like
> DocBook, and not tied to any particular system. I don't care that much
> which system we use, as long as all data is available in useful formats
> so that it can easily be repurposed outside of any system we use.


Absolutely - I don't want us to be tied to any system... the docs should all
formats and locations that make sense without the use of a tool...  I see
using forrest more as being a jumpstart than anything else...

-- Rob


Jonathan
>

Re: Documentation generation

[Jonathan Robie <jo...@redhat.com>]

Robert Godfrey wrote:
> I would like us to use forrest to get started with generating site documents
> which include docs from multiple source (e.g. doxygen and javadoc as well as
> docbook files in the svn repository)...  If this doesn;t work out then we
> can change to an ant based system or whatever.  I'm not religious about the
> tooling... just want us to get started with some sort of framework in place.
>   

OK. Is there anyone to do a prototype using Forrest?

My main concern is that all documentation be available in formats like 
DocBook, and not tied to any particular system. I don't care that much 
which system we use, as long as all data is available in useful formats 
so that it can easily be repurposed outside of any system we use.

Jonathan

Re: Documentation generation

[Robert Godfrey <ro...@gmail.com>]

On 11/09/2007, Jonathan Robie <jo...@redhat.com> wrote:
>
> Martin Ritchie wrote:
> >
> > As Rob says the important thing is to get aggreement that Docbook is
> > the way to go forward as we should have a single documentation format
> > for all languages.
> >
>
> That part I agree with.
>
> When you say "a single documentation format for all languages", does
> that imply that API documentation will also be brought into DocBook?


API documentation which is within the code (e.g. javadoc) should stay within
the code and not be separately checked in to svn (obviously)... End formats
for this documentation will be PDF, html... etc...

I would like us to use forrest to get started with generating site documents
which include docs from multiple source (e.g. doxygen and javadoc as well as
docbook files in the svn repository)...  If this doesn;t work out then we
can change to an ant based system or whatever.  I'm not religious about the
tooling... just want us to get started with some sort of framework in place.

-- Rob


Jonathan
>

Re: Documentation generation

[Jonathan Robie <jo...@redhat.com>]

Martin Ritchie wrote:
>
> As Rob says the important thing is to get aggreement that Docbook is
> the way to go forward as we should have a single documentation format
> for all languages.
>   

That part I agree with.

When you say "a single documentation format for all languages", does 
that imply that API documentation will also be brought into DocBook?

Jonathan

Re: Documentation generation

[Martin Ritchie <ri...@apache.org>]

On 11/09/2007, Robert Godfrey <ro...@gmail.com> wrote:
> On 11/09/2007, Jonathan Robie <jo...@redhat.com> wrote:
> >
> > Robert Godfrey wrote:
> > > +1 to the whole idea, but particularly Martin's comment that we need to
> > > *create* documentation first.
> > >
> > > Can we agree on using docbook as a format, and forrest for site
> > generation?
> > >
> >
> > Personally, I'm not sure that I have enough of an overview of what we
> > want to accomplish. What kinds of documentation will we have, and how
> > many pages total? Do we expect to be maintaining it for several versions
> > at a time?
>
>
> Certainly I expect us to be maintaining several versions at a time (we
> already are!)...
>
>
> > I would like to see what a forrest generated site would look like...
> > having
> > > something to review will also expose us to how little documentation we
> > > really have, I think.
> > >
> > I imagine Forrest is quite flexible in the formats it can generate.
> > Personally, I'd like to start with a list of what documentation we want
> > to have, our plan for maintaining multiple versions of the documentation
> > as we go, the formats we want to produce, etc. and then pick the tool.
>
>
>
> I think we are trying to use the tool to jumpstart our process.  We have
> been spectacularly bad at organising our documentation up until now...  As
> long as forrest doesn't impose significantly more constraints than using
> ant, then I think that using forrest is probably a better decision than
> starting building our own tool in ant :-)
>
>
>
>
> I'm new here, perhaps you already have that planned out. But to me, it's
> > usually a bad idea to pick the tool first and then figure out what you
> > want to do with it. If we don't know yet, I'd keep it simple, using
> > DocBook and simple ant scripts to get some documentation done to start
> > with, and keeping our options open for the big picture to come.
>
>
>
> Using a tool from Apache to achieve a task seems like a more "community"
> thing to do, at the very least, rather than inventing our own...  If we find
> Forrest doesn't meet our needs then we can move to another tool (or no tool
> at all).
>
> I think the important thing is to get past the tooling discussing and start
> doing the documentation.  I think we have people who are willing to work on
> getting Forrest set up.
>
> -- Rob
>
> > IMHO one of the key requirements for making this project a success will be
> > > the quality of our documentation.  Right now I see the lack of
> > documentation
> > > as a clear barrier to adoption of Qpid.
> > >
> >
> > I agree.
> >
> > Jonathan

As Rob says the important thing is to get aggreement that Docbook is
the way to go forward as we should have a single documentation format
for all languages.

-- 
Martin Ritchie

Fwd: Documentation generation

[Robert Godfrey <ro...@gmail.com>]

On 11/09/2007, Jonathan Robie <jo...@redhat.com> wrote:
>
> Robert Godfrey wrote:
> > +1 to the whole idea, but particularly Martin's comment that we need to
> > *create* documentation first.
> >
> > Can we agree on using docbook as a format, and forrest for site
> generation?
> >
>
> Personally, I'm not sure that I have enough of an overview of what we
> want to accomplish. What kinds of documentation will we have, and how
> many pages total? Do we expect to be maintaining it for several versions
> at a time?


Certainly I expect us to be maintaining several versions at a time (we
already are!)...


> I would like to see what a forrest generated site would look like...
> having
> > something to review will also expose us to how little documentation we
> > really have, I think.
> >
> I imagine Forrest is quite flexible in the formats it can generate.
> Personally, I'd like to start with a list of what documentation we want
> to have, our plan for maintaining multiple versions of the documentation
> as we go, the formats we want to produce, etc. and then pick the tool.



I think we are trying to use the tool to jumpstart our process.  We have
been spectacularly bad at organising our documentation up until now...  As
long as forrest doesn't impose significantly more constraints than using
ant, then I think that using forrest is probably a better decision than
starting building our own tool in ant :-)




I'm new here, perhaps you already have that planned out. But to me, it's
> usually a bad idea to pick the tool first and then figure out what you
> want to do with it. If we don't know yet, I'd keep it simple, using
> DocBook and simple ant scripts to get some documentation done to start
> with, and keeping our options open for the big picture to come.



Using a tool from Apache to achieve a task seems like a more "community"
thing to do, at the very least, rather than inventing our own...  If we find
Forrest doesn't meet our needs then we can move to another tool (or no tool
at all).

I think the important thing is to get past the tooling discussing and start
doing the documentation.  I think we have people who are willing to work on
getting Forrest set up.

-- Rob

> IMHO one of the key requirements for making this project a success will be
> > the quality of our documentation.  Right now I see the lack of
> documentation
> > as a clear barrier to adoption of Qpid.
> >
>
> I agree.
>
> Jonathan
>

Re: Documentation generation

[Jonathan Robie <jo...@redhat.com>]

Robert Godfrey wrote:
> +1 to the whole idea, but particularly Martin's comment that we need to
> *create* documentation first.
>
> Can we agree on using docbook as a format, and forrest for site generation?
>   

Personally, I'm not sure that I have enough of an overview of what we 
want to accomplish. What kinds of documentation will we have, and how 
many pages total? Do we expect to be maintaining it for several versions 
at a time?

> I would like to see what a forrest generated site would look like... having
> something to review will also expose us to how little documentation we
> really have, I think.
>   
I imagine Forrest is quite flexible in the formats it can generate. 
Personally, I'd like to start with a list of what documentation we want 
to have, our plan for maintaining multiple versions of the documentation 
as we go, the formats we want to produce, etc. and then pick the tool.

I'm new here, perhaps you already have that planned out. But to me, it's 
usually a bad idea to pick the tool first and then figure out what you 
want to do with it. If we don't know yet, I'd keep it simple, using 
DocBook and simple ant scripts to get some documentation done to start 
with, and keeping our options open for the big picture to come.

> IMHO one of the key requirements for making this project a success will be
> the quality of our documentation.  Right now I see the lack of documentation
> as a clear barrier to adoption of Qpid.
>   

I agree.

Jonathan

Re: Documentation generation

[Robert Greig <ro...@gmail.com>]

On 11/09/2007, Robert Godfrey <ro...@gmail.com> wrote:

> Can we agree on using docbook as a format, and forrest for site generation?

+1 from me certainly.

> I would like to see what a forrest generated site would look like... having
> something to review will also expose us to how little documentation we
> really have, I think.

I was going to pull what we have from the wiki into some docbook files
as a starting point. I don't know much about docbook but I am an
olympic champion with ctrl-c and ctrl-v.

> IMHO one of the key requirements for making this project a success will be
> the quality of our documentation.  Right now I see the lack of documentation
> as a clear barrier to adoption of Qpid.

Yes, certainly compared with other products in this space.

Other AMQP implementations don't have good docs either so this is a
way of getting further ahead - we are already IMO ahead on performance
and client implementations.

RG

Re: Documentation generation

[Robert Godfrey <ro...@gmail.com>]

> I mentioned the use of Forrest, and to answer Jonathan's question I'd
> like to see Forrest take the burden of generating our entire site see
> lenya.apache.org for an example. While this would replace the wiki in
> many of the ways we currently use it I think that the wiki still has a
> benefit as Rajith points out for proposals/discussions.
>
> I don't think we should put any effort in to writing our own ant (or
> other) based system for converting the docbook in to html or other
> format. Even if it is only a 3 line ant file. We should decide what
> our target for this docbook documentation is and generate it for them.
>
> I would say that this would mean generation for:
> - Web (html)
> - Developers (local html and/or pdf)
> - Release (local html, pdf?)
>
> Our focus should of course be to actually _create_ the content first
> and think about the presentation once we have something we can
> actually show. I don't mind taking the few minutes to look in to
> Forrest and generate an example of what things might look like. Would
> be good to have some content though to do that with :)


+1 to the whole idea, but particularly Martin's comment that we need to
*create* documentation first.

Can we agree on using docbook as a format, and forrest for site generation?
I would like to see what a forrest generated site would look like... having
something to review will also expose us to how little documentation we
really have, I think.

IMHO one of the key requirements for making this project a success will be
the quality of our documentation.  Right now I see the lack of documentation
as a clear barrier to adoption of Qpid.

Your thoughts?

Rob


--
> Martin Ritchie
>

Re: Documentation generation

[Martin Ritchie <ri...@apache.org>]

On 10/09/2007, Rajith Attapattu <ra...@gmail.com> wrote:
> I suggested this a while ago, but the stock answer was we have a wiki.
> I am glad that you guys brought up this topic.
> A lot of apache projects have version specific documentation in svn and it
> has proven to be a sucessful model.
>
> So +1 to the idea.
>
> Rajith
>
> P.S I am not opposed to putting documentation in wiki.
> IMO wiki is a great tool to publish the proposals/static
> documentation/design diagrams etc..
>
> But we need version specific documentation that is in svn and bundle that
> when we do a release.
>
>
> On 9/10/07, Robert Greig <ro...@gmail.com> wrote:
> >
> > On 10/09/2007, Jonathan Robie <jo...@redhat.com> wrote:
> >
> > > The Boost folks have created a DTD called BoostBook [1]
> > > which extends DocBook to provide explicit support for C++ libraries,
> > > specifically to add support for C++ constructs such as namespaces,
> > > classes, overloaded functions, templates, and specializations. It also
> > > has some support for coordinating program examples with test suites. I'm
> > > not sure whether we should use the BoostBook extensions or not, but we
> > > should look into that question.
> >
> > From the description it sounds like it would be potentially useful for
> > the C++ components but not necessarily for the others.
> >
> > > I'm not sure whether BoostBook currently
> > > has stylesheets for generating PDF - these are widely available for
> > DocBook.
> >
> > I think pdf documentation is essential. I personally prefer pdf to
> > html and if available I always tend to choose it over html versions.
> >
> > > With either DocBook or BoostBook, we may want to consider simplifying
> > > the DTD by creating a subset - that will make it simpler for authors,
> > > and result in better document consistency by preventing people from
> > > doing the same thing in different ways.
> >
> > Yes, I think that sounds sensible.
> >
> > > We should probably divide the document into entities to make version
> > > control easier.
> >
> > Yes, Spring does this.
> >
> > > I don't think Forrest is an alternative to DocBook, DocBook is one
> > > format you can use when generating documents with Forrest. Forrest is an
> > > integration and publishing framework, and it may be overkill for our
> > > documentation project. I'd be inclined to start just using ant files to
> > > build documents, and get fancier if we need to.
> >
> > Yes, I agree. Someone mentioned Forrest to me (I had never looked at
> > it before) so I thought I would throw it out there to see if others
> > had any opinions on it.
> >
> > > Do we plan to publish API documentation with this system too, or will we
> > > rely on doxygen / javadoc / pydoc / whatever Ruby or other languages
> > use?
> >
> > I would say we should continue to use javadoc for java API
> > documentation since most developers and developer tools expect that.
> >
> > RG
> >

I mentioned the use of Forrest, and to answer Jonathan's question I'd
like to see Forrest take the burden of generating our entire site see
lenya.apache.org for an example. While this would replace the wiki in
many of the ways we currently use it I think that the wiki still has a
benefit as Rajith points out for proposals/discussions.

I don't think we should put any effort in to writing our own ant (or
other) based system for converting the docbook in to html or other
format. Even if it is only a 3 line ant file. We should decide what
our target for this docbook documentation is and generate it for them.

I would say that this would mean generation for:
- Web (html)
- Developers (local html and/or pdf)
- Release (local html, pdf?)

Our focus should of course be to actually _create_ the content first
and think about the presentation once we have something we can
actually show. I don't mind taking the few minutes to look in to
Forrest and generate an example of what things might look like. Would
be good to have some content though to do that with :)


-- 
Martin Ritchie

Re: Documentation generation

[Rajith Attapattu <ra...@gmail.com>]

I suggested this a while ago, but the stock answer was we have a wiki.
I am glad that you guys brought up this topic.
A lot of apache projects have version specific documentation in svn and it
has proven to be a sucessful model.

So +1 to the idea.

Rajith

P.S I am not opposed to putting documentation in wiki.
IMO wiki is a great tool to publish the proposals/static
documentation/design diagrams etc..

But we need version specific documentation that is in svn and bundle that
when we do a release.


On 9/10/07, Robert Greig <ro...@gmail.com> wrote:
>
> On 10/09/2007, Jonathan Robie <jo...@redhat.com> wrote:
>
> > The Boost folks have created a DTD called BoostBook [1]
> > which extends DocBook to provide explicit support for C++ libraries,
> > specifically to add support for C++ constructs such as namespaces,
> > classes, overloaded functions, templates, and specializations. It also
> > has some support for coordinating program examples with test suites. I'm
> > not sure whether we should use the BoostBook extensions or not, but we
> > should look into that question.
>
> From the description it sounds like it would be potentially useful for
> the C++ components but not necessarily for the others.
>
> > I'm not sure whether BoostBook currently
> > has stylesheets for generating PDF - these are widely available for
> DocBook.
>
> I think pdf documentation is essential. I personally prefer pdf to
> html and if available I always tend to choose it over html versions.
>
> > With either DocBook or BoostBook, we may want to consider simplifying
> > the DTD by creating a subset - that will make it simpler for authors,
> > and result in better document consistency by preventing people from
> > doing the same thing in different ways.
>
> Yes, I think that sounds sensible.
>
> > We should probably divide the document into entities to make version
> > control easier.
>
> Yes, Spring does this.
>
> > I don't think Forrest is an alternative to DocBook, DocBook is one
> > format you can use when generating documents with Forrest. Forrest is an
> > integration and publishing framework, and it may be overkill for our
> > documentation project. I'd be inclined to start just using ant files to
> > build documents, and get fancier if we need to.
>
> Yes, I agree. Someone mentioned Forrest to me (I had never looked at
> it before) so I thought I would throw it out there to see if others
> had any opinions on it.
>
> > Do we plan to publish API documentation with this system too, or will we
> > rely on doxygen / javadoc / pydoc / whatever Ruby or other languages
> use?
>
> I would say we should continue to use javadoc for java API
> documentation since most developers and developer tools expect that.
>
> RG
>

Re: Documentation generation

[Robert Greig <ro...@gmail.com>]

On 10/09/2007, Jonathan Robie <jo...@redhat.com> wrote:

> The Boost folks have created a DTD called BoostBook [1]
> which extends DocBook to provide explicit support for C++ libraries,
> specifically to add support for C++ constructs such as namespaces,
> classes, overloaded functions, templates, and specializations. It also
> has some support for coordinating program examples with test suites. I'm
> not sure whether we should use the BoostBook extensions or not, but we
> should look into that question.

>From the description it sounds like it would be potentially useful for
the C++ components but not necessarily for the others.

> I'm not sure whether BoostBook currently
> has stylesheets for generating PDF - these are widely available for DocBook.

I think pdf documentation is essential. I personally prefer pdf to
html and if available I always tend to choose it over html versions.

> With either DocBook or BoostBook, we may want to consider simplifying
> the DTD by creating a subset - that will make it simpler for authors,
> and result in better document consistency by preventing people from
> doing the same thing in different ways.

Yes, I think that sounds sensible.

> We should probably divide the document into entities to make version
> control easier.

Yes, Spring does this.

> I don't think Forrest is an alternative to DocBook, DocBook is one
> format you can use when generating documents with Forrest. Forrest is an
> integration and publishing framework, and it may be overkill for our
> documentation project. I'd be inclined to start just using ant files to
> build documents, and get fancier if we need to.

Yes, I agree. Someone mentioned Forrest to me (I had never looked at
it before) so I thought I would throw it out there to see if others
had any opinions on it.

> Do we plan to publish API documentation with this system too, or will we
> rely on doxygen / javadoc / pydoc / whatever Ruby or other languages use?

I would say we should continue to use javadoc for java API
documentation since most developers and developer tools expect that.

RG

Re: Documentation generation

[Jonathan Robie <jo...@redhat.com>]

I agree 100% with the two requirements listed below.

DocBook, which was designed largely for technical documentation, 
articles, and books, is also used by O'Reilly for many of their 
technical books, and is widely supported by XML tools. The stylesheets 
for conversion to HTML and PDF are widely available and are 
customizable. The Boost folks have created a DTD called BoostBook [1] 
which extends DocBook to provide explicit support for C++ libraries, 
specifically to add support for C++ constructs such as namespaces, 
classes, overloaded functions, templates, and specializations. It also 
has some support for coordinating program examples with test suites. I'm 
not sure whether we should use the BoostBook extensions or not, but we 
should look into that question. I'm not sure whether BoostBook currently 
has stylesheets for generating PDF - these are widely available for DocBook.

With either DocBook or BoostBook, we may want to consider simplifying 
the DTD by creating a subset - that will make it simpler for authors, 
and result in better document consistency by preventing people from 
doing the same thing in different ways.

We should probably divide the document into entities to make version 
control easier.

I don't think Forrest is an alternative to DocBook, DocBook is one 
format you can use when generating documents with Forrest. Forrest is an 
integration and publishing framework, and it may be overkill for our 
documentation project. I'd be inclined to start just using ant files to 
build documents, and get fancier if we need to.

Do we plan to publish API documentation with this system too, or will we 
rely on doxygen / javadoc / pydoc / whatever Ruby or other languages use?

Jonathan


> Hi all,
>
> I think it is perhaps worth reconsidering how we generate our
> documentation. I think it would be good to have an approach that
> supported the following:
>
> 1) storing the documentation "source" in svn, with the code. That
> means it can be versioned and diffed easily
>
> 2) ability to generate at high quality HTML and PDF versions of the
> documentation
>
> The Hibernate and Spring projects use docbook as a format and IMO
> their documentation is excellent.
>
> One other option I have heard mentioned is Forrest
> (http://forrest.apache.org) but I don't have any experience of that.
> Can anyone share any opinions of it?
>
> RG
>
>