[Fwd: RefDoc - Neutral XML Document Format Input/Current State]

[Ross Gardler <rg...@apache.org>]

Bertrand is mentoring a GSoC project called RefDoc over at Cocoon. I'm 
listed as a co-mentor, but my involvement is only really to help 
Bertrand pick up the slack whilst he is busy. This is Bertrands project, 
not mine. Having said that, I think this is very useful for Forrest as well.

The project is intended to make Cocoon applications self documenting by 
adding javadoc like comments to the relevant files.

Below is a mail from Robert (the person doing the real work) asking for 
some feedback. It would be great if Forrest folk could have a look at 
this and let me know if there is anything they would like to see from a 
Forrest perspective. Please reply here I will pass on a summary of our 
observations to Robert.

Ross


-------- Original Message --------
From: Robert Graham <in...@gmail.com>
Reply-To: Robert Graham <in...@gmail.com>
To: dev@cocoon.apache.org
Subject: RefDoc - Neutral XML Document Format Input/Current State

Hello All,

I'd like any and all input on this as it would make it much better
than I will alone. However, I've sketched out something to start from.
First I'll give a brief description of what I'm talking about for
those that haven't kept up with RefDoc, then I'll present my ideas, and
finally I'll ask some questions that I've thought about in creating
these semantics. If you're current on RefDoc skip the next paragraph.
I apologize fo my long winded discussion.

RefDoc intersperses comments throughout various types of files (java,
xml, xsl, etc.) that are to be used as a form of documentation. They
aren't meant to replace javadoc, but to supplement it especially in
those cases where the files are not java files. These comments have
varying bits of metadata associated with them. The two pieces of
metadata that are well-defined for use now are 'key' and 'type'. In
publishing these documents they are sorted into groups of comments or
snippets with the same key being one document and then they are to be
sorted in some way by type.

I've created several types and other proposed 'well-defined' bits of
metadata to assist the publishing of these documents into a nice
format and easing the pain of parsing out such information. The types
are:

*method* - this type is for documenting methods or functions

*package* - this type indicates that the comment contains only the
package string for the file and perhaps the leading 'package'
declaration in java or a trailing semicolon

*class-declaration* - this type contains only the class declaration
(and maybe the trailing '{')

*codeblock* - this type is for indicating that the comment contains
code for publishing purposes

*name* - this type indicates the 'name' of the document of thing being
documented under
the current key (or specified key for this name

*warning* - this type expresses warnings

*example* - this type is used to indicate that the comment contains an 
example

*details* - this type must include another piece of metadata
indicating what it is detailing more on this in a minute

*variable* - this type is used to document a variable or parameter

*description* - this type is used to describe what the purpose of the
current thing being documented is as per the current/specified key

*see-also* - this type specifies that it contains a listing of doktor
keys that you might also want to look at/search for

A few new pieces of metadata:

*method*
*codeblock*
*example* (?) - those metadata are used for naming a comment
describing a method, example, or codeblock so that you can supply the
same metadata to "detail" comment types or other such constructions
for expounding on things. Perhaps even associating examples with
codeblocks or methods. This adds some possibilities to how things are
published but may introduce too much complexity.

*filetype* - this is usually supplied with the key for the document
and indicates what kind of document it is which may be used in a
publishing context

Issue #1:
The need to associate a codeblock with an example comes from
publishing interests. If I have an example with a block of text
explaining things and then a demostration in code then without
separating them but keeping them associated publishing it correctly
becomes a nightmare. I'm afraid though that this may introduce too
much complexity to deal with.

Issue #2:
Of course simple having to index so many different metadata keys might
be annoying and they should perhaps all be changed to one 'id' or
'name' field. I feel this should be done.

Issue #3:
I figured that the layout of the neutral XML would be similar to how a
javadoc document is arranged with the name and packge followed by a
description and then examples/methods/variables followed finally by
details and perhaps codeblocks. This may be too similar to javadoc,
however and I'd like input on formatting decisions.

Finally, any further comments, ideas, or discussion is welcomed.
Thanks for reading.

Cheers,
Robert Graham

Re: [Fwd: RefDoc - Neutral XML Document Format Input/Current State]

[Ross Gardler <rg...@apache.org>]

Diwaker Gupta wrote:
> This sounds exciting. Do we have access to any example/samplout output of 
> RefDoc?

http://svn.apache.org/repos/asf/cocoon/gsoc/rgraham/refdoc/

> One of the interesting possibilities is to generate examples automagically. 
> For instance by correlating the Java source file for a Cocoon Generator with 
> its usage in a processing pipeline.

Yes, I'll pass this over (of course, if devs are particularly interested
you can communicate directly with Robert on the Cocoon list).

> w.r.t Forrest, it would be nice to have a way of generating more friendly 
> documentation of the DTDs. The current DTD documentation is good, but I think 
> it could get better. Right now its a very "mechanical" kind of documentation 
> -- just listing the attributes and children and parents and so on. A little 
> more annotated DTD documentation would be great (what does each element mean, 
> what are recommended usages and so on).

I suspect this would be out of the scope of this project because DTD's
are not XML files and therefore would need a whole new parsing system.

It's a great idea though so I will passs that on as well.

Ross

> 
> On Monday 01 August 2005 3:49 am, Ross Gardler wrote:
> 
>>Bertrand is mentoring a GSoC project called RefDoc over at Cocoon. I'm
>>listed as a co-mentor, but my involvement is only really to help
>>Bertrand pick up the slack whilst he is busy. This is Bertrands project,
>>not mine. Having said that, I think this is very useful for Forrest as
>>well.
>>
>>The project is intended to make Cocoon applications self documenting by
>>adding javadoc like comments to the relevant files.
>>
>>Below is a mail from Robert (the person doing the real work) asking for
>>some feedback. It would be great if Forrest folk could have a look at
>>this and let me know if there is anything they would like to see from a
>>Forrest perspective. Please reply here I will pass on a summary of our
>>observations to Robert.
>>
>>Ross

Re: [Fwd: RefDoc - Neutral XML Document Format Input/Current State]

[David Crossley <cr...@apache.org>]

Cyriaque Dupoirieux wrote:
> Diwaker Gupta a ?crit :
> 
> >This sounds exciting. Do we have access to any example/samplout output of 
> >RefDoc?
> >
> >One of the interesting possibilities is to generate examples 
> >automagically. For instance by correlating the Java source file for a 
> >Cocoon Generator with its usage in a processing pipeline.
> >
> >w.r.t Forrest, it would be nice to have a way of generating more friendly 
> >documentation of the DTDs. The current DTD documentation is good, but I 
> >think it could get better.

Please see the DTDx plugin todo notes [1] and issue FOR-221 [2].

[1] http://forrest.apache.org/pluginDocs/plugins_0_70/org.apache.forrest.plugin.input.dtdx/todo.html
[2] http://issues.apache.org/jira/browse/FOR-221
    "NekoDTD generated dtd documentation needs improvement, some bugs".

The NekoDTD parser author would be pleased to receive patches.

I agree that other representations of the DTDs could also help.

> Right now its a very "mechanical" kind of 
> >documentation -- just listing the attributes and children and parents and 
> >so on. A little more annotated DTD documentation would be great (what does 
> >each element mean, what are recommended usages and so on).
>
> Excellent idea, I remember - when I was young - that I spent lot of time 
> to understand how I could have the former link and fork behaviour 
> whereas we only have <a> tag in document-v20.
> Maybe a little comment of the author used to be displayed in the 
> generated documentation should have helped me...

However the documented examples would have helped.
[3] http://forrest.apache.org/dtdx/document-v20.html#changes

Anyway, it will be very good if the RefDoc work can help
with enhanced DTD documentation.

How far can we go there? I mean, if the contents of [3] were
all embedded in the actual DTDs, then the DTDs would become
rather bloated. Perhaps have some minor explanations, but
still use specific example documents for the full story.

David

Re: [Fwd: RefDoc - Neutral XML Document Format Input/Current State]

[Cyriaque Dupoirieux <Cy...@pcotech.fr>]

Diwaker Gupta a écrit :

>This sounds exciting. Do we have access to any example/samplout output of 
>RefDoc?
>
>One of the interesting possibilities is to generate examples automagically. 
>For instance by correlating the Java source file for a Cocoon Generator with 
>its usage in a processing pipeline.
>
>w.r.t Forrest, it would be nice to have a way of generating more friendly 
>documentation of the DTDs. The current DTD documentation is good, but I think 
>it could get better. Right now its a very "mechanical" kind of documentation 
>-- just listing the attributes and children and parents and so on. A little 
>more annotated DTD documentation would be great (what does each element mean, 
>what are recommended usages and so on).
>  
>
Excellent idea, I remember - when I was young - that I spent lot of time 
to understand how I could have the former link and fork behaviour 
whereas we only have <a> tag in document-v20.
Maybe a little comment of the author used to be displayed in the 
generated documentation should have helped me...

Cyriaque,

>On Monday 01 August 2005 3:49 am, Ross Gardler wrote:
>  
>
>>Bertrand is mentoring a GSoC project called RefDoc over at Cocoon. I'm
>>listed as a co-mentor, but my involvement is only really to help
>>Bertrand pick up the slack whilst he is busy. This is Bertrands project,
>>not mine. Having said that, I think this is very useful for Forrest as
>>well.
>>
>>The project is intended to make Cocoon applications self documenting by
>>adding javadoc like comments to the relevant files.
>>
>>Below is a mail from Robert (the person doing the real work) asking for
>>some feedback. It would be great if Forrest folk could have a look at
>>this and let me know if there is anything they would like to see from a
>>Forrest perspective. Please reply here I will pass on a summary of our
>>observations to Robert.
>>
>>Ross
>>    
>>

Re: [Fwd: RefDoc - Neutral XML Document Format Input/Current State]

[Diwaker Gupta <di...@apache.org>]

This sounds exciting. Do we have access to any example/samplout output of 
RefDoc?

One of the interesting possibilities is to generate examples automagically. 
For instance by correlating the Java source file for a Cocoon Generator with 
its usage in a processing pipeline.

w.r.t Forrest, it would be nice to have a way of generating more friendly 
documentation of the DTDs. The current DTD documentation is good, but I think 
it could get better. Right now its a very "mechanical" kind of documentation 
-- just listing the attributes and children and parents and so on. A little 
more annotated DTD documentation would be great (what does each element mean, 
what are recommended usages and so on).

On Monday 01 August 2005 3:49 am, Ross Gardler wrote:
> Bertrand is mentoring a GSoC project called RefDoc over at Cocoon. I'm
> listed as a co-mentor, but my involvement is only really to help
> Bertrand pick up the slack whilst he is busy. This is Bertrands project,
> not mine. Having said that, I think this is very useful for Forrest as
> well.
>
> The project is intended to make Cocoon applications self documenting by
> adding javadoc like comments to the relevant files.
>
> Below is a mail from Robert (the person doing the real work) asking for
> some feedback. It would be great if Forrest folk could have a look at
> this and let me know if there is anything they would like to see from a
> Forrest perspective. Please reply here I will pass on a summary of our
> observations to Robert.
>
> Ross
-- 
Web/Blog/Gallery: http://floatingsun.net

Re: [Fwd: RefDoc - Neutral XML Document Format Input/Current State]

[Nicola Ken Barozzi <ni...@apache.org>]

Ross Gardler wrote:
> Bertrand is mentoring a GSoC project called RefDoc over at Cocoon. I'm
> listed as a co-mentor, but my involvement is only really to help
> Bertrand pick up the slack whilst he is busy. This is Bertrands project,
> not mine. Having said that, I think this is very useful for Forrest as
> well.
> 
> The project is intended to make Cocoon applications self documenting by
> adding javadoc like comments to the relevant files.
...

This is a perfect use-case for Forrest, with the preprocessing plugin I
talked about at ApacheConEU. We'll see how he manages it and then work
from there.

You can point him to our whiteboard where there is already something to
look at... or maybe it's better to not do it as what is there sucks a
bit ;-)

> Issue #2:
> Of course simple having to index so many different metadata keys might
> be annoying and they should perhaps all be changed to one 'id' or
> 'name' field. I feel this should be done.

Same thought here... only writing real code will make us understand what
is best.

> Issue #3:
> I figured that the layout of the neutral XML would be similar to how a
> javadoc document is arranged with the name and packge followed by a
> description and then examples/methods/variables followed finally by
> details and perhaps codeblocks. This may be too similar to javadoc,
> however and I'd like input on formatting decisions.

The more similar to Javadoc, the better. It's tried and tested, people
are used to it and it can be transformed well in a helpset.

-- 
Nicola Ken Barozzi                   nicolaken@apache.org
            - verba volant, scripta manent -
   (discussions get forgotten, just code remains)
---------------------------------------------------------------------