You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ant.apache.org by Kev Jackson <ke...@it.fts-vn.com> on 2006/01/12 06:15:04 UTC
sandbox gendoc
Hi,
I've had a look through the sandbox and I noticed Dominque's (sp?) xml'd
version of the docs.
As you know I had a play with DocBook-izing the manual last year and in
the end the effort of manually altering every html page to be valid
DocBook xml drove me mad (gibber etc). Also there seemed to be a
consensus (or rather a few well informed arguments against DocBook in
general) that DocBook was not the way to go for the ant manual[1]
So I've had a look at the gendoc version (all 2 manual pages + whatsnew)
and I thought that if this is at least in the sandbox, it's 99% more
official than choosing a.n.other xml.
Given this I'm preparing a little script to automate the conversion of
the html docs to the xmlised version in the sandbox.
So far I'm trying to get it to work correctly with echo, but most of the
code is generic so when I get a passable version of that I'll refactor
(a lot) and run it against the doc directory.
Given the formatting issues of html, I expect at least a little manual
intervention along the way, but manually converting all the docs would
be insane (there's over 400 in the core tasks directory alone)
Anyone interested in the results?
Kev
[1] http://marc.theaimsgroup.com/?l=ant-dev&m=111035433410638&w=2
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: sandbox gendoc
Posted by cs...@arielpartners.com.
>> Yes, DocBook, also a standard, is indeed very verbose and doesn't seem
>> to have the favor among commiters.
>
> I think think docbook is interesting but something that should be hidden
> from users by a docbook editor, not handwritten. Also its too low-level;
> I want to mark something as a <task>, not as a bit of code.
Agree 100%. I went through exactly this thought process myself.
I wanted to reuse Docbook as much as possible, but wanted a custom schema to
represent things like Design Patterns, Business Letters, Resumes, etc.
I found the answer: use Docbook 5.0 (currently in beta). Docbook5 uses
RelaxNG schemas and namespaces rather than DTDs.
I use Jing for validation, though libxml and Sun MSV work fine, too.
Each schema (pattern, letter, resume, ant task...) is implemented using a
small relaxng file that customizes docbook.rng and a small XSLT that
customizes docbook.xsl. The customizations are usually less than 100 LOC!
Here is a cut-down example, for a letter:
<grammar>
<include href="docbook.rng">
<start>
<element name="l:letter">
<a:documentation>A business letter</a:documentation>
<element name="l:salutation"><text/></element>
<element name="l:body">
<a:documentation>The main body of the letter</a:documentation>
<oneOrMore><ref name="db.all.blocks"/></oneOrMore>
</element>
</element>
</start>
</include>
</grammar>
This may be drifting off-topic. The main point I wanted to make is
that it is possible to have your custom schema but reuse Docbook as well,
via namespaces and relaxng with relatively little effort.
--Craeg
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: sandbox gendoc
Posted by Steve Loughran <st...@apache.org>.
Dominique Devienne wrote:
>>I've had a look through the sandbox and I noticed Dominique's (sp?) xml'd
>>version of the docs.
>
>
> what does (sp?) means in this context?
probably (spelling?)
>
>
>>As you know I had a play with DocBook-izing the manual last year and in
>>the end the effort of manually altering every html page to be valid
>>DocBook xml drove me mad (gibber etc). Also there seemed to be a
>>consensus (or rather a few well informed arguments against DocBook in
>>general) that DocBook was not the way to go for the ant manual[1]
>
>
> Yes, DocBook, also a standard, is indeed very verbose and doesn't seem
> to have the favor among commiters.
I think think docbook is interesting but something that should be hidden
from users by a docbook editor, not handwritten. Also its too low-level;
I want to mark something as a <task>, not as a bit of code.
>
>
>>So I've had a look at the gendoc version (all 2 manual pages + whatsnew)
>>and I thought that if this is at least in the sandbox, it's 99% more
>>official than choosing a.n.other xml.
>
>
> Well, it's hardly official in any way, but it's a start. Note though
> that there were objections regarding the verbosity of schema chosen,
> which I tried to address in another version not in SVN. I was trying
> to have a looser schema, easier to write by hand, which would be
> converted to the canonical, more verbose and structured XML, for
> further processing.
>
>
>>Given this I'm preparing a little script to automate the conversion of
>>the html docs to the xmlised version in the sandbox.
>>
>>So far I'm trying to get it to work correctly with echo, but most of the
>>code is generic so when I get a passable version of that I'll refactor
>>(a lot) and run it against the doc directory.
>>
>>Given the formatting issues of html, I expect at least a little manual
>>intervention along the way, but manually converting all the docs would
>>be insane (there's over 400 in the core tasks directory alone)
>
>
> Yes, I thought that would be the way to go as well down the line,
> parsing the HTML with tagsoup or the xerces-native-api based Sax
> reader for HTML (don't recall it's name). Given how the examples are
> usually all together, I thought there would need to be more that just
> a little editing, to sprinkle <div class="example"> all over the
> manual. But all in all, I didn't do any of the above, as I didn't much
> of the whole gendoc thing. I simply thought premature to write a
> script to convert the HTML manual into this XML, is the back end
> processing of the XML manual wasn't there.
>
>
>>Anyone interested in the results?
>
>
> I am. I think it would be valuable work, provided gendoc goes
> somewhere. As is obvious, I'm having trouble devote enough time to it.
> The one piece of advise I'd give you is to not throw write a one time
> throw away converter, as I'd foresee the HTML and XML manual to
> co-exist for some time, until the XML version matures enough. Doc
> fixes would go on in the HTML version most likely only. As long as a
> little structure is added to the HTML with <div>s and CSS classes, the
> converter should continue working.
I'd like to see a machine generated manual, with PDF alongside the
online stuff. I've been using things like hibernate, and their docs are
impressive.
>
> This would take care of the manually written manual, but remains the
> part that extracts from the sources and IntrospectionHelper all the
> relevant attribute/element information from tasks. --DD
If we set this up, I'll try and help with markup in the source. Vested
interest: I need to make another summary of all the tasks when I get
that far in the book.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: sandbox gendoc
Posted by Kev Jackson <ke...@it.fts-vn.com>.
>Java does regex just fine, albeit more verbose (when is Java not verbose ;-),
>but my main point is that you already have (Java) tools allow you to
>have an XML view of the existing HTML manual (tagsoup, etc...). Leave
>the parsing to these tools, and concentrate of transforming the
>"loose" HTML schema into a more structured XML, probably using XSL as
>the language rather than scripting. By adding a little more structure
>to the HTML with <div>s, the XML view of the HTML could be complete
>enough for robust transformation to XML, and perhaps even robust
>enough so that the HTML remains as the official "source" document of
>the manual (but stripped of all formatting, which would be added later
>in the XML processing pipeline). The main advantage of this would be
>that editing HTML using an HTML editor for manual editing can be
>easier/nicer and kinda wysiwyg, compared to editing the transformed
>XML.
>
>
>
I'm using libraries - I'm not writing my own html tokenizer :).
>>I'm aiming for a proof of concept script (for echo task) sometime in
>>the next week (if work doesn't get in the way too much). After that
>>I'll see how easy a refactoring job will be for making it generic.
>>
>>
>
>>From above, you can see that I envision the possibility of the HTML
>manual to remain, so it's all the more important that the transform is
>robust.
>
>
>
This suggests that the HTML manual is the "one true source" for the
manual, and all other versions are derived from it through some processing.
>Talks about the tokenizer being too greedy make me uneasy ;-) Leave
>the parsing to existing parsing tool, and just manipulate the
>structure of the document once it's been "reformatted" to a SAX event
>stream. In this form, it feeds easily and naturally to an XSL
>transform pipeline.
>
>That's my view of the whole thing anyway ;-) --DD
>
>
>
From this discussion my understanding is:
1 - Better to use Java + libs - presumably so that an Ant task can be
derived from it (Ant creates it's own manual would be rather nice I'd
have to say)
2 - Conversion util must be robust
3 - Conversion util will be long-lived
4 - Modification of existing HTML to make it easier for conversion util
would probably be a good idea
5 - Structure of XML is as yet undecided - Docbook with RelaxNG has now
been suggested as an alternative to a bespoke xml
Kev
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: sandbox gendoc
Posted by Dominique Devienne <dd...@gmail.com>.
> > Yes, I thought that would be the way to go as well down the line,
> > parsing the HTML with tagsoup [...]
>
> when I say script here I'm talking about a ruby script (partially as I
> saw this as being a converter for the manual for a limited amount of
> time, not immediate throwaway, but not a piece of code that's going to
> fester in svn for years after, partially because I know enough ruby to
> get it done in it (and I don't know enough perl/python)
>
> also for light text processing I think Java is probably too
> heavyweight (sledgehammer to crack a nut) and ruby has nice regexp
> capabilities.
Java does regex just fine, albeit more verbose (when is Java not verbose ;-),
but my main point is that you already have (Java) tools allow you to
have an XML view of the existing HTML manual (tagsoup, etc...). Leave
the parsing to these tools, and concentrate of transforming the
"loose" HTML schema into a more structured XML, probably using XSL as
the language rather than scripting. By adding a little more structure
to the HTML with <div>s, the XML view of the HTML could be complete
enough for robust transformation to XML, and perhaps even robust
enough so that the HTML remains as the official "source" document of
the manual (but stripped of all formatting, which would be added later
in the XML processing pipeline). The main advantage of this would be
that editing HTML using an HTML editor for manual editing can be
easier/nicer and kinda wysiwyg, compared to editing the transformed
XML.
> I'm aiming for a proof of concept script (for echo task) sometime in
> the next week (if work doesn't get in the way too much). After that
> I'll see how easy a refactoring job will be for making it generic.
>From above, you can see that I envision the possibility of the HTML
manual to remain, so it's all the more important that the transform is
robust.
> Right now I'm working with the source HTML as is (yes adding divs etc
> would help immensely) - to see how difficult it is. I've got a fair
> amount done, but the tokenizer I'm using is being a little greedy and
> I've got the examples table as part of the parameters (oops).
Talks about the tokenizer being too greedy make me uneasy ;-) Leave
the parsing to existing parsing tool, and just manipulate the
structure of the document once it's been "reformatted" to a SAX event
stream. In this form, it feeds easily and naturally to an XSL
transform pipeline.
That's my view of the whole thing anyway ;-) --DD
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: sandbox gendoc
Posted by Kevin Jackson <fo...@gmail.com>.
> what does (sp?) means in this context?
>
Sorry, it means I wasn't sure I got the spelling of your name right.
> Well, it's hardly official in any way, but it's a start. Note though
> that there were objections regarding the verbosity of schema chosen,
> which I tried to address in another version not in SVN. I was trying
> to have a looser schema, easier to write by hand, which would be
> converted to the canonical, more verbose and structured XML, for
> further processing.
>
yes I read that the <m:description> etc are too long <m:desc> etc
suggested instead.
> Yes, I thought that would be the way to go as well down the line,
> parsing the HTML with tagsoup or the xerces-native-api based Sax
> reader for HTML (don't recall it's name). Given how the examples are
> usually all together, I thought there would need to be more that just
> a little editing, to sprinkle <div class="example"> all over the
> manual. But all in all, I didn't do any of the above, as I didn't much
> of the whole gendoc thing. I simply thought premature to write a
> script to convert the HTML manual into this XML, is the back end
> processing of the XML manual wasn't there.
when I say script here I'm talking about a ruby script (partially as I
saw this as being a converter for the manual for a limited amount of
time, not immediate throwaway, but not a piece of code that's going to
fester in svn for years after, partially because I know enough ruby to
get it done in it (and I don't know enough perl/python)
also for light text processing I think Java is probably too
heavyweight (sledgehammer to crack a nut) and ruby has nice regexp
capabilities.
RE: processing xml, I thought the current opinion was to use xsl +
ants own style task for the HTML (forgive me if I'm wrong), for pdf
version of the manual there's fop (new version recently released) and
latex2pdf (was mentioned too). There's always forrest too, and I
suppose we could use XDoclet.
> I am. I think it would be valuable work, provided gendoc goes
> somewhere. As is obvious, I'm having trouble devote enough time to it.
> The one piece of advise I'd give you is to not throw write a one time
> throw away converter, as I'd foresee the HTML and XML manual to
> co-exist for some time, until the XML version matures enough. Doc
> fixes would go on in the HTML version most likely only. As long as a
> little structure is added to the HTML with <div>s and CSS classes, the
> converter should continue working.
I'm aiming for a proof of concept script (for echo task) sometime in
the next week (if work doesn't get in the way too much). After that
I'll see how easy a refactoring job will be for making it generic.
Right now I'm working with the source HTML as is (yes adding divs etc
would help immensely) - to see how difficult it is. I've got a fair
amount done, but the tokenizer I'm using is being a little greedy and
I've got the examples table as part of the parameters (oops).
>
> This would take care of the manually written manual, but remains the
> part that extracts from the sources and IntrospectionHelper all the
> relevant attribute/element information from tasks. --DD
>
Yes the 'literate programming' style of documentation (at least I
think this is correct usage of the term) is a different matter
altogether. Having something embedded in the source files would
require an example or two in some of the tasks, could this go into the
trunk or into sandbox (ie copy a taskdef into sandbox and use it to
experiment with the embedded docs).
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: sandbox gendoc
Posted by Dominique Devienne <dd...@gmail.com>.
> I've had a look through the sandbox and I noticed Dominique's (sp?) xml'd
> version of the docs.
what does (sp?) means in this context?
> As you know I had a play with DocBook-izing the manual last year and in
> the end the effort of manually altering every html page to be valid
> DocBook xml drove me mad (gibber etc). Also there seemed to be a
> consensus (or rather a few well informed arguments against DocBook in
> general) that DocBook was not the way to go for the ant manual[1]
Yes, DocBook, also a standard, is indeed very verbose and doesn't seem
to have the favor among commiters.
> So I've had a look at the gendoc version (all 2 manual pages + whatsnew)
> and I thought that if this is at least in the sandbox, it's 99% more
> official than choosing a.n.other xml.
Well, it's hardly official in any way, but it's a start. Note though
that there were objections regarding the verbosity of schema chosen,
which I tried to address in another version not in SVN. I was trying
to have a looser schema, easier to write by hand, which would be
converted to the canonical, more verbose and structured XML, for
further processing.
> Given this I'm preparing a little script to automate the conversion of
> the html docs to the xmlised version in the sandbox.
>
> So far I'm trying to get it to work correctly with echo, but most of the
> code is generic so when I get a passable version of that I'll refactor
> (a lot) and run it against the doc directory.
>
> Given the formatting issues of html, I expect at least a little manual
> intervention along the way, but manually converting all the docs would
> be insane (there's over 400 in the core tasks directory alone)
Yes, I thought that would be the way to go as well down the line,
parsing the HTML with tagsoup or the xerces-native-api based Sax
reader for HTML (don't recall it's name). Given how the examples are
usually all together, I thought there would need to be more that just
a little editing, to sprinkle <div class="example"> all over the
manual. But all in all, I didn't do any of the above, as I didn't much
of the whole gendoc thing. I simply thought premature to write a
script to convert the HTML manual into this XML, is the back end
processing of the XML manual wasn't there.
> Anyone interested in the results?
I am. I think it would be valuable work, provided gendoc goes
somewhere. As is obvious, I'm having trouble devote enough time to it.
The one piece of advise I'd give you is to not throw write a one time
throw away converter, as I'd foresee the HTML and XML manual to
co-exist for some time, until the XML version matures enough. Doc
fixes would go on in the HTML version most likely only. As long as a
little structure is added to the HTML with <div>s and CSS classes, the
converter should continue working.
This would take care of the manually written manual, but remains the
part that extracts from the sources and IntrospectionHelper all the
relevant attribute/element information from tasks. --DD
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org