You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by "James K. Lowden" <jk...@schemamania.org> on 2013/08/07 06:07:08 UTC
man pages for Subversion
http://www.schemamania.org/svn/man1/
e.g. http://www.schemamania.org/svn/man1/svn-propset.pdf
I offer the above mdoc man pages to the Subversion project as a token
of my appreciation, and in the hope they'll be included in future
versions.
I find using any command-line tool more convenient when it's documented
with proper man pages, so that "man foo" instantly displays the
reference manual. That's especially true of a complex tool like
subversion. The "svn help foo | less" feature is OK, but it's not as
convenient and lacks the boldface and underscores that assist in
scanning the text. HTML is better-looking, but many clicks away.
I hope you agree that the man pages are easier to read than plain
text, and the PDFs just stunning.
The pages represent perhaps 30 hours of work. The --help text was
processed through the bin/manualize Perl script to create a rough
mdoc draft. Then individual sections were edited by hand, sometimes to
put finishing touches on the formatting, sometimes to clarify the text.
The final result is surely a derived work, but it's not clear to me if
the --help text is copyrighted. I am prepare to assign it as befits
the project, if need be.
You can generate your own man pages from whatever version of Subversion
you have installed by unpacking the tarball and running "make".
Regards,
--jkl
Re: man pages for Subversion
Posted by Daniel Shahaf <da...@elego.de>.
Stefan Sperling wrote on Wed, Aug 07, 2013 at 11:27:13 +0200:
> On Wed, Aug 07, 2013 at 12:07:08AM -0400, James K. Lowden wrote:
> > The pages represent perhaps 30 hours of work. The --help text was
> > processed through the bin/manualize Perl script to create a rough
> > mdoc draft. Then individual sections were edited by hand, sometimes to
> > put finishing touches on the formatting, sometimes to clarify the text.
>
> So essentially you are running a script to produce those pages
> from 'svn help' output? Where can I find this script?
>
bin/manualize in http://www.schemamania.org/svn/svn-manual.pax.gz
Re: man pages for Subversion
Posted by Stefan Sperling <st...@elego.de>.
On Wed, Aug 07, 2013 at 12:07:08AM -0400, James K. Lowden wrote:
> http://www.schemamania.org/svn/man1/
> e.g. http://www.schemamania.org/svn/man1/svn-propset.pdf
>
> I offer the above mdoc man pages to the Subversion project as a token
> of my appreciation, and in the hope they'll be included in future
> versions.
>
> I find using any command-line tool more convenient when it's documented
> with proper man pages, so that "man foo" instantly displays the
> reference manual. That's especially true of a complex tool like
> subversion. The "svn help foo | less" feature is OK, but it's not as
> convenient and lacks the boldface and underscores that assist in
> scanning the text. HTML is better-looking, but many clicks away.
>
> I hope you agree that the man pages are easier to read than plain
> text, and the PDFs just stunning.
Thanks. I've been thinking about how to better produce mdoc output
for Subversion man pages for a while now.
A related discussion can be found at
http://mail-archives.apache.org/mod_mbox/subversion-dev/201209.mbox/%3C1347648306.66873.YahooMailNeo%40web87705.mail.ir2.yahoo.com%3E
Please also see the Improve 'svn help' item on this page:
http://subversion.apache.org/ideas.html#ideas
> The pages represent perhaps 30 hours of work. The --help text was
> processed through the bin/manualize Perl script to create a rough
> mdoc draft. Then individual sections were edited by hand, sometimes to
> put finishing touches on the formatting, sometimes to clarify the text.
So essentially you are running a script to produce those pages
from 'svn help' output? Where can I find this script?
Can the finishing touches you mention be made on the source
material, i.e. 'svn help' as well? Ideally, we'd produce man pages
as part of our build process and ship them with Subversion releases.
Can you help us with achieving that goal?
I don't mind whether we're post-processing 'svn help' output to
produce man pages, or whether we're generating 'svn help' output
and man pages from the same source material somehow. As long as
we can get a set of quality man pages to ship with a release.
> The final result is surely a derived work, but it's not clear to me if
> the --help text is copyrighted. I am prepare to assign it as befits
> the project, if need be.
I would say that the ASF copyright applies, i.e. the licence is APLv2.
> You can generate your own man pages from whatever version of Subversion
> you have installed by unpacking the tarball and running "make".
Which tarball?
Re: man pages for Subversion
Posted by Julian Foad <ju...@btopenworld.com>.
Thomas Åkesson wrote:
> Hi Julian et al,
> I think your XML approach is the way to go in order to fulfill current and
> future requirements. A single source to keep updated is vital. XML is an ideal
> source format for HTML, PDF while plaintext (svn help), man etc should be no
> problems to sort out with XSLT.
>
> I was unable to find the XML or XSLT in the patch.
Oops. Here, attached, is a complete patch (svn-help-from-xml-5.patch).
> Since the svnbook is Docbook
> I would go with that schema unless we find very specific requirements that we
> can't fulfill (usually doable with some creative use of attributes). The
> reasonable alternative, given that we prefer standards, would be DITA but that
> is a whole new level of complexity.
>
> How is the stuff that you removed from the .c files translated? Given that
> different languages take very different amounts of space the line breaks must to
> be... well, problematic.
I inserted manual "soft" line-break annotations into the XML in order to match the C code formatting exactly, with the intention that we would eventually remove those from the XML source and do some kind of automatic line splitting instead.
> Going with Docbook would enable the svnbook authors to quite easily generate a
> set of XML fragments from the svn XML. These XML fragments would be at minimum
> screen-elements and probably also some list-stuff. All of which can be XIncluded
> into the Svnbook instead of duplicated.
Yes -- I had that in mind but have not tried to do it.
> I can help out with schema discussions and XSLT programming (in September,
> preferably). I have a fair share of experience transforming Docbook-like schemas
> into XHTML, XSL-FO and other formats.
Cool.
Thanks.
- Julian
> On 12 aug 2013, at 12:38, Julian Foad wrote:
>> Hi James. I have one thing to throw into the mix, which you might
>> be interested in looking at. I experimented a few months ago with
>> generating both the C help strings and man pages from an XML source
>> file. My patch to do this is attached; it is complete and and usable,
>> although not finished or quite how I would want it.
>>
>> One of my requirements was to produce C strings identical to those we
>> have now, in order to prove that it can be done and to be able to
>> check whether I've missed anything. I invented my own XML schema and
>> hand-wrote the XSL transformations, and to be honest the XSL is ugly.
>> Before using a method like this, I would want to change it to use a
>> somewhat standard schema, or at least a standard schema as far as
>> possible plus our own extensions for the rest.
>>
>> If you want to play with this patch, it applies to svn trunk as of
>> r1456600 (March 2013); I haven't updated it since then. It's
>> integrated into the build system (Unix only, so far), using xsltproc.
>> To create the man page, use "make subversion/svn/svn-help.man", and
>> use something like "(cd subversion/svn && man -l svn.1)" to view it.
>> It currently creates a single man page for 'svn', listing all of the
>> subcommands within it.
>>
>> - Julian
Re: man pages for Subversion
Posted by Thomas Åkesson <th...@akesson.cc>.
Hi Julian et al,
I think your XML approach is the way to go in order to fulfill current and future requirements. A single source to keep updated is vital. XML is an ideal source format for HTML, PDF while plaintext (svn help), man etc should be no problems to sort out with XSLT.
I was unable to find the XML or XSLT in the patch. Since the svnbook is Docbook I would go with that schema unless we find very specific requirements that we can't fulfill (usually doable with some creative use of attributes). The reasonable alternative, given that we prefer standards, would be DITA but that is a whole new level of complexity.
How is the stuff that you removed from the .c files translated? Given that different languages take very different amounts of space the line breaks must to be... well, problematic.
Going with Docbook would enable the svnbook authors to quite easily generate a set of XML fragments from the svn XML. These XML fragments would be at minimum screen-elements and probably also some list-stuff. All of which can be XIncluded into the Svnbook instead of duplicated.
I can help out with schema discussions and XSLT programming (in September, preferably). I have a fair share of experience transforming Docbook-like schemas into XHTML, XSL-FO and other formats.
regards,
Thomas Å.
On 12 aug 2013, at 12:38, Julian Foad <ju...@btopenworld.com> wrote:
> Hi James. I have one thing to throw into the mix, which you might be interested in looking at. I experimented a few months ago with generating both the C help strings and man pages from an XML source file. My patch to do this is attached; it is complete and and usable, although not finished or quite how I would want it.
>
> One of my requirements was to produce C strings identical to those we have now, in order to prove that it can be done and to be able to check whether I've missed anything. I invented my own XML schema and hand-wrote the XSL transformations, and to be honest the XSL is ugly. Before using a method like this, I would want to change it to use a somewhat standard schema, or at least a standard schema as far as possible plus our own extensions for the rest.
>
> If you want to play with this patch, it applies to svn trunk as of r1456600 (March 2013); I haven't updated it since then. It's integrated into the build system (Unix only, so far), using xsltproc. To create the man page, use "make subversion/svn/svn-help.man", and use something like "(cd subversion/svn && man -l svn.1)" to view it. It currently creates a single man page for 'svn', listing all of the subcommands within it.
>
>
> - Julian
>
>
>
> --
> Join WANdisco's free daily demo sessions on Scaling Subversion for the Enterprise
> <http://www.wandisco.com/training/webinars>
>
>
>
> ----- Original Message -----
>> From: James K. Lowden <jk...@schemamania.org>
>> To: Daniel Shahaf <da...@elego.de>
>> Cc: dev@subversion.apache.org
>> Sent: Saturday, 10 August 2013, 4:22
>> Subject: Re: man pages for Subversion
>>
>> On Fri, 9 Aug 2013 11:21:23 +0300
>> Daniel Shahaf <da...@elego.de> wrote:
>>
>>>> 1. Get the sources for 1.9
>>>> 2. Examine svn_cl__cmd_table
>>>> 3. Prepare a doc build tree that creates two things:
>>>> a. man pages
>>>> b. plain text for --help, one file per subcommand
>>>>
>>>> The plain text filenames will have the extension ".in", so a
>>>> simple Makefile rule can convert them to ".h" for inclusion
>> into the
>>>> appropriate source module.
>>>
>>> You'd have to figure out something for windows as well, since we
>> don't
>>> use make that on that platform. (Several of our dependencies do, but
>>> we just generate Visual Studio projects.)
>>
>> Hmm. This is a sticky wicket.
>>
>> My plan is to keep mdoc sources and generate plain text for
>> inclusion in the help subsystem. That's done easily with groff, see
>> below. But I notice in INSTALL you support building on Windows not only
>> in a tarball but also directly in-tree. That means the Windows
>> build machine would have to be able to convert the mdoc sources to ASCII
>> strings.
>>
>> I'd like to point out, btw, that in 2013 this is the tail wagging the
>> dog. Windows command-line users are rare; most people use Tortoise.
>> Those that do use the command line don't rely on --help very much,
>> either. Consider:
>>
>> $ svn propset --help | wc -l
>> 95
>>
>> and that more.exe resembles UNIX more(1) circa 1984 and that most
>> Windows users don't even know it exists. "svn help" on Windows is
>> not
>> the model of ease of use.
>>
>> Preserving --help for anything more than a short recap of the options
>> isn't worth it, especially when PDFs are available! The time is much
>> better spent improving the documentation and Subversion itself.
>>
>> But let us suppose ad argumentum that we want to preserve --help, and
>> to support in-tree builds on Windows. That means something has to
>> convert the mdoc source to plain text. I don't see any better way than
>> to require mdocml or groff to be installed.
>>
>>> In general, it'll be best if you hook into the existing build system
>>> --- configure.ac, gen-make.py, and build.conf. The latter two are
>>> used on all platforms (on unix they generate build-outputs.mk).
>>
>> I don't want to get too deeply involved. I know mdoc, and I want to
>> see man pages become part of Subversion. I'm willing to provide
>> up-to-date mdoc source for them, and to generate plain text, and to
>> write a little Perl to convert the plain text to static constant
>> character arrays in C. I'm prepared to do the work to make it easy
>> -- for someone who knows how -- to integrate that work into the build
>> system. I don't want to peek inside gen-make.py.
>>
>> Below is a sample of what I'd expect from the mdoc -> ascii
>> conversion.
>>
>> --jkl
>>
>> $ groff -P -bc -Tascii -mdoc man1/svn-copy.1 \
>> | sed -ne '/NAME/,/specify/p'
>> NAME
>> svn copy -- Duplicate something in working copy or repository
>>
>> SYNOPSIS
>> svn copy (cp) [-Fmq] [--config-dir DIR] [--config-option OPT]
>> [--editor-cmd CMD] [--encoding ENC] [--file] [--force-log]
>> [--ignore-externals] [--message] [--no-auth-cache]
>> [--non-interactive] [--parents] [--password PWD] [--quiet]
>> [{-r | --revision} REV] [--trust-server-cert] [--username
>> USR] [--with-revprop PROP] SRC[@REV] ... DST
>>
>> DESCRIPTION
>> When copying multiple sources, they will be added as children of
>> DST, which must be a directory. SRC and DST can each be either a
>> working copy (WC) path or URL:
>> WC -> WC copy and schedule for addition (with history)
>> WC -> URL immediately commit a copy of WC to URL
>> URL -> WC check out URL into WC, schedule for addition
>> URL -> URL complete server-side copy; used to branch and tag
>> All the SRCs must be of the same type.
>>
>> OPTIONS
>> --editor-cmd CMD use CMD as external editor
>> --encoding ENC treat value as being in charset encoding ENC
>> -F --file FIL read log message from file FIL
>> --force-log force validity of log message source
>> --ignore-externals ignore externals definitions
>> -m --message MSG specify log message MSG
> <svn-help-from-xml-4.patch>
Re: man pages for Subversion
Posted by Branko Čibej <br...@wandisco.com>.
On 14.08.2013 23:42, Thomas Åkesson wrote:
>
> On 14 aug 2013, at 20:47, Mattias Engdegård <ma...@bredband.net> wrote:
>
>> 12 aug 2013 kl. 12.38 skrev Julian Foad:
>>
>>> Hi James. I have one thing to throw into the mix, which you might be interested in looking at. I experimented a few months ago with generating both the C help strings and man pages from an XML source file. My patch to do this is attached; it is complete and and usable, although not finished or quite how I would want it.
>> How about including the marked-up source in the svn binary (as C constant strings) and do the rendering at run time, when the user does "svn help"? Otherwise, it would be tricky to handle translations properly - the translator must work on the marked-up text and it should be possible to try out new translations without rebuilding the binaries.
> Is that possible today?
Of course. You can, for example, take an installed Subversion and
replace one of its .mo files, and it'll pick up the new translations.
If that doesn't work, I'd like to know about it because it'd be a
serious bug in how we use gettext.
-- Brane
--
Branko Čibej | Director of Subversion
WANdisco // Non-Stop Data
e. brane@wandisco.com
Re: man pages for Subversion
Posted by Thomas Åkesson <th...@bafast.se>.
On 14 aug 2013, at 20:47, Mattias Engdegård <ma...@bredband.net> wrote:
> 12 aug 2013 kl. 12.38 skrev Julian Foad:
>
>> Hi James. I have one thing to throw into the mix, which you might be interested in looking at. I experimented a few months ago with generating both the C help strings and man pages from an XML source file. My patch to do this is attached; it is complete and and usable, although not finished or quite how I would want it.
>
> How about including the marked-up source in the svn binary (as C constant strings) and do the rendering at run time, when the user does "svn help"? Otherwise, it would be tricky to handle translations properly - the translator must work on the marked-up text and it should be possible to try out new translations without rebuilding the binaries.
Is that possible today?
> This has the extra benefit of allowing the help text to be rendered to the width of the terminal and to (optionally) use terminal effects such as bold and underline.
Given the verbosity of some help texts I definitely think the source format should not care about terminal width. It must be a major pain to translate given the line breaks? Whether the line breaks are introduced during runtime or build matters less, but the later the better.
> The original could either live in a separate file, as in your patch, or as actual C constants. In any case, it probably makes things a lot easier for the translators if the text appears as strings in the same subversion.pot as everything else.
>
> (Personally I would prefer something more light-weight than XML, but perhaps it would give you an excuse to allow the --xml flag with svn help.)
Many recently developed software frameworks define their strings in XML because:
a) translation processes are well defined for XML, in a larger perspective since almost all documentation is done in XML (ok, not a great driver in this project except keeping svnbook up to date)
b) XML is so easy to transform into multiple distribution formats.
I know there is a lot of resistance toward the heavy-weight XML, but authoring is actually where XML excels (as opposed to data interchange where lighter formats can be preferred).
Cheers,
Thomas Å.
Re: man pages for Subversion
Posted by Mattias Engdegård <ma...@bredband.net>.
12 aug 2013 kl. 12.38 skrev Julian Foad:
> Hi James. I have one thing to throw into the mix, which you might
> be interested in looking at. I experimented a few months ago with
> generating both the C help strings and man pages from an XML source
> file. My patch to do this is attached; it is complete and and
> usable, although not finished or quite how I would want it.
How about including the marked-up source in the svn binary (as C
constant strings) and do the rendering at run time, when the user does
"svn help"? Otherwise, it would be tricky to handle translations
properly - the translator must work on the marked-up text and it
should be possible to try out new translations without rebuilding the
binaries.
This has the extra benefit of allowing the help text to be rendered to
the width of the terminal and to (optionally) use terminal effects
such as bold and underline.
The original could either live in a separate file, as in your patch,
or as actual C constants. In any case, it probably makes things a lot
easier for the translators if the text appears as strings in the same
subversion.pot as everything else.
(Personally I would prefer something more light-weight than XML, but
perhaps it would give you an excuse to allow the --xml flag with svn
help.)
Re: man pages for Subversion
Posted by Julian Foad <ju...@btopenworld.com>.
Hi James. I have one thing to throw into the mix, which you might be interested in looking at. I experimented a few months ago with generating both the C help strings and man pages from an XML source file. My patch to do this is attached; it is complete and and usable, although not finished or quite how I would want it.
One of my requirements was to produce C strings identical to those we have now, in order to prove that it can be done and to be able to check whether I've missed anything. I invented my own XML schema and hand-wrote the XSL transformations, and to be honest the XSL is ugly. Before using a method like this, I would want to change it to use a somewhat standard schema, or at least a standard schema as far as possible plus our own extensions for the rest.
If you want to play with this patch, it applies to svn trunk as of r1456600 (March 2013); I haven't updated it since then. It's integrated into the build system (Unix only, so far), using xsltproc. To create the man page, use "make subversion/svn/svn-help.man", and use something like "(cd subversion/svn && man -l svn.1)" to view it. It currently creates a single man page for 'svn', listing all of the subcommands within it.
- Julian
--
Join WANdisco's free daily demo sessions on Scaling Subversion for the Enterprise
<http://www.wandisco.com/training/webinars>
----- Original Message -----
> From: James K. Lowden <jk...@schemamania.org>
> To: Daniel Shahaf <da...@elego.de>
> Cc: dev@subversion.apache.org
> Sent: Saturday, 10 August 2013, 4:22
> Subject: Re: man pages for Subversion
>
> On Fri, 9 Aug 2013 11:21:23 +0300
> Daniel Shahaf <da...@elego.de> wrote:
>
>> > 1. Get the sources for 1.9
>> > 2. Examine svn_cl__cmd_table
>> > 3. Prepare a doc build tree that creates two things:
>> > a. man pages
>> > b. plain text for --help, one file per subcommand
>> >
>> > The plain text filenames will have the extension ".in", so a
>> > simple Makefile rule can convert them to ".h" for inclusion
> into the
>> > appropriate source module.
>>
>> You'd have to figure out something for windows as well, since we
> don't
>> use make that on that platform. (Several of our dependencies do, but
>> we just generate Visual Studio projects.)
>
> Hmm. This is a sticky wicket.
>
> My plan is to keep mdoc sources and generate plain text for
> inclusion in the help subsystem. That's done easily with groff, see
> below. But I notice in INSTALL you support building on Windows not only
> in a tarball but also directly in-tree. That means the Windows
> build machine would have to be able to convert the mdoc sources to ASCII
> strings.
>
> I'd like to point out, btw, that in 2013 this is the tail wagging the
> dog. Windows command-line users are rare; most people use Tortoise.
> Those that do use the command line don't rely on --help very much,
> either. Consider:
>
> $ svn propset --help | wc -l
> 95
>
> and that more.exe resembles UNIX more(1) circa 1984 and that most
> Windows users don't even know it exists. "svn help" on Windows is
> not
> the model of ease of use.
>
> Preserving --help for anything more than a short recap of the options
> isn't worth it, especially when PDFs are available! The time is much
> better spent improving the documentation and Subversion itself.
>
> But let us suppose ad argumentum that we want to preserve --help, and
> to support in-tree builds on Windows. That means something has to
> convert the mdoc source to plain text. I don't see any better way than
> to require mdocml or groff to be installed.
>
>> In general, it'll be best if you hook into the existing build system
>> --- configure.ac, gen-make.py, and build.conf. The latter two are
>> used on all platforms (on unix they generate build-outputs.mk).
>
> I don't want to get too deeply involved. I know mdoc, and I want to
> see man pages become part of Subversion. I'm willing to provide
> up-to-date mdoc source for them, and to generate plain text, and to
> write a little Perl to convert the plain text to static constant
> character arrays in C. I'm prepared to do the work to make it easy
> -- for someone who knows how -- to integrate that work into the build
> system. I don't want to peek inside gen-make.py.
>
> Below is a sample of what I'd expect from the mdoc -> ascii
> conversion.
>
> --jkl
>
> $ groff -P -bc -Tascii -mdoc man1/svn-copy.1 \
> | sed -ne '/NAME/,/specify/p'
> NAME
> svn copy -- Duplicate something in working copy or repository
>
> SYNOPSIS
> svn copy (cp) [-Fmq] [--config-dir DIR] [--config-option OPT]
> [--editor-cmd CMD] [--encoding ENC] [--file] [--force-log]
> [--ignore-externals] [--message] [--no-auth-cache]
> [--non-interactive] [--parents] [--password PWD] [--quiet]
> [{-r | --revision} REV] [--trust-server-cert] [--username
> USR] [--with-revprop PROP] SRC[@REV] ... DST
>
> DESCRIPTION
> When copying multiple sources, they will be added as children of
> DST, which must be a directory. SRC and DST can each be either a
> working copy (WC) path or URL:
> WC -> WC copy and schedule for addition (with history)
> WC -> URL immediately commit a copy of WC to URL
> URL -> WC check out URL into WC, schedule for addition
> URL -> URL complete server-side copy; used to branch and tag
> All the SRCs must be of the same type.
>
> OPTIONS
> --editor-cmd CMD use CMD as external editor
> --encoding ENC treat value as being in charset encoding ENC
> -F --file FIL read log message from file FIL
> --force-log force validity of log message source
> --ignore-externals ignore externals definitions
> -m --message MSG specify log message MSG
>
RE: man pages for Subversion
Posted by Bob Archer <Bo...@amsi.com>.
> On Fri, 9 Aug 2013 11:21:23 +0300
> Daniel Shahaf <da...@elego.de> wrote:
>
> > > 1. Get the sources for 1.9
> > > 2. Examine svn_cl__cmd_table
> > > 3. Prepare a doc build tree that creates two things:
> > > a. man pages
> > > b. plain text for --help, one file per subcommand
> > >
> > > The plain text filenames will have the extension ".in", so a simple
> > > Makefile rule can convert them to ".h" for inclusion into the
> > > appropriate source module.
> >
> > You'd have to figure out something for windows as well, since we don't
> > use make that on that platform. (Several of our dependencies do, but
> > we just generate Visual Studio projects.)
>
> Hmm. This is a sticky wicket.
>
> My plan is to keep mdoc sources and generate plain text for inclusion in the
> help subsystem. That's done easily with groff, see below. But I notice in
> INSTALL you support building on Windows not only in a tarball but also directly
> in-tree. That means the Windows build machine would have to be able to
> convert the mdoc sources to ASCII strings.
>
> I'd like to point out, btw, that in 2013 this is the tail wagging the dog. Windows
> command-line users are rare; most people use Tortoise.
Is this an assumption, a prejudice or wishful thinking? Or, do you have facts to back this up? Perhaps your assumption is correct wrt to your average mainstream web surfer, but for devs many of them use command line quite a bit. While I use tortoise for more common commands, I do drop to the command line quite frequently to use svn.exe and to read the help text.
> Those that do use the command line don't rely on --help very much, either.
> Consider:
>
> $ svn propset --help | wc -l
> 95
>
> and that more.exe resembles UNIX more(1) circa 1984 and that most Windows
> users don't even know it exists. "svn help" on Windows is not the model of ease
> of use.
>
> Preserving --help for anything more than a short recap of the options isn't
> worth it, especially when PDFs are available! The time is much better spent
> improving the documentation and Subversion itself.
>
> But let us suppose ad argumentum that we want to preserve --help, and to
> support in-tree builds on Windows. That means something has to convert the
> mdoc source to plain text. I don't see any better way than to require mdocml
> or groff to be installed.
>
> > In general, it'll be best if you hook into the existing build system
> > --- configure.ac, gen-make.py, and build.conf. The latter two are
> > used on all platforms (on unix they generate build-outputs.mk).
>
> I don't want to get too deeply involved. I know mdoc, and I want to see man
> pages become part of Subversion. I'm willing to provide up-to-date mdoc
> source for them, and to generate plain text, and to write a little Perl to convert
> the plain text to static constant character arrays in C. I'm prepared to do the
> work to make it easy
> -- for someone who knows how -- to integrate that work into the build system. I
> don't want to peek inside gen-make.py.
>
> Below is a sample of what I'd expect from the mdoc -> ascii conversion.
>
> --jkl
>
> $ groff -P -bc -Tascii -mdoc man1/svn-copy.1 \
> | sed -ne '/NAME/,/specify/p'
> NAME
> svn copy -- Duplicate something in working copy or repository
>
> SYNOPSIS
> svn copy (cp) [-Fmq] [--config-dir DIR] [--config-option OPT]
> [--editor-cmd CMD] [--encoding ENC] [--file] [--force-log]
> [--ignore-externals] [--message] [--no-auth-cache]
> [--non-interactive] [--parents] [--password PWD] [--quiet]
> [{-r | --revision} REV] [--trust-server-cert] [--username USR] [--with-
> revprop PROP] SRC[@REV] ... DST
>
> DESCRIPTION
> When copying multiple sources, they will be added as children of DST, which
> must be a directory. SRC and DST can each be either a working copy (WC) path
> or URL:
> WC -> WC copy and schedule for addition (with history)
> WC -> URL immediately commit a copy of WC to URL
> URL -> WC check out URL into WC, schedule for addition
> URL -> URL complete server-side copy; used to branch and tag
> All the SRCs must be of the same type.
>
> OPTIONS
> --editor-cmd CMD use CMD as external editor
> --encoding ENC treat value as being in charset encoding ENC
> -F --file FIL read log message from file FIL
> --force-log force validity of log message source
> --ignore-externals ignore externals definitions
> -m --message MSG specify log message MSG
>
Re: man pages for Subversion
Posted by "James K. Lowden" <jk...@schemamania.org>.
On Fri, 9 Aug 2013 11:21:23 +0300
Daniel Shahaf <da...@elego.de> wrote:
> > 1. Get the sources for 1.9
> > 2. Examine svn_cl__cmd_table
> > 3. Prepare a doc build tree that creates two things:
> > a. man pages
> > b. plain text for --help, one file per subcommand
> >
> > The plain text filenames will have the extension ".in", so a
> > simple Makefile rule can convert them to ".h" for inclusion into the
> > appropriate source module.
>
> You'd have to figure out something for windows as well, since we don't
> use make that on that platform. (Several of our dependencies do, but
> we just generate Visual Studio projects.)
Hmm. This is a sticky wicket.
My plan is to keep mdoc sources and generate plain text for
inclusion in the help subsystem. That's done easily with groff, see
below. But I notice in INSTALL you support building on Windows not only
in a tarball but also directly in-tree. That means the Windows
build machine would have to be able to convert the mdoc sources to ASCII
strings.
I'd like to point out, btw, that in 2013 this is the tail wagging the
dog. Windows command-line users are rare; most people use Tortoise.
Those that do use the command line don't rely on --help very much,
either. Consider:
$ svn propset --help | wc -l
95
and that more.exe resembles UNIX more(1) circa 1984 and that most
Windows users don't even know it exists. "svn help" on Windows is not
the model of ease of use.
Preserving --help for anything more than a short recap of the options
isn't worth it, especially when PDFs are available! The time is much
better spent improving the documentation and Subversion itself.
But let us suppose ad argumentum that we want to preserve --help, and
to support in-tree builds on Windows. That means something has to
convert the mdoc source to plain text. I don't see any better way than
to require mdocml or groff to be installed.
> In general, it'll be best if you hook into the existing build system
> --- configure.ac, gen-make.py, and build.conf. The latter two are
> used on all platforms (on unix they generate build-outputs.mk).
I don't want to get too deeply involved. I know mdoc, and I want to
see man pages become part of Subversion. I'm willing to provide
up-to-date mdoc source for them, and to generate plain text, and to
write a little Perl to convert the plain text to static constant
character arrays in C. I'm prepared to do the work to make it easy
-- for someone who knows how -- to integrate that work into the build
system. I don't want to peek inside gen-make.py.
Below is a sample of what I'd expect from the mdoc -> ascii
conversion.
--jkl
$ groff -P -bc -Tascii -mdoc man1/svn-copy.1 \
| sed -ne '/NAME/,/specify/p'
NAME
svn copy -- Duplicate something in working copy or repository
SYNOPSIS
svn copy (cp) [-Fmq] [--config-dir DIR] [--config-option OPT]
[--editor-cmd CMD] [--encoding ENC] [--file] [--force-log]
[--ignore-externals] [--message] [--no-auth-cache]
[--non-interactive] [--parents] [--password PWD] [--quiet]
[{-r | --revision} REV] [--trust-server-cert] [--username
USR] [--with-revprop PROP] SRC[@REV] ... DST
DESCRIPTION
When copying multiple sources, they will be added as children of
DST, which must be a directory. SRC and DST can each be either a
working copy (WC) path or URL:
WC -> WC copy and schedule for addition (with history)
WC -> URL immediately commit a copy of WC to URL
URL -> WC check out URL into WC, schedule for addition
URL -> URL complete server-side copy; used to branch and tag
All the SRCs must be of the same type.
OPTIONS
--editor-cmd CMD use CMD as external editor
--encoding ENC treat value as being in charset encoding ENC
-F --file FIL read log message from file FIL
--force-log force validity of log message source
--ignore-externals ignore externals definitions
-m --message MSG specify log message MSG
Re: man pages for Subversion
Posted by Daniel Shahaf <da...@elego.de>.
James K. Lowden wrote on Thu, Aug 08, 2013 at 22:25:08 -0400:
> Stefan Sperling <st...@elego.de> wrote:
> > If you manage to generate both the compiled-in strings and the man
> > pages (on UNIX only) from the same source format during the build,
> > I"m happy with that. If you manage to generate man pages from
> > 'svn help' output during the build process on UNIX, then I'm happy,
> > too.
>
> It sounds like we have agreement and the ball is back in my court. If
> there are no objections, I'll work on the following as a next step:
>
> 1. Get the sources for 1.9
> 2. Examine svn_cl__cmd_table
> 3. Prepare a doc build tree that creates two things:
> a. man pages
> b. plain text for --help, one file per subcommand
>
> The plain text filenames will have the extension ".in", so a
> simple Makefile rule can convert them to ".h" for inclusion into the
> appropriate source module.
You'd have to figure out something for windows as well, since we don't
use make that on that platform. (Several of our dependencies do, but we
just generate Visual Studio projects.)
In general, it'll be best if you hook into the existing build system ---
configure.ac, gen-make.py, and build.conf. The latter two are used on
all platforms (on unix they generate build-outputs.mk).
It would probably be worthwhile to familiarize yourself with the build
system on all platforms. Have a look at where translate_sql.py is
invoked from. (It's invoked at autogen.sh time on unix, by the way,
meaning end-users don't need to have Python to build + install
Subversion from a tarball.)
Re: man pages for Subversion
Posted by Stefan Sperling <st...@elego.de>.
On Thu, Aug 08, 2013 at 10:25:08PM -0400, James K. Lowden wrote:
> It sounds like we have agreement and the ball is back in my court. If
> there are no objections, I'll work on the following as a next step:
>
> 1. Get the sources for 1.9
> 2. Examine svn_cl__cmd_table
> 3. Prepare a doc build tree that creates two things:
> a. man pages
> b. plain text for --help, one file per subcommand
>
> The plain text filenames will have the extension ".in", so a
> simple Makefile rule can convert them to ".h" for inclusion into the
> appropriate source module.
Will the man pages and plain text files be derived from one another
in some way? Or will we have to maintain two versions of the docs?
Will the current 'svn help' output be changed in any way?
Re: man pages for Subversion
Posted by "James K. Lowden" <jk...@schemamania.org>.
On Thu, 8 Aug 2013 08:58:29 +0300
Daniel Shahaf <da...@elego.de> wrote:
> Perhaps this is a good time to point out that you were writing man
> pages for 1.7, which is in "Security fixes only" mode. The supported
> release is 1.8 and trunk (where new development should be targeted)
> is 1.9.
>
> > 1. Keep the man pages in mdoc format as the "souce code".
> > 2. Convert "svn foo --help" and "svn help foo" to invoke man(1), as
> > git does. Or use groff to generate plain text for static inclusion
> > in the the binary.
>
> As Bert said: not possible. We support Windows and that has no man(1)
> or nroff(1) binaries.
OK. It's easier to convert mdoc to ascii than the other way around.
What you want, I think, is one long C string (with embedded newlines)
that per subcommand that you can print in response to --help. Can do.
> So you'd need a solution that allows compiling the man pages source
> to C strings that uses only Python (and maybe Perl).
Yeah, the poverty of Windows has been a problem since ever. We can
leave the functionality of --help as is.
> You realise we already have such a database? svn_cl__cmd_table in
> subversion/svn/svn.c.
I will look into it. I suppose it sounds odd to this group, but I'm
just a Subversion user. I've never been tempted to hack on it, and
never looked at the code.
> > Contrary to popular wisdom, good old troff is the best system there
> > is
>
> Clearly you have a strong opinion on this.
Yes. :-) It's not as though I've been using troff for 30 years and
never learned anything else. I came to it reluctantly, having tried
several others, and found it actually works pretty well. As Dijkstra
said of Algol 60, "an advance over its predecessors, and over many of
its successors."
Stefan Sperling <st...@elego.de> wrote:
> If you manage to generate both the compiled-in strings and the man
> pages (on UNIX only) from the same source format during the build,
> I"m happy with that. If you manage to generate man pages from
> 'svn help' output during the build process on UNIX, then I'm happy,
> too.
It sounds like we have agreement and the ball is back in my court. If
there are no objections, I'll work on the following as a next step:
1. Get the sources for 1.9
2. Examine svn_cl__cmd_table
3. Prepare a doc build tree that creates two things:
a. man pages
b. plain text for --help, one file per subcommand
The plain text filenames will have the extension ".in", so a
simple Makefile rule can convert them to ".h" for inclusion into the
appropriate source module.
I'll also subscribe to the list to make it more convenient for us to
communicate.
Regards,
--jkl
Re: man pages for Subversion
Posted by Stefan Sperling <st...@elego.de>.
On Thu, Aug 08, 2013 at 08:58:29AM +0300, Daniel Shahaf wrote:
> James K. Lowden wrote on Wed, Aug 07, 2013 at 20:13:16 -0400:
> > 1. Keep the man pages in mdoc format as the "souce code".
> > 2. Convert "svn foo --help" and "svn help foo" to invoke man(1), as
> > git does. Or use groff to generate plain text for static inclusion in
> > the the binary.
>
> As Bert said: not possible. We support Windows and that has no man(1)
> or nroff(1) binaries.
Please do not change the behaviour of 'svn help' for this.
It needs compiled-in strings so that it works on any platform.
I think man pages only matter on UNIX-like platforms. They shouldn't
even be generated during builds on other platforms. They should be an
alternative to 'svn help' so that UNIX users have a familiar interface
to this documentation.
If you manage to generate both the compiled-in strings and the man
pages (on UNIX only) from the same source format during the build,
I"m happy with that. If you manage to generate man pages from
'svn help' output during the build process on UNIX, then I'm happy, too.
Re: man pages for Subversion
Posted by Daniel Shahaf <da...@elego.de>.
James K. Lowden wrote on Wed, Aug 07, 2013 at 20:13:16 -0400:
> On Wed, 7 Aug 2013 12:45:36 +0300
> Daniel Shahaf <da...@elego.de> wrote:
>
> > Then individual sections were edited by hand, sometimes to
> > > put finishing touches on the formatting, sometimes to clarify the
> > > text.
> >
> > How can we incorporate that into the dev cycle?
> >
> > if manual tweaks are required, we would have to follow a scheme
> > similar to translations: the master English output
> > (subversion/svn/svn.c:svn_cl__cmd_table) is updated, and afterwards
> > someone comes along, regenerates the manpage, re-applies the manual
> > edits, and commits. The problem is that a release may happen before
> > someone gets to update the semi-manually-generated man pages, which
> > will cause them to be out-of-date (with respect to 'svn help' output)
> > in the release.
> >
> > So, in addition to Stefan's point about incorporating your man pages
> > into the build system, I think it'll be useful to eliminate the need
> > for manual steps.
>
> First of all, let me say thanks for the positive response from
> everyone, and thanks for asking. :-) I wasn't sure how my message
> would be received, and I'm gratified I wasn't wasting my time.
>
Perhaps this is a good time to point out that you were writing man pages
for 1.7, which is in "Security fixes only" mode. The supported release
is 1.8 and trunk (where new development should be targeted) is 1.9.
> 1. Keep the man pages in mdoc format as the "souce code".
> 2. Convert "svn foo --help" and "svn help foo" to invoke man(1), as
> git does. Or use groff to generate plain text for static inclusion in
> the the binary.
As Bert said: not possible. We support Windows and that has no man(1)
or nroff(1) binaries.
The Windows build system already has Python available. (You *might* be
able to assume Perl since it's required for OpenSSL, but that requires
discussion.) So you'd need a solution that allows compiling the man
pages source to C strings that uses only Python (and maybe Perl).
> My manualize script is just a throwaway hack; it was easier (probably)
> to parse the help output to generate the synopsis and option lists.
> Going forward, it's probably not worthwhile to try to maintain a
> database of options keyed by subcommand or whatever. Although the
You realise we already have such a database? svn_cl__cmd_table in
subversion/svn/svn.c.
> Contrary to popular wisdom, good old troff is the best system there is
Clearly you have a strong opinion on this.
> So: unless you already have a database of options somewhere, I'd
See above.
> To deal with the copyright issue, I would put "Donated to the
> Subversion project by James K. Lowden" or similar in each file in the
> Authors section.
ASF generally recommends against @author tag on source code, but having
an acknowledgement on the compiled documentation might be okay I think
--- especially given that it's pretty common for man pages to carry such
acknowledgements.
Re: man pages for Subversion
Posted by Stefan Sperling <st...@elego.de>.
On Thu, Aug 08, 2013 at 09:20:16AM +0200, Alan Barrett wrote:
> On Wed, 07 Aug 2013, James K. Lowden wrote:
> >If HTML is desired, there are a couple of options. I haven't used it,
> >but troff2page sounds nice.
> >http://www.ccs.neu.edu/home/dorai/troff2page/index.html#node_toc_start
> >GNU troff also supports HTML output, but I haven't had great luck with
> >it.
>
> For rendering man pages to plain text, HTML, or PDF, please consider
> the BSD-licenced mandoc program (which, confusingly, is part of a
> package called "mdocml"). See <http://mdocml.bsd.lv>. The HTML and
> PDF output from mandoc is superior to the corresponding output from
> groff, but mandoc is more fussy about errors in the input.
>
> --apb (Alan Barrett)
I concur. mandoc should be used if available, otherwise fall back
to groff.
However, as a first step, having pages generated by groff would
be just fine. Hooking in another tool is easy once that's done.
Re: man pages for Subversion
Posted by Alan Barrett <ap...@cequrux.com>.
On Wed, 07 Aug 2013, James K. Lowden wrote:
>If HTML is desired, there are a couple of options. I haven't used it,
>but troff2page sounds nice.
>http://www.ccs.neu.edu/home/dorai/troff2page/index.html#node_toc_start
>GNU troff also supports HTML output, but I haven't had great luck with
>it.
For rendering man pages to plain text, HTML, or PDF, please
consider the BSD-licenced mandoc program (which, confusingly, is
part of a package called "mdocml"). See <http://mdocml.bsd.lv>.
The HTML and PDF output from mandoc is superior to the
corresponding output from groff, but mandoc is more fussy about errors
in the input.
--apb (Alan Barrett)
Re: man pages for Subversion
Posted by "James K. Lowden" <jk...@schemamania.org>.
On Wed, 7 Aug 2013 12:45:36 +0300
Daniel Shahaf <da...@elego.de> wrote:
> Then individual sections were edited by hand, sometimes to
> > put finishing touches on the formatting, sometimes to clarify the
> > text.
>
> How can we incorporate that into the dev cycle?
>
> if manual tweaks are required, we would have to follow a scheme
> similar to translations: the master English output
> (subversion/svn/svn.c:svn_cl__cmd_table) is updated, and afterwards
> someone comes along, regenerates the manpage, re-applies the manual
> edits, and commits. The problem is that a release may happen before
> someone gets to update the semi-manually-generated man pages, which
> will cause them to be out-of-date (with respect to 'svn help' output)
> in the release.
>
> So, in addition to Stefan's point about incorporating your man pages
> into the build system, I think it'll be useful to eliminate the need
> for manual steps.
First of all, let me say thanks for the positive response from
everyone, and thanks for asking. :-) I wasn't sure how my message
would be received, and I'm gratified I wasn't wasting my time.
It seems we agree having man pages would be a good thing; the next
question is how to keep them up to date. If we can agree on how to do
that, I'm willing to make changes that make the documentation more
amenable to the build system.
I suggest the following:
1. Keep the man pages in mdoc format as the "souce code".
2. Convert "svn foo --help" and "svn help foo" to invoke man(1), as
git does. Or use groff to generate plain text for static inclusion in
the the binary.
3. Add a cross-reference section to each page.
4. Consider keeping the manual as PDFs on the website. If you do, add
PDF escapes to the mdoc text to generate hyperlinks. (That's a
feature of groff nowadays.)
5. Consider a table of contents and an index.
If HTML is desired, there are a couple of options. I haven't used it,
but troff2page sounds nice.
http://www.ccs.neu.edu/home/dorai/troff2page/index.html#node_toc_start
GNU troff also supports HTML output, but I haven't had great luck with
it.
My manualize script is just a throwaway hack; it was easier (probably)
to parse the help output to generate the synopsis and option lists.
Going forward, it's probably not worthwhile to try to maintain a
database of options keyed by subcommand or whatever. Although the
synopsis and command-specific options are slightly redundant, keeping
them up to date and consistent won't be that much work. Subversion
doesn't change all that much at that level.
Contrary to popular wisdom, good old troff is the best system there is
for maintaining documentation. It's the least typing and the least
processing: the macros are just a few characters, and "processing"
mostly consists of copying the files to somewhere man(1) will find
them.
I say that having tried others: I used Doxygen and DocBook, and even
text2man. Troff is ugly, sure; Brian Kernighan called the syntax
"rebarbative". At the end of the day, though, you have to type
*something* to indicate "this is a list" and "this is an optional
function that takes an argument". mdoc is made for that, and it's
about 10% the size of e.g. DocBook, no matter how measured.
I'm completely sympathetic to the isssue of keeping the documentation
up to date. I've maintained the FreeTDS user guide for 10 years.
There's really nothing for it except to update as you go along, and to
review it (over and over) prior to release.
I confess I don't know how the Subversion source code is organized. IMO
the best place to keep documentation is in the source file directory.
I learned that trick from Dennis Ritchie. ;-)
Some of the text is shared between pages. To generate my drafts, I
just let ./manualize crank away. If mdoc is the canonical source, then
common text goes in a shared file. (There's really no need to assemble
the fragments; troff supports a ".so" directive that works like
#include in C.)
So: unless you already have a database of options somewhere, I'd
suggest maintaining the docs in mdoc format. If you like that idea,
I'm willing to make it easy. I would:
1. regenerate them using the HEAD version (or take the output from
someone who's currently hacking on it)
2. fix up the text some more
3. remove the shared text to shared files
4. add SEE ALSO cross references
5. deliver mdoc files and a GNU Makefile that with targets "install"
and "pdf"
More than that I won't promise, but we can discuss it. :-)
To deal with the copyright issue, I would put "Donated to the
Subversion project by James K. Lowden" or similar in each file in the
Authors section.
Happy to be of service. Let me know your thoughts.
Regards,
--jkl
Re: man pages for Subversion
Posted by Daniel Shahaf <da...@elego.de>.
James K. Lowden wrote on Wed, Aug 07, 2013 at 00:07:08 -0400:
> I offer the above mdoc man pages to the Subversion project as a token
> of my appreciation, and in the hope they'll be included in future
> versions.
>
Thanks!
> The pages represent perhaps 30 hours of work. The --help text was
> processed through the bin/manualize Perl script to create a rough
> mdoc draft. Then individual sections were edited by hand, sometimes to
> put finishing touches on the formatting, sometimes to clarify the text.
How can we incorporate that into the dev cycle?
We don't add new command-line options between patch releases, but we may
change the help output in other, bidirectionally-compatible ways. If we
can just run 'make' to generate updated manpages, then the manpages will
always be in sync with the HEAD code; but if manual tweaks are required,
we would have to follow a scheme similar to translations: the master
English output (subversion/svn/svn.c:svn_cl__cmd_table) is updated, and
afterwards someone comes along, regenerates the manpage, re-applies the
manual edits, and commits. The problem is that a release may happen
before someone gets to update the semi-manually-generated man pages,
which will cause them to be out-of-date (with respect to 'svn help'
output) in the release.
So, in addition to Stefan's point about incorporating your man pages
into the build system, I think it'll be useful to eliminate the need for
manual steps.
Re: man pages for Subversion
Posted by Daniel Shahaf <da...@elego.de>.
James K. Lowden wrote on Wed, Aug 07, 2013 at 00:07:08 -0400:
> The final result is surely a derived work, but it's not clear to me if
> the --help text is copyrighted. I am prepare to assign it as befits
> the project, if need be.
The ASF never requires copyright assignment. However, license grants
are welcome, in particular for larger contributions; see
https://www.apache.org/licenses/#clas