Re: [lucy-dev] Markdown for documentation (redux)

[Marvin Humphrey <ma...@rectangular.com>]

On Sun, Nov 23, 2014 at 10:05 AM, Nick Wellnhofer <we...@aevum.de> wrote:
> I made the necessary changes for both Lucy and Clownfish in the `markdown`
> branches. Man pages and HTML documentation for the C bindings, as well as
> Perl POD are now autogenerated from Markdown DocuComments.

It's really cool to see this implemented, Nick!

> I also uploaded a sample of the new HTML documentation for the C APIs:
>
> http://lucy.apache.org/docs/c/
> http://lucy.apache.org/docs/c/cfish.html
> http://lucy.apache.org/docs/c/lucy.html

The HTML looks spiffy!

The W3C validator reports a few problems with some of the generated HTML -- it
might be nice to clean those up.

> C API docs are only generated for public classes. Since not all classes are
> marked public yet, some pages are missing and some links are dead.

To be honest, I'm not thrilled with what we're putting out there as the public
API for either Clownfish or Lucy -- there are a number of things marked
"public" which really shouldn't be.  The subset of the API published in the
Perl docs is what I would consider canonical.

However, I'm not going to harp on this issue until after I've made substantial
progress on the higher priority of adding host languages.

> The Markdown documentation supports links to classes with a custom
> `clownfish` URI scheme. A link like
>
>     [RegexTokenizer](clownfish:class:lucy:RegexTokenizer)
>
> will be automatically converted to point to right location depending on the
> host language. This can also be (ab)used for other language-dependent stuff.

I think this is a reasonable extension.

Question: does CommonMark support conditional inclusion of code blocks?
It would be nice to put things such as the Lucy tutorial into core, but
providing host-specific code samples poses a challenge.

Marvin Humphrey

Re: [lucy-dev] Markdown for documentation (redux)

[Nick Wellnhofer <we...@aevum.de>]

On 01/12/2014 07:08, Marvin Humphrey wrote:
> The W3C validator reports a few problems with some of the generated HTML -- it
> might be nice to clean those up.

One thing is a missing <title>. I originally planned to allow custom HTML 
headers and footers for the generated docs but this would require a simple 
templating system to create useful titles. I think I'll defer this feature for 
now.

The other is a missing charset declaration. This raises the question whether 
we want to allow UTF-8 in our docucomments. This isn't a problem for HTML and 
POD. For man pages, UTF-8 support is system-dependent. My guess is that most 
modern systems support UTF-8. If this turns out to be a problem, we can also 
escape non-ASCII characters.

> To be honest, I'm not thrilled with what we're putting out there as the public
> API for either Clownfish or Lucy -- there are a number of things marked
> "public" which really shouldn't be.  The subset of the API published in the
> Perl docs is what I would consider canonical.

I only uploaded the HTML documentation for review. I also have no plans to 
publish C API documentation for Lucy. But I really want to get started with 
defining and publishing the Clownfish C API.

> Question: does CommonMark support conditional inclusion of code blocks?
> It would be nice to put things such as the Lucy tutorial into core, but
> providing host-specific code samples poses a challenge.

CommonMark supports the "fenced" code blocks of GitHub Flavored Markdown that 
allow to add an "info string":

     ```ruby
     require 'redcarpet'
     markdown = Redcarpet.new("Hello World!")
     puts markdown.to_html
     ```

It also allows to use tildes to be used instead of backticks:

     http://spec.commonmark.org/0.12/#fenced-code-blocks

The HTML renderer uses the first word of the info string to add a class like 
"language-ruby". The info string is also available in the parsed syntax tree, 
so it's easy to remove the code blocks we're not interested in.

Nick