Druid configuration documentation should not be organized in tables

[Roman Leventov <le...@apache.org>]

Rows in tables in markdown must fit a single line (unless you use
full-blown HTML <table><tr><td> syntax), including the configuration option
descriptions. Descriptions should properly be large, sometimes even
multi-paragraph texts with a discussion of how the parameter should be
used, how lowering and increasing the value affects the system, etc.
Instead, currently many configuration option descriptions in Druid consist
of a single sentence that doesn't explain much, if anything, beyond the
configuration option's name itself.

Instead, I suggest that configuration options are titled sections in the
documentation each on its own, for example:

### `druid.coordinator.someAwesomeOption`

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod
tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim
veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea
commodo consequat.

Duis aute irure dolor in reprehenderit in voluptate velit esse cillum
dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

*Default value:* `false`

*Required:* no

### `druid.coordinator.nextAwesomeOption`

...

https://github.com/apache/incubator-druid/issues/4861 would fix this
problem too.

Re: Druid configuration documentation should not be organized in tables

[Roman Leventov <le...@gmail.com>]

The control I didn't know the name of is "collapsible" and it's even
supported on Github:
https://gist.github.com/joyrexus/16041f2426450e73f5df9391f7f7ae5f

On Thu, 7 Mar 2019 at 20:13, Roman Leventov <le...@gmail.com> wrote:

> I've created an issue about this problem:
> https://github.com/apache/incubator-druid/issues/7210
>
> On Tue, 5 Mar 2019 at 20:36, Eyal Yurman
> <ey...@verizonmedia.com.invalid> wrote:
>
>> Just for comparison, this looks nice, isn't it?
>> https://spark.apache.org/docs/latest/configuration.html
>> Original .md:
>> https://github.com/apache/spark/blob/master/docs/configuration.md
>>
>> On Tue, Mar 5, 2019 at 3:20 PM Clint Wylie <cl...@imply.io> wrote:
>>
>> > I definitely agree that a lot of these configuration option docs have
>> > outgrown the tables. I do however think the giant table as a reference
>> to
>> > lookup configuration property names and default values is handy; I think
>> > maybe a system of links in the tables, like "see coordinator balancing
>> docs
>> > for more details" or whatever, where the implications of settings could
>> be
>> > described in greater detail might be more appropriate than embedding
>> all of
>> > the information in the tables. I do like the idea of like nested hidden
>> > larger descriptions so you can sort of have both a table and a fly out
>> > large description to keep it similar to what we have and not have to
>> update
>> > or add documentation in multiple places, but I have no idea how that
>> works
>> > in markdown world or if it's possible.
>> >
>> > On Tue, Mar 5, 2019 at 3:04 PM Roman Leventov <le...@gmail.com>
>> > wrote:
>> >
>> > > On Mon, 4 Mar 2019 at 21:36, Gian Merlino <gi...@apache.org> wrote:
>> > >
>> > > > I'm a bit scared that doing this would make the already large list
>> of
>> > > > configuration options (
>> > > > http://druid.io/docs/latest/configuration/index.html)
>> > > > become even more daunting and difficult to wrap one's head around.
>> > > >
>> > >
>> > > Sounds like a good nudge for ourselves to always seek for
>> opportunities
>> > to
>> > > remove configurations! Besides, we may reorganize the page to show
>> only
>> > > configuration names, and then a full description is open (like a
>> > "spoiler"
>> > > but not exactly, I'm not sure how this HTML control is called.
>> Similar to
>> > > how FAQ lists are organized sometimes) when the option name is
>> clicked.
>> > > Markdown->HTML converter that we use may support this, so the doc may
>> > need
>> > > to become a proper HTML page.
>> > >
>> > > Since, so often, multiple parameters interact with each other, maybe
>> it
>> > > > would make sense for the larger explanatory texts to be written in
>> > their
>> > > > own sections, with the parameter tables linking to them? It makes
>> sense
>> > > > with processing buffer size and num threads configs, for example,
>> since
>> > > > they aren't just important alone but they do relate to each other as
>> > > well.
>> > > >
>> > >
>> > > I'm afraid the necessity to find a place for a section, inner feeling
>> > that
>> > > a section should be "big enough" to justify its existence, the
>> necessity
>> > of
>> > > inter-linking in markdown will make it even less likely in practice
>> that
>> > > developers will create good docs with comprehensive discussions of
>> > > configuration options and their values.
>> > >
>> >
>>
>

Re: Druid configuration documentation should not be organized in tables

[Roman Leventov <le...@gmail.com>]

I've created an issue about this problem:
https://github.com/apache/incubator-druid/issues/7210

On Tue, 5 Mar 2019 at 20:36, Eyal Yurman <ey...@verizonmedia.com.invalid>
wrote:

> Just for comparison, this looks nice, isn't it?
> https://spark.apache.org/docs/latest/configuration.html
> Original .md:
> https://github.com/apache/spark/blob/master/docs/configuration.md
>
> On Tue, Mar 5, 2019 at 3:20 PM Clint Wylie <cl...@imply.io> wrote:
>
> > I definitely agree that a lot of these configuration option docs have
> > outgrown the tables. I do however think the giant table as a reference to
> > lookup configuration property names and default values is handy; I think
> > maybe a system of links in the tables, like "see coordinator balancing
> docs
> > for more details" or whatever, where the implications of settings could
> be
> > described in greater detail might be more appropriate than embedding all
> of
> > the information in the tables. I do like the idea of like nested hidden
> > larger descriptions so you can sort of have both a table and a fly out
> > large description to keep it similar to what we have and not have to
> update
> > or add documentation in multiple places, but I have no idea how that
> works
> > in markdown world or if it's possible.
> >
> > On Tue, Mar 5, 2019 at 3:04 PM Roman Leventov <le...@gmail.com>
> > wrote:
> >
> > > On Mon, 4 Mar 2019 at 21:36, Gian Merlino <gi...@apache.org> wrote:
> > >
> > > > I'm a bit scared that doing this would make the already large list of
> > > > configuration options (
> > > > http://druid.io/docs/latest/configuration/index.html)
> > > > become even more daunting and difficult to wrap one's head around.
> > > >
> > >
> > > Sounds like a good nudge for ourselves to always seek for opportunities
> > to
> > > remove configurations! Besides, we may reorganize the page to show only
> > > configuration names, and then a full description is open (like a
> > "spoiler"
> > > but not exactly, I'm not sure how this HTML control is called. Similar
> to
> > > how FAQ lists are organized sometimes) when the option name is clicked.
> > > Markdown->HTML converter that we use may support this, so the doc may
> > need
> > > to become a proper HTML page.
> > >
> > > Since, so often, multiple parameters interact with each other, maybe it
> > > > would make sense for the larger explanatory texts to be written in
> > their
> > > > own sections, with the parameter tables linking to them? It makes
> sense
> > > > with processing buffer size and num threads configs, for example,
> since
> > > > they aren't just important alone but they do relate to each other as
> > > well.
> > > >
> > >
> > > I'm afraid the necessity to find a place for a section, inner feeling
> > that
> > > a section should be "big enough" to justify its existence, the
> necessity
> > of
> > > inter-linking in markdown will make it even less likely in practice
> that
> > > developers will create good docs with comprehensive discussions of
> > > configuration options and their values.
> > >
> >
>

Re: Druid configuration documentation should not be organized in tables

[Eyal Yurman <ey...@verizonmedia.com.INVALID>]

Just for comparison, this looks nice, isn't it?
https://spark.apache.org/docs/latest/configuration.html
Original .md:
https://github.com/apache/spark/blob/master/docs/configuration.md

On Tue, Mar 5, 2019 at 3:20 PM Clint Wylie <cl...@imply.io> wrote:

> I definitely agree that a lot of these configuration option docs have
> outgrown the tables. I do however think the giant table as a reference to
> lookup configuration property names and default values is handy; I think
> maybe a system of links in the tables, like "see coordinator balancing docs
> for more details" or whatever, where the implications of settings could be
> described in greater detail might be more appropriate than embedding all of
> the information in the tables. I do like the idea of like nested hidden
> larger descriptions so you can sort of have both a table and a fly out
> large description to keep it similar to what we have and not have to update
> or add documentation in multiple places, but I have no idea how that works
> in markdown world or if it's possible.
>
> On Tue, Mar 5, 2019 at 3:04 PM Roman Leventov <le...@gmail.com>
> wrote:
>
> > On Mon, 4 Mar 2019 at 21:36, Gian Merlino <gi...@apache.org> wrote:
> >
> > > I'm a bit scared that doing this would make the already large list of
> > > configuration options (
> > > http://druid.io/docs/latest/configuration/index.html)
> > > become even more daunting and difficult to wrap one's head around.
> > >
> >
> > Sounds like a good nudge for ourselves to always seek for opportunities
> to
> > remove configurations! Besides, we may reorganize the page to show only
> > configuration names, and then a full description is open (like a
> "spoiler"
> > but not exactly, I'm not sure how this HTML control is called. Similar to
> > how FAQ lists are organized sometimes) when the option name is clicked.
> > Markdown->HTML converter that we use may support this, so the doc may
> need
> > to become a proper HTML page.
> >
> > Since, so often, multiple parameters interact with each other, maybe it
> > > would make sense for the larger explanatory texts to be written in
> their
> > > own sections, with the parameter tables linking to them? It makes sense
> > > with processing buffer size and num threads configs, for example, since
> > > they aren't just important alone but they do relate to each other as
> > well.
> > >
> >
> > I'm afraid the necessity to find a place for a section, inner feeling
> that
> > a section should be "big enough" to justify its existence, the necessity
> of
> > inter-linking in markdown will make it even less likely in practice that
> > developers will create good docs with comprehensive discussions of
> > configuration options and their values.
> >
>

Re: Druid configuration documentation should not be organized in tables

[Clint Wylie <cl...@imply.io>]

I definitely agree that a lot of these configuration option docs have
outgrown the tables. I do however think the giant table as a reference to
lookup configuration property names and default values is handy; I think
maybe a system of links in the tables, like "see coordinator balancing docs
for more details" or whatever, where the implications of settings could be
described in greater detail might be more appropriate than embedding all of
the information in the tables. I do like the idea of like nested hidden
larger descriptions so you can sort of have both a table and a fly out
large description to keep it similar to what we have and not have to update
or add documentation in multiple places, but I have no idea how that works
in markdown world or if it's possible.

On Tue, Mar 5, 2019 at 3:04 PM Roman Leventov <le...@gmail.com> wrote:

> On Mon, 4 Mar 2019 at 21:36, Gian Merlino <gi...@apache.org> wrote:
>
> > I'm a bit scared that doing this would make the already large list of
> > configuration options (
> > http://druid.io/docs/latest/configuration/index.html)
> > become even more daunting and difficult to wrap one's head around.
> >
>
> Sounds like a good nudge for ourselves to always seek for opportunities to
> remove configurations! Besides, we may reorganize the page to show only
> configuration names, and then a full description is open (like a "spoiler"
> but not exactly, I'm not sure how this HTML control is called. Similar to
> how FAQ lists are organized sometimes) when the option name is clicked.
> Markdown->HTML converter that we use may support this, so the doc may need
> to become a proper HTML page.
>
> Since, so often, multiple parameters interact with each other, maybe it
> > would make sense for the larger explanatory texts to be written in their
> > own sections, with the parameter tables linking to them? It makes sense
> > with processing buffer size and num threads configs, for example, since
> > they aren't just important alone but they do relate to each other as
> well.
> >
>
> I'm afraid the necessity to find a place for a section, inner feeling that
> a section should be "big enough" to justify its existence, the necessity of
> inter-linking in markdown will make it even less likely in practice that
> developers will create good docs with comprehensive discussions of
> configuration options and their values.
>

Re: Druid configuration documentation should not be organized in tables

[Roman Leventov <le...@gmail.com>]

On Mon, 4 Mar 2019 at 21:36, Gian Merlino <gi...@apache.org> wrote:

> I'm a bit scared that doing this would make the already large list of
> configuration options (
> http://druid.io/docs/latest/configuration/index.html)
> become even more daunting and difficult to wrap one's head around.
>

Sounds like a good nudge for ourselves to always seek for opportunities to
remove configurations! Besides, we may reorganize the page to show only
configuration names, and then a full description is open (like a "spoiler"
but not exactly, I'm not sure how this HTML control is called. Similar to
how FAQ lists are organized sometimes) when the option name is clicked.
Markdown->HTML converter that we use may support this, so the doc may need
to become a proper HTML page.

Since, so often, multiple parameters interact with each other, maybe it
> would make sense for the larger explanatory texts to be written in their
> own sections, with the parameter tables linking to them? It makes sense
> with processing buffer size and num threads configs, for example, since
> they aren't just important alone but they do relate to each other as well.
>

I'm afraid the necessity to find a place for a section, inner feeling that
a section should be "big enough" to justify its existence, the necessity of
inter-linking in markdown will make it even less likely in practice that
developers will create good docs with comprehensive discussions of
configuration options and their values.

Re: Druid configuration documentation should not be organized in tables

[Gian Merlino <gi...@apache.org>]

I'm a bit scared that doing this would make the already large list of
configuration options (http://druid.io/docs/latest/configuration/index.html)
become even more daunting and difficult to wrap one's head around.

Since, so often, multiple parameters interact with each other, maybe it
would make sense for the larger explanatory texts to be written in their
own sections, with the parameter tables linking to them? It makes sense
with processing buffer size and num threads configs, for example, since
they aren't just important alone but they do relate to each other as well.

On Mon, Mar 4, 2019 at 4:03 PM Roman Leventov <le...@apache.org> wrote:

> Rows in tables in markdown must fit a single line (unless you use
> full-blown HTML <table><tr><td> syntax), including the configuration option
> descriptions. Descriptions should properly be large, sometimes even
> multi-paragraph texts with a discussion of how the parameter should be
> used, how lowering and increasing the value affects the system, etc.
> Instead, currently many configuration option descriptions in Druid consist
> of a single sentence that doesn't explain much, if anything, beyond the
> configuration option's name itself.
>
> Instead, I suggest that configuration options are titled sections in the
> documentation each on its own, for example:
>
> ### `druid.coordinator.someAwesomeOption`
>
> Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod
> tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim
> veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea
> commodo consequat.
>
> Duis aute irure dolor in reprehenderit in voluptate velit esse cillum
> dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
> proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
>
> *Default value:* `false`
>
> *Required:* no
>
> ### `druid.coordinator.nextAwesomeOption`
>
> ...
>
> https://github.com/apache/incubator-druid/issues/4861 would fix this
> problem too.
>