[PROPOSAL] Improving Documentation for Apache Flume

[Israel Ekpo <is...@aicer.org>]

Hello Everyone,

I have been giving this a great deal of thought over the last 3 weeks.

I would really appreciate feedback on how we should proceed on this from
users, developers (committers), and project management perspectives.

It would really be nice for the documentation for the project to be moved
from git into the wiki and organized in a manner that makes it easy to
locate information on features, components, processes etc.

This will allow us to:

1. Break things down into easily digestible chunks of information rather
than having it in only long page.

2. Be able to update and publish information for previous and future
releases immediately.


Some of the wiki pages are buried deep down and may not be reachable from
the wiki home page.

It's also a bit confusing to find information from the two versions of the
wiki home page

https://cwiki.apache.org/FLUME/home.html

https://cwiki.apache.org/confluence/display/FLUME/Home

I like how projects like Apache Solr (http://wiki.apache.org/solr) and
PHP.net (http://www.php.net/manual/en) organize information.

I think we can learn a few things from how their project's documentation is
structured.

We could add comments to specific features, if they behave or should be
configured differently in different versions/releases.

What I would like to start with is:

1. Re-organization of the wiki home page to have a navigable table of
contents carefully broken down into easy-to-digest sections.
2. Sections that document the overall architecture of the product.
3. Sections that document the various features of the product (Sources,
Channels, Sinks, Interceptors, DeSerializers etc).
4. Sections that document tips, techniques and processes for contributors,
developers troubleshooting issues, active committers and PMC members.
5. FAQ index compiling and providing solutions to commonly-asked questions
from the user and developer mailing lists.
6. Re-introduction of Flume explaining what it is and what it isn't. Also
explaining use cases where Flume is applicable.

Making this information readily available for new comers will really
improve the rate of adoption and strengthen the community in the immediate
future.

I have created a brainstorming task here to track this:

https://issues.apache.org/jira/browse/FLUME-1987

Please add your responses and comments to the JIRA issue instead. It will
make things easier to track.

Thanks.