You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apex.apache.org by Andy Perlitch <an...@datatorrent.com> on 2015/11/17 04:46:11 UTC
Proposal for Apex and Malhar Docs
Hello Apexers,
I'd like to propose a process for creating, storing, updating, and
displaying documentation of the apex and malhar projects.
*## Goals*
The goals for docs that I had in mind are:
- Versioned with the code base
- Easy to contribute to by anyone
- Displayed on apex.incubator.apache.org
- Simple deploy process
- Indexable by search engines
- Easily navigable; table of contents, support for nested "groups" of docs,
responsive search
*## Proposal*
My proposal is as follows:
1. Source files for docs will live in their respective git repositories,
under a `docs` folder. Format for said files will be github-flavored
markdown <https://help.github.com/articles/github-flavored-markdown/> (In
the future, other formats could be supported). This takes care of the first
two goals listed above.
2. In addition to the docs themselves, a single json file acting as an
index for the rendered docs will exist in `docs/index.json`. This will be
used to render a left navigation bar in the docs section.
3. Create a gulp <http://gulpjs.com/> task which:
a. pulls down these files for each released version of apex and malhar
(using the Github API)
b. creates HTML versions of each project/version/doc, each with the same
left-side navigation
c. creates a JSON file (per version) containing the raw markdown content
that can be used by Javascript on the page to power a fast search function
in the left-side navigation.
4. As part of the release process, run the aforementioned gulp task and
push to the site using the current `build.sh` script
*## Mockups*
This would result in an index page on
*apex.incubator.apache.org/docs/index.html
<http://apex.incubator.apache.org/docs/index.html>*:
[image: Inline image 1]
And this is a mockup of looking at an individual doc, e.g.
*apex.incubator.apache.org/docs/v3.2.0-incubating/contributing.html
<http://apex.incubator.apache.org/docs/v3.2.0-incubating/contributing.html>*
:
[image: Inline image 2]
The search feature could filter the docs shown in the side bar in real-time:
[image: Inline image 3]
*## Caveats*
No plan is perfect, and this one is no exception. A few caveats to this
plan are:
- Any updates to the docs in the repos require a manual build and push to
the website.
- This would be a custom solution, and while I will rely on existing
modules as much as possible, there will be a little glue-code
- The JSON files used by the search feature may become very large if the
docs become extremely large. This may hurt browser performance. I doubt
this will ever be a problem though.
*## Alternatives*
We could approach the problem of docs in a few different ways. Most notably
are solutions hosted by others (i.e., Documentation as a Service). Some
examples of this solution: Atlassian Confluence
<https://developer.atlassian.com/opensource/>, Readme.io, and Readthedocs
<https://readthedocs.org/>. I personally take issue with these for the
following reasons:
- docs not versioned along with the code (readthedocs is the exception here)
- most solutions are not very user-friendly (no real-time search)
- css-skinning is difficult or impossible
- system exists outside of current projects/website
If the community rejects the proposal put forth here, my choice of
alternative would be ReadTheDocs (see the features
<http://read-the-docs.readthedocs.org/en/latest/features.html>).
*Pros of ReadTheDocs:*
- docs are also versioned with codebase in git repo
- git hook automatically updates docs without any manual step
- out-of-the-box support for showing different versions of the docs
- decent default theme
*Cons of ReadTheDocs are: *
- separately hosted
- learning curve for usage
- difficulty for skinning
- difficulty extending/adding functionality
- Slow search/frustrating left-hand navigation
*## Conclusions*
Documentation is one of the first places potential new community
members/developers will go when vetting Apex for use. Ideally, it should
dazzle and inspire. I think the status-quo for documentation on open-source
projects is a low bar, and I am willing to put in the bit of work to lay
the foundation for excellent docs. Again, if my custom proposal seems too
problematic, my vote is for using ReadTheDocs.
Please share your thoughts.
-Andy
--
Regards,
Andy Perlitch
Software Engineer
DataTorrent Inc
(408)829-9319
Re: Proposal for Apex and Malhar Docs
Posted by Thomas Weise <th...@datatorrent.com>.
Andy,
Looks good, thanks for taking the initiative.
We will need the ability to host docs for multiple versions, linked from
the download page.
We should have he documentation built from the release branches:
release-3.2 for 3.2.x versions etc. This will allow us to fix mistakes in
he docs outside of the release tag.
Thanks,
Thomas
On Wed, Nov 18, 2015 at 1:21 PM, Timothy Farkas <ti...@datatorrent.com> wrote:
> +1 Very nice :)
>
> On Wed, Nov 18, 2015 at 1:15 PM, Andy Perlitch <an...@datatorrent.com>
> wrote:
>
> > Sorry, seems like the images didn't get through.
> >
> > - index page for the docs: http://i.imgur.com/ZB8bzhO.png
> > - individual doc example: http://i.imgur.com/BcGaIEQ.png
> > - realtime searching (taken from angular docs as example)
> > http://i.imgur.com/K6247Q9.gif
> >
> >
> > Andy
> >
> > On Mon, Nov 16, 2015 at 7:46 PM, Andy Perlitch <an...@datatorrent.com>
> > wrote:
> >
> >> Hello Apexers,
> >>
> >> I'd like to propose a process for creating, storing, updating, and
> >> displaying documentation of the apex and malhar projects.
> >>
> >>
> >> *## Goals*
> >> The goals for docs that I had in mind are:
> >>
> >> - Versioned with the code base
> >> - Easy to contribute to by anyone
> >> - Displayed on apex.incubator.apache.org
> >> - Simple deploy process
> >> - Indexable by search engines
> >> - Easily navigable; table of contents, support for nested "groups" of
> >> docs, responsive search
> >>
> >> *## Proposal*
> >> My proposal is as follows:
> >>
> >> 1. Source files for docs will live in their respective git repositories,
> >> under a `docs` folder. Format for said files will be github-flavored
> >> markdown <https://help.github.com/articles/github-flavored-markdown/>
> (In
> >> the future, other formats could be supported). This takes care of the
> first
> >> two goals listed above.
> >> 2. In addition to the docs themselves, a single json file acting as an
> >> index for the rendered docs will exist in `docs/index.json`. This will
> be
> >> used to render a left navigation bar in the docs section.
> >> 3. Create a gulp <http://gulpjs.com/> task which:
> >> a. pulls down these files for each released version of apex and malhar
> >> (using the Github API)
> >> b. creates HTML versions of each project/version/doc, each with the
> >> same left-side navigation
> >> c. creates a JSON file (per version) containing the raw markdown
> >> content that can be used by Javascript on the page to power a fast
> search
> >> function in the left-side navigation.
> >> 4. As part of the release process, run the aforementioned gulp task and
> >> push to the site using the current `build.sh` script
> >>
> >>
> >> *## Mockups*
> >> This would result in an index page on *
> apex.incubator.apache.org/docs/index.html
> >> <http://apex.incubator.apache.org/docs/index.html>*:
> >> [image: Inline image 1]
> >>
> >> And this is a mockup of looking at an individual doc, e.g. *
> apex.incubator.apache.org/docs/v3.2.0-incubating/contributing.html
> >> <
> http://apex.incubator.apache.org/docs/v3.2.0-incubating/contributing.html
> >*
> >> :
> >> [image: Inline image 2]
> >>
> >> The search feature could filter the docs shown in the side bar in
> >> real-time:
> >> [image: Inline image 3]
> >>
> >> *## Caveats*
> >> No plan is perfect, and this one is no exception. A few caveats to this
> >> plan are:
> >>
> >> - Any updates to the docs in the repos require a manual build and push
> to
> >> the website.
> >> - This would be a custom solution, and while I will rely on existing
> >> modules as much as possible, there will be a little glue-code
> >> - The JSON files used by the search feature may become very large if the
> >> docs become extremely large. This may hurt browser performance. I doubt
> >> this will ever be a problem though.
> >>
> >> *## Alternatives*
> >> We could approach the problem of docs in a few different ways. Most
> >> notably are solutions hosted by others (i.e., Documentation as a
> Service).
> >> Some examples of this solution: Atlassian Confluence
> >> <https://developer.atlassian.com/opensource/>, Readme.io, and
> Readthedocs
> >> <https://readthedocs.org/>. I personally take issue with these for the
> >> following reasons:
> >>
> >> - docs not versioned along with the code (readthedocs is the exception
> >> here)
> >> - most solutions are not very user-friendly (no real-time search)
> >> - css-skinning is difficult or impossible
> >> - system exists outside of current projects/website
> >>
> >> If the community rejects the proposal put forth here, my choice of
> >> alternative would be ReadTheDocs (see the features
> >> <http://read-the-docs.readthedocs.org/en/latest/features.html>).
> >>
> >> *Pros of ReadTheDocs:*
> >> - docs are also versioned with codebase in git repo
> >> - git hook automatically updates docs without any manual step
> >> - out-of-the-box support for showing different versions of the docs
> >> - decent default theme
> >>
> >> *Cons of ReadTheDocs are: *
> >> - separately hosted
> >> - learning curve for usage
> >> - difficulty for skinning
> >> - difficulty extending/adding functionality
> >> - Slow search/frustrating left-hand navigation
> >>
> >>
> >> *## Conclusions*
> >> Documentation is one of the first places potential new community
> >> members/developers will go when vetting Apex for use. Ideally, it should
> >> dazzle and inspire. I think the status-quo for documentation on
> open-source
> >> projects is a low bar, and I am willing to put in the bit of work to lay
> >> the foundation for excellent docs. Again, if my custom proposal seems
> too
> >> problematic, my vote is for using ReadTheDocs.
> >>
> >> Please share your thoughts.
> >>
> >> -Andy
> >>
> >>
> >> --
> >> Regards,
> >> Andy Perlitch
> >> Software Engineer
> >> DataTorrent Inc
> >> (408)829-9319
> >>
> >
> >
> >
> > --
> > Regards,
> > Andy Perlitch
> > Software Engineer
> > DataTorrent Inc
> > (408)829-9319
> >
>
Re: Proposal for Apex and Malhar Docs
Posted by Pramod Immaneni <pr...@datatorrent.com>.
Amruta,
I don't have much knowledge about the documentation tools. Do these tools
integrate with source control and build systems. Ideally we would like the
code and documentation to be in one place so that they can be versioned
together and kept in sync w.r.t content in a straightforward way that is
less prone to errors. Also we typically don't like to keep the end-products
(outputs) in version control so we would keep the source material for the
docs and let the build systems or release processes kick-off the necessary
tools to generate the end-products that are consumed by users.
Thanks
On Mon, Nov 23, 2015 at 9:27 PM, Amruta Phansalkar <am...@datatorrent.com>
wrote:
> Hi All,
>
> I wanted to share a great example of a "one-stop shop". This is the
> documentation portal of Planview. Not only is it clean and sleek, but the
> workflows in the documents are well thought of. Please also see how
> everything comes together -- videos, text-based content, FAQs, and training
> resources. The idea of a one-stop shop is to have customers help themselves
> as far as possible. Given the kind of growth we are expecting, it will not
> be long before we see extensive customer interaction.
>
> Now for the link: https://success.planview.com/
>
> Thanks much,
>
> ~ Amruta
>
> On Tue, Nov 24, 2015 at 10:53 AM, Amruta Phansalkar <
> amruta@datatorrent.com>
> wrote:
>
> > Hi,
> >
> > To streamline the documentation lifecycle, a documentation process is
> > essential. Along with the other processes, we will also need to set one
> up
> > for documentation.
> >
> > In a typical environment, a documentation process includes writers
> > creating content. Developers, QA, support, and product management review
> > the documents extensively for ensuring a high level of accuracy.
> >
> > The advantages of this approach are:
> >
> > - Developers and engineers are freed from the task of creating
> > documentation, leaving them free to pursue other critical activities
> > - Documentation is created considering the end-users' perspective
> > - Two sets of eagle eyes; those of the writer, and those of the
> > reviewer
> > - More accurate, concise, meaningful content (And why not, given the
> > collaboration)
> >
> > Some time in the early half of next week, I will arrange a call to
> > demonstrate how a document lifecycle works.
> >
> > Thanks much,
> >
> > ~ Amruta
> >
> >
> >
> >
> >
> >
> > On Tue, Nov 24, 2015 at 6:51 AM, Sasha Parfenov <sa...@datatorrent.com>
> > wrote:
> >
> >> +1 for Andy's custom proposal, with ReadTheDocs or another pre-built
> >> processor as the backup.
> >>
> >> Apex documentation is primarily written by engineers, and needs to
> closely
> >> reflect the current state of the code. Many Apache projects follow the
> >> model proposed here, with variations on the markup language and
> processor
> >> to HTML used. I think this model will results in a good process for
> >> contributing, updating, and ensuring consistent look and feel for the
> >> docs.
> >>
> >> Thanks,
> >> Sasha
> >>
> >> On Wed, Nov 18, 2015 at 1:43 PM, Andy Perlitch <an...@datatorrent.com>
> >> wrote:
> >>
> >> > Here are some more pros and cons on my proposed solution and
> >> > ReadTheDocs.org:
> >> >
> >> > *# Custom Proposal*
> >> > Pros
> >> > - Unified doc set search in left nav bar
> >> >
> >> > Cons
> >> > - Harder to maintain as it is custom code (node and angular code to
> >> boot,
> >> > which is not a skill many community members will have)
> >> >
> >> > *# ReadTheDocs*
> >> > Pros
> >> > - Potentially easier to maintain by community
> >> >
> >> > Cons
> >> > - Core and Malhar would have to be two separate readthedocs.org
> >> endpoints,
> >> > which means you can't do a single search that queries both doc sets
> >> >
> >> > -Andy
> >> >
> >> > On Wed, Nov 18, 2015 at 1:21 PM, Timothy Farkas <ti...@datatorrent.com>
> >> > wrote:
> >> >
> >> > > +1 Very nice :)
> >> > >
> >> > > On Wed, Nov 18, 2015 at 1:15 PM, Andy Perlitch <
> andy@datatorrent.com>
> >> > > wrote:
> >> > >
> >> > > > Sorry, seems like the images didn't get through.
> >> > > >
> >> > > > - index page for the docs: http://i.imgur.com/ZB8bzhO.png
> >> > > > - individual doc example: http://i.imgur.com/BcGaIEQ.png
> >> > > > - realtime searching (taken from angular docs as example)
> >> > > > http://i.imgur.com/K6247Q9.gif
> >> > > >
> >> > > >
> >> > > > Andy
> >> > > >
> >> > > > On Mon, Nov 16, 2015 at 7:46 PM, Andy Perlitch <
> >> andy@datatorrent.com>
> >> > > > wrote:
> >> > > >
> >> > > >> Hello Apexers,
> >> > > >>
> >> > > >> I'd like to propose a process for creating, storing, updating,
> and
> >> > > >> displaying documentation of the apex and malhar projects.
> >> > > >>
> >> > > >>
> >> > > >> *## Goals*
> >> > > >> The goals for docs that I had in mind are:
> >> > > >>
> >> > > >> - Versioned with the code base
> >> > > >> - Easy to contribute to by anyone
> >> > > >> - Displayed on apex.incubator.apache.org
> >> > > >> - Simple deploy process
> >> > > >> - Indexable by search engines
> >> > > >> - Easily navigable; table of contents, support for nested
> "groups"
> >> of
> >> > > >> docs, responsive search
> >> > > >>
> >> > > >> *## Proposal*
> >> > > >> My proposal is as follows:
> >> > > >>
> >> > > >> 1. Source files for docs will live in their respective git
> >> > repositories,
> >> > > >> under a `docs` folder. Format for said files will be
> >> github-flavored
> >> > > >> markdown <
> >> https://help.github.com/articles/github-flavored-markdown/>
> >> > > (In
> >> > > >> the future, other formats could be supported). This takes care of
> >> the
> >> > > first
> >> > > >> two goals listed above.
> >> > > >> 2. In addition to the docs themselves, a single json file acting
> >> as an
> >> > > >> index for the rendered docs will exist in `docs/index.json`. This
> >> will
> >> > > be
> >> > > >> used to render a left navigation bar in the docs section.
> >> > > >> 3. Create a gulp <http://gulpjs.com/> task which:
> >> > > >> a. pulls down these files for each released version of apex and
> >> > malhar
> >> > > >> (using the Github API)
> >> > > >> b. creates HTML versions of each project/version/doc, each with
> >> the
> >> > > >> same left-side navigation
> >> > > >> c. creates a JSON file (per version) containing the raw
> markdown
> >> > > >> content that can be used by Javascript on the page to power a
> fast
> >> > > search
> >> > > >> function in the left-side navigation.
> >> > > >> 4. As part of the release process, run the aforementioned gulp
> task
> >> > and
> >> > > >> push to the site using the current `build.sh` script
> >> > > >>
> >> > > >>
> >> > > >> *## Mockups*
> >> > > >> This would result in an index page on *
> >> > > apex.incubator.apache.org/docs/index.html
> >> > > >> <http://apex.incubator.apache.org/docs/index.html>*:
> >> > > >> [image: Inline image 1]
> >> > > >>
> >> > > >> And this is a mockup of looking at an individual doc, e.g. *
> >> > > apex.incubator.apache.org/docs/v3.2.0-incubating/contributing.html
> >> > > >> <
> >> > >
> >> >
> >>
> http://apex.incubator.apache.org/docs/v3.2.0-incubating/contributing.html
> >> > > >*
> >> > > >> :
> >> > > >> [image: Inline image 2]
> >> > > >>
> >> > > >> The search feature could filter the docs shown in the side bar in
> >> > > >> real-time:
> >> > > >> [image: Inline image 3]
> >> > > >>
> >> > > >> *## Caveats*
> >> > > >> No plan is perfect, and this one is no exception. A few caveats
> to
> >> > this
> >> > > >> plan are:
> >> > > >>
> >> > > >> - Any updates to the docs in the repos require a manual build and
> >> push
> >> > > to
> >> > > >> the website.
> >> > > >> - This would be a custom solution, and while I will rely on
> >> existing
> >> > > >> modules as much as possible, there will be a little glue-code
> >> > > >> - The JSON files used by the search feature may become very large
> >> if
> >> > the
> >> > > >> docs become extremely large. This may hurt browser performance. I
> >> > doubt
> >> > > >> this will ever be a problem though.
> >> > > >>
> >> > > >> *## Alternatives*
> >> > > >> We could approach the problem of docs in a few different ways.
> Most
> >> > > >> notably are solutions hosted by others (i.e., Documentation as a
> >> > > Service).
> >> > > >> Some examples of this solution: Atlassian Confluence
> >> > > >> <https://developer.atlassian.com/opensource/>, Readme.io, and
> >> > > Readthedocs
> >> > > >> <https://readthedocs.org/>. I personally take issue with these
> for
> >> > the
> >> > > >> following reasons:
> >> > > >>
> >> > > >> - docs not versioned along with the code (readthedocs is the
> >> exception
> >> > > >> here)
> >> > > >> - most solutions are not very user-friendly (no real-time search)
> >> > > >> - css-skinning is difficult or impossible
> >> > > >> - system exists outside of current projects/website
> >> > > >>
> >> > > >> If the community rejects the proposal put forth here, my choice
> of
> >> > > >> alternative would be ReadTheDocs (see the features
> >> > > >> <http://read-the-docs.readthedocs.org/en/latest/features.html>).
> >> > > >>
> >> > > >> *Pros of ReadTheDocs:*
> >> > > >> - docs are also versioned with codebase in git repo
> >> > > >> - git hook automatically updates docs without any manual step
> >> > > >> - out-of-the-box support for showing different versions of the
> docs
> >> > > >> - decent default theme
> >> > > >>
> >> > > >> *Cons of ReadTheDocs are: *
> >> > > >> - separately hosted
> >> > > >> - learning curve for usage
> >> > > >> - difficulty for skinning
> >> > > >> - difficulty extending/adding functionality
> >> > > >> - Slow search/frustrating left-hand navigation
> >> > > >>
> >> > > >>
> >> > > >> *## Conclusions*
> >> > > >> Documentation is one of the first places potential new community
> >> > > >> members/developers will go when vetting Apex for use. Ideally, it
> >> > should
> >> > > >> dazzle and inspire. I think the status-quo for documentation on
> >> > > open-source
> >> > > >> projects is a low bar, and I am willing to put in the bit of work
> >> to
> >> > lay
> >> > > >> the foundation for excellent docs. Again, if my custom proposal
> >> seems
> >> > > too
> >> > > >> problematic, my vote is for using ReadTheDocs.
> >> > > >>
> >> > > >> Please share your thoughts.
> >> > > >>
> >> > > >> -Andy
> >> > > >>
> >> > > >>
> >> > > >> --
> >> > > >> Regards,
> >> > > >> Andy Perlitch
> >> > > >> Software Engineer
> >> > > >> DataTorrent Inc
> >> > > >> (408)829-9319
> >> > > >>
> >> > > >
> >> > > >
> >> > > >
> >> > > > --
> >> > > > Regards,
> >> > > > Andy Perlitch
> >> > > > Software Engineer
> >> > > > DataTorrent Inc
> >> > > > (408)829-9319
> >> > > >
> >> > >
> >> >
> >> >
> >> >
> >> > --
> >> > Regards,
> >> > Andy Perlitch
> >> > Software Engineer
> >> > DataTorrent Inc
> >> > (408)829-9319
> >> >
> >>
> >
> >
>
Re: Proposal for Apex and Malhar Docs
Posted by Amruta Phansalkar <am...@datatorrent.com>.
Hi All,
I wanted to share a great example of a "one-stop shop". This is the
documentation portal of Planview. Not only is it clean and sleek, but the
workflows in the documents are well thought of. Please also see how
everything comes together -- videos, text-based content, FAQs, and training
resources. The idea of a one-stop shop is to have customers help themselves
as far as possible. Given the kind of growth we are expecting, it will not
be long before we see extensive customer interaction.
Now for the link: https://success.planview.com/
Thanks much,
~ Amruta
On Tue, Nov 24, 2015 at 10:53 AM, Amruta Phansalkar <am...@datatorrent.com>
wrote:
> Hi,
>
> To streamline the documentation lifecycle, a documentation process is
> essential. Along with the other processes, we will also need to set one up
> for documentation.
>
> In a typical environment, a documentation process includes writers
> creating content. Developers, QA, support, and product management review
> the documents extensively for ensuring a high level of accuracy.
>
> The advantages of this approach are:
>
> - Developers and engineers are freed from the task of creating
> documentation, leaving them free to pursue other critical activities
> - Documentation is created considering the end-users' perspective
> - Two sets of eagle eyes; those of the writer, and those of the
> reviewer
> - More accurate, concise, meaningful content (And why not, given the
> collaboration)
>
> Some time in the early half of next week, I will arrange a call to
> demonstrate how a document lifecycle works.
>
> Thanks much,
>
> ~ Amruta
>
>
>
>
>
>
> On Tue, Nov 24, 2015 at 6:51 AM, Sasha Parfenov <sa...@datatorrent.com>
> wrote:
>
>> +1 for Andy's custom proposal, with ReadTheDocs or another pre-built
>> processor as the backup.
>>
>> Apex documentation is primarily written by engineers, and needs to closely
>> reflect the current state of the code. Many Apache projects follow the
>> model proposed here, with variations on the markup language and processor
>> to HTML used. I think this model will results in a good process for
>> contributing, updating, and ensuring consistent look and feel for the
>> docs.
>>
>> Thanks,
>> Sasha
>>
>> On Wed, Nov 18, 2015 at 1:43 PM, Andy Perlitch <an...@datatorrent.com>
>> wrote:
>>
>> > Here are some more pros and cons on my proposed solution and
>> > ReadTheDocs.org:
>> >
>> > *# Custom Proposal*
>> > Pros
>> > - Unified doc set search in left nav bar
>> >
>> > Cons
>> > - Harder to maintain as it is custom code (node and angular code to
>> boot,
>> > which is not a skill many community members will have)
>> >
>> > *# ReadTheDocs*
>> > Pros
>> > - Potentially easier to maintain by community
>> >
>> > Cons
>> > - Core and Malhar would have to be two separate readthedocs.org
>> endpoints,
>> > which means you can't do a single search that queries both doc sets
>> >
>> > -Andy
>> >
>> > On Wed, Nov 18, 2015 at 1:21 PM, Timothy Farkas <ti...@datatorrent.com>
>> > wrote:
>> >
>> > > +1 Very nice :)
>> > >
>> > > On Wed, Nov 18, 2015 at 1:15 PM, Andy Perlitch <an...@datatorrent.com>
>> > > wrote:
>> > >
>> > > > Sorry, seems like the images didn't get through.
>> > > >
>> > > > - index page for the docs: http://i.imgur.com/ZB8bzhO.png
>> > > > - individual doc example: http://i.imgur.com/BcGaIEQ.png
>> > > > - realtime searching (taken from angular docs as example)
>> > > > http://i.imgur.com/K6247Q9.gif
>> > > >
>> > > >
>> > > > Andy
>> > > >
>> > > > On Mon, Nov 16, 2015 at 7:46 PM, Andy Perlitch <
>> andy@datatorrent.com>
>> > > > wrote:
>> > > >
>> > > >> Hello Apexers,
>> > > >>
>> > > >> I'd like to propose a process for creating, storing, updating, and
>> > > >> displaying documentation of the apex and malhar projects.
>> > > >>
>> > > >>
>> > > >> *## Goals*
>> > > >> The goals for docs that I had in mind are:
>> > > >>
>> > > >> - Versioned with the code base
>> > > >> - Easy to contribute to by anyone
>> > > >> - Displayed on apex.incubator.apache.org
>> > > >> - Simple deploy process
>> > > >> - Indexable by search engines
>> > > >> - Easily navigable; table of contents, support for nested "groups"
>> of
>> > > >> docs, responsive search
>> > > >>
>> > > >> *## Proposal*
>> > > >> My proposal is as follows:
>> > > >>
>> > > >> 1. Source files for docs will live in their respective git
>> > repositories,
>> > > >> under a `docs` folder. Format for said files will be
>> github-flavored
>> > > >> markdown <
>> https://help.github.com/articles/github-flavored-markdown/>
>> > > (In
>> > > >> the future, other formats could be supported). This takes care of
>> the
>> > > first
>> > > >> two goals listed above.
>> > > >> 2. In addition to the docs themselves, a single json file acting
>> as an
>> > > >> index for the rendered docs will exist in `docs/index.json`. This
>> will
>> > > be
>> > > >> used to render a left navigation bar in the docs section.
>> > > >> 3. Create a gulp <http://gulpjs.com/> task which:
>> > > >> a. pulls down these files for each released version of apex and
>> > malhar
>> > > >> (using the Github API)
>> > > >> b. creates HTML versions of each project/version/doc, each with
>> the
>> > > >> same left-side navigation
>> > > >> c. creates a JSON file (per version) containing the raw markdown
>> > > >> content that can be used by Javascript on the page to power a fast
>> > > search
>> > > >> function in the left-side navigation.
>> > > >> 4. As part of the release process, run the aforementioned gulp task
>> > and
>> > > >> push to the site using the current `build.sh` script
>> > > >>
>> > > >>
>> > > >> *## Mockups*
>> > > >> This would result in an index page on *
>> > > apex.incubator.apache.org/docs/index.html
>> > > >> <http://apex.incubator.apache.org/docs/index.html>*:
>> > > >> [image: Inline image 1]
>> > > >>
>> > > >> And this is a mockup of looking at an individual doc, e.g. *
>> > > apex.incubator.apache.org/docs/v3.2.0-incubating/contributing.html
>> > > >> <
>> > >
>> >
>> http://apex.incubator.apache.org/docs/v3.2.0-incubating/contributing.html
>> > > >*
>> > > >> :
>> > > >> [image: Inline image 2]
>> > > >>
>> > > >> The search feature could filter the docs shown in the side bar in
>> > > >> real-time:
>> > > >> [image: Inline image 3]
>> > > >>
>> > > >> *## Caveats*
>> > > >> No plan is perfect, and this one is no exception. A few caveats to
>> > this
>> > > >> plan are:
>> > > >>
>> > > >> - Any updates to the docs in the repos require a manual build and
>> push
>> > > to
>> > > >> the website.
>> > > >> - This would be a custom solution, and while I will rely on
>> existing
>> > > >> modules as much as possible, there will be a little glue-code
>> > > >> - The JSON files used by the search feature may become very large
>> if
>> > the
>> > > >> docs become extremely large. This may hurt browser performance. I
>> > doubt
>> > > >> this will ever be a problem though.
>> > > >>
>> > > >> *## Alternatives*
>> > > >> We could approach the problem of docs in a few different ways. Most
>> > > >> notably are solutions hosted by others (i.e., Documentation as a
>> > > Service).
>> > > >> Some examples of this solution: Atlassian Confluence
>> > > >> <https://developer.atlassian.com/opensource/>, Readme.io, and
>> > > Readthedocs
>> > > >> <https://readthedocs.org/>. I personally take issue with these for
>> > the
>> > > >> following reasons:
>> > > >>
>> > > >> - docs not versioned along with the code (readthedocs is the
>> exception
>> > > >> here)
>> > > >> - most solutions are not very user-friendly (no real-time search)
>> > > >> - css-skinning is difficult or impossible
>> > > >> - system exists outside of current projects/website
>> > > >>
>> > > >> If the community rejects the proposal put forth here, my choice of
>> > > >> alternative would be ReadTheDocs (see the features
>> > > >> <http://read-the-docs.readthedocs.org/en/latest/features.html>).
>> > > >>
>> > > >> *Pros of ReadTheDocs:*
>> > > >> - docs are also versioned with codebase in git repo
>> > > >> - git hook automatically updates docs without any manual step
>> > > >> - out-of-the-box support for showing different versions of the docs
>> > > >> - decent default theme
>> > > >>
>> > > >> *Cons of ReadTheDocs are: *
>> > > >> - separately hosted
>> > > >> - learning curve for usage
>> > > >> - difficulty for skinning
>> > > >> - difficulty extending/adding functionality
>> > > >> - Slow search/frustrating left-hand navigation
>> > > >>
>> > > >>
>> > > >> *## Conclusions*
>> > > >> Documentation is one of the first places potential new community
>> > > >> members/developers will go when vetting Apex for use. Ideally, it
>> > should
>> > > >> dazzle and inspire. I think the status-quo for documentation on
>> > > open-source
>> > > >> projects is a low bar, and I am willing to put in the bit of work
>> to
>> > lay
>> > > >> the foundation for excellent docs. Again, if my custom proposal
>> seems
>> > > too
>> > > >> problematic, my vote is for using ReadTheDocs.
>> > > >>
>> > > >> Please share your thoughts.
>> > > >>
>> > > >> -Andy
>> > > >>
>> > > >>
>> > > >> --
>> > > >> Regards,
>> > > >> Andy Perlitch
>> > > >> Software Engineer
>> > > >> DataTorrent Inc
>> > > >> (408)829-9319
>> > > >>
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > Regards,
>> > > > Andy Perlitch
>> > > > Software Engineer
>> > > > DataTorrent Inc
>> > > > (408)829-9319
>> > > >
>> > >
>> >
>> >
>> >
>> > --
>> > Regards,
>> > Andy Perlitch
>> > Software Engineer
>> > DataTorrent Inc
>> > (408)829-9319
>> >
>>
>
>
Re: Proposal for Apex and Malhar Docs
Posted by Amruta Phansalkar <am...@datatorrent.com>.
Hi,
To streamline the documentation lifecycle, a documentation process is
essential. Along with the other processes, we will also need to set one up
for documentation.
In a typical environment, a documentation process includes writers creating
content. Developers, QA, support, and product management review the
documents extensively for ensuring a high level of accuracy.
The advantages of this approach are:
- Developers and engineers are freed from the task of creating
documentation, leaving them free to pursue other critical activities
- Documentation is created considering the end-users' perspective
- Two sets of eagle eyes; those of the writer, and those of the reviewer
- More accurate, concise, meaningful content (And why not, given the
collaboration)
Some time in the early half of next week, I will arrange a call to
demonstrate how a document lifecycle works.
Thanks much,
~ Amruta
On Tue, Nov 24, 2015 at 6:51 AM, Sasha Parfenov <sa...@datatorrent.com>
wrote:
> +1 for Andy's custom proposal, with ReadTheDocs or another pre-built
> processor as the backup.
>
> Apex documentation is primarily written by engineers, and needs to closely
> reflect the current state of the code. Many Apache projects follow the
> model proposed here, with variations on the markup language and processor
> to HTML used. I think this model will results in a good process for
> contributing, updating, and ensuring consistent look and feel for the docs.
>
> Thanks,
> Sasha
>
> On Wed, Nov 18, 2015 at 1:43 PM, Andy Perlitch <an...@datatorrent.com>
> wrote:
>
> > Here are some more pros and cons on my proposed solution and
> > ReadTheDocs.org:
> >
> > *# Custom Proposal*
> > Pros
> > - Unified doc set search in left nav bar
> >
> > Cons
> > - Harder to maintain as it is custom code (node and angular code to boot,
> > which is not a skill many community members will have)
> >
> > *# ReadTheDocs*
> > Pros
> > - Potentially easier to maintain by community
> >
> > Cons
> > - Core and Malhar would have to be two separate readthedocs.org
> endpoints,
> > which means you can't do a single search that queries both doc sets
> >
> > -Andy
> >
> > On Wed, Nov 18, 2015 at 1:21 PM, Timothy Farkas <ti...@datatorrent.com>
> > wrote:
> >
> > > +1 Very nice :)
> > >
> > > On Wed, Nov 18, 2015 at 1:15 PM, Andy Perlitch <an...@datatorrent.com>
> > > wrote:
> > >
> > > > Sorry, seems like the images didn't get through.
> > > >
> > > > - index page for the docs: http://i.imgur.com/ZB8bzhO.png
> > > > - individual doc example: http://i.imgur.com/BcGaIEQ.png
> > > > - realtime searching (taken from angular docs as example)
> > > > http://i.imgur.com/K6247Q9.gif
> > > >
> > > >
> > > > Andy
> > > >
> > > > On Mon, Nov 16, 2015 at 7:46 PM, Andy Perlitch <andy@datatorrent.com
> >
> > > > wrote:
> > > >
> > > >> Hello Apexers,
> > > >>
> > > >> I'd like to propose a process for creating, storing, updating, and
> > > >> displaying documentation of the apex and malhar projects.
> > > >>
> > > >>
> > > >> *## Goals*
> > > >> The goals for docs that I had in mind are:
> > > >>
> > > >> - Versioned with the code base
> > > >> - Easy to contribute to by anyone
> > > >> - Displayed on apex.incubator.apache.org
> > > >> - Simple deploy process
> > > >> - Indexable by search engines
> > > >> - Easily navigable; table of contents, support for nested "groups"
> of
> > > >> docs, responsive search
> > > >>
> > > >> *## Proposal*
> > > >> My proposal is as follows:
> > > >>
> > > >> 1. Source files for docs will live in their respective git
> > repositories,
> > > >> under a `docs` folder. Format for said files will be github-flavored
> > > >> markdown <
> https://help.github.com/articles/github-flavored-markdown/>
> > > (In
> > > >> the future, other formats could be supported). This takes care of
> the
> > > first
> > > >> two goals listed above.
> > > >> 2. In addition to the docs themselves, a single json file acting as
> an
> > > >> index for the rendered docs will exist in `docs/index.json`. This
> will
> > > be
> > > >> used to render a left navigation bar in the docs section.
> > > >> 3. Create a gulp <http://gulpjs.com/> task which:
> > > >> a. pulls down these files for each released version of apex and
> > malhar
> > > >> (using the Github API)
> > > >> b. creates HTML versions of each project/version/doc, each with
> the
> > > >> same left-side navigation
> > > >> c. creates a JSON file (per version) containing the raw markdown
> > > >> content that can be used by Javascript on the page to power a fast
> > > search
> > > >> function in the left-side navigation.
> > > >> 4. As part of the release process, run the aforementioned gulp task
> > and
> > > >> push to the site using the current `build.sh` script
> > > >>
> > > >>
> > > >> *## Mockups*
> > > >> This would result in an index page on *
> > > apex.incubator.apache.org/docs/index.html
> > > >> <http://apex.incubator.apache.org/docs/index.html>*:
> > > >> [image: Inline image 1]
> > > >>
> > > >> And this is a mockup of looking at an individual doc, e.g. *
> > > apex.incubator.apache.org/docs/v3.2.0-incubating/contributing.html
> > > >> <
> > >
> >
> http://apex.incubator.apache.org/docs/v3.2.0-incubating/contributing.html
> > > >*
> > > >> :
> > > >> [image: Inline image 2]
> > > >>
> > > >> The search feature could filter the docs shown in the side bar in
> > > >> real-time:
> > > >> [image: Inline image 3]
> > > >>
> > > >> *## Caveats*
> > > >> No plan is perfect, and this one is no exception. A few caveats to
> > this
> > > >> plan are:
> > > >>
> > > >> - Any updates to the docs in the repos require a manual build and
> push
> > > to
> > > >> the website.
> > > >> - This would be a custom solution, and while I will rely on existing
> > > >> modules as much as possible, there will be a little glue-code
> > > >> - The JSON files used by the search feature may become very large if
> > the
> > > >> docs become extremely large. This may hurt browser performance. I
> > doubt
> > > >> this will ever be a problem though.
> > > >>
> > > >> *## Alternatives*
> > > >> We could approach the problem of docs in a few different ways. Most
> > > >> notably are solutions hosted by others (i.e., Documentation as a
> > > Service).
> > > >> Some examples of this solution: Atlassian Confluence
> > > >> <https://developer.atlassian.com/opensource/>, Readme.io, and
> > > Readthedocs
> > > >> <https://readthedocs.org/>. I personally take issue with these for
> > the
> > > >> following reasons:
> > > >>
> > > >> - docs not versioned along with the code (readthedocs is the
> exception
> > > >> here)
> > > >> - most solutions are not very user-friendly (no real-time search)
> > > >> - css-skinning is difficult or impossible
> > > >> - system exists outside of current projects/website
> > > >>
> > > >> If the community rejects the proposal put forth here, my choice of
> > > >> alternative would be ReadTheDocs (see the features
> > > >> <http://read-the-docs.readthedocs.org/en/latest/features.html>).
> > > >>
> > > >> *Pros of ReadTheDocs:*
> > > >> - docs are also versioned with codebase in git repo
> > > >> - git hook automatically updates docs without any manual step
> > > >> - out-of-the-box support for showing different versions of the docs
> > > >> - decent default theme
> > > >>
> > > >> *Cons of ReadTheDocs are: *
> > > >> - separately hosted
> > > >> - learning curve for usage
> > > >> - difficulty for skinning
> > > >> - difficulty extending/adding functionality
> > > >> - Slow search/frustrating left-hand navigation
> > > >>
> > > >>
> > > >> *## Conclusions*
> > > >> Documentation is one of the first places potential new community
> > > >> members/developers will go when vetting Apex for use. Ideally, it
> > should
> > > >> dazzle and inspire. I think the status-quo for documentation on
> > > open-source
> > > >> projects is a low bar, and I am willing to put in the bit of work to
> > lay
> > > >> the foundation for excellent docs. Again, if my custom proposal
> seems
> > > too
> > > >> problematic, my vote is for using ReadTheDocs.
> > > >>
> > > >> Please share your thoughts.
> > > >>
> > > >> -Andy
> > > >>
> > > >>
> > > >> --
> > > >> Regards,
> > > >> Andy Perlitch
> > > >> Software Engineer
> > > >> DataTorrent Inc
> > > >> (408)829-9319
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > Regards,
> > > > Andy Perlitch
> > > > Software Engineer
> > > > DataTorrent Inc
> > > > (408)829-9319
> > > >
> > >
> >
> >
> >
> > --
> > Regards,
> > Andy Perlitch
> > Software Engineer
> > DataTorrent Inc
> > (408)829-9319
> >
>
Re: Proposal for Apex and Malhar Docs
Posted by Sasha Parfenov <sa...@datatorrent.com>.
+1 for Andy's custom proposal, with ReadTheDocs or another pre-built
processor as the backup.
Apex documentation is primarily written by engineers, and needs to closely
reflect the current state of the code. Many Apache projects follow the
model proposed here, with variations on the markup language and processor
to HTML used. I think this model will results in a good process for
contributing, updating, and ensuring consistent look and feel for the docs.
Thanks,
Sasha
On Wed, Nov 18, 2015 at 1:43 PM, Andy Perlitch <an...@datatorrent.com> wrote:
> Here are some more pros and cons on my proposed solution and
> ReadTheDocs.org:
>
> *# Custom Proposal*
> Pros
> - Unified doc set search in left nav bar
>
> Cons
> - Harder to maintain as it is custom code (node and angular code to boot,
> which is not a skill many community members will have)
>
> *# ReadTheDocs*
> Pros
> - Potentially easier to maintain by community
>
> Cons
> - Core and Malhar would have to be two separate readthedocs.org endpoints,
> which means you can't do a single search that queries both doc sets
>
> -Andy
>
> On Wed, Nov 18, 2015 at 1:21 PM, Timothy Farkas <ti...@datatorrent.com>
> wrote:
>
> > +1 Very nice :)
> >
> > On Wed, Nov 18, 2015 at 1:15 PM, Andy Perlitch <an...@datatorrent.com>
> > wrote:
> >
> > > Sorry, seems like the images didn't get through.
> > >
> > > - index page for the docs: http://i.imgur.com/ZB8bzhO.png
> > > - individual doc example: http://i.imgur.com/BcGaIEQ.png
> > > - realtime searching (taken from angular docs as example)
> > > http://i.imgur.com/K6247Q9.gif
> > >
> > >
> > > Andy
> > >
> > > On Mon, Nov 16, 2015 at 7:46 PM, Andy Perlitch <an...@datatorrent.com>
> > > wrote:
> > >
> > >> Hello Apexers,
> > >>
> > >> I'd like to propose a process for creating, storing, updating, and
> > >> displaying documentation of the apex and malhar projects.
> > >>
> > >>
> > >> *## Goals*
> > >> The goals for docs that I had in mind are:
> > >>
> > >> - Versioned with the code base
> > >> - Easy to contribute to by anyone
> > >> - Displayed on apex.incubator.apache.org
> > >> - Simple deploy process
> > >> - Indexable by search engines
> > >> - Easily navigable; table of contents, support for nested "groups" of
> > >> docs, responsive search
> > >>
> > >> *## Proposal*
> > >> My proposal is as follows:
> > >>
> > >> 1. Source files for docs will live in their respective git
> repositories,
> > >> under a `docs` folder. Format for said files will be github-flavored
> > >> markdown <https://help.github.com/articles/github-flavored-markdown/>
> > (In
> > >> the future, other formats could be supported). This takes care of the
> > first
> > >> two goals listed above.
> > >> 2. In addition to the docs themselves, a single json file acting as an
> > >> index for the rendered docs will exist in `docs/index.json`. This will
> > be
> > >> used to render a left navigation bar in the docs section.
> > >> 3. Create a gulp <http://gulpjs.com/> task which:
> > >> a. pulls down these files for each released version of apex and
> malhar
> > >> (using the Github API)
> > >> b. creates HTML versions of each project/version/doc, each with the
> > >> same left-side navigation
> > >> c. creates a JSON file (per version) containing the raw markdown
> > >> content that can be used by Javascript on the page to power a fast
> > search
> > >> function in the left-side navigation.
> > >> 4. As part of the release process, run the aforementioned gulp task
> and
> > >> push to the site using the current `build.sh` script
> > >>
> > >>
> > >> *## Mockups*
> > >> This would result in an index page on *
> > apex.incubator.apache.org/docs/index.html
> > >> <http://apex.incubator.apache.org/docs/index.html>*:
> > >> [image: Inline image 1]
> > >>
> > >> And this is a mockup of looking at an individual doc, e.g. *
> > apex.incubator.apache.org/docs/v3.2.0-incubating/contributing.html
> > >> <
> >
> http://apex.incubator.apache.org/docs/v3.2.0-incubating/contributing.html
> > >*
> > >> :
> > >> [image: Inline image 2]
> > >>
> > >> The search feature could filter the docs shown in the side bar in
> > >> real-time:
> > >> [image: Inline image 3]
> > >>
> > >> *## Caveats*
> > >> No plan is perfect, and this one is no exception. A few caveats to
> this
> > >> plan are:
> > >>
> > >> - Any updates to the docs in the repos require a manual build and push
> > to
> > >> the website.
> > >> - This would be a custom solution, and while I will rely on existing
> > >> modules as much as possible, there will be a little glue-code
> > >> - The JSON files used by the search feature may become very large if
> the
> > >> docs become extremely large. This may hurt browser performance. I
> doubt
> > >> this will ever be a problem though.
> > >>
> > >> *## Alternatives*
> > >> We could approach the problem of docs in a few different ways. Most
> > >> notably are solutions hosted by others (i.e., Documentation as a
> > Service).
> > >> Some examples of this solution: Atlassian Confluence
> > >> <https://developer.atlassian.com/opensource/>, Readme.io, and
> > Readthedocs
> > >> <https://readthedocs.org/>. I personally take issue with these for
> the
> > >> following reasons:
> > >>
> > >> - docs not versioned along with the code (readthedocs is the exception
> > >> here)
> > >> - most solutions are not very user-friendly (no real-time search)
> > >> - css-skinning is difficult or impossible
> > >> - system exists outside of current projects/website
> > >>
> > >> If the community rejects the proposal put forth here, my choice of
> > >> alternative would be ReadTheDocs (see the features
> > >> <http://read-the-docs.readthedocs.org/en/latest/features.html>).
> > >>
> > >> *Pros of ReadTheDocs:*
> > >> - docs are also versioned with codebase in git repo
> > >> - git hook automatically updates docs without any manual step
> > >> - out-of-the-box support for showing different versions of the docs
> > >> - decent default theme
> > >>
> > >> *Cons of ReadTheDocs are: *
> > >> - separately hosted
> > >> - learning curve for usage
> > >> - difficulty for skinning
> > >> - difficulty extending/adding functionality
> > >> - Slow search/frustrating left-hand navigation
> > >>
> > >>
> > >> *## Conclusions*
> > >> Documentation is one of the first places potential new community
> > >> members/developers will go when vetting Apex for use. Ideally, it
> should
> > >> dazzle and inspire. I think the status-quo for documentation on
> > open-source
> > >> projects is a low bar, and I am willing to put in the bit of work to
> lay
> > >> the foundation for excellent docs. Again, if my custom proposal seems
> > too
> > >> problematic, my vote is for using ReadTheDocs.
> > >>
> > >> Please share your thoughts.
> > >>
> > >> -Andy
> > >>
> > >>
> > >> --
> > >> Regards,
> > >> Andy Perlitch
> > >> Software Engineer
> > >> DataTorrent Inc
> > >> (408)829-9319
> > >>
> > >
> > >
> > >
> > > --
> > > Regards,
> > > Andy Perlitch
> > > Software Engineer
> > > DataTorrent Inc
> > > (408)829-9319
> > >
> >
>
>
>
> --
> Regards,
> Andy Perlitch
> Software Engineer
> DataTorrent Inc
> (408)829-9319
>
Re: Proposal for Apex and Malhar Docs
Posted by Andy Perlitch <an...@datatorrent.com>.
Here are some more pros and cons on my proposed solution and
ReadTheDocs.org:
*# Custom Proposal*
Pros
- Unified doc set search in left nav bar
Cons
- Harder to maintain as it is custom code (node and angular code to boot,
which is not a skill many community members will have)
*# ReadTheDocs*
Pros
- Potentially easier to maintain by community
Cons
- Core and Malhar would have to be two separate readthedocs.org endpoints,
which means you can't do a single search that queries both doc sets
-Andy
On Wed, Nov 18, 2015 at 1:21 PM, Timothy Farkas <ti...@datatorrent.com> wrote:
> +1 Very nice :)
>
> On Wed, Nov 18, 2015 at 1:15 PM, Andy Perlitch <an...@datatorrent.com>
> wrote:
>
> > Sorry, seems like the images didn't get through.
> >
> > - index page for the docs: http://i.imgur.com/ZB8bzhO.png
> > - individual doc example: http://i.imgur.com/BcGaIEQ.png
> > - realtime searching (taken from angular docs as example)
> > http://i.imgur.com/K6247Q9.gif
> >
> >
> > Andy
> >
> > On Mon, Nov 16, 2015 at 7:46 PM, Andy Perlitch <an...@datatorrent.com>
> > wrote:
> >
> >> Hello Apexers,
> >>
> >> I'd like to propose a process for creating, storing, updating, and
> >> displaying documentation of the apex and malhar projects.
> >>
> >>
> >> *## Goals*
> >> The goals for docs that I had in mind are:
> >>
> >> - Versioned with the code base
> >> - Easy to contribute to by anyone
> >> - Displayed on apex.incubator.apache.org
> >> - Simple deploy process
> >> - Indexable by search engines
> >> - Easily navigable; table of contents, support for nested "groups" of
> >> docs, responsive search
> >>
> >> *## Proposal*
> >> My proposal is as follows:
> >>
> >> 1. Source files for docs will live in their respective git repositories,
> >> under a `docs` folder. Format for said files will be github-flavored
> >> markdown <https://help.github.com/articles/github-flavored-markdown/>
> (In
> >> the future, other formats could be supported). This takes care of the
> first
> >> two goals listed above.
> >> 2. In addition to the docs themselves, a single json file acting as an
> >> index for the rendered docs will exist in `docs/index.json`. This will
> be
> >> used to render a left navigation bar in the docs section.
> >> 3. Create a gulp <http://gulpjs.com/> task which:
> >> a. pulls down these files for each released version of apex and malhar
> >> (using the Github API)
> >> b. creates HTML versions of each project/version/doc, each with the
> >> same left-side navigation
> >> c. creates a JSON file (per version) containing the raw markdown
> >> content that can be used by Javascript on the page to power a fast
> search
> >> function in the left-side navigation.
> >> 4. As part of the release process, run the aforementioned gulp task and
> >> push to the site using the current `build.sh` script
> >>
> >>
> >> *## Mockups*
> >> This would result in an index page on *
> apex.incubator.apache.org/docs/index.html
> >> <http://apex.incubator.apache.org/docs/index.html>*:
> >> [image: Inline image 1]
> >>
> >> And this is a mockup of looking at an individual doc, e.g. *
> apex.incubator.apache.org/docs/v3.2.0-incubating/contributing.html
> >> <
> http://apex.incubator.apache.org/docs/v3.2.0-incubating/contributing.html
> >*
> >> :
> >> [image: Inline image 2]
> >>
> >> The search feature could filter the docs shown in the side bar in
> >> real-time:
> >> [image: Inline image 3]
> >>
> >> *## Caveats*
> >> No plan is perfect, and this one is no exception. A few caveats to this
> >> plan are:
> >>
> >> - Any updates to the docs in the repos require a manual build and push
> to
> >> the website.
> >> - This would be a custom solution, and while I will rely on existing
> >> modules as much as possible, there will be a little glue-code
> >> - The JSON files used by the search feature may become very large if the
> >> docs become extremely large. This may hurt browser performance. I doubt
> >> this will ever be a problem though.
> >>
> >> *## Alternatives*
> >> We could approach the problem of docs in a few different ways. Most
> >> notably are solutions hosted by others (i.e., Documentation as a
> Service).
> >> Some examples of this solution: Atlassian Confluence
> >> <https://developer.atlassian.com/opensource/>, Readme.io, and
> Readthedocs
> >> <https://readthedocs.org/>. I personally take issue with these for the
> >> following reasons:
> >>
> >> - docs not versioned along with the code (readthedocs is the exception
> >> here)
> >> - most solutions are not very user-friendly (no real-time search)
> >> - css-skinning is difficult or impossible
> >> - system exists outside of current projects/website
> >>
> >> If the community rejects the proposal put forth here, my choice of
> >> alternative would be ReadTheDocs (see the features
> >> <http://read-the-docs.readthedocs.org/en/latest/features.html>).
> >>
> >> *Pros of ReadTheDocs:*
> >> - docs are also versioned with codebase in git repo
> >> - git hook automatically updates docs without any manual step
> >> - out-of-the-box support for showing different versions of the docs
> >> - decent default theme
> >>
> >> *Cons of ReadTheDocs are: *
> >> - separately hosted
> >> - learning curve for usage
> >> - difficulty for skinning
> >> - difficulty extending/adding functionality
> >> - Slow search/frustrating left-hand navigation
> >>
> >>
> >> *## Conclusions*
> >> Documentation is one of the first places potential new community
> >> members/developers will go when vetting Apex for use. Ideally, it should
> >> dazzle and inspire. I think the status-quo for documentation on
> open-source
> >> projects is a low bar, and I am willing to put in the bit of work to lay
> >> the foundation for excellent docs. Again, if my custom proposal seems
> too
> >> problematic, my vote is for using ReadTheDocs.
> >>
> >> Please share your thoughts.
> >>
> >> -Andy
> >>
> >>
> >> --
> >> Regards,
> >> Andy Perlitch
> >> Software Engineer
> >> DataTorrent Inc
> >> (408)829-9319
> >>
> >
> >
> >
> > --
> > Regards,
> > Andy Perlitch
> > Software Engineer
> > DataTorrent Inc
> > (408)829-9319
> >
>
--
Regards,
Andy Perlitch
Software Engineer
DataTorrent Inc
(408)829-9319
Re: Proposal for Apex and Malhar Docs
Posted by Timothy Farkas <ti...@datatorrent.com>.
+1 Very nice :)
On Wed, Nov 18, 2015 at 1:15 PM, Andy Perlitch <an...@datatorrent.com> wrote:
> Sorry, seems like the images didn't get through.
>
> - index page for the docs: http://i.imgur.com/ZB8bzhO.png
> - individual doc example: http://i.imgur.com/BcGaIEQ.png
> - realtime searching (taken from angular docs as example)
> http://i.imgur.com/K6247Q9.gif
>
>
> Andy
>
> On Mon, Nov 16, 2015 at 7:46 PM, Andy Perlitch <an...@datatorrent.com>
> wrote:
>
>> Hello Apexers,
>>
>> I'd like to propose a process for creating, storing, updating, and
>> displaying documentation of the apex and malhar projects.
>>
>>
>> *## Goals*
>> The goals for docs that I had in mind are:
>>
>> - Versioned with the code base
>> - Easy to contribute to by anyone
>> - Displayed on apex.incubator.apache.org
>> - Simple deploy process
>> - Indexable by search engines
>> - Easily navigable; table of contents, support for nested "groups" of
>> docs, responsive search
>>
>> *## Proposal*
>> My proposal is as follows:
>>
>> 1. Source files for docs will live in their respective git repositories,
>> under a `docs` folder. Format for said files will be github-flavored
>> markdown <https://help.github.com/articles/github-flavored-markdown/> (In
>> the future, other formats could be supported). This takes care of the first
>> two goals listed above.
>> 2. In addition to the docs themselves, a single json file acting as an
>> index for the rendered docs will exist in `docs/index.json`. This will be
>> used to render a left navigation bar in the docs section.
>> 3. Create a gulp <http://gulpjs.com/> task which:
>> a. pulls down these files for each released version of apex and malhar
>> (using the Github API)
>> b. creates HTML versions of each project/version/doc, each with the
>> same left-side navigation
>> c. creates a JSON file (per version) containing the raw markdown
>> content that can be used by Javascript on the page to power a fast search
>> function in the left-side navigation.
>> 4. As part of the release process, run the aforementioned gulp task and
>> push to the site using the current `build.sh` script
>>
>>
>> *## Mockups*
>> This would result in an index page on *apex.incubator.apache.org/docs/index.html
>> <http://apex.incubator.apache.org/docs/index.html>*:
>> [image: Inline image 1]
>>
>> And this is a mockup of looking at an individual doc, e.g. *apex.incubator.apache.org/docs/v3.2.0-incubating/contributing.html
>> <http://apex.incubator.apache.org/docs/v3.2.0-incubating/contributing.html>*
>> :
>> [image: Inline image 2]
>>
>> The search feature could filter the docs shown in the side bar in
>> real-time:
>> [image: Inline image 3]
>>
>> *## Caveats*
>> No plan is perfect, and this one is no exception. A few caveats to this
>> plan are:
>>
>> - Any updates to the docs in the repos require a manual build and push to
>> the website.
>> - This would be a custom solution, and while I will rely on existing
>> modules as much as possible, there will be a little glue-code
>> - The JSON files used by the search feature may become very large if the
>> docs become extremely large. This may hurt browser performance. I doubt
>> this will ever be a problem though.
>>
>> *## Alternatives*
>> We could approach the problem of docs in a few different ways. Most
>> notably are solutions hosted by others (i.e., Documentation as a Service).
>> Some examples of this solution: Atlassian Confluence
>> <https://developer.atlassian.com/opensource/>, Readme.io, and Readthedocs
>> <https://readthedocs.org/>. I personally take issue with these for the
>> following reasons:
>>
>> - docs not versioned along with the code (readthedocs is the exception
>> here)
>> - most solutions are not very user-friendly (no real-time search)
>> - css-skinning is difficult or impossible
>> - system exists outside of current projects/website
>>
>> If the community rejects the proposal put forth here, my choice of
>> alternative would be ReadTheDocs (see the features
>> <http://read-the-docs.readthedocs.org/en/latest/features.html>).
>>
>> *Pros of ReadTheDocs:*
>> - docs are also versioned with codebase in git repo
>> - git hook automatically updates docs without any manual step
>> - out-of-the-box support for showing different versions of the docs
>> - decent default theme
>>
>> *Cons of ReadTheDocs are: *
>> - separately hosted
>> - learning curve for usage
>> - difficulty for skinning
>> - difficulty extending/adding functionality
>> - Slow search/frustrating left-hand navigation
>>
>>
>> *## Conclusions*
>> Documentation is one of the first places potential new community
>> members/developers will go when vetting Apex for use. Ideally, it should
>> dazzle and inspire. I think the status-quo for documentation on open-source
>> projects is a low bar, and I am willing to put in the bit of work to lay
>> the foundation for excellent docs. Again, if my custom proposal seems too
>> problematic, my vote is for using ReadTheDocs.
>>
>> Please share your thoughts.
>>
>> -Andy
>>
>>
>> --
>> Regards,
>> Andy Perlitch
>> Software Engineer
>> DataTorrent Inc
>> (408)829-9319
>>
>
>
>
> --
> Regards,
> Andy Perlitch
> Software Engineer
> DataTorrent Inc
> (408)829-9319
>
Re: Proposal for Apex and Malhar Docs
Posted by Andy Perlitch <an...@datatorrent.com>.
Sorry, seems like the images didn't get through.
- index page for the docs: http://i.imgur.com/ZB8bzhO.png
- individual doc example: http://i.imgur.com/BcGaIEQ.png
- realtime searching (taken from angular docs as example)
http://i.imgur.com/K6247Q9.gif
Andy
On Mon, Nov 16, 2015 at 7:46 PM, Andy Perlitch <an...@datatorrent.com> wrote:
> Hello Apexers,
>
> I'd like to propose a process for creating, storing, updating, and
> displaying documentation of the apex and malhar projects.
>
>
> *## Goals*
> The goals for docs that I had in mind are:
>
> - Versioned with the code base
> - Easy to contribute to by anyone
> - Displayed on apex.incubator.apache.org
> - Simple deploy process
> - Indexable by search engines
> - Easily navigable; table of contents, support for nested "groups" of
> docs, responsive search
>
> *## Proposal*
> My proposal is as follows:
>
> 1. Source files for docs will live in their respective git repositories,
> under a `docs` folder. Format for said files will be github-flavored
> markdown <https://help.github.com/articles/github-flavored-markdown/> (In
> the future, other formats could be supported). This takes care of the first
> two goals listed above.
> 2. In addition to the docs themselves, a single json file acting as an
> index for the rendered docs will exist in `docs/index.json`. This will be
> used to render a left navigation bar in the docs section.
> 3. Create a gulp <http://gulpjs.com/> task which:
> a. pulls down these files for each released version of apex and malhar
> (using the Github API)
> b. creates HTML versions of each project/version/doc, each with the same
> left-side navigation
> c. creates a JSON file (per version) containing the raw markdown content
> that can be used by Javascript on the page to power a fast search function
> in the left-side navigation.
> 4. As part of the release process, run the aforementioned gulp task and
> push to the site using the current `build.sh` script
>
>
> *## Mockups*
> This would result in an index page on *apex.incubator.apache.org/docs/index.html
> <http://apex.incubator.apache.org/docs/index.html>*:
> [image: Inline image 1]
>
> And this is a mockup of looking at an individual doc, e.g. *apex.incubator.apache.org/docs/v3.2.0-incubating/contributing.html
> <http://apex.incubator.apache.org/docs/v3.2.0-incubating/contributing.html>*
> :
> [image: Inline image 2]
>
> The search feature could filter the docs shown in the side bar in
> real-time:
> [image: Inline image 3]
>
> *## Caveats*
> No plan is perfect, and this one is no exception. A few caveats to this
> plan are:
>
> - Any updates to the docs in the repos require a manual build and push to
> the website.
> - This would be a custom solution, and while I will rely on existing
> modules as much as possible, there will be a little glue-code
> - The JSON files used by the search feature may become very large if the
> docs become extremely large. This may hurt browser performance. I doubt
> this will ever be a problem though.
>
> *## Alternatives*
> We could approach the problem of docs in a few different ways. Most
> notably are solutions hosted by others (i.e., Documentation as a Service).
> Some examples of this solution: Atlassian Confluence
> <https://developer.atlassian.com/opensource/>, Readme.io, and Readthedocs
> <https://readthedocs.org/>. I personally take issue with these for the
> following reasons:
>
> - docs not versioned along with the code (readthedocs is the exception
> here)
> - most solutions are not very user-friendly (no real-time search)
> - css-skinning is difficult or impossible
> - system exists outside of current projects/website
>
> If the community rejects the proposal put forth here, my choice of
> alternative would be ReadTheDocs (see the features
> <http://read-the-docs.readthedocs.org/en/latest/features.html>).
>
> *Pros of ReadTheDocs:*
> - docs are also versioned with codebase in git repo
> - git hook automatically updates docs without any manual step
> - out-of-the-box support for showing different versions of the docs
> - decent default theme
>
> *Cons of ReadTheDocs are: *
> - separately hosted
> - learning curve for usage
> - difficulty for skinning
> - difficulty extending/adding functionality
> - Slow search/frustrating left-hand navigation
>
>
> *## Conclusions*
> Documentation is one of the first places potential new community
> members/developers will go when vetting Apex for use. Ideally, it should
> dazzle and inspire. I think the status-quo for documentation on open-source
> projects is a low bar, and I am willing to put in the bit of work to lay
> the foundation for excellent docs. Again, if my custom proposal seems too
> problematic, my vote is for using ReadTheDocs.
>
> Please share your thoughts.
>
> -Andy
>
>
> --
> Regards,
> Andy Perlitch
> Software Engineer
> DataTorrent Inc
> (408)829-9319
>
--
Regards,
Andy Perlitch
Software Engineer
DataTorrent Inc
(408)829-9319