Re: new website

[Aaron Markham <aa...@gmail.com>]

Thanks for the feedback.
We're starting to see merge conflicts as our base is wandering off
from master. The PRs are pretty massive, and there's a lot of moving
parts. I'd really appreciate reviews, so we can move forward.
The first PR only ADDs content and gives us the new site. Having the
new content in there would allow for people to add suggestions and
even cut PRs to fix cosmetic (like the dropdown) or content issues.

The second PR deletes things, but the 3rd PR sweeps up the publishing
and CI processes that are removed in the 2nd PR. It pretty difficult
to test all of these things, as I have made improvements in CI that I
can't easily use to test the new site material in the 1st PR.

An alternate strategy is to add the new site, add the new CI
functionality, then cleanup.

But... this has the drawback on not making a clean break and
potentially leaving a bunch of mess in repo that never gets deleted. I
feel that a clean repo is better. And we can always go back and
restore stuff that we actually needed.

On Fri, Aug 30, 2019 at 7:23 PM Chaitanya Bapat <ch...@gmail.com> wrote:
>
> First things first,
>
> Big shout out to you (Aaron) and the team for laying a strong foundation
> for the new website! We all knew that our original website needed
> improvements and it's criticality for user adoption and growth. But doing
> it well and in a timely manner. Great job, keep it up.
>
> Those 3 PRs are pretty massive. But would still try to review it by
> this weekend. PR descriptions were helpful in knowing what's happening
> amidst a lot of chaos.
>
> Nitpick: Aside from the known issues highlighted in the first PR [1/3], I
> found the MXNet version selection via the dropdown to be odd (Main page ->
> Getting started) Is there a cleaner way of doing this?
>
> Thanks once again.
>
>
> On Thu, 29 Aug 2019 at 18:06, Yuan Tang <te...@gmail.com> wrote:
>
> > I think for now PDF would still be used by a good amount of users since R
> > users are used to read PDF manual for packages that don't have websites.
> >
> > Nowadays Github pages + pkgdown combination is getting more and more
> > popular so we would see a trend soon towards web hosted docs for R
> > packages.
> >
> > On Thu, Aug 29, 2019 at 8:58 PM Aaron Markham <aa...@gmail.com>
> > wrote:
> >
> > > pkgdown makes some nice looking R microsites. Good idea. Do you know
> > > if many R users would still want the pdf or have things moved to use
> > > websites for reference like this?
> > > One of the nice things about the new pipelines for docs is that
> > > they're not wrapped by Sphinx, so our R contributors will have an
> > > easier time testing and adding this kind of feature.
> > >
> > > On Thu, Aug 29, 2019 at 5:34 PM Yuan Tang <te...@gmail.com>
> > wrote:
> > > >
> > > > Thanks for the update, Aaron.
> > > >
> > > > Regarding the R docs, one suggestion I have is to use pkgdown package (
> > > > https://pkgdown.r-lib.org/index.html) to automatically generated the
> > > > documentation pages (tutorials, API reference, etc.). I've seen huge
> > > > adoption of this package being used for documentations in the R
> > > community.
> > > >
> > > >
> > > > On Thu, Aug 29, 2019 at 8:26 PM Aaron Markham <
> > aaron.s.markham@gmail.com
> > > >
> > > > wrote:
> > > >
> > > > > Hi everyone,
> > > > >
> > > > > I'm very excited to share a preview and the pull requests for a new
> > > > > website and new documentation pipelines.
> > > > >
> > > > > The following link is using Apache's new staging site setup. It is
> > > > > built from the new docs publishing pipelines in CI where a Jekyll
> > > > > website is built, and documentation artifacts from Clojure, CPP,
> > Java,
> > > > > Julia, Python, R, and Scala are combined into one website.
> > > > >
> > > > > https://mxnet-beta.staged.apache.org
> > > > >
> > > > > It is the culmination of a lot of effort of several MXNet
> > contributors.
> > > > >
> > > > > * A huge shout out goes to Thomas Delteil for the work on the new
> > > > > Jekyll-backend and beautiful-looking website, and for helping me out
> > > > > whenever I'd get stuck on revamping the 7 different API docs systems
> > > > > in CI.
> > > > > * Soji Adeshina and Vishaal Kapoor both helping me with the system
> > > > > design for the new docs pipelines.
> > > > > * Per Goncalves da Silva and Marco de Abreu both helped me with
> > > > > figuring out CI issues.
> > > > > * We also ported over Mu Li's beta site for the Python & R APIs which
> > > > > had many contributors there. Thanks goes to Mu, Ivy Bazan, Jonas
> > > > > Mueller, Aston Zhang, and Zhi Zhang for their help & contributions. I
> > > > > apologize in advance if I missed anyone.
> > > > >
> > > > > Highlights:
> > > > >
> > > > > * R docs are now generated as part of CI. There were issues with R
> > > > > docs coming from beta repo. They were not reproducible. So I began
> > the
> > > > > process of creating the pdf doc that is expected by R users as an
> > > > > alternative. Thomas fixed a CPP bug that was blocking 90% of the docs
> > > > > from appearing. The R docs are 10x in length compared to the pdf
> > we're
> > > > > hosting now!
> > > > >
> > > > > * Each other API is built in a micro-site fashion. You will notice
> > > > > that the reference API links will open up the site that is generated
> > > > > by that language's docs tools. We tried to keep the navigation common
> > > > > and do this for the Python API. This is something that can be
> > expanded
> > > > > on for the other APIs in later updates to the website.
> > > > >
> > > > > * Each doc set can be generated separately with functions that will
> > > > > run in Docker and generate the docs artifacts. This means you can now
> > > > > focus on your preferred API and not have to deal with anything else.
> > > > >
> > > > > * Website changes are now much easier. You can serve Jekyll locally,
> > > > > and have it do incremental updates, so you can see your changes live
> > > > > without having to build MXNet or anything else. It's a pure front-end
> > > > > setup.
> > > > >
> > > > > * For website publishing, the MXNet binary is built once and then
> > > > > shared with the other docs generation pipelines.
> > > > >
> > > > > * For individual docs runs, you can run a "lite" binary build, then
> > > > > follow it up with the docs run you want.
> > > > >
> > > > > ---
> > > > >
> > > > > For example to build MXNet:
> > > > >
> > > > > ci/build.py --docker-registry mxnetcidev --platform ubuntu_cpu_lite
> > > > > /work/runtime_functions.sh build_ubuntu_cpu_docs
> > > > >
> > > > > Then to build the R docs:
> > > > >
> > > > > ci/build.py --docker-registry mxnetcidev --platform ubuntu_cpu_r
> > > > > /work/runtime_functions.sh build_r_docs
> > > > >
> > > > > There is now a Docker image and a runtime_function for each API
> > > > > (except Perl which is built offsite). Python is like this:
> > > > >
> > > > > ci/build.py --docker-registry mxnetcidev --platform ubuntu_cpu_python
> > > > > /work/runtime_functions.sh build_python_docs
> > > > >
> > > > > The pattern for platform is ubuntu_cpu_{api} and runtime_functions.sh
> > > > > is build_{api}_docs.
> > > > >
> > > > > Further information is on the developer wiki:
> > > > >
> > >
> > https://cwiki.apache.org/confluence/display/MXNET/Building+the+New+Website
> > > > > ----
> > > > >
> > > > > Ok, now this is where YOU come in. We need reviewers and testers.
> > > > >
> > > > > There are a lot of changes. My original PR was over 1,000 files with
> > > > > 83k additions and 55k deletions. So, Thomas broke this up into three
> > > > > pull requests that stack.
> > > > >
> > > > > Step 1 New Content
> > > https://github.com/apache/incubator-mxnet/pull/15884
> > > > > Step 2 Remove Old Content
> > > > > https://github.com/apache/incubator-mxnet/pull/15885
> > > > > Step 3 Setup New Jenkins
> > > > > https://github.com/apache/incubator-mxnet/pull/15886
> > > > >
> > > > > For reviewing purposes, start with the new content - what's easily
> > > > > visible on the preview website. This is mostly happening in the first
> > > > > PR:
> > > > > https://github.com/apache/incubator-mxnet/pull/15884
> > > > > You can also look at these helper PRs that show you the differences
> > so
> > > > > it is easier to review what's happening in Steps 2 and 3. You can
> > > > > review these now as well.
> > > > > Step 1->2: https://github.com/ThomasDelteil/incubator-mxnet/pull/5
> > > > > Step 2->3: https://github.com/ThomasDelteil/incubator-mxnet/pull/6
> > > > >
> > > > > I really appreciate everyone's support on this effort.
> > > > >
> > > > > Cheers,
> > > > > Aaron
> > > > >
> > >
> >
>
>
> --
> *Chaitanya Prakash Bapat*
> *+1 (973) 953-6299*
>
> [image: https://www.linkedin.com//in/chaibapat25]
> <https://github.com/ChaiBapchya>[image: https://www.facebook.com/chaibapat]
> <https://www.facebook.com/chaibapchya>[image:
> https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya>[image:
> https://www.linkedin.com//in/chaibapat25]
> <https://www.linkedin.com//in/chaibapchya/>