[DISCUSS] Splitting the sphinx-based Arrow docs into separate sphinx projects

[Joris Van den Bossche <jo...@gmail.com>]

Hi all,

We use the Sphinx documentation tool for the main Arrow documentation (the
format specs, the developer docs) and several of the languages (C++,
Python, part of the Java docs). This lives in the /docs subdirectory of the
Arrow repo.

Currently, this is a single large sphinx project. The advantage is that
this makes it easy to build the full set of docs, but it also has several
disadvantages. For one, it makes it harder to contribute to the docs. For
example, if you want to contribute a fix to the developer docs, and you
want to verify the change locally, you need to build the full docs which 1)
requires to have the dev version of C++ Arrow and PyArrow installed, and 2)
takes quite some time the first time.

My proposal is to split the single sphinx project into multiple projects,
that (for development) can easily be built independently.
I listed some reasons to do this and how to approach this technically in
the following google doc:
https://docs.google.com/document/d/1AXDNwU5CSnZ1cSeUISwy_xgvTzoYWeuqWApC8UEv97Q/edit?usp=sharing

What would people think of such a change? Feedback very welcome!

Joris

Re: [DISCUSS] Splitting the sphinx-based Arrow docs into separate sphinx projects

[Joris Van den Bossche <jo...@gmail.com>]

I have a draft PR experimenting with this at
https://github.com/apache/arrow/pull/11980

On Wed, 8 Dec 2021 at 17:17, Antoine Pitrou <an...@python.org> wrote:

>
> Hi Joris,
>
> I have a question: does Intersphinx handle glossary links as well?
>
> Good question. I didn't try explicitly (will do), but I did update other
references with intersphinx, and my understanding is that prefixing a
reference with the intersphinx domain works for any kind of
cross-reference. For references to sections this already worked.
(and
https://stackoverflow.com/questions/46473481/sphinx-how-to-link-to-terms-in-a-glossary-of-a-different-project
also confirms this should work for glossary items as well)


> (we don't have a glossary, but we may want to add one)
>
> Regards
>
> Antoine.
>
>
> Le 08/12/2021 à 14:20, Joris Van den Bossche a écrit :
> > Hi all,
> >
> > We use the Sphinx documentation tool for the main Arrow documentation
> (the
> > format specs, the developer docs) and several of the languages (C++,
> > Python, part of the Java docs). This lives in the /docs subdirectory of
> the
> > Arrow repo.
> >
> > Currently, this is a single large sphinx project. The advantage is that
> > this makes it easy to build the full set of docs, but it also has several
> > disadvantages. For one, it makes it harder to contribute to the docs. For
> > example, if you want to contribute a fix to the developer docs, and you
> > want to verify the change locally, you need to build the full docs which
> 1)
> > requires to have the dev version of C++ Arrow and PyArrow installed, and
> 2)
> > takes quite some time the first time.
> >
> > My proposal is to split the single sphinx project into multiple projects,
> > that (for development) can easily be built independently.
> > I listed some reasons to do this and how to approach this technically in
> > the following google doc:
> >
> https://docs.google.com/document/d/1AXDNwU5CSnZ1cSeUISwy_xgvTzoYWeuqWApC8UEv97Q/edit?usp=sharing
> >
> > What would people think of such a change? Feedback very welcome!
> >
> > Joris
> >
>

Re: [DISCUSS] Splitting the sphinx-based Arrow docs into separate sphinx projects

[Antoine Pitrou <an...@python.org>]

Hi Joris,

I have a question: does Intersphinx handle glossary links as well?

(we don't have a glossary, but we may want to add one)

Regards

Antoine.


Le 08/12/2021 à 14:20, Joris Van den Bossche a écrit :
> Hi all,
> 
> We use the Sphinx documentation tool for the main Arrow documentation (the
> format specs, the developer docs) and several of the languages (C++,
> Python, part of the Java docs). This lives in the /docs subdirectory of the
> Arrow repo.
> 
> Currently, this is a single large sphinx project. The advantage is that
> this makes it easy to build the full set of docs, but it also has several
> disadvantages. For one, it makes it harder to contribute to the docs. For
> example, if you want to contribute a fix to the developer docs, and you
> want to verify the change locally, you need to build the full docs which 1)
> requires to have the dev version of C++ Arrow and PyArrow installed, and 2)
> takes quite some time the first time.
> 
> My proposal is to split the single sphinx project into multiple projects,
> that (for development) can easily be built independently.
> I listed some reasons to do this and how to approach this technically in
> the following google doc:
> https://docs.google.com/document/d/1AXDNwU5CSnZ1cSeUISwy_xgvTzoYWeuqWApC8UEv97Q/edit?usp=sharing
> 
> What would people think of such a change? Feedback very welcome!
> 
> Joris
>