You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by Michael Smith <mi...@smith-li.com> on 2023/07/20 02:10:05 UTC
Implementing Python API Docs
I did a PR to implement a very old ticket, AVRO-312.
https://github.com/apache/avro/pull/2370
It builds the documentation in html and I can view it locally. What I need
is for someone who understands the process for publishing the avro website
to let me know if my changes will properly publish the Python API docs the
expected way, so they end up on the left navigation, following C# API on
avro.apache.org.
Can someone check that for me?
Thanks,
Michael
Re: Implementing Python API Docs
Posted by "Michael A. Smith" <mi...@smith-li.com>.
> or you mean something else?
I actually meant how to test it integrated with the rest of the website
build. There's no `doc` command for the root ./build.sh. I've been playing
with the part of that file that might become a standalone doc command, but
it is failing on a lot of stuff (probably macOS things) atm.
On Fri, Jul 21, 2023 at 5:56 PM Martin Grigorov <mg...@apache.org>
wrote:
>
> On Fri, Jul 21, 2023 at 10:47 PM Martin Grigorov <mg...@apache.org>
> wrote:
>
> >
> >
> > On Fri, Jul 21, 2023 at 10:18 PM Michael A. Smith <mi...@smith-li.com>
> > wrote:
> >
> >> I did the cherry-pick, but I'm not sure I know how to test if it
> >> works. https://github.com/apache/avro/pull/2380
> >
> >
> > git switch branch-1.11
> > cd lang/py
> > ./build.sh doc
> > open .../index.html
> >
> > or you mean something else ?
> >
>
> I just tried it and :
>
> ./build.sh doc
> /bin/python3: No module named tox
>
> I see that the "dist' target creates a virtual environment automatically.
> Does it make sense to do the same for doc/lint ?!
> Once the venv is created and activated, and tox installed the doc target
> fails with:
>
> The HTML pages are in docs/build/html.
> .pkg: _exit> python
>
/home/martin/git/apache/avro/lang/py/.env/lib/python3.11/site-packages/pyproject_api/_backend.py
> True setuptools.build_meta
> docs: OK (21.52=setup[17.53]+cmd[3.99] seconds)
> congratulations :) (21.59 seconds)
> cp: cannot create directory '../../build/avro-doc-1.12.0-SNAPSHOT/api/py':
> No such file or directory
>
>
> Fix:
>
> diff --git lang/py/build.sh lang/py/build.sh
> index a8f3febdb..5191fb38d 100755
> --- lang/py/build.sh
> +++ lang/py/build.sh
> @@ -55,6 +55,7 @@ doc() {
> local doc_dir
> [[ -s VERSION.txt ]] || cp ../../share/VERSION.txt .
> doc_dir="../../build/avro-doc-$(<VERSION.txt)/api/py"
> + mkdir -p $doc_dir
> python3 -m tox -e docs
> cp -a docs/build/* "$doc_dir"
>
>
>
> >
> >
> >
> >>
> >>
> >> On Fri, Jul 21, 2023 at 4:34 AM Ryan Skraba <ry...@skraba.com> wrote:
> >> >
> >> > Hey thanks for this work! Do you think the python doc generation
> >> > stuff can be cherry-picked back to 1.11? That would be a neat
> >> > addition to the website for the incoming 1.11.3!
> >> >
> >> > Building and deploying the website today is a really tricky
problem...
> >> > I think we're going to have to make some major changes to simplify
> >> > this soon...
> >> >
> >> > I suspect that the solution is going to look something like what
Flink
> >> > does: two separate static websites: one for the community and one
> >> > (well, one-per) for the release, but they should look and feel
> >> > integrated. These python docs should be generated for the
per-release
> >> > pages.
> >> >
> >> > There's a JIRA to investigate this, and I'm definitely on the low end
> >> > of this learning curve, but I think it should be doable!
> >> >
> >> > All my best, Ryan
> >> >
> >> >
> >> > On Fri, Jul 21, 2023 at 9:35 AM Martin Grigorov <mgrigorov@apache.org
>
> >> wrote:
> >> > >
> >> > > On Thu, Jul 20, 2023 at 7:20 PM Michael A. Smith <
> >> michael@smith-li.com>
> >> > > wrote:
> >> > >
> >> > > > OK, I've merged that PR and have made suggested changes to #2187.
> >> > > > Please let me know how I can help.
> >> > > >
> >> > >
> >> > > Thanks !
> >> > > Hopefully this PR will be merged soon!
> >> > >
> >> > >
> >> > >
> >> > > >
> >> > > > On Thu, Jul 20, 2023 at 11:27 AM Martin Grigorov <
> >> mgrigorov@apache.org>
> >> > > > wrote:
> >> > > > >
> >> > > > > On Thu, 20 Jul 2023 at 17:47, Michael A. Smith <
> >> michael@smith-li.com>
> >> > > > wrote:
> >> > > > >
> >> > > > > > Thanks, Martin,
> >> > > > > >
> >> > > > > > It seems like since that PR isn't merged yet, and it would be
> >> > > > > > complicated to add all the Sphinx stuff to it, I should
merge my
> >> > > > > > changes first, and then add the Python stuff to #2187 after
> >> that.
> >> > > > > >
> >> > > > > > Does that seem reasonable?
> >> > > > >
> >> > > > >
> >> > > > > Yep!
> >> > > > >
> >> > > > >
> >> > > > > >
> >> > > > > > On Thu, Jul 20, 2023 at 2:30 AM Martin Grigorov <
> >> mgrigorov@apache.org>
> >> > > > > > wrote:
> >> > > > > > >
> >> > > > > > > Hi Michael,
> >> > > > > > >
> >> > > > > > > The new website uses Hugo to build the static files.
> >> > > > > > > If you prefer to use Sphinx for the Python docs then I
> >> suggest to
> >> > > > follow
> >> > > > > > > the way of C/C++/C#/Java SDKs contribute their part of the
> >> docs in
> >> > > > this
> >> > > > > > PR
> >> > > > > > > -
> >> > > > > > >
> >> > > > > >
> >> > > >
> >>
https://github.com/apache/avro/pull/2187/files#diff-d54d69dbb27e75dae25cb4b2384310cb57707e419377cf572d5cb0ecc1f16877R76-R162
> >> > > > > > > You need to add a new CI job that installs Sphinx, make and
> >> other
> >> > > > > > > dependencies, then call "make" and finally upload the
HTMLs.
> >> Later
> >> > > > in the
> >> > > > > > > push-website job you need to download the HTMLs and copy
them
> >> to
> >> > > > > > > website/docs/++version++/api/py
> >> > > > > > >
> >> > > > > > > Let me know if you have any questions!
> >> > > > > > >
> >> > > > > > > Regards,
> >> > > > > > > Martin
> >> > > > > > >
> >> > > > > > > On Thu, Jul 20, 2023 at 5:10 AM Michael Smith <
> >> michael@smith-li.com>
> >> > > > > > wrote:
> >> > > > > > >
> >> > > > > > > > I did a PR to implement a very old ticket, AVRO-312.
> >> > > > > > > >
> >> > > > > > > > https://github.com/apache/avro/pull/2370
> >> > > > > > > >
> >> > > > > > > > It builds the documentation in html and I can view it
> >> locally.
> >> > > > What I
> >> > > > > > need
> >> > > > > > > > is for someone who understands the process for publishing
> >> the avro
> >> > > > > > website
> >> > > > > > > > to let me know if my changes will properly publish the
> >> Python API
> >> > > > docs
> >> > > > > > the
> >> > > > > > > > expected way, so they end up on the left navigation,
> >> following C#
> >> > > > API
> >> > > > > > on
> >> > > > > > > > avro.apache.org.
> >> > > > > > > >
> >> > > > > > > > Can someone check that for me?
> >> > > > > > > >
> >> > > > > > > > Thanks,
> >> > > > > > > > Michael
> >> > > > > > > >
> >> > > > > >
> >> > > >
> >>
> >
Re: Implementing Python API Docs
Posted by "Michael A. Smith" <mi...@smith-li.com>.
> + mkdir -p $doc_dir
Ah of course, thanks. I'll get that in.
> I see that the "dist' target creates a virtual environment automatically.
So it does. I didn't realize (or had forgotten). A bunch of the other
commands in `build.sh` use tox and don't create a virtualenv. It's tox
that creates most of the virtualenvs (and handles them much better
than we can). So `dist` is the odd one out.
So I'd prefer to just require tox for working with lang/py, which is
why I added it to the documentation in BUILD.md. lang/py still has the
shortest list of dependencies of any of the implementations.
Nevertheless, if there's a push the other way, I can be persuaded.
It'd just be a matter of bootstrapping in tox via a manual virtualenv.
Martin or anyone else on the list, please let me know if you have a
strong opinion about not having to install tox manually.
On Fri, Jul 21, 2023 at 5:56 PM Martin Grigorov <mg...@apache.org> wrote:
>
> On Fri, Jul 21, 2023 at 10:47 PM Martin Grigorov <mg...@apache.org>
> wrote:
>
> >
> >
> > On Fri, Jul 21, 2023 at 10:18 PM Michael A. Smith <mi...@smith-li.com>
> > wrote:
> >
> >> I did the cherry-pick, but I'm not sure I know how to test if it
> >> works. https://github.com/apache/avro/pull/2380
> >
> >
> > git switch branch-1.11
> > cd lang/py
> > ./build.sh doc
> > open .../index.html
> >
> > or you mean something else ?
> >
>
> I just tried it and :
>
> ./build.sh doc
> /bin/python3: No module named tox
>
> I see that the "dist' target creates a virtual environment automatically.
> Does it make sense to do the same for doc/lint ?!
> Once the venv is created and activated, and tox installed the doc target
> fails with:
>
> The HTML pages are in docs/build/html.
> .pkg: _exit> python
> /home/martin/git/apache/avro/lang/py/.env/lib/python3.11/site-packages/pyproject_api/_backend.py
> True setuptools.build_meta
> docs: OK (21.52=setup[17.53]+cmd[3.99] seconds)
> congratulations :) (21.59 seconds)
> cp: cannot create directory '../../build/avro-doc-1.12.0-SNAPSHOT/api/py':
> No such file or directory
>
>
> Fix:
>
> diff --git lang/py/build.sh lang/py/build.sh
> index a8f3febdb..5191fb38d 100755
> --- lang/py/build.sh
> +++ lang/py/build.sh
> @@ -55,6 +55,7 @@ doc() {
> local doc_dir
> [[ -s VERSION.txt ]] || cp ../../share/VERSION.txt .
> doc_dir="../../build/avro-doc-$(<VERSION.txt)/api/py"
> + mkdir -p $doc_dir
> python3 -m tox -e docs
> cp -a docs/build/* "$doc_dir"
>
>
>
> >
> >
> >
> >>
> >>
> >> On Fri, Jul 21, 2023 at 4:34 AM Ryan Skraba <ry...@skraba.com> wrote:
> >> >
> >> > Hey thanks for this work! Do you think the python doc generation
> >> > stuff can be cherry-picked back to 1.11? That would be a neat
> >> > addition to the website for the incoming 1.11.3!
> >> >
> >> > Building and deploying the website today is a really tricky problem...
> >> > I think we're going to have to make some major changes to simplify
> >> > this soon...
> >> >
> >> > I suspect that the solution is going to look something like what Flink
> >> > does: two separate static websites: one for the community and one
> >> > (well, one-per) for the release, but they should look and feel
> >> > integrated. These python docs should be generated for the per-release
> >> > pages.
> >> >
> >> > There's a JIRA to investigate this, and I'm definitely on the low end
> >> > of this learning curve, but I think it should be doable!
> >> >
> >> > All my best, Ryan
> >> >
> >> >
> >> > On Fri, Jul 21, 2023 at 9:35 AM Martin Grigorov <mg...@apache.org>
> >> wrote:
> >> > >
> >> > > On Thu, Jul 20, 2023 at 7:20 PM Michael A. Smith <
> >> michael@smith-li.com>
> >> > > wrote:
> >> > >
> >> > > > OK, I've merged that PR and have made suggested changes to #2187.
> >> > > > Please let me know how I can help.
> >> > > >
> >> > >
> >> > > Thanks !
> >> > > Hopefully this PR will be merged soon!
> >> > >
> >> > >
> >> > >
> >> > > >
> >> > > > On Thu, Jul 20, 2023 at 11:27 AM Martin Grigorov <
> >> mgrigorov@apache.org>
> >> > > > wrote:
> >> > > > >
> >> > > > > On Thu, 20 Jul 2023 at 17:47, Michael A. Smith <
> >> michael@smith-li.com>
> >> > > > wrote:
> >> > > > >
> >> > > > > > Thanks, Martin,
> >> > > > > >
> >> > > > > > It seems like since that PR isn't merged yet, and it would be
> >> > > > > > complicated to add all the Sphinx stuff to it, I should merge my
> >> > > > > > changes first, and then add the Python stuff to #2187 after
> >> that.
> >> > > > > >
> >> > > > > > Does that seem reasonable?
> >> > > > >
> >> > > > >
> >> > > > > Yep!
> >> > > > >
> >> > > > >
> >> > > > > >
> >> > > > > > On Thu, Jul 20, 2023 at 2:30 AM Martin Grigorov <
> >> mgrigorov@apache.org>
> >> > > > > > wrote:
> >> > > > > > >
> >> > > > > > > Hi Michael,
> >> > > > > > >
> >> > > > > > > The new website uses Hugo to build the static files.
> >> > > > > > > If you prefer to use Sphinx for the Python docs then I
> >> suggest to
> >> > > > follow
> >> > > > > > > the way of C/C++/C#/Java SDKs contribute their part of the
> >> docs in
> >> > > > this
> >> > > > > > PR
> >> > > > > > > -
> >> > > > > > >
> >> > > > > >
> >> > > >
> >> https://github.com/apache/avro/pull/2187/files#diff-d54d69dbb27e75dae25cb4b2384310cb57707e419377cf572d5cb0ecc1f16877R76-R162
> >> > > > > > > You need to add a new CI job that installs Sphinx, make and
> >> other
> >> > > > > > > dependencies, then call "make" and finally upload the HTMLs.
> >> Later
> >> > > > in the
> >> > > > > > > push-website job you need to download the HTMLs and copy them
> >> to
> >> > > > > > > website/docs/++version++/api/py
> >> > > > > > >
> >> > > > > > > Let me know if you have any questions!
> >> > > > > > >
> >> > > > > > > Regards,
> >> > > > > > > Martin
> >> > > > > > >
> >> > > > > > > On Thu, Jul 20, 2023 at 5:10 AM Michael Smith <
> >> michael@smith-li.com>
> >> > > > > > wrote:
> >> > > > > > >
> >> > > > > > > > I did a PR to implement a very old ticket, AVRO-312.
> >> > > > > > > >
> >> > > > > > > > https://github.com/apache/avro/pull/2370
> >> > > > > > > >
> >> > > > > > > > It builds the documentation in html and I can view it
> >> locally.
> >> > > > What I
> >> > > > > > need
> >> > > > > > > > is for someone who understands the process for publishing
> >> the avro
> >> > > > > > website
> >> > > > > > > > to let me know if my changes will properly publish the
> >> Python API
> >> > > > docs
> >> > > > > > the
> >> > > > > > > > expected way, so they end up on the left navigation,
> >> following C#
> >> > > > API
> >> > > > > > on
> >> > > > > > > > avro.apache.org.
> >> > > > > > > >
> >> > > > > > > > Can someone check that for me?
> >> > > > > > > >
> >> > > > > > > > Thanks,
> >> > > > > > > > Michael
> >> > > > > > > >
> >> > > > > >
> >> > > >
> >>
> >
Re: Implementing Python API Docs
Posted by Martin Grigorov <mg...@apache.org>.
On Fri, Jul 21, 2023 at 10:47 PM Martin Grigorov <mg...@apache.org>
wrote:
>
>
> On Fri, Jul 21, 2023 at 10:18 PM Michael A. Smith <mi...@smith-li.com>
> wrote:
>
>> I did the cherry-pick, but I'm not sure I know how to test if it
>> works. https://github.com/apache/avro/pull/2380
>
>
> git switch branch-1.11
> cd lang/py
> ./build.sh doc
> open .../index.html
>
> or you mean something else ?
>
I just tried it and :
./build.sh doc
/bin/python3: No module named tox
I see that the "dist' target creates a virtual environment automatically.
Does it make sense to do the same for doc/lint ?!
Once the venv is created and activated, and tox installed the doc target
fails with:
The HTML pages are in docs/build/html.
.pkg: _exit> python
/home/martin/git/apache/avro/lang/py/.env/lib/python3.11/site-packages/pyproject_api/_backend.py
True setuptools.build_meta
docs: OK (21.52=setup[17.53]+cmd[3.99] seconds)
congratulations :) (21.59 seconds)
cp: cannot create directory '../../build/avro-doc-1.12.0-SNAPSHOT/api/py':
No such file or directory
Fix:
diff --git lang/py/build.sh lang/py/build.sh
index a8f3febdb..5191fb38d 100755
--- lang/py/build.sh
+++ lang/py/build.sh
@@ -55,6 +55,7 @@ doc() {
local doc_dir
[[ -s VERSION.txt ]] || cp ../../share/VERSION.txt .
doc_dir="../../build/avro-doc-$(<VERSION.txt)/api/py"
+ mkdir -p $doc_dir
python3 -m tox -e docs
cp -a docs/build/* "$doc_dir"
>
>
>
>>
>>
>> On Fri, Jul 21, 2023 at 4:34 AM Ryan Skraba <ry...@skraba.com> wrote:
>> >
>> > Hey thanks for this work! Do you think the python doc generation
>> > stuff can be cherry-picked back to 1.11? That would be a neat
>> > addition to the website for the incoming 1.11.3!
>> >
>> > Building and deploying the website today is a really tricky problem...
>> > I think we're going to have to make some major changes to simplify
>> > this soon...
>> >
>> > I suspect that the solution is going to look something like what Flink
>> > does: two separate static websites: one for the community and one
>> > (well, one-per) for the release, but they should look and feel
>> > integrated. These python docs should be generated for the per-release
>> > pages.
>> >
>> > There's a JIRA to investigate this, and I'm definitely on the low end
>> > of this learning curve, but I think it should be doable!
>> >
>> > All my best, Ryan
>> >
>> >
>> > On Fri, Jul 21, 2023 at 9:35 AM Martin Grigorov <mg...@apache.org>
>> wrote:
>> > >
>> > > On Thu, Jul 20, 2023 at 7:20 PM Michael A. Smith <
>> michael@smith-li.com>
>> > > wrote:
>> > >
>> > > > OK, I've merged that PR and have made suggested changes to #2187.
>> > > > Please let me know how I can help.
>> > > >
>> > >
>> > > Thanks !
>> > > Hopefully this PR will be merged soon!
>> > >
>> > >
>> > >
>> > > >
>> > > > On Thu, Jul 20, 2023 at 11:27 AM Martin Grigorov <
>> mgrigorov@apache.org>
>> > > > wrote:
>> > > > >
>> > > > > On Thu, 20 Jul 2023 at 17:47, Michael A. Smith <
>> michael@smith-li.com>
>> > > > wrote:
>> > > > >
>> > > > > > Thanks, Martin,
>> > > > > >
>> > > > > > It seems like since that PR isn't merged yet, and it would be
>> > > > > > complicated to add all the Sphinx stuff to it, I should merge my
>> > > > > > changes first, and then add the Python stuff to #2187 after
>> that.
>> > > > > >
>> > > > > > Does that seem reasonable?
>> > > > >
>> > > > >
>> > > > > Yep!
>> > > > >
>> > > > >
>> > > > > >
>> > > > > > On Thu, Jul 20, 2023 at 2:30 AM Martin Grigorov <
>> mgrigorov@apache.org>
>> > > > > > wrote:
>> > > > > > >
>> > > > > > > Hi Michael,
>> > > > > > >
>> > > > > > > The new website uses Hugo to build the static files.
>> > > > > > > If you prefer to use Sphinx for the Python docs then I
>> suggest to
>> > > > follow
>> > > > > > > the way of C/C++/C#/Java SDKs contribute their part of the
>> docs in
>> > > > this
>> > > > > > PR
>> > > > > > > -
>> > > > > > >
>> > > > > >
>> > > >
>> https://github.com/apache/avro/pull/2187/files#diff-d54d69dbb27e75dae25cb4b2384310cb57707e419377cf572d5cb0ecc1f16877R76-R162
>> > > > > > > You need to add a new CI job that installs Sphinx, make and
>> other
>> > > > > > > dependencies, then call "make" and finally upload the HTMLs.
>> Later
>> > > > in the
>> > > > > > > push-website job you need to download the HTMLs and copy them
>> to
>> > > > > > > website/docs/++version++/api/py
>> > > > > > >
>> > > > > > > Let me know if you have any questions!
>> > > > > > >
>> > > > > > > Regards,
>> > > > > > > Martin
>> > > > > > >
>> > > > > > > On Thu, Jul 20, 2023 at 5:10 AM Michael Smith <
>> michael@smith-li.com>
>> > > > > > wrote:
>> > > > > > >
>> > > > > > > > I did a PR to implement a very old ticket, AVRO-312.
>> > > > > > > >
>> > > > > > > > https://github.com/apache/avro/pull/2370
>> > > > > > > >
>> > > > > > > > It builds the documentation in html and I can view it
>> locally.
>> > > > What I
>> > > > > > need
>> > > > > > > > is for someone who understands the process for publishing
>> the avro
>> > > > > > website
>> > > > > > > > to let me know if my changes will properly publish the
>> Python API
>> > > > docs
>> > > > > > the
>> > > > > > > > expected way, so they end up on the left navigation,
>> following C#
>> > > > API
>> > > > > > on
>> > > > > > > > avro.apache.org.
>> > > > > > > >
>> > > > > > > > Can someone check that for me?
>> > > > > > > >
>> > > > > > > > Thanks,
>> > > > > > > > Michael
>> > > > > > > >
>> > > > > >
>> > > >
>>
>
Re: Implementing Python API Docs
Posted by Martin Grigorov <mg...@apache.org>.
On Fri, Jul 21, 2023 at 10:18 PM Michael A. Smith <mi...@smith-li.com>
wrote:
> I did the cherry-pick, but I'm not sure I know how to test if it
> works. https://github.com/apache/avro/pull/2380
git switch branch-1.11
cd lang/py
./build.sh doc
open .../index.html
or you mean something else ?
>
>
> On Fri, Jul 21, 2023 at 4:34 AM Ryan Skraba <ry...@skraba.com> wrote:
> >
> > Hey thanks for this work! Do you think the python doc generation
> > stuff can be cherry-picked back to 1.11? That would be a neat
> > addition to the website for the incoming 1.11.3!
> >
> > Building and deploying the website today is a really tricky problem...
> > I think we're going to have to make some major changes to simplify
> > this soon...
> >
> > I suspect that the solution is going to look something like what Flink
> > does: two separate static websites: one for the community and one
> > (well, one-per) for the release, but they should look and feel
> > integrated. These python docs should be generated for the per-release
> > pages.
> >
> > There's a JIRA to investigate this, and I'm definitely on the low end
> > of this learning curve, but I think it should be doable!
> >
> > All my best, Ryan
> >
> >
> > On Fri, Jul 21, 2023 at 9:35 AM Martin Grigorov <mg...@apache.org>
> wrote:
> > >
> > > On Thu, Jul 20, 2023 at 7:20 PM Michael A. Smith <michael@smith-li.com
> >
> > > wrote:
> > >
> > > > OK, I've merged that PR and have made suggested changes to #2187.
> > > > Please let me know how I can help.
> > > >
> > >
> > > Thanks !
> > > Hopefully this PR will be merged soon!
> > >
> > >
> > >
> > > >
> > > > On Thu, Jul 20, 2023 at 11:27 AM Martin Grigorov <
> mgrigorov@apache.org>
> > > > wrote:
> > > > >
> > > > > On Thu, 20 Jul 2023 at 17:47, Michael A. Smith <
> michael@smith-li.com>
> > > > wrote:
> > > > >
> > > > > > Thanks, Martin,
> > > > > >
> > > > > > It seems like since that PR isn't merged yet, and it would be
> > > > > > complicated to add all the Sphinx stuff to it, I should merge my
> > > > > > changes first, and then add the Python stuff to #2187 after that.
> > > > > >
> > > > > > Does that seem reasonable?
> > > > >
> > > > >
> > > > > Yep!
> > > > >
> > > > >
> > > > > >
> > > > > > On Thu, Jul 20, 2023 at 2:30 AM Martin Grigorov <
> mgrigorov@apache.org>
> > > > > > wrote:
> > > > > > >
> > > > > > > Hi Michael,
> > > > > > >
> > > > > > > The new website uses Hugo to build the static files.
> > > > > > > If you prefer to use Sphinx for the Python docs then I suggest
> to
> > > > follow
> > > > > > > the way of C/C++/C#/Java SDKs contribute their part of the
> docs in
> > > > this
> > > > > > PR
> > > > > > > -
> > > > > > >
> > > > > >
> > > >
> https://github.com/apache/avro/pull/2187/files#diff-d54d69dbb27e75dae25cb4b2384310cb57707e419377cf572d5cb0ecc1f16877R76-R162
> > > > > > > You need to add a new CI job that installs Sphinx, make and
> other
> > > > > > > dependencies, then call "make" and finally upload the HTMLs.
> Later
> > > > in the
> > > > > > > push-website job you need to download the HTMLs and copy them
> to
> > > > > > > website/docs/++version++/api/py
> > > > > > >
> > > > > > > Let me know if you have any questions!
> > > > > > >
> > > > > > > Regards,
> > > > > > > Martin
> > > > > > >
> > > > > > > On Thu, Jul 20, 2023 at 5:10 AM Michael Smith <
> michael@smith-li.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > > I did a PR to implement a very old ticket, AVRO-312.
> > > > > > > >
> > > > > > > > https://github.com/apache/avro/pull/2370
> > > > > > > >
> > > > > > > > It builds the documentation in html and I can view it
> locally.
> > > > What I
> > > > > > need
> > > > > > > > is for someone who understands the process for publishing
> the avro
> > > > > > website
> > > > > > > > to let me know if my changes will properly publish the
> Python API
> > > > docs
> > > > > > the
> > > > > > > > expected way, so they end up on the left navigation,
> following C#
> > > > API
> > > > > > on
> > > > > > > > avro.apache.org.
> > > > > > > >
> > > > > > > > Can someone check that for me?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Michael
> > > > > > > >
> > > > > >
> > > >
>
Re: Implementing Python API Docs
Posted by "Michael A. Smith" <mi...@smith-li.com>.
I did the cherry-pick, but I'm not sure I know how to test if it
works. https://github.com/apache/avro/pull/2380
On Fri, Jul 21, 2023 at 4:34 AM Ryan Skraba <ry...@skraba.com> wrote:
>
> Hey thanks for this work! Do you think the python doc generation
> stuff can be cherry-picked back to 1.11? That would be a neat
> addition to the website for the incoming 1.11.3!
>
> Building and deploying the website today is a really tricky problem...
> I think we're going to have to make some major changes to simplify
> this soon...
>
> I suspect that the solution is going to look something like what Flink
> does: two separate static websites: one for the community and one
> (well, one-per) for the release, but they should look and feel
> integrated. These python docs should be generated for the per-release
> pages.
>
> There's a JIRA to investigate this, and I'm definitely on the low end
> of this learning curve, but I think it should be doable!
>
> All my best, Ryan
>
>
> On Fri, Jul 21, 2023 at 9:35 AM Martin Grigorov <mg...@apache.org> wrote:
> >
> > On Thu, Jul 20, 2023 at 7:20 PM Michael A. Smith <mi...@smith-li.com>
> > wrote:
> >
> > > OK, I've merged that PR and have made suggested changes to #2187.
> > > Please let me know how I can help.
> > >
> >
> > Thanks !
> > Hopefully this PR will be merged soon!
> >
> >
> >
> > >
> > > On Thu, Jul 20, 2023 at 11:27 AM Martin Grigorov <mg...@apache.org>
> > > wrote:
> > > >
> > > > On Thu, 20 Jul 2023 at 17:47, Michael A. Smith <mi...@smith-li.com>
> > > wrote:
> > > >
> > > > > Thanks, Martin,
> > > > >
> > > > > It seems like since that PR isn't merged yet, and it would be
> > > > > complicated to add all the Sphinx stuff to it, I should merge my
> > > > > changes first, and then add the Python stuff to #2187 after that.
> > > > >
> > > > > Does that seem reasonable?
> > > >
> > > >
> > > > Yep!
> > > >
> > > >
> > > > >
> > > > > On Thu, Jul 20, 2023 at 2:30 AM Martin Grigorov <mg...@apache.org>
> > > > > wrote:
> > > > > >
> > > > > > Hi Michael,
> > > > > >
> > > > > > The new website uses Hugo to build the static files.
> > > > > > If you prefer to use Sphinx for the Python docs then I suggest to
> > > follow
> > > > > > the way of C/C++/C#/Java SDKs contribute their part of the docs in
> > > this
> > > > > PR
> > > > > > -
> > > > > >
> > > > >
> > > https://github.com/apache/avro/pull/2187/files#diff-d54d69dbb27e75dae25cb4b2384310cb57707e419377cf572d5cb0ecc1f16877R76-R162
> > > > > > You need to add a new CI job that installs Sphinx, make and other
> > > > > > dependencies, then call "make" and finally upload the HTMLs. Later
> > > in the
> > > > > > push-website job you need to download the HTMLs and copy them to
> > > > > > website/docs/++version++/api/py
> > > > > >
> > > > > > Let me know if you have any questions!
> > > > > >
> > > > > > Regards,
> > > > > > Martin
> > > > > >
> > > > > > On Thu, Jul 20, 2023 at 5:10 AM Michael Smith <mi...@smith-li.com>
> > > > > wrote:
> > > > > >
> > > > > > > I did a PR to implement a very old ticket, AVRO-312.
> > > > > > >
> > > > > > > https://github.com/apache/avro/pull/2370
> > > > > > >
> > > > > > > It builds the documentation in html and I can view it locally.
> > > What I
> > > > > need
> > > > > > > is for someone who understands the process for publishing the avro
> > > > > website
> > > > > > > to let me know if my changes will properly publish the Python API
> > > docs
> > > > > the
> > > > > > > expected way, so they end up on the left navigation, following C#
> > > API
> > > > > on
> > > > > > > avro.apache.org.
> > > > > > >
> > > > > > > Can someone check that for me?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Michael
> > > > > > >
> > > > >
> > >
Re: Implementing Python API Docs
Posted by Ryan Skraba <ry...@skraba.com>.
Hey thanks for this work! Do you think the python doc generation
stuff can be cherry-picked back to 1.11? That would be a neat
addition to the website for the incoming 1.11.3!
Building and deploying the website today is a really tricky problem...
I think we're going to have to make some major changes to simplify
this soon...
I suspect that the solution is going to look something like what Flink
does: two separate static websites: one for the community and one
(well, one-per) for the release, but they should look and feel
integrated. These python docs should be generated for the per-release
pages.
There's a JIRA to investigate this, and I'm definitely on the low end
of this learning curve, but I think it should be doable!
All my best, Ryan
On Fri, Jul 21, 2023 at 9:35 AM Martin Grigorov <mg...@apache.org> wrote:
>
> On Thu, Jul 20, 2023 at 7:20 PM Michael A. Smith <mi...@smith-li.com>
> wrote:
>
> > OK, I've merged that PR and have made suggested changes to #2187.
> > Please let me know how I can help.
> >
>
> Thanks !
> Hopefully this PR will be merged soon!
>
>
>
> >
> > On Thu, Jul 20, 2023 at 11:27 AM Martin Grigorov <mg...@apache.org>
> > wrote:
> > >
> > > On Thu, 20 Jul 2023 at 17:47, Michael A. Smith <mi...@smith-li.com>
> > wrote:
> > >
> > > > Thanks, Martin,
> > > >
> > > > It seems like since that PR isn't merged yet, and it would be
> > > > complicated to add all the Sphinx stuff to it, I should merge my
> > > > changes first, and then add the Python stuff to #2187 after that.
> > > >
> > > > Does that seem reasonable?
> > >
> > >
> > > Yep!
> > >
> > >
> > > >
> > > > On Thu, Jul 20, 2023 at 2:30 AM Martin Grigorov <mg...@apache.org>
> > > > wrote:
> > > > >
> > > > > Hi Michael,
> > > > >
> > > > > The new website uses Hugo to build the static files.
> > > > > If you prefer to use Sphinx for the Python docs then I suggest to
> > follow
> > > > > the way of C/C++/C#/Java SDKs contribute their part of the docs in
> > this
> > > > PR
> > > > > -
> > > > >
> > > >
> > https://github.com/apache/avro/pull/2187/files#diff-d54d69dbb27e75dae25cb4b2384310cb57707e419377cf572d5cb0ecc1f16877R76-R162
> > > > > You need to add a new CI job that installs Sphinx, make and other
> > > > > dependencies, then call "make" and finally upload the HTMLs. Later
> > in the
> > > > > push-website job you need to download the HTMLs and copy them to
> > > > > website/docs/++version++/api/py
> > > > >
> > > > > Let me know if you have any questions!
> > > > >
> > > > > Regards,
> > > > > Martin
> > > > >
> > > > > On Thu, Jul 20, 2023 at 5:10 AM Michael Smith <mi...@smith-li.com>
> > > > wrote:
> > > > >
> > > > > > I did a PR to implement a very old ticket, AVRO-312.
> > > > > >
> > > > > > https://github.com/apache/avro/pull/2370
> > > > > >
> > > > > > It builds the documentation in html and I can view it locally.
> > What I
> > > > need
> > > > > > is for someone who understands the process for publishing the avro
> > > > website
> > > > > > to let me know if my changes will properly publish the Python API
> > docs
> > > > the
> > > > > > expected way, so they end up on the left navigation, following C#
> > API
> > > > on
> > > > > > avro.apache.org.
> > > > > >
> > > > > > Can someone check that for me?
> > > > > >
> > > > > > Thanks,
> > > > > > Michael
> > > > > >
> > > >
> >
Re: Implementing Python API Docs
Posted by Martin Grigorov <mg...@apache.org>.
On Thu, Jul 20, 2023 at 7:20 PM Michael A. Smith <mi...@smith-li.com>
wrote:
> OK, I've merged that PR and have made suggested changes to #2187.
> Please let me know how I can help.
>
Thanks !
Hopefully this PR will be merged soon!
>
> On Thu, Jul 20, 2023 at 11:27 AM Martin Grigorov <mg...@apache.org>
> wrote:
> >
> > On Thu, 20 Jul 2023 at 17:47, Michael A. Smith <mi...@smith-li.com>
> wrote:
> >
> > > Thanks, Martin,
> > >
> > > It seems like since that PR isn't merged yet, and it would be
> > > complicated to add all the Sphinx stuff to it, I should merge my
> > > changes first, and then add the Python stuff to #2187 after that.
> > >
> > > Does that seem reasonable?
> >
> >
> > Yep!
> >
> >
> > >
> > > On Thu, Jul 20, 2023 at 2:30 AM Martin Grigorov <mg...@apache.org>
> > > wrote:
> > > >
> > > > Hi Michael,
> > > >
> > > > The new website uses Hugo to build the static files.
> > > > If you prefer to use Sphinx for the Python docs then I suggest to
> follow
> > > > the way of C/C++/C#/Java SDKs contribute their part of the docs in
> this
> > > PR
> > > > -
> > > >
> > >
> https://github.com/apache/avro/pull/2187/files#diff-d54d69dbb27e75dae25cb4b2384310cb57707e419377cf572d5cb0ecc1f16877R76-R162
> > > > You need to add a new CI job that installs Sphinx, make and other
> > > > dependencies, then call "make" and finally upload the HTMLs. Later
> in the
> > > > push-website job you need to download the HTMLs and copy them to
> > > > website/docs/++version++/api/py
> > > >
> > > > Let me know if you have any questions!
> > > >
> > > > Regards,
> > > > Martin
> > > >
> > > > On Thu, Jul 20, 2023 at 5:10 AM Michael Smith <mi...@smith-li.com>
> > > wrote:
> > > >
> > > > > I did a PR to implement a very old ticket, AVRO-312.
> > > > >
> > > > > https://github.com/apache/avro/pull/2370
> > > > >
> > > > > It builds the documentation in html and I can view it locally.
> What I
> > > need
> > > > > is for someone who understands the process for publishing the avro
> > > website
> > > > > to let me know if my changes will properly publish the Python API
> docs
> > > the
> > > > > expected way, so they end up on the left navigation, following C#
> API
> > > on
> > > > > avro.apache.org.
> > > > >
> > > > > Can someone check that for me?
> > > > >
> > > > > Thanks,
> > > > > Michael
> > > > >
> > >
>
Re: Implementing Python API Docs
Posted by "Michael A. Smith" <mi...@smith-li.com>.
OK, I've merged that PR and have made suggested changes to #2187.
Please let me know how I can help.
On Thu, Jul 20, 2023 at 11:27 AM Martin Grigorov <mg...@apache.org> wrote:
>
> On Thu, 20 Jul 2023 at 17:47, Michael A. Smith <mi...@smith-li.com> wrote:
>
> > Thanks, Martin,
> >
> > It seems like since that PR isn't merged yet, and it would be
> > complicated to add all the Sphinx stuff to it, I should merge my
> > changes first, and then add the Python stuff to #2187 after that.
> >
> > Does that seem reasonable?
>
>
> Yep!
>
>
> >
> > On Thu, Jul 20, 2023 at 2:30 AM Martin Grigorov <mg...@apache.org>
> > wrote:
> > >
> > > Hi Michael,
> > >
> > > The new website uses Hugo to build the static files.
> > > If you prefer to use Sphinx for the Python docs then I suggest to follow
> > > the way of C/C++/C#/Java SDKs contribute their part of the docs in this
> > PR
> > > -
> > >
> > https://github.com/apache/avro/pull/2187/files#diff-d54d69dbb27e75dae25cb4b2384310cb57707e419377cf572d5cb0ecc1f16877R76-R162
> > > You need to add a new CI job that installs Sphinx, make and other
> > > dependencies, then call "make" and finally upload the HTMLs. Later in the
> > > push-website job you need to download the HTMLs and copy them to
> > > website/docs/++version++/api/py
> > >
> > > Let me know if you have any questions!
> > >
> > > Regards,
> > > Martin
> > >
> > > On Thu, Jul 20, 2023 at 5:10 AM Michael Smith <mi...@smith-li.com>
> > wrote:
> > >
> > > > I did a PR to implement a very old ticket, AVRO-312.
> > > >
> > > > https://github.com/apache/avro/pull/2370
> > > >
> > > > It builds the documentation in html and I can view it locally. What I
> > need
> > > > is for someone who understands the process for publishing the avro
> > website
> > > > to let me know if my changes will properly publish the Python API docs
> > the
> > > > expected way, so they end up on the left navigation, following C# API
> > on
> > > > avro.apache.org.
> > > >
> > > > Can someone check that for me?
> > > >
> > > > Thanks,
> > > > Michael
> > > >
> >
Re: Implementing Python API Docs
Posted by Martin Grigorov <mg...@apache.org>.
On Thu, 20 Jul 2023 at 17:47, Michael A. Smith <mi...@smith-li.com> wrote:
> Thanks, Martin,
>
> It seems like since that PR isn't merged yet, and it would be
> complicated to add all the Sphinx stuff to it, I should merge my
> changes first, and then add the Python stuff to #2187 after that.
>
> Does that seem reasonable?
Yep!
>
> On Thu, Jul 20, 2023 at 2:30 AM Martin Grigorov <mg...@apache.org>
> wrote:
> >
> > Hi Michael,
> >
> > The new website uses Hugo to build the static files.
> > If you prefer to use Sphinx for the Python docs then I suggest to follow
> > the way of C/C++/C#/Java SDKs contribute their part of the docs in this
> PR
> > -
> >
> https://github.com/apache/avro/pull/2187/files#diff-d54d69dbb27e75dae25cb4b2384310cb57707e419377cf572d5cb0ecc1f16877R76-R162
> > You need to add a new CI job that installs Sphinx, make and other
> > dependencies, then call "make" and finally upload the HTMLs. Later in the
> > push-website job you need to download the HTMLs and copy them to
> > website/docs/++version++/api/py
> >
> > Let me know if you have any questions!
> >
> > Regards,
> > Martin
> >
> > On Thu, Jul 20, 2023 at 5:10 AM Michael Smith <mi...@smith-li.com>
> wrote:
> >
> > > I did a PR to implement a very old ticket, AVRO-312.
> > >
> > > https://github.com/apache/avro/pull/2370
> > >
> > > It builds the documentation in html and I can view it locally. What I
> need
> > > is for someone who understands the process for publishing the avro
> website
> > > to let me know if my changes will properly publish the Python API docs
> the
> > > expected way, so they end up on the left navigation, following C# API
> on
> > > avro.apache.org.
> > >
> > > Can someone check that for me?
> > >
> > > Thanks,
> > > Michael
> > >
>
Re: Implementing Python API Docs
Posted by "Michael A. Smith" <mi...@smith-li.com>.
Thanks, Martin,
It seems like since that PR isn't merged yet, and it would be
complicated to add all the Sphinx stuff to it, I should merge my
changes first, and then add the Python stuff to #2187 after that.
Does that seem reasonable?
On Thu, Jul 20, 2023 at 2:30 AM Martin Grigorov <mg...@apache.org> wrote:
>
> Hi Michael,
>
> The new website uses Hugo to build the static files.
> If you prefer to use Sphinx for the Python docs then I suggest to follow
> the way of C/C++/C#/Java SDKs contribute their part of the docs in this PR
> -
> https://github.com/apache/avro/pull/2187/files#diff-d54d69dbb27e75dae25cb4b2384310cb57707e419377cf572d5cb0ecc1f16877R76-R162
> You need to add a new CI job that installs Sphinx, make and other
> dependencies, then call "make" and finally upload the HTMLs. Later in the
> push-website job you need to download the HTMLs and copy them to
> website/docs/++version++/api/py
>
> Let me know if you have any questions!
>
> Regards,
> Martin
>
> On Thu, Jul 20, 2023 at 5:10 AM Michael Smith <mi...@smith-li.com> wrote:
>
> > I did a PR to implement a very old ticket, AVRO-312.
> >
> > https://github.com/apache/avro/pull/2370
> >
> > It builds the documentation in html and I can view it locally. What I need
> > is for someone who understands the process for publishing the avro website
> > to let me know if my changes will properly publish the Python API docs the
> > expected way, so they end up on the left navigation, following C# API on
> > avro.apache.org.
> >
> > Can someone check that for me?
> >
> > Thanks,
> > Michael
> >
Re: Implementing Python API Docs
Posted by Martin Grigorov <mg...@apache.org>.
Hi Michael,
The new website uses Hugo to build the static files.
If you prefer to use Sphinx for the Python docs then I suggest to follow
the way of C/C++/C#/Java SDKs contribute their part of the docs in this PR
-
https://github.com/apache/avro/pull/2187/files#diff-d54d69dbb27e75dae25cb4b2384310cb57707e419377cf572d5cb0ecc1f16877R76-R162
You need to add a new CI job that installs Sphinx, make and other
dependencies, then call "make" and finally upload the HTMLs. Later in the
push-website job you need to download the HTMLs and copy them to
website/docs/++version++/api/py
Let me know if you have any questions!
Regards,
Martin
On Thu, Jul 20, 2023 at 5:10 AM Michael Smith <mi...@smith-li.com> wrote:
> I did a PR to implement a very old ticket, AVRO-312.
>
> https://github.com/apache/avro/pull/2370
>
> It builds the documentation in html and I can view it locally. What I need
> is for someone who understands the process for publishing the avro website
> to let me know if my changes will properly publish the Python API docs the
> expected way, so they end up on the left navigation, following C# API on
> avro.apache.org.
>
> Can someone check that for me?
>
> Thanks,
> Michael
>