You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2024/01/15 01:46:46 UTC
[PR] [INFRA] Pin `sphinxcontrib-applehelp==1.0.4` [spark]
zhengruifeng opened a new pull request, #44727:
URL: https://github.com/apache/spark/pull/44727
### What changes were proposed in this pull request?
Pin `sphinxcontrib-applehelp==1.0.4`
previously,
`Install Python linter dependencies` install `sphinxcontrib-applehelp-1.0.7`, and then `Install dependencies for documentation generation` reinstall it with `sphinxcontrib-applehelp-1.0.4`;
now,
`Install Python linter dependencies` install `sphinxcontrib-applehelp-1.0.8`, and `Install dependencies for documentation generation` keep this intallation:
`Requirement already satisfied: sphinxcontrib-applehelp in /usr/local/lib/python3.9/dist-packages (from sphinx==4.5.0) (1.0.8)`
### Why are the changes needed?
doc build is failing with:
```
Sphinx version error:
The sphinxcontrib.applehelp extension used by this project needs at least Sphinx v5.0; it therefore cannot be built with this version.
make: *** [Makefile:35: html] Error 2
------------------------------------------------
Jekyll 4.3.3 Please append `--trace` to the `build` command
for any additional information or backtrace.
------------------------------------------------
/__w/spark/spark/docs/_plugins/copy_api_dirs.rb:131:in `<top (required)>': Python doc generation failed (RuntimeError)
from /__w/spark/spark/docs/.local_ruby_bundle/ruby/2.7.0/gems/jekyll-4.3.3/lib/jekyll/external.rb:57:in `require'
from /__w/spark/spark/docs/.local_ruby_bundle/ruby/2.7.0/gems/jekyll-4.3.3/lib/jekyll/external.rb:57:in `block in require_with_graceful_fail'
```
### Does this PR introduce _any_ user-facing change?
no
### How was this patch tested?
ci
### Was this patch authored or co-authored using generative AI tooling?
no
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [INFRA] Pin `sphinxcontrib-applehelp==1.0.4` [spark]
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #44727:
URL: https://github.com/apache/spark/pull/44727#issuecomment-1891216444
> Mind creating a JIRA ticket @zhengruifeng?
sure, it's still under test, maybe we need to pin more deps.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [INFRA] Pin `sphinxcontrib-applehelp==1.0.4` [spark]
Posted by "yaooqinn (via GitHub)" <gi...@apache.org>.
yaooqinn commented on PR #44727:
URL: https://github.com/apache/spark/pull/44727#issuecomment-1891216040
Mind creating a JIRA ticket @zhengruifeng?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-46715][INFRA] Pin `sphinxcontrib-*` [spark]
Posted by "nchammas (via GitHub)" <gi...@apache.org>.
nchammas commented on PR #44727:
URL: https://github.com/apache/spark/pull/44727#issuecomment-1891270353
The proximate reason for this failure is that we install `pydata_sphinx_theme` without a version qualifier here:
https://github.com/apache/spark/blob/9f061b0d03a3847f48329f70e05e2936aeac91c9/.github/workflows/build_and_test.yml#L705
Note that this is separate from the constraint that `sphinx==4.5.0`, so pip installs `pydata_sphinx_theme` at version 0.15.1. This in turn requires `sphinx>=5.0`, which pulls in the incompatible Sphinx extensions mentioned in the error you are seeing.
A longer term solution would be to compile all of our requirements per Python version and install them in one go. That's the only way pip is going to have the information it needs to prevent this kind of problem from occurring.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-46715][INFRA] Pin `sphinxcontrib-*` [spark]
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on PR #44727:
URL: https://github.com/apache/spark/pull/44727#issuecomment-1891240669
> > How about upgrade sphinx to 5.0.0 directly? I am conducting tests, and it seems ok:
> > #44729
> > https://github.com/LuciferYang/spark/actions/runs/7523170642/job/20476201559
> > 
>
> I remember @itholic had compared multiple versions of sphinx, and chose 4.5.0. Other versions cause some render issues without failing in CI.
OK, let's continue using 4.5.0 for now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-46715][INFRA] Pin `sphinxcontrib-*` [spark]
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #44727:
URL: https://github.com/apache/spark/pull/44727#issuecomment-1891266024
thanks all
merged to master to restore CI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-46715][INFRA] Pin `sphinxcontrib-*` [spark]
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on PR #44727:
URL: https://github.com/apache/spark/pull/44727#issuecomment-1891238112
How about upgrade sphinx to 5.0.0 directly? I am conducting tests, and it seems ok:
https://github.com/apache/spark/pull/44729
https://github.com/LuciferYang/spark/actions/runs/7523170642/job/20476201559
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-46715][INFRA] Pin `sphinxcontrib-*` [spark]
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng closed pull request #44727: [SPARK-46715][INFRA] Pin `sphinxcontrib-*`
URL: https://github.com/apache/spark/pull/44727
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-46715][INFRA] Pin `sphinxcontrib-*` [spark]
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #44727:
URL: https://github.com/apache/spark/pull/44727#issuecomment-1891425213
@wbo4958 It seems the `pydata_sphinx_theme` installations is not changed:
it installs `pydata_sphinx_theme-0.15.1` in `Install Python linter dependencies`, and then `pydata_sphinx_theme-0.13.3` in `Install dependencies for documentation generation`, in both a successful run (https://github.com/apache/spark/actions/runs/7507799677/job/20442119109) and a failed one (https://github.com/apache/spark/actions/runs/7522706520/job/20475105206)
The reason I think is `pydata_sphinx_theme` itself (both `0.15.1` and `0.13.3`) doesn't specify the versions of `sphinxcontrib-*`:
```
│ ├── sphinxcontrib-applehelp sphinxcontrib-applehelp is a Sphinx extension which outputs Apple help books
│ ├── sphinxcontrib-devhelp sphinxcontrib-devhelp is a sphinx extension which outputs Devhelp documents
│ ├── sphinxcontrib-htmlhelp>=2.0.0 sphinxcontrib-htmlhelp is a sphinx extension which renders HTML help files
│ ├── sphinxcontrib-jsmath A sphinx extension which renders display math in HTML via JavaScript
│ ├── sphinxcontrib-qthelp sphinxcontrib-qthelp is a sphinx extension which outputs QtHelp documents
│ └── sphinxcontrib-serializinghtml>=1.1.9 sphinxcontrib-serializinghtml is a sphinx extension which outputs "serialized"
```
then latest `sphinxcontrib-*` is not compatabile with sphinx.
As to 3.5, I am not sure, since it uses `sphinx<3.1.0`, probably needs a separate fix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-46715][INFRA] Pin `sphinxcontrib-*` [spark]
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #44727:
URL: https://github.com/apache/spark/pull/44727#issuecomment-1891239995
> How about upgrade sphinx to 5.0.0 directly? I am conducting tests, and it seems ok:
>
> #44729
>
> https://github.com/LuciferYang/spark/actions/runs/7523170642/job/20476201559
>
> 
I remember @itholic had compared multiple versions of sphinx, and chose 4.5.0.
Other versions cause some render issues without failing in CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-46715][INFRA] Pin `sphinxcontrib-*` [spark]
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #44727:
URL: https://github.com/apache/spark/pull/44727#issuecomment-1891246229
FYI, 4.2.0 is chosen in https://github.com/apache/spark/pull/44012#discussion_r1405239729 and then 4.5.0 for performance in https://github.com/apache/spark/commit/5eec99df01f82f5b247b6523f4169cfc05e6cd11
let's stick to 4.x before more investigation on the build docs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-46715][INFRA] Pin `sphinxcontrib-*` [spark]
Posted by "wbo4958 (via GitHub)" <gi...@apache.org>.
wbo4958 commented on PR #44727:
URL: https://github.com/apache/spark/pull/44727#issuecomment-1891303552
@zhengruifeng, seems this issue is till in branch-3.5, is it possible to back-port to branch 3.5?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-46715][INFRA] Pin `sphinxcontrib-*` [spark]
Posted by "wbo4958 (via GitHub)" <gi...@apache.org>.
wbo4958 commented on PR #44727:
URL: https://github.com/apache/spark/pull/44727#issuecomment-1892918125
Thx for your fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org