You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by gu...@apache.org on 2021/05/12 01:14:41 UTC

[spark] branch branch-3.1 updated: [SPARK-35375][INFRA] Use Jinja2 < 3.0.0 for Python linter dependency in GA

This is an automated email from the ASF dual-hosted git repository.

gurwls223 pushed a commit to branch branch-3.1
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-3.1 by this push:
     new f9de071  [SPARK-35375][INFRA] Use Jinja2 < 3.0.0 for Python linter dependency in GA
f9de071 is described below

commit f9de071d9b6b19f7f9828a2e34d362ff596c7b59
Author: Kousuke Saruta <sa...@oss.nttdata.com>
AuthorDate: Wed May 12 10:13:38 2021 +0900

    [SPARK-35375][INFRA] Use Jinja2 < 3.0.0 for Python linter dependency in GA
    
    ### What changes were proposed in this pull request?
    
    From a few hours ago, Python linter fails in GA.
    The latest Jinja 3.0.0 seems to cause this failure.
    https://pypi.org/project/Jinja2/
    
    ```
    Run ./dev/lint-python
    starting python compilation test...
    python compilation succeeded.
    
    starting pycodestyle test...
    pycodestyle checks passed.
    
    starting flake8 test...
    flake8 checks passed.
    
    starting mypy test...
    mypy checks passed.
    
    starting sphinx-build tests...
    sphinx-build checks failed:
    Running Sphinx v3.0.4
    making output directory... done
    [autosummary] generating autosummary for: development/contributing.rst, development/debugging.rst, development/index.rst, development/setting_ide.rst, development/testing.rst, getting_started/index.rst, getting_started/install.rst, getting_started/quickstart.ipynb, index.rst, migration_guide/index.rst, ..., reference/pyspark.ml.rst, reference/pyspark.mllib.rst, reference/pyspark.resource.rst, reference/pyspark.rst, reference/pyspark.sql.rst, reference/pyspark.ss.rst, reference/pyspark [...]
    
    Exception occurred:
      File "/__w/spark/spark/python/docs/source/_templates/autosummary/class_with_docs.rst", line 26, in top-level template code
        {% if '__init__' in methods %}
    jinja2.exceptions.UndefinedError: 'methods' is undefined
    The full traceback has been saved in /tmp/sphinx-err-ypgyi75y.log, if you want to report the issue to the developers.
    Please also report this if it was a user error, so that a better error message can be provided next time.
    A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
    make: *** [Makefile:20: html] Error 2
    
    re-running make html to print full warning list:
    Running Sphinx v3.0.4
    making output directory... done
    [autosummary] generating autosummary for: development/contributing.rst, development/debugging.rst, development/index.rst, development/setting_ide.rst, development/testing.rst, getting_started/index.rst, getting_started/install.rst, getting_started/quickstart.ipynb, index.rst, migration_guide/index.rst, ..., reference/pyspark.ml.rst, reference/pyspark.mllib.rst, reference/pyspark.resource.rst, reference/pyspark.rst, reference/pyspark.sql.rst, reference/pyspark.ss.rst, reference/pyspark [...]
    
    Exception occurred:
      File "/__w/spark/spark/python/docs/source/_templates/autosummary/class_with_docs.rst", line 26, in top-level template code
        {% if '__init__' in methods %}
    jinja2.exceptions.UndefinedError: 'methods' is undefined
    The full traceback has been saved in /tmp/sphinx-err-fvtmvvwv.log, if you want to report the issue to the developers.
    Please also report this if it was a user error, so that a better error message can be provided next time.
    A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
    make: *** [Makefile:20: html] Error 2
    Error: Process completed with exit code 2.
    ```
    
    ### Why are the changes needed?
    
    To recover GA build.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    GA.
    
    Closes #32509 from sarutak/fix-python-lint-error.
    
    Authored-by: Kousuke Saruta <sa...@oss.nttdata.com>
    Signed-off-by: Hyukjin Kwon <gu...@apache.org>
    (cherry picked from commit af0d99cce63a10aebd8104f12937e462f2873eac)
    Signed-off-by: Hyukjin Kwon <gu...@apache.org>
---
 .github/workflows/build_and_test.yml | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/.github/workflows/build_and_test.yml b/.github/workflows/build_and_test.yml
index b7e4e84..7b4f82d 100644
--- a/.github/workflows/build_and_test.yml
+++ b/.github/workflows/build_and_test.yml
@@ -332,7 +332,9 @@ jobs:
       run: |
         # TODO(SPARK-32407): Sphinx 3.1+ does not correctly index nested classes.
         #   See also https://github.com/sphinx-doc/sphinx/issues/7551.
-        python3.6 -m pip install flake8 'sphinx<3.1.0' numpy pydata_sphinx_theme ipython nbsphinx mypy numpydoc
+        # Jinja2 3.0.0+ causes error when building with Sphinx.
+        #   See also https://issues.apache.org/jira/browse/SPARK-35375.
+        python3.6 -m pip install flake8 'sphinx<3.1.0' numpy pydata_sphinx_theme ipython nbsphinx mypy numpydoc 'jinja2<3.0.0'
     - name: Install R linter dependencies and SparkR
       run: |
         apt-get install -y libcurl4-openssl-dev libgit2-dev libssl-dev libxml2-dev
@@ -345,7 +347,9 @@ jobs:
         apt-get install -y libcurl4-openssl-dev pandoc
         # TODO(SPARK-32407): Sphinx 3.1+ does not correctly index nested classes.
         #   See also https://github.com/sphinx-doc/sphinx/issues/7551.
-        python3.6 -m pip install 'sphinx<3.1.0' mkdocs numpy pydata_sphinx_theme ipython nbsphinx numpydoc
+        # Jinja2 3.0.0+ causes error when building with Sphinx.
+        #   See also https://issues.apache.org/jira/browse/SPARK-35375.
+        python3.6 -m pip install 'sphinx<3.1.0' mkdocs numpy pydata_sphinx_theme ipython nbsphinx numpydoc 'jinja2<3.0.0'
         apt-get update -y
         apt-get install -y ruby ruby-dev
         gem install jekyll jekyll-redirect-from rouge

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org