You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2023/01/19 20:03:00 UTC
[jira] [Commented] (IMPALA-11846) impala-shell packaging is broken with python 3.8 and setuptools 66.0.0
[ https://issues.apache.org/jira/browse/IMPALA-11846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17678862#comment-17678862 ]
ASF subversion and git services commented on IMPALA-11846:
----------------------------------------------------------
Commit 1056e16a27c75d3bc8c645a75feb3ba5ece50ebc in impala's branch refs/heads/master from Gergely Fürnstáhl
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=1056e16a2 ]
IMPALA-11846: Fix builds with setuptools>=66.0.0
setuptools 66.0.0 introduced a breaking change, it does not support non
PEP440 compliant version names. This breaks impala_shell's packaging and
installing test if the system python3's version is 3.8+.
This is a quick fix to unblock builds. The rest of the work will be done
in IMPALA-11849 (e.g. stabilizing the python environments version).
impala_shell releases should not be affected by this, as the version
number we generate is already PEP440 compliant.
Testing:
- Built locally with python3.8
Change-Id: I4eb0957fb576e590b86b6fe570216cfb72d11aef
Reviewed-on: http://gerrit.cloudera.org:8080/19431
Reviewed-by: Joe McDonnell <jo...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> impala-shell packaging is broken with python 3.8 and setuptools 66.0.0
> ----------------------------------------------------------------------
>
> Key: IMPALA-11846
> URL: https://issues.apache.org/jira/browse/IMPALA-11846
> Project: IMPALA
> Issue Type: Bug
> Reporter: Gergely Fürnstáhl
> Assignee: Gergely Fürnstáhl
> Priority: Major
>
> setuptools introduced a breaking change with 66.0.0 release affecting impala build:
> [https://setuptools.pypa.io/en/stable/history.html#breaking-changes]
> It affects building with python 3.8. setuptools package is part of the build system and installed with python and pip during an earlier step, the version number cannot be specified in the setup.py file.
> {code:java}
> Failed to build impala-shell
> Installing collected packages: bitarray, configparser, kerberos, prettytable, six, sasl, sqlparse, thrift, pure-sasl, thrift-sasl, impala-shell
> Attempting uninstall: thrift
> Found existing installation: thrift 0.11.0
> Not uninstalling thrift at /home/gfurnstahl/Impala/toolchain/toolchain-packages-gcc10.4.0/thrift-0.11.0-p5/python/lib/python2.7/site-packages, outside environment /home/gfurnstahl/Impala/shell/build/py2_venv
> Can't uninstall 'thrift'. No files were found to uninstall.
> Preparing metadata (setup.py): finished with status 'error'
> error: subprocess-exited-with-error
>
> × python setup.py egg_info did not run successfully.
> │ exit code: 1
> ╰─> [27 lines of output]
> running egg_info
> /home/gfurnstahl/Impala/shell/build/py3_venv/lib/python3.8/site-packages/setuptools/dist.py:543: UserWarning: The version specified ('install-test') is an invalid version, this may not work as expected with newer versions of setuptools, pip, and PyPI. Please see PEP 440 for more details.
> warnings.warn(
> Traceback (most recent call last):
> File "<string>", line 2, in <module>
> File "<pip-setuptools-caller>", line 34, in <module>
> File "/tmp/pip-req-build-qi4d4ya9/setup.py", line 136, in <module>
> setup(
> File "/home/gfurnstahl/Impala/shell/build/py3_venv/lib/python3.8/site-packages/setuptools/__init__.py", line 87, in setup
> return distutils.core.setup(**attrs)
> File "/home/gfurnstahl/Impala/shell/build/py3_venv/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 185, in setup
> return run_commands(dist)
> File "/home/gfurnstahl/Impala/shell/build/py3_venv/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
> dist.run_commands()
> File "/home/gfurnstahl/Impala/shell/build/py3_venv/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
> self.run_command(cmd)
> File "/home/gfurnstahl/Impala/shell/build/py3_venv/lib/python3.8/site-packages/setuptools/dist.py", line 1208, in run_command
> super().run_command(command)
> File "/home/gfurnstahl/Impala/shell/build/py3_venv/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
> cmd_obj.ensure_finalized()
> File "/home/gfurnstahl/Impala/shell/build/py3_venv/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 111, in ensure_finalized
> self.finalize_options()
> File "/home/gfurnstahl/Impala/shell/build/py3_venv/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 219, in finalize_options
> parsed_version = parse_version(self.egg_version)
> File "/home/gfurnstahl/Impala/shell/build/py3_venv/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/version.py", line 266, in __init__
> raise InvalidVersion(f"Invalid version: '{version}'")
> pkg_resources.extern.packaging.version.InvalidVersion: Invalid version: 'install-test'
> [end of output]
>
> note: This error originates from a subprocess, and is likely not a problem with pip.
> error: metadata-generation-failed
> × Encountered error while generating package metadata.
> ╰─> See above for output. {code}
> Jenkins jobs does not seem to be affected due to using python 3.5 and installing older version of setuptools.
> {code:java}
> 19:13:52 DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality.
> 19:13:52 Requirement already satisfied: setuptools>=36.8.0 in ./build/py3_venv/lib/python3.5/site-packages (from impala-shell===install-test) (50.3.2) {code}
> A solution could be to use PEP440 compilant name for the test package (e.g. replace install-test with 1.0.0.dev1), or extend the packaging using PEP518 and pyproject.toml to specify setuptools version:
> [https://peps.python.org/pep-0518/]
> Followup Jira to track improvements in the stability of python env:
> https://issues.apache.org/jira/browse/IMPALA-11849
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org