You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@bigtop.apache.org by "Kengo Seki (Jira)" <ji...@apache.org> on 2020/07/10 00:07:00 UTC
[jira] [Created] (BIGTOP-3375) Building Livy fails due to the lack
of some Python2 libraries
Kengo Seki created BIGTOP-3375:
----------------------------------
Summary: Building Livy fails due to the lack of some Python2 libraries
Key: BIGTOP-3375
URL: https://issues.apache.org/jira/browse/BIGTOP-3375
Project: Bigtop
Issue Type: Bug
Reporter: Kengo Seki
Assignee: Kengo Seki
We changed the way to install flake8 and setuptools to use Python3 instead of Python2 in BIGTOP-3364, but I just noticed Livy 0.6.0 only supports Python 2.x, so running {{./gradlew livy-pkg-ind}} fails as follows.
{code:java}
$ ./gradlew livy-clean livy-pkg-ind
(snip)
Installed /bigtop/build/livy/rpm/BUILD/livy-0.6.0/python-api/UNKNOWN-0.0.0-py2.7.egg
warning: no previously-included files matching '*.pyc' found anywhere in distribution
no previously-included directories found matching 'docs/build/'
warning: install_lib: 'build/lib' does not exist -- no Python modules to install
zip_safe flag not set; analyzing archive contents...
Traceback (most recent call last):
File "setup.py", line 57, in <module>
tests_require=['pytest']
File "/usr/lib64/python2.7/distutils/core.py", line 112, in setup
_setup_distribution = dist = klass(attrs)
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 265, in __init__
self.fetch_build_eggs(attrs.pop('setup_requires'))
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 289, in fetch_build_eggs
parse_requirements(requires), installer=self.fetch_build_egg
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 626, in resolve
raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: flake8
(snip)
[INFO] livy-examples ...................................... SUCCESS [ 9.374 s]
[INFO] livy-python-api 0.6.0-incubating ................... FAILURE [ 2.545 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 10:10 min
[INFO] Finished at: 2020-07-08T23:20:10Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:exec (python-api install) on project livy-python-api: Command execution failed.: Process exited with an error: 1 (Exit value: 1) -> [Help 1]
{code}
I tried to install the python-flake8 package to avoid this, but I got another error:
{code:java}
[INFO] --- exec-maven-plugin:1.2.1:exec (python-api install) @ livy-python-api ---
Traceback (most recent call last):
File "setup.py", line 57, in <module>
tests_require=['pytest']
File "/usr/lib64/python2.7/distutils/core.py", line 112, in setup
_setup_distribution = dist = klass(attrs)
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 265, in __init__
self.fetch_build_eggs(attrs.pop('setup_requires'))
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 289, in fetch_build_eggs
parse_requirements(requires), installer=self.fetch_build_egg
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 618, in resolve
dist = best[req.key] = env.best_match(req, self, installer)
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 862, in best_match
return self.obtain(req, installer) # try and download/install
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 874, in obtain
return installer(requirement)
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 339, in fetch_build_egg
return cmd.easy_install(req)
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 623, in easy_install
return self.install_item(spec, dist.location, tmpdir, deps)
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 653, in install_item
dists = self.install_eggs(spec, download, tmpdir)
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 849, in install_eggs
return self.build_and_install(setup_script, setup_base)
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1130, in build_and_install
self.run_setup(setup_script, setup_base, args)
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1115, in run_setup
run_setup(setup_script, args)
File "/usr/lib/python2.7/site-packages/setuptools/sandbox.py", line 69, in run_setup
lambda: execfile(
File "/usr/lib/python2.7/site-packages/setuptools/sandbox.py", line 120, in run
return func()
File "/usr/lib/python2.7/site-packages/setuptools/sandbox.py", line 71, in <lambda>
{'__file__':setup_script, '__name__':'__main__'}
File "setup.py", line 21, in <module>
File "/usr/lib64/python2.7/distutils/core.py", line 112, in setup
_setup_distribution = dist = klass(attrs)
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 269, in __init__
_Distribution.__init__(self,attrs)
File "/usr/lib64/python2.7/distutils/dist.py", line 287, in __init__
self.finalize_options()
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 302, in finalize_options
ep.load()(self, ep.name, value)
File "build/bdist.linux-x86_64/egg/setuptools_scm/integration.py", line 9, in version_keyword
File "build/bdist.linux-x86_64/egg/setuptools_scm/version.py", line 60, in _warn_if_setuptools_outdated
setuptools_scm.version.SetuptoolsOutdatedWarning: your setuptools is too old (<12)
(snip)
[INFO] livy-examples ...................................... SUCCESS [ 4.536 s]
[INFO] livy-python-api 0.6.0-incubating ................... FAILURE [ 0.887 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:21 min
[INFO] Finished at: 2020-07-09T22:57:06Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:exec (python-api install) on project livy-python-api: Command execution failed.: Process exited with an error: 1 (Exit value: 1) -> [Help 1]
{code}
CentOS 7 provides the python-setuptools package, but its version is 0.9.8. So we may have to install setuptools as a Python2 library via pip2, as we did before.
As Yuqi mentioned before, the latest version of setuptools has already dropped Python2 support, but prior to v45.0.0 seems to work with Python2.
[https://setuptools.readthedocs.io/en/latest/history.html#v45-0-0]
So I'll try to revert BIGTOP-3364 and pin setuptools' version to 44.0.0, and check if both Zeppelin and Livy are successfully built on all distros.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)