You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@bigtop.apache.org by "Kengo Seki (Jira)" <ji...@apache.org> on 2020/07/20 02:13:00 UTC
[jira] [Updated] (BIGTOP-3375) Bump Livy to 0.7.0 and build it with
Python3
[ https://issues.apache.org/jira/browse/BIGTOP-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kengo Seki updated BIGTOP-3375:
-------------------------------
Summary: Bump Livy to 0.7.0 and build it with Python3 (was: Building Livy fails due to the lack of some Python2 libraries)
> Bump Livy to 0.7.0 and build it with Python3
> --------------------------------------------
>
> Key: BIGTOP-3375
> URL: https://issues.apache.org/jira/browse/BIGTOP-3375
> Project: Bigtop
> Issue Type: Bug
> Reporter: Kengo Seki
> Assignee: Kengo Seki
> Priority: Major
>
> We changed the way to install flake8 and setuptools to use Python3 instead of Python2 in BIGTOP-3364, but I just noticed Livy 0.6.0 only supports Python 2.x, so running {{./gradlew livy-pkg-ind}} fails as follows.
> {code:java}
> $ ./gradlew livy-clean livy-pkg-ind
> (snip)
> Installed /bigtop/build/livy/rpm/BUILD/livy-0.6.0/python-api/UNKNOWN-0.0.0-py2.7.egg
> warning: no previously-included files matching '*.pyc' found anywhere in distribution
> no previously-included directories found matching 'docs/build/'
> warning: install_lib: 'build/lib' does not exist -- no Python modules to install
> zip_safe flag not set; analyzing archive contents...
> Traceback (most recent call last):
> File "setup.py", line 57, in <module>
> tests_require=['pytest']
> File "/usr/lib64/python2.7/distutils/core.py", line 112, in setup
> _setup_distribution = dist = klass(attrs)
> File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 265, in __init__
> self.fetch_build_eggs(attrs.pop('setup_requires'))
> File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 289, in fetch_build_eggs
> parse_requirements(requires), installer=self.fetch_build_egg
> File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 626, in resolve
> raise DistributionNotFound(req)
> pkg_resources.DistributionNotFound: flake8
> (snip)
> [INFO] livy-examples ...................................... SUCCESS [ 9.374 s]
> [INFO] livy-python-api 0.6.0-incubating ................... FAILURE [ 2.545 s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 10:10 min
> [INFO] Finished at: 2020-07-08T23:20:10Z
> [INFO] ------------------------------------------------------------------------
> [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:exec (python-api install) on project livy-python-api: Command execution failed.: Process exited with an error: 1 (Exit value: 1) -> [Help 1]
> {code}
> I tried to install the python-flake8 package to avoid this, but I got another error:
> {code:java}
> [INFO] --- exec-maven-plugin:1.2.1:exec (python-api install) @ livy-python-api ---
> Traceback (most recent call last):
> File "setup.py", line 57, in <module>
> tests_require=['pytest']
> File "/usr/lib64/python2.7/distutils/core.py", line 112, in setup
> _setup_distribution = dist = klass(attrs)
> File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 265, in __init__
> self.fetch_build_eggs(attrs.pop('setup_requires'))
> File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 289, in fetch_build_eggs
> parse_requirements(requires), installer=self.fetch_build_egg
> File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 618, in resolve
> dist = best[req.key] = env.best_match(req, self, installer)
> File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 862, in best_match
> return self.obtain(req, installer) # try and download/install
> File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 874, in obtain
> return installer(requirement)
> File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 339, in fetch_build_egg
> return cmd.easy_install(req)
> File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 623, in easy_install
> return self.install_item(spec, dist.location, tmpdir, deps)
> File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 653, in install_item
> dists = self.install_eggs(spec, download, tmpdir)
> File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 849, in install_eggs
> return self.build_and_install(setup_script, setup_base)
> File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1130, in build_and_install
> self.run_setup(setup_script, setup_base, args)
> File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1115, in run_setup
> run_setup(setup_script, args)
> File "/usr/lib/python2.7/site-packages/setuptools/sandbox.py", line 69, in run_setup
> lambda: execfile(
> File "/usr/lib/python2.7/site-packages/setuptools/sandbox.py", line 120, in run
> return func()
> File "/usr/lib/python2.7/site-packages/setuptools/sandbox.py", line 71, in <lambda>
> {'__file__':setup_script, '__name__':'__main__'}
> File "setup.py", line 21, in <module>
>
> File "/usr/lib64/python2.7/distutils/core.py", line 112, in setup
> _setup_distribution = dist = klass(attrs)
> File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 269, in __init__
> _Distribution.__init__(self,attrs)
> File "/usr/lib64/python2.7/distutils/dist.py", line 287, in __init__
> self.finalize_options()
> File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 302, in finalize_options
> ep.load()(self, ep.name, value)
> File "build/bdist.linux-x86_64/egg/setuptools_scm/integration.py", line 9, in version_keyword
> File "build/bdist.linux-x86_64/egg/setuptools_scm/version.py", line 60, in _warn_if_setuptools_outdated
> setuptools_scm.version.SetuptoolsOutdatedWarning: your setuptools is too old (<12)
> (snip)
> [INFO] livy-examples ...................................... SUCCESS [ 4.536 s]
> [INFO] livy-python-api 0.6.0-incubating ................... FAILURE [ 0.887 s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 01:21 min
> [INFO] Finished at: 2020-07-09T22:57:06Z
> [INFO] ------------------------------------------------------------------------
> [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:exec (python-api install) on project livy-python-api: Command execution failed.: Process exited with an error: 1 (Exit value: 1) -> [Help 1]
> {code}
> CentOS 7 provides the python-setuptools package, but its version is 0.9.8. So we may have to install setuptools as a Python2 library via pip2, as we did before.
> As Yuqi mentioned before, the latest version of setuptools has already dropped Python2 support, but prior to v45.0.0 seems to work with Python2.
> [https://setuptools.readthedocs.io/en/latest/history.html#v45-0-0]
> So I'll try to revert BIGTOP-3364 and pin setuptools' version to 44.0.0, and check if both Zeppelin and Livy are successfully built on all distros.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)