[kudu] 02/02: build-and-test.sh: fix Python 2.7 build

[ad...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

adar pushed a commit to branch branch-1.9.x
in repository https://gitbox.apache.org/repos/asf/kudu.git

commit ceb096453194f30aa8901600573d6aaa78c345eb
Author: Adar Dembo <ad...@cloudera.com>
AuthorDate: Tue Dec 10 13:57:49 2019 -0800

    build-and-test.sh: fix Python 2.7 build
    
    When building with Python 2.7, the new setuptools 42.0.0 breaks the
    kudu-python build with an error like this:
    
      [1/3] Cythonizing kudu/client.pyx
      [2/3] Cythonizing kudu/errors.pyx
      [3/3] Cythonizing kudu/schema.pyx
      WARNING: The wheel package is not available.
      ERROR: 'pip wheel' requires the 'wheel' package. To fix this, run: pip install wheel
      Traceback (most recent call last):
      File "setup.py", line 216, in <module>
        test_suite="kudu.tests"
      File "/home/jenkins-slave/workspace/kudu-master/3/build/debug/py_env/local/lib/python2.7/site-packages/setuptools/__init__.py", line 144, in setup
        _install_setup_requires(attrs)
      File "/home/jenkins-slave/workspace/kudu-master/3/build/debug/py_env/local/lib/python2.7/site-packages/setuptools/__init__.py", line 139, in _install_setup_requires
        dist.fetch_build_eggs(dist.setup_requires)
      File "/home/jenkins-slave/workspace/kudu-master/3/build/debug/py_env/local/lib/python2.7/site-packages/setuptools/dist.py", line 721, in fetch_build_eggs
        replace_conflicting=True,
      File "/home/jenkins-slave/workspace/kudu-master/3/build/debug/py_env/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 782, in resolve
        replace_conflicting=replace_conflicting
      File "/home/jenkins-slave/workspace/kudu-master/3/build/debug/py_env/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1065, in best_match
        return self.obtain(req, installer)
      File "/home/jenkins-slave/workspace/kudu-master/3/build/debug/py_env/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1077, in obtain
        return installer(requirement)
      File "/home/jenkins-slave/workspace/kudu-master/3/build/debug/py_env/local/lib/python2.7/site-packages/setuptools/dist.py", line 777, in fetch_build_egg
        return fetch_build_egg(self, req)
      File "/home/jenkins-slave/workspace/kudu-master/3/build/debug/py_env/local/lib/python2.7/site-packages/setuptools/installer.py", line 130, in fetch_build_egg
        raise DistutilsError(str(e))
      distutils.errors.DistutilsError: Command '['/home/jenkins-slave/workspace/kudu-master/3/build/debug/py_env/bin/python', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpK8eEW3', '--quiet', 'pytest-runner']' returned non-zero exit status 1
    
    An easy fix is to pin to an earlier version of setuptools. I don't know why
    this doesn't happen on Python 3.
    
    Change-Id: Ia37e164f0319236265165748de47e414a057338f
    Reviewed-on: http://gerrit.cloudera.org:8080/14875
    Reviewed-by: Andrew Wong <aw...@cloudera.com>
    Reviewed-by: Bankim Bhavsar <ba...@cloudera.com>
    Reviewed-by: Alexey Serbin <as...@cloudera.com>
    Tested-by: Adar Dembo <ad...@cloudera.com>
    (cherry picked from commit 6ac9191c731d6955a97f08d4add02a96198f6fbf)
    Reviewed-on: http://gerrit.cloudera.org:8080/14887
    Reviewed-by: Adar Dembo <ad...@cloudera.com>
---
 build-support/jenkins/build-and-test.sh | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)

diff --git a/build-support/jenkins/build-and-test.sh b/build-support/jenkins/build-and-test.sh
index b6e6b01..6832352 100755
--- a/build-support/jenkins/build-and-test.sh
+++ b/build-support/jenkins/build-and-test.sh
@@ -444,13 +444,16 @@ if [ "$BUILD_PYTHON" == "1" ]; then
   #
   # The absence of $PIP_FLAGS is intentional: older versions of pip may not
   # support the flags that we want to use.
-  pip install -i https://pypi.python.org/simple $PIP_INSTALL_FLAGS --upgrade 'pip < 10.0.0b1'
+  pip install -i https://pypi.python.org/simple $PIP_INSTALL_FLAGS --upgrade 'pip <10.0.0b1'
 
   # New versions of pip raise an exception when upgrading old versions of
   # setuptools (such as the one found in el6). The workaround is to upgrade
   # setuptools on its own, outside of requirements.txt, and with the pip version
   # check disabled.
-  pip $PIP_FLAGS install --disable-pip-version-check $PIP_INSTALL_FLAGS --upgrade 'setuptools >= 0.8'
+  #
+  # Setuptools 42.0.0 changes something that causes build_ext to fail with a
+  # missing wheel package. Let's pin to an older version known to work.
+  pip $PIP_FLAGS install --disable-pip-version-check $PIP_INSTALL_FLAGS --upgrade 'setuptools >=0.8,<42.0.0'
 
   # One of our dependencies is pandas, installed below. It depends on numpy, and
   # if we don't install numpy directly, the pandas installation will install the
@@ -461,7 +464,7 @@ if [ "$BUILD_PYTHON" == "1" ]; then
   # so we must pass in the current values of CC and CXX.
   #
   # See https://github.com/numpy/numpy/releases/tag/v1.12.0 for more details.
-  CC=$CLANG CXX=$CLANG++ pip $PIP_FLAGS install $PIP_INSTALL_FLAGS 'numpy < 1.12.0'
+  CC=$CLANG CXX=$CLANG++ pip $PIP_FLAGS install $PIP_INSTALL_FLAGS 'numpy <1.12.0'
 
   # We've got a new pip and new setuptools. We can now install the rest of the
   # Python client's requirements.
@@ -478,7 +481,7 @@ if [ "$BUILD_PYTHON" == "1" ]; then
   #
   # pandas 0.18 dropped support for python 2.6. See https://pandas.pydata.org/pandas-docs/version/0.23.0/whatsnew.html#v0-18-0-march-13-2016
   # for more details.
-  CC=$CLANG CXX=$CLANG++ pip $PIP_FLAGS install $PIP_INSTALL_FLAGS 'pandas < 0.18'
+  CC=$CLANG CXX=$CLANG++ pip $PIP_FLAGS install $PIP_INSTALL_FLAGS 'pandas <0.18'
 
   # Delete old Cython extensions to force them to be rebuilt.
   rm -Rf build kudu_python.egg-info kudu/*.so
@@ -533,13 +536,13 @@ if [ "$BUILD_PYTHON3" == "1" ]; then
   # support the flags that we want to use.
   #
   # 1. https://github.com/pypa/pip/issues/6175
-  pip install -i https://pypi.python.org/simple $PIP_INSTALL_FLAGS --upgrade 'pip < 19.0'
+  pip install -i https://pypi.python.org/simple $PIP_INSTALL_FLAGS --upgrade 'pip <19.0'
 
   # New versions of pip raise an exception when upgrading old versions of
   # setuptools (such as the one found in el6). The workaround is to upgrade
   # setuptools on its own, outside of requirements.txt, and with the pip version
   # check disabled.
-  pip $PIP_FLAGS install --disable-pip-version-check $PIP_INSTALL_FLAGS --upgrade 'setuptools >= 0.8'
+  pip $PIP_FLAGS install --disable-pip-version-check $PIP_INSTALL_FLAGS --upgrade 'setuptools >=0.8'
 
   # One of our dependencies is pandas, installed below. It depends on numpy, and
   # if we don't install numpy directly, the pandas installation will install the
@@ -551,7 +554,7 @@ if [ "$BUILD_PYTHON3" == "1" ]; then
   # so we must pass in the current values of CC and CXX.
   #
   # See https://github.com/numpy/numpy/releases/tag/v1.16.0rc1 for more details.
-  CC=$CLANG CXX=$CLANG++ pip $PIP_FLAGS install $PIP_INSTALL_FLAGS 'numpy < 1.16.0'
+  CC=$CLANG CXX=$CLANG++ pip $PIP_FLAGS install $PIP_INSTALL_FLAGS 'numpy <1.16.0'
 
   # We've got a new pip and new setuptools. We can now install the rest of the
   # Python client's requirements.