incubator-impala git commit: IMPALA-3774: fix download_requirements for older Python versions

[ta...@apache.org]

Repository: incubator-impala
Updated Branches:
  refs/heads/master 36b4f88bd -> a07021775


IMPALA-3774: fix download_requirements for older Python versions

Pip always runs the setup.py file in downloaded tarballs to get
metadata. Impyla's setup.py does not work in some older python
installations since find_packages() in setuptools does not support
the 'include' argument.

As a workaround, use our pip_download.py script to download Impyla
instead of pip.

Testing:
Confirmed that Jenkins build successfully downloaded the pip packages
and was able to bootstrap the virtualenv.

Change-Id: Id8801493c0f4caab2273383333ffbe2729b8339b
Reviewed-on: http://gerrit.cloudera.org:8080/3574
Reviewed-by: Tim Armstrong <ta...@cloudera.com>
Tested-by: Tim Armstrong <ta...@cloudera.com>


Project: http://git-wip-us.apache.org/repos/asf/incubator-impala/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-impala/commit/a0702177
Tree: http://git-wip-us.apache.org/repos/asf/incubator-impala/tree/a0702177
Diff: http://git-wip-us.apache.org/repos/asf/incubator-impala/diff/a0702177

Branch: refs/heads/master
Commit: a070217750d4006a41426e97f41d94c93489457e
Parents: 36b4f88
Author: Tim Armstrong <ta...@cloudera.com>
Authored: Tue Jul 5 14:02:36 2016 -0700
Committer: Tim Armstrong <ta...@cloudera.com>
Committed: Wed Jul 6 14:40:54 2016 -0700

----------------------------------------------------------------------
 infra/python/deps/download_requirements | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/a0702177/infra/python/deps/download_requirements
----------------------------------------------------------------------
diff --git a/infra/python/deps/download_requirements b/infra/python/deps/download_requirements
index c72eabb..eee7a1b 100755
--- a/infra/python/deps/download_requirements
+++ b/infra/python/deps/download_requirements
@@ -16,7 +16,9 @@ fi
 
 DIR=$(dirname "$0")
 # Download but don't install all packages listed in requirements.txt.
-"$PIP" -q install --download "$DIR" -r "$DIR"/requirements.txt
+# Don't use pip to download Impyla because its setup.py requires a newer version
+# of setuptools than is available on some systems we support (e.g. CentOS 6).
+"$PIP" -q install --download "$DIR" -r <(grep -v "impyla" "$DIR"/requirements.txt)
 
 # Directly download required packages not listed in requirements.txt.
 # For virtualenv, other scripts rely on the .tar.gz package (not a .whl package).
@@ -25,4 +27,6 @@ DIR=$(dirname "$0")
 pushd $DIR
 $PYTHON ./pip_download.py virtualenv 13.1.0
 $PYTHON ./pip_download.py kudu-python 0.1.1
+IMPYLA_VERSION=$(grep impyla ./requirements.txt | grep -o '[0-9.]*')
+$PYTHON ./pip_download.py impyla ${IMPYLA_VERSION}
 popd