You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Joe McDonnell (Code Review)" <ge...@cloudera.org> on 2023/05/06 21:32:57 UTC

[Impala-ASF-CR] IMPALA-12117: Use separate cache dirs for shell pip installs

Hello Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/19813

to look at the new patch set (#2).

Change subject: IMPALA-12117: Use separate cache dirs for shell pip installs
......................................................................

IMPALA-12117: Use separate cache dirs for shell pip installs

Pip sporadically hits an error when installing impala-shell into
a virtualenv. An example symptom is this (though the issue is
not specific to thrift):
WARNING: Skipping page https://pypi.org/simple/thrift/ because the
   GET request got Content-Type: Unknown. The only supported
   Content-Types are application/vnd.pypi.simple.v1+json,
   application/vnd.pypi.simple.v1+html, and text/html
ERROR: Could not find a version that satisfies the requirement
   thrift==0.16.0 (from impala-shell) (from versions: none)
ERROR: No matching distribution found for thrift==0.16.0

It appears that this error can occur when two pip processes
are installing into virtualenvs simultaneously and share a
cache directory. This happens for our impala-shell build,
because we are doing pip install for Python 2 and Python 3
simultaneously. The impala-python/impala-python3 virtualenvs
do not use a cache directory and are not impacted.

This changes the shell's pip install to give the Python 2 and
Python 3 separate cache directories. The cache directories are
placed in ~/.cache like the regular pip cache. These do not
consume much space (a couple MB).

Testing:
 - Ran all-build-options-ub2004 ten times without seeing the failure

Change-Id: I3f834b9f8c8cbc09830745ad132677a2fe17e07b
---
M shell/CMakeLists.txt
1 file changed, 7 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/13/19813/2
-- 
To view, visit http://gerrit.cloudera.org:8080/19813
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3f834b9f8c8cbc09830745ad132677a2fe17e07b
Gerrit-Change-Number: 19813
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>