[jira] [Created] (IMPALA-12117) Intermittent pip error when building shell's venv

["Joe McDonnell (Jira)" <ji...@apache.org>]

Joe McDonnell created IMPALA-12117:
--------------------------------------

             Summary: Intermittent pip error when building shell's venv
                 Key: IMPALA-12117
                 URL: https://issues.apache.org/jira/browse/IMPALA-12117
             Project: IMPALA
          Issue Type: Bug
          Components: Infrastructure
    Affects Versions: Impala 4.3.0
            Reporter: Joe McDonnell


The all-build-options-ub2004 job intermittently hits this pip error when building the shell virtualenvs:

 
{noformat}
WARNING: Skipping page https://pypi.org/simple/thrift/ because the GET request got Content-Type: Unknown. The only supported Content-Types are application/vnd.pypi.simple.v1+json, application/vnd.pypi.simple.v1+html, and text/html
ERROR: Could not find a version that satisfies the requirement thrift==0.16.0 (from impala-shell) (from versions: none)
ERROR: No matching distribution found for thrift==0.16.0
{noformat}
This error doesn't always apply to thrift. It can be any of the shell dependencies. It doesn't seem to impact the impala-python or impala-python3 virtualenv (which don't use prebuilt binaries).

Searching around, a theory on the forums is that this can happen when there are multiple pip installs running simultaneously writing to the same pip cache. Impala builds a python 2 virtualenv and a python 3 virtualenv simultaneously, so that seems like a plausible theory.

We should try using separate pip caches for the python 2 vs python 3 shell virtualenvs.

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org