You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Joe McDonnell (Jira)" <ji...@apache.org> on 2023/06/21 00:45:00 UTC
[jira] [Commented] (IMPALA-12203) Use python = python 3 for modern toolchain docker images
[ https://issues.apache.org/jira/browse/IMPALA-12203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17735493#comment-17735493 ]
Joe McDonnell commented on IMPALA-12203:
----------------------------------------
Addressed in [https://github.com/cloudera/native-toolchain/commit/e22dd49de95c4b35994eeba021296b7621571776] :
{noformat}
commit e22dd49de95c4b35994eeba021296b7621571776
Author: Joe McDonnell <jo...@cloudera.com>
Date: Wed Jun 7 16:37:45 2023 -0700 IMPALA-12203: Default to Python 3 on modern distributions
This modifies the postinstall.sh script to set python to
point to python 3 for Linux distributions that do not
offer python 2. This will apply to Redhat 9 and Ubuntu 22
when support is added, but it does not apply to any
older distributions. The docker images can be used for
building older native-toolchain versions, so none of
the older distributions can be modified to use python 3.
To support this, this cleans up some old python logic
in a few packages. LLVM and Kudu can use system python
without any issues, and they both support python 3 for
this use case.
This removes some obsolete code from assert-dependencies-present.py
that used the Python 2 'platform' package. These checks
do not apply to any supported distributions.
This also changes postinstall.sh to use the last versions
of pip (20.3.4) and awscli (1.19.112) that support Python 2.
These are recent enough to also support newer Python 3 like
Python 3.10.
Testing:
- Built Docker images and ran the build
- Also ran the build for several older native-toolchain commits
with the new Docker images to make sure that they
continue to work.
Change-Id: Ib10589927509eeaa44b628709dcdb5e45c4dd034
Reviewed-on: http://gerrit.cloudera.org:8080/20063
Reviewed-by: Michael Smith <mi...@cloudera.com>
Reviewed-by: Laszlo Gaal <la...@cloudera.com>
Tested-by: Joe McDonnell <jo...@cloudera.com>
{noformat}
> Use python = python 3 for modern toolchain docker images
> --------------------------------------------------------
>
> Key: IMPALA-12203
> URL: https://issues.apache.org/jira/browse/IMPALA-12203
> Project: IMPALA
> Issue Type: Improvement
> Components: Infrastructure
> Affects Versions: Impala 4.3.0
> Reporter: Joe McDonnell
> Assignee: Joe McDonnell
> Priority: Major
>
> Newer Linux distributions like Redhat 9 no longer include Python 2 in the default repositories. In general, Python 2 is dead and the toolchain code should be able to operate on distributions without Python 2.
> Specifically, we would set python to point to python 3 and fix any issues that come up. The issues known so far:
> * The version of pip that we install in the docker images is too old to work appropriately with recent Python 3. It would need to be upgraded to 20.3.4 (the last version that also supports Python 2.7).
> * The version of awscli that we install doesn't work with Python 3.10 (used on Ubuntu 22). This will need to be upgraded to 1.19.112, which is the last version to support Python 2.7. This is new enough to work with Python 3.10.
> For bonus points, this logic can be extended to other modern distributions like Ubuntu 20, SLES 15, and Redhat 8. There is no need to extend this to legacy distributions like Redhat 7, SLES 12, or Ubuntu 16/18.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org