You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Michael Smith (Code Review)" <ge...@cloudera.org> on 2023/09/07 23:52:37 UTC
[native-toolchain-CR] [tools] Install aws to Python 3 when available
Michael Smith has uploaded this change for review. ( http://gerrit.cloudera.org:8080/20465
Change subject: [tools] Install aws to Python 3 when available
......................................................................
[tools] Install aws to Python 3 when available
I encountered failures building build containers for Ubuntu 22.04, and
CentOS 8 on ARM, when installing awscli. We should start moving to
Python 3 anyway, so this patch installs awscli to Python 3 when it
detects pip3 is available. Other platforms will continue to install pip
and use Python 2.7.
Change-Id: I4667181f9a6ef66c757623f1b128c3faa2057114
---
M docker/all/assert-dependencies-present.py
M docker/all/postinstall.sh
M docker/ubuntu2204.df
3 files changed, 8 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/native-toolchain refs/changes/65/20465/1
--
To view, visit http://gerrit.cloudera.org:8080/20465
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I4667181f9a6ef66c757623f1b128c3faa2057114
Gerrit-Change-Number: 20465
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
[native-toolchain-CR] IMPALA-12435: Install pinned pip
Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/20465 )
Change subject: IMPALA-12435: Install pinned pip
......................................................................
IMPALA-12435: Install pinned pip
Fixes installing the desired version of pip. Pulling a specific version
of the get-pip script is insufficient to install that version of pip.
Installing awscli started failing due to config deprecated in later
versions of pip.
Change-Id: I4667181f9a6ef66c757623f1b128c3faa2057114
Reviewed-on: http://gerrit.cloudera.org:8080/20465
Reviewed-by: Joe McDonnell <jo...@cloudera.com>
Tested-by: Michael Smith <mi...@cloudera.com>
---
M docker/all/assert-dependencies-present.py
M docker/all/postinstall.sh
2 files changed, 2 insertions(+), 2 deletions(-)
Approvals:
Joe McDonnell: Looks good to me, approved
Michael Smith: Verified
--
To view, visit http://gerrit.cloudera.org:8080/20465
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I4667181f9a6ef66c757623f1b128c3faa2057114
Gerrit-Change-Number: 20465
Gerrit-PatchSet: 4
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
[native-toolchain-CR] IMPALA-12435: Install pinned pip
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/20465 )
Change subject: IMPALA-12435: Install pinned pip
......................................................................
Patch Set 3:
> > Patch Set 3: Code-Review+2
> >
> > Thanks for fixing this
> >
> > Does this help with the ARM container builds?
>
> No, it runs into a different build failure in ARM containers.
> Planning to move to Python 3.8 for them.
Darn, well at least we fixed this piece.
--
To view, visit http://gerrit.cloudera.org:8080/20465
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4667181f9a6ef66c757623f1b128c3faa2057114
Gerrit-Change-Number: 20465
Gerrit-PatchSet: 3
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Fri, 08 Sep 2023 22:26:22 +0000
Gerrit-HasComments: No
[native-toolchain-CR] IMPALA-12435: Install pinned pip
Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/20465 )
Change subject: IMPALA-12435: Install pinned pip
......................................................................
Patch Set 3: Verified+1
All images but SLES 12 built successfully. Ran into an unrelated error with SLES 12 trying to pull opensuse.org repos.
--
To view, visit http://gerrit.cloudera.org:8080/20465
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4667181f9a6ef66c757623f1b128c3faa2057114
Gerrit-Change-Number: 20465
Gerrit-PatchSet: 3
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Fri, 08 Sep 2023 23:09:32 +0000
Gerrit-HasComments: No
[native-toolchain-CR] IMPALA-12435: Install pinned pip
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/20465 )
Change subject: IMPALA-12435: Install pinned pip
......................................................................
Patch Set 3: Code-Review+2
Thanks for fixing this
Does this help with the ARM container builds?
--
To view, visit http://gerrit.cloudera.org:8080/20465
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4667181f9a6ef66c757623f1b128c3faa2057114
Gerrit-Change-Number: 20465
Gerrit-PatchSet: 3
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Fri, 08 Sep 2023 22:19:57 +0000
Gerrit-HasComments: No
[native-toolchain-CR] IMPALA-12435: Install pinned pip
Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/20465 )
Change subject: IMPALA-12435: Install pinned pip
......................................................................
Patch Set 3:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/20465/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/20465/1//COMMIT_MSG@7
PS1, Line 7: IMPALA-12435: Install pinned pip
> Please file a JIRA and then also give a bit of info about the error seen. I
Done
http://gerrit.cloudera.org:8080/#/c/20465/1/docker/all/postinstall.sh
File docker/all/postinstall.sh:
http://gerrit.cloudera.org:8080/#/c/20465/1/docker/all/postinstall.sh@65
PS1, Line 65: dl_verify https://r
> This will not remain constant over time. Is there a specific version that w
There's no single version that works. I decided to take this change out for now.
http://gerrit.cloudera.org:8080/#/c/20465/1/docker/all/postinstall.sh@72
PS1, Line 72:
> I noticed that this command doesn't install 20.3.4 like I thought it did. O
Done
--
To view, visit http://gerrit.cloudera.org:8080/20465
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4667181f9a6ef66c757623f1b128c3faa2057114
Gerrit-Change-Number: 20465
Gerrit-PatchSet: 3
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Fri, 08 Sep 2023 18:52:51 +0000
Gerrit-HasComments: Yes
[native-toolchain-CR] IMPALA-12435: Install pinned pip
Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Hello Laszlo Gaal, Joe McDonnell,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20465
to look at the new patch set (#3).
Change subject: IMPALA-12435: Install pinned pip
......................................................................
IMPALA-12435: Install pinned pip
Fixes installing the desired version of pip. Pulling a specific version
of the get-pip script is insufficient to install that version of pip.
Installing awscli started failing due to config deprecated in later
versions of pip.
Change-Id: I4667181f9a6ef66c757623f1b128c3faa2057114
---
M docker/all/assert-dependencies-present.py
M docker/all/postinstall.sh
2 files changed, 2 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/native-toolchain refs/changes/65/20465/3
--
To view, visit http://gerrit.cloudera.org:8080/20465
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4667181f9a6ef66c757623f1b128c3faa2057114
Gerrit-Change-Number: 20465
Gerrit-PatchSet: 3
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
[native-toolchain-CR] IMPALA-12435: Install pinned pip, use pip3 when available
Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Hello Laszlo Gaal, Joe McDonnell,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20465
to look at the new patch set (#2).
Change subject: IMPALA-12435: Install pinned pip, use pip3 when available
......................................................................
IMPALA-12435: Install pinned pip, use pip3 when available
Fixes installing the desired version of pip. Pulling a specific version
of the get-pip script is insufficient to install that version of pip.
Installing awscli started failing due to config deprecated in later
versions of pip.
Also moves to installing awscli to Python 3 when it detects pip3 is
available. Other platforms will continue to install pip and use Python
2.7.
Change-Id: I4667181f9a6ef66c757623f1b128c3faa2057114
---
M docker/all/assert-dependencies-present.py
M docker/all/postinstall.sh
M docker/ubuntu2204.df
3 files changed, 9 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/native-toolchain refs/changes/65/20465/2
--
To view, visit http://gerrit.cloudera.org:8080/20465
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4667181f9a6ef66c757623f1b128c3faa2057114
Gerrit-Change-Number: 20465
Gerrit-PatchSet: 2
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
[native-toolchain-CR] IMPALA-12435: Install pinned pip
Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/20465 )
Change subject: IMPALA-12435: Install pinned pip
......................................................................
Patch Set 3:
> Patch Set 3: Code-Review+2
>
> Thanks for fixing this
>
> Does this help with the ARM container builds?
No, it runs into a different build failure in ARM containers. Planning to move to Python 3.8 for them.
--
To view, visit http://gerrit.cloudera.org:8080/20465
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4667181f9a6ef66c757623f1b128c3faa2057114
Gerrit-Change-Number: 20465
Gerrit-PatchSet: 3
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Fri, 08 Sep 2023 22:20:25 +0000
Gerrit-HasComments: No
[native-toolchain-CR] [tools] Install aws to Python 3 when available
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/20465 )
Change subject: [tools] Install aws to Python 3 when available
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/20465/1/docker/all/postinstall.sh
File docker/all/postinstall.sh:
http://gerrit.cloudera.org:8080/#/c/20465/1/docker/all/postinstall.sh@72
PS1, Line 72: python get-pip.py
I noticed that this command doesn't install 20.3.4 like I thought it did. On my box, changing this to force pip==20.3.4 makes the Ubuntu 22 image build on x86:
python get-pip.py "pip==20.3.4"
--
To view, visit http://gerrit.cloudera.org:8080/20465
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4667181f9a6ef66c757623f1b128c3faa2057114
Gerrit-Change-Number: 20465
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Fri, 08 Sep 2023 00:36:59 +0000
Gerrit-HasComments: Yes
[native-toolchain-CR] [tools] Install aws to Python 3 when available
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/20465 )
Change subject: [tools] Install aws to Python 3 when available
......................................................................
Patch Set 1:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/20465/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/20465/1//COMMIT_MSG@7
PS1, Line 7: [tools] Install aws to Python 3 when available
Please file a JIRA and then also give a bit of info about the error seen. It looks like Ubuntu 22 is failing on x86, so this is just a generic issue not specific to ARM.
http://gerrit.cloudera.org:8080/#/c/20465/1/docker/all/postinstall.sh
File docker/all/postinstall.sh:
http://gerrit.cloudera.org:8080/#/c/20465/1/docker/all/postinstall.sh@65
PS1, Line 65: pip3 install awscli
This will not remain constant over time. Is there a specific version that works?
--
To view, visit http://gerrit.cloudera.org:8080/20465
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4667181f9a6ef66c757623f1b128c3faa2057114
Gerrit-Change-Number: 20465
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Fri, 08 Sep 2023 00:26:28 +0000
Gerrit-HasComments: Yes