You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Riza Suminto (Code Review)" <ge...@cloudera.org> on 2025/08/13 17:36:53 UTC
[Impala-ASF-CR] WIP: workaround impala python3 make failure
Riza Suminto has uploaded this change for review. ( http://gerrit.cloudera.org:8080/23292
Change subject: WIP: workaround impala_python3 make failure
......................................................................
WIP: workaround impala_python3 make failure
Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
---
M infra/python/bootstrap_virtualenv.py
1 file changed, 2 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/23292/1
--
To view, visit http://gerrit.cloudera.org:8080/23292
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
Gerrit-Change-Number: 23292
Gerrit-PatchSet: 1
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
[Impala-ASF-CR] IMPALA-14308: Workaround failure in impala python3 build
Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Hello Laszlo Gaal, Jason Fehr, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23292
to look at the new patch set (#3).
Change subject: IMPALA-14308: Workaround failure in impala_python3 build
......................................................................
IMPALA-14308: Workaround failure in impala_python3 build
Construction of the impala-virtualenv fails since PyPI released version
7.0.0 of pbr. This blocks all precommit runs, since the Impala
virtualenv is required for all end-to-end tests.
The failure happen during pywebhdfs==0.3.2 installation. It is expected
to pullthe pinned version pbr==3.1.1, but the latest pbr==7.0.0 was
pulled instead. pbr==7.0.0 then broke with this error message:
ModuleNotFoundError: No module named 'packaging.requirements'
This patch adds workaround in bootstrap_virtualenv.py to install
packaging==24.1 early for python3. Installing it early managed to
unblock `make -j impala_python3`. packaging==24.1 package is already
listed in infra/python/deps/gcovr-requirements.txt, which installed in
later step and in python3 virtualenv only.
Testing:
Pass shell/ tests in Ubuntu 22.04 and Rocky 9.2.
Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
---
M infra/python/bootstrap_virtualenv.py
1 file changed, 2 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/23292/3
--
To view, visit http://gerrit.cloudera.org:8080/23292
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
Gerrit-Change-Number: 23292
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
[Impala-ASF-CR] IMPALA-14308: Workaround failure in impala python3 build
Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Jason Fehr has posted comments on this change. ( http://gerrit.cloudera.org:8080/23292 )
Change subject: IMPALA-14308: Workaround failure in impala_python3 build
......................................................................
Patch Set 3: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/23292
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
Gerrit-Change-Number: 23292
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Aug 2025 20:55:54 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14308: Workaround failure in impala python3 build
Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/23292 )
Change subject: IMPALA-14308: Workaround failure in impala_python3 build
......................................................................
Patch Set 4:
Thank you Laszlo and Jason!
--
To view, visit http://gerrit.cloudera.org:8080/23292
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
Gerrit-Change-Number: 23292
Gerrit-PatchSet: 4
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Aug 2025 20:57:24 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14308: Workaround failure in impala python3 build
Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/23292 )
Change subject: IMPALA-14308: Workaround failure in impala_python3 build
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/23292/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/23292/2//COMMIT_MSG@20
PS2, Line 20: Pass shell/ tests in Ubuntu 22.04 and Rocky 9.2.
Test still running
https://jenkins.impala.io/view/all/job/ubuntu-22.04-from-scratch/8/console
https://jenkins.impala.io/job/rocky-9.2-from-scratch/35/console
--
To view, visit http://gerrit.cloudera.org:8080/23292
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
Gerrit-Change-Number: 23292
Gerrit-PatchSet: 2
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Aug 2025 18:41:22 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] WIP: workaround impala python3 make failure
Posted by "Laszlo Gaal (Code Review)" <ge...@cloudera.org>.
Laszlo Gaal has posted comments on this change. ( http://gerrit.cloudera.org:8080/23292 )
Change subject: WIP: workaround impala_python3 make failure
......................................................................
Patch Set 1: Code-Review+1
Feels a bit like a kludge to me, but if it passes all tests, I'm OK with the fix: IMO it is more important to unblock builds than to come up with a perfect solution. Improvements can always be added later.
Would be great to have other eyes on it though.
--
To view, visit http://gerrit.cloudera.org:8080/23292
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
Gerrit-Change-Number: 23292
Gerrit-PatchSet: 1
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Aug 2025 18:26:39 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14308: Workaround failure in impala python3 build
Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/23292 )
Change subject: IMPALA-14308: Workaround failure in impala_python3 build
......................................................................
IMPALA-14308: Workaround failure in impala_python3 build
Construction of the impala-virtualenv fails since PyPI released version
7.0.0 of pbr. This blocks all precommit runs, since the Impala
virtualenv is required for all end-to-end tests.
The failure happen during pywebhdfs==0.3.2 installation. It is expected
to pullthe pinned version pbr==3.1.1, but the latest pbr==7.0.0 was
pulled instead. pbr==7.0.0 then broke with this error message:
ModuleNotFoundError: No module named 'packaging.requirements'
This patch adds workaround in bootstrap_virtualenv.py to install
packaging==24.1 early for python3. Installing it early managed to
unblock `make -j impala_python3`. packaging==24.1 package is already
listed in infra/python/deps/gcovr-requirements.txt, which installed in
later step and in python3 virtualenv only.
Testing:
Pass shell/ tests in Ubuntu 22.04 and Rocky 9.2.
Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
Reviewed-on: http://gerrit.cloudera.org:8080/23292
Reviewed-by: Jason Fehr <jf...@cloudera.com>
Tested-by: Jason Fehr <jf...@cloudera.com>
---
M infra/python/bootstrap_virtualenv.py
1 file changed, 2 insertions(+), 0 deletions(-)
Approvals:
Jason Fehr: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/23292
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
Gerrit-Change-Number: 23292
Gerrit-PatchSet: 4
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
[Impala-ASF-CR] IMPALA-14308: Workaround failure in impala python3 build
Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Hello Laszlo Gaal, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23292
to look at the new patch set (#2).
Change subject: IMPALA-14308: Workaround failure in impala_python3 build
......................................................................
IMPALA-14308: Workaround failure in impala_python3 build
Construction of the impala-virtualenv fails since PyPI released version
7.0.0 of pbr. This blocks all precommit runs, since the Impala
virtualenv is required for all end-to-end tests.
This patch adds workaround in bootstrap_virtualenv.py to install
packaging==24.1 early for python3. This package is listed in
infra/python/deps/gcovr-requirements.txt, which only installed in
python3 virtualenv. Installing it early managed to unblock
`make -j impala_python3`.
Testing:
Pass shell/ tests in Ubuntu 22.04 and Rocky 9.2.
Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
---
M infra/python/bootstrap_virtualenv.py
1 file changed, 2 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/23292/2
--
To view, visit http://gerrit.cloudera.org:8080/23292
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
Gerrit-Change-Number: 23292
Gerrit-PatchSet: 2
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
[Impala-ASF-CR] WIP: workaround impala python3 make failure
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23292 )
Change subject: WIP: workaround impala_python3 make failure
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/20147/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/23292
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
Gerrit-Change-Number: 23292
Gerrit-PatchSet: 1
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Aug 2025 18:03:51 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14308: Workaround failure in impala python3 build
Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Jason Fehr has posted comments on this change. ( http://gerrit.cloudera.org:8080/23292 )
Change subject: IMPALA-14308: Workaround failure in impala_python3 build
......................................................................
Patch Set 3: Code-Review+2
(1 comment)
http://gerrit.cloudera.org:8080/#/c/23292/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/23292/2//COMMIT_MSG@13
PS2, Line 13: The failure happen during pywebhdfs==0.3.2 installation. It is expected
> Done.
Thanks. I do not understand either why this workaround fixed the issue.
--
To view, visit http://gerrit.cloudera.org:8080/23292
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
Gerrit-Change-Number: 23292
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Aug 2025 20:55:13 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14308: Workaround failure in impala python3 build
Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Jason Fehr has posted comments on this change. ( http://gerrit.cloudera.org:8080/23292 )
Change subject: IMPALA-14308: Workaround failure in impala_python3 build
......................................................................
Patch Set 3:
Setting verified +1 since the gerrit-verify-dryrun build will fail until this patch is merged.
--
To view, visit http://gerrit.cloudera.org:8080/23292
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
Gerrit-Change-Number: 23292
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Aug 2025 20:56:50 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14308: Workaround failure in impala python3 build
Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Jason Fehr has posted comments on this change. ( http://gerrit.cloudera.org:8080/23292 )
Change subject: IMPALA-14308: Workaround failure in impala_python3 build
......................................................................
Patch Set 2: Code-Review+1
(1 comment)
http://gerrit.cloudera.org:8080/#/c/23292/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/23292/2//COMMIT_MSG@13
PS2, Line 13: This patch adds workaround in bootstrap_virtualenv.py to install
Can you explain more about why this workaround works?
--
To view, visit http://gerrit.cloudera.org:8080/23292
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
Gerrit-Change-Number: 23292
Gerrit-PatchSet: 2
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Aug 2025 20:07:40 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14308: Workaround failure in impala python3 build
Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/23292 )
Change subject: IMPALA-14308: Workaround failure in impala_python3 build
......................................................................
Patch Set 3:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/23292/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/23292/2//COMMIT_MSG@13
PS2, Line 13: The failure happen during pywebhdfs==0.3.2 installation. It is expected
> Can you explain more about why this workaround works?
Done.
This is the best explanation that I can make. Not 100% sure why it works though.
Maybe pbr==7.0.0 still being pulled, but having packaging==24.1 installed first allowed it pass.
I did `pip3 freeze` later on and see only pbr=3.1.1 installed in the virtualenv.
http://gerrit.cloudera.org:8080/#/c/23292/2//COMMIT_MSG@20
PS2, Line 20: packaging==24.1 early for python3. Installing it
> Test still running
Both has passed.
--
To view, visit http://gerrit.cloudera.org:8080/23292
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
Gerrit-Change-Number: 23292
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Aug 2025 20:22:43 +0000
Gerrit-HasComments: Yes