You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Riza Suminto (Code Review)" <ge...@cloudera.org> on 2025/08/13 17:36:53 UTC

[Impala-ASF-CR] WIP: workaround impala python3 make failure

Riza Suminto has uploaded this change for review. ( http://gerrit.cloudera.org:8080/23292


Change subject: WIP: workaround impala_python3 make failure
......................................................................

WIP: workaround impala_python3 make failure

Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
---
M infra/python/bootstrap_virtualenv.py
1 file changed, 2 insertions(+), 0 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/23292/1
-- 
To view, visit http://gerrit.cloudera.org:8080/23292
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
Gerrit-Change-Number: 23292
Gerrit-PatchSet: 1
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>

[Impala-ASF-CR] IMPALA-14308: Workaround failure in impala python3 build

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Hello Laszlo Gaal, Jason Fehr, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/23292

to look at the new patch set (#3).

Change subject: IMPALA-14308: Workaround failure in impala_python3 build
......................................................................

IMPALA-14308: Workaround failure in impala_python3 build

Construction of the impala-virtualenv fails since PyPI released version
7.0.0 of pbr. This blocks all precommit runs, since the Impala
virtualenv is required for all end-to-end tests.

The failure happen during pywebhdfs==0.3.2 installation. It is expected
to pullthe pinned version pbr==3.1.1, but the latest pbr==7.0.0 was
pulled instead. pbr==7.0.0 then broke with this error message:

  ModuleNotFoundError: No module named 'packaging.requirements'

This patch adds workaround in bootstrap_virtualenv.py to install
packaging==24.1 early for python3. Installing it early managed to
unblock `make -j impala_python3`. packaging==24.1 package is already
listed in infra/python/deps/gcovr-requirements.txt, which installed in
later step and in python3 virtualenv only.

Testing:
Pass shell/ tests in Ubuntu 22.04 and Rocky 9.2.

Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
---
M infra/python/bootstrap_virtualenv.py
1 file changed, 2 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/23292/3
-- 
To view, visit http://gerrit.cloudera.org:8080/23292
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
Gerrit-Change-Number: 23292
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>

[Impala-ASF-CR] IMPALA-14308: Workaround failure in impala python3 build

Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Jason Fehr has posted comments on this change. ( http://gerrit.cloudera.org:8080/23292 )

Change subject: IMPALA-14308: Workaround failure in impala_python3 build
......................................................................


Patch Set 3: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/23292
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
Gerrit-Change-Number: 23292
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Aug 2025 20:55:54 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14308: Workaround failure in impala python3 build

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/23292 )

Change subject: IMPALA-14308: Workaround failure in impala_python3 build
......................................................................


Patch Set 4:

Thank you Laszlo and Jason!


-- 
To view, visit http://gerrit.cloudera.org:8080/23292
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
Gerrit-Change-Number: 23292
Gerrit-PatchSet: 4
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Aug 2025 20:57:24 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14308: Workaround failure in impala python3 build

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/23292 )

Change subject: IMPALA-14308: Workaround failure in impala_python3 build
......................................................................


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/23292/2//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/23292/2//COMMIT_MSG@20
PS2, Line 20: Pass shell/ tests in Ubuntu 22.04 and Rocky 9.2.
Test still running
https://jenkins.impala.io/view/all/job/ubuntu-22.04-from-scratch/8/console
https://jenkins.impala.io/job/rocky-9.2-from-scratch/35/console



-- 
To view, visit http://gerrit.cloudera.org:8080/23292
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
Gerrit-Change-Number: 23292
Gerrit-PatchSet: 2
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Aug 2025 18:41:22 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] WIP: workaround impala python3 make failure

Posted by "Laszlo Gaal (Code Review)" <ge...@cloudera.org>.
Laszlo Gaal has posted comments on this change. ( http://gerrit.cloudera.org:8080/23292 )

Change subject: WIP: workaround impala_python3 make failure
......................................................................


Patch Set 1: Code-Review+1

Feels a bit like a kludge to me, but if it passes all tests, I'm OK with the fix: IMO it is more important to unblock builds than to come up with a perfect solution. Improvements can always be added later.

Would be great to have other eyes on it though.


-- 
To view, visit http://gerrit.cloudera.org:8080/23292
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
Gerrit-Change-Number: 23292
Gerrit-PatchSet: 1
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Aug 2025 18:26:39 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14308: Workaround failure in impala python3 build

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/23292 )

Change subject: IMPALA-14308: Workaround failure in impala_python3 build
......................................................................

IMPALA-14308: Workaround failure in impala_python3 build

Construction of the impala-virtualenv fails since PyPI released version
7.0.0 of pbr. This blocks all precommit runs, since the Impala
virtualenv is required for all end-to-end tests.

The failure happen during pywebhdfs==0.3.2 installation. It is expected
to pullthe pinned version pbr==3.1.1, but the latest pbr==7.0.0 was
pulled instead. pbr==7.0.0 then broke with this error message:

  ModuleNotFoundError: No module named 'packaging.requirements'

This patch adds workaround in bootstrap_virtualenv.py to install
packaging==24.1 early for python3. Installing it early managed to
unblock `make -j impala_python3`. packaging==24.1 package is already
listed in infra/python/deps/gcovr-requirements.txt, which installed in
later step and in python3 virtualenv only.

Testing:
Pass shell/ tests in Ubuntu 22.04 and Rocky 9.2.

Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
Reviewed-on: http://gerrit.cloudera.org:8080/23292
Reviewed-by: Jason Fehr <jf...@cloudera.com>
Tested-by: Jason Fehr <jf...@cloudera.com>
---
M infra/python/bootstrap_virtualenv.py
1 file changed, 2 insertions(+), 0 deletions(-)

Approvals:
  Jason Fehr: Looks good to me, approved; Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/23292
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
Gerrit-Change-Number: 23292
Gerrit-PatchSet: 4
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>

[Impala-ASF-CR] IMPALA-14308: Workaround failure in impala python3 build

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Hello Laszlo Gaal, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/23292

to look at the new patch set (#2).

Change subject: IMPALA-14308: Workaround failure in impala_python3 build
......................................................................

IMPALA-14308: Workaround failure in impala_python3 build

Construction of the impala-virtualenv fails since PyPI released version
7.0.0 of pbr. This blocks all precommit runs, since the Impala
virtualenv is required for all end-to-end tests.

This patch adds workaround in bootstrap_virtualenv.py to install
packaging==24.1 early for python3. This package is listed in
infra/python/deps/gcovr-requirements.txt, which only installed in
python3 virtualenv. Installing it early managed to unblock
`make -j impala_python3`.

Testing:
Pass shell/ tests in Ubuntu 22.04 and Rocky 9.2.

Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
---
M infra/python/bootstrap_virtualenv.py
1 file changed, 2 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/23292/2
-- 
To view, visit http://gerrit.cloudera.org:8080/23292
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
Gerrit-Change-Number: 23292
Gerrit-PatchSet: 2
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>

[Impala-ASF-CR] WIP: workaround impala python3 make failure

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23292 )

Change subject: WIP: workaround impala_python3 make failure
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/20147/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/23292
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
Gerrit-Change-Number: 23292
Gerrit-PatchSet: 1
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Aug 2025 18:03:51 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14308: Workaround failure in impala python3 build

Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Jason Fehr has posted comments on this change. ( http://gerrit.cloudera.org:8080/23292 )

Change subject: IMPALA-14308: Workaround failure in impala_python3 build
......................................................................


Patch Set 3: Code-Review+2

(1 comment)

http://gerrit.cloudera.org:8080/#/c/23292/2//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/23292/2//COMMIT_MSG@13
PS2, Line 13: The failure happen during pywebhdfs==0.3.2 installation. It is expected
> Done.
Thanks. I do not understand either why this workaround fixed the issue.



-- 
To view, visit http://gerrit.cloudera.org:8080/23292
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
Gerrit-Change-Number: 23292
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Aug 2025 20:55:13 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-14308: Workaround failure in impala python3 build

Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Jason Fehr has posted comments on this change. ( http://gerrit.cloudera.org:8080/23292 )

Change subject: IMPALA-14308: Workaround failure in impala_python3 build
......................................................................


Patch Set 3:

Setting verified +1 since the gerrit-verify-dryrun build will fail until this patch is merged.


-- 
To view, visit http://gerrit.cloudera.org:8080/23292
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
Gerrit-Change-Number: 23292
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Aug 2025 20:56:50 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14308: Workaround failure in impala python3 build

Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Jason Fehr has posted comments on this change. ( http://gerrit.cloudera.org:8080/23292 )

Change subject: IMPALA-14308: Workaround failure in impala_python3 build
......................................................................


Patch Set 2: Code-Review+1

(1 comment)

http://gerrit.cloudera.org:8080/#/c/23292/2//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/23292/2//COMMIT_MSG@13
PS2, Line 13: This patch adds workaround in bootstrap_virtualenv.py to install
Can you explain more about why this workaround works?



-- 
To view, visit http://gerrit.cloudera.org:8080/23292
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
Gerrit-Change-Number: 23292
Gerrit-PatchSet: 2
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Aug 2025 20:07:40 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-14308: Workaround failure in impala python3 build

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/23292 )

Change subject: IMPALA-14308: Workaround failure in impala_python3 build
......................................................................


Patch Set 3:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/23292/2//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/23292/2//COMMIT_MSG@13
PS2, Line 13: The failure happen during pywebhdfs==0.3.2 installation. It is expected
> Can you explain more about why this workaround works?
Done.

This is the best explanation that I can make. Not 100% sure why it works though.
Maybe pbr==7.0.0 still being pulled, but having packaging==24.1 installed first allowed it pass.
I did `pip3 freeze` later on and see only pbr=3.1.1 installed in the virtualenv.


http://gerrit.cloudera.org:8080/#/c/23292/2//COMMIT_MSG@20
PS2, Line 20: packaging==24.1 early for python3. Installing it
> Test still running
Both has passed.



-- 
To view, visit http://gerrit.cloudera.org:8080/23292
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0167fb5e1e0637cdde64d0d3beaf6b154afc06b1
Gerrit-Change-Number: 23292
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Aug 2025 20:22:43 +0000
Gerrit-HasComments: Yes