You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Joe McDonnell (Code Review)" <ge...@cloudera.org> on 2023/04/25 23:20:55 UTC
[Impala-ASF-CR] IMPALA-11785: Warn if Thrift fastbinary is not working for impala-shell
Joe McDonnell has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19806
Change subject: IMPALA-11785: Warn if Thrift fastbinary is not working for impala-shell
......................................................................
IMPALA-11785: Warn if Thrift fastbinary is not working for impala-shell
Thrift's fastbinary module provides native code that
accelerations the BinaryProtocol. It can make a large
performance difference when using the Hiveserver2
protocol with impala-shell. If the fastbinary is not
working, it silently falls back to interpreted code.
This can happen because the fastbinary couldn't load
a particular library, etc.
This adds a warning on impala-shell startup when
it detects that Thrift's fastbinary is not working.
Testing:
- Added a simple test that verifies the impala-shell
does not output the warning
Change-Id: Id5d0e5db5cfdf1db4521b00f912b4697a7f646e8
---
M shell/impala_shell.py
M tests/shell/test_shell_commandline.py
2 files changed, 14 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/06/19806/1
--
To view, visit http://gerrit.cloudera.org:8080/19806
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Id5d0e5db5cfdf1db4521b00f912b4697a7f646e8
Gerrit-Change-Number: 19806
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
[Impala-ASF-CR] IMPALA-11785: Warn if Thrift fastbinary is not working for impala-shell
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19806 )
Change subject: IMPALA-11785: Warn if Thrift fastbinary is not working for impala-shell
......................................................................
Patch Set 3:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/9336/ DRY_RUN=true
--
To view, visit http://gerrit.cloudera.org:8080/19806
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id5d0e5db5cfdf1db4521b00f912b4697a7f646e8
Gerrit-Change-Number: 19806
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Mon, 22 May 2023 21:01:33 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11785: Warn if Thrift fastbinary is not working for impala-shell
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19806 )
Change subject: IMPALA-11785: Warn if Thrift fastbinary is not working for impala-shell
......................................................................
Patch Set 3: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/19806
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id5d0e5db5cfdf1db4521b00f912b4697a7f646e8
Gerrit-Change-Number: 19806
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Tue, 23 May 2023 02:17:10 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11785: Warn if Thrift fastbinary is not working for impala-shell
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19806 )
Change subject: IMPALA-11785: Warn if Thrift fastbinary is not working for impala-shell
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/12867/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/19806
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id5d0e5db5cfdf1db4521b00f912b4697a7f646e8
Gerrit-Change-Number: 19806
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Tue, 25 Apr 2023 23:41:15 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11785: Warn if Thrift fastbinary is not working for impala-shell
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/19806 )
Change subject: IMPALA-11785: Warn if Thrift fastbinary is not working for impala-shell
......................................................................
Patch Set 3: Code-Review+2
Thanks for adding the warning, I have also bumped into the issue and it would have saved some time.
--
To view, visit http://gerrit.cloudera.org:8080/19806
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id5d0e5db5cfdf1db4521b00f912b4697a7f646e8
Gerrit-Change-Number: 19806
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Mon, 22 May 2023 11:44:45 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11785: Warn if Thrift fastbinary is not working for impala-shell
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19806 )
Change subject: IMPALA-11785: Warn if Thrift fastbinary is not working for impala-shell
......................................................................
IMPALA-11785: Warn if Thrift fastbinary is not working for impala-shell
Thrift's fastbinary module provides native code that
accelerations the BinaryProtocol. It can make a large
performance difference when using the Hiveserver2
protocol with impala-shell. If the fastbinary is not
working, it silently falls back to interpreted code.
This can happen because the fastbinary couldn't load
a particular library, etc.
This adds a warning on impala-shell startup when
it detects that Thrift's fastbinary is not working.
When bin/impala-shell.sh is modified to use python3,
impala-shell outputs this error (shortened for legibility):
WARNING: Failed to load Thrift's fastbinary module. Thrift's
BinaryProtocol will not be accelerated, which can reduce performance.
Error was '{path to Python2 thrift fastbinary.so}: undefined symbol: _Py_ZeroStruct'
Testing:
- Added a simple test that verifies the impala-shell
does not output the warning
- Outputs warning when Python 2 thrift used for Python 3 shell
Change-Id: Id5d0e5db5cfdf1db4521b00f912b4697a7f646e8
Reviewed-on: http://gerrit.cloudera.org:8080/19806
Reviewed-by: Csaba Ringhofer <cs...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M shell/impala_shell.py
M tests/shell/test_shell_commandline.py
2 files changed, 14 insertions(+), 0 deletions(-)
Approvals:
Csaba Ringhofer: Looks good to me, approved
Impala Public Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/19806
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Id5d0e5db5cfdf1db4521b00f912b4697a7f646e8
Gerrit-Change-Number: 19806
Gerrit-PatchSet: 4
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
[Impala-ASF-CR] IMPALA-11785: Warn if Thrift fastbinary is not working for impala-shell
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/19806
to look at the new patch set (#2).
Change subject: IMPALA-11785: Warn if Thrift fastbinary is not working for impala-shell
......................................................................
IMPALA-11785: Warn if Thrift fastbinary is not working for impala-shell
Thrift's fastbinary module provides native code that
accelerations the BinaryProtocol. It can make a large
performance difference when using the Hiveserver2
protocol with impala-shell. If the fastbinary is not
working, it silently falls back to interpreted code.
This can happen because the fastbinary couldn't load
a particular library, etc.
This adds a warning on impala-shell startup when
it detects that Thrift's fastbinary is not working.
When bin/impala-shell.sh is modified to use python3,
impala-shell outputs this error (shortened for legibility):
WARNING: Failed to load Thrift's fastbinary module. Thrift's
BinaryProtocol will not be accelerated, which can reduce performance.
Error was '{path to Python2 thrift fastbinary.so}: undefined symbol: _Py_ZeroStruct'
Testing:
- Added a simple test that verifies the impala-shell
does not output the warning
- Outputs warning when Python 2 thrift used for Python 3 shell
Change-Id: Id5d0e5db5cfdf1db4521b00f912b4697a7f646e8
---
M shell/impala_shell.py
M tests/shell/test_shell_commandline.py
2 files changed, 14 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/06/19806/2
--
To view, visit http://gerrit.cloudera.org:8080/19806
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Id5d0e5db5cfdf1db4521b00f912b4697a7f646e8
Gerrit-Change-Number: 19806
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-11785: Warn if Thrift fastbinary is not working for impala-shell
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19806 )
Change subject: IMPALA-11785: Warn if Thrift fastbinary is not working for impala-shell
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/19806/1/shell/impala_shell.py
File shell/impala_shell.py:
http://gerrit.cloudera.org:8080/#/c/19806/1/shell/impala_shell.py@2133
PS1, Line 2133: i
flake8: F401 'thrift.protocol.fastbinary' imported but unused
--
To view, visit http://gerrit.cloudera.org:8080/19806
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id5d0e5db5cfdf1db4521b00f912b4697a7f646e8
Gerrit-Change-Number: 19806
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Tue, 25 Apr 2023 23:21:47 +0000
Gerrit-HasComments: Yes