You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Quanlong Huang (Code Review)" <ge...@cloudera.org> on 2021/02/21 08:41:00 UTC
[Impala-ASF-CR] IMPALA-10523: Fix impala-shell crash in printing error messages that contain UTF-8 characters
Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17099
Change subject: IMPALA-10523: Fix impala-shell crash in printing error messages that contain UTF-8 characters
......................................................................
IMPALA-10523: Fix impala-shell crash in printing error messages that contain UTF-8 characters
In Python2, print() converts all non-keyword arguments to strings like
str() does and writes them to the stream. str() on QueryStateException
returns its value(i.e. error message) which could be in unicode type.
Python2 will implicitly encode it to str type using the default
encoding, ‘ascii’. This could result in UnicodeEncodeError when there
are non-ascii characters in the error message.
This patch explicitly encodes the error message using ‘utf-8’ encoding
if it’s in unicode type and the shell is run in Python2.
Tests:
- Add test in test_shell_interactive.py
Change-Id: Ie10f5b03ecc5877053c2fbada1afaf256b423a71
---
M shell/impala_shell.py
M tests/shell/test_shell_interactive.py
2 files changed, 13 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/99/17099/1
--
To view, visit http://gerrit.cloudera.org:8080/17099
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie10f5b03ecc5877053c2fbada1afaf256b423a71
Gerrit-Change-Number: 17099
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
[Impala-ASF-CR] IMPALA-10523: Fix impala-shell crash in printing error messages that contain UTF-8 characters
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/17099 )
Change subject: IMPALA-10523: Fix impala-shell crash in printing error messages that contain UTF-8 characters
......................................................................
Patch Set 3:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/17099/2/shell/impala_shell.py
File shell/impala_shell.py:
http://gerrit.cloudera.org:8080/#/c/17099/2/shell/impala_shell.py@1321
PS2, Line 1321: # Python2 will implicitly convert unicode to str when printing to stderr. It's done
> nit: could you add a short one line comment that explains this condition?
Done
--
To view, visit http://gerrit.cloudera.org:8080/17099
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie10f5b03ecc5877053c2fbada1afaf256b423a71
Gerrit-Change-Number: 17099
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Tamas Mate <tm...@cloudera.com>
Gerrit-Comment-Date: Fri, 12 Mar 2021 08:17:15 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10523: Fix impala-shell crash in printing error messages that contain UTF-8 characters
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Hello Tamas Mate, Laszlo Gaal, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/17099
to look at the new patch set (#3).
Change subject: IMPALA-10523: Fix impala-shell crash in printing error messages that contain UTF-8 characters
......................................................................
IMPALA-10523: Fix impala-shell crash in printing error messages that contain UTF-8 characters
In Python2, print() converts all non-keyword arguments to strings like
str() does and writes them to the stream. str() on QueryStateException
returns its value(i.e. error message) which could be in unicode type.
Python2 will implicitly encode it to str type using the default
encoding, 'ascii'. This could result in UnicodeEncodeError when there
are non-ascii characters in the error message.
This patch explicitly encodes the error message using 'utf-8' encoding
if it's in unicode type and the shell is run in Python2.
Tests:
- Add test in test_shell_interactive.py
Change-Id: Ie10f5b03ecc5877053c2fbada1afaf256b423a71
---
M shell/impala_shell.py
M tests/shell/test_shell_interactive.py
2 files changed, 16 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/99/17099/3
--
To view, visit http://gerrit.cloudera.org:8080/17099
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie10f5b03ecc5877053c2fbada1afaf256b423a71
Gerrit-Change-Number: 17099
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Tamas Mate <tm...@cloudera.com>
[Impala-ASF-CR] IMPALA-10523: Fix impala-shell crash in printing error messages that contain UTF-8 characters
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17099 )
Change subject: IMPALA-10523: Fix impala-shell crash in printing error messages that contain UTF-8 characters
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/8182/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/17099
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie10f5b03ecc5877053c2fbada1afaf256b423a71
Gerrit-Change-Number: 17099
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Sun, 21 Feb 2021 09:00:10 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10523: Fix impala-shell crash in printing error messages that contain UTF-8 characters
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/17099
to look at the new patch set (#2).
Change subject: IMPALA-10523: Fix impala-shell crash in printing error messages that contain UTF-8 characters
......................................................................
IMPALA-10523: Fix impala-shell crash in printing error messages that contain UTF-8 characters
In Python2, print() converts all non-keyword arguments to strings like
str() does and writes them to the stream. str() on QueryStateException
returns its value(i.e. error message) which could be in unicode type.
Python2 will implicitly encode it to str type using the default
encoding, 'ascii'. This could result in UnicodeEncodeError when there
are non-ascii characters in the error message.
This patch explicitly encodes the error message using 'utf-8' encoding
if it's in unicode type and the shell is run in Python2.
Tests:
- Add test in test_shell_interactive.py
Change-Id: Ie10f5b03ecc5877053c2fbada1afaf256b423a71
---
M shell/impala_shell.py
M tests/shell/test_shell_interactive.py
2 files changed, 13 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/99/17099/2
--
To view, visit http://gerrit.cloudera.org:8080/17099
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie10f5b03ecc5877053c2fbada1afaf256b423a71
Gerrit-Change-Number: 17099
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-10523: Fix impala-shell crash in printing error messages that contain UTF-8 characters
Posted by "Tamas Mate (Code Review)" <ge...@cloudera.org>.
Tamas Mate has posted comments on this change. ( http://gerrit.cloudera.org:8080/17099 )
Change subject: IMPALA-10523: Fix impala-shell crash in printing error messages that contain UTF-8 characters
......................................................................
Patch Set 3: Code-Review+1
Hi Quanlong, thanks for adding the comment, LGTM!
--
To view, visit http://gerrit.cloudera.org:8080/17099
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie10f5b03ecc5877053c2fbada1afaf256b423a71
Gerrit-Change-Number: 17099
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Tamas Mate <tm...@cloudera.com>
Gerrit-Comment-Date: Fri, 12 Mar 2021 08:46:26 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10523: Fix impala-shell crash in printing error messages that contain UTF-8 characters
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17099 )
Change subject: IMPALA-10523: Fix impala-shell crash in printing error messages that contain UTF-8 characters
......................................................................
Patch Set 3:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6959/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/17099
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie10f5b03ecc5877053c2fbada1afaf256b423a71
Gerrit-Change-Number: 17099
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Tamas Mate <tm...@cloudera.com>
Gerrit-Comment-Date: Fri, 12 Mar 2021 12:31:42 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10523: Fix impala-shell crash in printing error messages that contain UTF-8 characters
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17099 )
Change subject: IMPALA-10523: Fix impala-shell crash in printing error messages that contain UTF-8 characters
......................................................................
Patch Set 3: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/17099
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie10f5b03ecc5877053c2fbada1afaf256b423a71
Gerrit-Change-Number: 17099
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Tamas Mate <tm...@cloudera.com>
Gerrit-Comment-Date: Fri, 12 Mar 2021 18:19:08 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10523: Fix impala-shell crash in printing error messages that contain UTF-8 characters
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17099 )
Change subject: IMPALA-10523: Fix impala-shell crash in printing error messages that contain UTF-8 characters
......................................................................
Patch Set 3:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/8348/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/17099
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie10f5b03ecc5877053c2fbada1afaf256b423a71
Gerrit-Change-Number: 17099
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Tamas Mate <tm...@cloudera.com>
Gerrit-Comment-Date: Fri, 12 Mar 2021 08:37:25 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10523: Fix impala-shell crash in printing error messages that contain UTF-8 characters
Posted by "Laszlo Gaal (Code Review)" <ge...@cloudera.org>.
Laszlo Gaal has posted comments on this change. ( http://gerrit.cloudera.org:8080/17099 )
Change subject: IMPALA-10523: Fix impala-shell crash in printing error messages that contain UTF-8 characters
......................................................................
Patch Set 3: Code-Review+2
LGTM; thanks for the fix, Quanlong!
--
To view, visit http://gerrit.cloudera.org:8080/17099
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie10f5b03ecc5877053c2fbada1afaf256b423a71
Gerrit-Change-Number: 17099
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Tamas Mate <tm...@cloudera.com>
Gerrit-Comment-Date: Fri, 12 Mar 2021 10:02:18 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10523: Fix impala-shell crash in printing error messages that contain UTF-8 characters
Posted by "Tamas Mate (Code Review)" <ge...@cloudera.org>.
Tamas Mate has posted comments on this change. ( http://gerrit.cloudera.org:8080/17099 )
Change subject: IMPALA-10523: Fix impala-shell crash in printing error messages that contain UTF-8 characters
......................................................................
Patch Set 2:
(1 comment)
Hi Quanlong, looks good to me, just one minor observation.
http://gerrit.cloudera.org:8080/#/c/17099/2/shell/impala_shell.py
File shell/impala_shell.py:
http://gerrit.cloudera.org:8080/#/c/17099/2/shell/impala_shell.py@1321
PS2, Line 1321: if sys.version_info.major == 2 and isinstance(msg, unicode):
nit: could you add a short one line comment that explains this condition?
--
To view, visit http://gerrit.cloudera.org:8080/17099
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie10f5b03ecc5877053c2fbada1afaf256b423a71
Gerrit-Change-Number: 17099
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Tamas Mate <tm...@cloudera.com>
Gerrit-Comment-Date: Tue, 02 Mar 2021 12:33:11 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10523: Fix impala-shell crash in printing error messages that contain UTF-8 characters
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17099 )
Change subject: IMPALA-10523: Fix impala-shell crash in printing error messages that contain UTF-8 characters
......................................................................
IMPALA-10523: Fix impala-shell crash in printing error messages that contain UTF-8 characters
In Python2, print() converts all non-keyword arguments to strings like
str() does and writes them to the stream. str() on QueryStateException
returns its value(i.e. error message) which could be in unicode type.
Python2 will implicitly encode it to str type using the default
encoding, 'ascii'. This could result in UnicodeEncodeError when there
are non-ascii characters in the error message.
This patch explicitly encodes the error message using 'utf-8' encoding
if it's in unicode type and the shell is run in Python2.
Tests:
- Add test in test_shell_interactive.py
Change-Id: Ie10f5b03ecc5877053c2fbada1afaf256b423a71
Reviewed-on: http://gerrit.cloudera.org:8080/17099
Reviewed-by: Tamas Mate <tm...@cloudera.com>
Reviewed-by: Laszlo Gaal <la...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M shell/impala_shell.py
M tests/shell/test_shell_interactive.py
2 files changed, 16 insertions(+), 1 deletion(-)
Approvals:
Tamas Mate: Looks good to me, but someone else must approve
Laszlo Gaal: Looks good to me, approved
Impala Public Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/17099
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ie10f5b03ecc5877053c2fbada1afaf256b423a71
Gerrit-Change-Number: 17099
Gerrit-PatchSet: 4
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Tamas Mate <tm...@cloudera.com>