You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org> on 2023/05/16 14:24:11 UTC

[Impala-ASF-CR] IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)

Csaba Ringhofer has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19893


Change subject: IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)
......................................................................

IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)

As __future__.unicode_literals is imported in impala-shell
concatenating an str with a literal leads to decoding the
string with 'ascii' codec which fails if there are non-ascii
characters. Converting the literal to str solves the issue.

Testing:
- added regression test + ran related EE tests

Change-Id: I99b72dd262fc7c382e8baee1dce7592880c84de2
---
M shell/impala_shell.py
M tests/shell/test_shell_commandline.py
2 files changed, 13 insertions(+), 0 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/93/19893/1
-- 
To view, visit http://gerrit.cloudera.org:8080/19893
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I99b72dd262fc7c382e8baee1dce7592880c84de2
Gerrit-Change-Number: 19893
Gerrit-PatchSet: 1
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>

[Impala-ASF-CR] IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)

Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Hello Jason Fehr, Joe McDonnell, Michael Smith, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/19893

to look at the new patch set (#3).

Change subject: IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)
......................................................................

IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)

As __future__.unicode_literals is imported in impala-shell
concatenating an str with a literal leads to decoding the
string with 'ascii' codec which fails if there are non-ascii
characters. Converting the literal to str solves the issue.

Testing:
- added regression test + ran related EE tests

Change-Id: I99b72dd262fc7c382e8baee1dce7592880c84de2
---
M shell/impala_shell.py
M shell/shell_output.py
M tests/shell/test_shell_commandline.py
3 files changed, 36 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/93/19893/3
-- 
To view, visit http://gerrit.cloudera.org:8080/19893
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I99b72dd262fc7c382e8baee1dce7592880c84de2
Gerrit-Change-Number: 19893
Gerrit-PatchSet: 3
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>

[Impala-ASF-CR] IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19893 )

Change subject: IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)
......................................................................


Patch Set 4: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/19893
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I99b72dd262fc7c382e8baee1dce7592880c84de2
Gerrit-Change-Number: 19893
Gerrit-PatchSet: 4
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Thu, 25 May 2023 00:33:33 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)

Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Hello Jason Fehr, Joe McDonnell, Michael Smith, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/19893

to look at the new patch set (#2).

Change subject: IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)
......................................................................

IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)

As __future__.unicode_literals is imported in impala-shell
concatenating an str with a literal leads to decoding the
string with 'ascii' codec which fails if there are non-ascii
characters. Converting the literal to str solves the issue.

Testing:
- added regression test + ran related EE tests

Change-Id: I99b72dd262fc7c382e8baee1dce7592880c84de2
---
M shell/impala_shell.py
M shell/shell_output.py
M tests/shell/test_shell_commandline.py
3 files changed, 36 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/93/19893/2
-- 
To view, visit http://gerrit.cloudera.org:8080/19893
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I99b72dd262fc7c382e8baee1dce7592880c84de2
Gerrit-Change-Number: 19893
Gerrit-PatchSet: 2
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>

[Impala-ASF-CR] IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/19893 )

Change subject: IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)
......................................................................


Patch Set 3: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/19893
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I99b72dd262fc7c382e8baee1dce7592880c84de2
Gerrit-Change-Number: 19893
Gerrit-PatchSet: 3
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Wed, 24 May 2023 15:26:36 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19893 )

Change subject: IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/13055/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/19893
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I99b72dd262fc7c382e8baee1dce7592880c84de2
Gerrit-Change-Number: 19893
Gerrit-PatchSet: 1
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Tue, 16 May 2023 14:45:06 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)

Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/19893 )

Change subject: IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)
......................................................................


Patch Set 3:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/19893/3/shell/shell_output.py
File shell/shell_output.py:

http://gerrit.cloudera.org:8080/#/c/19893/3/shell/shell_output.py@32
PS3, Line 32: def match_string_type(str_to_convert, reference_str):
> The reference_str looks a bit weird when looking at a use of this function,
I couldn't think of a better name (open to suggestions)


http://gerrit.cloudera.org:8080/#/c/19893/3/shell/shell_output.py@32
PS3, Line 32: 
            :   
I don't know about unit tests for the impala-shell - in tests/shell the tests are calling impala-shell through the command line.

I agree that it would be good to have unit test that call the Python functions directly instead of the current end-to-end tests. Is it ok to do the unit test in another commit? As it would the first of its kind, it may be better to have some larger discussion about where to put the test and I don't want to block this simple fix with it.



-- 
To view, visit http://gerrit.cloudera.org:8080/19893
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I99b72dd262fc7c382e8baee1dce7592880c84de2
Gerrit-Change-Number: 19893
Gerrit-PatchSet: 3
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Mon, 22 May 2023 14:58:49 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19893 )

Change subject: IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)
......................................................................


Patch Set 4: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/19893
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I99b72dd262fc7c382e8baee1dce7592880c84de2
Gerrit-Change-Number: 19893
Gerrit-PatchSet: 4
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Wed, 24 May 2023 19:09:18 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)

Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Jason Fehr has posted comments on this change. ( http://gerrit.cloudera.org:8080/19893 )

Change subject: IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)
......................................................................


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19893/3/shell/shell_output.py
File shell/shell_output.py:

http://gerrit.cloudera.org:8080/#/c/19893/3/shell/shell_output.py@33
PS3, Line 33:   """ Returns 'str_to_convert' converted to the same type as 'reference_str'.
It would be good to have specific unit tests for this function especially since the different pytest vectors will exercise the python 2 and 3 paths through this function.



-- 
To view, visit http://gerrit.cloudera.org:8080/19893
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I99b72dd262fc7c382e8baee1dce7592880c84de2
Gerrit-Change-Number: 19893
Gerrit-PatchSet: 3
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Thu, 18 May 2023 20:45:29 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/19893 )

Change subject: IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)
......................................................................


Patch Set 3: Code-Review+1

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19893/3/shell/shell_output.py
File shell/shell_output.py:

http://gerrit.cloudera.org:8080/#/c/19893/3/shell/shell_output.py@32
PS3, Line 32: def match_string_type(str_to_convert, reference_str):
The reference_str looks a bit weird when looking at a use of this function, but it makes sense.



-- 
To view, visit http://gerrit.cloudera.org:8080/19893
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I99b72dd262fc7c382e8baee1dce7592880c84de2
Gerrit-Change-Number: 19893
Gerrit-PatchSet: 3
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Wed, 17 May 2023 23:24:03 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/19893 )

Change subject: IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19893/1/shell/impala_shell.py
File shell/impala_shell.py:

http://gerrit.cloudera.org:8080/#/c/19893/1/shell/impala_shell.py@1144
PS1, Line 1144:           query_profile_prefix = query_profile_prefix.encode('utf-8')
> It looks like our current version of Thrift generates python that returns R
Maybe this only impacts GetRuntimeProfile. At the least I think we also need to check line 1148.



-- 
To view, visit http://gerrit.cloudera.org:8080/19893
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I99b72dd262fc7c382e8baee1dce7592880c84de2
Gerrit-Change-Number: 19893
Gerrit-PatchSet: 1
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Tue, 16 May 2023 16:39:40 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)

Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/19893 )

Change subject: IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)
......................................................................


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19893/1/shell/impala_shell.py
File shell/impala_shell.py:

http://gerrit.cloudera.org:8080/#/c/19893/1/shell/impala_shell.py@1144
PS1, Line 1144:           print(query_profile_prefix + profile)
> Tests in the test_shell_commandline.py file (specifically 'test_internation
Added conversion for the failed_profile case too. Adding test for this is a bit harder as this needs a query retry to happen.

>Should we convert all RPC results to unicode?
This is a bit tricky topic as there are conflicting goals:
- run in Python 2 and Python 3 too
- have good performance when printing query results
- handle strings that are not valid utf-8 sequences - this is possible as STRING can be used to store binary data in Impala (the BINARY type is relatively new) - so we replace unprintable characters (instead of throwing an exception and failing the query)

In the end the stream we print the results to expects utf-8 encoded byte array. The profile is a bit special case as we print it to stdout and not use the redirection set in -o parameter.



-- 
To view, visit http://gerrit.cloudera.org:8080/19893
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I99b72dd262fc7c382e8baee1dce7592880c84de2
Gerrit-Change-Number: 19893
Gerrit-PatchSet: 2
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Wed, 17 May 2023 11:45:53 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19893 )

Change subject: IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)
......................................................................


Patch Set 2:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/13062/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/19893
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I99b72dd262fc7c382e8baee1dce7592880c84de2
Gerrit-Change-Number: 19893
Gerrit-PatchSet: 2
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Wed, 17 May 2023 11:58:08 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19893 )

Change subject: IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)
......................................................................

IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)

As __future__.unicode_literals is imported in impala-shell
concatenating an str with a literal leads to decoding the
string with 'ascii' codec which fails if there are non-ascii
characters. Converting the literal to str solves the issue.

Testing:
- added regression test + ran related EE tests

Change-Id: I99b72dd262fc7c382e8baee1dce7592880c84de2
Reviewed-on: http://gerrit.cloudera.org:8080/19893
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M shell/impala_shell.py
M shell/shell_output.py
M tests/shell/test_shell_commandline.py
3 files changed, 36 insertions(+), 3 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/19893
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I99b72dd262fc7c382e8baee1dce7592880c84de2
Gerrit-Change-Number: 19893
Gerrit-PatchSet: 5
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>

[Impala-ASF-CR] IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)

Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Jason Fehr has posted comments on this change. ( http://gerrit.cloudera.org:8080/19893 )

Change subject: IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19893/1/shell/impala_shell.py
File shell/impala_shell.py:

http://gerrit.cloudera.org:8080/#/c/19893/1/shell/impala_shell.py@1144
PS1, Line 1144:           query_profile_prefix = query_profile_prefix.encode('utf-8')
> Maybe this only impacts GetRuntimeProfile. At the least I think we also nee
Tests in the test_shell_commandline.py file (specifically 'test_international_characters', 'test_international_characters_prettyprint', and 'test_international_characters_prettyprint_tabs') assert that using Russian UTF-8 characters in a select query succeeds.

I agree that line 1148 needs to be UTF-8 encoded as well since it's a string literal.



-- 
To view, visit http://gerrit.cloudera.org:8080/19893
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I99b72dd262fc7c382e8baee1dce7592880c84de2
Gerrit-Change-Number: 19893
Gerrit-PatchSet: 1
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Tue, 16 May 2023 17:57:12 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/19893 )

Change subject: IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19893/1/shell/impala_shell.py
File shell/impala_shell.py:

http://gerrit.cloudera.org:8080/#/c/19893/1/shell/impala_shell.py@1144
PS1, Line 1144:           query_profile_prefix = query_profile_prefix.encode('utf-8')
> I'm concerned that this is just one case impacted by using unicode_literals
It looks like our current version of Thrift generates python that returns RPC results as byte strings. Should we convert all RPC results to unicode? Or patch literals that might be concatenated with RPC results so they're byte strings in Python 2 and unicode in Python 3? That partially defeats the point of setting unicode_literal.



-- 
To view, visit http://gerrit.cloudera.org:8080/19893
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I99b72dd262fc7c382e8baee1dce7592880c84de2
Gerrit-Change-Number: 19893
Gerrit-PatchSet: 1
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Tue, 16 May 2023 16:35:38 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19893 )

Change subject: IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)
......................................................................


Patch Set 2:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/19893/2/shell/shell_output.py
File shell/shell_output.py:

http://gerrit.cloudera.org:8080/#/c/19893/2/shell/shell_output.py@31
PS2, Line 31: def match_string_type(str_to_convert, reference_str):
flake8: E302 expected 2 blank lines, found 1


http://gerrit.cloudera.org:8080/#/c/19893/2/shell/shell_output.py@48
PS2, Line 48: n
flake8: F841 local variable 'needs_conversion' is assigned to but never used



-- 
To view, visit http://gerrit.cloudera.org:8080/19893
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I99b72dd262fc7c382e8baee1dce7592880c84de2
Gerrit-Change-Number: 19893
Gerrit-PatchSet: 2
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Wed, 17 May 2023 11:36:11 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19893 )

Change subject: IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)
......................................................................


Patch Set 3:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/13063/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/19893
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I99b72dd262fc7c382e8baee1dce7592880c84de2
Gerrit-Change-Number: 19893
Gerrit-PatchSet: 3
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Wed, 17 May 2023 13:34:43 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19893 )

Change subject: IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)
......................................................................


Patch Set 4:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/9347/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/19893
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I99b72dd262fc7c382e8baee1dce7592880c84de2
Gerrit-Change-Number: 19893
Gerrit-PatchSet: 4
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Wed, 24 May 2023 19:09:18 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/19893 )

Change subject: IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19893/1/shell/impala_shell.py
File shell/impala_shell.py:

http://gerrit.cloudera.org:8080/#/c/19893/1/shell/impala_shell.py@1144
PS1, Line 1144:           query_profile_prefix = query_profile_prefix.encode('utf-8')
I'm concerned that this is just one case impacted by using unicode_literals. Will line 1148 also error?



-- 
To view, visit http://gerrit.cloudera.org:8080/19893
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I99b72dd262fc7c382e8baee1dce7592880c84de2
Gerrit-Change-Number: 19893
Gerrit-PatchSet: 1
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Tue, 16 May 2023 16:16:34 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)

Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Jason Fehr has posted comments on this change. ( http://gerrit.cloudera.org:8080/19893 )

Change subject: IMPALA-12145: Fix profiles with non-ascii character in impala-shell (python2)
......................................................................


Patch Set 3: Code-Review+1

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19893/3/shell/shell_output.py
File shell/shell_output.py:

http://gerrit.cloudera.org:8080/#/c/19893/3/shell/shell_output.py@32
PS3, Line 32: 
            :   
> I don't know about unit tests for the impala-shell - in tests/shell the tes
Yeah, can always do a follow up.  If all the code paths get exercised through the command line, then specific unit tests for this function is not needed.



-- 
To view, visit http://gerrit.cloudera.org:8080/19893
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I99b72dd262fc7c382e8baee1dce7592880c84de2
Gerrit-Change-Number: 19893
Gerrit-PatchSet: 3
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Mon, 22 May 2023 15:20:50 +0000
Gerrit-HasComments: Yes