You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Adam Tamas (Code Review)" <ge...@cloudera.org> on 2020/08/13 10:04:37 UTC
[Impala-ASF-CR] WIP IMPALA-10012: ds hll sketch() results ascii codec decoding error fix
Adam Tamas has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16305
Change subject: WIP IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix
......................................................................
WIP IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix
While the ds_hll_sketch() generates a string value as output the date
is not an ascii encoded text but a bitsketch, because of this, when
the shell get this data it disconnect while it tries to decode it.
This patch contains a solution, where we replace any not UTF-8
decodeable characters if we run into an UnicodeDecodeError.
This solution is working with the Thrift 0.9.3 autogegnerated gen-py
but still fails with Thrift 0.11.0.
Testing:
-manual testing with these protocols: 'hs2-http', 'hs2', 'beeswax'
(-test added to tests/shell/test_shell_interactive.py)
Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8
---
M shell/impala_client.py
M shell/impala_shell.py
M shell/shell_output.py
M tests/shell/test_shell_interactive.py
4 files changed, 44 insertions(+), 5 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/05/16305/4
--
To view, visit http://gerrit.cloudera.org:8080/16305
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8
Gerrit-Change-Number: 16305
Gerrit-PatchSet: 4
Gerrit-Owner: Adam Tamas <ta...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
[Impala-ASF-CR] IMPALA-10012: ds hll sketch() results ascii codec decoding error fix
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/16305 )
Change subject: IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix
......................................................................
Patch Set 11:
Sorry, I somehow believed that there was already a successful jenkins run but actually there wasn't. Reverted the commit and asked Tamás to upload it again
--
To view, visit http://gerrit.cloudera.org:8080/16305
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8
Gerrit-Change-Number: 16305
Gerrit-PatchSet: 11
Gerrit-Owner: Adam Tamas <en...@gmail.com>
Gerrit-Reviewer: Adam Tamas <en...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 04 Sep 2020 12:31:08 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10012: ds hll sketch() results ascii codec decoding error fix
Posted by "Adam Tamas (Code Review)" <ge...@cloudera.org>.
Adam Tamas has posted comments on this change. ( http://gerrit.cloudera.org:8080/16305 )
Change subject: IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix
......................................................................
Patch Set 10:
(6 comments)
http://gerrit.cloudera.org:8080/#/c/16305/9//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/16305/9//COMMIT_MSG@20
PS9, Line 20:
> typo
Done
http://gerrit.cloudera.org:8080/#/c/16305/9//COMMIT_MSG@21
PS9, Line 21: This solution is working with the Th
> Can you be more specific about the error thrown?
Done
http://gerrit.cloudera.org:8080/#/c/16305/9//COMMIT_MSG@24
PS9, Line 24: For Thrift 0.11.0 the error is catched and an error message is sen
> can you add a bit more info, e.g. the issue can be reproduced with SELECT u
Done
http://gerrit.cloudera.org:8080/#/c/16305/9/shell/impala_shell.py
File shell/impala_shell.py:
http://gerrit.cloudera.org:8080/#/c/16305/9/shell/impala_shell.py@1232
PS9, Line 1232: 'to find the possible source of the error.' % (e,), file=sys.stderr)
> nit: +4 indent
Done
http://gerrit.cloudera.org:8080/#/c/16305/9/shell/shell_output.py
File shell/shell_output.py:
http://gerrit.cloudera.org:8080/#/c/16305/9/shell/shell_output.py@40
PS9, Line 40: decodable.
> nit: decodable
Done
http://gerrit.cloudera.org:8080/#/c/16305/9/shell/shell_output.py@41
PS9, Line 41: row an
> typo
Done
--
To view, visit http://gerrit.cloudera.org:8080/16305
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8
Gerrit-Change-Number: 16305
Gerrit-PatchSet: 10
Gerrit-Owner: Adam Tamas <en...@gmail.com>
Gerrit-Reviewer: Adam Tamas <en...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 04 Sep 2020 11:41:26 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10012: ds hll sketch() results ascii codec decoding error fix
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/16305 )
Change subject: IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix
......................................................................
Patch Set 9:
(6 comments)
only more nits
http://gerrit.cloudera.org:8080/#/c/16305/9//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/16305/9//COMMIT_MSG@20
PS9, Line 20: messenge
typo
http://gerrit.cloudera.org:8080/#/c/16305/9//COMMIT_MSG@21
PS9, Line 21: (not working with beeswax protocol).
Can you be more specific about the error thrown?
http://gerrit.cloudera.org:8080/#/c/16305/9//COMMIT_MSG@24
PS9, Line 24: -manual testing with these protocols: 'hs2-http', 'hs2', 'beeswax'
can you add a bit more info, e.g. the issue can be reproduced with SELECT unhex("aa")
http://gerrit.cloudera.org:8080/#/c/16305/9/shell/impala_shell.py
File shell/impala_shell.py:
http://gerrit.cloudera.org:8080/#/c/16305/9/shell/impala_shell.py@1232
PS9, Line 1232: 'to find the possible source of the error.' % (e,), file=sys.stderr)
nit: +4 indent
http://gerrit.cloudera.org:8080/#/c/16305/9/shell/shell_output.py
File shell/shell_output.py:
http://gerrit.cloudera.org:8080/#/c/16305/9/shell/shell_output.py@40
PS9, Line 40: decodeable
nit: decodable
http://gerrit.cloudera.org:8080/#/c/16305/9/shell/shell_output.py@41
PS9, Line 41: rowand
typo
--
To view, visit http://gerrit.cloudera.org:8080/16305
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8
Gerrit-Change-Number: 16305
Gerrit-PatchSet: 9
Gerrit-Owner: Adam Tamas <en...@gmail.com>
Gerrit-Reviewer: Adam Tamas <en...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 04 Sep 2020 11:23:36 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10012: ds hll sketch() results ascii codec decoding error fix
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16305 )
Change subject: IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix
......................................................................
Patch Set 8:
Build Failed
https://jenkins.impala.io/job/gerrit-code-review-checks/7100/ : Initial code review checks failed. See linked job for details on the failure.
--
To view, visit http://gerrit.cloudera.org:8080/16305
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8
Gerrit-Change-Number: 16305
Gerrit-PatchSet: 8
Gerrit-Owner: Adam Tamas <ta...@cloudera.com>
Gerrit-Reviewer: Adam Tamas <ta...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 04 Sep 2020 08:31:38 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10012: ds hll sketch() results ascii codec decoding error fix
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/16305 )
Change subject: IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix
......................................................................
Patch Set 10: Verified+1
There was no code change since the last verify job, so I give +1 for verified too.
--
To view, visit http://gerrit.cloudera.org:8080/16305
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8
Gerrit-Change-Number: 16305
Gerrit-PatchSet: 10
Gerrit-Owner: Adam Tamas <en...@gmail.com>
Gerrit-Reviewer: Adam Tamas <en...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 04 Sep 2020 12:18:18 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10012: ds hll sketch() results ascii codec decoding error fix
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16305 )
Change subject: IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix
......................................................................
Patch Set 10:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/7103/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/16305
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8
Gerrit-Change-Number: 16305
Gerrit-PatchSet: 10
Gerrit-Owner: Adam Tamas <en...@gmail.com>
Gerrit-Reviewer: Adam Tamas <en...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 04 Sep 2020 12:03:19 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10012: ds hll sketch() results ascii codec decoding error fix
Posted by "Adam Tamas (Code Review)" <ge...@cloudera.org>.
Adam Tamas has posted comments on this change. ( http://gerrit.cloudera.org:8080/16305 )
Change subject: IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix
......................................................................
Patch Set 9:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/16305/5//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/16305/5//COMMIT_MSG@14
PS5, Line 14:
> nit: please wrap at 72
Done
http://gerrit.cloudera.org:8080/#/c/16305/5//COMMIT_MSG@14
PS5, Line 14: decodable
> typo: I think it is decodable
Done
http://gerrit.cloudera.org:8080/#/c/16305/5//COMMIT_MSG@17
PS5, Line 17: autogenerated
> typo
Done
http://gerrit.cloudera.org:8080/#/c/16305/5//COMMIT_MSG@18
PS5, Line 18: but still fails with Thrift 0.11.0
> Can you create a ticket for this? As we have discussed, fixing this would n
Done (IMPALA-10145)
http://gerrit.cloudera.org:8080/#/c/16305/5/shell/impala_shell.py
File shell/impala_shell.py:
http://gerrit.cloudera.org:8080/#/c/16305/5/shell/impala_shell.py@1187
PS5, Line 1187: self.output_stream.write(rows)
> nit: extra line
Done
--
To view, visit http://gerrit.cloudera.org:8080/16305
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8
Gerrit-Change-Number: 16305
Gerrit-PatchSet: 9
Gerrit-Owner: Adam Tamas <ta...@cloudera.com>
Gerrit-Reviewer: Adam Tamas <ta...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 04 Sep 2020 10:53:57 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10012: ds hll sketch() results ascii codec decoding error fix
Posted by "Adam Tamas (Code Review)" <ge...@cloudera.org>.
Adam Tamas has uploaded a new patch set (#9). ( http://gerrit.cloudera.org:8080/16305 )
Change subject: IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix
......................................................................
IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix
While the ds_hll_sketch() generates a string value as output the data
is not an ascii encoded text but a bitsketch, because of this, when
the shell get this data it disconnect while it tries to decode it.
This patch contains a solution, where we replace any not UTF-8
decodable characters if we run into an UnicodeDecodeError after
fetching it.
This solution is working with the Thrift 0.9.3 autogenerated gen-py
but still fails with Thrift 0.11.0.
For Thrift 0.11.0 the error is catched and an error messenge is sent
(not working with beeswax protocol).
Testing:
-manual testing with these protocols: 'hs2-http', 'hs2', 'beeswax'
Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8
---
M shell/impala_client.py
M shell/impala_shell.py
M shell/shell_output.py
3 files changed, 37 insertions(+), 5 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/05/16305/9
--
To view, visit http://gerrit.cloudera.org:8080/16305
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8
Gerrit-Change-Number: 16305
Gerrit-PatchSet: 9
Gerrit-Owner: Adam Tamas <ta...@cloudera.com>
Gerrit-Reviewer: Adam Tamas <ta...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] WIP IMPALA-10012: ds hll sketch() results ascii codec decoding error fix
Posted by "Adam Tamas (Code Review)" <ge...@cloudera.org>.
Adam Tamas has posted comments on this change. ( http://gerrit.cloudera.org:8080/16305 )
Change subject: WIP IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix
......................................................................
Patch Set 4:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/16305/3/shell/impala_client.py
File shell/impala_client.py:
http://gerrit.cloudera.org:8080/#/c/16305/3/shell/impala_client.py@1100
PS3, Line 1100: try:
> It could be potentially faster if the exception was caught per row instead
Done
http://gerrit.cloudera.org:8080/#/c/16305/3/shell/impala_shell.py
File shell/impala_shell.py:
http://gerrit.cloudera.org:8080/#/c/16305/3/shell/impala_shell.py@1191
PS3, Line 1191: # retrieve the error log
> Where does the exception come from?
Here, it can come from 2 different sources depending on which formatter we are using (DelimitedOutputFormatter or PrettyOutputFormatter from the shell_output.py) Moved the check there.
This is only true, if we are using the gen-py generated by thrift 0.9.3, if it is 0.11.0 the error is coming from the thrift autogenerated files. (Here is no solution for thrift 0.11.0 yet)
http://gerrit.cloudera.org:8080/#/c/16305/3/shell/impala_shell.py@1197
PS3, Line 1197: if
> What is "i", a character? I would prefer a name like "ch" in that case, as
THANKS!
This was a problem I was looking a solution for.
(The Dwarves delved too greedily and too deep)
http://gerrit.cloudera.org:8080/#/c/16305/3/tests/shell/test_shell_interactive.py
File tests/shell/test_shell_interactive.py:
http://gerrit.cloudera.org:8080/#/c/16305/3/tests/shell/test_shell_interactive.py@289
PS3, Line 289:
> I would prefer to test against a predefined string, e.g. one that contains
While it is theoretically done, since the tests use thrift 0.11.0 this test will fail for now.
http://gerrit.cloudera.org:8080/#/c/16305/3/tests/shell/test_shell_interactive.py@292
PS3, Line 292: child_proc = spawn_shell(shell_cmd)
> This path tests the shell when it uses the pretty printer - can you also te
Done
--
To view, visit http://gerrit.cloudera.org:8080/16305
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8
Gerrit-Change-Number: 16305
Gerrit-PatchSet: 4
Gerrit-Owner: Adam Tamas <ta...@cloudera.com>
Gerrit-Reviewer: Adam Tamas <ta...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 13 Aug 2020 10:05:42 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10012: ds hll sketch() results ascii codec decoding error fix
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/16305 )
Change subject: IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix
......................................................................
Patch Set 10: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/16305
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8
Gerrit-Change-Number: 16305
Gerrit-PatchSet: 10
Gerrit-Owner: Adam Tamas <en...@gmail.com>
Gerrit-Reviewer: Adam Tamas <en...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 04 Sep 2020 12:16:14 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-10012: ds hll sketch() results ascii codec decoding error fix
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16305 )
Change subject: WIP IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix
......................................................................
Patch Set 4:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/6905/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/16305
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8
Gerrit-Change-Number: 16305
Gerrit-PatchSet: 4
Gerrit-Owner: Adam Tamas <ta...@cloudera.com>
Gerrit-Reviewer: Adam Tamas <ta...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 13 Aug 2020 10:25:41 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10012: ds hll sketch() results ascii codec decoding error fix
Posted by "Adam Tamas (Code Review)" <ge...@cloudera.org>.
Adam Tamas has uploaded a new patch set (#10). ( http://gerrit.cloudera.org:8080/16305 )
Change subject: IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix
......................................................................
IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix
While the ds_hll_sketch() generates a string value as output the data
is not an ascii encoded text but a bitsketch, because of this, when
the shell get this data it disconnect while it tries to decode it.
The issue can be reproduced with a simple method like using unhex
with a wrong input.
Example: SELECT unhex("aa");
This patch contains a solution, where we replace any not UTF-8
decodable characters if we run into an UnicodeDecodeError after
fetching it.
This solution is working with the Thrift 0.9.3 autogenerated gen-py
but still fails with Thrift 0.11.0.
For Thrift 0.11.0 the error is catched and an error message is sent
(not working with beeswax protocol, because it generates a different
error (TypeError) which can come for other reasons too).
Testing:
-manual testing with these protocols: 'hs2-http', 'hs2', 'beeswax'
Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8
---
M shell/impala_client.py
M shell/impala_shell.py
M shell/shell_output.py
3 files changed, 37 insertions(+), 5 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/05/16305/10
--
To view, visit http://gerrit.cloudera.org:8080/16305
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8
Gerrit-Change-Number: 16305
Gerrit-PatchSet: 10
Gerrit-Owner: Adam Tamas <en...@gmail.com>
Gerrit-Reviewer: Adam Tamas <en...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-10012: ds hll sketch() results ascii codec decoding error fix
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16305 )
Change subject: IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix
......................................................................
Patch Set 9:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6402/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/16305
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8
Gerrit-Change-Number: 16305
Gerrit-PatchSet: 9
Gerrit-Owner: Adam Tamas <ta...@cloudera.com>
Gerrit-Reviewer: Adam Tamas <ta...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 04 Sep 2020 09:43:39 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10012: ds hll sketch() results ascii codec decoding error fix
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16305 )
Change subject: IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix
......................................................................
Patch Set 9:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/7101/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/16305
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8
Gerrit-Change-Number: 16305
Gerrit-PatchSet: 9
Gerrit-Owner: Adam Tamas <ta...@cloudera.com>
Gerrit-Reviewer: Adam Tamas <ta...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 04 Sep 2020 08:57:28 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10012: ds hll sketch() results ascii codec decoding error fix
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16305 )
Change subject: IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix
......................................................................
IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix
While the ds_hll_sketch() generates a string value as output the data
is not an ascii encoded text but a bitsketch, because of this, when
the shell get this data it disconnect while it tries to decode it.
The issue can be reproduced with a simple method like using unhex
with a wrong input.
Example: SELECT unhex("aa");
This patch contains a solution, where we replace any not UTF-8
decodable characters if we run into an UnicodeDecodeError after
fetching it.
This solution is working with the Thrift 0.9.3 autogenerated gen-py
but still fails with Thrift 0.11.0.
For Thrift 0.11.0 the error is catched and an error message is sent
(not working with beeswax protocol, because it generates a different
error (TypeError) which can come for other reasons too).
Testing:
-manual testing with these protocols: 'hs2-http', 'hs2', 'beeswax'
Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8
Reviewed-on: http://gerrit.cloudera.org:8080/16305
Reviewed-by: Csaba Ringhofer <cs...@cloudera.com>
Tested-by: Csaba Ringhofer <cs...@cloudera.com>
---
M shell/impala_client.py
M shell/impala_shell.py
M shell/shell_output.py
3 files changed, 37 insertions(+), 5 deletions(-)
Approvals:
Csaba Ringhofer: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/16305
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8
Gerrit-Change-Number: 16305
Gerrit-PatchSet: 11
Gerrit-Owner: Adam Tamas <en...@gmail.com>
Gerrit-Reviewer: Adam Tamas <en...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-10012: ds hll sketch() results ascii codec decoding error fix
Posted by "Adam Tamas (Code Review)" <ge...@cloudera.org>.
Adam Tamas has uploaded a new patch set (#8). ( http://gerrit.cloudera.org:8080/16305 )
Change subject: IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix
......................................................................
IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix
While the ds_hll_sketch() generates a string value as output the data
is not an ascii encoded text but a bitsketch, because of this, when
the shell get this data it disconnect while it tries to decode it.
This patch contains a solution, where we replace any not UTF-8
decodable characters if we run into an UnicodeDecodeError after
fetching it.
This solution is working with the Thrift 0.9.3 autogenerated gen-py
but still fails with Thrift 0.11.0.
For Thrift 0.11.0 the error is catched and an error messenge is sent
(not working with beeswax protocol).
Testing:
-manual testing with these protocols: 'hs2-http', 'hs2', 'beeswax'
Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8
---
M shell/impala_client.py
M shell/impala_shell.py
M shell/shell_output.py
3 files changed, 37 insertions(+), 5 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/05/16305/8
--
To view, visit http://gerrit.cloudera.org:8080/16305
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8
Gerrit-Change-Number: 16305
Gerrit-PatchSet: 8
Gerrit-Owner: Adam Tamas <ta...@cloudera.com>
Gerrit-Reviewer: Adam Tamas <ta...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>