[jira] [Commented] (IMPALA-2195) Improper handling of comments in queries

["ASF subversion and git services (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16798505#comment-16798505 ] 

ASF subversion and git services commented on IMPALA-2195:
---------------------------------------------------------

Commit 9ad9a1624a3ccc3caadab6d7dd4eb7de96e110dd in impala's branch refs/heads/master from Andrew Sherman
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=9ad9a16 ]

IMPALA-8325: Leading Unicode comments cause Impala Shell failure.

This change fixes a regression introduced by "IMPALA-2195 Improper
handling of comments in queries."

The Impala Shell parses input text into several strings using the
sqlparse library. One of the returned strings is the sql command, this
is used to determine the correct do_<command> method to call. Another of
the returned strings is the leading comment, which is a comment that
appears before legal sql text.

Python2 has strings with multiple encodings. The strings returned from
the sqlparse library have the Unicode encoding. Impala Shell converts
the sql command string to utf-8 encoding before using it.

If the Impala Shell needs to send the sql command to an Impala
Coordinator then it (re)constructs the query out of the strings
returned by the sqlparse library. This query is sent to the Coordinator
via Beeswax protocol. The query is converted to an ascii string before
being sent. The conversion can fail if the leading comment string
contains Unicode characters, which can't be directly converted to ascii.
So the trigger for the bug is that the leading comment contains Unicode.

The fix is that the leading comment string should be converted to utf-8
in the same way as the sql command.

TESTING:

Ran all end -to-end tests.
Added two test cases to tests/shell/test_shell_interactive.py

Change-Id: I8633935b6e0ca33594afd32ad242779555e09944
Reviewed-on: http://gerrit.cloudera.org:8080/12812
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>


> Improper handling of comments in queries
> ----------------------------------------
>
>                 Key: IMPALA-2195
>                 URL: https://issues.apache.org/jira/browse/IMPALA-2195
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Clients
>    Affects Versions: Impala 2.2, Impala 2.3.0
>            Reporter: bharath v
>            Assignee: Fredy Wijaya
>            Priority: Minor
>              Labels: shell
>             Fix For: Impala 2.13.0, Impala 3.1.0
>
>
> Shell doesn't handle comments properly while executing the queries. For example, 
> {noformat}
> [impalad:21000] > ----- random comment 1
> -----### random comment 2
> insert overwrite test2 select * from test1;
> Query: ----- random comment 1
> -----### random comment 2
> insert overwrite test2 select * from test1
> Fetched 0 row(s) in 3.53s
> {noformat}
> Stderr for an insert query shows up as "Fetched 0 row(s)" instead of "Inserted 1 row(s)" . This happens because the shell parser doesn't detect this as an insert query due to the presence of comments.  We don't strip the comments before sending them to Cmd control, due which the default() method is applied instead of do_insert().



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org