You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (JIRA)" <ji...@apache.org> on 2019/03/23 23:39:00 UTC
[jira] [Commented] (IMPALA-6223) Gracefully handle malformed 'with'
queries in impala-shell
[ https://issues.apache.org/jira/browse/IMPALA-6223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16799846#comment-16799846 ]
ASF subversion and git services commented on IMPALA-6223:
---------------------------------------------------------
Commit 1aae0c6b4e7d835a461dd0fcc2a418a8269b645b in impala's branch refs/heads/2.x from poojanilangekar
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=1aae0c6 ]
IMPALA-6223: Gracefully handle malformed 'with' queries in impala-shell
The change handles the exception thrown by shlex while parsing a
malformed query.
This patch was tested by adding both commandline and interactive
shell tests.
Change-Id: Ibb1e9238ac67b8ad3b2caa1748a18b04f384802d
Reviewed-on: http://gerrit.cloudera.org:8080/10876
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-on: http://gerrit.cloudera.org:8080/12800
Reviewed-by: Fredy Wijaya <fw...@cloudera.com>
> Gracefully handle malformed 'with' queries in impala-shell
> ----------------------------------------------------------
>
> Key: IMPALA-6223
> URL: https://issues.apache.org/jira/browse/IMPALA-6223
> Project: IMPALA
> Issue Type: Bug
> Components: Clients
> Affects Versions: Impala 2.10.0
> Reporter: bharath v
> Assignee: Pooja Nilangekar
> Priority: Minor
> Labels: newbie
>
> Impala shell can throw a lexer error if it encounters a malformed "with" query.
> {noformat}
> impala-shell.sh -q "with foo as (select bar from temp where temp.a='"
> Starting Impala Shell without Kerberos authentication
> Connected to localhost:21000
> Server version: impalad version 2.11.0-SNAPSHOT DEBUG (build 0ee1765f38082bc5c10aa37b23cb8e57caa57d4e)
> Traceback (most recent call last):
> File "/home/bharath/Impala/shell/impala_shell.py", line 1463, in <module>
> execute_queries_non_interactive_mode(options, query_options)
> File "/home/bharath/Impala/shell/impala_shell.py", line 1338, in execute_queries_non_interactive_mode
> shell.execute_query_list(queries)):
> File "/home/bharath/Impala/shell/impala_shell.py", line 1218, in execute_query_list
> if self.onecmd(q) is CmdStatus.ERROR:
> File "/home/bharath/Impala/shell/impala_shell.py", line 505, in onecmd
> return cmd.Cmd.onecmd(self, line)
> File "/usr/lib/python2.7/cmd.py", line 221, in onecmd
> return func(arg)
> File "/home/bharath/Impala/shell/impala_shell.py", line 1024, in do_with
> tokens = list(lexer)
> File "/usr/lib/python2.7/shlex.py", line 269, in next
> token = self.get_token()
> File "/usr/lib/python2.7/shlex.py", line 96, in get_token
> raw = self.read_token()
> File "/usr/lib/python2.7/shlex.py", line 172, in read_token
> raise ValueError, "No closing quotation"
> ValueError: No closing quotation
> {noformat}
> This happens because we use shlex to parse the input query to determine if its a DML and it can throw if the input doesn't have balanced quotes.
> {noformat}
> def do_with(self, args):
> """Executes a query with a WITH clause, fetching all rows"""
> query = self.imp_client.create_beeswax_query("with %s" % args,
> self.set_query_options)
> # Set posix=True and add "'" to escaped quotes
> # to deal with escaped quotes in string literals
> lexer = shlex.shlex(query.query.lstrip(), posix=True)
> lexer.escapedquotes += "'"
> # Because the WITH clause may precede DML or SELECT queries,
> # just checking the first token is insufficient.
> is_dml = False
> tokens = list(lexer) <----
> {noformat}
> A simple shlex repro of that is as follows,
> {noformat}
> >>> lexer = shlex.shlex("with foo as (select bar from temp where temp.a='", posix=True);
> >>> list(lexer)
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File "/usr/lib/python2.7/shlex.py", line 269, in next
> token = self.get_token()
> File "/usr/lib/python2.7/shlex.py", line 96, in get_token
> raw = self.read_token()
> File "/usr/lib/python2.7/shlex.py", line 172, in read_token
> raise ValueError, "No closing quotation"
> ValueError: No closing quotation
> {noformat}
> Fix: Either catch the exception and handle it gracefully or have a better way to figure out the query type, using a SQL parser (more involved).
> This query also repros it:
> {code}
> with v as (select 1)
> select foo('\\'), ('bar
> ;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org