[jira] [Created] (IMPALA-6223) Gracefully handle malformed 'with' queries in impala-shell

["bharath v (JIRA)" <ji...@apache.org>]

bharath v created IMPALA-6223:
---------------------------------

             Summary: Gracefully handle malformed 'with' queries in impala-shell
                 Key: IMPALA-6223
                 URL: https://issues.apache.org/jira/browse/IMPALA-6223
             Project: IMPALA
          Issue Type: Bug
          Components: Clients
    Affects Versions: Impala 2.10.0
            Reporter: bharath v
            Priority: Minor


Impala shell can throw a lexer error if it encounters a malformed "with" query.

{noformat}
impala-shell.sh -q "with foo as (select bar from temp where temp.a='"
Starting Impala Shell without Kerberos authentication
Connected to localhost:21000
Server version: impalad version 2.11.0-SNAPSHOT DEBUG (build 0ee1765f38082bc5c10aa37b23cb8e57caa57d4e)
Traceback (most recent call last):
  File "/home/bharath/Impala/shell/impala_shell.py", line 1463, in <module>
    execute_queries_non_interactive_mode(options, query_options)
  File "/home/bharath/Impala/shell/impala_shell.py", line 1338, in execute_queries_non_interactive_mode
    shell.execute_query_list(queries)):
  File "/home/bharath/Impala/shell/impala_shell.py", line 1218, in execute_query_list
    if self.onecmd(q) is CmdStatus.ERROR:
  File "/home/bharath/Impala/shell/impala_shell.py", line 505, in onecmd
    return cmd.Cmd.onecmd(self, line)
  File "/usr/lib/python2.7/cmd.py", line 221, in onecmd
    return func(arg)
  File "/home/bharath/Impala/shell/impala_shell.py", line 1024, in do_with
    tokens = list(lexer)
  File "/usr/lib/python2.7/shlex.py", line 269, in next
    token = self.get_token()
  File "/usr/lib/python2.7/shlex.py", line 96, in get_token
    raw = self.read_token()
  File "/usr/lib/python2.7/shlex.py", line 172, in read_token
    raise ValueError, "No closing quotation"
ValueError: No closing quotation
{noformat}

This happens because we use shlex to parse the input query to determine if its a DML and it can throw if the input doesn't have balanced quotes.

{noformat}
def do_with(self, args):
    """Executes a query with a WITH clause, fetching all rows"""
    query = self.imp_client.create_beeswax_query("with %s" % args,
                                                 self.set_query_options)
    # Set posix=True and add "'" to escaped quotes
    # to deal with escaped quotes in string literals
    lexer = shlex.shlex(query.query.lstrip(), posix=True)
    lexer.escapedquotes += "'"
    # Because the WITH clause may precede DML or SELECT queries,
    # just checking the first token is insufficient.
    is_dml = False
    tokens = list(lexer)  <----
{noformat}

A simple shlex repro of that is as follows,

{noformat}
>>> lexer = shlex.shlex("with foo as (select bar from temp where temp.a='", posix=True);
>>> list(lexer)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/shlex.py", line 269, in next
    token = self.get_token()
  File "/usr/lib/python2.7/shlex.py", line 96, in get_token
    raw = self.read_token()
  File "/usr/lib/python2.7/shlex.py", line 172, in read_token
    raise ValueError, "No closing quotation"
ValueError: No closing quotation
{noformat}

Fix: Either catch the exception and handle it gracefully or have a better way to figure out the query type, using a SQL parser (more involved).





--
This message was sent by Atlassian JIRA
(v6.4.14#64029)