You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Tamas Mate (Jira)" <ji...@apache.org> on 2020/08/06 12:45:00 UTC
[jira] [Commented] (IMPALA-10051) impala-shell exits with
ValueError with WITH clauses
[ https://issues.apache.org/jira/browse/IMPALA-10051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17172327#comment-17172327 ]
Tamas Mate commented on IMPALA-10051:
-------------------------------------
In the ImpalaShell.do_with method shlex.split is used to tokenize the query so that it can be decided whether the query is DML or not.
However, whitespace bound tokens in shlex by default. Therefore, whitespace in a query string can cause the closure of a token prematurely before the final quote could be evaluated. Which triggers the 'No closing quotation' exception.
In this example the space after the + sign causes the issue.
impala-shell.sh -q 'with select regexp_replace(column_name,"[a-zA-Z]","+ ");'
> impala-shell exits with ValueError with WITH clauses
> ----------------------------------------------------
>
> Key: IMPALA-10051
> URL: https://issues.apache.org/jira/browse/IMPALA-10051
> Project: IMPALA
> Issue Type: Bug
> Components: Clients
> Affects Versions: Impala 4.0
> Reporter: Tamas Mate
> Assignee: Tamas Mate
> Priority: Major
>
> Some strings can cause shlex to throw an exception in WITH clauses, for example in a regexp_replace. This should be handled more gracefully and correctly.
> Working query (impala-shell forwards the query for analysis):
> {code:java}
> impala-shell.sh -q 'with select regexp_replace(column_name, "[a-zA-Z]", "+ ");'
> {code}
> While same query fails with ValueError when empty spaces are removed from the arguments of the regexp_replace:
> {code:java}
> tmate@tmate-box:~/Projects/Impala$ impala-shell.sh -q 'with select regexp_replace(column_name,"[a-zA-Z]","+ ");'
> Starting Impala Shell with no authentication using Python 2.7.16
> Warning: live_progress only applies to interactive shell sessions, and is being skipped for now.
> Opened TCP connection to localhost:21000
> Connected to localhost:21000
> Server version: impalad version 4.0.0-SNAPSHOT DEBUG (build b29cb4ca82a4f05ea7dc0eadc330a64fbe685ef0)
> Traceback (most recent call last):
> File "/home/tmate/Projects/Impala/shell/impala_shell.py", line 1973, in <module>
> impala_shell_main()
> File "/home/tmate/Projects/Impala/shell/impala_shell.py", line 1927, in impala_shell_main
> if execute_queries_non_interactive_mode(options, query_options):
> File "/home/tmate/Projects/Impala/shell/impala_shell.py", line 1731, in execute_queries_non_interactive_mode
> shell.execute_query_list(queries))
> File "/home/tmate/Projects/Impala/shell/impala_shell.py", line 1564, in execute_query_list
> if self.onecmd(q) is CmdStatus.ERROR:
> File "/home/tmate/Projects/Impala/shell/impala_shell.py", line 675, in onecmd
> return func(arg)
> File "/home/tmate/Projects/Impala/shell/impala_shell.py", line 1276, in do_with
> tokens = shlex.split(strip_comments(query.lstrip()), posix=False)
> File "/home/tmate/Projects/Impala/toolchain/toolchain-packages-gcc7.5.0/python-2.7.16/lib/python2.7/shlex.py", line 279, in split
> return list(lex)
> File "/home/tmate/Projects/Impala/toolchain/toolchain-packages-gcc7.5.0/python-2.7.16/lib/python2.7/shlex.py", line 269, in next
> token = self.get_token()
> File "/home/tmate/Projects/Impala/toolchain/toolchain-packages-gcc7.5.0/python-2.7.16/lib/python2.7/shlex.py", line 96, in get_token
> raw = self.read_token()
> File "/home/tmate/Projects/Impala/toolchain/toolchain-packages-gcc7.5.0/python-2.7.16/lib/python2.7/shlex.py", line 172, in read_token
> raise ValueError, "No closing quotation"
> ValueError: No closing quotation
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org