[impala] branch master updated: IMPALA-8652 Illegal delimiter error in shell has unknown error

[ta...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

tarmstrong pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git


The following commit(s) were added to refs/heads/master by this push:
     new 3a4f8b3  IMPALA-8652 Illegal delimiter error in shell has unknown error
3a4f8b3 is described below

commit 3a4f8b3ae1a293fb350778a44ac9db6a6036d33b
Author: Jiawei Wang <ji...@cloudera.com>
AuthorDate: Thu Jun 20 14:43:32 2019 -0700

    IMPALA-8652 Illegal delimiter error in shell has unknown error
    
    Problem:
    When assign --output_delimiter to invalid value, the validation of
    the argument is done only after the query is running, ValueError is
    raised in DelimitedOutputFormatter and caught in _exec_stmt in shell
    
    Solution:
    Add --output_delimiter option check before impala-shell initialization
    Remove delimiter length check in DelimitedOutputFormatter
    
    Testing:
    tests/shell/test_shell_commandline.py passed
    
    Example:
    $ impala-shell.sh -B --output_delimiter '||' -q 'select 1,1,1'
    Illegal delimiter ||, the delimiter must be a 1-character string.
    
    Change-Id: I7ee2fccd305b104b3aff44c57659b6f14f2f4a05
    Reviewed-on: http://gerrit.cloudera.org:8080/13690
    Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
    Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
 shell/impala_shell.py | 7 +++++++
 shell/shell_output.py | 6 ++----
 2 files changed, 9 insertions(+), 4 deletions(-)

diff --git a/shell/impala_shell.py b/shell/impala_shell.py
index 553e787..1fdd86e 100755
--- a/shell/impala_shell.py
+++ b/shell/impala_shell.py
@@ -1729,6 +1729,13 @@ def impala_shell_main():
     print VERSION_STRING
     return
 
+  if options.write_delimited:
+    delim = options.output_delimiter.decode('string-escape')
+    if len(delim) != 1:
+      print_to_stderr("Illegal delimiter %s, the delimiter "
+                      "must be a 1-character string." % delim)
+      raise FatalShellException()
+
   if options.use_kerberos and options.use_ldap:
     print_to_stderr("Please specify at most one authentication mechanism (-k or -l)")
     raise FatalShellException()
diff --git a/shell/shell_output.py b/shell/shell_output.py
index 8ab3bee..190e8bc 100644
--- a/shell/shell_output.py
+++ b/shell/shell_output.py
@@ -52,10 +52,8 @@ class DelimitedOutputFormatter(object):
   def __init__(self, field_delim="\t"):
     if field_delim:
       self.field_delim = field_delim.decode('string-escape')
-      if len(self.field_delim) != 1:
-        error_msg = ("Illegal delimiter %s, the delimiter "
-                     "must be a 1-character string." % self.field_delim)
-        raise ValueError, error_msg
+      # IMPALA-8652, the delimiter should be a 1-character string and verified already
+      assert len(self.field_delim) == 1
 
   def format(self, rows):
     """Returns string containing UTF-8-encoded representation of the table data."""