[jira] [Resolved] (IMPALA-11251) TestImpalaShellInteractive.test_unicode_input fails on Ubuntu 20

["Joe McDonnell (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-11251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Joe McDonnell resolved IMPALA-11251.
------------------------------------
    Fix Version/s: Impala 4.2.0
       Resolution: Fixed

> TestImpalaShellInteractive.test_unicode_input fails on Ubuntu 20
> ----------------------------------------------------------------
>
>                 Key: IMPALA-11251
>                 URL: https://issues.apache.org/jira/browse/IMPALA-11251
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Infrastructure
>    Affects Versions: Impala 4.1.0
>            Reporter: Joe McDonnell
>            Assignee: Joe McDonnell
>            Priority: Major
>             Fix For: Impala 4.2.0
>
>
> Running TestImpalaShellInteractive.test_unicode_input() on Ubuntu results in this failure:
>  
> {noformat}
> shell/test_shell_interactive.py:377: in test_unicode_input
>     child_proc.expect(PROMPT_REGEX)
> ../infra/python/env-gcc7.5.0/lib/python2.7/site-packages/pexpect/__init__.py:1451: in expect
>     timeout, searchwindowsize)
> ../infra/python/env-gcc7.5.0/lib/python2.7/site-packages/pexpect/__init__.py:1466: in expect_list
>     timeout, searchwindowsize)
> ../infra/python/env-gcc7.5.0/lib/python2.7/site-packages/pexpect/__init__.py:1554: in expect_loop
>     raise EOF(str(err) + '\n' + str(self))
> E   EOF: End Of File (EOF). Exception style platform.
> E   <pexpect.spawn object at 0x7fe089a9e890>
> E   version: 3.3
> E   command: /home/impdev/Impala/shell/build/impala-shell-4.1.0-SNAPSHOT/impala-shell
> E   args: ['/home/impdev/Impala/shell/build/impala-shell-4.1.0-SNAPSHOT/impala-shell', '--protocol=beeswax', '-ilocalhost:21000']
> E   searcher: <pexpect.searcher_re object at 0x7fe089a9ec10>
> E   buffer (last 100 chars): ''
> E   before (last 100 chars): ' (required by /home/impdev/.python-eggs/sasl-0.2.1-py2.7-linux-x86_64.egg-tmp/sasl/saslwrapper.so)\r\n'
> E   after: <class 'pexpect.EOF'>
> E   match: None
> E   match_index: None
> E   exitstatus: None
> E   flag_eof: True
> E   pid: 1109550
> E   child_fd: 23
> E   closed: False
> E   timeout: 30
> E   delimiter: <class 'pexpect.EOF'>
> E   logfile: None
> E   logfile_read: None
> E   logfile_send: None
> E   maxread: 2000
> E   ignorecase: False
> E   searchwindowsize: None
> E   delaybeforesend: 0.05
> E   delayafterclose: 0.1
> E   delayafterterminate: 0.1{noformat}
> What seems to be happening is that impala-shell is built against the system compiler / system libstdc++ (GCC 9). If we then run it with LD_LIBRARY_PATH set to point to toolchain compiler / libstdc++, then it won't find the symbols it needs to run. The solution is to use shell/util.py's spawn_shell function, which cleans up the environment for the pexpect:
>  
>  
> {noformat}
> def build_shell_env(env=None):
>   """ Construct the environment for the shell to run in based on 'env', or the current
>   process's environment if env is None."""
>   if not env: env = os.environ
>   # Don't inherit PYTHONPATH or LD_LIBRARY_PATH - the shell launch script must set
>   # these to include dependencies. Copy 'env' to avoid mutating argument or os.environ.
>   env = dict(env)
>   if "PYTHONPATH" in env:
>     del env["PYTHONPATH"]
>   if "LD_LIBRARY_PATH" in env:
>     del env["LD_LIBRARY_PATH"]
>   return env
> ...
> def spawn_shell(shell_cmd):
>   """Spawn a shell process with the provided command line. Returns the Pexpect object."""
>   return pexpect.spawn(shell_cmd[0], shell_cmd[1:], env=build_shell_env()){noformat}
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.7#820007)