You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2022/05/26 13:22:00 UTC
[jira] [Commented] (IMPALA-11251) TestImpalaShellInteractive.test_unicode_input fails on Ubuntu 20
[ https://issues.apache.org/jira/browse/IMPALA-11251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17542481#comment-17542481 ]
ASF subversion and git services commented on IMPALA-11251:
----------------------------------------------------------
Commit f77f57707417379635880efa1caa1ec40f2314b2 in impala's branch refs/heads/master from Joe McDonnell
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=f77f57707 ]
IMPALA-11251: Fix TestImpalaShellInteractive.test_unicode_input on Ubuntu 20
TestImpalaShellInteractive.test_unicode_input() was spawning
impala-shell using a command that preserves the environment. This
is a problem on Ubuntu 20, because Ubuntu 20 uses a newer GCC/libstdc++.
Preserving the environment keeps an LD_LIBRARY_PATH setting that
uses an older libstdc++ and causes the shell to fail at startup.
This switches the test to use shell/util.py's spawn_shell(),
which cleans up the environment before running the shell.
Testing:
- Ran tests on Ubuntu 20
Change-Id: Ib07f557ab3c21d6b39f814dcfc0bf9eb1b61f090
Reviewed-on: http://gerrit.cloudera.org:8080/18558
Reviewed-by: Joe McDonnell <jo...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> TestImpalaShellInteractive.test_unicode_input fails on Ubuntu 20
> ----------------------------------------------------------------
>
> Key: IMPALA-11251
> URL: https://issues.apache.org/jira/browse/IMPALA-11251
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Affects Versions: Impala 4.1.0
> Reporter: Joe McDonnell
> Priority: Major
>
> Running TestImpalaShellInteractive.test_unicode_input() on Ubuntu results in this failure:
>
> {noformat}
> shell/test_shell_interactive.py:377: in test_unicode_input
> child_proc.expect(PROMPT_REGEX)
> ../infra/python/env-gcc7.5.0/lib/python2.7/site-packages/pexpect/__init__.py:1451: in expect
> timeout, searchwindowsize)
> ../infra/python/env-gcc7.5.0/lib/python2.7/site-packages/pexpect/__init__.py:1466: in expect_list
> timeout, searchwindowsize)
> ../infra/python/env-gcc7.5.0/lib/python2.7/site-packages/pexpect/__init__.py:1554: in expect_loop
> raise EOF(str(err) + '\n' + str(self))
> E EOF: End Of File (EOF). Exception style platform.
> E <pexpect.spawn object at 0x7fe089a9e890>
> E version: 3.3
> E command: /home/impdev/Impala/shell/build/impala-shell-4.1.0-SNAPSHOT/impala-shell
> E args: ['/home/impdev/Impala/shell/build/impala-shell-4.1.0-SNAPSHOT/impala-shell', '--protocol=beeswax', '-ilocalhost:21000']
> E searcher: <pexpect.searcher_re object at 0x7fe089a9ec10>
> E buffer (last 100 chars): ''
> E before (last 100 chars): ' (required by /home/impdev/.python-eggs/sasl-0.2.1-py2.7-linux-x86_64.egg-tmp/sasl/saslwrapper.so)\r\n'
> E after: <class 'pexpect.EOF'>
> E match: None
> E match_index: None
> E exitstatus: None
> E flag_eof: True
> E pid: 1109550
> E child_fd: 23
> E closed: False
> E timeout: 30
> E delimiter: <class 'pexpect.EOF'>
> E logfile: None
> E logfile_read: None
> E logfile_send: None
> E maxread: 2000
> E ignorecase: False
> E searchwindowsize: None
> E delaybeforesend: 0.05
> E delayafterclose: 0.1
> E delayafterterminate: 0.1{noformat}
> What seems to be happening is that impala-shell is built against the system compiler / system libstdc++ (GCC 9). If we then run it with LD_LIBRARY_PATH set to point to toolchain compiler / libstdc++, then it won't find the symbols it needs to run. The solution is to use shell/util.py's spawn_shell function, which cleans up the environment for the pexpect:
>
>
> {noformat}
> def build_shell_env(env=None):
> """ Construct the environment for the shell to run in based on 'env', or the current
> process's environment if env is None."""
> if not env: env = os.environ
> # Don't inherit PYTHONPATH or LD_LIBRARY_PATH - the shell launch script must set
> # these to include dependencies. Copy 'env' to avoid mutating argument or os.environ.
> env = dict(env)
> if "PYTHONPATH" in env:
> del env["PYTHONPATH"]
> if "LD_LIBRARY_PATH" in env:
> del env["LD_LIBRARY_PATH"]
> return env
> ...
> def spawn_shell(shell_cmd):
> """Spawn a shell process with the provided command line. Returns the Pexpect object."""
> return pexpect.spawn(shell_cmd[0], shell_cmd[1:], env=build_shell_env()){noformat}
>
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org