You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Johannes Hugo Kitschke <jo...@rwth-aachen.de> on 2015/04/15 10:40:25 UTC
ShellBolt / Bash invocation: Source environment variables
Hi,
I am currently experiencing a problem when submitting a topology built
with pyleus to my storm cluster:
java.lang.RuntimeException: Error when launching multilang subprocess
pyleus_venv/bin/python: error while loading shared libraries:
libpython2.7.so.1.0: cannot open shared object file: No such file or
directory
Maybe I know what causes this error: On our cluster I have to execute
'module load python' to setup the environment for python. I have this in
my .bashrc but my assumption is, that .bashrc does not get sourced when
setting up the ShellBot (is this true?). This is based on the following
observation: Local mode performs as expected, unless python module is
not loaded. Then it fails with the error above.
Long story short: Which shell is used? Which files are sourced when the
shell is launched for the ShellBolts?
Johannes
Re: ShellBolt / Bash invocation: Source environment variables
Posted by Johannes Hugo Kitschke <jo...@rwth-aachen.de>.
To answer my own question (maybe it's useful for someone else): My
findings are:
- bash is launched as non-login, non-interactive shell for the shell bolt
- in this mode bash looks for a path/file in the environment variable
'BASH_ENV', if it exists that file is sourced
- now I only had to create a file that loads the module and point the
env. var. to the location of the file
That's it. Now it works.
On 04/15/2015 10:40 AM, Johannes Hugo Kitschke wrote:
> Hi,
>
> I am currently experiencing a problem when submitting a topology built
> with pyleus to my storm cluster:
>
> java.lang.RuntimeException: Error when launching multilang subprocess
> pyleus_venv/bin/python: error while loading shared libraries:
> libpython2.7.so.1.0: cannot open shared object file: No such file or
> directory
>
> Maybe I know what causes this error: On our cluster I have to execute
> 'module load python' to setup the environment for python. I have this
> in my .bashrc but my assumption is, that .bashrc does not get sourced
> when setting up the ShellBot (is this true?). This is based on the
> following observation: Local mode performs as expected, unless python
> module is not loaded. Then it fails with the error above.
>
> Long story short: Which shell is used? Which files are sourced when
> the shell is launched for the ShellBolts?
>
> Johannes