[GitHub] fhieber opened a new issue #8532: mxnet-mkl (v0.12.0) crash when using (conda-installed) numpy with MKL

[GitBox <gi...@apache.org>]

fhieber opened a new issue #8532: mxnet-mkl (v0.12.0) crash when using (conda-installed) numpy with MKL
URL: https://github.com/apache/incubator-mxnet/issues/8532
 
 
   We have observed crashes with any mkl-enabled pip package of mxnet-0.12.0 in combination with numpy if installed through conda (which by default also uses MKL).
   
   In this case, mxnet trainings crash with the following error message:
   ```
   OMP: Error #15: Initializing libiomp5.so, but found libiomp5.so already initialized.
   OMP: Hint: This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you
   OMP: Hint: This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://www.intel.com/software/products/support/.
   ```
   
   Numpy from conda links against the libmkl_rt.so, distributed through conda:
   ```
   libmkl_rt.so => /opt/conda/lib/python3.6/site-packages/numpy/core/../../../../libmkl_rt.so (0x00007f05256e0000)
   	libmkl_rt.so => /opt/conda/lib/python3.6/site-packages/numpy/linalg/../../../../libmkl_rt.so (0x00007f367e1d5000)
   	libmkl_rt.so => /opt/conda/lib/python3.6/site-packages/numpy/linalg/../../../../libmkl_rt.so (0x00007fd39e751000)
   ```
   
   whereas MXNet links to its own .so:
   ```
   ldd /opt/conda/lib/python3.6/site-packages/mxnet/libmxnet.so
   [...]
   	libmklml_intel.so => /opt/conda/lib/python3.6/site-packages/mxnet/libmklml_intel.so (0x00007f8c85b94000)
   	libiomp5.so => /opt/conda/lib/python3.6/site-packages/mxnet/libiomp5.so (0x00007f8c857f1000)
   [...]
   ```
   
   This prevents people from using numpy w/ MKL in combination with mxnet-mkl==0.12.0.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services