You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2017/12/07 04:27:17 UTC
[GitHub] popfido opened a new issue #8976: static TLS errors from jemalloc 5 built on CentOS 7
popfido opened a new issue #8976: static TLS errors from jemalloc 5 built on CentOS 7
URL: https://github.com/apache/incubator-mxnet/issues/8976
## Description
when enable USE_JEMALLOC with jemalloc 5.0.0(same for 5.0.1) to built MXNet from source on CentOS 7 with devtoolset-3, the compiling program will stop and produce an TLS error when linking libmxnet.so with python OpWrapper
## Environment info
----------Python Info----------
Version : 3.6.3
Compiler : GCC 7.2.0
Build : ('default', 'Nov 20 2017 20:41:42')
Arch : ('64bit', '')
------------Pip Info-----------
Version : 9.0.1
Directory : /data/anaconda2/envs/fidoenv3/lib/python3.6/site-packages/pip
----------MXNet Info-----------
Traceback (most recent call last):
File "tools/diagnose.py", line 103, in check_mxnet
import mxnet
File "/home/rd/mxnet/python/mxnet/__init__.py", line 25, in <module>
from . import engine
File "/home/rd/mxnet/python/mxnet/engine.py", line 23, in <module>
from .base import _LIB, check_call
File "/home/rd/mxnet/python/mxnet/base.py", line 111, in <module>
_LIB = _load_lib()
File "/home/rd/mxnet/python/mxnet/base.py", line 103, in _load_lib
lib = ctypes.CDLL(lib_path[0], ctypes.RTLD_LOCAL)
File "/data/anaconda2/envs/fidoenv3/lib/python3.6/ctypes/__init__.py", line 348, in __init__
self._handle = _dlopen(self._name, mode)
OSError: /usr/local/lib/libjemalloc.so.2: cannot allocate memory in static TLS block
----------System Info----------
Platform : Linux-3.10.0-514.16.1.el7.x86_64-x86_64-with-centos-7.3.1611-Core
system : Linux
node : dm-dl-test-bjd-001
release : 3.10.0-514.16.1.el7.x86_64
version : #1 SMP Wed Apr 12 15:04:24 UTC 2017
----------Hardware Info----------
machine : x86_64
processor : x86_64
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Thread(s) per core: 1
Core(s) per socket: 1
Socket(s): 4
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 63
Model name: Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz
Stepping: 2
CPU MHz: 2394.454
BogoMIPS: 4788.90
Hypervisor vendor: KVM
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 20480K
NUMA node0 CPU(s): 0-3
----------Network Test----------
Setting timeout: 10
Timing for MXNet: https://github.com/apache/incubator-mxnet, DNS: 0.0427 sec, LOAD: 1.3971 sec.
Timing for Gluon Tutorial(en): http://gluon.mxnet.io, DNS: 1.1900 sec, LOAD: 1.1925 sec.
Timing for Gluon Tutorial(cn): https://zh.gluon.ai, DNS: 3.4383 sec, LOAD: 3.6338 sec.
Timing for FashionMNIST: https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz, DNS: 0.8646 sec, LOAD: 2.3594 sec.
Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.0453 sec, LOAD: 0.8300 sec.
Timing for Conda: https://repo.continuum.io/pkgs/free/, DNS: 0.2181 sec, LOAD: 1.0059 sec.
## Build info
Compiler (gcc/clang/mingw/visual studio): gcc 4.9.2
MXNet commit hash:
branch v1.0.0
Build config:
USE_CUDA = 1
USE_CUDNN = 1
USE_NVRTC = 1
USE_OPENCV = 0
USE_OPENMP = 1
USE_MKL2017 = 1
USE_MKL2017_EXPERIMENTAL = 1
USE_NNPACK = 1
USE_LAPACK = 1
USE_OPERATOR_TUNING = 1
USE_JEMALLOC = 1
## Error Message:
cd /home/rd/mxnet/dmlc-core; make libdmlc.a USE_SSE=1 config=/home/rd/mxnet/config.mk; cd /home/rd/mxnet
make[1]: Entering directory `/home/rd/mxnet/dmlc-core'
make[1]: `libdmlc.a' is up to date.
make[1]: Leaving directory `/home/rd/mxnet/dmlc-core'
(cd cpp-package/scripts; python OpWrapperGenerator.py /home/rd/mxnet/lib/libmxnet.so)
Traceback (most recent call last):
File "OpWrapperGenerator.py", line 425, in <module>
raise(e)
File "OpWrapperGenerator.py", line 419, in <module>
f.write(patternStr % ParseAllOps())
File "OpWrapperGenerator.py", line 314, in ParseAllOps
cdll.libmxnet = cdll.LoadLibrary(sys.argv[1])
File "/data/anaconda2/envs/fidoenv3/lib/python3.6/ctypes/__init__.py", line 426, in LoadLibrary
return self._dlltype(name)
File "/data/anaconda2/envs/fidoenv3/lib/python3.6/ctypes/__init__.py", line 348, in __init__
self._handle = _dlopen(self._name, mode)
OSError: /usr/local/lib/libjemalloc.so.2: cannot allocate memory in static TLS block
make: *** [cpp-package/include/mxnet-cpp/op.h] Error 1
## Minimum reproducible example
(If you are using your own code, please provide a short script that reproduces the error. Otherwise, please provide link to the existing example.)
## Steps to reproduce
(Paste the commands you ran that produced the error.)
1. git clone https://github.com/apache/incubator-mxnet
2. cd incubator-mxnet
3. make -jX
## What have you tried to solve it?
1. Use different version of jemalloc: 5.0.1 doesn't works, 4.5.0 works
2. Refer [jemalloc#937](https://github.com/jemalloc/jemalloc/issues/937)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services