[GitHub] yuantangliang opened a new issue #9520: Image augumention crash when set MXNET_CPU_WORKER_NTHREADS bigger than 3

[GitBox <gi...@apache.org>]

yuantangliang opened a new issue #9520: Image augumention crash when set   MXNET_CPU_WORKER_NTHREADS bigger than 3
URL: https://github.com/apache/incubator-mxnet/issues/9520
 
 
   Note: Providing complete information in the most concise form is the best way to get help. This issue template serves as the checklist for essential information to most of the technical issues and bug reports. For non-technical issues and feature requests, feel free to present the information in what you believe is the best form.
   
   For Q & A and discussion, please start a discussion thread at https://discuss.mxnet.io 
   
   ## Description
   Image augumention crash when set   MXNET_CPU_WORKER_NTHREADS bigger than 3
   
   ## Environment info (Required)
   
   ----------Python Info----------
   ('Version      :', '2.7.6')
   ('Compiler     :', 'GCC 4.8.4')
   ('Build        :', ('default', 'Oct 26 2016 20:30:19'))
   ('Arch         :', ('64bit', 'ELF'))
   ------------Pip Info-----------
   ('Version      :', '9.0.1')
   ('Directory    :', '/usr/local/lib/python2.7/dist-packages/pip')
   ----------MXNet Info-----------
   ('Version      :', '0.12.1')
   ('Directory    :', '/usr/local/lib/python2.7/dist-packages/mxnet')
   ('Commit Hash   :', 'e0c7906693f0c79b0ce34a4d777c26a6bf1903c1')
   ----------System Info----------
   ('Platform     :', 'Linux-4.4.0-64-generic-x86_64-with-Ubuntu-14.04-trusty')
   ('system       :', 'Linux')
   ('node         :', 'meter')
   ('release      :', '4.4.0-64-generic')
   ('version      :', '#85~14.04.1-Ubuntu SMP Mon Feb 20 12:10:54 UTC 2017')
   ----------Hardware Info----------
   ('machine      :', 'x86_64')
   ('processor    :', 'x86_64')
   Architecture:          x86_64
   CPU op-mode(s):        32-bit, 64-bit
   Byte Order:            Little Endian
   CPU(s):                8
   On-line CPU(s) list:   0-7
   Thread(s) per core:    2
   Core(s) per socket:    4
   Socket(s):             1
   NUMA node(s):          1
   Vendor ID:             GenuineIntel
   CPU family:            6
   Model:                 94
   Stepping:              3
   CPU MHz:               4200.000
   BogoMIPS:              8016.71
   Virtualization:        VT-x
   L1d cache:             32K
   L1i cache:             32K
   L2 cache:              256K
   L3 cache:              8192K
   NUMA node0 CPU(s):     0-7
   ----------Network Test----------
   Setting timeout: 10
   Timing for MXNet: https://github.com/apache/incubator-mxnet, DNS: 0.0072 sec, LOAD: 1.6354 sec.
   Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.0060 sec, LOAD: 0.5680 sec.
   Timing for FashionMNIST: https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz, DNS: 0.1315 sec, LOAD: 0.9401 sec.
   Timing for Conda: https://repo.continuum.io/pkgs/free/, DNS: 0.1766 sec, LOAD: 1.0127 sec.
   Timing for Gluon Tutorial(en): http://gluon.mxnet.io, DNS: 0.0067 sec, LOAD: 0.3688 sec.
   Error open Gluon Tutorial(cn): https://zh.gluon.ai, <urlopen error [Errno 1] _ssl.c:510: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure>, DNS finished in 0.277799129486 sec.
   
   
   Package used (Python/R/Scala/Julia):
   Python 
   
   ## Error Message:
   BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
   
   ## Minimum reproducible example
   ```
   default_train_augument = {}
   
   default_train_augument['mean'] = np.array([123.68, 116.28, 103.53])
   default_train_augument['rand_crop'] = 1 # random crop ????
   default_train_augument['rand_mirror'] = 1
   default_train_augument['rand_pad'] = 0.65 # pad ???????
   default_train_augument['rand_gray'] = 0.2
   default_train_augument['brightness'] = 0.7#
   default_train_augument['contrast'] = 0.7 # ????
   default_train_augument['saturation'] = 0.7 #
   default_train_augument['pca_noise'] = 0.7 #
   default_train_augument['hue'] = 0.7 #
   default_train_augument['min_object_covered'] = 0
   default_train_augument['aspect_ratio_range'] =(0.8, 1.22)
   default_train_augument['area_range'] =(0.3, 2.4)
   default_train_augument['min_eject_coverage'] = 0.4 # ??????box?????
   
   os.environ["MXNET_CPU_WORKER_NTHREADS"] = "%d" % num_worker
   
   def create_mx_det_iter():
       file_name = os.path.join(RECORD_PERSON_ROOPATH, 'person_test.rec')
       id_file_name = os.path.join(RECORD_PERSON_ROOPATH, 'person_test.idx')
       iter1 = ImageDetIter(20,(3,480,640), path_imgrec=file_name, path_imgidx=id_file_name, **default_train_augument)
       return iter1
   
   def record_iterator_test_all(det_iter,batch_size=32):
       import time
       import mxnet as mx
       mx.profiler
       i = 0
       det_iter.reset()
       tic = time.time()
       for batch in det_iter:
           i+=1
           print(batch_size * i / (time.time() - tic))
   
   det_iter = create_mx_det_iter()
   record_iterator_test_all(det_iter,32)
   
   ````
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services