You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2019/07/17 18:18:02 UTC

[GitHub] [incubator-mxnet] arcadiaphy opened a new issue #15576: Multi-threaded inference broken with MKLDNN

arcadiaphy opened a new issue #15576: Multi-threaded inference broken with MKLDNN
URL: https://github.com/apache/incubator-mxnet/issues/15576
 
 
   ## Description
   I want to do multi-threaded inference with shared model parameters, so I'm testing the `MXPredCreateMultiThread` API in [cpp example](https://github.com/apache/incubator-mxnet/tree/master/example/image-classification/predict-cpp). I find that the example is broken with more than 1 threads on MKLDNN build: the output of model inference is not deterministic. If I run it with openblas build, everything is normal.
   
   ## Environment info (Required)
   
   ```
   ----------Python Info----------
   ('Version      :', '2.7.16')
   ('Compiler     :', 'GCC 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.46.4)')
   ('Build        :', ('default', 'Jun 19 2019 07:40:37'))
   ('Arch         :', ('64bit', ''))
   ------------Pip Info-----------
   ('Version      :', '19.1.1')
   ('Directory    :', '/usr/local/lib/python2.7/site-packages/pip')
   ----------MXNet Info-----------
   ('Version      :', '1.5.0')
   ('Directory    :', '/Users/arcadia/work/mxnet-1.5/python/mxnet')
   Commit hash file "/Users/arcadia/work/mxnet-1.5/python/mxnet/COMMIT_HASH" not found. Not installed from pre-built package or built from source.
   ('Library      :', ['/Users/arcadia/work/mxnet-1.5/python/mxnet/../../lib/libmxnet.so'])
   Build features:
   ✖ CUDA
   ✖ CUDNN
   ✖ NCCL
   ✖ CUDA_RTC
   ✖ TENSORRT
   ✔ CPU_SSE
   ✔ CPU_SSE2
   ✔ CPU_SSE3
   ✔ CPU_SSE4_1
   ✔ CPU_SSE4_2
   ✖ CPU_SSE4A
   ✔ CPU_AVX
   ✖ CPU_AVX2
   ✖ OPENMP
   ✖ SSE
   ✔ F16C
   ✖ JEMALLOC
   ✖ BLAS_OPEN
   ✖ BLAS_ATLAS
   ✖ BLAS_MKL
   ✖ BLAS_APPLE
   ✖ LAPACK
   ✔ MKLDNN
   ✔ OPENCV
   ✖ CAFFE
   ✖ PROFILER
   ✖ DIST_KVSTORE
   ✖ CXX14
   ✖ INT64_TENSOR_SIZE
   ✖ SIGNAL_HANDLER
   ✖ DEBUG
   ----------System Info----------
   ('Platform     :', 'Darwin-18.2.0-x86_64-i386-64bit')
   ('system       :', 'Darwin')
   ('node         :', 'MacBookPro')
   ('release      :', '18.2.0')
   ('version      :', 'Darwin Kernel Version 18.2.0: Fri Oct  5 19:41:49 PDT 2018; root:xnu-4903.221.2~2/RELEASE_X86_64')
   ```
   
   ## Build info (Required if built from source)
   
   clang
   
   MXNet commit hash:
   latest commit
   4d07d78e1aff36f603231568bfff88242d073756
   
   ## Minimum reproducible example
   I've slightly modified the cpp example to print the first 10 numbers in the output ndarray:
   download [the code](https://github.com/apache/incubator-mxnet/files/3403216/predict.tar.gz) and replace the example/image-classification/predict-cpp folder.
   
   In order to run the example, the changes in PR #15574 is needed to patch the mxnet code.
   
   ```
   MXNET_ENGINE_TYPE=NaiveEngine ./image-classification-predict path/to/image [thread number]
   ```
   
   ## The output
   openblas 2 threads:
   ```
   ./model/mobilenetv2_0.25-symbol.json ... 146314 bytes
   ./model/mobilenetv2_0.25-0000.params ... 6135676 bytes
   [02:11:17] src/engine/engine.cc:55: MXNet start using engine: NaiveEngine
   3.16592 -2.01711 -3.29867 -10.9845 -1.56596 2.30844 -4.57165 -0.211675 0.300732 -4.35327
   3.16592 -2.01711 -3.29867 -10.9845 -1.56596 2.30844 -4.57165 -0.211675 0.300732 -4.35327
   run successfully
   ```
   
   mkldnn 2 threads:
   ```
   ./model/mobilenetv2_0.25-symbol.json ... 146314 bytes
   ./model/mobilenetv2_0.25-0000.params ... 6135676 bytes
   [02:12:40] src/engine/engine.cc:55: MXNet start using engine: NaiveEngine
   -4.33902 -8.3967 -3.97791 -2.16607 -7.36041 -7.65456 -6.08846 -3.10948 -6.65086 4.17571
   -0.0463818 0.440686 -0.907998 -7.17286 4.8312 -3.03199 4.31585 0.0692099 -0.844623 -8.1773
   run successfully
   ```
   
   The results change randomly with every execution.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services