[GitHub] pengzhao-intel commented on issue #11129: [MXNET-497]Test kAddTo request for mkldnn operators

[GitBox <gi...@apache.org>]

pengzhao-intel commented on issue #11129: [MXNET-497]Test kAddTo request for mkldnn operators
URL: https://github.com/apache/incubator-mxnet/pull/11129#issuecomment-396779411
 
 
   @azai91 Really thanks your efforts for improving the quality of MKL-DNN backend.
   
   @azai91 @zheng-da @eric-haibin-lin @marcoabreu
   
   Two suggestions:
   
   **1) avoiding change the MKL-DNN implementations when adding the test cases.** 
      It will be hard to track back the changes. If it really needs to change (for the bugfix or other reasons), I think we can file another PR.
   
      One example is #11026, this PR is marked as adding test cases, so we don't pay much attention because it's always a good thing for more cases. But the implementation was changed and led to a performance regression for resnet-50 inference (from 238 drop to 179), see below.
   
   **2) running the `benchmark_score.py` to verify the performance for each of CI**
   
   ```
   commit 92286c9106dd63d2bfd062f9abb0e53b071a46e4
   Author: Alexander Zai azai91@gmail.com
   Date:   Tue May 29 17:48:33 2018 -0700
   
   [lvtao@mlt-skx052 image-classification]$ python benchmark_score.py
   INFO:root:network: resnet-50
   INFO:root:device: cpu(0)
   /home/lvtao/Workspace/mxnet-official/python/mxnet/module/base_module.py:66: UserWarning: Data provided by label_shapes don't match names specified by label_names ([] vs. ['softmax_label'])
     warnings.warn(msg)
   [22:09:59] src/operator/nn/mkldnn/mkldnn_base.cc:72: Allocate 9437184 bytes with malloc directly
   INFO:root:batch size  1, image/sec: 83.039246
   INFO:root:batch size  2, image/sec: 118.022742
   INFO:root:batch size  4, image/sec: 149.483457
   INFO:root:batch size  8, image/sec: 162.208255
   INFO:root:batch size 16, image/sec: 166.800091
   INFO:root:batch size 32, image/sec: 165.821744
   INFO:root:batch size 64, image/sec: 175.934763
   INFO:root:batch size 128, image/sec: 179.160898
   INFO:root:batch size 256, image/sec: 178.767242
   
   
   commit 9514a1e39f8356f8fee6202cd86c8f20fbf301b6
   Author: kpmurali 37911926+kpmurali@users.noreply.github.com
   Date:   Tue May 29 17:36:35 2018 -0700
   
   [lvtao@mlt-skx052 image-classification]$ python benchmark_score.py
   INFO:root:network: resnet-50
   INFO:root:device: cpu(0)
   /home/lvtao/Workspace/mxnet-official/python/mxnet/module/base_module.py:66: UserWarning: Data provided by label_shapes don't match names specified by label_names ([] vs. ['softmax_label'])
     warnings.warn(msg)
   [22:19:23] src/operator/nn/mkldnn/mkldnn_base.cc:72: Allocate 147456 bytes with malloc directly
   [22:19:23] src/operator/nn/mkldnn/mkldnn_base.cc:72: Allocate 589824 bytes with malloc directly
   [22:19:23] src/operator/nn/mkldnn/mkldnn_base.cc:72: Allocate 2359296 bytes with malloc directly
   [22:19:23] src/operator/nn/mkldnn/mkldnn_base.cc:72: Allocate 9437184 bytes with malloc directly
   INFO:root:batch size  1, image/sec: 89.205967
   INFO:root:batch size  2, image/sec: 128.775569
   INFO:root:batch size  4, image/sec: 156.363125
   INFO:root:batch size  8, image/sec: 195.993911
   INFO:root:batch size 16, image/sec: 219.215664
   INFO:root:batch size 32, image/sec: 224.414152
   INFO:root:batch size 64, image/sec: 238.657344
   INFO:root:batch size 128, image/sec: 230.266770
   INFO:root:batch size 256, image/sec: 225.139635
   
   ```
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services