[GitHub] eric-haibin-lin commented on issue #9396: inference speed drop after updating mxnet from 0.10.0 to 1.0.0

[GitBox <gi...@apache.org>]

eric-haibin-lin commented on issue #9396: inference speed drop after updating mxnet from 0.10.0 to 1.0.0
URL: https://github.com/apache/incubator-mxnet/issues/9396#issuecomment-373480684
 
 
   @nicklhy I bisected the changes between 0.10.0 and 1.0 and found the following on a p2.xlarge(K80) instance. The commits are patched with the fix in PR 9055
   ```bash
   git checkout xxx; 
   git submodule update --recursive; 
   git cherry-pick 9cc8ea3be23fb7adf4630e4cf065a2473094fbc8 -X theirs
   make
   ```
   and below is the result
   ```
   ff21e1f Changed FullyConnected to use new linalg gemm, plus TensorCore if fp16 I/O. (#7505)
   
   speed test for batch size: 1
           avg forward speed: 24.484983 samples/s
           avg forward time: mean = 0.040839 s, std = 0.000095 s
   ```
   ```
   56eae58 Fixed Makefile so a null CUDA_ARCH is treated like an unset one. (#7515) - Fast
   
   ########################################################
   speed test for batch size: 1
           avg forward speed: 25.461191 samples/s
           avg forward time: mean = 0.039270 s, std = 0.000095 s
   ########################################################
   ```
   Looks like the commit ff21e1f caused the 4% slowdown during inference. 
   
   @DickJC123 were you aware of this? 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services