You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2019/06/17 14:37:55 UTC
[GitHub] [incubator-mxnet] pengxin99 commented on issue #12893: Dummy-data
based benchmarking script for Gluon RNN-API
pengxin99 commented on issue #12893: Dummy-data based benchmarking script for Gluon RNN-API
URL: https://github.com/apache/incubator-mxnet/pull/12893#issuecomment-502710383
@roywei thanks for you review, and I test our script on 8180-1s with 29 cores, below is the results:
inference benchmark results：
```
INFO:root:lstm inference benchmark.
INFO:root:For BS = 64, Layers = 1, Shape=[64, 15, 500, 500], SPS=20993.399 sps
INFO:root:For BS = 64, Layers = 1, Shape=[64, 20, 500, 500], SPS=17156.602 sps
INFO:root:For BS = 64, Layers = 1, Shape=[64, 25, 500, 500], SPS=14634.234 sps
INFO:root:For BS = 64, Layers = 1, Shape=[64, 30, 500, 500], SPS=13103.086 sps
INFO:root:For BS = 64, Layers = 1, Shape=[64, 35, 500, 500], SPS=11681.630 sps
INFO:root:For BS = 64, Layers = 1, Shape=[64, 40, 500, 500], SPS=10475.330 sps
INFO:root:For BS = 64, Layers = 1, Shape=[64, 45, 500, 500], SPS=9504.317 sps
INFO:root:For BS = 64, Layers = 1, Shape=[64, 50, 500, 500], SPS=8842.211 sps
INFO:root:For BS = 16, Layers = 1, Shape=[16, 25, 512, 512], SPS=6095.115 sps
INFO:root:For BS = 32, Layers = 1, Shape=[32, 25, 512, 512], SPS=8296.429 sps
INFO:root:For BS = 64, Layers = 1, Shape=[64, 25, 512, 512], SPS=10112.136 sps
INFO:root:For BS = 128, Layers = 1, Shape=[128, 25, 512, 512], SPS=12306.829 sps
INFO:root:For BS = 16, Layers = 1, Shape=[16, 25, 1024, 1024], SPS=1196.153 sps
INFO:root:For BS = 32, Layers = 1, Shape=[32, 25, 1024, 1024], SPS=1705.147 sps
INFO:root:For BS = 64, Layers = 1, Shape=[64, 25, 1024, 1024], SPS=2248.357 sps
INFO:root:For BS = 128, Layers = 1, Shape=[128, 25, 1024, 1024], SPS=3900.332 sps
INFO:root:For BS = 16, Layers = 1, Shape=[16, 25, 2048, 2048], SPS=280.221 sps
INFO:root:For BS = 32, Layers = 1, Shape=[32, 25, 2048, 2048], SPS=351.963 sps
INFO:root:For BS = 64, Layers = 1, Shape=[64, 25, 2048, 2048], SPS=487.378 sps
INFO:root:For BS = 128, Layers = 1, Shape=[128, 25, 2048, 2048], SPS=1070.538 sps
INFO:root:For BS = 16, Layers = 1, Shape=[16, 25, 4096, 4096], SPS=65.910 sps
INFO:root:For BS = 32, Layers = 1, Shape=[32, 25, 4096, 4096], SPS=85.363 sps
INFO:root:For BS = 64, Layers = 1, Shape=[64, 25, 4096, 4096], SPS=90.826 sps
INFO:root:For BS = 128, Layers = 1, Shape=[128, 25, 4096, 4096], SPS=281.542 sps
INFO:root:gru inference benchmark.
INFO:root:For BS = 64, Layers = 1, Shape=[64, 15, 500, 500], SPS=14952.023 sps
INFO:root:For BS = 64, Layers = 1, Shape=[64, 20, 500, 500], SPS=12003.757 sps
INFO:root:For BS = 64, Layers = 1, Shape=[64, 25, 500, 500], SPS=9956.502 sps
INFO:root:For BS = 64, Layers = 1, Shape=[64, 30, 500, 500], SPS=8532.300 sps
INFO:root:For BS = 64, Layers = 1, Shape=[64, 35, 500, 500], SPS=7446.000 sps
INFO:root:For BS = 64, Layers = 1, Shape=[64, 40, 500, 500], SPS=6586.347 sps
INFO:root:For BS = 64, Layers = 1, Shape=[64, 45, 500, 500], SPS=5961.260 sps
INFO:root:For BS = 64, Layers = 1, Shape=[64, 50, 500, 500], SPS=5449.126 sps
INFO:root:For BS = 16, Layers = 1, Shape=[16, 25, 512, 512], SPS=4347.048 sps
INFO:root:For BS = 32, Layers = 1, Shape=[32, 25, 512, 512], SPS=6195.475 sps
INFO:root:For BS = 64, Layers = 1, Shape=[64, 25, 512, 512], SPS=8700.350 sps
INFO:root:For BS = 128, Layers = 1, Shape=[128, 25, 512, 512], SPS=11402.683 sps
INFO:root:For BS = 16, Layers = 1, Shape=[16, 25, 1024, 1024], SPS=1327.629 sps
INFO:root:For BS = 32, Layers = 1, Shape=[32, 25, 1024, 1024], SPS=2022.031 sps
INFO:root:For BS = 64, Layers = 1, Shape=[64, 25, 1024, 1024], SPS=2940.143 sps
INFO:root:For BS = 128, Layers = 1, Shape=[128, 25, 1024, 1024], SPS=3987.736 sps
INFO:root:For BS = 16, Layers = 1, Shape=[16, 25, 2048, 2048], SPS=359.937 sps
INFO:root:For BS = 32, Layers = 1, Shape=[32, 25, 2048, 2048], SPS=592.503 sps
INFO:root:For BS = 64, Layers = 1, Shape=[64, 25, 2048, 2048], SPS=862.949 sps
INFO:root:For BS = 128, Layers = 1, Shape=[128, 25, 2048, 2048], SPS=1260.849 sps
INFO:root:For BS = 16, Layers = 1, Shape=[16, 25, 4096, 4096], SPS=88.472 sps
INFO:root:For BS = 32, Layers = 1, Shape=[32, 25, 4096, 4096], SPS=151.096 sps
INFO:root:For BS = 64, Layers = 1, Shape=[64, 25, 4096, 4096], SPS=240.240 sps
INFO:root:For BS = 128, Layers = 1, Shape=[128, 25, 4096, 4096], SPS=369.249 sps
```
latency benchmark result:
```
INFO:root:lstm inference benchmark.
INFO:root:For BS = 1, Layers = 1, Shape=[1, 15, 500, 500], latency=0.003145 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 20, 500, 500], latency=0.003825 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 500, 500], latency=0.004469 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 30, 500, 500], latency=0.005054 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 35, 500, 500], latency=0.005685 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 40, 500, 500], latency=0.006307 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 45, 500, 500], latency=0.007022 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 50, 500, 500], latency=0.007478 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 512, 512], latency=0.002655 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 512, 512], latency=0.003892 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 512, 512], latency=0.006760 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 512, 512], latency=0.010551 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 1024, 1024], latency=0.013230 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 1024, 1024], latency=0.018453 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 1024, 1024], latency=0.029798 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 1024, 1024], latency=0.034038 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 2048, 2048], latency=0.057398 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 2048, 2048], latency=0.090056 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 2048, 2048], latency=0.138633 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 2048, 2048], latency=0.126094 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 4096, 4096], latency=0.290858 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 4096, 4096], latency=0.410212 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 4096, 4096], latency=0.735229 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 4096, 4096], latency=0.472423 ms
INFO:root:gru inference benchmark.
INFO:root:For BS = 1, Layers = 1, Shape=[1, 15, 500, 500], latency=0.001689 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 20, 500, 500], latency=0.001951 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 500, 500], latency=0.002212 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 30, 500, 500], latency=0.002472 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 35, 500, 500], latency=0.002746 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 40, 500, 500], latency=0.003000 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 45, 500, 500], latency=0.003244 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 50, 500, 500], latency=0.003519 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 512, 512], latency=0.002322 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 512, 512], latency=0.002309 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 512, 512], latency=0.002331 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 512, 512], latency=0.002314 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 1024, 1024], latency=0.006032 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 1024, 1024], latency=0.006082 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 1024, 1024], latency=0.006082 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 1024, 1024], latency=0.005999 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 2048, 2048], latency=0.022420 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 2048, 2048], latency=0.022164 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 2048, 2048], latency=0.022387 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 2048, 2048], latency=0.022163 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 4096, 4096], latency=0.100981 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 4096, 4096], latency=0.101482 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 4096, 4096], latency=0.102126 ms
INFO:root:For BS = 1, Layers = 1, Shape=[1, 25, 4096, 4096], latency=0.101685 ms
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services