[GitHub] aseyboldt opened a new issue #8219: Broadcasting ops are slow

[gi...@git.apache.org]

aseyboldt opened a new issue #8219: Broadcasting ops are slow
URL: https://github.com/apache/incubator-mxnet/issues/8219
 
 
   I can't see a good reason why `broadcast_add(array, value)` should be much slower than `array + scalar`, but the speed difference is almost 100x:
   
   ```python
   import mxnet as mx
   
   a = mx.sym.var('a')
   b = mx.sym.var('b')
   
   a_ = mx.nd.ones((2**20,))
   b1 = mx.nd.ones((2**20,))
   b2 = mx.nd.ones((1,))
   
   func1 = (a + b).bind(mx.cpu(), {'a': a_, 'b': b1})
   func2 = mx.sym.broadcast_add(a, b).bind(mx.cpu(), {'a': a_, 'b': b2})
   func3 = (a + 1.).bind(mx.cpu(), {'a': a_})
   
   # array + array
   %timeit func1.forward()[0].wait_to_read()
   # 409 ?s ? 12.7 ?s per loop (mean ? std. dev. of 7 runs, 1000 loops each)
   
   # boadcast_add(scalar, array)
   %timeit func2.forward()[0].wait_to_read()
   # 7.02 ms ? 7.08 ?s per loop (mean ? std. dev. of 7 runs, 100 loops each)
   
   # array + scalar
   %timeit func3.forward()[0].wait_to_read()
   # 88.7 ?s ? 939 ns per loop (mean ? std. dev. of 7 runs, 10000 loops each)
   ```
   
   A bit of digging led me to this function:
   
   https://github.com/apache/incubator-mxnet/blob/bbf1c0b9d49b432fc46b911656f7f3fdd2521f98/src/operator/tensor/broadcast_reduce-inl.h#L142-L152
   
   This is called for each element of the result array, and does a lot of index juggling. Broadcasting can be implemented much faster by using strides of 0 in dimensions where we want to broadcast. Additionally it is possible to optimize some inner loops further in many cases. The best reference I know of would be the numpy iterator api https://docs.scipy.org/doc/numpy-dev/reference/c-api.iterator.html
   
   The same problem seem to be in the reduction ops in the same file.
   
   For the benchmark I used git commit 573a010879583885a0193e30dc0b8c848d80869b on a linux machine with intel cpu.
   
   
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services