[GitHub] haojin2 edited a comment on issue #11795: Fix problematic backward of take & embedding

[GitBox <gi...@apache.org>]

haojin2 edited a comment on issue #11795: Fix problematic backward of take & embedding
URL: https://github.com/apache/incubator-mxnet/pull/11795#issuecomment-405808567
 
 
   Benchmark script:
   ```Python
   import mxnet as mx
   import numpy as np
   import time
   
   N = 50000
   ctx = mx.gpu(0)
   
   embedding = mx.gluon.nn.Embedding(N, 300)
   embedding.initialize(ctx=ctx)
   
   i = 0
   np.random.seed(1)
   idx = mx.nd.array(np.random.randint(0, N, size=(1024, 160)), ctx=ctx)
   
   print(np.max(np.bincount(idx.asnumpy().flatten().astype(np.int64))))
   
   a = time.time()
   for i in range(500000):
       with mx.autograd.record():
           emb_in = embedding(idx)
           loss = emb_in.sum()
           loss.backward()
   print(time.time() - a)
   ```
   Benchmark results:
   Original implementation: 146.33599019050598
   New Implementation: 147.44622945785522
   
   Slowdown is negligible after switching to new kernel.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services