[GitHub] coldsheephot opened a new issue #11299: row_sparse_pull, push row_sparse gradient is too slow

[GitBox <gi...@apache.org>]

coldsheephot opened a new issue #11299: row_sparse_pull,push row_sparse gradient is too slow
URL: https://github.com/apache/incubator-mxnet/issues/11299
 
 
   # result:
   when I use row_sparse api to develop deep gradient_compression algorithm ,i find the api is too slowly than dense gradient, can you help me to save more time ?
   
   <pre>
   kv: local,ctx=mx.cpu()
   row_sparse push and pull time: 12.5247938633
   default push and pull time: 0.194673061371
   </pre>
   <pre>
   kv: device,ctx=mx.gpu(0)
   row_sparse push and pull time: 3.08180809021
   default push and pull time: 0.197321891785
   </pre>
   # code:
   In this code ,I even don't use merge and update, and also tostype even use once, but it also too slowly than dense gradient
   <pre>
   
   import mxnet as mx
   import numpy as np
   
   shape = (512*512*3*3, )
   key = 3
   
   name = 'local'
   kv = mx.kv.create(name)
   context = mx.cpu()
   
   for i in range(100):
       kv.init(i, mx.nd.zeros(shape=shape,ctx=context).tostype('row_sparse'))
   
   for i in range(100,200):
       kv.init(i, mx.nd.zeros(shape=shape,ctx=context))
   
   a = mx.nd.ones(shape=shape,ctx=context).tostype('row_sparse')
   all_row_ids = mx.nd.array(np.arange(shape[0]), dtype=np.int64)
   
   
   def test_row_sparse_pull():
       out = a
       for i in range(100):
           kv.push(i, a)
           kv.row_sparse_pull(i, out=out, priority=i, row_ids=all_row_ids)
           # out.wait_to_read()
           # mx.base._LIB.MXNDArrayWaitToWrite(a.handle)
       mx.nd.waitall()
   
   
   b = mx.nd.ones(shape=shape,ctx=context)
   
   
   def test_default_pull():
       out = b
       for i in range(100, 200):
           kv.push(i, b)
           kv.pull(i, out=out, priority=i)
           # mx.base._LIB.MXNDArrayWaitToWrite(b.handle)
           # out.wait_to_read()
       mx.nd.waitall()
   
   
   import time
   t1 = time.time()
   test_row_sparse_pull()
   t2 = time.time()
   print "row_sparse push and pull time:", t2-t1
   
   
   t1 = time.time()
   test_default_pull()
   t2 = time.time()
   
   print "default push and pull time:", t2-t1
   </pre>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services