[GitHub] eric-haibin-lin commented on issue #9266: something wrong with SparseEmbedding?

[GitBox <gi...@apache.org>]

eric-haibin-lin commented on issue #9266: something wrong with SparseEmbedding?
URL: https://github.com/apache/incubator-mxnet/issues/9266#issuecomment-354632895
 
 
   I see. Are you using CPU / single GPU / multiple GPUs? 
   One thing to notice is that our Adam optimizer by default performs lazy update if it sees sparse weight & sparse grad, which is similar to tf.LazyAdamOptimizer and behaves different from standard Adam optimizer. https://github.com/apache/incubator-mxnet/blob/master/python/mxnet/optimizer.py#L675-L688
   
   There're some effort supporting standard updates with sparse grad, but the one for Adam is not there yet. But it will be available soon  (cc @ZiyueHuang )
   
   > using Embedding, it started to converge , but the result was unexpected.
   
   What do you mean by unexpected result? 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services