You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2017/10/23 04:58:35 UTC

[GitHub] rahul003 commented on issue #8373: distribute training in fp16

rahul003 commented on issue #8373: distribute training in fp16
URL: https://github.com/apache/incubator-mxnet/pull/8373#issuecomment-338548123
 
 
   I agree we should provide similar interfaces for this and 2bit right now, at least on the user facing API first.
   
   A problem is that this solution requires creating a server for this dtype of 16bit. Which means we need to know type of compression when creating a kvstore. In the 2bit PR, since we still use float32 to represent values (for 2bit compression, 16 values represented in one float32), the original server was sent a message to know the type of compression.
   
   The only way I see to merge interfaces is to have compression parameters even in my case to come in at the time of creating kvstore. To start this, I would like to understand how to give the argument defined in python/mxnet/kvstore_server.py to start the server.
   
   General n-bit compression can be done. But neither this PR nor my PR can be directly extended as of now. In my PR, although the framework in kvstores can be generalized easily, the operator is 2bit specific. Either that can be extended by other having operators for different specific n, or if possible coming up with a generic operator. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services