[GitHub] StephanieYuan commented on issue #12376: [MXNET-854] SVRG Optimization in Python Module API

[GitBox <gi...@apache.org>]

StephanieYuan commented on issue #12376: [MXNET-854] SVRG Optimization in Python Module API
URL: https://github.com/apache/incubator-mxnet/pull/12376#issuecomment-418745603
 
 
   Thanks everyone for the feedback and input! 
   
   I've rebased and resolved the conflict with master.  I have also added some more unittests for SVRGModule especially on verifying SVRG gradients calculations and the test coverage is now:
   mxnet/contrib/svrg_optimization/svrg_module.py                        175     21    88%
   mxnet/contrib/svrg_optimization/svrg_optimizer.py                      38      1    97%
   
   @sandeep-krishnamurthy I'm working on creating the MNIST example but I'm expecting it will take a few more days as I have other tasks in queue. Maybe I can include the example in my next PR on SVRG in distributed mode?
   
   @anirudhacharya SVRGOptimizer is designed to be used with SVRGModule alone. The actual SVRG optimization logic is implemented in the SVRGModule and the SVRGOptimizer is designed to assist the full gradients accumulation in the KVStore. I've made SVRGOptimizer a private class and added the reasonings behind the design in greater detail in the docs/api/python/contrib/svrg_optimization. Hopefully it clarifies things a bit. 
   
   @aaronmarkham I've added pointer about the svrg_optimization in index.md and refined the docs in the example/svrg_module/README. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services