You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2018/01/29 19:04:55 UTC
[GitHub] piiswrong commented on a change in pull request #8918: Added in Large-Batch SGD with a warmup, and a LARS startegy. Also add?
piiswrong commented on a change in pull request #8918: Added in Large-Batch SGD with a warmup, and a LARS startegy. Also add?
URL: https://github.com/apache/incubator-mxnet/pull/8918#discussion_r164531156
##########
File path: python/mxnet/optimizer.py
##########
@@ -645,6 +645,195 @@ def update(self, index, weight, grad, state):
ftml_update(weight, grad, prev_d, prev_v, prev_z, out=weight,
lr=lr, wd=wd, **kwargs)
+@register
+class LBSGD(Optimizer):
+ """The Large Batch SGD optimizer with momentum and weight decay.
+
+ The optimizer updates the weight by::
+
+ state = momentum * state + lr * rescale_grad * clip(grad, clip_gradient) + wd * weight
+ weight = weight - state
+
+ For details of the update algorithm see :class:`~mxnet.ndarray.lbsgd_update` and
Review comment:
@ashokei @zhreshold
Where is lbsgd_update defined? I don't see it.
Please add proper reference to relevant papar
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services