[GitHub] piiswrong commented on a change in pull request #8107: "add warmup lr_scheduler" create a new pr

[GitBox <gi...@apache.org>]

piiswrong commented on a change in pull request #8107:  "add warmup lr_scheduler" create a new pr
URL: https://github.com/apache/incubator-mxnet/pull/8107#discussion_r151878202
 
 

 ##########
 File path: python/mxnet/lr_scheduler.py
 ##########
 @@ -100,16 +100,27 @@ class MultiFactorScheduler(LRScheduler):
 
     Then calculate the new learning rate by::
 
-       base_lr * pow(factor, k+1)
+        base_lr * pow(factor, k+1)
+
+    When warmup_step>1, warmup the learning rate by a const value for first warmup_step steps.
+    It returns a new learning rate by::
+
+        begin_lr + (num_update - 1) * const_update
 
     Parameters
     ----------
     step: list of int
-        The list of steps to schedule a change
+        The list of steps to schedule a change.
     factor: float
         The factor to change the learning rate.
+    warmup_step : int
+        Changes the learning rate for first 'warmup_step' updates.
+    begin_lr : float, optional
+        The learning rate at begin.
+    stop_lr : float, optional
+        Stop updating the learning rate if it is less than this value.
 
 Review comment:
   begin_lr and stop_lr are confusing names and the does doesn't explain exactly what they mean

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services