You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@madlib.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2018/03/10 00:36:00 UTC
[jira] [Commented] (MADLIB-1212) Incorrect step_size update in MLP
[ https://issues.apache.org/jira/browse/MADLIB-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393870#comment-16393870 ]
ASF GitHub Bot commented on MADLIB-1212:
----------------------------------------
GitHub user njayaram2 opened a pull request:
https://github.com/apache/madlib/pull/240
MLP: Fix step size initialization based on learning rate policy
JIRA: MADLIB-1212
The step_size is supposed to be updated based on the learning rate. The
formulae for different policies depend on the current iteration number,
which was not consumed correctly. This commit fixes it by using
it.iteration for the current iteration number in that update.
Co-authored-by: Nikhil Kak <nk...@pivotal.io>
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/madlib/madlib bugfix/mlp-learning-rate-policy
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/madlib/pull/240.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #240
----
commit 72c6c8bdf5bfb50132e0566ca9600f82e99b7700
Author: Nandish Jayaram <nj...@...>
Date: 2018-03-06T19:10:50Z
MLP: Fix step size initialization based on learning rate policy
JIRA: MADLIB-1212
The step_size is supposed to be updated based on the learning rate. The
formulae for different policies depend on the current iteration number,
which was not consumed correctly. This commit fixes it by using
it.iteration for the current iteration number in that update.
Co-authored-by: Nikhil Kak <nk...@pivotal.io>
----
> Incorrect step_size update in MLP
> ---------------------------------
>
> Key: MADLIB-1212
> URL: https://issues.apache.org/jira/browse/MADLIB-1212
> Project: Apache MADlib
> Issue Type: Bug
> Components: Module: Neural Networks
> Reporter: Nandish Jayaram
> Priority: Major
> Fix For: v1.14
>
>
> The initialization and update of step_size based on the learning rate policy seems to be broken in MLP.
> {code}
> zero_indexed_iteration = current_iteration - 1
> if learning_rate_policy == "exp":
> step_size = step_size_init * gamma**zero_indexed_iteration
> elif learning_rate_policy == "inv":
> step_size = step_size_init * (current_iteration)**(-power)
> elif learning_rate_policy == "step":
> step_size = step_size_init * gamma**(
> math.floor(zero_indexed_iteration / iterations_per_step))
> {code}
> The variable current_iteration in the above code snippet (from mlp_igd.py_in) does not seem to be updated in MLP, so the step_size will remain the same.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)