You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Trevor Grant (JIRA)" <ji...@apache.org> on 2015/08/18 14:51:45 UTC
[jira] [Commented] (FLINK-1994) Add different gain calculation
schemes to SGD
[ https://issues.apache.org/jira/browse/FLINK-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14701200#comment-14701200 ]
Trevor Grant commented on FLINK-1994:
-------------------------------------
I more or less have this working with a few new optimizers based on sklearn- specifically the constant, optimal, and inverse scaling.
Reading this paper to find others, as well as including bulleted ones in issue.
> Add different gain calculation schemes to SGD
> ---------------------------------------------
>
> Key: FLINK-1994
> URL: https://issues.apache.org/jira/browse/FLINK-1994
> Project: Flink
> Issue Type: Improvement
> Components: Machine Learning Library
> Reporter: Till Rohrmann
> Assignee: Trevor Grant
> Priority: Minor
> Labels: ML, Starter
>
> The current SGD implementation uses as gain for the weight updates the formula {{stepsize/sqrt(iterationNumber)}}. It would be good to make the gain calculation configurable and to provide different strategies for that. For example:
> * stepsize/(1 + iterationNumber)
> * stepsize*(1 + regularization * stepsize * iterationNumber)^(-3/4)
> See also how to properly select the gains [1].
> Resources:
> [1] http://arxiv.org/pdf/1107.2490.pdf
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)