You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@hivemall.apache.org by myui <gi...@git.apache.org> on 2018/12/12 08:51:47 UTC

[GitHub] incubator-hivemall pull request #175: [WIP][HIVEMALL-230] Revise Optimizer I...

GitHub user myui opened a pull request:

    https://github.com/apache/incubator-hivemall/pull/175

    [WIP][HIVEMALL-230] Revise Optimizer Implementation

    ## What changes were proposed in this pull request?
    
    Revise Optimizer implementation. 
    
    1. Revise default hyperparameters of AdaDelta and Adam. 
    2. Support AdamW, AdamHD, Eve, and YellowFin optimizer.
    
    * Fixing Weight Decay Regularization in Adam
    https://openreview.net/forum?id=rk6qdGgCZ
    * On the Convergence of Adam and Beyond 
    https://openreview.net/forum?id=ryQu7f-RZ
    * AdamHD (Adam with Hypergradient descent)
    https://arxiv.org/pdf/1703.04782.pdf
    • Eve: A Gradient Based Optimization Method with Locally and Globally Adaptive Learning Rates
    https://arxiv.org/abs/1611.01505
    • YellowFin and the Art of Momentum Tuning
    https://arxiv.org/abs/1706.03471
    
    ## What type of PR is it?
    
    Improvement, Feature
    
    ## What is the Jira issue?
    
    https://issues.apache.org/jira/browse/HIVEMALL-230
    
    ## How was this patch tested?
    
    unit tests, emr (to appear)
    
    ## How to use this feature?
    
    to appear
    
    ## Checklist
    
    - [x] Did you apply source code formatter, i.e., `./bin/format_code.sh`, for your commit?
    - [ ] Did you run system tests on Hive (or Spark)?


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/myui/incubator-hivemall adam_test

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-hivemall/pull/175.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #175
    
----
commit 5168cf06bf03c38f005d435a4415ce8cb8140891
Author: Makoto Yui <my...@...>
Date:   2018-12-03T07:04:29Z

    Added ongoing unit test files

commit ed1b6302183a687a3584fe62ce5fa92b26c828ad
Author: Makoto Yui <my...@...>
Date:   2018-12-04T09:41:42Z

    Fixed to show ETA in debug log

commit 5c9d63f9fc184f05eed28f03986c6269c4ea6e93
Author: Makoto Yui <my...@...>
Date:   2018-12-04T09:42:02Z

    Added unit tests

commit 243f4b40899b960f4942c75f89c0c4c94974b03b
Author: Makoto Yui <my...@...>
Date:   2018-12-05T09:48:17Z

    Added comments

commit ae29e9a669dcd311b154615e19900ec4b01fd4d8
Author: Makoto Yui <my...@...>
Date:   2018-12-06T07:08:48Z

    Refactored

commit c25ce02db537570c6ed75db74d9a3783b316c694
Author: Makoto Yui <my...@...>
Date:   2018-12-06T07:10:05Z

    Added square() method

commit 71671d10138aa54c0485809b6126753a54dbe3e8
Author: Makoto Yui <my...@...>
Date:   2018-12-06T07:10:42Z

    Added helper methods

commit 6f4edbbaaac37884533132dea00c81f36da45e50
Author: Makoto Yui <my...@...>
Date:   2018-12-06T07:22:51Z

    Refactored ADAM implementation

commit e61f22afaa46bdf705c2760cebaa601929a77608
Author: Makoto Yui <my...@...>
Date:   2018-12-06T08:52:08Z

    Added logging message

commit 22c3f7c132fc01528c93c6e15d40a2b70f1771c0
Author: Makoto Yui <my...@...>
Date:   2018-12-06T08:53:01Z

    Improved -eta option to take eta0 for Fixed ETA estimator

commit e9b9b1420c3b573b5cbe15e4340d862251fac81d
Author: Makoto Yui <my...@...>
Date:   2018-12-06T08:53:28Z

    Added unit test

commit 7c6e4a1da5eaeb99c02a9a83f1519d5274131037
Author: Makoto Yui <my...@...>
Date:   2018-12-06T09:06:16Z

    Made eta default hyper-parameter flexible for each optimizer

commit a92293906d43c25ce47032644774723a0cf713d9
Author: Makoto Yui <my...@...>
Date:   2018-12-06T09:36:26Z

    Changed the default hyperparameter of AdaDelta

commit 1494ea298497a846650b2d9f6799add77105ae77
Author: Makoto Yui <my...@...>
Date:   2018-12-07T05:03:21Z

    Reduced the size of test data

commit 79197a84ca4d840ab3150730d5e6d4a5ad96e719
Author: Makoto Yui <my...@...>
Date:   2018-12-07T05:39:13Z

    Improved -help option handling

commit 4fdcf6c84ec81c174f5e107038660b1200b1a9a5
Author: Makoto Yui <my...@...>
Date:   2018-12-07T05:48:07Z

    Added assertions

commit e1c7a68df679a65f496268bd4acc286b19d0a964
Author: Makoto Yui <my...@...>
Date:   2018-12-07T07:39:58Z

    Fixed AdaDelta eta to 1.0

commit b8e5698ecd7e7d2758ef85a338c053f5bbcc663d
Author: Makoto Yui <my...@...>
Date:   2018-12-07T09:13:48Z

    Supported -amsgrad in Adam

commit aa512c3b71039f97c2ac08b598fcb11f1cfc4d80
Author: Makoto Yui <my...@...>
Date:   2018-12-07T09:59:59Z

    Supported -decay option in ADAM optimizer

commit 19bd276ff9867ba93f42c241feb9aa5aafd0836c
Author: Makoto Yui <my...@...>
Date:   2018-12-07T10:15:24Z

    Revise the default eta0/alpha value

commit 19fa61145e8be18c3f86988905b35f171e1ee50e
Author: Makoto Yui <my...@...>
Date:   2018-12-10T08:37:05Z

    Revised ADAM hyperparameter treatment

----


---

[GitHub] incubator-hivemall pull request #175: [HIVEMALL-230] Revise Optimizer Implem...

Posted by asfgit <gi...@git.apache.org>.

Github user asfgit closed the pull request at:

    https://github.com/apache/incubator-hivemall/pull/175


---