[jira] [Commented] (SINGA-413) Hyper-parameter configuration API

["Ngin Yun Chuan (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SINGA-413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16706150#comment-16706150 ] 

Ngin Yun Chuan commented on SINGA-413:
--------------------------------------

Hi [~wangwei.cs], I've pushed the API changes to the [`improve_knob_api` branch|https://github.com/nginyc/rafiki/compare/improve_knob_api]. The worker-side implementation for this improvement is not complete yet - wanted to get your feedback on the revised model knob configuration API before continuing.


> Hyper-parameter configuration API
> ---------------------------------
>
>                 Key: SINGA-413
>                 URL: https://issues.apache.org/jira/browse/SINGA-413
>             Project: Singa
>          Issue Type: Improvement
>            Reporter: wangwei
>            Priority: Major
>
> The current API for hyper-parameter configuration in Rafiki requires the model contributor to implement
> {code:java}
> def get_knob_config(self):
>     {
>     'knobs': {
>            'hidden_layer_units': {
>            'type': 'int',
>            'range': [2, 128]
>            },
>      ...
>      }
> def init(self, knobs):
>     self.hidden_layer_units = knobs.get(hidden_layer_units){code}
> The json style can be replaced by
> {code}
> def check(knobs, val):
>    if knobs[0].value() == 4 and val == 4:
>       return False
>    else:
>       return True
> def register_knobs(self):
>    self.hidden_layer_units = Knob(32, kInt, range=(2, 128))
>    self.learning_rate = Knob(0.02, kExpFloat, range=(1e-1, 1e-4))
>    self.batch_size = Knob(8, kIntCat, range = [4, 8, 16]))
>    self.num_epoch = Knob(4, kIntCat, range = [4, 8, 16], depends=self.batch_size, callback=check()))
> def train(self):
>    for i in range(self.num_epoch.value()):
>       ...{code}
> The worker's workflow is like
> {code:java}
> for trial in range(total_trials):
>    model = Model()
>    model.register_knobs()
>    knobs = [x for x, y in model.__dict__.getitems() if isinstance(y, Knob)]
>    if trial == 0:
>      reigster_knobs_to_advisor(knobs)
>    get_knobs_from_advisor(knobs){code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)