You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@madlib.apache.org by fmcquillan99 <gi...@git.apache.org> on 2018/04/04 20:56:19 UTC
[GitHub] madlib issue #251: MLP: Simplify initialization of model coefficients
Github user fmcquillan99 commented on the issue:
https://github.com/apache/madlib/pull/251
Using the data set from
http://madlib.apache.org/docs/latest/group__grp__nn.html#example
the warm start seems to be functioning OK in the sense that it is picking up where it left off:
```
DROP TABLE IF EXISTS mlp_model, mlp_model_summary, mlp_model_standardization;
-- Set seed so results are reproducible
SELECT setseed(0);
SELECT madlib.mlp_classification(
'iris_data', -- Source table
'mlp_model', -- Destination table
'attributes', -- Input features
'class_text', -- Label
ARRAY[5], -- Number of units per layer
'learning_rate_init=0.003,
n_iterations=10,
tolerance=0', -- Optimizer params
'tanh', -- Activation function
NULL, -- Default weight (1)
FALSE, -- No warm start
TRUE -- Not verbose
);
```
produces
```
INFO: Iteration: 2, Loss: <1.52802997435>
INFO: Iteration: 3, Loss: <1.48759899134>
INFO: Iteration: 4, Loss: <1.45013924747>
INFO: Iteration: 5, Loss: <1.41474941405>
INFO: Iteration: 6, Loss: <1.38065258666>
INFO: Iteration: 7, Loss: <1.34717834047>
INFO: Iteration: 8, Loss: <1.31375187164>
INFO: Iteration: 9, Loss: <1.27988891477>
```
Multiple calls of
```
SELECT madlib.mlp_classification(
'iris_data', -- Source table
'mlp_model', -- Destination table
'attributes', -- Input features
'class_text', -- Label
ARRAY[5], -- Number of units per layer
'learning_rate_init=0.003,
n_iterations=10,
tolerance=0', -- Optimizer params
'tanh', -- Activation function
NULL, -- Default weight (1)
TRUE, -- Warm start
TRUE -- Not verbose
);
```
produces
```
INFO: Iteration: 2, Loss: <1.17220527126>
INFO: Iteration: 3, Loss: <1.1336000279>
INFO: Iteration: 4, Loss: <1.09355192251>
INFO: Iteration: 5, Loss: <1.05216070965>
INFO: Iteration: 6, Loss: <1.00962024929>
INFO: Iteration: 7, Loss: <0.966205768022>
INFO: Iteration: 8, Loss: <0.922255641274>
INFO: Iteration: 9, Loss: <0.878149011376>
```
```
INFO: Iteration: 2, Loss: <0.748780710824>
INFO: Iteration: 3, Loss: <0.707821562402>
INFO: Iteration: 4, Loss: <0.668421333885>
INFO: Iteration: 5, Loss: <0.630777478881>
INFO: Iteration: 6, Loss: <0.595026837568>
INFO: Iteration: 7, Loss: <0.56124977596>
INFO: Iteration: 8, Loss: <0.529476974538>
INFO: Iteration: 9, Loss: <0.499697614318>
```
INFO: Iteration: 2, Loss: <0.421765880284>
INFO: Iteration: 3, Loss: <0.399310193088>
INFO: Iteration: 4, Loss: <0.378448795893>
INFO: Iteration: 5, Loss: <0.359075938873>
INFO: Iteration: 6, Loss: <0.341086602199>
INFO: Iteration: 7, Loss: <0.324378642438>
INFO: Iteration: 8, Loss: <0.308854261119>
INFO: Iteration: 9, Loss: <0.294420938642>
```
INFO: Iteration: 2, Loss: <0.256830594513>
INFO: Iteration: 3, Loss: <0.245954682329>
INFO: Iteration: 4, Loss: <0.235795731167>
INFO: Iteration: 5, Loss: <0.226295883531>
INFO: Iteration: 6, Loss: <0.217402237653>
INFO: Iteration: 7, Loss: <0.209066485211>
INFO: Iteration: 8, Loss: <0.201244549645>
INFO: Iteration: 9, Loss: <0.193896234701>
```
If I run with 50 iterations at one time (no warm start), it ends up with
```
INFO: Iteration: 49, Loss: <0.173285421454>
```
which is in the ballpark of the 5 separate runs above, so I think this looks OK.
It would be better if the loss for the 1st and last iterations was printed out in verbose mode.
---