You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@madlib.apache.org by GitBox <gi...@apache.org> on 2019/05/11 20:10:20 UTC
[GitHub] [madlib] fmcquillan99 edited a comment on issue #385: DL: Make
metrics and loss optional compile parameters
fmcquillan99 edited a comment on issue #385: DL: Make metrics and loss optional compile parameters
URL: https://github.com/apache/madlib/pull/385#issuecomment-491540303
I guess I am not clear on what are mandatory params and what are optional params, and how errors are handled.
(1)
regular test
```
DROP TABLE IF EXISTS model1, model1_summary;
SELECT madlib.madlib_keras_fit('mnist_train_packed', -- source_table
'model1', -- model
'dependent_var',
'independent_var',
'model_arch_library', -- model_arch_table
1, -- model_arch_id
$$ loss='categorical_crossentropy', optimizer='adam', metrics=['acc'] $$, -- compile_params
$$ batch_size=128, epochs=4 $$, -- fit_params
5, -- num_iterations
0, -- gpus per host
NULL, -- validation_table
'Frank', -- name
'A test model' -- description
);
INFO: Processed 30000 images: Fit took 4.37503194809 sec, Total was 5.91591286659 sec (seg0 slice1 10.128.0.41:40000 pid=2400)
CONTEXT: PL/Python function "fit_transition"
INFO: Processed 30000 images: Fit took 4.39241290092 sec, Total was 5.95022010803 sec (seg1 slice1 10.128.0.41:40001 pid=2401)
CONTEXT: PL/Python function "fit_transition"
INFO: Time for iteration 5: 8.3350250721 sec
CONTEXT: PL/Python function "madlib_keras_fit"
INFO: Average loss after training iteration 5: 0.0441886410117
CONTEXT: PL/Python function "madlib_keras_fit"
INFO: Average accuracy after training iteration 5: 0.986333310604
```
OK
(2)
no metrics param
```
DROP TABLE IF EXISTS model1, model1_summary;
SELECT madlib.madlib_keras_fit('mnist_train_packed', -- source_table
'model1', -- model
'dependent_var', -- dependent_varname
'independent_var', -- independent_varname
'model_arch_library', -- model_arch_table
1, -- model_arch_id
$$ loss='categorical_crossentropy', optimizer='adam' $$, -- compile_params
$$ batch_size=128, epochs=4 $$, -- fit_params
1, -- num_iterations
0, -- gpus per host
NULL, -- validation_table
'Frank', -- name
'A test model' -- description
);
INFO: Model architecture size: 1KB
CONTEXT: PL/Python function "madlib_keras_fit"
INFO: Model state (serialized) size: 0MB
CONTEXT: PL/Python function "madlib_keras_fit"
ERROR: plpy.SPIError: KeyError: 'acc' (plpython.c:5038) (seg1 slice1 10.128.0.41:40001 pid=3000) (plpython.c:5038)
DETAIL:
Traceback (most recent call last):
PL/Python function "fit_transition", line 6, in <module>
return madlib_keras.fit_transition(**globals())
PL/Python function "fit_transition", line 431, in fit_transition
PL/Python function "fit_transition"
CONTEXT: Traceback (most recent call last):
PL/Python function "madlib_keras_fit", line 21, in <module>
madlib_keras.fit(**globals())
PL/Python function "madlib_keras_fit", line 173, in fit
PL/Python function "madlib_keras_fit"
```
Is this the right error?
(3)
no loss param
```
DROP TABLE IF EXISTS model1, model1_summary;
SELECT madlib.madlib_keras_fit('mnist_train_packed', -- source_table
'model1', -- model
'dependent_var',
'independent_var',
'model_arch_library', -- model_arch_table
1, -- model_arch_id
$$ optimizer='adam', metrics=['acc'] $$, -- compile_params
$$ batch_size=128, epochs=4 $$, -- fit_params
5, -- num_iterations
0, -- gpus per host
NULL, -- validation_table
'Frank', -- name
'A test model' -- description
);
INFO: Model architecture size: 1KB
CONTEXT: PL/Python function "madlib_keras_fit"
INFO: Model state (serialized) size: 0MB
CONTEXT: PL/Python function "madlib_keras_fit"
ERROR: plpy.SPIError: plpy.Error: loss is a required parameter for compile (plpython.c:5038) (seg0 slice1 10.128.0.41:40000 pid=3238) (plpython.c:5038)
DETAIL:
Traceback (most recent call last):
PL/Python function "fit_transition", line 6, in <module>
return madlib_keras.fit_transition(**globals())
PL/Python function "fit_transition", line 408, in fit_transition
PL/Python function "fit_transition", line 101, in compile_and_set_weights
PL/Python function "fit_transition", line 274, in compile_model
PL/Python function "fit_transition", line 175, in parse_and_validate_compile_params
PL/Python function "fit_transition", line 96, in _assert
PL/Python function "fit_transition"
CONTEXT: Traceback (most recent call last):
PL/Python function "madlib_keras_fit", line 21, in <module>
madlib_keras.fit(**globals())
PL/Python function "madlib_keras_fit", line 173, in fit
PL/Python function "madlib_keras_fit"
```
Is this the right error to throw
(4)
No fit params
```
DROP TABLE IF EXISTS model1, model1_summary;
SELECT madlib.madlib_keras_fit('mnist_train_packed', -- source_table
'model1', -- model
'dependent_var',
'independent_var',
'model_arch_library', -- model_arch_table
1, -- model_arch_id
$$ loss='categorical_crossentropy', optimizer='adam', metrics=['acc'] $$, -- compile_params
NULL, -- fit_params
5, -- num_iterations
0, -- gpus per host
NULL, -- validation_table
'Frank', -- name
'A test model' -- description
);
ERROR: TypeError: cannot concatenate 'str' and 'NoneType' objects (plpython.c:5038)
CONTEXT: Traceback (most recent call last):
PL/Python function "madlib_keras_fit", line 21, in <module>
madlib_keras.fit(**globals())
PL/Python function "madlib_keras_fit", line 142, in fit
PL/Python function "madlib_keras_fit"
```
I would have thought that defaults would be used.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services