You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@madlib.apache.org by "Frank McQuillan (JIRA)" <ji...@apache.org> on 2019/06/21 18:20:00 UTC
[jira] [Closed] (MADLIB-1363) Reduce verbose output to console with
fit()
[ https://issues.apache.org/jira/browse/MADLIB-1363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Frank McQuillan closed MADLIB-1363.
-----------------------------------
Resolution: Fixed
https://github.com/apache/madlib/pull/415
> Reduce verbose output to console with fit()
> -------------------------------------------
>
> Key: MADLIB-1363
> URL: https://issues.apache.org/jira/browse/MADLIB-1363
> Project: Apache MADlib
> Issue Type: Improvement
> Components: Deep Learning
> Reporter: Frank McQuillan
> Priority: Minor
> Fix For: v1.16
>
>
> fit() INFO and CONTEXT messages
> (1) no validation_table, metrics_compute_frequency=0
> {code}
> SELECT madlib.madlib_keras_fit('iris_train_packed', -- source table
> 'iris_model', -- model output table
> 'model_arch_library', -- model arch table
> 1, -- model arch id
> $$ loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'] $$, -- compile_params
> $$ batch_size=5, epochs=3 $$, -- fit_params
> 10 -- num_iterations
> );
> INFO: Processed 60 images: Fit took 0.567000865936 sec, Total was 0.757196903229 sec (seg0 slice1 10.128.0.41:40000 pid=13317)
> CONTEXT: PL/Python function "fit_transition"
> INFO: Processed 60 images: Fit took 0.55348110199 sec, Total was 0.741441011429 sec (seg1 slice1 10.128.0.41:40001 pid=13318)
> CONTEXT: PL/Python function "fit_transition"
> INFO: Time for training in iteration 1: 2.45737695694 sec
> CONTEXT: PL/Python function "madlib_keras_fit"
> {code}
> change to
> {code}
> INFO: Time for training in iteration 1: 2.45737695694 sec
> CONTEXT: PL/Python function "madlib_keras_fit"
> {code}
> (2) no validation_table, metrics_compute_frequency!=0
> {code}
> SELECT madlib.madlib_keras_fit('iris_train_packed', -- source table
> 'iris_model', -- model output table
> 'model_arch_library', -- model arch table
> 1, -- model arch id
> $$ loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'] $$, -- compile_params
> $$ batch_size=5, epochs=3 $$, -- fit_params
> 10, -- num_iterations
> 0, -- gpus per host
> NULL, -- validation table
> 1 -- metrics compute frequency
> );
> INFO: Processed 60 images: Fit took 0.534310817719 sec, Total was 0.712550878525 sec (seg0 slice1 10.128.0.41:40000 pid=14501)
> CONTEXT: PL/Python function "fit_transition"
> INFO: Processed 60 images: Fit took 0.564456939697 sec, Total was 0.751413106918 sec (seg1 slice1 10.128.0.41:40001 pid=14502)
> CONTEXT: PL/Python function "fit_transition"
> INFO: Time for training in iteration 1: 2.28858995438 sec
> CONTEXT: PL/Python function "madlib_keras_fit"
> INFO: Time for evaluation in iteration 1: 0.188971996307 sec.
> CONTEXT: PL/Python function "madlib_keras_fit"
> INFO: Training set metric after iteration 1: 0.649999976158.
> CONTEXT: PL/Python function "madlib_keras_fit"
> INFO: Training set loss after iteration 1: 1.1202558279.
> CONTEXT: PL/Python function "madlib_keras_fit"
> {code}
> change to
> {code}
> INFO: Time for training in iteration 1: 2.28858995438 sec
> Time for evaluation in iteration 1: 0.188971996307 sec
> Training set metric after iteration 1: 0.649999976158
> Training set loss after iteration 1: 1.1202558279
> CONTEXT: PL/Python function "madlib_keras_fit"
> {code}
> (3) yes validation_table, metrics_compute_frequency=0
> {code}
> SELECT madlib.madlib_keras_fit('iris_train_packed', -- source table
> 'iris_model', -- model output table
> 'model_arch_library', -- model arch table
> 1, -- model arch id
> $$ loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'] $$, -- compile_params
> $$ batch_size=5, epochs=3 $$, -- fit_params
> 10, -- num_iterations
> 0, -- GPUs per host
> 'iris_test_packed' -- validation dataset
> );
> INFO: Processed 60 images: Fit took 0.552575826645 sec, Total was 0.734694004059 sec (seg0 slice1 10.128.0.41:40000 pid=18431)
> CONTEXT: PL/Python function "fit_transition"
> INFO: Processed 60 images: Fit took 0.549551010132 sec, Total was 0.734927892685 sec (seg1 slice1 10.128.0.41:40001 pid=18432)
> CONTEXT: PL/Python function "fit_transition"
> INFO: Time for training in iteration 1: 2.36340904236 sec
> CONTEXT: PL/Python function "madlib_keras_fit"
> {code}
> change to
> {code}
> INFO: Time for training in iteration 1: 2.45737695694 sec
> CONTEXT: PL/Python function "madlib_keras_fit"
> {code}
> (4) yes validation_table, metrics_compute_frequency=!0
> {code}
> SELECT madlib.madlib_keras_fit('iris_train_packed', -- source table
> 'iris_model', -- model output table
> 'model_arch_library', -- model arch table
> 1, -- model arch id
> $$ loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'] $$, -- compile_params
> $$ batch_size=5, epochs=3 $$, -- fit_params
> 10, -- num_iterations
> 0, -- GPUs per host
> 'iris_test_packed', -- validation dataset
> 1 -- metrics compute frequency
> );
> INFO: Processed 60 images: Fit took 0.57217502594 sec, Total was 0.817452907562 sec (seg0 slice1 10.128.0.41:40000 pid=19573)
> CONTEXT: PL/Python function "fit_transition"
> INFO: Processed 60 images: Fit took 0.554927110672 sec, Total was 0.800101041794 sec (seg1 slice1 10.128.0.41:40001 pid=19574)
> CONTEXT: PL/Python function "fit_transition"
> INFO: Time for training in iteration 1: 2.43148899078 sec
> CONTEXT: PL/Python function "madlib_keras_fit"
> INFO: Time for evaluation in iteration 1: 0.217161893845 sec.
> CONTEXT: PL/Python function "madlib_keras_fit"
> INFO: Training set metric after iteration 1: 0.524999976158.
> CONTEXT: PL/Python function "madlib_keras_fit"
> INFO: Training set loss after iteration 1: 0.984773635864.
> CONTEXT: PL/Python function "madlib_keras_fit"
> INFO: Time for evaluation in iteration 1: 0.205282926559 sec.
> CONTEXT: PL/Python function "madlib_keras_fit"
> INFO: Validation set metric after iteration 1: 0.600000023842.
> CONTEXT: PL/Python function "madlib_keras_fit"
> INFO: Validation set loss after iteration 1: 0.940379023552.
> CONTEXT: PL/Python function "madlib_keras_fit"
> {code}
> change to
> {code}
> INFO: Time for training in iteration 1: 2.43148899078 sec
> Time for evaluating training dataset in iteration 1: 0.217161893845 sec
> Training set metric after iteration 1: 0.524999976158
> Training set loss after iteration 1: 0.984773635864
> Time for evaluating validation dataset in iteration 1: 0.205282926559 sec
> Validation set metric after iteration 1: 0.600000023842
> Validation set loss after iteration 1: 0.940379023552
> CONTEXT: PL/Python function "madlib_keras_fit"
> {code}
> Note change in wording ^^^ because there are 2 evaluation times, 1 for training dataset and 1 for validation dataset.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)