[jira] [Closed] (SYSTEMML-1567) Remove conditionals from nn layers

["Niketan Pansare (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SYSTEMML-1567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Niketan Pansare closed SYSTEMML-1567.
-------------------------------------
       Resolution: Duplicate
    Fix Version/s: SystemML 1.0

This issue should be fixed by https://issues.apache.org/jira/browse/SYSTEMML-1554 and https://issues.apache.org/jira/browse/SYSTEMML-1561

> Remove conditionals from nn layers
> ----------------------------------
>
>                 Key: SYSTEMML-1567
>                 URL: https://issues.apache.org/jira/browse/SYSTEMML-1567
>             Project: SystemML
>          Issue Type: Improvement
>          Components: APIs
>    Affects Versions: SystemML 1.0
>            Reporter: Niketan Pansare
>             Fix For: SystemML 1.0
>
>
> Conditionals in nn layers introduce transient read/write variables that disables fused operators such as CP relu_maxpooling_backward and hence redundant execute sparsity-introducing sel+ operator. This operator causes unnecessary dense-to-sparse-to-dense conversion and becomes the heavy hitter after native BLAS change. Note: some fused operators such as CP relu_maxpooling are still applied because there is no conditional in between those layers.
> Without conditionals in dropout layer: https://github.com/apache/incubator-systemml/blob/master/scripts/nn/layers/dropout.dml#L49-L53 
> {code}
> Iter:2000.0, training loss:0.003149394810197065, training accuracy:100.0
> Iter:2000.0, validation loss:191.9888157354513, validation accuracy:96.875
> SystemML Statistics:
> Total elapsed time:             416.609 sec.
> Total compilation time:         0.000 sec.
> Total execution time:           416.609 sec.
> Number of compiled Spark inst:  69.
> Number of executed Spark inst:  2.
> Native mkl calls (LibMatrixMult/LibMatrixDNN):  4270/10553.
> Cache hits (Mem, WB, FS, HDFS): 277973/0/0/0.
> Cache writes (WB, FS, HDFS):    143616/0/0.
> Cache times (ACQr/m, RLS, EXP): 0.101/0.080/1.988/0.000 sec.
> HOP DAGs recompiled (PRED, SB): 0/2277.
> HOP DAGs recompile time:        6.146 sec.
> Spark ctx create time (lazy):   0.027 sec.
> Spark trans counts (par,bc,col):0/0/0.
> Spark trans times (par,bc,col): 0.000/0.000/0.000 secs.
> Total JIT compile time:         37.746 sec.
> Total JVM GC count:             3949.
> Total JVM GC time:              56.609 sec.
> Heavy hitter instructions (name, time, count):
> -- 1)   conv2d_bias_add         48.984 sec      4514
> -- 2)   conv2d_backward_filter  47.780 sec      4026
> -- 3)   -*      38.246 sec      16104
> -- 4)   +*      35.902 sec      8052
> -- 5)   +       34.227 sec      30566
> -- 6)   ba+*    30.643 sec      12566
> -- 7)   relu_maxpooling_backward        29.678 sec      4026
> -- 8)   conv2d_backward_data    28.520 sec      2013
> -- 9)   *       26.825 sec      35275
> -- 10)  relu_backward   24.842 sec      6039
> {code}
> With conditional, we add sel+ to the heavy hitter:
> {code}
> -- 1)   sel+    55.054 sec      6283
> {code}
> [~mwdusenb@us.ibm.com] Since you created the layers, I think you should decide how best to restructure the DML. My recommendation would be to create two layers in case of conditionals.
> [~mboehm7] [~reinwald]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)