[jira] [Closed] (SYSTEMML-2120) Improve codegen optimizer (pruning effectiveness)

["Matthias Boehm (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SYSTEMML-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Matthias Boehm closed SYSTEMML-2120.
------------------------------------
       Resolution: Done
         Assignee: Matthias Boehm
    Fix Version/s: SystemML 1.1

> Improve codegen optimizer (pruning effectiveness)
> -------------------------------------------------
>
>                 Key: SYSTEMML-2120
>                 URL: https://issues.apache.org/jira/browse/SYSTEMML-2120
>             Project: SystemML
>          Issue Type: Sub-task
>            Reporter: Matthias Boehm
>            Assignee: Matthias Boehm
>            Priority: Major
>             Fix For: SystemML 1.1
>
>
> The codegen optimizer applies various techniques to prune the search space, including the partitioning into independent problems, the restriction to interesting points, as well as cost-based and structural pruning. These techniques work very well for most use case rendering the codegen overhead negligible. However, there are still exceptions where the additional compilation overhead leads to a slowdown. One of these examples is lenet, the below codegen statistics are obtained from 1000 iterations over mnist60k.
> {code}
> HOP DAGs recompiled (PRED, SB): 0/1154.
> HOP DAGs recompile time:        237.579 sec.
> Codegen compile (DAG,CP,JC):    1176/17450/36.
> Codegen enum (ALLt/p,EVALt/p):  1064354993445/16387318/11342971/11331811.
> Codegen compile times (DAG,JC): 234.174/0.835 sec.
> Codegen plan cache hits:        17408/17444.
> {code}
> Pruning reduced the number of plans from > 1 trillion to 11 million, which is still too high causing an overhead of 234s for a total execution time of 425s.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)