[jira] [Closed] (SYSTEMML-1884) Unnecessary spark/mr instructions on removeEmpty select

["Matthias Boehm (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SYSTEMML-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Matthias Boehm closed SYSTEMML-1884.
------------------------------------

> Unnecessary spark/mr instructions on removeEmpty select
> -------------------------------------------------------
>
>                 Key: SYSTEMML-1884
>                 URL: https://issues.apache.org/jira/browse/SYSTEMML-1884
>             Project: SystemML
>          Issue Type: Bug
>    Affects Versions: SystemML 0.14
>            Reporter: Matthias Boehm
>            Assignee: Matthias Boehm
>             Fix For: SystemML 0.15
>
>
> The builtin function {{removeEmpty}} takes an optional select parameter. If this parameter is provided, we directly use it instead of explicitly computing it. However, the compilation of removeEmpty handled this incorrectly, setting the forced execution type to the initially compiled execution type of removeEmpty. This is problematic in scenarios with dynamic recompilation. For example, consider the following scenario:
> {code}
> C = replace(target=A, pattern=2, replacement=0);
> D = removeEmpty(B, margin="rows", select=C);
> {code}
> If the {{removeEmpty}} is initially compiled to SPARK, this also mistakenly set the forced execution type of {{replace}} to SPARK. During recompilation, the {{removeEmpty}} might be compiled to CP but because {{replace}} has a forced execution type, it will remain in SPARK. 
> Workaround: The workaround for this issue is to put a {{while(FALSE){}}} just before the {{removeEmpty}} call (more precisely, C in above example needs to be computed in a separate HOP DAG).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)