[jira] [Created] (SYSTEMML-1884) Unnecessary spark/mr instructions on removeEmpty select

["Matthias Boehm (JIRA)" <ji...@apache.org>]

Matthias Boehm created SYSTEMML-1884:
----------------------------------------

             Summary: Unnecessary spark/mr instructions on removeEmpty select
                 Key: SYSTEMML-1884
                 URL: https://issues.apache.org/jira/browse/SYSTEMML-1884
             Project: SystemML
          Issue Type: Bug
            Reporter: Matthias Boehm


The builtin function {{removeEmpty}} takes an optional select parameter. If this parameter is provided, we directly use it instead of explicitly computing it. However, the compilation of removeEmpty handled this incorrectly, setting the forced execution type to the initially compiled execution type of removeEmpty. This is problematic in scenarios with dynamic recompilation. For example, consider the following scenario:

{code}
C = replace(target=A, pattern=2, replacement=0);
D = removeEmpty(B, margin="rows", select=C);
{code}

If the {{removeEmpty}} is initially compiled to SPARK, this also mistakenly set the forced execution type of {{replace}} to SPARK. During recompilation, the {{removeEmpty}} might be compiled to CP but because {{replace}} has a forced execution type, it will remain in SPARK. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)