[jira] [Commented] (SYSTEMML-1946) Incorrect operation parallelism w/ constrained parfor optimizer and specified k

["Matthias Boehm (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SYSTEMML-1946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16195940#comment-16195940 ] 

Matthias Boehm commented on SYSTEMML-1946:
------------------------------------------

with the fix, the generated plan looks as follows:
{code}
----------------------------
 EXPLAIN OPT TREE (type=ABSTRACT_PLAN, size=6)
----------------------------
--PARFOR, exec=CP, k=4, dp=NONE, tp=FACTORING, rm=LOCAL_MEM
----GENERIC (lines 4-4), exec=CP, k=1
------u(print), exec=CP, k=1
------ua(+RC), exec=CP, k=4
------ba(+*), exec=CP, k=4
------dg(rand), exec=CP, k=4
----------------------------
{code}

> Incorrect operation parallelism w/ constrained parfor optimizer and specified k
> -------------------------------------------------------------------------------
>
>                 Key: SYSTEMML-1946
>                 URL: https://issues.apache.org/jira/browse/SYSTEMML-1946
>             Project: SystemML
>          Issue Type: Bug
>            Reporter: Matthias Boehm
>
> When fixing the parfor degree of parallelism with 
> {code}
> parfor(i in 1:100, opt=CONSTRAINED, par=4, log=DEBUG)
>   print(sum(X %*% rand(rows=1000, cols=1)));
> {code}
> the constrained parfor optimizer misses to set the operation parallelism accordingly, which produces the following plan and thus, potentially large CPU over-provisioning that hurts performance.
> {code}
> ----------------------------
>  EXPLAIN OPT TREE (type=ABSTRACT_PLAN, size=6)
> ----------------------------
> --PARFOR, exec=CP, k=4, dp=NONE, tp=FACTORING, rm=LOCAL_MEM
> ----GENERIC (lines 4-4), exec=CP, k=1
> ------u(print), exec=CP, k=16
> ------ua(+RC), exec=CP, k=16
> ------ba(+*), exec=CP, k=16
> ------dg(rand), exec=CP, k=16
> ----------------------------
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)