[jira] [Commented] (SYSTEMML-2413) Paramserv performance bottleneck on the DAG recompilation

["LI Guobao (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SYSTEMML-2413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518195#comment-16518195 ] 

LI Guobao commented on SYSTEMML-2413:
-------------------------------------

[~mboehm7] , in fact, I found that maybe this expensive dag recompilation is introduced by a code fault in workers. It is that I wrongly used always the same original function block to generate the instruction. But we should use the recompiled new function block. So to be more clear, I remove the function blocks which are not related to update function or agg function. Additionnally, I could observe in the code that the dags have been already deep copied when copying the function because they need the recompilation.

> Paramserv performance bottleneck on the DAG recompilation
> ---------------------------------------------------------
>
>                 Key: SYSTEMML-2413
>                 URL: https://issues.apache.org/jira/browse/SYSTEMML-2413
>             Project: SystemML
>          Issue Type: Bug
>            Reporter: LI Guobao
>            Assignee: LI Guobao
>            Priority: Major
>
> We could observe that the DAG recompilation of each experiment is expensive. Hence, we could improve this bottleneck by deep copying the dags into each workers' execution context. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)