[jira] [Updated] (SYSTEMML-1127) Synchronize creation of buffer pool on remote workers

["Glenn Weidner (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SYSTEMML-1127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Glenn Weidner updated SYSTEMML-1127:
------------------------------------
    Fix Version/s:     (was: SystemML 1.0)
                   SystemML 0.12

> Synchronize creation of buffer pool on remote workers
> -----------------------------------------------------
>
>                 Key: SYSTEMML-1127
>                 URL: https://issues.apache.org/jira/browse/SYSTEMML-1127
>             Project: SystemML
>          Issue Type: Bug
>          Components: ParFor
>            Reporter: Felix Schüler
>            Assignee: Felix Schüler
>             Fix For: SystemML 0.12
>
>
> When executing a Spark parfor, the SparkParforWorker throws an exception which states that the localtmpdir could not be created. This is due to the fact that multiple executors are running multithreaded on the same worker. 
> The createDistributedUniqueID() method in the IDHander.java creates unique IDs only per pid and host, not per thread. This could potentially be solved by adding the threadID to the unique ID. The question is if every thread should have its own cache or if the logic should be changed so that the first creation will be successful and then the threads share one cache.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)