[jira] [Commented] (REEF-1842) Making IMRU task and input data association deterministic

["Markus Weimer (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/REEF-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16102552#comment-16102552 ] 

Markus Weimer commented on REEF-1842:
-------------------------------------

Are you envisioning something where the user supplies the mapping? Or more like a hash based approach which is random, but deterministic?

> Making IMRU task and input data association deterministic
> ---------------------------------------------------------
>
>                 Key: REEF-1842
>                 URL: https://issues.apache.org/jira/browse/REEF-1842
>             Project: REEF
>          Issue Type: Task
>            Reporter: Julia
>            Assignee: Julia
>            Priority: Critical
>
> Currently, the tasks added to the typology on communication group is based on the sequence of contexts received. While the input data assigned to context can be random especially when some evaluators fail and new evaluators/contexts are requested. This makes the training result undeterministic given same input data set. 
> To make the result deterministic for given set of input data, we would like to make tasks in the typology keep the same order with the same input data file associated. To achieve it, the mapping between the context id and partitionDescriptor id should be fixed once assigned. And when adding tasks to the communication group, the order should be based on the sorted id to ensure it is deterministic. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)