[jira] [Commented] (REEF-1224) Fault Tolerant - Separate Data downloading from Task injection

["Dhruv Mahajan (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/REEF-1224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15194159#comment-15194159 ] 

Dhruv Mahajan commented on REEF-1224:
-------------------------------------

[~markus.weimer] [~afchung90] [~juliaw]

So, at high level design, here are the steps we need to do:

a) Start DataLoading task on the root context and service.Basically configurations from IDataPartition interface will be used at this stage.
b) Data Loading task will bind The DataSet of type T as singleton to the context.
c) Submit Group communication service to the evaluators.
d) Start the IMRU tasks on the evaluators.

I have few questions:

a) Whenever I submit context and or service from driver, I just need to use SubmitContextAndService function right?
b) Once DataLoading Task generates DataSet of Type T, we will need to bind it to current service rather than context right so that subsequent contexts can also see it.
c) What do I need on evaluator side to actually bind it? Is there an example piece of code for this?


> Fault Tolerant - Separate Data downloading from Task injection
> --------------------------------------------------------------
>
>                 Key: REEF-1224
>                 URL: https://issues.apache.org/jira/browse/REEF-1224
>             Project: REEF
>          Issue Type: Task
>            Reporter: Julia
>            Assignee: Dhruv Mahajan
>
> Currently in IMRU, data downloading happens during the Task injection. It couples the data and Task object. In Fault tolerant case, we would like to only resubmit a task but use the data that have been downloaded, That requires us to decouple those two portions. For example, data downloading portion can be attached to Context, and we can then resubmit a task on the same context. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)