[jira] [Commented] (REEF-936) Support scale-down in REEF

["Boris Shulman (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/REEF-936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131718#comment-15131718 ] 

Boris Shulman commented on REEF-936:
------------------------------------

My plan is as following:
1) MultiRuntime will provide driver side handlers (RequestHandler, ReleaseHandler, LaunchHandler, ResourceManagerStartHandler and ResourceManagerStopHandler)that will receive the actual calls and delegate the calls for the actual runtime.
2) All the handlers above will reference RuntimesContainer that will host the actual runtimes. 
3) RuntimesConainer will receive serialized runtime configurations and an injector in its constructor. It will initialize different runtimes by instantianting different configurations (different injector for each configuration). We will copy needed objects (RuntimeParameters.ResourceStatusHandler, RuntimeParameters.NodeDescriptorHandler, RuntimeParameters.ResourceAllocationHandler and RuntimeParameters.RuntimeStatusHandler) from the original injector to the new injectors.
4) As tang does not support Maps, the RuntimesContainer will receive list of runtime configurations and a list of runtime names. The first runtime in the list will be used as a default runtime.

WDYT?


> Support scale-down in REEF
> --------------------------
>
>                 Key: REEF-936
>                 URL: https://issues.apache.org/jira/browse/REEF-936
>             Project: REEF
>          Issue Type: Improvement
>          Components: REEF
>            Reporter: Markus Weimer
>            Assignee: Boris Shulman
>
> The minimal useful REEF job right now consists of two containers: One for the Driver and one for the Evaluator. This is a fitting design for applications that need to scale out.
> However, some apps might need to be elastic to workloads the size of a single Task, at which point the overhead for the Driver becomes substantial.
> While there is no plan of action yet, let's use this JIRA as the link target for future work towards this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)