[jira] [Created] (REEF-2054) Elastic Broadcast

["Matteo Interlandi (JIRA)" <ji...@apache.org>]

Matteo Interlandi created REEF-2054:
---------------------------------------

             Summary: Elastic Broadcast
                 Key: REEF-2054
                 URL: https://issues.apache.org/jira/browse/REEF-2054
             Project: REEF
          Issue Type: Sub-task
            Reporter: Matteo Interlandi
            Assignee: Matteo Interlandi


This subtask is used to track the implementation of the elastic broadcast operator.

To have the elastic broadcast operator working the following components are required:

Driver side:
 * a context implementation providing the entry point to the elastic service
 * a stage implementation wrapping pipelines of +operators+ into logical schedulable units
 * a task set object managing the scheduling and communication of tasks

Task side:
 * a context implementation providing the entrypoint for tasks to the elastic service
 * a stage implementation wrapping the +workflow+ of operators
 * a workflow object surfacing the list of operators to the task and enabling to manager failures events by the elastic framework
 * a communication layer between tasks and between tasks and the driver

Operators:
 * for now only the broadcast operator will be implemented
 * each operator will have:
 ** a set user defined policies in case failures occur
 ** a set of mechanisms implementing the policies
 ** a topology mirroring the logical organization of nodes in the network. For the moment only a flat topology (1 master, n nodes) will be implemented
 ** user defined +failure machine+ defining when policies are triggered
 ** policies, mechanism and failure machines are used also at stage and context level (in a sort of hierarchical way) to provide high level behaviors under failures.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)