[jira] [Created] (REEF-33) Support pushing Task messages from Evaluator to Driver.

["Byung-Gon Chun (JIRA)" <ji...@apache.org>]

Byung-Gon Chun created REEF-33:
----------------------------------

             Summary: Support pushing Task messages from Evaluator to Driver.
                 Key: REEF-33
                 URL: https://issues.apache.org/jira/browse/REEF-33
             Project: REEF
          Issue Type: New Feature
            Reporter: Byung-Gon Chun
            Priority: Critical


Currently REEF supports sending Task messages by piggybacking them to heartbeats from Evaluator to Driver. This is great for scalability, but there are cases where Task needs to send messages immediately. This question has been asked multiple times in the dev mailing list; our experience of building Surf, an in-memory caching service on REEF, confirms it. 

Although the default way of sending Task messages should use the heartbeat mechanism, we should provide an option to push a TaskMessage from Evaluator to Driver. Otherwise, we have to set up a separate channel (RemoteManager) between Evaluator and Driver, which does not make sense.

A good approach is to reuse REEF RemoteManager between Evaluator and Driver and to provide an interface to push TaskMessages (as an option). 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)