[jira] [Commented] (REEF-30) Run REEF on Mesos

["Markus Weimer (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/REEF-30?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14193599#comment-14193599 ] 

Markus Weimer commented on REEF-30:
-----------------------------------

This is great! One question: In REEF-runtime-YARN, we allow users to use native YARN resource requests in addition to the {{EvaluatorRequest}} interface. Are you thinking of something similar? For instance, we could expose the resource offers to a application-provided {{EventHandler}}. That way, REEF provides a abstraction for resource requests that is portable, but also allows code to circumvent this abstraction in a sane way.

> Run REEF on Mesos
> -----------------
>
>                 Key: REEF-30
>                 URL: https://issues.apache.org/jira/browse/REEF-30
>             Project: REEF
>          Issue Type: Task
>          Components: REEF
>            Reporter: Byung-Gon Chun
>            Assignee: John Yang
>
> Add reef-runtime-mesos to support running REEF on Mesos. 
> The main work's done for translating Mesos resource offers to REEF resource requests and running evaluators on Mesos executors.
> On resource requests from REEF Driver, launches Mesos executors which register Evaluator Launch/Release handlers
> On evaluator launch/release requests from REEF Driver, uses the registered handlers to start/stop evaluator processes on the Mesos executors.
> One thing to note is that Mesos runs a framework scheduler as a client (i.e., it does not run on a slave mode). This means that with Mesos the REEF driver runs at the machine you launch the job.
> There're meta-frameworks like Marathon and Aurora that allows to run the framework scheduler on a slave node. We will use Mesos natively. Integrating with Marathon or Aurora can be future work.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)