FLIP-6 + Mesos - Questions

[Jared Stehler <ja...@intellifylearning.com>]

According to the docs: 

The Mesos scheduler currently resides with the JobManager but will be started independently of the JobManager in future versions (see FLIP-6 <https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65147077>). The proposed changes will also add a Dipsatcher component which will be the central point for job submission and monitoring.

I’m assuming that the dispatcher will implement a similar API as the current job manager does, for job submission etc. 

Will the dispatcher spawn a new job manager per job, and calculate and spawn the required number of task managers as well? 

Will the task manager containers be killed along with the job manager container when that job is canceled or fails? 

Will it be possible to specify different task manager memory settings at the job submission level, so that we might spawn task managers with X heap for job A, and Y heap for job B?

Is this slated for the 1.5 release? I’m trying to get a sense for what flip-6 will look like when it drops, and what its current state is on the master / 1.5 branch; I’m interested in helping test it out.


--
Jared Stehler
Chief Architect - Intellify Learning
o: 617.701.6330 x703