Mesos Scheduler/Executor coordination mechanism

[Vladimir Vivien <vl...@gmail.com>]

Hi.  New to Mesos and still trying to wrap my head around its concept.

Question:
When an executor is launched by Mesos, how does the Scheduler know about
the Executor it just launched and vise versa ?  It appears that the slave
launches an OS process, but I am unclear how the coordination is done
between the running OS process and mesos.
-- 
Vladimir Vivien

Re: Mesos Scheduler/Executor coordination mechanism

[Sam Taha <ta...@gmail.com>]

When you create the executor (in your Scheduler code) you specify the
command line script to launch on the Slave. So if you are running a java
executor on the slave, your script will launch a jvm with all the usually
command line trimmings to run the Executor.main(). The mesos Slave process
that invokes the script, I assume passes the executor ID and related setup
data to the java executor shell script when it is started to properly setup
the executor configs.

Thanks,
Sam Taha

http://www.grandlogic.com






On Thu, Sep 26, 2013 at 3:26 PM, Vladimir Vivien
<vl...@gmail.com>wrote:

> Vinod
> Thanks.  Your reply added more knowledge to my understanding.
>
> From my one week understanding of mesos, it seems when a slave launches a
> task, it does so via an OS call of "/bin/sh -c".  Now, how does the slave
> process reach into the running process to trigger the callbacks ?
>
>
> On Thu, Sep 26, 2013 at 3:17 PM, Vinod Kone <vi...@gmail.com> wrote:
>
>> The way it works is as follows:
>>
>> Scheduler launches "tasks". A task has an "executor id" (optional)
>> attached to it. When a slave gets a task, it launches an executor if an
>> executor with that "executor id" is not running. All subsequent tasks
>> tagged with the same "executor id" are just passed on to the running
>> executor.
>>
>> Typically executors send TASK_STARTING or TASK_RUNNING updates when a
>> task gets launched. This is the signal that could be used by the schedulers
>> to know that a task (and its corresponding executor) has launched.
>>
>> HTH,
>>
>>
>> On Thu, Sep 26, 2013 at 12:03 PM, Vladimir Vivien <
>> vladimir.vivien@gmail.com> wrote:
>>
>>> Hi.  New to Mesos and still trying to wrap my head around its concept.
>>>
>>> Question:
>>> When an executor is launched by Mesos, how does the Scheduler know about
>>> the Executor it just launched and vise versa ?  It appears that the slave
>>> launches an OS process, but I am unclear how the coordination is done
>>> between the running OS process and mesos.
>>>  --
>>> Vladimir Vivien
>>>
>>
>>
>
>
> --
> Vladimir Vivien
>

Re: Mesos Scheduler/Executor coordination mechanism

[Vladimir Vivien <vl...@gmail.com>]

Vinod
Thanks.  Your reply added more knowledge to my understanding.

>From my one week understanding of mesos, it seems when a slave launches a
task, it does so via an OS call of "/bin/sh -c".  Now, how does the slave
process reach into the running process to trigger the callbacks ?


On Thu, Sep 26, 2013 at 3:17 PM, Vinod Kone <vi...@gmail.com> wrote:

> The way it works is as follows:
>
> Scheduler launches "tasks". A task has an "executor id" (optional)
> attached to it. When a slave gets a task, it launches an executor if an
> executor with that "executor id" is not running. All subsequent tasks
> tagged with the same "executor id" are just passed on to the running
> executor.
>
> Typically executors send TASK_STARTING or TASK_RUNNING updates when a task
> gets launched. This is the signal that could be used by the schedulers to
> know that a task (and its corresponding executor) has launched.
>
> HTH,
>
>
> On Thu, Sep 26, 2013 at 12:03 PM, Vladimir Vivien <
> vladimir.vivien@gmail.com> wrote:
>
>> Hi.  New to Mesos and still trying to wrap my head around its concept.
>>
>> Question:
>> When an executor is launched by Mesos, how does the Scheduler know about
>> the Executor it just launched and vise versa ?  It appears that the slave
>> launches an OS process, but I am unclear how the coordination is done
>> between the running OS process and mesos.
>>  --
>> Vladimir Vivien
>>
>
>


-- 
Vladimir Vivien

Re: Mesos Scheduler/Executor coordination mechanism

[Vinod Kone <vi...@gmail.com>]

The way it works is as follows:

Scheduler launches "tasks". A task has an "executor id" (optional) attached
to it. When a slave gets a task, it launches an executor if an executor
with that "executor id" is not running. All subsequent tasks tagged with
the same "executor id" are just passed on to the running executor.

Typically executors send TASK_STARTING or TASK_RUNNING updates when a task
gets launched. This is the signal that could be used by the schedulers to
know that a task (and its corresponding executor) has launched.

HTH,


On Thu, Sep 26, 2013 at 12:03 PM, Vladimir Vivien <vladimir.vivien@gmail.com
> wrote:

> Hi.  New to Mesos and still trying to wrap my head around its concept.
>
> Question:
> When an executor is launched by Mesos, how does the Scheduler know about
> the Executor it just launched and vise versa ?  It appears that the slave
> launches an OS process, but I am unclear how the coordination is done
> between the running OS process and mesos.
>  --
> Vladimir Vivien
>