When do executors shutdown?

[Hans van den Bogert <ha...@gmail.com>]

I have difficulty understanding Mesos’ model.

A framework can, for every accepted resource offer,  mention an executor besides the tasks descriptions it submits to Mesos. However does every use of offered resources, start a new executor? Thus for instance if the scenario occurs that two resource offers are used (shortly after each other),  which happen to be of the same slave, then are two executors started at one point? Or is the second batch of tasks given to the first started executor?

I hope my question is clear, if not, let me know,

Hans van den Bogert

Re: When do executors shutdown?

[haosdent <ha...@gmail.com>]

Every offer have a id. Once an offer is accepted, it could not used by
others. If you use same offer id to start two executors, you would get an
error.

On Tue, Jun 30, 2015 at 6:08 PM, Hans van den Bogert <ha...@gmail.com>
wrote:

> I have difficulty understanding Mesos’ model.
>
> A framework can, for every accepted resource offer,  mention an executor
> besides the tasks descriptions it submits to Mesos. However does every use
> of offered resources, start a new executor? Thus for instance if the
> scenario occurs that two resource offers are used (shortly after each
> other),  which happen to be of the same slave, then are two executors
> started at one point? Or is the second batch of tasks given to the first
> started executor?
>
> I hope my question is clear, if not, let me know,
>
> Hans van den Bogert




-- 
Best Regards,
Haosdent Huang

Re: When do executors shutdown?

[Hans van den Bogert <ha...@gmail.com>]

Thanks Alex and Vino, very clear.

Hans

On 30 Jun 2015, at 19:36, Vinod Kone <vi...@gmail.com> wrote:

> To clarify Alex's response. An executor is not shutdown if it has no running tasks. It is only shutdown when the framework asks it to (or the framework itself shuts down).

Re: When do executors shutdown?

[Vinod Kone <vi...@gmail.com>]

To clarify Alex's response. An executor is not shutdown if it has no
running tasks. It is only shutdown when the framework asks it to (or the
framework itself shuts down).

Re: When do executors shutdown?

[Alex Rukletsov <al...@mesosphere.com>]

An executor is terminated by Mesos if it misbehaves (e.g. sends
TASK_STAGING updates or uses too much memory), killed by an
oversubscription QoSController, a framework shuts down, or a scheduler
sends a scheduler::Call::Shutdown request to Mesos. Note that an executor
may also fail or decide to commit suicide.

On Tue, Jun 30, 2015 at 12:38 PM, Hans van den Bogert <ha...@gmail.com>
wrote:

> Exactly what I needed to know, one follow-up question though:
>
> An executor is terminated by Mesos if it has no running tasks
>
> Does this mean there is some timeout? Or does the “parent” framework
> actively have to give a command to shutdown the executor? Because using
> Spark in fine-grained mode for example, I don’t see the executors getting
> shutdown, even though they might not have tasks for a while. (I am glad
> they don’t get killed without consent of Spark, because we would lose our
> in memory data).
>
> On 30 Jun 2015, at 12:32, Alex Rukletsov <al...@mesosphere.com> wrote:
>
> There are two types of tasks: (1) those that specify an executor and (2)
> those, that specify a command.
>
> When a task of ttype (1) arrives to a slave, the slave checks whether an
> executor with the same executorID already exists on this slave. If yes, the
> task is redirected to the executor; if not, then an executor instance is
> created. An executor is terminated by Mesos if it has no running tasks and
> all status updated for terminated tasks have been delivered.
>
> For tasks of type (2) a special executor (called MesosExecutor) is created
> for each task. When such task terminates or is killed, the corresponding
> executor shuts down as well.
>
> On Tue, Jun 30, 2015 at 12:08 PM, Hans van den Bogert <
> hansbogert@gmail.com> wrote:
>
>> I have difficulty understanding Mesos’ model.
>>
>> A framework can, for every accepted resource offer,  mention an executor
>> besides the tasks descriptions it submits to Mesos. However does every use
>> of offered resources, start a new executor? Thus for instance if the
>> scenario occurs that two resource offers are used (shortly after each
>> other),  which happen to be of the same slave, then are two executors
>> started at one point? Or is the second batch of tasks given to the first
>> started executor?
>>
>> I hope my question is clear, if not, let me know,
>>
>> Hans van den Bogert
>
>
>
>

Re: When do executors shutdown?

[Hans van den Bogert <ha...@gmail.com>]

Exactly what I needed to know, one follow-up question though:
> An executor is terminated by Mesos if it has no running tasks
Does this mean there is some timeout? Or does the “parent” framework actively have to give a command to shutdown the executor? Because using Spark in fine-grained mode for example, I don’t see the executors getting shutdown, even though they might not have tasks for a while. (I am glad they don’t get killed without consent of Spark, because we would lose our in memory data).

On 30 Jun 2015, at 12:32, Alex Rukletsov <al...@mesosphere.com> wrote:

> There are two types of tasks: (1) those that specify an executor and (2) those, that specify a command. 
> 
> When a task of ttype (1) arrives to a slave, the slave checks whether an executor with the same executorID already exists on this slave. If yes, the task is redirected to the executor; if not, then an executor instance is created. An executor is terminated by Mesos if it has no running tasks and all status updated for terminated tasks have been delivered.
> 
> For tasks of type (2) a special executor (called MesosExecutor) is created for each task. When such task terminates or is killed, the corresponding executor shuts down as well.
> 
> On Tue, Jun 30, 2015 at 12:08 PM, Hans van den Bogert <ha...@gmail.com> wrote:
> I have difficulty understanding Mesos’ model.
> 
> A framework can, for every accepted resource offer,  mention an executor besides the tasks descriptions it submits to Mesos. However does every use of offered resources, start a new executor? Thus for instance if the scenario occurs that two resource offers are used (shortly after each other),  which happen to be of the same slave, then are two executors started at one point? Or is the second batch of tasks given to the first started executor?
> 
> I hope my question is clear, if not, let me know,
> 
> Hans van den Bogert
> 

Re: When do executors shutdown?

[Alex Rukletsov <al...@mesosphere.com>]

There are two types of tasks: (1) those that specify an executor and (2)
those, that specify a command.

When a task of ttype (1) arrives to a slave, the slave checks whether an
executor with the same executorID already exists on this slave. If yes, the
task is redirected to the executor; if not, then an executor instance is
created. An executor is terminated by Mesos if it has no running tasks and
all status updated for terminated tasks have been delivered.

For tasks of type (2) a special executor (called MesosExecutor) is created
for each task. When such task terminates or is killed, the corresponding
executor shuts down as well.

On Tue, Jun 30, 2015 at 12:08 PM, Hans van den Bogert <ha...@gmail.com>
wrote:

> I have difficulty understanding Mesos’ model.
>
> A framework can, for every accepted resource offer,  mention an executor
> besides the tasks descriptions it submits to Mesos. However does every use
> of offered resources, start a new executor? Thus for instance if the
> scenario occurs that two resource offers are used (shortly after each
> other),  which happen to be of the same slave, then are two executors
> started at one point? Or is the second batch of tasks given to the first
> started executor?
>
> I hope my question is clear, if not, let me know,
>
> Hans van den Bogert

Re: When do executors shutdown?

[Tom Arnfeld <to...@duedil.com>]

It depends on the framework, Mesos imposes no rules on the relationship between a task an executor. The framework can specify an Executor ID with the tasks it's submitting, and if two tasks land on the same slave with the same Executor ID, Mesos will take care of ensuring they share the same Executor process.


The slave actually has a queue of task launch operations it manages and tasks are launched in sequence, so if you respond to two offers with the same Executor ID on the same slave in quick succession, the slave will launch only one instance of the executor and send both tasks to it.




I hope that helps!




Note: Not all tasks have to have executors, in this instance Mesos will transparent launch an Executor on your behalf, one per task.


--


Tom Arnfeld

Developer // DueDil

On Tue, Jun 30, 2015 at 11:08 AM, Hans van den Bogert
<ha...@gmail.com> wrote:

> I have difficulty understanding Mesos’ model.
> A framework can, for every accepted resource offer,  mention an executor besides the tasks descriptions it submits to Mesos. However does every use of offered resources, start a new executor? Thus for instance if the scenario occurs that two resource offers are used (shortly after each other),  which happen to be of the same slave, then are two executors started at one point? Or is the second batch of tasks given to the first started executor?
> I hope my question is clear, if not, let me know,
> Hans van den Bogert