Re: Question on executors

[Benjamin Mahler <be...@gmail.com>]

>
> I have the status of TASK_COMPLETED being sent via the driver, followed by
> a wait of about 5 secs


This is needed because of https://issues.apache.org/jira/browse/MESOS-243.
A 1 second sleep should be ample.


> However, I'd still like to hear any thoughts on the approach of using one
> task per executor to simplify my executor code by not having the forking
> and process management. Plus, the benefit of resource isolation now
> applying to tasks. Especially if there's reasons not to do this.


This is perfectly fine and is what is done by several general purpose
frameworks, such as Aurora. There's no reason *not* to do this, especially
if your tasks are independent and are arbitrary forked processes. You're
correct that task-level isolation is what you'll want, in this case.

Other frameworks, such as Spark, may have different semantics for a Task. A
Task may simply be an operation performed in a thread inside the Executor,
rather than an arbitrary forked process. The semantics of a Task are
entirely up to you.

If you don't want to implement an Executor and your Task merely consists of
forking an arbitrary process, you can use the built-in "Command Executor".
You can launch a task directly in this manner by specifying a CommandInfo
inside your TaskInfo (see the documentation in mesos.proto). Unless you're
using the "Command Executor", you will still need to implement "forking and
process management".

Also, I assume you mean TASK_FINISHED instead of TASK_COMPLETED?


On Mon, Mar 10, 2014 at 4:39 PM, Sharma Podila <sp...@netflix.com> wrote:

> I will have to take back the question on why I am getting a TASK_LOST; I
> had a bug that prevented the TASK_COMPLETED to be sent out before exiting.
> I noticed it only after sending my email out!
>
> However, I'd still like to hear any thoughts on the approach of using one
> task per executor to simplify my executor code by not having the forking
> and process management. Plus, the benefit of resource isolation now
> applying to tasks. Especially if there's reasons not to do this.
>
> Thanks.
>
> Sharma
>
>
> On Mon, Mar 10, 2014 at 4:09 PM, Sharma Podila <sp...@netflix.com>wrote:
>
>> I have a question on the Executors. I see that a framework can use 1 or
>> more executors on a slave. At finest granularity, one could launch each
>> task in a separate executor (our tasks are relatively long lived). This at
>> least seems like a good choice since resource isolation of executors now
>> happens for each task. And, I could eliminate the work in my executor of
>> forking processes to execute tasks.
>>
>> When a task completes (either normally or by getting a kill request), I
>> would like the executor to exit as well. Playing with this, I have the
>> status of TASK_COMPLETED being sent via the driver, followed by a wait of
>> about 5 secs (not sure I need this for any mesos communication to happen
>> for status), and then, say, a System.exit(). However, this is producing a
>> TASK_LOST to my framework scheduler. Shouldn't the previous sending of
>> TASK_COMPLETED status prevent the TASK_LOST update?
>>
>> I am using Mesos 0.16.
>>
>> Sharma
>>
>>
>

Re: Question on executors

[Sharma Podila <sp...@netflix.com>]

Yes, all good questions. These questions are a part of the design for our
system, which runs user provided Java code written specifically to our API.
While it is possible we may separate it out in the future as our system
evolves, the current needs are met with running user code in the same JVM.
Thanks, that does help.
Sharma


On Tue, Mar 11, 2014 at 11:56 AM, Benjamin Mahler <benjamin.mahler@gmail.com
> wrote:

> If you're just loading a .jar then I suppose you are ok, but I'm not sure
> if this is a wise design for running arbitrary .jars. You'll be able to
> share the JVM overhead but you'll also be sharing the same JVM runtime, is
> that well supported?
>
> What happens if both your executor .jar and the task .jar have conflicting
> library dependencies? What happens if the task .jar crashes the JVM? I'm
> not a Java expert so it would be good to discuss this with someone else who
> better understands the implications of this shared JVM design.
>
> Hope that helps,
> Ben
>
>
> On Mon, Mar 10, 2014 at 10:03 PM, Sharma Podila <sp...@netflix.com>wrote:
>
>> Thank you for the confirmation and the pointer to the 1 sec sleep.
>>  Yes, I meant TASK_FINISHED.
>>
>>
>>> If you don't want to implement an Executor and your Task merely consists
>>> of forking an arbitrary process, you can use the built-in "Command
>>> Executor". You can launch a task directly in this manner by specifying a
>>> CommandInfo inside your TaskInfo (see the documentation in mesos.proto).
>>> Unless you're using the "Command Executor", you will still need to
>>> implement "forking and process management".
>>>
>>>
>> I do have an Executor implementation in Java to handle all the callbacks
>> from the driver. The launchTask implementation simply loads the task's jar
>> and runs the code in the same JVM. In this case, it sounds like I don't
>> need to implement forking and process management. Unless there's something
>> else I am missing?
>>
>>
>

Re: Question on executors

[Benjamin Mahler <be...@gmail.com>]

If you're just loading a .jar then I suppose you are ok, but I'm not sure
if this is a wise design for running arbitrary .jars. You'll be able to
share the JVM overhead but you'll also be sharing the same JVM runtime, is
that well supported?

What happens if both your executor .jar and the task .jar have conflicting
library dependencies? What happens if the task .jar crashes the JVM? I'm
not a Java expert so it would be good to discuss this with someone else who
better understands the implications of this shared JVM design.

Hope that helps,
Ben


On Mon, Mar 10, 2014 at 10:03 PM, Sharma Podila <sp...@netflix.com> wrote:

> Thank you for the confirmation and the pointer to the 1 sec sleep.
>  Yes, I meant TASK_FINISHED.
>
>
>> If you don't want to implement an Executor and your Task merely consists
>> of forking an arbitrary process, you can use the built-in "Command
>> Executor". You can launch a task directly in this manner by specifying a
>> CommandInfo inside your TaskInfo (see the documentation in mesos.proto).
>> Unless you're using the "Command Executor", you will still need to
>> implement "forking and process management".
>>
>>
> I do have an Executor implementation in Java to handle all the callbacks
> from the driver. The launchTask implementation simply loads the task's jar
> and runs the code in the same JVM. In this case, it sounds like I don't
> need to implement forking and process management. Unless there's something
> else I am missing?
>
>

Re: Question on executors

[Benjamin Mahler <be...@gmail.com>]

If you're just loading a .jar then I suppose you are ok, but I'm not sure
if this is a wise design for running arbitrary .jars. You'll be able to
share the JVM overhead but you'll also be sharing the same JVM runtime, is
that well supported?

What happens if both your executor .jar and the task .jar have conflicting
library dependencies? What happens if the task .jar crashes the JVM? I'm
not a Java expert so it would be good to discuss this with someone else who
better understands the implications of this shared JVM design.

Hope that helps,
Ben


On Mon, Mar 10, 2014 at 10:03 PM, Sharma Podila <sp...@netflix.com> wrote:

> Thank you for the confirmation and the pointer to the 1 sec sleep.
>  Yes, I meant TASK_FINISHED.
>
>
>> If you don't want to implement an Executor and your Task merely consists
>> of forking an arbitrary process, you can use the built-in "Command
>> Executor". You can launch a task directly in this manner by specifying a
>> CommandInfo inside your TaskInfo (see the documentation in mesos.proto).
>> Unless you're using the "Command Executor", you will still need to
>> implement "forking and process management".
>>
>>
> I do have an Executor implementation in Java to handle all the callbacks
> from the driver. The launchTask implementation simply loads the task's jar
> and runs the code in the same JVM. In this case, it sounds like I don't
> need to implement forking and process management. Unless there's something
> else I am missing?
>
>

Re: Question on executors

[Sharma Podila <sp...@netflix.com>]

Thank you for the confirmation and the pointer to the 1 sec sleep.
Yes, I meant TASK_FINISHED.


> If you don't want to implement an Executor and your Task merely consists
> of forking an arbitrary process, you can use the built-in "Command
> Executor". You can launch a task directly in this manner by specifying a
> CommandInfo inside your TaskInfo (see the documentation in mesos.proto).
> Unless you're using the "Command Executor", you will still need to
> implement "forking and process management".
>
>
I do have an Executor implementation in Java to handle all the callbacks
from the driver. The launchTask implementation simply loads the task's jar
and runs the code in the same JVM. In this case, it sounds like I don't
need to implement forking and process management. Unless there's something
else I am missing?