Re: Sending TASK_STARTING in the built-in executors

[Benno Evers <be...@mesosphere.com>]

As a follow-up, this change now landed and will likely be part of Mesos
1.5.0.

We did our best to verify that we don't accidentally break existing
frameworks, and the only issue we could find was with chronos (where a fix
was since merged into the stable and master branches).

If you discover that some framework you depend on would choke on this, now
would be a good time to update it before upgrading to Mesos 1.5.0 ;)

Best regards,
-- 
Benno Evers
Software Engineer, Mesosphere

On Wed, Aug 23, 2017 at 11:38 AM, Benno Evers <be...@mesosphere.com> wrote:

> Hi all,
>
> when starting a task, an executor can send out the following status
> updates:
>
>   - [optional] TASK_STARTING: Sent by the executor when it received the
> launch command
>   - TASK_RUNNING: Sent by the executor when the task is running
>
> The built-in executors currently don't send out TASK_STARTING updates. I
> think this discards potentially valuable information, because TASK_RUNNING
> informs us about the current status of the task, but not about the status
> change.
>
> For example, if the network connection between scheduler and master is
> interrupted during task start, it has no good way to estimate the tasks
> start time, because the TASK_RUNNING update that it eventually gets might
> be a much later one. Also, for tasks with a long delay between STARTING and
> RUNNING, to an outside observer it will look the same as if the task was
> stuck in STAGING.
>
> There is a small risk that sending an additional update could break
> existing frameworks. We briefly looked through some of the most popular
> open-source frameworks and didn't find any major issues, but of course it's
> impossible to do an exhaustive check.
>
> In particular, a framework will break if
>
>  1. It runs tasks using one of the built-in mesos executors, and
>  2. it doesn't handle the possibility of receiving TASK_STARTING update,
> and
>  3. it reports an error whenever it encounters an unexpected task states
> in an update.
>
>
> If you are aware of any such framework, please speak up so we can consider
> it.
>
>
> Thanks,
> --
> Benno Evers
> Software Engineer, Mesosphere
>



-- 
Benno Evers
Software Engineer, Mesosphere