You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@aurora.apache.org by Michael Schenck <sc...@hioscar.com> on 2015/04/09 17:49:07 UTC
Plotting state transitions
Hi all,
I need to be able to plot state transitions from a Job-level at bare
minimum. I've put together a patch to thermos-observer (attached:
thermos-statsd.patch).
Now, I understand that the Observer is likely to be going away, so I was
thinking about creating a similar patch for the executor, possibly for
possibly in ThermosTaskRunner.compute_status
<https://github.com/mschenck/aurora/blob/master/src/main/python/apache/aurora/executor/thermos_task_runner.py#L183-L205>.
My last thought would be to include a Process that *tails* the checkpoint
log similarly to the Observer.
What is the preferred approach to accomplishing this goal?
Best regards,
Michael Schenck
Re: Plotting state transitions
Posted by Michael Schenck <sc...@hioscar.com>.
I also pushed what I already have to ReviewBoard:
https://reviews.apache.org/r/33019/
On Thu, Apr 9, 2015 at 12:33 PM, Michael Schenck <sc...@hioscar.com>
wrote:
> I have a branch (in Github) on my fork of the repo:
> https://github.com/mschenck/aurora/tree/thermos_statsd_state_transitions
>
> I'm happy to push the patch to ReviewBoard, as well.
>
> Can you clarify what you mean by "plan"?
>
> - Michael Schenck
>
> On Thu, Apr 9, 2015 at 12:23 PM, Bill Farner <wf...@apache.org> wrote:
>
>> I may be going on a tangent from what's in your mind, but i think a means
>> to visualize this stuff would be really useful. I can see value in a few
>> different distinct, but complementary things:
>>
>> 1. state flow of instances in a job. This would cover transitions between
>> states like PENDING and RUNNING, and what happens after termination (based
>> on whether the job is a service, cron, etc)
>>
>> 2. *plan* for Processes in a task, as followed by thermos. This would be
>> handy to see that parallel/sequential invocations are as you expect.
>>
>> 3. plan *execution* of processes in a task, as executed thus far by
>> thermos. Sounds like this is what you described above, so this would be a
>> yet-to-finish view of (2).
>>
>> As for your approach thus far, the apache mail server seems to eat
>> attachments, so you'll probably be better off posting a diff to our
>> ReviewBoard, or pushing a branch to github for folks to peek at.
>>
>>
>> -=Bill
>>
>> On Thu, Apr 9, 2015 at 8:49 AM, Michael Schenck <sc...@hioscar.com>
>> wrote:
>>
>> > Hi all,
>> >
>> > I need to be able to plot state transitions from a Job-level at bare
>> > minimum. I've put together a patch to thermos-observer (attached:
>> > thermos-statsd.patch).
>> >
>> > Now, I understand that the Observer is likely to be going away, so I was
>> > thinking about creating a similar patch for the executor, possibly for
>> > possibly in ThermosTaskRunner.compute_status
>> > <
>> https://github.com/mschenck/aurora/blob/master/src/main/python/apache/aurora/executor/thermos_task_runner.py#L183-L205
>> >.
>> > My last thought would be to include a Process that *tails* the
>> checkpoint
>> > log similarly to the Observer.
>> >
>> > What is the preferred approach to accomplishing this goal?
>> >
>> > Best regards,
>> > Michael Schenck
>> >
>>
>
>
Re: Plotting state transitions
Posted by Michael Schenck <sc...@hioscar.com>.
I have a branch (in Github) on my fork of the repo:
https://github.com/mschenck/aurora/tree/thermos_statsd_state_transitions
I'm happy to push the patch to ReviewBoard, as well.
Can you clarify what you mean by "plan"?
- Michael Schenck
On Thu, Apr 9, 2015 at 12:23 PM, Bill Farner <wf...@apache.org> wrote:
> I may be going on a tangent from what's in your mind, but i think a means
> to visualize this stuff would be really useful. I can see value in a few
> different distinct, but complementary things:
>
> 1. state flow of instances in a job. This would cover transitions between
> states like PENDING and RUNNING, and what happens after termination (based
> on whether the job is a service, cron, etc)
>
> 2. *plan* for Processes in a task, as followed by thermos. This would be
> handy to see that parallel/sequential invocations are as you expect.
>
> 3. plan *execution* of processes in a task, as executed thus far by
> thermos. Sounds like this is what you described above, so this would be a
> yet-to-finish view of (2).
>
> As for your approach thus far, the apache mail server seems to eat
> attachments, so you'll probably be better off posting a diff to our
> ReviewBoard, or pushing a branch to github for folks to peek at.
>
>
> -=Bill
>
> On Thu, Apr 9, 2015 at 8:49 AM, Michael Schenck <sc...@hioscar.com>
> wrote:
>
> > Hi all,
> >
> > I need to be able to plot state transitions from a Job-level at bare
> > minimum. I've put together a patch to thermos-observer (attached:
> > thermos-statsd.patch).
> >
> > Now, I understand that the Observer is likely to be going away, so I was
> > thinking about creating a similar patch for the executor, possibly for
> > possibly in ThermosTaskRunner.compute_status
> > <
> https://github.com/mschenck/aurora/blob/master/src/main/python/apache/aurora/executor/thermos_task_runner.py#L183-L205
> >.
> > My last thought would be to include a Process that *tails* the checkpoint
> > log similarly to the Observer.
> >
> > What is the preferred approach to accomplishing this goal?
> >
> > Best regards,
> > Michael Schenck
> >
>
Re: Plotting state transitions
Posted by Bill Farner <wf...@apache.org>.
I may be going on a tangent from what's in your mind, but i think a means
to visualize this stuff would be really useful. I can see value in a few
different distinct, but complementary things:
1. state flow of instances in a job. This would cover transitions between
states like PENDING and RUNNING, and what happens after termination (based
on whether the job is a service, cron, etc)
2. *plan* for Processes in a task, as followed by thermos. This would be
handy to see that parallel/sequential invocations are as you expect.
3. plan *execution* of processes in a task, as executed thus far by
thermos. Sounds like this is what you described above, so this would be a
yet-to-finish view of (2).
As for your approach thus far, the apache mail server seems to eat
attachments, so you'll probably be better off posting a diff to our
ReviewBoard, or pushing a branch to github for folks to peek at.
-=Bill
On Thu, Apr 9, 2015 at 8:49 AM, Michael Schenck <sc...@hioscar.com> wrote:
> Hi all,
>
> I need to be able to plot state transitions from a Job-level at bare
> minimum. I've put together a patch to thermos-observer (attached:
> thermos-statsd.patch).
>
> Now, I understand that the Observer is likely to be going away, so I was
> thinking about creating a similar patch for the executor, possibly for
> possibly in ThermosTaskRunner.compute_status
> <https://github.com/mschenck/aurora/blob/master/src/main/python/apache/aurora/executor/thermos_task_runner.py#L183-L205>.
> My last thought would be to include a Process that *tails* the checkpoint
> log similarly to the Observer.
>
> What is the preferred approach to accomplishing this goal?
>
> Best regards,
> Michael Schenck
>