You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@aurora.apache.org by Bill Farner <wf...@apache.org> on 2016/04/01 01:11:26 UTC
Are we ready to remove the observer?
Assuming that the vast majority of utility provided by the observer is
sandbox/log browsing - can we remove it and link to sandbox browsing that
mesos provides?
The rest of the information could be (or already is) logged in the sandbox
for the rare debugging scenarios that call for it.
Re: Are we ready to remove the observer?
Posted by Bill Farner <wf...@apache.org>.
>
> There is no process to host the observer UI when a task terminates.
Aha, i didn't realize that was in reference to Steve's comment "have the
executor itself host the observer web UI".
I agree, that approach is encumbered for terminal tasks.
On Mon, Apr 4, 2016 at 5:10 PM, Maxim Khutornenko <ma...@apache.org> wrote:
> Sorry, I should have tried harder explaining myself. There is no
> process to host the observer UI when a task terminates. We still want
> (and arguably more so) to look at terminal task details but since
> there is no process left to host the http server, there is no way to
> access that data in the current way of things.
>
> On Mon, Apr 4, 2016 at 3:54 PM, Bill Farner <wf...@apache.org> wrote:
> >>
> >> It falls apart for terminal tasks when executor process is not running
> >> anymore.
> >
> >
> > This sounds important...can you recall what would not work in that
> > scenario? I figured it would work ~identically because the observer
> > follows the lifecycle of the task sandbox directory.
> >
> > On Mon, Apr 4, 2016 at 2:43 PM, Maxim Khutornenko <ma...@apache.org>
> wrote:
> >
> >> I think we've discussed this option before. It falls apart for
> >> terminal tasks when executor process is not running anymore.
> >>
> >> One of the possible ways forward could be extending Mesos UI to
> >> opportunistically consume task data periodically dumped by an executor
> >> into a json file. That could cover the functionality gap created by
> >> killing the observer and let other frameworks customize their task
> >> views in a standard and pluggable way.
> >>
> >> On Mon, Apr 4, 2016 at 2:31 PM, Steve Niemitz <sn...@apache.org>
> wrote:
> >> > It seems like the easiest path forward would be to have the executor
> >> itself
> >> > host the observer web UI, if the HTTP port for the UI were configured
> as
> >> > just another port on the task, the aurora UI could just link to /mname
> >> for
> >> > that instance.
> >> >
> >> > I think the overall "what is running on this machine" view the
> observer
> >> > displays (if you go to it without a task ID) is much less useful and
> >> could
> >> > probably be removed without much sadness.
> >> >
> >> > On Mon, Apr 4, 2016 at 5:23 PM, Bill Farner <wf...@apache.org>
> wrote:
> >> >
> >> >> >
> >> >> > why don't we revisit the problem from the other direction and see
> if
> >> we
> >> >> > can remove checkpoints?
> >> >>
> >> >>
> >> >> Simplicity, again :-) If it turns out we don't need the observer
> >> anyhow,
> >> >> it saves a lot of time. I'm just poking at different parts to make
> >> sure we
> >> >> can still justify their weight.
> >> >>
> >> >> On Mon, Apr 4, 2016 at 1:54 PM, Zameer Manji <zm...@apache.org>
> wrote:
> >> >>
> >> >> > On Mon, Apr 4, 2016 at 1:47 PM, Bill Farner <wf...@apache.org>
> >> wrote:
> >> >> >
> >> >> > > We clearly have different experiences - i've never really
> benefited
> >> >> from
> >> >> > > viewing the process graph, as most jobs have very simple
> sequences
> >> that
> >> >> > > could be easily explained by a text file in the sandbox. On the
> >> >> > contrary,
> >> >> > > i've encountered people confused by the process graph, the
> observer,
> >> >> and
> >> >> > > sandbox browsing...so i must respectfully disagree that it is
> >> >> universally
> >> >> > > appreciated.
> >> >> > >
> >> >> > > What i'm trying to achieve is simplicity. The observer is an
> extra
> >> >> > moving
> >> >> > > part, and another thing for operators to understand and maintain.
> >> It
> >> >> > also
> >> >> > > couples Aurora to one relatively specific way of running tasks,
> >> which
> >> >> > makes
> >> >> > > it difficult to open new use cases like Docker tasks. Removing
> the
> >> >> > > observer starts to pull on a thread of complexity that i don't
> think
> >> >> > Aurora
> >> >> > > benefits much from, for example state checkpointing by the
> executor.
> >> >> > >
> >> >> > > My goal is not to apply pressure, but to perform a gut check. If
> >> the
> >> >> > > answer is "No", that's fine.
> >> >> > >
> >> >> >
> >> >> >
> >> >> > Bill,
> >> >> >
> >> >> > I think you are pulling on the right thread here but I think
> >> revisiting
> >> >> the
> >> >> > observer is the wrong way of approaching the problem. I also agree
> >> that
> >> >> > Aurora doesn't benefit much from state checkpointing by the
> executor
> >> and
> >> >> > the observer is an extension of that since it provides a read only
> >> human
> >> >> > friendly view of the data in the checkpoints. However, instead of
> >> >> removing
> >> >> > the observer (and degrading the UX around accessing the data in the
> >> >> > checkpoints), why don't we revisit the problem from the other
> >> direction
> >> >> and
> >> >> > see if we can remove checkpoints?
> >> >> >
> >> >> >
> >> >> > --
> >> >> > Zameer Manji
> >> >> >
> >> >>
> >>
>
Re: Are we ready to remove the observer?
Posted by Maxim Khutornenko <ma...@apache.org>.
Sorry, I should have tried harder explaining myself. There is no
process to host the observer UI when a task terminates. We still want
(and arguably more so) to look at terminal task details but since
there is no process left to host the http server, there is no way to
access that data in the current way of things.
On Mon, Apr 4, 2016 at 3:54 PM, Bill Farner <wf...@apache.org> wrote:
>>
>> It falls apart for terminal tasks when executor process is not running
>> anymore.
>
>
> This sounds important...can you recall what would not work in that
> scenario? I figured it would work ~identically because the observer
> follows the lifecycle of the task sandbox directory.
>
> On Mon, Apr 4, 2016 at 2:43 PM, Maxim Khutornenko <ma...@apache.org> wrote:
>
>> I think we've discussed this option before. It falls apart for
>> terminal tasks when executor process is not running anymore.
>>
>> One of the possible ways forward could be extending Mesos UI to
>> opportunistically consume task data periodically dumped by an executor
>> into a json file. That could cover the functionality gap created by
>> killing the observer and let other frameworks customize their task
>> views in a standard and pluggable way.
>>
>> On Mon, Apr 4, 2016 at 2:31 PM, Steve Niemitz <sn...@apache.org> wrote:
>> > It seems like the easiest path forward would be to have the executor
>> itself
>> > host the observer web UI, if the HTTP port for the UI were configured as
>> > just another port on the task, the aurora UI could just link to /mname
>> for
>> > that instance.
>> >
>> > I think the overall "what is running on this machine" view the observer
>> > displays (if you go to it without a task ID) is much less useful and
>> could
>> > probably be removed without much sadness.
>> >
>> > On Mon, Apr 4, 2016 at 5:23 PM, Bill Farner <wf...@apache.org> wrote:
>> >
>> >> >
>> >> > why don't we revisit the problem from the other direction and see if
>> we
>> >> > can remove checkpoints?
>> >>
>> >>
>> >> Simplicity, again :-) If it turns out we don't need the observer
>> anyhow,
>> >> it saves a lot of time. I'm just poking at different parts to make
>> sure we
>> >> can still justify their weight.
>> >>
>> >> On Mon, Apr 4, 2016 at 1:54 PM, Zameer Manji <zm...@apache.org> wrote:
>> >>
>> >> > On Mon, Apr 4, 2016 at 1:47 PM, Bill Farner <wf...@apache.org>
>> wrote:
>> >> >
>> >> > > We clearly have different experiences - i've never really benefited
>> >> from
>> >> > > viewing the process graph, as most jobs have very simple sequences
>> that
>> >> > > could be easily explained by a text file in the sandbox. On the
>> >> > contrary,
>> >> > > i've encountered people confused by the process graph, the observer,
>> >> and
>> >> > > sandbox browsing...so i must respectfully disagree that it is
>> >> universally
>> >> > > appreciated.
>> >> > >
>> >> > > What i'm trying to achieve is simplicity. The observer is an extra
>> >> > moving
>> >> > > part, and another thing for operators to understand and maintain.
>> It
>> >> > also
>> >> > > couples Aurora to one relatively specific way of running tasks,
>> which
>> >> > makes
>> >> > > it difficult to open new use cases like Docker tasks. Removing the
>> >> > > observer starts to pull on a thread of complexity that i don't think
>> >> > Aurora
>> >> > > benefits much from, for example state checkpointing by the executor.
>> >> > >
>> >> > > My goal is not to apply pressure, but to perform a gut check. If
>> the
>> >> > > answer is "No", that's fine.
>> >> > >
>> >> >
>> >> >
>> >> > Bill,
>> >> >
>> >> > I think you are pulling on the right thread here but I think
>> revisiting
>> >> the
>> >> > observer is the wrong way of approaching the problem. I also agree
>> that
>> >> > Aurora doesn't benefit much from state checkpointing by the executor
>> and
>> >> > the observer is an extension of that since it provides a read only
>> human
>> >> > friendly view of the data in the checkpoints. However, instead of
>> >> removing
>> >> > the observer (and degrading the UX around accessing the data in the
>> >> > checkpoints), why don't we revisit the problem from the other
>> direction
>> >> and
>> >> > see if we can remove checkpoints?
>> >> >
>> >> >
>> >> > --
>> >> > Zameer Manji
>> >> >
>> >>
>>
Re: Are we ready to remove the observer?
Posted by Bill Farner <wf...@apache.org>.
>
> It falls apart for terminal tasks when executor process is not running
> anymore.
This sounds important...can you recall what would not work in that
scenario? I figured it would work ~identically because the observer
follows the lifecycle of the task sandbox directory.
On Mon, Apr 4, 2016 at 2:43 PM, Maxim Khutornenko <ma...@apache.org> wrote:
> I think we've discussed this option before. It falls apart for
> terminal tasks when executor process is not running anymore.
>
> One of the possible ways forward could be extending Mesos UI to
> opportunistically consume task data periodically dumped by an executor
> into a json file. That could cover the functionality gap created by
> killing the observer and let other frameworks customize their task
> views in a standard and pluggable way.
>
> On Mon, Apr 4, 2016 at 2:31 PM, Steve Niemitz <sn...@apache.org> wrote:
> > It seems like the easiest path forward would be to have the executor
> itself
> > host the observer web UI, if the HTTP port for the UI were configured as
> > just another port on the task, the aurora UI could just link to /mname
> for
> > that instance.
> >
> > I think the overall "what is running on this machine" view the observer
> > displays (if you go to it without a task ID) is much less useful and
> could
> > probably be removed without much sadness.
> >
> > On Mon, Apr 4, 2016 at 5:23 PM, Bill Farner <wf...@apache.org> wrote:
> >
> >> >
> >> > why don't we revisit the problem from the other direction and see if
> we
> >> > can remove checkpoints?
> >>
> >>
> >> Simplicity, again :-) If it turns out we don't need the observer
> anyhow,
> >> it saves a lot of time. I'm just poking at different parts to make
> sure we
> >> can still justify their weight.
> >>
> >> On Mon, Apr 4, 2016 at 1:54 PM, Zameer Manji <zm...@apache.org> wrote:
> >>
> >> > On Mon, Apr 4, 2016 at 1:47 PM, Bill Farner <wf...@apache.org>
> wrote:
> >> >
> >> > > We clearly have different experiences - i've never really benefited
> >> from
> >> > > viewing the process graph, as most jobs have very simple sequences
> that
> >> > > could be easily explained by a text file in the sandbox. On the
> >> > contrary,
> >> > > i've encountered people confused by the process graph, the observer,
> >> and
> >> > > sandbox browsing...so i must respectfully disagree that it is
> >> universally
> >> > > appreciated.
> >> > >
> >> > > What i'm trying to achieve is simplicity. The observer is an extra
> >> > moving
> >> > > part, and another thing for operators to understand and maintain.
> It
> >> > also
> >> > > couples Aurora to one relatively specific way of running tasks,
> which
> >> > makes
> >> > > it difficult to open new use cases like Docker tasks. Removing the
> >> > > observer starts to pull on a thread of complexity that i don't think
> >> > Aurora
> >> > > benefits much from, for example state checkpointing by the executor.
> >> > >
> >> > > My goal is not to apply pressure, but to perform a gut check. If
> the
> >> > > answer is "No", that's fine.
> >> > >
> >> >
> >> >
> >> > Bill,
> >> >
> >> > I think you are pulling on the right thread here but I think
> revisiting
> >> the
> >> > observer is the wrong way of approaching the problem. I also agree
> that
> >> > Aurora doesn't benefit much from state checkpointing by the executor
> and
> >> > the observer is an extension of that since it provides a read only
> human
> >> > friendly view of the data in the checkpoints. However, instead of
> >> removing
> >> > the observer (and degrading the UX around accessing the data in the
> >> > checkpoints), why don't we revisit the problem from the other
> direction
> >> and
> >> > see if we can remove checkpoints?
> >> >
> >> >
> >> > --
> >> > Zameer Manji
> >> >
> >>
>
Re: Are we ready to remove the observer?
Posted by Maxim Khutornenko <ma...@apache.org>.
I think we've discussed this option before. It falls apart for
terminal tasks when executor process is not running anymore.
One of the possible ways forward could be extending Mesos UI to
opportunistically consume task data periodically dumped by an executor
into a json file. That could cover the functionality gap created by
killing the observer and let other frameworks customize their task
views in a standard and pluggable way.
On Mon, Apr 4, 2016 at 2:31 PM, Steve Niemitz <sn...@apache.org> wrote:
> It seems like the easiest path forward would be to have the executor itself
> host the observer web UI, if the HTTP port for the UI were configured as
> just another port on the task, the aurora UI could just link to /mname for
> that instance.
>
> I think the overall "what is running on this machine" view the observer
> displays (if you go to it without a task ID) is much less useful and could
> probably be removed without much sadness.
>
> On Mon, Apr 4, 2016 at 5:23 PM, Bill Farner <wf...@apache.org> wrote:
>
>> >
>> > why don't we revisit the problem from the other direction and see if we
>> > can remove checkpoints?
>>
>>
>> Simplicity, again :-) If it turns out we don't need the observer anyhow,
>> it saves a lot of time. I'm just poking at different parts to make sure we
>> can still justify their weight.
>>
>> On Mon, Apr 4, 2016 at 1:54 PM, Zameer Manji <zm...@apache.org> wrote:
>>
>> > On Mon, Apr 4, 2016 at 1:47 PM, Bill Farner <wf...@apache.org> wrote:
>> >
>> > > We clearly have different experiences - i've never really benefited
>> from
>> > > viewing the process graph, as most jobs have very simple sequences that
>> > > could be easily explained by a text file in the sandbox. On the
>> > contrary,
>> > > i've encountered people confused by the process graph, the observer,
>> and
>> > > sandbox browsing...so i must respectfully disagree that it is
>> universally
>> > > appreciated.
>> > >
>> > > What i'm trying to achieve is simplicity. The observer is an extra
>> > moving
>> > > part, and another thing for operators to understand and maintain. It
>> > also
>> > > couples Aurora to one relatively specific way of running tasks, which
>> > makes
>> > > it difficult to open new use cases like Docker tasks. Removing the
>> > > observer starts to pull on a thread of complexity that i don't think
>> > Aurora
>> > > benefits much from, for example state checkpointing by the executor.
>> > >
>> > > My goal is not to apply pressure, but to perform a gut check. If the
>> > > answer is "No", that's fine.
>> > >
>> >
>> >
>> > Bill,
>> >
>> > I think you are pulling on the right thread here but I think revisiting
>> the
>> > observer is the wrong way of approaching the problem. I also agree that
>> > Aurora doesn't benefit much from state checkpointing by the executor and
>> > the observer is an extension of that since it provides a read only human
>> > friendly view of the data in the checkpoints. However, instead of
>> removing
>> > the observer (and degrading the UX around accessing the data in the
>> > checkpoints), why don't we revisit the problem from the other direction
>> and
>> > see if we can remove checkpoints?
>> >
>> >
>> > --
>> > Zameer Manji
>> >
>>
Re: Are we ready to remove the observer?
Posted by Steve Niemitz <sn...@apache.org>.
It seems like the easiest path forward would be to have the executor itself
host the observer web UI, if the HTTP port for the UI were configured as
just another port on the task, the aurora UI could just link to /mname for
that instance.
I think the overall "what is running on this machine" view the observer
displays (if you go to it without a task ID) is much less useful and could
probably be removed without much sadness.
On Mon, Apr 4, 2016 at 5:23 PM, Bill Farner <wf...@apache.org> wrote:
> >
> > why don't we revisit the problem from the other direction and see if we
> > can remove checkpoints?
>
>
> Simplicity, again :-) If it turns out we don't need the observer anyhow,
> it saves a lot of time. I'm just poking at different parts to make sure we
> can still justify their weight.
>
> On Mon, Apr 4, 2016 at 1:54 PM, Zameer Manji <zm...@apache.org> wrote:
>
> > On Mon, Apr 4, 2016 at 1:47 PM, Bill Farner <wf...@apache.org> wrote:
> >
> > > We clearly have different experiences - i've never really benefited
> from
> > > viewing the process graph, as most jobs have very simple sequences that
> > > could be easily explained by a text file in the sandbox. On the
> > contrary,
> > > i've encountered people confused by the process graph, the observer,
> and
> > > sandbox browsing...so i must respectfully disagree that it is
> universally
> > > appreciated.
> > >
> > > What i'm trying to achieve is simplicity. The observer is an extra
> > moving
> > > part, and another thing for operators to understand and maintain. It
> > also
> > > couples Aurora to one relatively specific way of running tasks, which
> > makes
> > > it difficult to open new use cases like Docker tasks. Removing the
> > > observer starts to pull on a thread of complexity that i don't think
> > Aurora
> > > benefits much from, for example state checkpointing by the executor.
> > >
> > > My goal is not to apply pressure, but to perform a gut check. If the
> > > answer is "No", that's fine.
> > >
> >
> >
> > Bill,
> >
> > I think you are pulling on the right thread here but I think revisiting
> the
> > observer is the wrong way of approaching the problem. I also agree that
> > Aurora doesn't benefit much from state checkpointing by the executor and
> > the observer is an extension of that since it provides a read only human
> > friendly view of the data in the checkpoints. However, instead of
> removing
> > the observer (and degrading the UX around accessing the data in the
> > checkpoints), why don't we revisit the problem from the other direction
> and
> > see if we can remove checkpoints?
> >
> >
> > --
> > Zameer Manji
> >
>
Re: Are we ready to remove the observer?
Posted by Bill Farner <wf...@apache.org>.
>
> why don't we revisit the problem from the other direction and see if we
> can remove checkpoints?
Simplicity, again :-) If it turns out we don't need the observer anyhow,
it saves a lot of time. I'm just poking at different parts to make sure we
can still justify their weight.
On Mon, Apr 4, 2016 at 1:54 PM, Zameer Manji <zm...@apache.org> wrote:
> On Mon, Apr 4, 2016 at 1:47 PM, Bill Farner <wf...@apache.org> wrote:
>
> > We clearly have different experiences - i've never really benefited from
> > viewing the process graph, as most jobs have very simple sequences that
> > could be easily explained by a text file in the sandbox. On the
> contrary,
> > i've encountered people confused by the process graph, the observer, and
> > sandbox browsing...so i must respectfully disagree that it is universally
> > appreciated.
> >
> > What i'm trying to achieve is simplicity. The observer is an extra
> moving
> > part, and another thing for operators to understand and maintain. It
> also
> > couples Aurora to one relatively specific way of running tasks, which
> makes
> > it difficult to open new use cases like Docker tasks. Removing the
> > observer starts to pull on a thread of complexity that i don't think
> Aurora
> > benefits much from, for example state checkpointing by the executor.
> >
> > My goal is not to apply pressure, but to perform a gut check. If the
> > answer is "No", that's fine.
> >
>
>
> Bill,
>
> I think you are pulling on the right thread here but I think revisiting the
> observer is the wrong way of approaching the problem. I also agree that
> Aurora doesn't benefit much from state checkpointing by the executor and
> the observer is an extension of that since it provides a read only human
> friendly view of the data in the checkpoints. However, instead of removing
> the observer (and degrading the UX around accessing the data in the
> checkpoints), why don't we revisit the problem from the other direction and
> see if we can remove checkpoints?
>
>
> --
> Zameer Manji
>
Re: Are we ready to remove the observer?
Posted by Zameer Manji <zm...@apache.org>.
On Mon, Apr 4, 2016 at 1:47 PM, Bill Farner <wf...@apache.org> wrote:
> We clearly have different experiences - i've never really benefited from
> viewing the process graph, as most jobs have very simple sequences that
> could be easily explained by a text file in the sandbox. On the contrary,
> i've encountered people confused by the process graph, the observer, and
> sandbox browsing...so i must respectfully disagree that it is universally
> appreciated.
>
> What i'm trying to achieve is simplicity. The observer is an extra moving
> part, and another thing for operators to understand and maintain. It also
> couples Aurora to one relatively specific way of running tasks, which makes
> it difficult to open new use cases like Docker tasks. Removing the
> observer starts to pull on a thread of complexity that i don't think Aurora
> benefits much from, for example state checkpointing by the executor.
>
> My goal is not to apply pressure, but to perform a gut check. If the
> answer is "No", that's fine.
>
Bill,
I think you are pulling on the right thread here but I think revisiting the
observer is the wrong way of approaching the problem. I also agree that
Aurora doesn't benefit much from state checkpointing by the executor and
the observer is an extension of that since it provides a read only human
friendly view of the data in the checkpoints. However, instead of removing
the observer (and degrading the UX around accessing the data in the
checkpoints), why don't we revisit the problem from the other direction and
see if we can remove checkpoints?
--
Zameer Manji
Re: Are we ready to remove the observer?
Posted by Jeff Schroeder <je...@computer.org>.
But the executor would still stay around thereby distributing healthchecks,
correct?
On Monday, April 4, 2016, Bill Farner <wf...@apache.org> wrote:
> We clearly have different experiences - i've never really benefited from
> viewing the process graph, as most jobs have very simple sequences that
> could be easily explained by a text file in the sandbox. On the contrary,
> i've encountered people confused by the process graph, the observer, and
> sandbox browsing...so i must respectfully disagree that it is universally
> appreciated.
>
> What i'm trying to achieve is simplicity. The observer is an extra moving
> part, and another thing for operators to understand and maintain. It also
> couples Aurora to one relatively specific way of running tasks, which makes
> it difficult to open new use cases like Docker tasks. Removing the
> observer starts to pull on a thread of complexity that i don't think Aurora
> benefits much from, for example state checkpointing by the executor.
>
> My goal is not to apply pressure, but to perform a gut check. If the
> answer is "No", that's fine.
>
> On Mon, Apr 4, 2016 at 1:01 PM, Maxim Khutornenko <maxim@apache.org
> <javascript:;>> wrote:
>
> > I am with Josh on this one. Thermos Observer UI (and especially its
> > process graph) is one of the features universally appreciated by our
> > customers. I am all for deprecating the Observer but only in way that
> > retains parity with the existing functionality and hopefully enhances
> > it. What are we trying to achieve here that would justify losing some
> > of our feature set?
> >
> > On Mon, Apr 4, 2016 at 12:42 PM, Erb, Stephan
> > <Stephan.Erb@blue-yonder.com <javascript:;>> wrote:
> > > Have you recently looked at the Mesos UI, Joshua? It offers sandbox
> > browsing similar to the chroot link of Thermos. So at least you don't
> have
> > to do SSH into any box. We could link to that Mesos UI instead of the
> > Thermos one, and Mesos could then serve a nice index.html that contains
> the
> > content that was formerly served by Thermos.
> > >
> > > When dropping Thermos and relying on Mesos instead, we could profit
> from
> > the recent addition such as authentication.
> > >
> > >
> > > ________________________________________
> > > From: Joshua Cohen <jcohen@apache.org <javascript:;>>
> > > Sent: Monday, April 4, 2016 18:42
> > > To: dev@aurora.apache.org <javascript:;>
> > > Subject: Re: Are we ready to remove the observer?
> > >
> > > If you're suggesting just going to the task directory and pulling them
> > out
> > > of the executor logs. Yes, I could ssh into the host the task is
> running
> > on
> > > and grep the task directory out of the mesos agent logs and then trawl
> > the
> > > logs (or cat task.json), but that's much more effort than going to the
> > > observer's task UI (i.e. it'd take a minute, rather than a few
> seconds).
> > > I'd also posit that it's much easier for new Aurora operators to come
> to
> > > grips with the process tree via the UI rather than a JSON blob.
> > >
> > > If you're suggesting something else (i.e. new UI to expose these
> separate
> > > from the Observer), I'm fine with that, that's what I was implying
> above
> > > would be necessary before I think we could retire the Observer.
> > >
> > > A counter question: do people feel that updating/deploying the Observer
> > is
> > > a major obstacle? I know we've got the process well automated, so it's
> > > relatively painless. I'd love to replace the Observer with something
> > > better, but I don't feel like it's a major drag on our productivity as
> it
> > > exists today to warrant killing it off entirely. My opinion may be
> > colored
> > > by the deploy automation we have in place though!
> > >
> > > On Mon, Apr 4, 2016 at 9:32 AM, Bill Farner <wfarner@apache.org
> <javascript:;>> wrote:
> > >
> > >> >
> > >> > 2) Providing an easy view of a process's command-line
> > >> > 3) Providing a holistic view of the task config
> > >>
> > >>
> > >> Just to check my understanding - these could be trivially handled in
> > >> text/log format, right?
> > >>
> > >> On Mon, Apr 4, 2016 at 9:30 AM, Joshua Cohen <jcohen@apache.org
> <javascript:;>> wrote:
> > >>
> > >> > I'm -1 on this until we have an actual replacement for the
> Observer. I
> > >> > think that the observer provides significant value outside of just
> > >> sandbox
> > >> > browsing:
> > >> >
> > >> > 1) Exporting task-level statistics.
> > >> > 2) Providing an easy view of a process's command-line
> > >> > 3) Providing a holistic view of the task config
> > >> > 4) Real time utilization stats
> > >> >
> > >> > As a cluster operator, I use all of these features on a daily basis
> > >> > (especially when I'm on call) in addition to sandbox browsing, so I
> > don't
> > >> > think that these uses cases are that rare.
> > >> >
> > >> > On Fri, Apr 1, 2016 at 6:55 AM, Steve Niemitz <sniemitz@apache.org
> <javascript:;>>
> > >> wrote:
> > >> >
> > >> > > The per-process stats have never been very useful to us (since
> they
> > >> don't
> > >> > > work for docker), however, even being able to see the processes
> that
> > >> are
> > >> > > running, how many times they've restarted, when they launched, etc
> > is
> > >> > > invaluable.
> > >> > >
> > >> > > I think there would be big pushback from users if they were to
> lose
> > the
> > >> > > functionality it provided currently (beyond log viewing).
> > >> > >
> > >> > > On Fri, Apr 1, 2016 at 6:58 AM, Erb, Stephan <
> > >> > Stephan.Erb@blue-yonder.com <javascript:;>>
> > >> > > wrote:
> > >> > >
> > >> > > > From an operator and Aurora developer perspective, it would be
> > really
> > >> > > > great to get rid of the thermos observer quickly.
> > >> > > >
> > >> > > > However, from a user perspective the usability gap between
> > observer
> > >> and
> > >> > > > plain Mesos sandbox browsing is quite large right now. I agree
> > with
> > >> > > > Benjamin here that it would probably work if we generate html
> > pages
> > >> > ready
> > >> > > > for user consumption.
> > >> > > >
> > >> > > > These are the relevant tickets in our tracker:
> > >> > > > * https://issues.apache.org/jira/browse/AURORA-725
> > >> > > > * https://issues.apache.org/jira/browse/AURORA-777
> > >> > > >
> > >> > > > ________________________________________
> > >> > > > From: benley@gmail.com <javascript:;> <benley@gmail.com
> <javascript:;>>
> > >> > > > Sent: Friday, April 1, 2016 02:35
> > >> > > > To: dev@aurora.apache.org <javascript:;>
> > >> > > > Subject: Re: Are we ready to remove the observer?
> > >> > > >
> > >> > > > Is there any chance we can keep the per-process cpu and ram
> > >> utilization
> > >> > > > stats? That's one of the coolest things about aurora, imo. The
> > >> > executor
> > >> > > > is already writing those checkpoints inside the mesos sandbox (I
> > >> > think?),
> > >> > > > so perhaps it could also produce the html pages that the
> observer
> > >> > > currently
> > >> > > > renders?
> > >> > > >
> > >> > > > On Thu, Mar 31, 2016 at 4:33 PM Zhitao Li <
> zhitaoli.cs@gmail.com <javascript:;>>
> > >> > wrote:
> > >> > > >
> > >> > > > > +1.
> > >> > > > >
> > >> > > > > On Thu, Mar 31, 2016 at 4:11 PM, Bill Farner <
> > wfarner@apache.org <javascript:;>>
> > >> > > wrote:
> > >> > > > >
> > >> > > > > > Assuming that the vast majority of utility provided by the
> > >> observer
> > >> > > is
> > >> > > > > > sandbox/log browsing - can we remove it and link to sandbox
> > >> > browsing
> > >> > > > that
> > >> > > > > > mesos provides?
> > >> > > > > >
> > >> > > > > > The rest of the information could be (or already is) logged
> in
> > >> the
> > >> > > > > sandbox
> > >> > > > > > for the rare debugging scenarios that call for it.
> > >> > > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > > --
> > >> > > > > Cheers,
> > >> > > > >
> > >> > > > > Zhitao Li
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
>
--
Text by Jeff, typos by iPhone
Re: Are we ready to remove the observer?
Posted by Bill Farner <wf...@apache.org>.
We clearly have different experiences - i've never really benefited from
viewing the process graph, as most jobs have very simple sequences that
could be easily explained by a text file in the sandbox. On the contrary,
i've encountered people confused by the process graph, the observer, and
sandbox browsing...so i must respectfully disagree that it is universally
appreciated.
What i'm trying to achieve is simplicity. The observer is an extra moving
part, and another thing for operators to understand and maintain. It also
couples Aurora to one relatively specific way of running tasks, which makes
it difficult to open new use cases like Docker tasks. Removing the
observer starts to pull on a thread of complexity that i don't think Aurora
benefits much from, for example state checkpointing by the executor.
My goal is not to apply pressure, but to perform a gut check. If the
answer is "No", that's fine.
On Mon, Apr 4, 2016 at 1:01 PM, Maxim Khutornenko <ma...@apache.org> wrote:
> I am with Josh on this one. Thermos Observer UI (and especially its
> process graph) is one of the features universally appreciated by our
> customers. I am all for deprecating the Observer but only in way that
> retains parity with the existing functionality and hopefully enhances
> it. What are we trying to achieve here that would justify losing some
> of our feature set?
>
> On Mon, Apr 4, 2016 at 12:42 PM, Erb, Stephan
> <St...@blue-yonder.com> wrote:
> > Have you recently looked at the Mesos UI, Joshua? It offers sandbox
> browsing similar to the chroot link of Thermos. So at least you don't have
> to do SSH into any box. We could link to that Mesos UI instead of the
> Thermos one, and Mesos could then serve a nice index.html that contains the
> content that was formerly served by Thermos.
> >
> > When dropping Thermos and relying on Mesos instead, we could profit from
> the recent addition such as authentication.
> >
> >
> > ________________________________________
> > From: Joshua Cohen <jc...@apache.org>
> > Sent: Monday, April 4, 2016 18:42
> > To: dev@aurora.apache.org
> > Subject: Re: Are we ready to remove the observer?
> >
> > If you're suggesting just going to the task directory and pulling them
> out
> > of the executor logs. Yes, I could ssh into the host the task is running
> on
> > and grep the task directory out of the mesos agent logs and then trawl
> the
> > logs (or cat task.json), but that's much more effort than going to the
> > observer's task UI (i.e. it'd take a minute, rather than a few seconds).
> > I'd also posit that it's much easier for new Aurora operators to come to
> > grips with the process tree via the UI rather than a JSON blob.
> >
> > If you're suggesting something else (i.e. new UI to expose these separate
> > from the Observer), I'm fine with that, that's what I was implying above
> > would be necessary before I think we could retire the Observer.
> >
> > A counter question: do people feel that updating/deploying the Observer
> is
> > a major obstacle? I know we've got the process well automated, so it's
> > relatively painless. I'd love to replace the Observer with something
> > better, but I don't feel like it's a major drag on our productivity as it
> > exists today to warrant killing it off entirely. My opinion may be
> colored
> > by the deploy automation we have in place though!
> >
> > On Mon, Apr 4, 2016 at 9:32 AM, Bill Farner <wf...@apache.org> wrote:
> >
> >> >
> >> > 2) Providing an easy view of a process's command-line
> >> > 3) Providing a holistic view of the task config
> >>
> >>
> >> Just to check my understanding - these could be trivially handled in
> >> text/log format, right?
> >>
> >> On Mon, Apr 4, 2016 at 9:30 AM, Joshua Cohen <jc...@apache.org> wrote:
> >>
> >> > I'm -1 on this until we have an actual replacement for the Observer. I
> >> > think that the observer provides significant value outside of just
> >> sandbox
> >> > browsing:
> >> >
> >> > 1) Exporting task-level statistics.
> >> > 2) Providing an easy view of a process's command-line
> >> > 3) Providing a holistic view of the task config
> >> > 4) Real time utilization stats
> >> >
> >> > As a cluster operator, I use all of these features on a daily basis
> >> > (especially when I'm on call) in addition to sandbox browsing, so I
> don't
> >> > think that these uses cases are that rare.
> >> >
> >> > On Fri, Apr 1, 2016 at 6:55 AM, Steve Niemitz <sn...@apache.org>
> >> wrote:
> >> >
> >> > > The per-process stats have never been very useful to us (since they
> >> don't
> >> > > work for docker), however, even being able to see the processes that
> >> are
> >> > > running, how many times they've restarted, when they launched, etc
> is
> >> > > invaluable.
> >> > >
> >> > > I think there would be big pushback from users if they were to lose
> the
> >> > > functionality it provided currently (beyond log viewing).
> >> > >
> >> > > On Fri, Apr 1, 2016 at 6:58 AM, Erb, Stephan <
> >> > Stephan.Erb@blue-yonder.com>
> >> > > wrote:
> >> > >
> >> > > > From an operator and Aurora developer perspective, it would be
> really
> >> > > > great to get rid of the thermos observer quickly.
> >> > > >
> >> > > > However, from a user perspective the usability gap between
> observer
> >> and
> >> > > > plain Mesos sandbox browsing is quite large right now. I agree
> with
> >> > > > Benjamin here that it would probably work if we generate html
> pages
> >> > ready
> >> > > > for user consumption.
> >> > > >
> >> > > > These are the relevant tickets in our tracker:
> >> > > > * https://issues.apache.org/jira/browse/AURORA-725
> >> > > > * https://issues.apache.org/jira/browse/AURORA-777
> >> > > >
> >> > > > ________________________________________
> >> > > > From: benley@gmail.com <be...@gmail.com>
> >> > > > Sent: Friday, April 1, 2016 02:35
> >> > > > To: dev@aurora.apache.org
> >> > > > Subject: Re: Are we ready to remove the observer?
> >> > > >
> >> > > > Is there any chance we can keep the per-process cpu and ram
> >> utilization
> >> > > > stats? That's one of the coolest things about aurora, imo. The
> >> > executor
> >> > > > is already writing those checkpoints inside the mesos sandbox (I
> >> > think?),
> >> > > > so perhaps it could also produce the html pages that the observer
> >> > > currently
> >> > > > renders?
> >> > > >
> >> > > > On Thu, Mar 31, 2016 at 4:33 PM Zhitao Li <zh...@gmail.com>
> >> > wrote:
> >> > > >
> >> > > > > +1.
> >> > > > >
> >> > > > > On Thu, Mar 31, 2016 at 4:11 PM, Bill Farner <
> wfarner@apache.org>
> >> > > wrote:
> >> > > > >
> >> > > > > > Assuming that the vast majority of utility provided by the
> >> observer
> >> > > is
> >> > > > > > sandbox/log browsing - can we remove it and link to sandbox
> >> > browsing
> >> > > > that
> >> > > > > > mesos provides?
> >> > > > > >
> >> > > > > > The rest of the information could be (or already is) logged in
> >> the
> >> > > > > sandbox
> >> > > > > > for the rare debugging scenarios that call for it.
> >> > > > > >
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > --
> >> > > > > Cheers,
> >> > > > >
> >> > > > > Zhitao Li
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>
Re: Are we ready to remove the observer?
Posted by Maxim Khutornenko <ma...@apache.org>.
I am with Josh on this one. Thermos Observer UI (and especially its
process graph) is one of the features universally appreciated by our
customers. I am all for deprecating the Observer but only in way that
retains parity with the existing functionality and hopefully enhances
it. What are we trying to achieve here that would justify losing some
of our feature set?
On Mon, Apr 4, 2016 at 12:42 PM, Erb, Stephan
<St...@blue-yonder.com> wrote:
> Have you recently looked at the Mesos UI, Joshua? It offers sandbox browsing similar to the chroot link of Thermos. So at least you don't have to do SSH into any box. We could link to that Mesos UI instead of the Thermos one, and Mesos could then serve a nice index.html that contains the content that was formerly served by Thermos.
>
> When dropping Thermos and relying on Mesos instead, we could profit from the recent addition such as authentication.
>
>
> ________________________________________
> From: Joshua Cohen <jc...@apache.org>
> Sent: Monday, April 4, 2016 18:42
> To: dev@aurora.apache.org
> Subject: Re: Are we ready to remove the observer?
>
> If you're suggesting just going to the task directory and pulling them out
> of the executor logs. Yes, I could ssh into the host the task is running on
> and grep the task directory out of the mesos agent logs and then trawl the
> logs (or cat task.json), but that's much more effort than going to the
> observer's task UI (i.e. it'd take a minute, rather than a few seconds).
> I'd also posit that it's much easier for new Aurora operators to come to
> grips with the process tree via the UI rather than a JSON blob.
>
> If you're suggesting something else (i.e. new UI to expose these separate
> from the Observer), I'm fine with that, that's what I was implying above
> would be necessary before I think we could retire the Observer.
>
> A counter question: do people feel that updating/deploying the Observer is
> a major obstacle? I know we've got the process well automated, so it's
> relatively painless. I'd love to replace the Observer with something
> better, but I don't feel like it's a major drag on our productivity as it
> exists today to warrant killing it off entirely. My opinion may be colored
> by the deploy automation we have in place though!
>
> On Mon, Apr 4, 2016 at 9:32 AM, Bill Farner <wf...@apache.org> wrote:
>
>> >
>> > 2) Providing an easy view of a process's command-line
>> > 3) Providing a holistic view of the task config
>>
>>
>> Just to check my understanding - these could be trivially handled in
>> text/log format, right?
>>
>> On Mon, Apr 4, 2016 at 9:30 AM, Joshua Cohen <jc...@apache.org> wrote:
>>
>> > I'm -1 on this until we have an actual replacement for the Observer. I
>> > think that the observer provides significant value outside of just
>> sandbox
>> > browsing:
>> >
>> > 1) Exporting task-level statistics.
>> > 2) Providing an easy view of a process's command-line
>> > 3) Providing a holistic view of the task config
>> > 4) Real time utilization stats
>> >
>> > As a cluster operator, I use all of these features on a daily basis
>> > (especially when I'm on call) in addition to sandbox browsing, so I don't
>> > think that these uses cases are that rare.
>> >
>> > On Fri, Apr 1, 2016 at 6:55 AM, Steve Niemitz <sn...@apache.org>
>> wrote:
>> >
>> > > The per-process stats have never been very useful to us (since they
>> don't
>> > > work for docker), however, even being able to see the processes that
>> are
>> > > running, how many times they've restarted, when they launched, etc is
>> > > invaluable.
>> > >
>> > > I think there would be big pushback from users if they were to lose the
>> > > functionality it provided currently (beyond log viewing).
>> > >
>> > > On Fri, Apr 1, 2016 at 6:58 AM, Erb, Stephan <
>> > Stephan.Erb@blue-yonder.com>
>> > > wrote:
>> > >
>> > > > From an operator and Aurora developer perspective, it would be really
>> > > > great to get rid of the thermos observer quickly.
>> > > >
>> > > > However, from a user perspective the usability gap between observer
>> and
>> > > > plain Mesos sandbox browsing is quite large right now. I agree with
>> > > > Benjamin here that it would probably work if we generate html pages
>> > ready
>> > > > for user consumption.
>> > > >
>> > > > These are the relevant tickets in our tracker:
>> > > > * https://issues.apache.org/jira/browse/AURORA-725
>> > > > * https://issues.apache.org/jira/browse/AURORA-777
>> > > >
>> > > > ________________________________________
>> > > > From: benley@gmail.com <be...@gmail.com>
>> > > > Sent: Friday, April 1, 2016 02:35
>> > > > To: dev@aurora.apache.org
>> > > > Subject: Re: Are we ready to remove the observer?
>> > > >
>> > > > Is there any chance we can keep the per-process cpu and ram
>> utilization
>> > > > stats? That's one of the coolest things about aurora, imo. The
>> > executor
>> > > > is already writing those checkpoints inside the mesos sandbox (I
>> > think?),
>> > > > so perhaps it could also produce the html pages that the observer
>> > > currently
>> > > > renders?
>> > > >
>> > > > On Thu, Mar 31, 2016 at 4:33 PM Zhitao Li <zh...@gmail.com>
>> > wrote:
>> > > >
>> > > > > +1.
>> > > > >
>> > > > > On Thu, Mar 31, 2016 at 4:11 PM, Bill Farner <wf...@apache.org>
>> > > wrote:
>> > > > >
>> > > > > > Assuming that the vast majority of utility provided by the
>> observer
>> > > is
>> > > > > > sandbox/log browsing - can we remove it and link to sandbox
>> > browsing
>> > > > that
>> > > > > > mesos provides?
>> > > > > >
>> > > > > > The rest of the information could be (or already is) logged in
>> the
>> > > > > sandbox
>> > > > > > for the rare debugging scenarios that call for it.
>> > > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > --
>> > > > > Cheers,
>> > > > >
>> > > > > Zhitao Li
>> > > > >
>> > > >
>> > >
>> >
>>
Re: Are we ready to remove the observer?
Posted by "Erb, Stephan" <St...@blue-yonder.com>.
Have you recently looked at the Mesos UI, Joshua? It offers sandbox browsing similar to the chroot link of Thermos. So at least you don't have to do SSH into any box. We could link to that Mesos UI instead of the Thermos one, and Mesos could then serve a nice index.html that contains the content that was formerly served by Thermos.
When dropping Thermos and relying on Mesos instead, we could profit from the recent addition such as authentication.
________________________________________
From: Joshua Cohen <jc...@apache.org>
Sent: Monday, April 4, 2016 18:42
To: dev@aurora.apache.org
Subject: Re: Are we ready to remove the observer?
If you're suggesting just going to the task directory and pulling them out
of the executor logs. Yes, I could ssh into the host the task is running on
and grep the task directory out of the mesos agent logs and then trawl the
logs (or cat task.json), but that's much more effort than going to the
observer's task UI (i.e. it'd take a minute, rather than a few seconds).
I'd also posit that it's much easier for new Aurora operators to come to
grips with the process tree via the UI rather than a JSON blob.
If you're suggesting something else (i.e. new UI to expose these separate
from the Observer), I'm fine with that, that's what I was implying above
would be necessary before I think we could retire the Observer.
A counter question: do people feel that updating/deploying the Observer is
a major obstacle? I know we've got the process well automated, so it's
relatively painless. I'd love to replace the Observer with something
better, but I don't feel like it's a major drag on our productivity as it
exists today to warrant killing it off entirely. My opinion may be colored
by the deploy automation we have in place though!
On Mon, Apr 4, 2016 at 9:32 AM, Bill Farner <wf...@apache.org> wrote:
> >
> > 2) Providing an easy view of a process's command-line
> > 3) Providing a holistic view of the task config
>
>
> Just to check my understanding - these could be trivially handled in
> text/log format, right?
>
> On Mon, Apr 4, 2016 at 9:30 AM, Joshua Cohen <jc...@apache.org> wrote:
>
> > I'm -1 on this until we have an actual replacement for the Observer. I
> > think that the observer provides significant value outside of just
> sandbox
> > browsing:
> >
> > 1) Exporting task-level statistics.
> > 2) Providing an easy view of a process's command-line
> > 3) Providing a holistic view of the task config
> > 4) Real time utilization stats
> >
> > As a cluster operator, I use all of these features on a daily basis
> > (especially when I'm on call) in addition to sandbox browsing, so I don't
> > think that these uses cases are that rare.
> >
> > On Fri, Apr 1, 2016 at 6:55 AM, Steve Niemitz <sn...@apache.org>
> wrote:
> >
> > > The per-process stats have never been very useful to us (since they
> don't
> > > work for docker), however, even being able to see the processes that
> are
> > > running, how many times they've restarted, when they launched, etc is
> > > invaluable.
> > >
> > > I think there would be big pushback from users if they were to lose the
> > > functionality it provided currently (beyond log viewing).
> > >
> > > On Fri, Apr 1, 2016 at 6:58 AM, Erb, Stephan <
> > Stephan.Erb@blue-yonder.com>
> > > wrote:
> > >
> > > > From an operator and Aurora developer perspective, it would be really
> > > > great to get rid of the thermos observer quickly.
> > > >
> > > > However, from a user perspective the usability gap between observer
> and
> > > > plain Mesos sandbox browsing is quite large right now. I agree with
> > > > Benjamin here that it would probably work if we generate html pages
> > ready
> > > > for user consumption.
> > > >
> > > > These are the relevant tickets in our tracker:
> > > > * https://issues.apache.org/jira/browse/AURORA-725
> > > > * https://issues.apache.org/jira/browse/AURORA-777
> > > >
> > > > ________________________________________
> > > > From: benley@gmail.com <be...@gmail.com>
> > > > Sent: Friday, April 1, 2016 02:35
> > > > To: dev@aurora.apache.org
> > > > Subject: Re: Are we ready to remove the observer?
> > > >
> > > > Is there any chance we can keep the per-process cpu and ram
> utilization
> > > > stats? That's one of the coolest things about aurora, imo. The
> > executor
> > > > is already writing those checkpoints inside the mesos sandbox (I
> > think?),
> > > > so perhaps it could also produce the html pages that the observer
> > > currently
> > > > renders?
> > > >
> > > > On Thu, Mar 31, 2016 at 4:33 PM Zhitao Li <zh...@gmail.com>
> > wrote:
> > > >
> > > > > +1.
> > > > >
> > > > > On Thu, Mar 31, 2016 at 4:11 PM, Bill Farner <wf...@apache.org>
> > > wrote:
> > > > >
> > > > > > Assuming that the vast majority of utility provided by the
> observer
> > > is
> > > > > > sandbox/log browsing - can we remove it and link to sandbox
> > browsing
> > > > that
> > > > > > mesos provides?
> > > > > >
> > > > > > The rest of the information could be (or already is) logged in
> the
> > > > > sandbox
> > > > > > for the rare debugging scenarios that call for it.
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Cheers,
> > > > >
> > > > > Zhitao Li
> > > > >
> > > >
> > >
> >
>
Re: Are we ready to remove the observer?
Posted by Joshua Cohen <jc...@apache.org>.
If you're suggesting just going to the task directory and pulling them out
of the executor logs. Yes, I could ssh into the host the task is running on
and grep the task directory out of the mesos agent logs and then trawl the
logs (or cat task.json), but that's much more effort than going to the
observer's task UI (i.e. it'd take a minute, rather than a few seconds).
I'd also posit that it's much easier for new Aurora operators to come to
grips with the process tree via the UI rather than a JSON blob.
If you're suggesting something else (i.e. new UI to expose these separate
from the Observer), I'm fine with that, that's what I was implying above
would be necessary before I think we could retire the Observer.
A counter question: do people feel that updating/deploying the Observer is
a major obstacle? I know we've got the process well automated, so it's
relatively painless. I'd love to replace the Observer with something
better, but I don't feel like it's a major drag on our productivity as it
exists today to warrant killing it off entirely. My opinion may be colored
by the deploy automation we have in place though!
On Mon, Apr 4, 2016 at 9:32 AM, Bill Farner <wf...@apache.org> wrote:
> >
> > 2) Providing an easy view of a process's command-line
> > 3) Providing a holistic view of the task config
>
>
> Just to check my understanding - these could be trivially handled in
> text/log format, right?
>
> On Mon, Apr 4, 2016 at 9:30 AM, Joshua Cohen <jc...@apache.org> wrote:
>
> > I'm -1 on this until we have an actual replacement for the Observer. I
> > think that the observer provides significant value outside of just
> sandbox
> > browsing:
> >
> > 1) Exporting task-level statistics.
> > 2) Providing an easy view of a process's command-line
> > 3) Providing a holistic view of the task config
> > 4) Real time utilization stats
> >
> > As a cluster operator, I use all of these features on a daily basis
> > (especially when I'm on call) in addition to sandbox browsing, so I don't
> > think that these uses cases are that rare.
> >
> > On Fri, Apr 1, 2016 at 6:55 AM, Steve Niemitz <sn...@apache.org>
> wrote:
> >
> > > The per-process stats have never been very useful to us (since they
> don't
> > > work for docker), however, even being able to see the processes that
> are
> > > running, how many times they've restarted, when they launched, etc is
> > > invaluable.
> > >
> > > I think there would be big pushback from users if they were to lose the
> > > functionality it provided currently (beyond log viewing).
> > >
> > > On Fri, Apr 1, 2016 at 6:58 AM, Erb, Stephan <
> > Stephan.Erb@blue-yonder.com>
> > > wrote:
> > >
> > > > From an operator and Aurora developer perspective, it would be really
> > > > great to get rid of the thermos observer quickly.
> > > >
> > > > However, from a user perspective the usability gap between observer
> and
> > > > plain Mesos sandbox browsing is quite large right now. I agree with
> > > > Benjamin here that it would probably work if we generate html pages
> > ready
> > > > for user consumption.
> > > >
> > > > These are the relevant tickets in our tracker:
> > > > * https://issues.apache.org/jira/browse/AURORA-725
> > > > * https://issues.apache.org/jira/browse/AURORA-777
> > > >
> > > > ________________________________________
> > > > From: benley@gmail.com <be...@gmail.com>
> > > > Sent: Friday, April 1, 2016 02:35
> > > > To: dev@aurora.apache.org
> > > > Subject: Re: Are we ready to remove the observer?
> > > >
> > > > Is there any chance we can keep the per-process cpu and ram
> utilization
> > > > stats? That's one of the coolest things about aurora, imo. The
> > executor
> > > > is already writing those checkpoints inside the mesos sandbox (I
> > think?),
> > > > so perhaps it could also produce the html pages that the observer
> > > currently
> > > > renders?
> > > >
> > > > On Thu, Mar 31, 2016 at 4:33 PM Zhitao Li <zh...@gmail.com>
> > wrote:
> > > >
> > > > > +1.
> > > > >
> > > > > On Thu, Mar 31, 2016 at 4:11 PM, Bill Farner <wf...@apache.org>
> > > wrote:
> > > > >
> > > > > > Assuming that the vast majority of utility provided by the
> observer
> > > is
> > > > > > sandbox/log browsing - can we remove it and link to sandbox
> > browsing
> > > > that
> > > > > > mesos provides?
> > > > > >
> > > > > > The rest of the information could be (or already is) logged in
> the
> > > > > sandbox
> > > > > > for the rare debugging scenarios that call for it.
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Cheers,
> > > > >
> > > > > Zhitao Li
> > > > >
> > > >
> > >
> >
>
Re: Are we ready to remove the observer?
Posted by Bill Farner <wf...@apache.org>.
>
> 2) Providing an easy view of a process's command-line
> 3) Providing a holistic view of the task config
Just to check my understanding - these could be trivially handled in
text/log format, right?
On Mon, Apr 4, 2016 at 9:30 AM, Joshua Cohen <jc...@apache.org> wrote:
> I'm -1 on this until we have an actual replacement for the Observer. I
> think that the observer provides significant value outside of just sandbox
> browsing:
>
> 1) Exporting task-level statistics.
> 2) Providing an easy view of a process's command-line
> 3) Providing a holistic view of the task config
> 4) Real time utilization stats
>
> As a cluster operator, I use all of these features on a daily basis
> (especially when I'm on call) in addition to sandbox browsing, so I don't
> think that these uses cases are that rare.
>
> On Fri, Apr 1, 2016 at 6:55 AM, Steve Niemitz <sn...@apache.org> wrote:
>
> > The per-process stats have never been very useful to us (since they don't
> > work for docker), however, even being able to see the processes that are
> > running, how many times they've restarted, when they launched, etc is
> > invaluable.
> >
> > I think there would be big pushback from users if they were to lose the
> > functionality it provided currently (beyond log viewing).
> >
> > On Fri, Apr 1, 2016 at 6:58 AM, Erb, Stephan <
> Stephan.Erb@blue-yonder.com>
> > wrote:
> >
> > > From an operator and Aurora developer perspective, it would be really
> > > great to get rid of the thermos observer quickly.
> > >
> > > However, from a user perspective the usability gap between observer and
> > > plain Mesos sandbox browsing is quite large right now. I agree with
> > > Benjamin here that it would probably work if we generate html pages
> ready
> > > for user consumption.
> > >
> > > These are the relevant tickets in our tracker:
> > > * https://issues.apache.org/jira/browse/AURORA-725
> > > * https://issues.apache.org/jira/browse/AURORA-777
> > >
> > > ________________________________________
> > > From: benley@gmail.com <be...@gmail.com>
> > > Sent: Friday, April 1, 2016 02:35
> > > To: dev@aurora.apache.org
> > > Subject: Re: Are we ready to remove the observer?
> > >
> > > Is there any chance we can keep the per-process cpu and ram utilization
> > > stats? That's one of the coolest things about aurora, imo. The
> executor
> > > is already writing those checkpoints inside the mesos sandbox (I
> think?),
> > > so perhaps it could also produce the html pages that the observer
> > currently
> > > renders?
> > >
> > > On Thu, Mar 31, 2016 at 4:33 PM Zhitao Li <zh...@gmail.com>
> wrote:
> > >
> > > > +1.
> > > >
> > > > On Thu, Mar 31, 2016 at 4:11 PM, Bill Farner <wf...@apache.org>
> > wrote:
> > > >
> > > > > Assuming that the vast majority of utility provided by the observer
> > is
> > > > > sandbox/log browsing - can we remove it and link to sandbox
> browsing
> > > that
> > > > > mesos provides?
> > > > >
> > > > > The rest of the information could be (or already is) logged in the
> > > > sandbox
> > > > > for the rare debugging scenarios that call for it.
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Cheers,
> > > >
> > > > Zhitao Li
> > > >
> > >
> >
>
Re: Are we ready to remove the observer?
Posted by Joshua Cohen <jc...@apache.org>.
I'm -1 on this until we have an actual replacement for the Observer. I
think that the observer provides significant value outside of just sandbox
browsing:
1) Exporting task-level statistics.
2) Providing an easy view of a process's command-line
3) Providing a holistic view of the task config
4) Real time utilization stats
As a cluster operator, I use all of these features on a daily basis
(especially when I'm on call) in addition to sandbox browsing, so I don't
think that these uses cases are that rare.
On Fri, Apr 1, 2016 at 6:55 AM, Steve Niemitz <sn...@apache.org> wrote:
> The per-process stats have never been very useful to us (since they don't
> work for docker), however, even being able to see the processes that are
> running, how many times they've restarted, when they launched, etc is
> invaluable.
>
> I think there would be big pushback from users if they were to lose the
> functionality it provided currently (beyond log viewing).
>
> On Fri, Apr 1, 2016 at 6:58 AM, Erb, Stephan <St...@blue-yonder.com>
> wrote:
>
> > From an operator and Aurora developer perspective, it would be really
> > great to get rid of the thermos observer quickly.
> >
> > However, from a user perspective the usability gap between observer and
> > plain Mesos sandbox browsing is quite large right now. I agree with
> > Benjamin here that it would probably work if we generate html pages ready
> > for user consumption.
> >
> > These are the relevant tickets in our tracker:
> > * https://issues.apache.org/jira/browse/AURORA-725
> > * https://issues.apache.org/jira/browse/AURORA-777
> >
> > ________________________________________
> > From: benley@gmail.com <be...@gmail.com>
> > Sent: Friday, April 1, 2016 02:35
> > To: dev@aurora.apache.org
> > Subject: Re: Are we ready to remove the observer?
> >
> > Is there any chance we can keep the per-process cpu and ram utilization
> > stats? That's one of the coolest things about aurora, imo. The executor
> > is already writing those checkpoints inside the mesos sandbox (I think?),
> > so perhaps it could also produce the html pages that the observer
> currently
> > renders?
> >
> > On Thu, Mar 31, 2016 at 4:33 PM Zhitao Li <zh...@gmail.com> wrote:
> >
> > > +1.
> > >
> > > On Thu, Mar 31, 2016 at 4:11 PM, Bill Farner <wf...@apache.org>
> wrote:
> > >
> > > > Assuming that the vast majority of utility provided by the observer
> is
> > > > sandbox/log browsing - can we remove it and link to sandbox browsing
> > that
> > > > mesos provides?
> > > >
> > > > The rest of the information could be (or already is) logged in the
> > > sandbox
> > > > for the rare debugging scenarios that call for it.
> > > >
> > >
> > >
> > >
> > > --
> > > Cheers,
> > >
> > > Zhitao Li
> > >
> >
>
Re: Are we ready to remove the observer?
Posted by Steve Niemitz <sn...@apache.org>.
The per-process stats have never been very useful to us (since they don't
work for docker), however, even being able to see the processes that are
running, how many times they've restarted, when they launched, etc is
invaluable.
I think there would be big pushback from users if they were to lose the
functionality it provided currently (beyond log viewing).
On Fri, Apr 1, 2016 at 6:58 AM, Erb, Stephan <St...@blue-yonder.com>
wrote:
> From an operator and Aurora developer perspective, it would be really
> great to get rid of the thermos observer quickly.
>
> However, from a user perspective the usability gap between observer and
> plain Mesos sandbox browsing is quite large right now. I agree with
> Benjamin here that it would probably work if we generate html pages ready
> for user consumption.
>
> These are the relevant tickets in our tracker:
> * https://issues.apache.org/jira/browse/AURORA-725
> * https://issues.apache.org/jira/browse/AURORA-777
>
> ________________________________________
> From: benley@gmail.com <be...@gmail.com>
> Sent: Friday, April 1, 2016 02:35
> To: dev@aurora.apache.org
> Subject: Re: Are we ready to remove the observer?
>
> Is there any chance we can keep the per-process cpu and ram utilization
> stats? That's one of the coolest things about aurora, imo. The executor
> is already writing those checkpoints inside the mesos sandbox (I think?),
> so perhaps it could also produce the html pages that the observer currently
> renders?
>
> On Thu, Mar 31, 2016 at 4:33 PM Zhitao Li <zh...@gmail.com> wrote:
>
> > +1.
> >
> > On Thu, Mar 31, 2016 at 4:11 PM, Bill Farner <wf...@apache.org> wrote:
> >
> > > Assuming that the vast majority of utility provided by the observer is
> > > sandbox/log browsing - can we remove it and link to sandbox browsing
> that
> > > mesos provides?
> > >
> > > The rest of the information could be (or already is) logged in the
> > sandbox
> > > for the rare debugging scenarios that call for it.
> > >
> >
> >
> >
> > --
> > Cheers,
> >
> > Zhitao Li
> >
>
Re: Are we ready to remove the observer?
Posted by "Erb, Stephan" <St...@blue-yonder.com>.
>From an operator and Aurora developer perspective, it would be really great to get rid of the thermos observer quickly.
However, from a user perspective the usability gap between observer and plain Mesos sandbox browsing is quite large right now. I agree with Benjamin here that it would probably work if we generate html pages ready for user consumption.
These are the relevant tickets in our tracker:
* https://issues.apache.org/jira/browse/AURORA-725
* https://issues.apache.org/jira/browse/AURORA-777
________________________________________
From: benley@gmail.com <be...@gmail.com>
Sent: Friday, April 1, 2016 02:35
To: dev@aurora.apache.org
Subject: Re: Are we ready to remove the observer?
Is there any chance we can keep the per-process cpu and ram utilization
stats? That's one of the coolest things about aurora, imo. The executor
is already writing those checkpoints inside the mesos sandbox (I think?),
so perhaps it could also produce the html pages that the observer currently
renders?
On Thu, Mar 31, 2016 at 4:33 PM Zhitao Li <zh...@gmail.com> wrote:
> +1.
>
> On Thu, Mar 31, 2016 at 4:11 PM, Bill Farner <wf...@apache.org> wrote:
>
> > Assuming that the vast majority of utility provided by the observer is
> > sandbox/log browsing - can we remove it and link to sandbox browsing that
> > mesos provides?
> >
> > The rest of the information could be (or already is) logged in the
> sandbox
> > for the rare debugging scenarios that call for it.
> >
>
>
>
> --
> Cheers,
>
> Zhitao Li
>
Re: Are we ready to remove the observer?
Posted by "benley@gmail.com" <be...@gmail.com>.
Is there any chance we can keep the per-process cpu and ram utilization
stats? That's one of the coolest things about aurora, imo. The executor
is already writing those checkpoints inside the mesos sandbox (I think?),
so perhaps it could also produce the html pages that the observer currently
renders?
On Thu, Mar 31, 2016 at 4:33 PM Zhitao Li <zh...@gmail.com> wrote:
> +1.
>
> On Thu, Mar 31, 2016 at 4:11 PM, Bill Farner <wf...@apache.org> wrote:
>
> > Assuming that the vast majority of utility provided by the observer is
> > sandbox/log browsing - can we remove it and link to sandbox browsing that
> > mesos provides?
> >
> > The rest of the information could be (or already is) logged in the
> sandbox
> > for the rare debugging scenarios that call for it.
> >
>
>
>
> --
> Cheers,
>
> Zhitao Li
>
Re: Are we ready to remove the observer?
Posted by Zhitao Li <zh...@gmail.com>.
+1.
On Thu, Mar 31, 2016 at 4:11 PM, Bill Farner <wf...@apache.org> wrote:
> Assuming that the vast majority of utility provided by the observer is
> sandbox/log browsing - can we remove it and link to sandbox browsing that
> mesos provides?
>
> The rest of the information could be (or already is) logged in the sandbox
> for the rare debugging scenarios that call for it.
>
--
Cheers,
Zhitao Li