You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@aurora.apache.org by Bill Farner <wf...@apache.org> on 2016/01/27 00:03:51 UTC
[PROPOSAL] Revisit task ID format
Context: a task ID is a unique identifier for a task. Aurora and Mesos
both require this uniqueness. Within mesos, frameworks are required to
craft their own task IDs as they see fit.
Our task ID format is currently [1]
TIMESTAMP-ROLE-ENV-JOBNAME-INSTANCE-UUID
for an example:
1453847837931-vagrant-test-http_example_docker-1-a23f55e2-151f-409e-9cea-76ec79482533
In addition to guaranteed uniqueness, this format has the benefit of being
somewhat human-friendly. Within logs, it is somewhat possible to partially
recognize a task based solely on the text ID.
*I would like to propose that we remove the TIMESTAMP- prefix from the task
ID.* It was originally included so that task IDs would be temporally
sortable for scheduling prioritization. At present, tasks are not sorted
using the ID.
While proposing the above, i think it's also prudent to take the
opportunity to consider a complete overhaul of the ID contents. *An
alternative approach would be to only use the UUID.* This has the benefit
of decoupling arbitrary user input from the various ways task IDs are used
(as an example - mesos uses them in file names, limiting length and allowed
characters). Task IDs also become fixed width, which offers a (very)
marginal memory reduction over the status quo, and makes console line
wrapping more consistent when perusing logs. Additionally, it eschews the
potential problem of users parsing task IDs and coupling to its format.
Any thoughts on this?
[1]
https://github.com/apache/aurora/blob/master/src/main/java/org/apache/aurora/scheduler/TaskIdGenerator.java
Re: [PROPOSAL] Revisit task ID format
Posted by "Erb, Stephan" <St...@blue-yonder.com>.
Additional thought: I guess if we use the unmangled jobkey as a task name [1] we could totally go for the pure UUID task ids.
[1] https://github.com/apache/aurora/blob/749f83502f059ae6d2b229cf76c1ed44ccf3d255/src/main/java/org/apache/aurora/scheduler/configuration/executor/ExecutorModule.java#L135
________________________________________
From: Erb, Stephan <St...@blue-yonder.com>
Sent: Wednesday, January 27, 2016 12:17 AM
To: dev@aurora.apache.org
Subject: Re: [PROPOSAL] Revisit task ID format
+1 for dropping the timestamp
However, I am not sure regarding the mangled jobkey. It tends to make it easier to correlate Mesos tasks to Aurora jobs when skimming log files, viewing the Mesos-UI or even when using the Thermos [1]. I guess the traceability of all of those usecases could be improved, but that would probably additional work.
[1] https://github.com/apache/aurora/blob/master/docs/images/runningtask.png
________________________________________
From: Bill Farner <wf...@apache.org>
Sent: Wednesday, January 27, 2016 12:03 AM
To: dev@aurora.apache.org
Subject: [PROPOSAL] Revisit task ID format
Context: a task ID is a unique identifier for a task. Aurora and Mesos
both require this uniqueness. Within mesos, frameworks are required to
craft their own task IDs as they see fit.
Our task ID format is currently [1]
TIMESTAMP-ROLE-ENV-JOBNAME-INSTANCE-UUID
for an example:
1453847837931-vagrant-test-http_example_docker-1-a23f55e2-151f-409e-9cea-76ec79482533
In addition to guaranteed uniqueness, this format has the benefit of being
somewhat human-friendly. Within logs, it is somewhat possible to partially
recognize a task based solely on the text ID.
*I would like to propose that we remove the TIMESTAMP- prefix from the task
ID.* It was originally included so that task IDs would be temporally
sortable for scheduling prioritization. At present, tasks are not sorted
using the ID.
While proposing the above, i think it's also prudent to take the
opportunity to consider a complete overhaul of the ID contents. *An
alternative approach would be to only use the UUID.* This has the benefit
of decoupling arbitrary user input from the various ways task IDs are used
(as an example - mesos uses them in file names, limiting length and allowed
characters). Task IDs also become fixed width, which offers a (very)
marginal memory reduction over the status quo, and makes console line
wrapping more consistent when perusing logs. Additionally, it eschews the
potential problem of users parsing task IDs and coupling to its format.
Any thoughts on this?
[1]
https://github.com/apache/aurora/blob/master/src/main/java/org/apache/aurora/scheduler/TaskIdGenerator.java
Re: [PROPOSAL] Revisit task ID format
Posted by Bill Farner <wf...@apache.org>.
It has now been submitted.
On Fri, Feb 19, 2016 at 9:32 AM, Chris Lambert <ch...@gmail.com>
wrote:
> 'Morning,
>
> What's the eta for this patch, Bill? Planning to commit soon?
>
> Chris
>
>
> On Wed, Jan 27, 2016 at 8:53 PM, Bill Farner <wf...@apache.org> wrote:
>
> > Thanks for the input, folks. Patch is up here:
> > https://reviews.apache.org/r/42896/
> >
> > On Wed, Jan 27, 2016 at 8:09 PM, Maxim Khutornenko <ma...@apache.org>
> > wrote:
> >
> > > You're right, ignore me. I guess I was mourning the loss of
> > StorageBackfill
> > > too hard :) Obviously, we don't have to force this change over existing
> > > tasks and let them die out naturally. Some user scraping tools may be
> > > broken due to this but we always advised against taking a dependency on
> > our
> > > task ID format.
> > >
> > > +1 to the proposal.
> > >
> > > On Wed, Jan 27, 2016 at 5:46 PM, Bill Farner <wf...@apache.org>
> wrote:
> > >
> > > > I don't believe backwards compatibility is an issue here. This would
> > be
> > > an
> > > > alteration to generation of new task IDs. AFAIK we don't do anything
> > > that
> > > > requires comprehension or synthesis of previously-generated task IDs.
> > > >
> > > > On Wed, Jan 27, 2016 at 5:39 PM, Maxim Khutornenko <maxim@apache.org
> >
> > > > wrote:
> > > >
> > > > > What's the cluster upgrade story going to look like? Since task IDs
> > are
> > > > > used as unique identifiers for Mesos, I expect this change would
> > > require
> > > > > rebooting the entire cluster to the new format? I am not really
> sure
> > > how
> > > > > this can be done in a graceful manner without sending the entire
> > > cluster
> > > > to
> > > > > LOST on first reconciliation run.
> > > > >
> > > > > Unless we have answers to the above, I am -1 to this proposal. The
> > > > benefits
> > > > > don't seem significant enough to offset the pain of migrating to
> the
> > > new
> > > > > format.
> > > > >
> > > > > On Tue, Jan 26, 2016 at 6:25 PM, Mauricio Garavaglia <
> > > > > mauriciogaravaglia@gmail.com> wrote:
> > > > >
> > > > > > +1 to dropping the timestamp.
> > > > > >
> > > > > > Agree that having the jobkey at hand has been helpful for
> > debugging.
> > > > > >
> > > > > > On Tue, Jan 26, 2016 at 10:39 PM, Zameer Manji <
> zmanji@apache.org>
> > > > > wrote:
> > > > > >
> > > > > > > +1 to removing the timestamp.
> > > > > > >
> > > > > > > The timestamp has not provided me with any benefit as an
> > operator.
> > > > The
> > > > > > > mangled jobkey and UUID have been very useful in grepping logs
> > and
> > > > > > > diagnosing failing jobs.
> > > > > > >
> > > > > > > On Tue, Jan 26, 2016 at 3:49 PM, Zhitao Li <
> > zhitaoli.cs@gmail.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > > +1 for dropping the time and keeping the mangled jobkey.
> Unless
> > > we
> > > > > are
> > > > > > > sure
> > > > > > > > that all internal logging of Mesos master and agent contains
> an
> > > > > > > identifier
> > > > > > > > with user some user generated data, changing it to UUID will
> > make
> > > > > adhoc
> > > > > > > > debugging through Mesos logging harder.
> > > > > > > >
> > > > > > > > On Tue, Jan 26, 2016 at 3:17 PM, Erb, Stephan <
> > > > > > > Stephan.Erb@blue-yonder.com
> > > > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > +1 for dropping the timestamp
> > > > > > > > >
> > > > > > > > > However, I am not sure regarding the mangled jobkey. It
> tends
> > > to
> > > > > make
> > > > > > > it
> > > > > > > > > easier to correlate Mesos tasks to Aurora jobs when
> skimming
> > > log
> > > > > > files,
> > > > > > > > > viewing the Mesos-UI or even when using the Thermos [1]. I
> > > guess
> > > > > the
> > > > > > > > > traceability of all of those usecases could be improved,
> but
> > > that
> > > > > > would
> > > > > > > > > probably additional work.
> > > > > > > > >
> > > > > > > > > [1]
> > > > > > > > >
> > > > > > >
> > > > >
> > >
> https://github.com/apache/aurora/blob/master/docs/images/runningtask.png
> > > > > > > > > ________________________________________
> > > > > > > > > From: Bill Farner <wf...@apache.org>
> > > > > > > > > Sent: Wednesday, January 27, 2016 12:03 AM
> > > > > > > > > To: dev@aurora.apache.org
> > > > > > > > > Subject: [PROPOSAL] Revisit task ID format
> > > > > > > > >
> > > > > > > > > Context: a task ID is a unique identifier for a task.
> Aurora
> > > and
> > > > > > Mesos
> > > > > > > > > both require this uniqueness. Within mesos, frameworks are
> > > > > required
> > > > > > to
> > > > > > > > > craft their own task IDs as they see fit.
> > > > > > > > >
> > > > > > > > > Our task ID format is currently [1]
> > > > > > > > >
> > > > > > > > > TIMESTAMP-ROLE-ENV-JOBNAME-INSTANCE-UUID
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > for an example:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> 1453847837931-vagrant-test-http_example_docker-1-a23f55e2-151f-409e-9cea-76ec79482533
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > In addition to guaranteed uniqueness, this format has the
> > > benefit
> > > > > of
> > > > > > > > being
> > > > > > > > > somewhat human-friendly. Within logs, it is somewhat
> > possible
> > > to
> > > > > > > > partially
> > > > > > > > > recognize a task based solely on the text ID.
> > > > > > > > >
> > > > > > > > > *I would like to propose that we remove the TIMESTAMP-
> prefix
> > > > from
> > > > > > the
> > > > > > > > task
> > > > > > > > > ID.* It was originally included so that task IDs would be
> > > > > temporally
> > > > > > > > > sortable for scheduling prioritization. At present, tasks
> > are
> > > > not
> > > > > > > sorted
> > > > > > > > > using the ID.
> > > > > > > > >
> > > > > > > > > While proposing the above, i think it's also prudent to
> take
> > > the
> > > > > > > > > opportunity to consider a complete overhaul of the ID
> > contents.
> > > > > *An
> > > > > > > > > alternative approach would be to only use the UUID.* This
> > has
> > > > the
> > > > > > > > benefit
> > > > > > > > > of decoupling arbitrary user input from the various ways
> task
> > > IDs
> > > > > are
> > > > > > > > used
> > > > > > > > > (as an example - mesos uses them in file names, limiting
> > length
> > > > and
> > > > > > > > allowed
> > > > > > > > > characters). Task IDs also become fixed width, which
> offers
> > a
> > > > > (very)
> > > > > > > > > marginal memory reduction over the status quo, and makes
> > > console
> > > > > line
> > > > > > > > > wrapping more consistent when perusing logs. Additionally,
> > it
> > > > > > eschews
> > > > > > > > the
> > > > > > > > > potential problem of users parsing task IDs and coupling to
> > its
> > > > > > format.
> > > > > > > > >
> > > > > > > > > Any thoughts on this?
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > [1]
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/aurora/blob/master/src/main/java/org/apache/aurora/scheduler/TaskIdGenerator.java
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Cheers,
> > > > > > > >
> > > > > > > > Zhitao Li
> > > > > > > >
> > > > > > > > --
> > > > > > > > Zameer Manji
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [PROPOSAL] Revisit task ID format
Posted by Chris Lambert <ch...@gmail.com>.
'Morning,
What's the eta for this patch, Bill? Planning to commit soon?
Chris
On Wed, Jan 27, 2016 at 8:53 PM, Bill Farner <wf...@apache.org> wrote:
> Thanks for the input, folks. Patch is up here:
> https://reviews.apache.org/r/42896/
>
> On Wed, Jan 27, 2016 at 8:09 PM, Maxim Khutornenko <ma...@apache.org>
> wrote:
>
> > You're right, ignore me. I guess I was mourning the loss of
> StorageBackfill
> > too hard :) Obviously, we don't have to force this change over existing
> > tasks and let them die out naturally. Some user scraping tools may be
> > broken due to this but we always advised against taking a dependency on
> our
> > task ID format.
> >
> > +1 to the proposal.
> >
> > On Wed, Jan 27, 2016 at 5:46 PM, Bill Farner <wf...@apache.org> wrote:
> >
> > > I don't believe backwards compatibility is an issue here. This would
> be
> > an
> > > alteration to generation of new task IDs. AFAIK we don't do anything
> > that
> > > requires comprehension or synthesis of previously-generated task IDs.
> > >
> > > On Wed, Jan 27, 2016 at 5:39 PM, Maxim Khutornenko <ma...@apache.org>
> > > wrote:
> > >
> > > > What's the cluster upgrade story going to look like? Since task IDs
> are
> > > > used as unique identifiers for Mesos, I expect this change would
> > require
> > > > rebooting the entire cluster to the new format? I am not really sure
> > how
> > > > this can be done in a graceful manner without sending the entire
> > cluster
> > > to
> > > > LOST on first reconciliation run.
> > > >
> > > > Unless we have answers to the above, I am -1 to this proposal. The
> > > benefits
> > > > don't seem significant enough to offset the pain of migrating to the
> > new
> > > > format.
> > > >
> > > > On Tue, Jan 26, 2016 at 6:25 PM, Mauricio Garavaglia <
> > > > mauriciogaravaglia@gmail.com> wrote:
> > > >
> > > > > +1 to dropping the timestamp.
> > > > >
> > > > > Agree that having the jobkey at hand has been helpful for
> debugging.
> > > > >
> > > > > On Tue, Jan 26, 2016 at 10:39 PM, Zameer Manji <zm...@apache.org>
> > > > wrote:
> > > > >
> > > > > > +1 to removing the timestamp.
> > > > > >
> > > > > > The timestamp has not provided me with any benefit as an
> operator.
> > > The
> > > > > > mangled jobkey and UUID have been very useful in grepping logs
> and
> > > > > > diagnosing failing jobs.
> > > > > >
> > > > > > On Tue, Jan 26, 2016 at 3:49 PM, Zhitao Li <
> zhitaoli.cs@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > > +1 for dropping the time and keeping the mangled jobkey. Unless
> > we
> > > > are
> > > > > > sure
> > > > > > > that all internal logging of Mesos master and agent contains an
> > > > > > identifier
> > > > > > > with user some user generated data, changing it to UUID will
> make
> > > > adhoc
> > > > > > > debugging through Mesos logging harder.
> > > > > > >
> > > > > > > On Tue, Jan 26, 2016 at 3:17 PM, Erb, Stephan <
> > > > > > Stephan.Erb@blue-yonder.com
> > > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > +1 for dropping the timestamp
> > > > > > > >
> > > > > > > > However, I am not sure regarding the mangled jobkey. It tends
> > to
> > > > make
> > > > > > it
> > > > > > > > easier to correlate Mesos tasks to Aurora jobs when skimming
> > log
> > > > > files,
> > > > > > > > viewing the Mesos-UI or even when using the Thermos [1]. I
> > guess
> > > > the
> > > > > > > > traceability of all of those usecases could be improved, but
> > that
> > > > > would
> > > > > > > > probably additional work.
> > > > > > > >
> > > > > > > > [1]
> > > > > > > >
> > > > > >
> > > >
> > https://github.com/apache/aurora/blob/master/docs/images/runningtask.png
> > > > > > > > ________________________________________
> > > > > > > > From: Bill Farner <wf...@apache.org>
> > > > > > > > Sent: Wednesday, January 27, 2016 12:03 AM
> > > > > > > > To: dev@aurora.apache.org
> > > > > > > > Subject: [PROPOSAL] Revisit task ID format
> > > > > > > >
> > > > > > > > Context: a task ID is a unique identifier for a task. Aurora
> > and
> > > > > Mesos
> > > > > > > > both require this uniqueness. Within mesos, frameworks are
> > > > required
> > > > > to
> > > > > > > > craft their own task IDs as they see fit.
> > > > > > > >
> > > > > > > > Our task ID format is currently [1]
> > > > > > > >
> > > > > > > > TIMESTAMP-ROLE-ENV-JOBNAME-INSTANCE-UUID
> > > > > > > >
> > > > > > > >
> > > > > > > > for an example:
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> 1453847837931-vagrant-test-http_example_docker-1-a23f55e2-151f-409e-9cea-76ec79482533
> > > > > > > >
> > > > > > > >
> > > > > > > > In addition to guaranteed uniqueness, this format has the
> > benefit
> > > > of
> > > > > > > being
> > > > > > > > somewhat human-friendly. Within logs, it is somewhat
> possible
> > to
> > > > > > > partially
> > > > > > > > recognize a task based solely on the text ID.
> > > > > > > >
> > > > > > > > *I would like to propose that we remove the TIMESTAMP- prefix
> > > from
> > > > > the
> > > > > > > task
> > > > > > > > ID.* It was originally included so that task IDs would be
> > > > temporally
> > > > > > > > sortable for scheduling prioritization. At present, tasks
> are
> > > not
> > > > > > sorted
> > > > > > > > using the ID.
> > > > > > > >
> > > > > > > > While proposing the above, i think it's also prudent to take
> > the
> > > > > > > > opportunity to consider a complete overhaul of the ID
> contents.
> > > > *An
> > > > > > > > alternative approach would be to only use the UUID.* This
> has
> > > the
> > > > > > > benefit
> > > > > > > > of decoupling arbitrary user input from the various ways task
> > IDs
> > > > are
> > > > > > > used
> > > > > > > > (as an example - mesos uses them in file names, limiting
> length
> > > and
> > > > > > > allowed
> > > > > > > > characters). Task IDs also become fixed width, which offers
> a
> > > > (very)
> > > > > > > > marginal memory reduction over the status quo, and makes
> > console
> > > > line
> > > > > > > > wrapping more consistent when perusing logs. Additionally,
> it
> > > > > eschews
> > > > > > > the
> > > > > > > > potential problem of users parsing task IDs and coupling to
> its
> > > > > format.
> > > > > > > >
> > > > > > > > Any thoughts on this?
> > > > > > > >
> > > > > > > >
> > > > > > > > [1]
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/aurora/blob/master/src/main/java/org/apache/aurora/scheduler/TaskIdGenerator.java
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Cheers,
> > > > > > >
> > > > > > > Zhitao Li
> > > > > > >
> > > > > > > --
> > > > > > > Zameer Manji
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [PROPOSAL] Revisit task ID format
Posted by Bill Farner <wf...@apache.org>.
Thanks for the input, folks. Patch is up here:
https://reviews.apache.org/r/42896/
On Wed, Jan 27, 2016 at 8:09 PM, Maxim Khutornenko <ma...@apache.org> wrote:
> You're right, ignore me. I guess I was mourning the loss of StorageBackfill
> too hard :) Obviously, we don't have to force this change over existing
> tasks and let them die out naturally. Some user scraping tools may be
> broken due to this but we always advised against taking a dependency on our
> task ID format.
>
> +1 to the proposal.
>
> On Wed, Jan 27, 2016 at 5:46 PM, Bill Farner <wf...@apache.org> wrote:
>
> > I don't believe backwards compatibility is an issue here. This would be
> an
> > alteration to generation of new task IDs. AFAIK we don't do anything
> that
> > requires comprehension or synthesis of previously-generated task IDs.
> >
> > On Wed, Jan 27, 2016 at 5:39 PM, Maxim Khutornenko <ma...@apache.org>
> > wrote:
> >
> > > What's the cluster upgrade story going to look like? Since task IDs are
> > > used as unique identifiers for Mesos, I expect this change would
> require
> > > rebooting the entire cluster to the new format? I am not really sure
> how
> > > this can be done in a graceful manner without sending the entire
> cluster
> > to
> > > LOST on first reconciliation run.
> > >
> > > Unless we have answers to the above, I am -1 to this proposal. The
> > benefits
> > > don't seem significant enough to offset the pain of migrating to the
> new
> > > format.
> > >
> > > On Tue, Jan 26, 2016 at 6:25 PM, Mauricio Garavaglia <
> > > mauriciogaravaglia@gmail.com> wrote:
> > >
> > > > +1 to dropping the timestamp.
> > > >
> > > > Agree that having the jobkey at hand has been helpful for debugging.
> > > >
> > > > On Tue, Jan 26, 2016 at 10:39 PM, Zameer Manji <zm...@apache.org>
> > > wrote:
> > > >
> > > > > +1 to removing the timestamp.
> > > > >
> > > > > The timestamp has not provided me with any benefit as an operator.
> > The
> > > > > mangled jobkey and UUID have been very useful in grepping logs and
> > > > > diagnosing failing jobs.
> > > > >
> > > > > On Tue, Jan 26, 2016 at 3:49 PM, Zhitao Li <zh...@gmail.com>
> > > > wrote:
> > > > >
> > > > > > +1 for dropping the time and keeping the mangled jobkey. Unless
> we
> > > are
> > > > > sure
> > > > > > that all internal logging of Mesos master and agent contains an
> > > > > identifier
> > > > > > with user some user generated data, changing it to UUID will make
> > > adhoc
> > > > > > debugging through Mesos logging harder.
> > > > > >
> > > > > > On Tue, Jan 26, 2016 at 3:17 PM, Erb, Stephan <
> > > > > Stephan.Erb@blue-yonder.com
> > > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > +1 for dropping the timestamp
> > > > > > >
> > > > > > > However, I am not sure regarding the mangled jobkey. It tends
> to
> > > make
> > > > > it
> > > > > > > easier to correlate Mesos tasks to Aurora jobs when skimming
> log
> > > > files,
> > > > > > > viewing the Mesos-UI or even when using the Thermos [1]. I
> guess
> > > the
> > > > > > > traceability of all of those usecases could be improved, but
> that
> > > > would
> > > > > > > probably additional work.
> > > > > > >
> > > > > > > [1]
> > > > > > >
> > > > >
> > >
> https://github.com/apache/aurora/blob/master/docs/images/runningtask.png
> > > > > > > ________________________________________
> > > > > > > From: Bill Farner <wf...@apache.org>
> > > > > > > Sent: Wednesday, January 27, 2016 12:03 AM
> > > > > > > To: dev@aurora.apache.org
> > > > > > > Subject: [PROPOSAL] Revisit task ID format
> > > > > > >
> > > > > > > Context: a task ID is a unique identifier for a task. Aurora
> and
> > > > Mesos
> > > > > > > both require this uniqueness. Within mesos, frameworks are
> > > required
> > > > to
> > > > > > > craft their own task IDs as they see fit.
> > > > > > >
> > > > > > > Our task ID format is currently [1]
> > > > > > >
> > > > > > > TIMESTAMP-ROLE-ENV-JOBNAME-INSTANCE-UUID
> > > > > > >
> > > > > > >
> > > > > > > for an example:
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> 1453847837931-vagrant-test-http_example_docker-1-a23f55e2-151f-409e-9cea-76ec79482533
> > > > > > >
> > > > > > >
> > > > > > > In addition to guaranteed uniqueness, this format has the
> benefit
> > > of
> > > > > > being
> > > > > > > somewhat human-friendly. Within logs, it is somewhat possible
> to
> > > > > > partially
> > > > > > > recognize a task based solely on the text ID.
> > > > > > >
> > > > > > > *I would like to propose that we remove the TIMESTAMP- prefix
> > from
> > > > the
> > > > > > task
> > > > > > > ID.* It was originally included so that task IDs would be
> > > temporally
> > > > > > > sortable for scheduling prioritization. At present, tasks are
> > not
> > > > > sorted
> > > > > > > using the ID.
> > > > > > >
> > > > > > > While proposing the above, i think it's also prudent to take
> the
> > > > > > > opportunity to consider a complete overhaul of the ID contents.
> > > *An
> > > > > > > alternative approach would be to only use the UUID.* This has
> > the
> > > > > > benefit
> > > > > > > of decoupling arbitrary user input from the various ways task
> IDs
> > > are
> > > > > > used
> > > > > > > (as an example - mesos uses them in file names, limiting length
> > and
> > > > > > allowed
> > > > > > > characters). Task IDs also become fixed width, which offers a
> > > (very)
> > > > > > > marginal memory reduction over the status quo, and makes
> console
> > > line
> > > > > > > wrapping more consistent when perusing logs. Additionally, it
> > > > eschews
> > > > > > the
> > > > > > > potential problem of users parsing task IDs and coupling to its
> > > > format.
> > > > > > >
> > > > > > > Any thoughts on this?
> > > > > > >
> > > > > > >
> > > > > > > [1]
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/aurora/blob/master/src/main/java/org/apache/aurora/scheduler/TaskIdGenerator.java
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Cheers,
> > > > > >
> > > > > > Zhitao Li
> > > > > >
> > > > > > --
> > > > > > Zameer Manji
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [PROPOSAL] Revisit task ID format
Posted by Maxim Khutornenko <ma...@apache.org>.
You're right, ignore me. I guess I was mourning the loss of StorageBackfill
too hard :) Obviously, we don't have to force this change over existing
tasks and let them die out naturally. Some user scraping tools may be
broken due to this but we always advised against taking a dependency on our
task ID format.
+1 to the proposal.
On Wed, Jan 27, 2016 at 5:46 PM, Bill Farner <wf...@apache.org> wrote:
> I don't believe backwards compatibility is an issue here. This would be an
> alteration to generation of new task IDs. AFAIK we don't do anything that
> requires comprehension or synthesis of previously-generated task IDs.
>
> On Wed, Jan 27, 2016 at 5:39 PM, Maxim Khutornenko <ma...@apache.org>
> wrote:
>
> > What's the cluster upgrade story going to look like? Since task IDs are
> > used as unique identifiers for Mesos, I expect this change would require
> > rebooting the entire cluster to the new format? I am not really sure how
> > this can be done in a graceful manner without sending the entire cluster
> to
> > LOST on first reconciliation run.
> >
> > Unless we have answers to the above, I am -1 to this proposal. The
> benefits
> > don't seem significant enough to offset the pain of migrating to the new
> > format.
> >
> > On Tue, Jan 26, 2016 at 6:25 PM, Mauricio Garavaglia <
> > mauriciogaravaglia@gmail.com> wrote:
> >
> > > +1 to dropping the timestamp.
> > >
> > > Agree that having the jobkey at hand has been helpful for debugging.
> > >
> > > On Tue, Jan 26, 2016 at 10:39 PM, Zameer Manji <zm...@apache.org>
> > wrote:
> > >
> > > > +1 to removing the timestamp.
> > > >
> > > > The timestamp has not provided me with any benefit as an operator.
> The
> > > > mangled jobkey and UUID have been very useful in grepping logs and
> > > > diagnosing failing jobs.
> > > >
> > > > On Tue, Jan 26, 2016 at 3:49 PM, Zhitao Li <zh...@gmail.com>
> > > wrote:
> > > >
> > > > > +1 for dropping the time and keeping the mangled jobkey. Unless we
> > are
> > > > sure
> > > > > that all internal logging of Mesos master and agent contains an
> > > > identifier
> > > > > with user some user generated data, changing it to UUID will make
> > adhoc
> > > > > debugging through Mesos logging harder.
> > > > >
> > > > > On Tue, Jan 26, 2016 at 3:17 PM, Erb, Stephan <
> > > > Stephan.Erb@blue-yonder.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > +1 for dropping the timestamp
> > > > > >
> > > > > > However, I am not sure regarding the mangled jobkey. It tends to
> > make
> > > > it
> > > > > > easier to correlate Mesos tasks to Aurora jobs when skimming log
> > > files,
> > > > > > viewing the Mesos-UI or even when using the Thermos [1]. I guess
> > the
> > > > > > traceability of all of those usecases could be improved, but that
> > > would
> > > > > > probably additional work.
> > > > > >
> > > > > > [1]
> > > > > >
> > > >
> > https://github.com/apache/aurora/blob/master/docs/images/runningtask.png
> > > > > > ________________________________________
> > > > > > From: Bill Farner <wf...@apache.org>
> > > > > > Sent: Wednesday, January 27, 2016 12:03 AM
> > > > > > To: dev@aurora.apache.org
> > > > > > Subject: [PROPOSAL] Revisit task ID format
> > > > > >
> > > > > > Context: a task ID is a unique identifier for a task. Aurora and
> > > Mesos
> > > > > > both require this uniqueness. Within mesos, frameworks are
> > required
> > > to
> > > > > > craft their own task IDs as they see fit.
> > > > > >
> > > > > > Our task ID format is currently [1]
> > > > > >
> > > > > > TIMESTAMP-ROLE-ENV-JOBNAME-INSTANCE-UUID
> > > > > >
> > > > > >
> > > > > > for an example:
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> 1453847837931-vagrant-test-http_example_docker-1-a23f55e2-151f-409e-9cea-76ec79482533
> > > > > >
> > > > > >
> > > > > > In addition to guaranteed uniqueness, this format has the benefit
> > of
> > > > > being
> > > > > > somewhat human-friendly. Within logs, it is somewhat possible to
> > > > > partially
> > > > > > recognize a task based solely on the text ID.
> > > > > >
> > > > > > *I would like to propose that we remove the TIMESTAMP- prefix
> from
> > > the
> > > > > task
> > > > > > ID.* It was originally included so that task IDs would be
> > temporally
> > > > > > sortable for scheduling prioritization. At present, tasks are
> not
> > > > sorted
> > > > > > using the ID.
> > > > > >
> > > > > > While proposing the above, i think it's also prudent to take the
> > > > > > opportunity to consider a complete overhaul of the ID contents.
> > *An
> > > > > > alternative approach would be to only use the UUID.* This has
> the
> > > > > benefit
> > > > > > of decoupling arbitrary user input from the various ways task IDs
> > are
> > > > > used
> > > > > > (as an example - mesos uses them in file names, limiting length
> and
> > > > > allowed
> > > > > > characters). Task IDs also become fixed width, which offers a
> > (very)
> > > > > > marginal memory reduction over the status quo, and makes console
> > line
> > > > > > wrapping more consistent when perusing logs. Additionally, it
> > > eschews
> > > > > the
> > > > > > potential problem of users parsing task IDs and coupling to its
> > > format.
> > > > > >
> > > > > > Any thoughts on this?
> > > > > >
> > > > > >
> > > > > > [1]
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/aurora/blob/master/src/main/java/org/apache/aurora/scheduler/TaskIdGenerator.java
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Cheers,
> > > > >
> > > > > Zhitao Li
> > > > >
> > > > > --
> > > > > Zameer Manji
> > > > >
> > > > >
> > > >
> > >
> >
>
Re: [PROPOSAL] Revisit task ID format
Posted by Bill Farner <wf...@apache.org>.
I don't believe backwards compatibility is an issue here. This would be an
alteration to generation of new task IDs. AFAIK we don't do anything that
requires comprehension or synthesis of previously-generated task IDs.
On Wed, Jan 27, 2016 at 5:39 PM, Maxim Khutornenko <ma...@apache.org> wrote:
> What's the cluster upgrade story going to look like? Since task IDs are
> used as unique identifiers for Mesos, I expect this change would require
> rebooting the entire cluster to the new format? I am not really sure how
> this can be done in a graceful manner without sending the entire cluster to
> LOST on first reconciliation run.
>
> Unless we have answers to the above, I am -1 to this proposal. The benefits
> don't seem significant enough to offset the pain of migrating to the new
> format.
>
> On Tue, Jan 26, 2016 at 6:25 PM, Mauricio Garavaglia <
> mauriciogaravaglia@gmail.com> wrote:
>
> > +1 to dropping the timestamp.
> >
> > Agree that having the jobkey at hand has been helpful for debugging.
> >
> > On Tue, Jan 26, 2016 at 10:39 PM, Zameer Manji <zm...@apache.org>
> wrote:
> >
> > > +1 to removing the timestamp.
> > >
> > > The timestamp has not provided me with any benefit as an operator. The
> > > mangled jobkey and UUID have been very useful in grepping logs and
> > > diagnosing failing jobs.
> > >
> > > On Tue, Jan 26, 2016 at 3:49 PM, Zhitao Li <zh...@gmail.com>
> > wrote:
> > >
> > > > +1 for dropping the time and keeping the mangled jobkey. Unless we
> are
> > > sure
> > > > that all internal logging of Mesos master and agent contains an
> > > identifier
> > > > with user some user generated data, changing it to UUID will make
> adhoc
> > > > debugging through Mesos logging harder.
> > > >
> > > > On Tue, Jan 26, 2016 at 3:17 PM, Erb, Stephan <
> > > Stephan.Erb@blue-yonder.com
> > > > >
> > > > wrote:
> > > >
> > > > > +1 for dropping the timestamp
> > > > >
> > > > > However, I am not sure regarding the mangled jobkey. It tends to
> make
> > > it
> > > > > easier to correlate Mesos tasks to Aurora jobs when skimming log
> > files,
> > > > > viewing the Mesos-UI or even when using the Thermos [1]. I guess
> the
> > > > > traceability of all of those usecases could be improved, but that
> > would
> > > > > probably additional work.
> > > > >
> > > > > [1]
> > > > >
> > >
> https://github.com/apache/aurora/blob/master/docs/images/runningtask.png
> > > > > ________________________________________
> > > > > From: Bill Farner <wf...@apache.org>
> > > > > Sent: Wednesday, January 27, 2016 12:03 AM
> > > > > To: dev@aurora.apache.org
> > > > > Subject: [PROPOSAL] Revisit task ID format
> > > > >
> > > > > Context: a task ID is a unique identifier for a task. Aurora and
> > Mesos
> > > > > both require this uniqueness. Within mesos, frameworks are
> required
> > to
> > > > > craft their own task IDs as they see fit.
> > > > >
> > > > > Our task ID format is currently [1]
> > > > >
> > > > > TIMESTAMP-ROLE-ENV-JOBNAME-INSTANCE-UUID
> > > > >
> > > > >
> > > > > for an example:
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> 1453847837931-vagrant-test-http_example_docker-1-a23f55e2-151f-409e-9cea-76ec79482533
> > > > >
> > > > >
> > > > > In addition to guaranteed uniqueness, this format has the benefit
> of
> > > > being
> > > > > somewhat human-friendly. Within logs, it is somewhat possible to
> > > > partially
> > > > > recognize a task based solely on the text ID.
> > > > >
> > > > > *I would like to propose that we remove the TIMESTAMP- prefix from
> > the
> > > > task
> > > > > ID.* It was originally included so that task IDs would be
> temporally
> > > > > sortable for scheduling prioritization. At present, tasks are not
> > > sorted
> > > > > using the ID.
> > > > >
> > > > > While proposing the above, i think it's also prudent to take the
> > > > > opportunity to consider a complete overhaul of the ID contents.
> *An
> > > > > alternative approach would be to only use the UUID.* This has the
> > > > benefit
> > > > > of decoupling arbitrary user input from the various ways task IDs
> are
> > > > used
> > > > > (as an example - mesos uses them in file names, limiting length and
> > > > allowed
> > > > > characters). Task IDs also become fixed width, which offers a
> (very)
> > > > > marginal memory reduction over the status quo, and makes console
> line
> > > > > wrapping more consistent when perusing logs. Additionally, it
> > eschews
> > > > the
> > > > > potential problem of users parsing task IDs and coupling to its
> > format.
> > > > >
> > > > > Any thoughts on this?
> > > > >
> > > > >
> > > > > [1]
> > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/aurora/blob/master/src/main/java/org/apache/aurora/scheduler/TaskIdGenerator.java
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Cheers,
> > > >
> > > > Zhitao Li
> > > >
> > > > --
> > > > Zameer Manji
> > > >
> > > >
> > >
> >
>
Re: [PROPOSAL] Revisit task ID format
Posted by Maxim Khutornenko <ma...@apache.org>.
What's the cluster upgrade story going to look like? Since task IDs are
used as unique identifiers for Mesos, I expect this change would require
rebooting the entire cluster to the new format? I am not really sure how
this can be done in a graceful manner without sending the entire cluster to
LOST on first reconciliation run.
Unless we have answers to the above, I am -1 to this proposal. The benefits
don't seem significant enough to offset the pain of migrating to the new
format.
On Tue, Jan 26, 2016 at 6:25 PM, Mauricio Garavaglia <
mauriciogaravaglia@gmail.com> wrote:
> +1 to dropping the timestamp.
>
> Agree that having the jobkey at hand has been helpful for debugging.
>
> On Tue, Jan 26, 2016 at 10:39 PM, Zameer Manji <zm...@apache.org> wrote:
>
> > +1 to removing the timestamp.
> >
> > The timestamp has not provided me with any benefit as an operator. The
> > mangled jobkey and UUID have been very useful in grepping logs and
> > diagnosing failing jobs.
> >
> > On Tue, Jan 26, 2016 at 3:49 PM, Zhitao Li <zh...@gmail.com>
> wrote:
> >
> > > +1 for dropping the time and keeping the mangled jobkey. Unless we are
> > sure
> > > that all internal logging of Mesos master and agent contains an
> > identifier
> > > with user some user generated data, changing it to UUID will make adhoc
> > > debugging through Mesos logging harder.
> > >
> > > On Tue, Jan 26, 2016 at 3:17 PM, Erb, Stephan <
> > Stephan.Erb@blue-yonder.com
> > > >
> > > wrote:
> > >
> > > > +1 for dropping the timestamp
> > > >
> > > > However, I am not sure regarding the mangled jobkey. It tends to make
> > it
> > > > easier to correlate Mesos tasks to Aurora jobs when skimming log
> files,
> > > > viewing the Mesos-UI or even when using the Thermos [1]. I guess the
> > > > traceability of all of those usecases could be improved, but that
> would
> > > > probably additional work.
> > > >
> > > > [1]
> > > >
> > https://github.com/apache/aurora/blob/master/docs/images/runningtask.png
> > > > ________________________________________
> > > > From: Bill Farner <wf...@apache.org>
> > > > Sent: Wednesday, January 27, 2016 12:03 AM
> > > > To: dev@aurora.apache.org
> > > > Subject: [PROPOSAL] Revisit task ID format
> > > >
> > > > Context: a task ID is a unique identifier for a task. Aurora and
> Mesos
> > > > both require this uniqueness. Within mesos, frameworks are required
> to
> > > > craft their own task IDs as they see fit.
> > > >
> > > > Our task ID format is currently [1]
> > > >
> > > > TIMESTAMP-ROLE-ENV-JOBNAME-INSTANCE-UUID
> > > >
> > > >
> > > > for an example:
> > > >
> > > >
> > > >
> > >
> >
> 1453847837931-vagrant-test-http_example_docker-1-a23f55e2-151f-409e-9cea-76ec79482533
> > > >
> > > >
> > > > In addition to guaranteed uniqueness, this format has the benefit of
> > > being
> > > > somewhat human-friendly. Within logs, it is somewhat possible to
> > > partially
> > > > recognize a task based solely on the text ID.
> > > >
> > > > *I would like to propose that we remove the TIMESTAMP- prefix from
> the
> > > task
> > > > ID.* It was originally included so that task IDs would be temporally
> > > > sortable for scheduling prioritization. At present, tasks are not
> > sorted
> > > > using the ID.
> > > >
> > > > While proposing the above, i think it's also prudent to take the
> > > > opportunity to consider a complete overhaul of the ID contents. *An
> > > > alternative approach would be to only use the UUID.* This has the
> > > benefit
> > > > of decoupling arbitrary user input from the various ways task IDs are
> > > used
> > > > (as an example - mesos uses them in file names, limiting length and
> > > allowed
> > > > characters). Task IDs also become fixed width, which offers a (very)
> > > > marginal memory reduction over the status quo, and makes console line
> > > > wrapping more consistent when perusing logs. Additionally, it
> eschews
> > > the
> > > > potential problem of users parsing task IDs and coupling to its
> format.
> > > >
> > > > Any thoughts on this?
> > > >
> > > >
> > > > [1]
> > > >
> > > >
> > >
> >
> https://github.com/apache/aurora/blob/master/src/main/java/org/apache/aurora/scheduler/TaskIdGenerator.java
> > > >
> > >
> > >
> > >
> > > --
> > > Cheers,
> > >
> > > Zhitao Li
> > >
> > > --
> > > Zameer Manji
> > >
> > >
> >
>
Re: [PROPOSAL] Revisit task ID format
Posted by Mauricio Garavaglia <ma...@gmail.com>.
+1 to dropping the timestamp.
Agree that having the jobkey at hand has been helpful for debugging.
On Tue, Jan 26, 2016 at 10:39 PM, Zameer Manji <zm...@apache.org> wrote:
> +1 to removing the timestamp.
>
> The timestamp has not provided me with any benefit as an operator. The
> mangled jobkey and UUID have been very useful in grepping logs and
> diagnosing failing jobs.
>
> On Tue, Jan 26, 2016 at 3:49 PM, Zhitao Li <zh...@gmail.com> wrote:
>
> > +1 for dropping the time and keeping the mangled jobkey. Unless we are
> sure
> > that all internal logging of Mesos master and agent contains an
> identifier
> > with user some user generated data, changing it to UUID will make adhoc
> > debugging through Mesos logging harder.
> >
> > On Tue, Jan 26, 2016 at 3:17 PM, Erb, Stephan <
> Stephan.Erb@blue-yonder.com
> > >
> > wrote:
> >
> > > +1 for dropping the timestamp
> > >
> > > However, I am not sure regarding the mangled jobkey. It tends to make
> it
> > > easier to correlate Mesos tasks to Aurora jobs when skimming log files,
> > > viewing the Mesos-UI or even when using the Thermos [1]. I guess the
> > > traceability of all of those usecases could be improved, but that would
> > > probably additional work.
> > >
> > > [1]
> > >
> https://github.com/apache/aurora/blob/master/docs/images/runningtask.png
> > > ________________________________________
> > > From: Bill Farner <wf...@apache.org>
> > > Sent: Wednesday, January 27, 2016 12:03 AM
> > > To: dev@aurora.apache.org
> > > Subject: [PROPOSAL] Revisit task ID format
> > >
> > > Context: a task ID is a unique identifier for a task. Aurora and Mesos
> > > both require this uniqueness. Within mesos, frameworks are required to
> > > craft their own task IDs as they see fit.
> > >
> > > Our task ID format is currently [1]
> > >
> > > TIMESTAMP-ROLE-ENV-JOBNAME-INSTANCE-UUID
> > >
> > >
> > > for an example:
> > >
> > >
> > >
> >
> 1453847837931-vagrant-test-http_example_docker-1-a23f55e2-151f-409e-9cea-76ec79482533
> > >
> > >
> > > In addition to guaranteed uniqueness, this format has the benefit of
> > being
> > > somewhat human-friendly. Within logs, it is somewhat possible to
> > partially
> > > recognize a task based solely on the text ID.
> > >
> > > *I would like to propose that we remove the TIMESTAMP- prefix from the
> > task
> > > ID.* It was originally included so that task IDs would be temporally
> > > sortable for scheduling prioritization. At present, tasks are not
> sorted
> > > using the ID.
> > >
> > > While proposing the above, i think it's also prudent to take the
> > > opportunity to consider a complete overhaul of the ID contents. *An
> > > alternative approach would be to only use the UUID.* This has the
> > benefit
> > > of decoupling arbitrary user input from the various ways task IDs are
> > used
> > > (as an example - mesos uses them in file names, limiting length and
> > allowed
> > > characters). Task IDs also become fixed width, which offers a (very)
> > > marginal memory reduction over the status quo, and makes console line
> > > wrapping more consistent when perusing logs. Additionally, it eschews
> > the
> > > potential problem of users parsing task IDs and coupling to its format.
> > >
> > > Any thoughts on this?
> > >
> > >
> > > [1]
> > >
> > >
> >
> https://github.com/apache/aurora/blob/master/src/main/java/org/apache/aurora/scheduler/TaskIdGenerator.java
> > >
> >
> >
> >
> > --
> > Cheers,
> >
> > Zhitao Li
> >
> > --
> > Zameer Manji
> >
> >
>
Re: [PROPOSAL] Revisit task ID format
Posted by Zameer Manji <zm...@apache.org>.
+1 to removing the timestamp.
The timestamp has not provided me with any benefit as an operator. The
mangled jobkey and UUID have been very useful in grepping logs and
diagnosing failing jobs.
On Tue, Jan 26, 2016 at 3:49 PM, Zhitao Li <zh...@gmail.com> wrote:
> +1 for dropping the time and keeping the mangled jobkey. Unless we are sure
> that all internal logging of Mesos master and agent contains an identifier
> with user some user generated data, changing it to UUID will make adhoc
> debugging through Mesos logging harder.
>
> On Tue, Jan 26, 2016 at 3:17 PM, Erb, Stephan <Stephan.Erb@blue-yonder.com
> >
> wrote:
>
> > +1 for dropping the timestamp
> >
> > However, I am not sure regarding the mangled jobkey. It tends to make it
> > easier to correlate Mesos tasks to Aurora jobs when skimming log files,
> > viewing the Mesos-UI or even when using the Thermos [1]. I guess the
> > traceability of all of those usecases could be improved, but that would
> > probably additional work.
> >
> > [1]
> > https://github.com/apache/aurora/blob/master/docs/images/runningtask.png
> > ________________________________________
> > From: Bill Farner <wf...@apache.org>
> > Sent: Wednesday, January 27, 2016 12:03 AM
> > To: dev@aurora.apache.org
> > Subject: [PROPOSAL] Revisit task ID format
> >
> > Context: a task ID is a unique identifier for a task. Aurora and Mesos
> > both require this uniqueness. Within mesos, frameworks are required to
> > craft their own task IDs as they see fit.
> >
> > Our task ID format is currently [1]
> >
> > TIMESTAMP-ROLE-ENV-JOBNAME-INSTANCE-UUID
> >
> >
> > for an example:
> >
> >
> >
> 1453847837931-vagrant-test-http_example_docker-1-a23f55e2-151f-409e-9cea-76ec79482533
> >
> >
> > In addition to guaranteed uniqueness, this format has the benefit of
> being
> > somewhat human-friendly. Within logs, it is somewhat possible to
> partially
> > recognize a task based solely on the text ID.
> >
> > *I would like to propose that we remove the TIMESTAMP- prefix from the
> task
> > ID.* It was originally included so that task IDs would be temporally
> > sortable for scheduling prioritization. At present, tasks are not sorted
> > using the ID.
> >
> > While proposing the above, i think it's also prudent to take the
> > opportunity to consider a complete overhaul of the ID contents. *An
> > alternative approach would be to only use the UUID.* This has the
> benefit
> > of decoupling arbitrary user input from the various ways task IDs are
> used
> > (as an example - mesos uses them in file names, limiting length and
> allowed
> > characters). Task IDs also become fixed width, which offers a (very)
> > marginal memory reduction over the status quo, and makes console line
> > wrapping more consistent when perusing logs. Additionally, it eschews
> the
> > potential problem of users parsing task IDs and coupling to its format.
> >
> > Any thoughts on this?
> >
> >
> > [1]
> >
> >
> https://github.com/apache/aurora/blob/master/src/main/java/org/apache/aurora/scheduler/TaskIdGenerator.java
> >
>
>
>
> --
> Cheers,
>
> Zhitao Li
>
> --
> Zameer Manji
>
>
Re: [PROPOSAL] Revisit task ID format
Posted by Zhitao Li <zh...@gmail.com>.
+1 for dropping the time and keeping the mangled jobkey. Unless we are sure
that all internal logging of Mesos master and agent contains an identifier
with user some user generated data, changing it to UUID will make adhoc
debugging through Mesos logging harder.
On Tue, Jan 26, 2016 at 3:17 PM, Erb, Stephan <St...@blue-yonder.com>
wrote:
> +1 for dropping the timestamp
>
> However, I am not sure regarding the mangled jobkey. It tends to make it
> easier to correlate Mesos tasks to Aurora jobs when skimming log files,
> viewing the Mesos-UI or even when using the Thermos [1]. I guess the
> traceability of all of those usecases could be improved, but that would
> probably additional work.
>
> [1]
> https://github.com/apache/aurora/blob/master/docs/images/runningtask.png
> ________________________________________
> From: Bill Farner <wf...@apache.org>
> Sent: Wednesday, January 27, 2016 12:03 AM
> To: dev@aurora.apache.org
> Subject: [PROPOSAL] Revisit task ID format
>
> Context: a task ID is a unique identifier for a task. Aurora and Mesos
> both require this uniqueness. Within mesos, frameworks are required to
> craft their own task IDs as they see fit.
>
> Our task ID format is currently [1]
>
> TIMESTAMP-ROLE-ENV-JOBNAME-INSTANCE-UUID
>
>
> for an example:
>
>
> 1453847837931-vagrant-test-http_example_docker-1-a23f55e2-151f-409e-9cea-76ec79482533
>
>
> In addition to guaranteed uniqueness, this format has the benefit of being
> somewhat human-friendly. Within logs, it is somewhat possible to partially
> recognize a task based solely on the text ID.
>
> *I would like to propose that we remove the TIMESTAMP- prefix from the task
> ID.* It was originally included so that task IDs would be temporally
> sortable for scheduling prioritization. At present, tasks are not sorted
> using the ID.
>
> While proposing the above, i think it's also prudent to take the
> opportunity to consider a complete overhaul of the ID contents. *An
> alternative approach would be to only use the UUID.* This has the benefit
> of decoupling arbitrary user input from the various ways task IDs are used
> (as an example - mesos uses them in file names, limiting length and allowed
> characters). Task IDs also become fixed width, which offers a (very)
> marginal memory reduction over the status quo, and makes console line
> wrapping more consistent when perusing logs. Additionally, it eschews the
> potential problem of users parsing task IDs and coupling to its format.
>
> Any thoughts on this?
>
>
> [1]
>
> https://github.com/apache/aurora/blob/master/src/main/java/org/apache/aurora/scheduler/TaskIdGenerator.java
>
--
Cheers,
Zhitao Li
Re: [PROPOSAL] Revisit task ID format
Posted by "Erb, Stephan" <St...@blue-yonder.com>.
+1 for dropping the timestamp
However, I am not sure regarding the mangled jobkey. It tends to make it easier to correlate Mesos tasks to Aurora jobs when skimming log files, viewing the Mesos-UI or even when using the Thermos [1]. I guess the traceability of all of those usecases could be improved, but that would probably additional work.
[1] https://github.com/apache/aurora/blob/master/docs/images/runningtask.png
________________________________________
From: Bill Farner <wf...@apache.org>
Sent: Wednesday, January 27, 2016 12:03 AM
To: dev@aurora.apache.org
Subject: [PROPOSAL] Revisit task ID format
Context: a task ID is a unique identifier for a task. Aurora and Mesos
both require this uniqueness. Within mesos, frameworks are required to
craft their own task IDs as they see fit.
Our task ID format is currently [1]
TIMESTAMP-ROLE-ENV-JOBNAME-INSTANCE-UUID
for an example:
1453847837931-vagrant-test-http_example_docker-1-a23f55e2-151f-409e-9cea-76ec79482533
In addition to guaranteed uniqueness, this format has the benefit of being
somewhat human-friendly. Within logs, it is somewhat possible to partially
recognize a task based solely on the text ID.
*I would like to propose that we remove the TIMESTAMP- prefix from the task
ID.* It was originally included so that task IDs would be temporally
sortable for scheduling prioritization. At present, tasks are not sorted
using the ID.
While proposing the above, i think it's also prudent to take the
opportunity to consider a complete overhaul of the ID contents. *An
alternative approach would be to only use the UUID.* This has the benefit
of decoupling arbitrary user input from the various ways task IDs are used
(as an example - mesos uses them in file names, limiting length and allowed
characters). Task IDs also become fixed width, which offers a (very)
marginal memory reduction over the status quo, and makes console line
wrapping more consistent when perusing logs. Additionally, it eschews the
potential problem of users parsing task IDs and coupling to its format.
Any thoughts on this?
[1]
https://github.com/apache/aurora/blob/master/src/main/java/org/apache/aurora/scheduler/TaskIdGenerator.java