You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@myriad.apache.org by Paul Read <pd...@gmail.com> on 2015/03/27 17:42:23 UTC
myriad issue #10, data transport ?
<https://github.com/mesos/myriad> Mapping NM NodeId to Mesos TaskId
Having studied the problem I think I'm ready to implement a solution. There
are two ways, that I can see, of transporting the NM id to the scheduler.
The first would be to coop the TaskStatus "data" field which does not
appear to be used either in myriad or mesos or a better way would be to add
either a "sub_task" field or a "child" field to the TaskStatus structure.
Since this is a mesos baseline issue and not a myriad issue I would like
the thoughts of those better positioned to make this decision.
Thanks
Paul
Re: myriad issue #10, data transport ?
Posted by Adam Bordelon <ad...@mesosphere.io>.
I also remembered that you can use the `TaskStatus.message` string field
instead of the 'data' byte array. It's only used by Mesos for
failure/lost/error cases, so you can put whatever you want there for an
arbitrary TASK_RUNNING status update, and even send multiple TASK_RUNNINGs
from a custom executor, if you want to, for example, announce state changes
to a new phase for your task (starting, recovering, formatted, initialized,
bootstrapped, running, unhealthy, etc.). So you can put
json/protobuf/whatever string data you want directly in the message field.
On Sun, Mar 29, 2015 at 5:45 AM, Paul Read <pd...@gmail.com> wrote:
> Great. I was wondering which forum to use. Also I was considering using
> JSON as the data payload so it would be extensible.
>
> On Fri, Mar 27, 2015 at 1:55 PM, Adam Bordelon <ad...@mesosphere.io> wrote:
>
> > And feel free to have these discussions on the github issue itself:
> > https://github.com/mesos/myriad/issues/10
> >
> > On Fri, Mar 27, 2015 at 10:54 AM, Adam Bordelon <ad...@mesosphere.io>
> > wrote:
> >
> > > Go ahead and use TaskStatus.data; that's what it's there for, although
> at
> > > some point you may need to make the data itself a serialized object if
> we
> > > need to communicate multiple fields.
> > > Adding new fields to TaskStatus itself would require a change in Mesos
> > > proper, which could take a while. However, it could be useful to add a
> > > general key-value "labels" map to TaskStatus to help frameworks that do
> > > want to pass info back.
> > >
> > > On Fri, Mar 27, 2015 at 9:42 AM, Paul Read <pd...@gmail.com>
> wrote:
> > >
> > >> <https://github.com/mesos/myriad> Mapping NM NodeId to Mesos TaskId
> > >>
> > >> Having studied the problem I think I'm ready to implement a solution.
> > >> There
> > >> are two ways, that I can see, of transporting the NM id to the
> > scheduler.
> > >> The first would be to coop the TaskStatus "data" field which does not
> > >> appear to be used either in myriad or mesos or a better way would be
> to
> > >> add
> > >> either a "sub_task" field or a "child" field to the TaskStatus
> > structure.
> > >>
> > >> Since this is a mesos baseline issue and not a myriad issue I would
> like
> > >> the thoughts of those better positioned to make this decision.
> > >>
> > >> Thanks
> > >>
> > >> Paul
> > >>
> > >
> > >
> >
>
Re: myriad issue #10, data transport ?
Posted by Paul Read <pd...@gmail.com>.
Great. I was wondering which forum to use. Also I was considering using
JSON as the data payload so it would be extensible.
On Fri, Mar 27, 2015 at 1:55 PM, Adam Bordelon <ad...@mesosphere.io> wrote:
> And feel free to have these discussions on the github issue itself:
> https://github.com/mesos/myriad/issues/10
>
> On Fri, Mar 27, 2015 at 10:54 AM, Adam Bordelon <ad...@mesosphere.io>
> wrote:
>
> > Go ahead and use TaskStatus.data; that's what it's there for, although at
> > some point you may need to make the data itself a serialized object if we
> > need to communicate multiple fields.
> > Adding new fields to TaskStatus itself would require a change in Mesos
> > proper, which could take a while. However, it could be useful to add a
> > general key-value "labels" map to TaskStatus to help frameworks that do
> > want to pass info back.
> >
> > On Fri, Mar 27, 2015 at 9:42 AM, Paul Read <pd...@gmail.com> wrote:
> >
> >> <https://github.com/mesos/myriad> Mapping NM NodeId to Mesos TaskId
> >>
> >> Having studied the problem I think I'm ready to implement a solution.
> >> There
> >> are two ways, that I can see, of transporting the NM id to the
> scheduler.
> >> The first would be to coop the TaskStatus "data" field which does not
> >> appear to be used either in myriad or mesos or a better way would be to
> >> add
> >> either a "sub_task" field or a "child" field to the TaskStatus
> structure.
> >>
> >> Since this is a mesos baseline issue and not a myriad issue I would like
> >> the thoughts of those better positioned to make this decision.
> >>
> >> Thanks
> >>
> >> Paul
> >>
> >
> >
>
Re: myriad issue #10, data transport ?
Posted by Adam Bordelon <ad...@mesosphere.io>.
And feel free to have these discussions on the github issue itself:
https://github.com/mesos/myriad/issues/10
On Fri, Mar 27, 2015 at 10:54 AM, Adam Bordelon <ad...@mesosphere.io> wrote:
> Go ahead and use TaskStatus.data; that's what it's there for, although at
> some point you may need to make the data itself a serialized object if we
> need to communicate multiple fields.
> Adding new fields to TaskStatus itself would require a change in Mesos
> proper, which could take a while. However, it could be useful to add a
> general key-value "labels" map to TaskStatus to help frameworks that do
> want to pass info back.
>
> On Fri, Mar 27, 2015 at 9:42 AM, Paul Read <pd...@gmail.com> wrote:
>
>> <https://github.com/mesos/myriad> Mapping NM NodeId to Mesos TaskId
>>
>> Having studied the problem I think I'm ready to implement a solution.
>> There
>> are two ways, that I can see, of transporting the NM id to the scheduler.
>> The first would be to coop the TaskStatus "data" field which does not
>> appear to be used either in myriad or mesos or a better way would be to
>> add
>> either a "sub_task" field or a "child" field to the TaskStatus structure.
>>
>> Since this is a mesos baseline issue and not a myriad issue I would like
>> the thoughts of those better positioned to make this decision.
>>
>> Thanks
>>
>> Paul
>>
>
>
Re: myriad issue #10, data transport ?
Posted by Adam Bordelon <ad...@mesosphere.io>.
Go ahead and use TaskStatus.data; that's what it's there for, although at
some point you may need to make the data itself a serialized object if we
need to communicate multiple fields.
Adding new fields to TaskStatus itself would require a change in Mesos
proper, which could take a while. However, it could be useful to add a
general key-value "labels" map to TaskStatus to help frameworks that do
want to pass info back.
On Fri, Mar 27, 2015 at 9:42 AM, Paul Read <pd...@gmail.com> wrote:
> <https://github.com/mesos/myriad> Mapping NM NodeId to Mesos TaskId
>
> Having studied the problem I think I'm ready to implement a solution. There
> are two ways, that I can see, of transporting the NM id to the scheduler.
> The first would be to coop the TaskStatus "data" field which does not
> appear to be used either in myriad or mesos or a better way would be to add
> either a "sub_task" field or a "child" field to the TaskStatus structure.
>
> Since this is a mesos baseline issue and not a myriad issue I would like
> the thoughts of those better positioned to make this decision.
>
> Thanks
>
> Paul
>