You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by James DeFelice <ja...@gmail.com> on 2015/03/15 19:45:03 UTC
executor source not set
Is there a reason that the native executor driver bindings don't check
for/enforce a TaskStatus.Source of SOURCE_EXECUTOR?
https://github.com/apache/mesos/blob/master/src/exec/exec.cpp#L490
Or is that supposed to be handled somewhere else?
-James
Re: executor source not set
Posted by James DeFelice <ja...@gmail.com>.
filed https://issues.apache.org/jira/browse/MESOS-2499 to track this
On Sun, Mar 15, 2015 at 11:20 PM, Vinod Kone <vi...@apache.org> wrote:
> Ah crap. You are right! Please file a JIRA. We should have a test for this.
>
> On Sun, Mar 15, 2015 at 8:05 PM, James DeFelice <ja...@gmail.com>
> wrote:
>
> > Vinod, I suspect that the "status.set_source" is meaningless since it's
> > likely updating an object that's a copy of what's actually inside the
> > StatusUpdate that's forward()ed from the slave. In the master code, I see
> > references to things like "message.mutable_update().mutable_status()"
> that
> > seem obtain a pointer to a status object to which changes are made. I
> > suspect the same should be performed in the slave code.
> >
> > If this sounds right to you, I'll file a JIRA for it.
> >
> >
> > On Sun, Mar 15, 2015 at 10:13 PM, James DeFelice <
> james.defelice@gmail.com
> > >
> > wrote:
> >
> > > Yes, generated by the executor.
> > >
> > > I do see a source for status updates generated by reconciliation
> > > processes. For example:
> > >
> > >
> >
> reason="REASON_RECONCILIATION",source="SOURCE_MASTER",state="TASK_RUNNING"
> > >
> > > But for status updates generated by my executor, the scheduler sees
> this:
> > >
> > > reason="none",source="none",state="TASK_RUNNING" (I print none when
> the
> > > Source/Reason are nil)
> > >
> > >
> > > On Sun, Mar 15, 2015 at 10:06 PM, Vinod Kone <vi...@apache.org>
> > wrote:
> > >
> > >> Are these updates generated by the executor? Note that updates
> > *generated*
> > >> by slave or master will have a different source. Are you not seeing
> any
> > >> source at all in the updates?
> > >>
> > >> On Sun, Mar 15, 2015 at 6:59 PM, James DeFelice <
> > james.defelice@gmail.com
> > >> >
> > >> wrote:
> > >>
> > >> > Thanks, I found the same. For some reason I'm not seeing a Source
> set
> > in
> > >> > the updates that the scheduler is receiving. The updates are being
> > >> > generated via the mesos-go bindings. The bindings shouldn't matter
> > since
> > >> > the slave is supposed to fill in the blank. Not sure what's going
> on,
> > >> needs
> > >> > more investigation. FWIW I'm observing this on 0.21.0.
> > >> >
> > >> > On Sun, Mar 15, 2015 at 9:55 PM, Vinod Kone <vi...@apache.org>
> > >> wrote:
> > >> >
> > >> > > It is enforced by the slave to be future proof (a future where
> there
> > >> will
> > >> > > be no executor driver).
> > >> > >
> > >> > >
> > https://github.com/apache/mesos/blob/master/src/slave/slave.cpp#L2491
> > >> > >
> > >> > > On Sun, Mar 15, 2015 at 11:45 AM, James DeFelice <
> > >> > james.defelice@gmail.com
> > >> > > >
> > >> > > wrote:
> > >> > >
> > >> > > > Is there a reason that the native executor driver bindings don't
> > >> check
> > >> > > > for/enforce a TaskStatus.Source of SOURCE_EXECUTOR?
> > >> > > >
> > >> > > >
> > https://github.com/apache/mesos/blob/master/src/exec/exec.cpp#L490
> > >> > > >
> > >> > > > Or is that supposed to be handled somewhere else?
> > >> > > >
> > >> > > > -James
> > >> > > >
> > >> > >
> > >> >
> > >> >
> > >> >
> > >> > --
> > >> > James DeFelice
> > >> > 585.241.9488 (voice)
> > >> > 650.649.6071 (fax)
> > >> >
> > >>
> > >
> > >
> > >
> > > --
> > > James DeFelice
> > > 585.241.9488 (voice)
> > > 650.649.6071 (fax)
> > >
> >
> >
> >
> > --
> > James DeFelice
> > 585.241.9488 (voice)
> > 650.649.6071 (fax)
> >
>
--
James DeFelice
585.241.9488 (voice)
650.649.6071 (fax)
Re: executor source not set
Posted by Vinod Kone <vi...@apache.org>.
Ah crap. You are right! Please file a JIRA. We should have a test for this.
On Sun, Mar 15, 2015 at 8:05 PM, James DeFelice <ja...@gmail.com>
wrote:
> Vinod, I suspect that the "status.set_source" is meaningless since it's
> likely updating an object that's a copy of what's actually inside the
> StatusUpdate that's forward()ed from the slave. In the master code, I see
> references to things like "message.mutable_update().mutable_status()" that
> seem obtain a pointer to a status object to which changes are made. I
> suspect the same should be performed in the slave code.
>
> If this sounds right to you, I'll file a JIRA for it.
>
>
> On Sun, Mar 15, 2015 at 10:13 PM, James DeFelice <james.defelice@gmail.com
> >
> wrote:
>
> > Yes, generated by the executor.
> >
> > I do see a source for status updates generated by reconciliation
> > processes. For example:
> >
> >
> reason="REASON_RECONCILIATION",source="SOURCE_MASTER",state="TASK_RUNNING"
> >
> > But for status updates generated by my executor, the scheduler sees this:
> >
> > reason="none",source="none",state="TASK_RUNNING" (I print none when the
> > Source/Reason are nil)
> >
> >
> > On Sun, Mar 15, 2015 at 10:06 PM, Vinod Kone <vi...@apache.org>
> wrote:
> >
> >> Are these updates generated by the executor? Note that updates
> *generated*
> >> by slave or master will have a different source. Are you not seeing any
> >> source at all in the updates?
> >>
> >> On Sun, Mar 15, 2015 at 6:59 PM, James DeFelice <
> james.defelice@gmail.com
> >> >
> >> wrote:
> >>
> >> > Thanks, I found the same. For some reason I'm not seeing a Source set
> in
> >> > the updates that the scheduler is receiving. The updates are being
> >> > generated via the mesos-go bindings. The bindings shouldn't matter
> since
> >> > the slave is supposed to fill in the blank. Not sure what's going on,
> >> needs
> >> > more investigation. FWIW I'm observing this on 0.21.0.
> >> >
> >> > On Sun, Mar 15, 2015 at 9:55 PM, Vinod Kone <vi...@apache.org>
> >> wrote:
> >> >
> >> > > It is enforced by the slave to be future proof (a future where there
> >> will
> >> > > be no executor driver).
> >> > >
> >> > >
> https://github.com/apache/mesos/blob/master/src/slave/slave.cpp#L2491
> >> > >
> >> > > On Sun, Mar 15, 2015 at 11:45 AM, James DeFelice <
> >> > james.defelice@gmail.com
> >> > > >
> >> > > wrote:
> >> > >
> >> > > > Is there a reason that the native executor driver bindings don't
> >> check
> >> > > > for/enforce a TaskStatus.Source of SOURCE_EXECUTOR?
> >> > > >
> >> > > >
> https://github.com/apache/mesos/blob/master/src/exec/exec.cpp#L490
> >> > > >
> >> > > > Or is that supposed to be handled somewhere else?
> >> > > >
> >> > > > -James
> >> > > >
> >> > >
> >> >
> >> >
> >> >
> >> > --
> >> > James DeFelice
> >> > 585.241.9488 (voice)
> >> > 650.649.6071 (fax)
> >> >
> >>
> >
> >
> >
> > --
> > James DeFelice
> > 585.241.9488 (voice)
> > 650.649.6071 (fax)
> >
>
>
>
> --
> James DeFelice
> 585.241.9488 (voice)
> 650.649.6071 (fax)
>
Re: executor source not set
Posted by James DeFelice <ja...@gmail.com>.
Vinod, I suspect that the "status.set_source" is meaningless since it's
likely updating an object that's a copy of what's actually inside the
StatusUpdate that's forward()ed from the slave. In the master code, I see
references to things like "message.mutable_update().mutable_status()" that
seem obtain a pointer to a status object to which changes are made. I
suspect the same should be performed in the slave code.
If this sounds right to you, I'll file a JIRA for it.
On Sun, Mar 15, 2015 at 10:13 PM, James DeFelice <ja...@gmail.com>
wrote:
> Yes, generated by the executor.
>
> I do see a source for status updates generated by reconciliation
> processes. For example:
>
> reason="REASON_RECONCILIATION",source="SOURCE_MASTER",state="TASK_RUNNING"
>
> But for status updates generated by my executor, the scheduler sees this:
>
> reason="none",source="none",state="TASK_RUNNING" (I print none when the
> Source/Reason are nil)
>
>
> On Sun, Mar 15, 2015 at 10:06 PM, Vinod Kone <vi...@apache.org> wrote:
>
>> Are these updates generated by the executor? Note that updates *generated*
>> by slave or master will have a different source. Are you not seeing any
>> source at all in the updates?
>>
>> On Sun, Mar 15, 2015 at 6:59 PM, James DeFelice <james.defelice@gmail.com
>> >
>> wrote:
>>
>> > Thanks, I found the same. For some reason I'm not seeing a Source set in
>> > the updates that the scheduler is receiving. The updates are being
>> > generated via the mesos-go bindings. The bindings shouldn't matter since
>> > the slave is supposed to fill in the blank. Not sure what's going on,
>> needs
>> > more investigation. FWIW I'm observing this on 0.21.0.
>> >
>> > On Sun, Mar 15, 2015 at 9:55 PM, Vinod Kone <vi...@apache.org>
>> wrote:
>> >
>> > > It is enforced by the slave to be future proof (a future where there
>> will
>> > > be no executor driver).
>> > >
>> > > https://github.com/apache/mesos/blob/master/src/slave/slave.cpp#L2491
>> > >
>> > > On Sun, Mar 15, 2015 at 11:45 AM, James DeFelice <
>> > james.defelice@gmail.com
>> > > >
>> > > wrote:
>> > >
>> > > > Is there a reason that the native executor driver bindings don't
>> check
>> > > > for/enforce a TaskStatus.Source of SOURCE_EXECUTOR?
>> > > >
>> > > > https://github.com/apache/mesos/blob/master/src/exec/exec.cpp#L490
>> > > >
>> > > > Or is that supposed to be handled somewhere else?
>> > > >
>> > > > -James
>> > > >
>> > >
>> >
>> >
>> >
>> > --
>> > James DeFelice
>> > 585.241.9488 (voice)
>> > 650.649.6071 (fax)
>> >
>>
>
>
>
> --
> James DeFelice
> 585.241.9488 (voice)
> 650.649.6071 (fax)
>
--
James DeFelice
585.241.9488 (voice)
650.649.6071 (fax)
Re: executor source not set
Posted by James DeFelice <ja...@gmail.com>.
Yes, generated by the executor.
I do see a source for status updates generated by reconciliation processes.
For example:
reason="REASON_RECONCILIATION",source="SOURCE_MASTER",state="TASK_RUNNING"
But for status updates generated by my executor, the scheduler sees this:
reason="none",source="none",state="TASK_RUNNING" (I print none when the
Source/Reason are nil)
On Sun, Mar 15, 2015 at 10:06 PM, Vinod Kone <vi...@apache.org> wrote:
> Are these updates generated by the executor? Note that updates *generated*
> by slave or master will have a different source. Are you not seeing any
> source at all in the updates?
>
> On Sun, Mar 15, 2015 at 6:59 PM, James DeFelice <ja...@gmail.com>
> wrote:
>
> > Thanks, I found the same. For some reason I'm not seeing a Source set in
> > the updates that the scheduler is receiving. The updates are being
> > generated via the mesos-go bindings. The bindings shouldn't matter since
> > the slave is supposed to fill in the blank. Not sure what's going on,
> needs
> > more investigation. FWIW I'm observing this on 0.21.0.
> >
> > On Sun, Mar 15, 2015 at 9:55 PM, Vinod Kone <vi...@apache.org>
> wrote:
> >
> > > It is enforced by the slave to be future proof (a future where there
> will
> > > be no executor driver).
> > >
> > > https://github.com/apache/mesos/blob/master/src/slave/slave.cpp#L2491
> > >
> > > On Sun, Mar 15, 2015 at 11:45 AM, James DeFelice <
> > james.defelice@gmail.com
> > > >
> > > wrote:
> > >
> > > > Is there a reason that the native executor driver bindings don't
> check
> > > > for/enforce a TaskStatus.Source of SOURCE_EXECUTOR?
> > > >
> > > > https://github.com/apache/mesos/blob/master/src/exec/exec.cpp#L490
> > > >
> > > > Or is that supposed to be handled somewhere else?
> > > >
> > > > -James
> > > >
> > >
> >
> >
> >
> > --
> > James DeFelice
> > 585.241.9488 (voice)
> > 650.649.6071 (fax)
> >
>
--
James DeFelice
585.241.9488 (voice)
650.649.6071 (fax)
Re: executor source not set
Posted by Vinod Kone <vi...@apache.org>.
Are these updates generated by the executor? Note that updates *generated*
by slave or master will have a different source. Are you not seeing any
source at all in the updates?
On Sun, Mar 15, 2015 at 6:59 PM, James DeFelice <ja...@gmail.com>
wrote:
> Thanks, I found the same. For some reason I'm not seeing a Source set in
> the updates that the scheduler is receiving. The updates are being
> generated via the mesos-go bindings. The bindings shouldn't matter since
> the slave is supposed to fill in the blank. Not sure what's going on, needs
> more investigation. FWIW I'm observing this on 0.21.0.
>
> On Sun, Mar 15, 2015 at 9:55 PM, Vinod Kone <vi...@apache.org> wrote:
>
> > It is enforced by the slave to be future proof (a future where there will
> > be no executor driver).
> >
> > https://github.com/apache/mesos/blob/master/src/slave/slave.cpp#L2491
> >
> > On Sun, Mar 15, 2015 at 11:45 AM, James DeFelice <
> james.defelice@gmail.com
> > >
> > wrote:
> >
> > > Is there a reason that the native executor driver bindings don't check
> > > for/enforce a TaskStatus.Source of SOURCE_EXECUTOR?
> > >
> > > https://github.com/apache/mesos/blob/master/src/exec/exec.cpp#L490
> > >
> > > Or is that supposed to be handled somewhere else?
> > >
> > > -James
> > >
> >
>
>
>
> --
> James DeFelice
> 585.241.9488 (voice)
> 650.649.6071 (fax)
>
Re: executor source not set
Posted by James DeFelice <ja...@gmail.com>.
Thanks, I found the same. For some reason I'm not seeing a Source set in
the updates that the scheduler is receiving. The updates are being
generated via the mesos-go bindings. The bindings shouldn't matter since
the slave is supposed to fill in the blank. Not sure what's going on, needs
more investigation. FWIW I'm observing this on 0.21.0.
On Sun, Mar 15, 2015 at 9:55 PM, Vinod Kone <vi...@apache.org> wrote:
> It is enforced by the slave to be future proof (a future where there will
> be no executor driver).
>
> https://github.com/apache/mesos/blob/master/src/slave/slave.cpp#L2491
>
> On Sun, Mar 15, 2015 at 11:45 AM, James DeFelice <james.defelice@gmail.com
> >
> wrote:
>
> > Is there a reason that the native executor driver bindings don't check
> > for/enforce a TaskStatus.Source of SOURCE_EXECUTOR?
> >
> > https://github.com/apache/mesos/blob/master/src/exec/exec.cpp#L490
> >
> > Or is that supposed to be handled somewhere else?
> >
> > -James
> >
>
--
James DeFelice
585.241.9488 (voice)
650.649.6071 (fax)
Re: executor source not set
Posted by Vinod Kone <vi...@apache.org>.
It is enforced by the slave to be future proof (a future where there will
be no executor driver).
https://github.com/apache/mesos/blob/master/src/slave/slave.cpp#L2491
On Sun, Mar 15, 2015 at 11:45 AM, James DeFelice <ja...@gmail.com>
wrote:
> Is there a reason that the native executor driver bindings don't check
> for/enforce a TaskStatus.Source of SOURCE_EXECUTOR?
>
> https://github.com/apache/mesos/blob/master/src/exec/exec.cpp#L490
>
> Or is that supposed to be handled somewhere else?
>
> -James
>