You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@reef.apache.org by Dhruv Mahajan <dh...@gmail.com> on 2016/03/22 06:48:35 UTC
Issues with IActiveContext.SubmitContextAndService
HI
In C# driver, when I call IActiveContext.SubmitContextAndService() I got
the "Not implemented" error below. Without this functionality how can we
build the context stacks?
Dhruv
ERROR: Exception in Call_ClrSystemActiveContextHandler_OnNextencountered
error [System.NotImplementedException: The method or operation is not
implemented.
at
Org.Apache.REEF.Driver.Bridge.Events.ActiveContext.SubmitContextAndService(IConfiguration
contextConfiguration, IConfiguration serviceConfiguration)
at Org.Apache.REEF.IMRU.OnREEF.Driver.IMRUDriver`4.OnNext(IActiveContext
activeContext)
at Org.Apache.REEF.Driver.Bridge.ClrSystemHandler`1.OnNext(T value)
at
Org.Apache.REEF.Driver.Bridge.ClrSystemHandlerWrapper.Call_ClrSystemActiveContextHandler_OnNext(UInt64
handle, IActiveContextClr2Java clr2Java)
at
Java_org_apache_reef_javabridge_NativeInterop_clrSystemActiveContextHandlerOnNext(JNIEnv_*
env, _jclass* cls, Int64 handle, _jobject* jactiveContextBridge, _jobject*
jlogger)] with mesage [The method or operation is not implemented.] and
stack trace [ at
Org.Apache.REEF.Driver.Bridge.Events.ActiveContext.SubmitContextAndService(IConfiguration
contextConfiguration, IConfiguration serviceConfiguration)
at Org.Apache.REEF.IMRU.OnREEF.Driver.IMRUDriver`4.OnNext(IActiveContext
activeContext)
at Org.Apache.REEF.Driver.Bridge.ClrSystemHandler`1.OnNext(T value)
at
Org.Apache.REEF.Driver.Bridge.ClrSystemHandlerWrapper.Call_ClrSystemActiveContextHandler_OnNext(UInt64
handle, IActiveContextClr2Java clr2Java)
at
Java_org_apache_reef_javabridge_NativeInterop_clrSystemActiveContextHandlerOnNext(JNIEnv_*
env, _jclass* cls, Int64 handle, _jobject* jactiveContextBridge, _jobject*
jlogger)]
<C++>InteropUtil Start: 0 : 2016-03-21T22:41:26.4129786-07:00 0008
RE: Issues with IActiveContext.SubmitContextAndService
Posted by "Julia Wang (QIUHE)" <Qi...@microsoft.com>.
> Yes, we could catch these exceptions in the `MapTask` and deal with them locally. That way, we
> might not even have to fail the task, as this is an expected exit for a Task, leading to an `ICompletedTask` with the appropriate message inside.
That's right. We should differentiate proper task complete vs. enforced to complete. Using message (either normal task message or message inside TaskException) can be a way.
Julia
-----Original Message-----
From: Markus Weimer [mailto:markus@weimo.de]
Sent: Tuesday, March 22, 2016 12:19 PM
To: dev@reef.apache.org
Subject: Re: Issues with IActiveContext.SubmitContextAndService
On 2016-03-22 11:14, Julia Wang (QIUHE) wrote:
> The other is to put GC services inside Task, then we will look at
> FailedTask
The tricky part is that we need to distinguish this `IFailedTask` from other cases. Here, we expect the task to fail while in many or even all other cases, receiving an `IFailedTask` should fail the job.
As I mentioned, we can't use the timing of the `IFailedTask` relative to the `IFailedEvaluator` event, as there is no guarantee on event ordering across different evaluators.
Hence, we need to differentiate on the type of exception in the `IFailedTask` we receive. Or, as you pointed out:
> If we can control such chained error in Task, for example, task can
> catch such exception and react, we can leverage task massage to tell
> driver what exactly happens.
Yes, we could catch these exceptions in the `MapTask` and deal with them locally. That way, we might not even have to fail the task, as this is an expected exit for a Task, leading to an `ICompletedTask` with the appropriate message inside.
Markus
Re: Issues with IActiveContext.SubmitContextAndService
Posted by Markus Weimer <ma...@weimo.de>.
On 2016-03-22 11:14, Julia Wang (QIUHE) wrote:
> The other is to put GC services inside Task, then we will look at
> FailedTask
The tricky part is that we need to distinguish this `IFailedTask` from
other cases. Here, we expect the task to fail while in many or even all
other cases, receiving an `IFailedTask` should fail the job.
As I mentioned, we can't use the timing of the `IFailedTask` relative to
the `IFailedEvaluator` event, as there is no guarantee on event ordering
across different evaluators.
Hence, we need to differentiate on the type of exception in the
`IFailedTask` we receive. Or, as you pointed out:
> If we can control such chained error in Task, for example, task can
> catch such exception and react, we can leverage task massage to tell
> driver what exactly happens.
Yes, we could catch these exceptions in the `MapTask` and deal with them
locally. That way, we might not even have to fail the task, as this is
an expected exit for a Task, leading to an `ICompletedTask` with the
appropriate message inside.
Markus
RE: Issues with IActiveContext.SubmitContextAndService
Posted by "Julia Wang (QIUHE)" <Qi...@microsoft.com>.
Hi Markus,
I am totally on the same page with you now. The complexity is chained effect. That is REEF-1225 wants to follow up.
I agree the two options you mentioned below,
one is to put GC services in a separate context layer, then we need to distinguish Failed context from FailedEvaluator.
The other is to put GC services inside Task, then we will look at FailedTask
If we can control such chained error in Task, for example, task can catch such exception and react, we can leverage task massage to tell driver what exactly happens. If it is at context level, we need to check if we are able to send proper massage back from context back to driver.
Julia
-----Original Message-----
From: Markus Weimer [mailto:markus@weimo.de]
Sent: Tuesday, March 22, 2016 10:55 AM
To: dev@reef.apache.org
Subject: Re: Issues with IActiveContext.SubmitContextAndService
On 2016-03-22 10:41, Julia Wang (QIUHE) wrote:
> For this phase, what we want to handle is if evaluators/contexts fail,
> request new evaluators, stop un-impacted tasks, then start entire
> group again.
I think you mean the same thing. Let me try to be more precise to sort out the misunderstanding.
Let's say we run an IMRU job on a lot of Evaluators, and one of them, let's call it "F" fails. That is: on the Driver, we receive an `IFailedEvaluator` for Evaluator F.
Today, this means that the job fails, because IMRU doesn't have a handler bound for `IFailedEvaluator`.
In REEF-1223, we aim to add such an event handler. The goal is to react to the failure in the following way:
1. Request a replacement F' for F.
2. Load the partition managed in F into F'
and in parallel:
3. Shut down the current set of still running Tasks.
After which we can
4. Start a fresh set of IMRU tasks on the Evaluators, including F'.
Now, Dhruv points out that when an Evaluator fails, the group communication in the "neighboring" Evaluators will also fail, which leads to a cascade of related failures we receive in the Driver.
We cannot allow these failures to take down the otherwise fine Evaluators. Dhruv's proposal is to separate the Data and the Group Communications into separate Contexts on the Evaluators. That way, the failing group communications won't affect the data stored in those Evaluators.
An alternative design is to keep the Group Communications in the Tasks.
That way, we receive a bunch of `IFailedTask` events in the Driver, and the data kept in the (only) Context in the Evaluators is unaffected.
Of course, both of these approaches hinge on our ability to identify these `IFailedContext` or `IFailedTask` events as part of the (expected) failures after we received an `IFailedEvaluator`. This can be very tricky, as there is no guarantee that the `IFailedEvaluator` is actually received first in time.
Markus
Re: Issues with IActiveContext.SubmitContextAndService
Posted by Markus Weimer <ma...@weimo.de>.
On 2016-03-22 10:41, Julia Wang (QIUHE) wrote:
> For this phase, what we want to handle is if evaluators/contexts
> fail, request new evaluators, stop un-impacted tasks, then start
> entire group again.
I think you mean the same thing. Let me try to be more precise to sort
out the misunderstanding.
Let's say we run an IMRU job on a lot of Evaluators, and one of them,
let's call it "F" fails. That is: on the Driver, we receive an
`IFailedEvaluator` for Evaluator F.
Today, this means that the job fails, because IMRU doesn't have a
handler bound for `IFailedEvaluator`.
In REEF-1223, we aim to add such an event handler. The goal is to react
to the failure in the following way:
1. Request a replacement F' for F.
2. Load the partition managed in F into F'
and in parallel:
3. Shut down the current set of still running Tasks.
After which we can
4. Start a fresh set of IMRU tasks on the Evaluators, including F'.
Now, Dhruv points out that when an Evaluator fails, the group
communication in the "neighboring" Evaluators will also fail, which
leads to a cascade of related failures we receive in the Driver.
We cannot allow these failures to take down the otherwise fine
Evaluators. Dhruv's proposal is to separate the Data and the Group
Communications into separate Contexts on the Evaluators. That way, the
failing group communications won't affect the data stored in those
Evaluators.
An alternative design is to keep the Group Communications in the Tasks.
That way, we receive a bunch of `IFailedTask` events in the Driver, and
the data kept in the (only) Context in the Evaluators is unaffected.
Of course, both of these approaches hinge on our ability to identify
these `IFailedContext` or `IFailedTask` events as part of the (expected)
failures after we received an `IFailedEvaluator`. This can be very
tricky, as there is no guarantee that the `IFailedEvaluator` is actually
received first in time.
Markus
RE: Issues with IActiveContext.SubmitContextAndService
Posted by "Julia Wang (QIUHE)" <Qi...@microsoft.com>.
For this phase, what we want to handle is if evaluators/contexts fail, request new evaluators, stop un-impacted tasks, then start entire group again.
Do we want to distinguish Evaluator failure and context failure? We can discuss about it to see if it brings enough benefit with lower cost.
Thanks,
Julia
-----Original Message-----
From: Dhruv Mahajan [mailto:dhruv.mahajan@gmail.com]
Sent: Tuesday, March 22, 2016 10:26 AM
To: dev@reef.apache.org
Subject: Re: Issues with IActiveContext.SubmitContextAndService
So, We need to shut down evaluator only in case if they totally fail. As I said one evlauator failing will lead to series of context failures in all other evaluators because of dependency on each other via streams. So we need proper handlers for IFailedContext and IFailedTasks and not simply discard evaluator if this happens.
Dhruv
On Tue, Mar 22, 2016 at 10:19 AM, Julia Wang (QIUHE) < Qiuhe.Wang@microsoft.com> wrote:
> Do you mean you want to separate evaluator failure from context failures?
> At this stage, we will restart evaluator for this case.
>
> -----Original Message-----
> From: Dhruv Mahajan [mailto:dhruv.mahajan@gmail.com]
> Sent: Tuesday, March 22, 2016 10:16 AM
> To: dev@reef.apache.org
> Cc: dev@reef.incubator.apache.org; Jaliya Ekanayake <
> jaliyaek@microsoft.com>
> Subject: Re: Issues with IActiveContext.SubmitContextAndService
>
> Another critical reason for not doing it:
>
> Assuming some evaluator failed, Streams will close and there will be
> stream failures in other evaluators which will lead to failed contexts
> in Group Comm. service. Since DataLoading is part of same context this
> means we have ot load data again even for evaluators that suffered
> failed contexts. Does that make sense?
>
> Dhruv
>
> On Tue, Mar 22, 2016 at 10:07 AM, Dhruv Mahajan
> <dh...@gmail.com>
> wrote:
>
> > In near future, we will need to build ML workflow where driver has
> > to execute series of jobs, say IMRU followed by Parameter server and so on.
> > Common things between all of them will be data and not Group
> > Comm.....Hence, I would like to keep it as a separate layer.
> >
> > Dhruv
> >
> > On Tue, Mar 22, 2016 at 10:04 AM, Julia Wang (QIUHE) <
> > Qiuhe.Wang@microsoft.com> wrote:
> >
> >> Are you submitting the second context? Why not to submit everything
> >> with the IAllocatedEvaluator. SubmitContextAndSErvice()? What is
> >> the consideration to use second context?
> >>
> >> -----Original Message-----
> >> From: Dhruv Mahajan [mailto:dhruv.mahajan@gmail.com]
> >> Sent: Tuesday, March 22, 2016 10:00 AM
> >> To: dev@reef.apache.org
> >> Cc: dev@reef.incubator.apache.org; Jaliya Ekanayake <
> >> jaliyaek@microsoft.com>
> >> Subject: Re: Issues with IActiveContext.SubmitContextAndService
> >>
> >> So first step:
> >>
> >> a) When we get IAllocatedEvaluator, we first submit DataLoading
> >> Context and Service and Data gets read.
> >> b) Then once we get IActiveContext with data loaded, we need to
> >> call
> >> IActiveContext.SubmitContextAndSErvice() for group comm. service.
> >> c) Now once we get the IActiveContext back with Group comm. service
> >> also instanitates we will submit the IMRU task.
> >>
> >> We do not want to mix the two. Data Loading has to be separate from
> >> everything else.
> >>
> >> Dhruv
> >>
> >> On Tue, Mar 22, 2016 at 9:56 AM, Julia Wang (QIUHE) <
> >> Qiuhe.Wang@microsoft.com> wrote:
> >>
> >> > In which scenario you need to use IActiveContext to
> >> > SubmitContextAndService?
> >> >
> >> > My understanding is that is fault tolerant case, you
> >> > SubmitContextAndService with IAllocatedEvaluator at first time.
> >> > If any evaluator fails, we resubmit a new evaluator.
> >> >
> >> > Thanks,
> >> > Julia
> >> >
> >> > -----Original Message-----
> >> > From: Dhruv Mahajan [mailto:dhruv.mahajan@gmail.com]
> >> > Sent: Tuesday, March 22, 2016 9:28 AM
> >> > To: dev@reef.apache.org
> >> > Cc: dev@reef.incubator.apache.org
> >> > Subject: Re: Issues with IActiveContext.SubmitContextAndService
> >> >
> >> > So are we stuck with REEF-1224 till this is resolved? Or for now
> >> > passing whatever Group Communication service configuration
> >> > generate as part of context service will work just fine? We are
> >> > not going to stack it further once we put GC context.
> >> >
> >> > Dhruv
> >> >
> >> > On Tue, Mar 22, 2016 at 8:44 AM, Markus Weimer <ma...@weimo.de>
> wrote:
> >> >
> >> > > On 2016-03-21 23:46, Julia Wang (QIUHE) wrote:
> >> > >
> >> > >> With IActiveContext, we support SubmitContext() but not
> >> > >> SubmitContextAndService yet I believe.
> >> > >>
> >> > >
> >> > > OK. I filed
> >> > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2f
> >> > is
> >> > sue
> >> > s.apache.org%2fjira%2fbrowse%2fREEF-1267&data=01%7c01%7cQiuhe.Wan
> >> > g%
> >> > 40m
> >> > icrosoft.com%7c8e535ba1ae2d4790953108d3526eec76%7c72f988bf86f141a
> >> > f9
> >> > 1ab
> >> > 2d7cd011db47%7c1&sdata=mLGuPsaFUimYnjutmazr9xZSk2U7XTEweiauEXBUdf
> >> > 0%
> >> > 3d
> >> > for this.
> >> > >
> >> > > Markus
> >> > >
> >> >
> >>
> >
> >
>
Re: Issues with IActiveContext.SubmitContextAndService
Posted by Dhruv Mahajan <dh...@gmail.com>.
So, We need to shut down evaluator only in case if they totally fail. As I
said one evlauator failing will lead to series of context failures in all
other evaluators because of dependency on each other via streams. So we
need proper handlers for IFailedContext and IFailedTasks and not simply
discard evaluator if this happens.
Dhruv
On Tue, Mar 22, 2016 at 10:19 AM, Julia Wang (QIUHE) <
Qiuhe.Wang@microsoft.com> wrote:
> Do you mean you want to separate evaluator failure from context failures?
> At this stage, we will restart evaluator for this case.
>
> -----Original Message-----
> From: Dhruv Mahajan [mailto:dhruv.mahajan@gmail.com]
> Sent: Tuesday, March 22, 2016 10:16 AM
> To: dev@reef.apache.org
> Cc: dev@reef.incubator.apache.org; Jaliya Ekanayake <
> jaliyaek@microsoft.com>
> Subject: Re: Issues with IActiveContext.SubmitContextAndService
>
> Another critical reason for not doing it:
>
> Assuming some evaluator failed, Streams will close and there will be
> stream failures in other evaluators which will lead to failed contexts in
> Group Comm. service. Since DataLoading is part of same context this means
> we have ot load data again even for evaluators that suffered failed
> contexts. Does that make sense?
>
> Dhruv
>
> On Tue, Mar 22, 2016 at 10:07 AM, Dhruv Mahajan <dh...@gmail.com>
> wrote:
>
> > In near future, we will need to build ML workflow where driver has to
> > execute series of jobs, say IMRU followed by Parameter server and so on.
> > Common things between all of them will be data and not Group
> > Comm.....Hence, I would like to keep it as a separate layer.
> >
> > Dhruv
> >
> > On Tue, Mar 22, 2016 at 10:04 AM, Julia Wang (QIUHE) <
> > Qiuhe.Wang@microsoft.com> wrote:
> >
> >> Are you submitting the second context? Why not to submit everything
> >> with the IAllocatedEvaluator. SubmitContextAndSErvice()? What is the
> >> consideration to use second context?
> >>
> >> -----Original Message-----
> >> From: Dhruv Mahajan [mailto:dhruv.mahajan@gmail.com]
> >> Sent: Tuesday, March 22, 2016 10:00 AM
> >> To: dev@reef.apache.org
> >> Cc: dev@reef.incubator.apache.org; Jaliya Ekanayake <
> >> jaliyaek@microsoft.com>
> >> Subject: Re: Issues with IActiveContext.SubmitContextAndService
> >>
> >> So first step:
> >>
> >> a) When we get IAllocatedEvaluator, we first submit DataLoading
> >> Context and Service and Data gets read.
> >> b) Then once we get IActiveContext with data loaded, we need to call
> >> IActiveContext.SubmitContextAndSErvice() for group comm. service.
> >> c) Now once we get the IActiveContext back with Group comm. service
> >> also instanitates we will submit the IMRU task.
> >>
> >> We do not want to mix the two. Data Loading has to be separate from
> >> everything else.
> >>
> >> Dhruv
> >>
> >> On Tue, Mar 22, 2016 at 9:56 AM, Julia Wang (QIUHE) <
> >> Qiuhe.Wang@microsoft.com> wrote:
> >>
> >> > In which scenario you need to use IActiveContext to
> >> > SubmitContextAndService?
> >> >
> >> > My understanding is that is fault tolerant case, you
> >> > SubmitContextAndService with IAllocatedEvaluator at first time. If
> >> > any evaluator fails, we resubmit a new evaluator.
> >> >
> >> > Thanks,
> >> > Julia
> >> >
> >> > -----Original Message-----
> >> > From: Dhruv Mahajan [mailto:dhruv.mahajan@gmail.com]
> >> > Sent: Tuesday, March 22, 2016 9:28 AM
> >> > To: dev@reef.apache.org
> >> > Cc: dev@reef.incubator.apache.org
> >> > Subject: Re: Issues with IActiveContext.SubmitContextAndService
> >> >
> >> > So are we stuck with REEF-1224 till this is resolved? Or for now
> >> > passing whatever Group Communication service configuration generate
> >> > as part of context service will work just fine? We are not going to
> >> > stack it further once we put GC context.
> >> >
> >> > Dhruv
> >> >
> >> > On Tue, Mar 22, 2016 at 8:44 AM, Markus Weimer <ma...@weimo.de>
> wrote:
> >> >
> >> > > On 2016-03-21 23:46, Julia Wang (QIUHE) wrote:
> >> > >
> >> > >> With IActiveContext, we support SubmitContext() but not
> >> > >> SubmitContextAndService yet I believe.
> >> > >>
> >> > >
> >> > > OK. I filed
> >> > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fis
> >> > sue
> >> > s.apache.org%2fjira%2fbrowse%2fREEF-1267&data=01%7c01%7cQiuhe.Wang%
> >> > 40m
> >> > icrosoft.com%7c8e535ba1ae2d4790953108d3526eec76%7c72f988bf86f141af9
> >> > 1ab
> >> > 2d7cd011db47%7c1&sdata=mLGuPsaFUimYnjutmazr9xZSk2U7XTEweiauEXBUdf0%
> >> > 3d
> >> > for this.
> >> > >
> >> > > Markus
> >> > >
> >> >
> >>
> >
> >
>
RE: Issues with IActiveContext.SubmitContextAndService
Posted by "Julia Wang (QIUHE)" <Qi...@microsoft.com>.
Do you mean you want to separate evaluator failure from context failures? At this stage, we will restart evaluator for this case.
-----Original Message-----
From: Dhruv Mahajan [mailto:dhruv.mahajan@gmail.com]
Sent: Tuesday, March 22, 2016 10:16 AM
To: dev@reef.apache.org
Cc: dev@reef.incubator.apache.org; Jaliya Ekanayake <ja...@microsoft.com>
Subject: Re: Issues with IActiveContext.SubmitContextAndService
Another critical reason for not doing it:
Assuming some evaluator failed, Streams will close and there will be stream failures in other evaluators which will lead to failed contexts in Group Comm. service. Since DataLoading is part of same context this means we have ot load data again even for evaluators that suffered failed contexts. Does that make sense?
Dhruv
On Tue, Mar 22, 2016 at 10:07 AM, Dhruv Mahajan <dh...@gmail.com>
wrote:
> In near future, we will need to build ML workflow where driver has to
> execute series of jobs, say IMRU followed by Parameter server and so on.
> Common things between all of them will be data and not Group
> Comm.....Hence, I would like to keep it as a separate layer.
>
> Dhruv
>
> On Tue, Mar 22, 2016 at 10:04 AM, Julia Wang (QIUHE) <
> Qiuhe.Wang@microsoft.com> wrote:
>
>> Are you submitting the second context? Why not to submit everything
>> with the IAllocatedEvaluator. SubmitContextAndSErvice()? What is the
>> consideration to use second context?
>>
>> -----Original Message-----
>> From: Dhruv Mahajan [mailto:dhruv.mahajan@gmail.com]
>> Sent: Tuesday, March 22, 2016 10:00 AM
>> To: dev@reef.apache.org
>> Cc: dev@reef.incubator.apache.org; Jaliya Ekanayake <
>> jaliyaek@microsoft.com>
>> Subject: Re: Issues with IActiveContext.SubmitContextAndService
>>
>> So first step:
>>
>> a) When we get IAllocatedEvaluator, we first submit DataLoading
>> Context and Service and Data gets read.
>> b) Then once we get IActiveContext with data loaded, we need to call
>> IActiveContext.SubmitContextAndSErvice() for group comm. service.
>> c) Now once we get the IActiveContext back with Group comm. service
>> also instanitates we will submit the IMRU task.
>>
>> We do not want to mix the two. Data Loading has to be separate from
>> everything else.
>>
>> Dhruv
>>
>> On Tue, Mar 22, 2016 at 9:56 AM, Julia Wang (QIUHE) <
>> Qiuhe.Wang@microsoft.com> wrote:
>>
>> > In which scenario you need to use IActiveContext to
>> > SubmitContextAndService?
>> >
>> > My understanding is that is fault tolerant case, you
>> > SubmitContextAndService with IAllocatedEvaluator at first time. If
>> > any evaluator fails, we resubmit a new evaluator.
>> >
>> > Thanks,
>> > Julia
>> >
>> > -----Original Message-----
>> > From: Dhruv Mahajan [mailto:dhruv.mahajan@gmail.com]
>> > Sent: Tuesday, March 22, 2016 9:28 AM
>> > To: dev@reef.apache.org
>> > Cc: dev@reef.incubator.apache.org
>> > Subject: Re: Issues with IActiveContext.SubmitContextAndService
>> >
>> > So are we stuck with REEF-1224 till this is resolved? Or for now
>> > passing whatever Group Communication service configuration generate
>> > as part of context service will work just fine? We are not going to
>> > stack it further once we put GC context.
>> >
>> > Dhruv
>> >
>> > On Tue, Mar 22, 2016 at 8:44 AM, Markus Weimer <ma...@weimo.de> wrote:
>> >
>> > > On 2016-03-21 23:46, Julia Wang (QIUHE) wrote:
>> > >
>> > >> With IActiveContext, we support SubmitContext() but not
>> > >> SubmitContextAndService yet I believe.
>> > >>
>> > >
>> > > OK. I filed
>> > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fis
>> > sue
>> > s.apache.org%2fjira%2fbrowse%2fREEF-1267&data=01%7c01%7cQiuhe.Wang%
>> > 40m
>> > icrosoft.com%7c8e535ba1ae2d4790953108d3526eec76%7c72f988bf86f141af9
>> > 1ab
>> > 2d7cd011db47%7c1&sdata=mLGuPsaFUimYnjutmazr9xZSk2U7XTEweiauEXBUdf0%
>> > 3d
>> > for this.
>> > >
>> > > Markus
>> > >
>> >
>>
>
>
Re: Issues with IActiveContext.SubmitContextAndService
Posted by Dhruv Mahajan <dh...@gmail.com>.
Another critical reason for not doing it:
Assuming some evaluator failed, Streams will close and there will be stream
failures in other evaluators which will lead to failed contexts in Group
Comm. service. Since DataLoading is part of same context this means we have
ot load data again even for evaluators that suffered failed contexts. Does
that make sense?
Dhruv
On Tue, Mar 22, 2016 at 10:07 AM, Dhruv Mahajan <dh...@gmail.com>
wrote:
> In near future, we will need to build ML workflow where driver has to
> execute series of jobs, say IMRU followed by Parameter server and so on.
> Common things between all of them will be data and not Group
> Comm.....Hence, I would like to keep it as a separate layer.
>
> Dhruv
>
> On Tue, Mar 22, 2016 at 10:04 AM, Julia Wang (QIUHE) <
> Qiuhe.Wang@microsoft.com> wrote:
>
>> Are you submitting the second context? Why not to submit everything with
>> the IAllocatedEvaluator. SubmitContextAndSErvice()? What is the
>> consideration to use second context?
>>
>> -----Original Message-----
>> From: Dhruv Mahajan [mailto:dhruv.mahajan@gmail.com]
>> Sent: Tuesday, March 22, 2016 10:00 AM
>> To: dev@reef.apache.org
>> Cc: dev@reef.incubator.apache.org; Jaliya Ekanayake <
>> jaliyaek@microsoft.com>
>> Subject: Re: Issues with IActiveContext.SubmitContextAndService
>>
>> So first step:
>>
>> a) When we get IAllocatedEvaluator, we first submit DataLoading Context
>> and Service and Data gets read.
>> b) Then once we get IActiveContext with data loaded, we need to call
>> IActiveContext.SubmitContextAndSErvice() for group comm. service.
>> c) Now once we get the IActiveContext back with Group comm. service also
>> instanitates we will submit the IMRU task.
>>
>> We do not want to mix the two. Data Loading has to be separate from
>> everything else.
>>
>> Dhruv
>>
>> On Tue, Mar 22, 2016 at 9:56 AM, Julia Wang (QIUHE) <
>> Qiuhe.Wang@microsoft.com> wrote:
>>
>> > In which scenario you need to use IActiveContext to
>> > SubmitContextAndService?
>> >
>> > My understanding is that is fault tolerant case, you
>> > SubmitContextAndService with IAllocatedEvaluator at first time. If any
>> > evaluator fails, we resubmit a new evaluator.
>> >
>> > Thanks,
>> > Julia
>> >
>> > -----Original Message-----
>> > From: Dhruv Mahajan [mailto:dhruv.mahajan@gmail.com]
>> > Sent: Tuesday, March 22, 2016 9:28 AM
>> > To: dev@reef.apache.org
>> > Cc: dev@reef.incubator.apache.org
>> > Subject: Re: Issues with IActiveContext.SubmitContextAndService
>> >
>> > So are we stuck with REEF-1224 till this is resolved? Or for now
>> > passing whatever Group Communication service configuration generate as
>> > part of context service will work just fine? We are not going to stack
>> > it further once we put GC context.
>> >
>> > Dhruv
>> >
>> > On Tue, Mar 22, 2016 at 8:44 AM, Markus Weimer <ma...@weimo.de> wrote:
>> >
>> > > On 2016-03-21 23:46, Julia Wang (QIUHE) wrote:
>> > >
>> > >> With IActiveContext, we support SubmitContext() but not
>> > >> SubmitContextAndService yet I believe.
>> > >>
>> > >
>> > > OK. I filed
>> > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fissue
>> > s.apache.org%2fjira%2fbrowse%2fREEF-1267&data=01%7c01%7cQiuhe.Wang%40m
>> > icrosoft.com%7c8e535ba1ae2d4790953108d3526eec76%7c72f988bf86f141af91ab
>> > 2d7cd011db47%7c1&sdata=mLGuPsaFUimYnjutmazr9xZSk2U7XTEweiauEXBUdf0%3d
>> > for this.
>> > >
>> > > Markus
>> > >
>> >
>>
>
>
Re: Issues with IActiveContext.SubmitContextAndService
Posted by Dhruv Mahajan <dh...@gmail.com>.
In near future, we will need to build ML workflow where driver has to
execute series of jobs, say IMRU followed by Parameter server and so on.
Common things between all of them will be data and not Group
Comm.....Hence, I would like to keep it as a separate layer.
Dhruv
On Tue, Mar 22, 2016 at 10:04 AM, Julia Wang (QIUHE) <
Qiuhe.Wang@microsoft.com> wrote:
> Are you submitting the second context? Why not to submit everything with
> the IAllocatedEvaluator. SubmitContextAndSErvice()? What is the
> consideration to use second context?
>
> -----Original Message-----
> From: Dhruv Mahajan [mailto:dhruv.mahajan@gmail.com]
> Sent: Tuesday, March 22, 2016 10:00 AM
> To: dev@reef.apache.org
> Cc: dev@reef.incubator.apache.org; Jaliya Ekanayake <
> jaliyaek@microsoft.com>
> Subject: Re: Issues with IActiveContext.SubmitContextAndService
>
> So first step:
>
> a) When we get IAllocatedEvaluator, we first submit DataLoading Context
> and Service and Data gets read.
> b) Then once we get IActiveContext with data loaded, we need to call
> IActiveContext.SubmitContextAndSErvice() for group comm. service.
> c) Now once we get the IActiveContext back with Group comm. service also
> instanitates we will submit the IMRU task.
>
> We do not want to mix the two. Data Loading has to be separate from
> everything else.
>
> Dhruv
>
> On Tue, Mar 22, 2016 at 9:56 AM, Julia Wang (QIUHE) <
> Qiuhe.Wang@microsoft.com> wrote:
>
> > In which scenario you need to use IActiveContext to
> > SubmitContextAndService?
> >
> > My understanding is that is fault tolerant case, you
> > SubmitContextAndService with IAllocatedEvaluator at first time. If any
> > evaluator fails, we resubmit a new evaluator.
> >
> > Thanks,
> > Julia
> >
> > -----Original Message-----
> > From: Dhruv Mahajan [mailto:dhruv.mahajan@gmail.com]
> > Sent: Tuesday, March 22, 2016 9:28 AM
> > To: dev@reef.apache.org
> > Cc: dev@reef.incubator.apache.org
> > Subject: Re: Issues with IActiveContext.SubmitContextAndService
> >
> > So are we stuck with REEF-1224 till this is resolved? Or for now
> > passing whatever Group Communication service configuration generate as
> > part of context service will work just fine? We are not going to stack
> > it further once we put GC context.
> >
> > Dhruv
> >
> > On Tue, Mar 22, 2016 at 8:44 AM, Markus Weimer <ma...@weimo.de> wrote:
> >
> > > On 2016-03-21 23:46, Julia Wang (QIUHE) wrote:
> > >
> > >> With IActiveContext, we support SubmitContext() but not
> > >> SubmitContextAndService yet I believe.
> > >>
> > >
> > > OK. I filed
> > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fissue
> > s.apache.org%2fjira%2fbrowse%2fREEF-1267&data=01%7c01%7cQiuhe.Wang%40m
> > icrosoft.com%7c8e535ba1ae2d4790953108d3526eec76%7c72f988bf86f141af91ab
> > 2d7cd011db47%7c1&sdata=mLGuPsaFUimYnjutmazr9xZSk2U7XTEweiauEXBUdf0%3d
> > for this.
> > >
> > > Markus
> > >
> >
>
RE: Issues with IActiveContext.SubmitContextAndService
Posted by "Julia Wang (QIUHE)" <Qi...@microsoft.com>.
Are you submitting the second context? Why not to submit everything with the IAllocatedEvaluator. SubmitContextAndSErvice()? What is the consideration to use second context?
-----Original Message-----
From: Dhruv Mahajan [mailto:dhruv.mahajan@gmail.com]
Sent: Tuesday, March 22, 2016 10:00 AM
To: dev@reef.apache.org
Cc: dev@reef.incubator.apache.org; Jaliya Ekanayake <ja...@microsoft.com>
Subject: Re: Issues with IActiveContext.SubmitContextAndService
So first step:
a) When we get IAllocatedEvaluator, we first submit DataLoading Context and Service and Data gets read.
b) Then once we get IActiveContext with data loaded, we need to call
IActiveContext.SubmitContextAndSErvice() for group comm. service.
c) Now once we get the IActiveContext back with Group comm. service also instanitates we will submit the IMRU task.
We do not want to mix the two. Data Loading has to be separate from everything else.
Dhruv
On Tue, Mar 22, 2016 at 9:56 AM, Julia Wang (QIUHE) < Qiuhe.Wang@microsoft.com> wrote:
> In which scenario you need to use IActiveContext to
> SubmitContextAndService?
>
> My understanding is that is fault tolerant case, you
> SubmitContextAndService with IAllocatedEvaluator at first time. If any
> evaluator fails, we resubmit a new evaluator.
>
> Thanks,
> Julia
>
> -----Original Message-----
> From: Dhruv Mahajan [mailto:dhruv.mahajan@gmail.com]
> Sent: Tuesday, March 22, 2016 9:28 AM
> To: dev@reef.apache.org
> Cc: dev@reef.incubator.apache.org
> Subject: Re: Issues with IActiveContext.SubmitContextAndService
>
> So are we stuck with REEF-1224 till this is resolved? Or for now
> passing whatever Group Communication service configuration generate as
> part of context service will work just fine? We are not going to stack
> it further once we put GC context.
>
> Dhruv
>
> On Tue, Mar 22, 2016 at 8:44 AM, Markus Weimer <ma...@weimo.de> wrote:
>
> > On 2016-03-21 23:46, Julia Wang (QIUHE) wrote:
> >
> >> With IActiveContext, we support SubmitContext() but not
> >> SubmitContextAndService yet I believe.
> >>
> >
> > OK. I filed
> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fissue
> s.apache.org%2fjira%2fbrowse%2fREEF-1267&data=01%7c01%7cQiuhe.Wang%40m
> icrosoft.com%7c8e535ba1ae2d4790953108d3526eec76%7c72f988bf86f141af91ab
> 2d7cd011db47%7c1&sdata=mLGuPsaFUimYnjutmazr9xZSk2U7XTEweiauEXBUdf0%3d
> for this.
> >
> > Markus
> >
>
RE: Issues with IActiveContext.SubmitContextAndService
Posted by "Julia Wang (QIUHE)" <Qi...@microsoft.com>.
We had discussions of removing services in future. You can always leverage IObserver<OnContextStart> bound to the context for services.
-----Original Message-----
From: Dhruv Mahajan [mailto:dhruv.mahajan@gmail.com]
Sent: Tuesday, March 22, 2016 10:10 AM
To: dev@reef.apache.org
Subject: Re: Issues with IActiveContext.SubmitContextAndService
Markus, assuming we do this, why do we need to keep separation at task level. As discussed earlier, we can put data loading in the context itself by binding IObserver<OnContextStart>. Am I missing something?
Dhruv
On Tue, Mar 22, 2016 at 10:07 AM, Markus Weimer <ma...@weimo.de> wrote:
> On 2016-03-22 09:59, Dhruv Mahajan wrote:
> > We do not want to mix the two. Data Loading has to be separate from
> > everything else.
>
> You could bind both these services into the same context for now. The
> separation can happen at the Task level: A first Task loads the data,
> and a second Task performs the ML.
>
> The upside is that you are unblocked, the downside is that we probably
> need to revisit this in the future.
>
> Markus
>
Re: Issues with IActiveContext.SubmitContextAndService
Posted by Dhruv Mahajan <dh...@gmail.com>.
Markus, assuming we do this, why do we need to keep separation at task
level. As discussed earlier, we can put data loading in the context itself
by binding IObserver<OnContextStart>. Am I missing something?
Dhruv
On Tue, Mar 22, 2016 at 10:07 AM, Markus Weimer <ma...@weimo.de> wrote:
> On 2016-03-22 09:59, Dhruv Mahajan wrote:
> > We do not want to mix the two. Data Loading has to be separate from
> > everything else.
>
> You could bind both these services into the same context for now. The
> separation can happen at the Task level: A first Task loads the data,
> and a second Task performs the ML.
>
> The upside is that you are unblocked, the downside is that we probably
> need to revisit this in the future.
>
> Markus
>
Re: Issues with IActiveContext.SubmitContextAndService
Posted by Markus Weimer <ma...@weimo.de>.
On 2016-03-22 09:59, Dhruv Mahajan wrote:
> We do not want to mix the two. Data Loading has to be separate from
> everything else.
You could bind both these services into the same context for now. The
separation can happen at the Task level: A first Task loads the data,
and a second Task performs the ML.
The upside is that you are unblocked, the downside is that we probably
need to revisit this in the future.
Markus
Re: Issues with IActiveContext.SubmitContextAndService
Posted by Dhruv Mahajan <dh...@gmail.com>.
So first step:
a) When we get IAllocatedEvaluator, we first submit DataLoading Context and
Service and Data gets read.
b) Then once we get IActiveContext with data loaded, we need to call
IActiveContext.SubmitContextAndSErvice() for group comm. service.
c) Now once we get the IActiveContext back with Group comm. service also
instanitates we will submit the IMRU task.
We do not want to mix the two. Data Loading has to be separate from
everything else.
Dhruv
On Tue, Mar 22, 2016 at 9:56 AM, Julia Wang (QIUHE) <
Qiuhe.Wang@microsoft.com> wrote:
> In which scenario you need to use IActiveContext to
> SubmitContextAndService?
>
> My understanding is that is fault tolerant case, you
> SubmitContextAndService with IAllocatedEvaluator at first time. If any
> evaluator fails, we resubmit a new evaluator.
>
> Thanks,
> Julia
>
> -----Original Message-----
> From: Dhruv Mahajan [mailto:dhruv.mahajan@gmail.com]
> Sent: Tuesday, March 22, 2016 9:28 AM
> To: dev@reef.apache.org
> Cc: dev@reef.incubator.apache.org
> Subject: Re: Issues with IActiveContext.SubmitContextAndService
>
> So are we stuck with REEF-1224 till this is resolved? Or for now passing
> whatever Group Communication service configuration generate as part of
> context service will work just fine? We are not going to stack it further
> once we put GC context.
>
> Dhruv
>
> On Tue, Mar 22, 2016 at 8:44 AM, Markus Weimer <ma...@weimo.de> wrote:
>
> > On 2016-03-21 23:46, Julia Wang (QIUHE) wrote:
> >
> >> With IActiveContext, we support SubmitContext() but not
> >> SubmitContextAndService yet I believe.
> >>
> >
> > OK. I filed
> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fissues.apache.org%2fjira%2fbrowse%2fREEF-1267&data=01%7c01%7cQiuhe.Wang%40microsoft.com%7c8e535ba1ae2d4790953108d3526eec76%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=mLGuPsaFUimYnjutmazr9xZSk2U7XTEweiauEXBUdf0%3d
> for this.
> >
> > Markus
> >
>
RE: Issues with IActiveContext.SubmitContextAndService
Posted by "Julia Wang (QIUHE)" <Qi...@microsoft.com>.
In which scenario you need to use IActiveContext to SubmitContextAndService?
My understanding is that is fault tolerant case, you SubmitContextAndService with IAllocatedEvaluator at first time. If any evaluator fails, we resubmit a new evaluator.
Thanks,
Julia
-----Original Message-----
From: Dhruv Mahajan [mailto:dhruv.mahajan@gmail.com]
Sent: Tuesday, March 22, 2016 9:28 AM
To: dev@reef.apache.org
Cc: dev@reef.incubator.apache.org
Subject: Re: Issues with IActiveContext.SubmitContextAndService
So are we stuck with REEF-1224 till this is resolved? Or for now passing whatever Group Communication service configuration generate as part of context service will work just fine? We are not going to stack it further once we put GC context.
Dhruv
On Tue, Mar 22, 2016 at 8:44 AM, Markus Weimer <ma...@weimo.de> wrote:
> On 2016-03-21 23:46, Julia Wang (QIUHE) wrote:
>
>> With IActiveContext, we support SubmitContext() but not
>> SubmitContextAndService yet I believe.
>>
>
> OK. I filed https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fissues.apache.org%2fjira%2fbrowse%2fREEF-1267&data=01%7c01%7cQiuhe.Wang%40microsoft.com%7c8e535ba1ae2d4790953108d3526eec76%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=mLGuPsaFUimYnjutmazr9xZSk2U7XTEweiauEXBUdf0%3d for this.
>
> Markus
>
Re: Issues with IActiveContext.SubmitContextAndService
Posted by Dhruv Mahajan <dh...@gmail.com>.
So are we stuck with REEF-1224 till this is resolved? Or for now passing
whatever Group Communication service configuration generate as part of
context service will work just fine? We are not going to stack it further
once we put GC context.
Dhruv
On Tue, Mar 22, 2016 at 8:44 AM, Markus Weimer <ma...@weimo.de> wrote:
> On 2016-03-21 23:46, Julia Wang (QIUHE) wrote:
>
>> With IActiveContext, we support SubmitContext() but not
>> SubmitContextAndService yet I believe.
>>
>
> OK. I filed https://issues.apache.org/jira/browse/REEF-1267 for this.
>
> Markus
>
Re: Issues with IActiveContext.SubmitContextAndService
Posted by Markus Weimer <ma...@weimo.de>.
On 2016-03-21 23:46, Julia Wang (QIUHE) wrote:
> With IActiveContext, we support SubmitContext() but not
> SubmitContextAndService yet I believe.
OK. I filed https://issues.apache.org/jira/browse/REEF-1267 for this.
Markus
RE: Issues with IActiveContext.SubmitContextAndService
Posted by "Julia Wang (QIUHE)" <Qi...@microsoft.com>.
With IActiveContext, we support SubmitContext() but not SubmitContextAndService yet I believe.
-----Original Message-----
From: Dhruv Mahajan [mailto:dhruv.mahajan@gmail.com]
Sent: Monday, March 21, 2016 11:39 PM
To: dev@reef.apache.org
Cc: dev@reef.incubator.apache.org
Subject: Re: Issues with IActiveContext.SubmitContextAndService
Julia: WE have always been using
IAllocatedEvaluator.SubmitContextAndService(). I am talking about
IActiveContext.SubmitContextAndService() which is used to build context stacks.
Dhruv
On Mon, Mar 21, 2016 at 10:59 PM, Julia Wang (QIUHE) < Qiuhe.Wang@microsoft.com> wrote:
> SubmitContextAndService is used in many places in the code, an
> example is in BroadcastReduceDriver. The test case has been passing.
>
> Did you change any code around it? If not, I would suggest you to do
> mvn clean install and then build C# solution.
>
> Julia
>
> -----Original Message-----
> From: Dhruv Mahajan [mailto:dhruv.mahajan@gmail.com]
> Sent: Monday, March 21, 2016 10:49 PM
> To: dev@reef.incubator.apache.org
> Subject: Issues with IActiveContext.SubmitContextAndService
>
> HI
>
> In C# driver, when I call IActiveContext.SubmitContextAndService() I
> got the "Not implemented" error below. Without this functionality how
> can we build the context stacks?
>
> Dhruv
>
>
> ERROR: Exception in
> Call_ClrSystemActiveContextHandler_OnNextencountered
> error [System.NotImplementedException: The method or operation is not
> implemented.
> at
>
> Org.Apache.REEF.Driver.Bridge.Events.ActiveContext.SubmitContextAndSer
> vice(IConfiguration contextConfiguration, IConfiguration
> serviceConfiguration)
> at
> Org.Apache.REEF.IMRU.OnREEF.Driver.IMRUDriver`4.OnNext(IActiveContext
> activeContext)
> at Org.Apache.REEF.Driver.Bridge.ClrSystemHandler`1.OnNext(T value)
> at
>
> Org.Apache.REEF.Driver.Bridge.ClrSystemHandlerWrapper.Call_ClrSystemAc
> tiveContextHandler_OnNext(UInt64 handle, IActiveContextClr2Java
> clr2Java)
> at
>
> Java_org_apache_reef_javabridge_NativeInterop_clrSystemActiveContextHa
> ndlerOnNext(JNIEnv_* env, _jclass* cls, Int64 handle, _jobject*
> jactiveContextBridge, _jobject* jlogger)] with mesage [The method or
> operation is not implemented.] and
> stack trace [ at
>
> Org.Apache.REEF.Driver.Bridge.Events.ActiveContext.SubmitContextAndSer
> vice(IConfiguration contextConfiguration, IConfiguration
> serviceConfiguration)
> at
> Org.Apache.REEF.IMRU.OnREEF.Driver.IMRUDriver`4.OnNext(IActiveContext
> activeContext)
> at Org.Apache.REEF.Driver.Bridge.ClrSystemHandler`1.OnNext(T value)
> at
>
> Org.Apache.REEF.Driver.Bridge.ClrSystemHandlerWrapper.Call_ClrSystemAc
> tiveContextHandler_OnNext(UInt64 handle, IActiveContextClr2Java
> clr2Java)
> at
>
> Java_org_apache_reef_javabridge_NativeInterop_clrSystemActiveContextHa
> ndlerOnNext(JNIEnv_* env, _jclass* cls, Int64 handle, _jobject*
> jactiveContextBridge, _jobject* jlogger)] <C++>InteropUtil Start: 0 :
> 2016-03-21T22:41:26.4129786-07:00 0008
>
Re: Issues with IActiveContext.SubmitContextAndService
Posted by Dhruv Mahajan <dh...@gmail.com>.
Julia: WE have always been using
IAllocatedEvaluator.SubmitContextAndService(). I am talking about
IActiveContext.SubmitContextAndService() which is used to build context
stacks.
Dhruv
On Mon, Mar 21, 2016 at 10:59 PM, Julia Wang (QIUHE) <
Qiuhe.Wang@microsoft.com> wrote:
> SubmitContextAndService is used in many places in the code, an example is
> in BroadcastReduceDriver. The test case has been passing.
>
> Did you change any code around it? If not, I would suggest you to do mvn
> clean install and then build C# solution.
>
> Julia
>
> -----Original Message-----
> From: Dhruv Mahajan [mailto:dhruv.mahajan@gmail.com]
> Sent: Monday, March 21, 2016 10:49 PM
> To: dev@reef.incubator.apache.org
> Subject: Issues with IActiveContext.SubmitContextAndService
>
> HI
>
> In C# driver, when I call IActiveContext.SubmitContextAndService() I got
> the "Not implemented" error below. Without this functionality how can we
> build the context stacks?
>
> Dhruv
>
>
> ERROR: Exception in Call_ClrSystemActiveContextHandler_OnNextencountered
> error [System.NotImplementedException: The method or operation is not
> implemented.
> at
>
> Org.Apache.REEF.Driver.Bridge.Events.ActiveContext.SubmitContextAndService(IConfiguration
> contextConfiguration, IConfiguration serviceConfiguration)
> at Org.Apache.REEF.IMRU.OnREEF.Driver.IMRUDriver`4.OnNext(IActiveContext
> activeContext)
> at Org.Apache.REEF.Driver.Bridge.ClrSystemHandler`1.OnNext(T value)
> at
>
> Org.Apache.REEF.Driver.Bridge.ClrSystemHandlerWrapper.Call_ClrSystemActiveContextHandler_OnNext(UInt64
> handle, IActiveContextClr2Java clr2Java)
> at
>
> Java_org_apache_reef_javabridge_NativeInterop_clrSystemActiveContextHandlerOnNext(JNIEnv_*
> env, _jclass* cls, Int64 handle, _jobject* jactiveContextBridge, _jobject*
> jlogger)] with mesage [The method or operation is not implemented.] and
> stack trace [ at
>
> Org.Apache.REEF.Driver.Bridge.Events.ActiveContext.SubmitContextAndService(IConfiguration
> contextConfiguration, IConfiguration serviceConfiguration)
> at Org.Apache.REEF.IMRU.OnREEF.Driver.IMRUDriver`4.OnNext(IActiveContext
> activeContext)
> at Org.Apache.REEF.Driver.Bridge.ClrSystemHandler`1.OnNext(T value)
> at
>
> Org.Apache.REEF.Driver.Bridge.ClrSystemHandlerWrapper.Call_ClrSystemActiveContextHandler_OnNext(UInt64
> handle, IActiveContextClr2Java clr2Java)
> at
>
> Java_org_apache_reef_javabridge_NativeInterop_clrSystemActiveContextHandlerOnNext(JNIEnv_*
> env, _jclass* cls, Int64 handle, _jobject* jactiveContextBridge, _jobject*
> jlogger)] <C++>InteropUtil Start: 0 : 2016-03-21T22:41:26.4129786-07:00 0008
>
RE: Issues with IActiveContext.SubmitContextAndService
Posted by "Julia Wang (QIUHE)" <Qi...@microsoft.com>.
SubmitContextAndService is used in many places in the code, an example is in BroadcastReduceDriver. The test case has been passing.
Did you change any code around it? If not, I would suggest you to do mvn clean install and then build C# solution.
Julia
-----Original Message-----
From: Dhruv Mahajan [mailto:dhruv.mahajan@gmail.com]
Sent: Monday, March 21, 2016 10:49 PM
To: dev@reef.incubator.apache.org
Subject: Issues with IActiveContext.SubmitContextAndService
HI
In C# driver, when I call IActiveContext.SubmitContextAndService() I got the "Not implemented" error below. Without this functionality how can we build the context stacks?
Dhruv
ERROR: Exception in Call_ClrSystemActiveContextHandler_OnNextencountered
error [System.NotImplementedException: The method or operation is not implemented.
at
Org.Apache.REEF.Driver.Bridge.Events.ActiveContext.SubmitContextAndService(IConfiguration
contextConfiguration, IConfiguration serviceConfiguration)
at Org.Apache.REEF.IMRU.OnREEF.Driver.IMRUDriver`4.OnNext(IActiveContext
activeContext)
at Org.Apache.REEF.Driver.Bridge.ClrSystemHandler`1.OnNext(T value)
at
Org.Apache.REEF.Driver.Bridge.ClrSystemHandlerWrapper.Call_ClrSystemActiveContextHandler_OnNext(UInt64
handle, IActiveContextClr2Java clr2Java)
at
Java_org_apache_reef_javabridge_NativeInterop_clrSystemActiveContextHandlerOnNext(JNIEnv_*
env, _jclass* cls, Int64 handle, _jobject* jactiveContextBridge, _jobject* jlogger)] with mesage [The method or operation is not implemented.] and
stack trace [ at
Org.Apache.REEF.Driver.Bridge.Events.ActiveContext.SubmitContextAndService(IConfiguration
contextConfiguration, IConfiguration serviceConfiguration)
at Org.Apache.REEF.IMRU.OnREEF.Driver.IMRUDriver`4.OnNext(IActiveContext
activeContext)
at Org.Apache.REEF.Driver.Bridge.ClrSystemHandler`1.OnNext(T value)
at
Org.Apache.REEF.Driver.Bridge.ClrSystemHandlerWrapper.Call_ClrSystemActiveContextHandler_OnNext(UInt64
handle, IActiveContextClr2Java clr2Java)
at
Java_org_apache_reef_javabridge_NativeInterop_clrSystemActiveContextHandlerOnNext(JNIEnv_*
env, _jclass* cls, Int64 handle, _jobject* jactiveContextBridge, _jobject* jlogger)] <C++>InteropUtil Start: 0 : 2016-03-21T22:41:26.4129786-07:00 0008