You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@reef.apache.org by Andrew Chung <af...@gmail.com> on 2016/02/02 01:25:08 UTC
REEF Services
Hi All,
Recently, I have been going over the .NET Evaluator code to implement
various missing features and have been cross comparing the code with
Java code.
I have come across a few potential issues and questions with regards
to REEF Services:
1. What is the scope of a REEF Service? Are they tied to a Context?
According to the way the code is currently structured (see
ContextRuntime.java), I believe this is the case.
2. Currently, the Java code does not use the variable `services` from
`serviceInjector.GetInstance`. I believe it is possible for certain
implementations of JVM JIT compilers to optimize away services, so
users may end up not having the Services created at all.
If a REEF Service is tied to a Context and is only visible by it and
its children Contexts, I believe we should do the following:
1. Create a `Service` interface that extends `AutoCloseable` and
expose the method `Start()`.
2. Make `ContextRuntime` implement `AutoCloseable`.
3. Close `Service` only when `ContextRuntime` is closed.
How does this sound? Note that this may require modifications to the
`ServiceConfiguration` API we expose now.
Thanks,
Andrew
Re: REEF Services
Posted by Markus Weimer <ma...@weimo.de>.
Sorry for being late to the thread!
On 2016-02-01 16:25, Andrew Chung wrote:
> I have come across a few potential issues and questions with regards
> to REEF Services:
:) One of the least documented parts of the REEF design indeed. You are
in good company.
> 1. What is the scope of a REEF Service? Are they tied to a Context?
> According to the way the code is currently structured (see
> ContextRuntime.java), I believe this is the case.
Yes. The idea of a REEF Service is to expand the API surface of REEF
with plug-ins. We really should have called them that, plug-ins. They
come in two flavors:
(1) Driver side Services: These are basically event handlers bound *in
addition to* the event handlers bound by the user. The name service is
one of these, so is the group communications. Further, the Driver side
Services provide injectable objects for the application Driver to use,
e.g. `GroupCommunicationsDriver`. Frequently, those objects provide
facilities to configure the Evaluator side of the same service.
(2) Evaluator side Services: These services predominantly provide
additional injectable objects for the Task to use. Services live in
Contexts, and receive events based on that: `ContextStart`,
`ContextStop`, ... . Further, they receive Task level events `TaskStart`
and `TaskStop`. The reason for putting Services into Contexts is to
provide a life-cycle for the Services that is independent both of the
Evaluator and the Task life-cycle: A Context (and therefore Services)
can span one Task, some Tasks or the entire life of an Evaluator.
> 2. Currently, the Java code does not use the variable `services` from
> `serviceInjector.GetInstance`. I believe it is possible for certain
> implementations of JVM JIT compilers to optimize away services, so
> users may end up not having the Services created at all.
Which file is this in? It might be a bug we need to resolve?
> If a REEF Service is tied to a Context and is only visible by it and
> its children Contexts, I believe we should do the following: 1.
> Create a `Service` interface that extends `AutoCloseable` and expose
> the method `Start()`.
Why not use events for start and stop?
> Note that this may require modifications to the
> `ServiceConfiguration` API we expose now.
Given that Services aren't implemented in .NET right now, that should be
fine :)
Markus
Re: REEF Services
Posted by Markus Weimer <ma...@weimo.de>.
On 2016-03-03 14:22, Andrew Chung wrote:
> what happens when a user binds an object in a `SharedContextInjector`
> but does not instantiate it until the user pushes a child Context? In
> this case,[...] the object will not be shared across the two children
> Contexts.
Correct. The solution Shravan proposed for this is to have the
`ContextStart` handler of the "Service" depend on the object. That way,
it gets instantiated at pretty much the same time as today. And we can
remove a code path from REEF.
Markus
Re: REEF Services
Posted by Andrew Chung <af...@gmail.com>.
Hi Shravan and Markus,
Sorry for being late to the discussion. Maybe I'm missing something,
but what happens when a user binds an object in a
`SharedContextInjector` but does not instantiate it until the user
pushes a child Context? In this case, it seems like if the first child
Context of the original Context is popped off and a second child
Context is pushed on that requires the object to be injected, the
object will not be shared across the two children Contexts. The
different resulting behavior is pretty subtle here.
Thanks,
Andrew
On Fri, Feb 26, 2016 at 10:31 AM, Shravan Matthur Narayanamurthy
<sh...@gmail.com> wrote:
> Hi Markus,
>
> I think you have captured our conversation very well. It makes perfect
> sense to me :)
> One thing, though, is whether we want to call the ContextConfiguration as
> LocalContextConfiguration or with some other prefix to make it explicit
> that it is not shared.
>
> @Julia: The notion of Service in REEF was to create a mechanism to share
> configuration and the associated state with all the descendants. I think
> creating objects separately was a certain nuance of the design that we feel
> is not necessary after building a couple of "Services". Since the term
> Service is confusing, we just want to get rid of it and use the Start &
> Stop Handlers for controlling the lifecycle of objects that are a part of
> your state which is shared with the descendants. As you rightly point out
> the handlers are tied to the Context's life-cycle. Since we want the state
> to be shared it makes sense to tie the scope and extent of that state
> to the lifetime of the contexts they are a part of.
>
> As a part of this I think we should also create different Start/Stop
> handlers for both the local/shared contexts so that we give finer-grained
> control on the scope & lifetime of the state to the user.
>
> --Shravan
>
>
>
> On Thu, Feb 25, 2016 at 12:55 PM, Julia Wang (QIUHE) <
> Qiuhe.Wang@microsoft.com> wrote:
>
>> Make sense.
>>
>> The only thing is Context Handlers are more tight to ContextLlifeCycle.
>> They are designed (or at least from Name) for doing something when context
>> starts and stops. The services are just some objects which are instantiated
>> with Context objects.
>>
>> -----Original Message-----
>> From: Markus Weimer [mailto:markus@weimo.de]
>> Sent: Thursday, February 25, 2016 10:30 AM
>> To: dev@reef.apache.org
>> Subject: Re: REEF Services
>>
>> Hi,
>>
>> Shravan and I just had a bit of a brainstorm on this least clear concept
>> of the REEF API. Here is what we came up with as a proposal:
>>
>> (1) Remove the explicit notion of `Service` from the API. If an
>> application wants to make sure objects are instantiated before
>> the Tasks or other Contexts are spawned, those objects can be
>> depended upon by the `ContextStart` handlers.
>>
>> (2) Rename `ServiceConfiguration` and `ServiceInjector` and related
>> items to `SharedContextConfiguration` and
>> `SharedContextInjector`. That way, their function is clear: Stuff
>> in them will be inherited by Contexts spawned. Stuff in the
>> `ContextConfiguration` and `ContextInjector` won't be. I use
>> "stuff" here as an overloaded technical term: It denotes all the
>> bindings inside a `Configuration` as well as all the already
>> created instances in an `Injector.
>>
>> Shravan, did I capture our conversation correctly?
>>
>> Does this make sense to anybody but Shravan & me?
>>
>> Markus
>>
Re: REEF Services
Posted by Shravan Matthur Narayanamurthy <sh...@gmail.com>.
Hi Markus,
I think you have captured our conversation very well. It makes perfect
sense to me :)
One thing, though, is whether we want to call the ContextConfiguration as
LocalContextConfiguration or with some other prefix to make it explicit
that it is not shared.
@Julia: The notion of Service in REEF was to create a mechanism to share
configuration and the associated state with all the descendants. I think
creating objects separately was a certain nuance of the design that we feel
is not necessary after building a couple of "Services". Since the term
Service is confusing, we just want to get rid of it and use the Start &
Stop Handlers for controlling the lifecycle of objects that are a part of
your state which is shared with the descendants. As you rightly point out
the handlers are tied to the Context's life-cycle. Since we want the state
to be shared it makes sense to tie the scope and extent of that state
to the lifetime of the contexts they are a part of.
As a part of this I think we should also create different Start/Stop
handlers for both the local/shared contexts so that we give finer-grained
control on the scope & lifetime of the state to the user.
--Shravan
On Thu, Feb 25, 2016 at 12:55 PM, Julia Wang (QIUHE) <
Qiuhe.Wang@microsoft.com> wrote:
> Make sense.
>
> The only thing is Context Handlers are more tight to ContextLlifeCycle.
> They are designed (or at least from Name) for doing something when context
> starts and stops. The services are just some objects which are instantiated
> with Context objects.
>
> -----Original Message-----
> From: Markus Weimer [mailto:markus@weimo.de]
> Sent: Thursday, February 25, 2016 10:30 AM
> To: dev@reef.apache.org
> Subject: Re: REEF Services
>
> Hi,
>
> Shravan and I just had a bit of a brainstorm on this least clear concept
> of the REEF API. Here is what we came up with as a proposal:
>
> (1) Remove the explicit notion of `Service` from the API. If an
> application wants to make sure objects are instantiated before
> the Tasks or other Contexts are spawned, those objects can be
> depended upon by the `ContextStart` handlers.
>
> (2) Rename `ServiceConfiguration` and `ServiceInjector` and related
> items to `SharedContextConfiguration` and
> `SharedContextInjector`. That way, their function is clear: Stuff
> in them will be inherited by Contexts spawned. Stuff in the
> `ContextConfiguration` and `ContextInjector` won't be. I use
> "stuff" here as an overloaded technical term: It denotes all the
> bindings inside a `Configuration` as well as all the already
> created instances in an `Injector.
>
> Shravan, did I capture our conversation correctly?
>
> Does this make sense to anybody but Shravan & me?
>
> Markus
>
RE: REEF Services
Posted by "Julia Wang (QIUHE)" <Qi...@microsoft.com>.
Make sense.
The only thing is Context Handlers are more tight to ContextLlifeCycle. They are designed (or at least from Name) for doing something when context starts and stops. The services are just some objects which are instantiated with Context objects.
-----Original Message-----
From: Markus Weimer [mailto:markus@weimo.de]
Sent: Thursday, February 25, 2016 10:30 AM
To: dev@reef.apache.org
Subject: Re: REEF Services
Hi,
Shravan and I just had a bit of a brainstorm on this least clear concept of the REEF API. Here is what we came up with as a proposal:
(1) Remove the explicit notion of `Service` from the API. If an
application wants to make sure objects are instantiated before
the Tasks or other Contexts are spawned, those objects can be
depended upon by the `ContextStart` handlers.
(2) Rename `ServiceConfiguration` and `ServiceInjector` and related
items to `SharedContextConfiguration` and
`SharedContextInjector`. That way, their function is clear: Stuff
in them will be inherited by Contexts spawned. Stuff in the
`ContextConfiguration` and `ContextInjector` won't be. I use
"stuff" here as an overloaded technical term: It denotes all the
bindings inside a `Configuration` as well as all the already
created instances in an `Injector.
Shravan, did I capture our conversation correctly?
Does this make sense to anybody but Shravan & me?
Markus
Re: REEF Services
Posted by Markus Weimer <ma...@weimo.de>.
Hi,
Shravan and I just had a bit of a brainstorm on this least clear concept
of the REEF API. Here is what we came up with as a proposal:
(1) Remove the explicit notion of `Service` from the API. If an
application wants to make sure objects are instantiated before
the Tasks or other Contexts are spawned, those objects can be
depended upon by the `ContextStart` handlers.
(2) Rename `ServiceConfiguration` and `ServiceInjector` and related
items to `SharedContextConfiguration` and
`SharedContextInjector`. That way, their function is clear: Stuff
in them will be inherited by Contexts spawned. Stuff in the
`ContextConfiguration` and `ContextInjector` won't be. I use
"stuff" here as an overloaded technical term: It denotes all the
bindings inside a `Configuration` as well as all the already
created instances in an `Injector.
Shravan, did I capture our conversation correctly?
Does this make sense to anybody but Shravan & me?
Markus
Re: REEF Services
Posted by Markus Weimer <ma...@weimo.de>.
On Wed, Feb 24, 2016 at 4:10 PM, Shravan Matthur Narayanamurthy
<sh...@gmail.com> wrote:
> Why do we need this distinction? Can we not have only inherited?
Unfortunately not. A Context in the end becomes a `Injector` instance
on the Evaluator. Creating a new Context is synonymous with forking
the Injector of the parent Context. So far, your proposal would work.
However, when forking Injectors in Tang, the same rules apply as when
merging `Configuration` instances: The `Configuration` passed into the
`fork()` call is not allowed to conflict with the Configuration
already in the Injector. And each Context uses the same named
parameters, e.g. for its ID. Hence, there are some things we can't
inherit to subcontexts.
Makes sense?
Markus
Re: REEF Services
Posted by Shravan Matthur Narayanamurthy <sh...@gmail.com>.
Why do we need this distinction? Can we not have only inherited? It will
only be extra references anyway right? If there is some state in the your
hierarchy that is not shared, it is not possible to get rid of the storage
for that anyways.
--Shravan
On Wed, Feb 24, 2016 at 3:04 PM, Markus Weimer <ma...@weimo.de> wrote:
> Yes, that could work. Especially for .NET,where we don't have a legacy
> problem with that.
>
> However, we still need two configurations: One which is inherited by
> sub-contexts and one which isn't. What would be good names for these?
>
> Markus
>
> On Wed, Feb 24, 2016 at 2:50 PM, Shravan Matthur Narayanamurthy
> <sh...@gmail.com> wrote:
> > How about we abandon the Service notion and use the ContextStart/Stop
> > handlers for what we wanted to achieve with Services. If you want the
> > functionality of a Service, when you configure the Context add all the
> > configuration and also implement ContextStart/Stop handlers for
> > implementing lifecycle of the Service with the required objects getting
> > injected into the constructors of the Start handler. This is actually
> what
> > I see in all the implementations I see today.
> > --Shravan
> >
> > On Tue, Feb 16, 2016 at 9:13 AM, Markus Weimer <ma...@weimo.de> wrote:
> >
> >> On 2016-02-12 11:40, Andrew Chung wrote:
> >>
> >>> My understanding is that they are just a set of injectable objects
> >>> that don't necessarily have anything in common.
> >>>
> >>
> >> Correct. That name is misleading, not my finest hour in naming things :)
> >> As the introduction of Services pre-dates the Open Source release of
> >> REEF, let me add a bit of back story:
> >>
> >> Services are meant to be the extension hooks the API of the Driver and
> >> the Evaluators. Tang solves much of this, as one can just merge
> >> `Configuration` for additional libraries to the Driver / Evaluator
> >> `Configuration` and then depend on instances from those libraries.
> >>
> >> On the Driver, this is really all we need, as there is only one Driver
> >> and the life cycle of the Service completely matches the one of the
> Driver.
> >>
> >> On the Evaluator, the story got more tricky: The Evaluator's life cycle
> >> is decoupled from the Task life cycle. There is no guarantee that
> >> subsequent Tasks will need the same Services. Some Services may be
> >> detrimental to the performance of one Task, while required by another.
> >> Think of caching layers which occupy lots of memory as an example. Even
> >> worse, some Services are incompatible with one another (version hell).
> >> Hence, we can't bind Services to the Evaluator's life cycle.
> >>
> >> But we can't bind them to the Task life cycle either: There is no
> >> guarantee that subsequent Tasks on the same Evaluator do not need the
> >> same Services either. And important state might be lost if we cleared
> >> out the Services in between Tasks.
> >>
> >> Hence, we need a way to control the life cycle of these Services
> >> independently of the Evaluator and the Task: Contexts. Mind you,
> >> Services on the Evaluator are just what they are on the Driver:
> >> injectable instances.
> >>
> >> However, we now need to control the singleton property of these objects.
> >> Logically, a Service should be a singleton for the duration of a
> >> Context. In order to make that happen, we need to instantiate them
> >> before the Task gets instantiated. In order to do this, we need an
> >> enumeration of the classes considered part of the Service, hence the
> >> `ServiceConfiguration`: Its main purpose is to tell the Evaluator which
> >> objects it shall instantiate before the Task and hang onto in between
> >> tasks. That is the only thing distinguishing a Service from any other
> >> injectable object.
> >>
> >> Long story short: There is no need for them to all implement a common
> >> base interface. Or that need hasn't come up yet :)
> >>
> >> Markus
> >>
>
Re: REEF Services
Posted by Markus Weimer <ma...@weimo.de>.
If we do that, we'd loose the flexibility of stacking contexts. In the
current design, there is no difference between root and other
contexts, which is more general.
Markus
On Wed, Feb 24, 2016 at 3:32 PM, Julia Wang (QIUHE)
<Qi...@microsoft.com> wrote:
> We can have handlers attached to RootContext level so that they can be inherited by all the sub-contexts, and individual Context level handlers. How about:
>
> RootContextStart/StopHandler
> ContextStart/StopHandler
>
> Thanks,
> Julia
>
> -----Original Message-----
> From: Markus Weimer [mailto:markus@weimo.de]
> Sent: Wednesday, February 24, 2016 3:04 PM
> To: dev@reef.apache.org
> Subject: Re: REEF Services
>
> Yes, that could work. Especially for .NET,where we don't have a legacy problem with that.
>
> However, we still need two configurations: One which is inherited by sub-contexts and one which isn't. What would be good names for these?
>
> Markus
>
> On Wed, Feb 24, 2016 at 2:50 PM, Shravan Matthur Narayanamurthy <sh...@gmail.com> wrote:
>> How about we abandon the Service notion and use the ContextStart/Stop
>> handlers for what we wanted to achieve with Services. If you want the
>> functionality of a Service, when you configure the Context add all the
>> configuration and also implement ContextStart/Stop handlers for
>> implementing lifecycle of the Service with the required objects
>> getting injected into the constructors of the Start handler. This is
>> actually what I see in all the implementations I see today.
>> --Shravan
>>
>> On Tue, Feb 16, 2016 at 9:13 AM, Markus Weimer <ma...@weimo.de> wrote:
>>
>>> On 2016-02-12 11:40, Andrew Chung wrote:
>>>
>>>> My understanding is that they are just a set of injectable objects
>>>> that don't necessarily have anything in common.
>>>>
>>>
>>> Correct. That name is misleading, not my finest hour in naming things
>>> :) As the introduction of Services pre-dates the Open Source release
>>> of REEF, let me add a bit of back story:
>>>
>>> Services are meant to be the extension hooks the API of the Driver
>>> and the Evaluators. Tang solves much of this, as one can just merge
>>> `Configuration` for additional libraries to the Driver / Evaluator
>>> `Configuration` and then depend on instances from those libraries.
>>>
>>> On the Driver, this is really all we need, as there is only one
>>> Driver and the life cycle of the Service completely matches the one of the Driver.
>>>
>>> On the Evaluator, the story got more tricky: The Evaluator's life
>>> cycle is decoupled from the Task life cycle. There is no guarantee
>>> that subsequent Tasks will need the same Services. Some Services may
>>> be detrimental to the performance of one Task, while required by another.
>>> Think of caching layers which occupy lots of memory as an example.
>>> Even worse, some Services are incompatible with one another (version hell).
>>> Hence, we can't bind Services to the Evaluator's life cycle.
>>>
>>> But we can't bind them to the Task life cycle either: There is no
>>> guarantee that subsequent Tasks on the same Evaluator do not need the
>>> same Services either. And important state might be lost if we cleared
>>> out the Services in between Tasks.
>>>
>>> Hence, we need a way to control the life cycle of these Services
>>> independently of the Evaluator and the Task: Contexts. Mind you,
>>> Services on the Evaluator are just what they are on the Driver:
>>> injectable instances.
>>>
>>> However, we now need to control the singleton property of these objects.
>>> Logically, a Service should be a singleton for the duration of a
>>> Context. In order to make that happen, we need to instantiate them
>>> before the Task gets instantiated. In order to do this, we need an
>>> enumeration of the classes considered part of the Service, hence the
>>> `ServiceConfiguration`: Its main purpose is to tell the Evaluator
>>> which objects it shall instantiate before the Task and hang onto in
>>> between tasks. That is the only thing distinguishing a Service from
>>> any other injectable object.
>>>
>>> Long story short: There is no need for them to all implement a common
>>> base interface. Or that need hasn't come up yet :)
>>>
>>> Markus
>>>
RE: REEF Services
Posted by "Julia Wang (QIUHE)" <Qi...@microsoft.com>.
We can have handlers attached to RootContext level so that they can be inherited by all the sub-contexts, and individual Context level handlers. How about:
RootContextStart/StopHandler
ContextStart/StopHandler
Thanks,
Julia
-----Original Message-----
From: Markus Weimer [mailto:markus@weimo.de]
Sent: Wednesday, February 24, 2016 3:04 PM
To: dev@reef.apache.org
Subject: Re: REEF Services
Yes, that could work. Especially for .NET,where we don't have a legacy problem with that.
However, we still need two configurations: One which is inherited by sub-contexts and one which isn't. What would be good names for these?
Markus
On Wed, Feb 24, 2016 at 2:50 PM, Shravan Matthur Narayanamurthy <sh...@gmail.com> wrote:
> How about we abandon the Service notion and use the ContextStart/Stop
> handlers for what we wanted to achieve with Services. If you want the
> functionality of a Service, when you configure the Context add all the
> configuration and also implement ContextStart/Stop handlers for
> implementing lifecycle of the Service with the required objects
> getting injected into the constructors of the Start handler. This is
> actually what I see in all the implementations I see today.
> --Shravan
>
> On Tue, Feb 16, 2016 at 9:13 AM, Markus Weimer <ma...@weimo.de> wrote:
>
>> On 2016-02-12 11:40, Andrew Chung wrote:
>>
>>> My understanding is that they are just a set of injectable objects
>>> that don't necessarily have anything in common.
>>>
>>
>> Correct. That name is misleading, not my finest hour in naming things
>> :) As the introduction of Services pre-dates the Open Source release
>> of REEF, let me add a bit of back story:
>>
>> Services are meant to be the extension hooks the API of the Driver
>> and the Evaluators. Tang solves much of this, as one can just merge
>> `Configuration` for additional libraries to the Driver / Evaluator
>> `Configuration` and then depend on instances from those libraries.
>>
>> On the Driver, this is really all we need, as there is only one
>> Driver and the life cycle of the Service completely matches the one of the Driver.
>>
>> On the Evaluator, the story got more tricky: The Evaluator's life
>> cycle is decoupled from the Task life cycle. There is no guarantee
>> that subsequent Tasks will need the same Services. Some Services may
>> be detrimental to the performance of one Task, while required by another.
>> Think of caching layers which occupy lots of memory as an example.
>> Even worse, some Services are incompatible with one another (version hell).
>> Hence, we can't bind Services to the Evaluator's life cycle.
>>
>> But we can't bind them to the Task life cycle either: There is no
>> guarantee that subsequent Tasks on the same Evaluator do not need the
>> same Services either. And important state might be lost if we cleared
>> out the Services in between Tasks.
>>
>> Hence, we need a way to control the life cycle of these Services
>> independently of the Evaluator and the Task: Contexts. Mind you,
>> Services on the Evaluator are just what they are on the Driver:
>> injectable instances.
>>
>> However, we now need to control the singleton property of these objects.
>> Logically, a Service should be a singleton for the duration of a
>> Context. In order to make that happen, we need to instantiate them
>> before the Task gets instantiated. In order to do this, we need an
>> enumeration of the classes considered part of the Service, hence the
>> `ServiceConfiguration`: Its main purpose is to tell the Evaluator
>> which objects it shall instantiate before the Task and hang onto in
>> between tasks. That is the only thing distinguishing a Service from
>> any other injectable object.
>>
>> Long story short: There is no need for them to all implement a common
>> base interface. Or that need hasn't come up yet :)
>>
>> Markus
>>
Re: REEF Services
Posted by Markus Weimer <ma...@weimo.de>.
Yes, that could work. Especially for .NET,where we don't have a legacy
problem with that.
However, we still need two configurations: One which is inherited by
sub-contexts and one which isn't. What would be good names for these?
Markus
On Wed, Feb 24, 2016 at 2:50 PM, Shravan Matthur Narayanamurthy
<sh...@gmail.com> wrote:
> How about we abandon the Service notion and use the ContextStart/Stop
> handlers for what we wanted to achieve with Services. If you want the
> functionality of a Service, when you configure the Context add all the
> configuration and also implement ContextStart/Stop handlers for
> implementing lifecycle of the Service with the required objects getting
> injected into the constructors of the Start handler. This is actually what
> I see in all the implementations I see today.
> --Shravan
>
> On Tue, Feb 16, 2016 at 9:13 AM, Markus Weimer <ma...@weimo.de> wrote:
>
>> On 2016-02-12 11:40, Andrew Chung wrote:
>>
>>> My understanding is that they are just a set of injectable objects
>>> that don't necessarily have anything in common.
>>>
>>
>> Correct. That name is misleading, not my finest hour in naming things :)
>> As the introduction of Services pre-dates the Open Source release of
>> REEF, let me add a bit of back story:
>>
>> Services are meant to be the extension hooks the API of the Driver and
>> the Evaluators. Tang solves much of this, as one can just merge
>> `Configuration` for additional libraries to the Driver / Evaluator
>> `Configuration` and then depend on instances from those libraries.
>>
>> On the Driver, this is really all we need, as there is only one Driver
>> and the life cycle of the Service completely matches the one of the Driver.
>>
>> On the Evaluator, the story got more tricky: The Evaluator's life cycle
>> is decoupled from the Task life cycle. There is no guarantee that
>> subsequent Tasks will need the same Services. Some Services may be
>> detrimental to the performance of one Task, while required by another.
>> Think of caching layers which occupy lots of memory as an example. Even
>> worse, some Services are incompatible with one another (version hell).
>> Hence, we can't bind Services to the Evaluator's life cycle.
>>
>> But we can't bind them to the Task life cycle either: There is no
>> guarantee that subsequent Tasks on the same Evaluator do not need the
>> same Services either. And important state might be lost if we cleared
>> out the Services in between Tasks.
>>
>> Hence, we need a way to control the life cycle of these Services
>> independently of the Evaluator and the Task: Contexts. Mind you,
>> Services on the Evaluator are just what they are on the Driver:
>> injectable instances.
>>
>> However, we now need to control the singleton property of these objects.
>> Logically, a Service should be a singleton for the duration of a
>> Context. In order to make that happen, we need to instantiate them
>> before the Task gets instantiated. In order to do this, we need an
>> enumeration of the classes considered part of the Service, hence the
>> `ServiceConfiguration`: Its main purpose is to tell the Evaluator which
>> objects it shall instantiate before the Task and hang onto in between
>> tasks. That is the only thing distinguishing a Service from any other
>> injectable object.
>>
>> Long story short: There is no need for them to all implement a common
>> base interface. Or that need hasn't come up yet :)
>>
>> Markus
>>
Re: REEF Services
Posted by Shravan Matthur Narayanamurthy <sh...@gmail.com>.
How about we abandon the Service notion and use the ContextStart/Stop
handlers for what we wanted to achieve with Services. If you want the
functionality of a Service, when you configure the Context add all the
configuration and also implement ContextStart/Stop handlers for
implementing lifecycle of the Service with the required objects getting
injected into the constructors of the Start handler. This is actually what
I see in all the implementations I see today.
--Shravan
On Tue, Feb 16, 2016 at 9:13 AM, Markus Weimer <ma...@weimo.de> wrote:
> On 2016-02-12 11:40, Andrew Chung wrote:
>
>> My understanding is that they are just a set of injectable objects
>> that don't necessarily have anything in common.
>>
>
> Correct. That name is misleading, not my finest hour in naming things :)
> As the introduction of Services pre-dates the Open Source release of
> REEF, let me add a bit of back story:
>
> Services are meant to be the extension hooks the API of the Driver and
> the Evaluators. Tang solves much of this, as one can just merge
> `Configuration` for additional libraries to the Driver / Evaluator
> `Configuration` and then depend on instances from those libraries.
>
> On the Driver, this is really all we need, as there is only one Driver
> and the life cycle of the Service completely matches the one of the Driver.
>
> On the Evaluator, the story got more tricky: The Evaluator's life cycle
> is decoupled from the Task life cycle. There is no guarantee that
> subsequent Tasks will need the same Services. Some Services may be
> detrimental to the performance of one Task, while required by another.
> Think of caching layers which occupy lots of memory as an example. Even
> worse, some Services are incompatible with one another (version hell).
> Hence, we can't bind Services to the Evaluator's life cycle.
>
> But we can't bind them to the Task life cycle either: There is no
> guarantee that subsequent Tasks on the same Evaluator do not need the
> same Services either. And important state might be lost if we cleared
> out the Services in between Tasks.
>
> Hence, we need a way to control the life cycle of these Services
> independently of the Evaluator and the Task: Contexts. Mind you,
> Services on the Evaluator are just what they are on the Driver:
> injectable instances.
>
> However, we now need to control the singleton property of these objects.
> Logically, a Service should be a singleton for the duration of a
> Context. In order to make that happen, we need to instantiate them
> before the Task gets instantiated. In order to do this, we need an
> enumeration of the classes considered part of the Service, hence the
> `ServiceConfiguration`: Its main purpose is to tell the Evaluator which
> objects it shall instantiate before the Task and hang onto in between
> tasks. That is the only thing distinguishing a Service from any other
> injectable object.
>
> Long story short: There is no need for them to all implement a common
> base interface. Or that need hasn't come up yet :)
>
> Markus
>
Re: REEF Services
Posted by Markus Weimer <ma...@weimo.de>.
On 2016-02-12 11:40, Andrew Chung wrote:
> My understanding is that they are just a set of injectable objects
> that don't necessarily have anything in common.
Correct. That name is misleading, not my finest hour in naming things :)
As the introduction of Services pre-dates the Open Source release of
REEF, let me add a bit of back story:
Services are meant to be the extension hooks the API of the Driver and
the Evaluators. Tang solves much of this, as one can just merge
`Configuration` for additional libraries to the Driver / Evaluator
`Configuration` and then depend on instances from those libraries.
On the Driver, this is really all we need, as there is only one Driver
and the life cycle of the Service completely matches the one of the Driver.
On the Evaluator, the story got more tricky: The Evaluator's life cycle
is decoupled from the Task life cycle. There is no guarantee that
subsequent Tasks will need the same Services. Some Services may be
detrimental to the performance of one Task, while required by another.
Think of caching layers which occupy lots of memory as an example. Even
worse, some Services are incompatible with one another (version hell).
Hence, we can't bind Services to the Evaluator's life cycle.
But we can't bind them to the Task life cycle either: There is no
guarantee that subsequent Tasks on the same Evaluator do not need the
same Services either. And important state might be lost if we cleared
out the Services in between Tasks.
Hence, we need a way to control the life cycle of these Services
independently of the Evaluator and the Task: Contexts. Mind you,
Services on the Evaluator are just what they are on the Driver:
injectable instances.
However, we now need to control the singleton property of these objects.
Logically, a Service should be a singleton for the duration of a
Context. In order to make that happen, we need to instantiate them
before the Task gets instantiated. In order to do this, we need an
enumeration of the classes considered part of the Service, hence the
`ServiceConfiguration`: Its main purpose is to tell the Evaluator which
objects it shall instantiate before the Task and hang onto in between
tasks. That is the only thing distinguishing a Service from any other
injectable object.
Long story short: There is no need for them to all implement a common
base interface. Or that need hasn't come up yet :)
Markus
Re: REEF Services
Posted by Andrew Chung <af...@gmail.com>.
My understanding is that they are just a set of injectable objects
that don't necessarily have anything in common. They are called
"service" in the sense that they are accessible as singletons across
the context stack above the context at which the "service" object is
bound and injected.
Thanks,
Andrew
On Fri, Feb 12, 2016 at 11:13 AM, Julia Wang (QIUHE)
<Qi...@microsoft.com> wrote:
> Technically, yes, services are just a set of objects injected from driver, and cab be shared cross contexts, per context, or per task. They are not really services.
>
> Question is shall we still call them as Services?
>
> If we think those classes have something in common, or at least logically we want group them as services, we should then have an interface for those classes such as IService. If not, and they are just some objects, shall we use different names instead of service?
>
> Thanks,
> Julia
>
> -----Original Message-----
> From: Andrew Chung [mailto:afchung90@gmail.com]
> Sent: Tuesday, February 2, 2016 10:17 PM
> To: dev@reef.apache.org
> Subject: Re: REEF Services
>
> Looks like I've had some misunderstanding about the concept of Services earlier -- turns out that "Service" was just a device to allow Tang to have shared objects across contexts, so the objects resulting from the injection doesn't actually need to be used. As a result, we will not need a start/stop function or a Service interface either. I assume Java won't be able to optimize away the `getInstance` call since it has potential side effects, so this seems to be a non-issue. Sorry for the confusion, and thanks for the clarification, Markus!
>
> Thanks,
> Andrew
>
> On Tue, Feb 2, 2016 at 10:14 AM, Julia Wang (QIUHE) <Qi...@microsoft.com> wrote:
>> > * Singleton in the Evaluator: Put the service into the root context,
>> > which is the first context created on an Evaluator. All other
>> > contexts inherit its objects.
>> > * Per-Context: Just put it into that Context.
>> > * Per-Task: Launch a Context just for that Task.
>>
>> This is pretty clear! Thanks for the summary. In current .Net Evaluator, as we only use one active context, so the first and second cases are treated in the same way.
>> '
>> -----Original Message-----
>> From: Markus Weimer [mailto:markus@weimo.de]
>> Sent: Tuesday, February 2, 2016 10:09 AM
>> To: dev@reef.apache.org
>> Subject: Re: REEF Services
>>
>> On 2016-02-02 10:02, Julia Wang (QIUHE) wrote:
>>> If we need one service per context, then it should be tight to
>>> context. If we only need singleton in entire evaluator, it should be
>>> tight to evaluator. If any service is at task level, then it should
>>> be tight to task I guess.
>>
>> That's what the Context stacking is for, and it covers the three cases you mentioned:
>>
>> * Singleton in the Evaluator: Put the service into the root context,
>> which is the first context created on an Evaluator. All other
>> contexts inherit its objects.
>> * Per-Context: Just put it into that Context.
>> * Per-Task: Launch a Context just for that Task.
>>
>> Markus
RE: REEF Services
Posted by "Julia Wang (QIUHE)" <Qi...@microsoft.com>.
Technically, yes, services are just a set of objects injected from driver, and cab be shared cross contexts, per context, or per task. They are not really services.
Question is shall we still call them as Services?
If we think those classes have something in common, or at least logically we want group them as services, we should then have an interface for those classes such as IService. If not, and they are just some objects, shall we use different names instead of service?
Thanks,
Julia
-----Original Message-----
From: Andrew Chung [mailto:afchung90@gmail.com]
Sent: Tuesday, February 2, 2016 10:17 PM
To: dev@reef.apache.org
Subject: Re: REEF Services
Looks like I've had some misunderstanding about the concept of Services earlier -- turns out that "Service" was just a device to allow Tang to have shared objects across contexts, so the objects resulting from the injection doesn't actually need to be used. As a result, we will not need a start/stop function or a Service interface either. I assume Java won't be able to optimize away the `getInstance` call since it has potential side effects, so this seems to be a non-issue. Sorry for the confusion, and thanks for the clarification, Markus!
Thanks,
Andrew
On Tue, Feb 2, 2016 at 10:14 AM, Julia Wang (QIUHE) <Qi...@microsoft.com> wrote:
> > * Singleton in the Evaluator: Put the service into the root context,
> > which is the first context created on an Evaluator. All other
> > contexts inherit its objects.
> > * Per-Context: Just put it into that Context.
> > * Per-Task: Launch a Context just for that Task.
>
> This is pretty clear! Thanks for the summary. In current .Net Evaluator, as we only use one active context, so the first and second cases are treated in the same way.
> '
> -----Original Message-----
> From: Markus Weimer [mailto:markus@weimo.de]
> Sent: Tuesday, February 2, 2016 10:09 AM
> To: dev@reef.apache.org
> Subject: Re: REEF Services
>
> On 2016-02-02 10:02, Julia Wang (QIUHE) wrote:
>> If we need one service per context, then it should be tight to
>> context. If we only need singleton in entire evaluator, it should be
>> tight to evaluator. If any service is at task level, then it should
>> be tight to task I guess.
>
> That's what the Context stacking is for, and it covers the three cases you mentioned:
>
> * Singleton in the Evaluator: Put the service into the root context,
> which is the first context created on an Evaluator. All other
> contexts inherit its objects.
> * Per-Context: Just put it into that Context.
> * Per-Task: Launch a Context just for that Task.
>
> Markus
Re: REEF Services
Posted by Andrew Chung <af...@gmail.com>.
Looks like I've had some misunderstanding about the concept of
Services earlier -- turns out that "Service" was just a device to
allow Tang to have shared objects across contexts, so the objects
resulting from the injection doesn't actually need to be used. As a
result, we will not need a start/stop function or a Service interface
either. I assume Java won't be able to optimize away the `getInstance`
call since it has potential side effects, so this seems to be a
non-issue. Sorry for the confusion, and thanks for the clarification,
Markus!
Thanks,
Andrew
On Tue, Feb 2, 2016 at 10:14 AM, Julia Wang (QIUHE)
<Qi...@microsoft.com> wrote:
> > * Singleton in the Evaluator: Put the service into the root context,
> > which is the first context created on an Evaluator. All other
> > contexts inherit its objects.
> > * Per-Context: Just put it into that Context.
> > * Per-Task: Launch a Context just for that Task.
>
> This is pretty clear! Thanks for the summary. In current .Net Evaluator, as we only use one active context, so the first and second cases are treated in the same way.
> '
> -----Original Message-----
> From: Markus Weimer [mailto:markus@weimo.de]
> Sent: Tuesday, February 2, 2016 10:09 AM
> To: dev@reef.apache.org
> Subject: Re: REEF Services
>
> On 2016-02-02 10:02, Julia Wang (QIUHE) wrote:
>> If we need one service per context, then it should be tight to
>> context. If we only need singleton in entire evaluator, it should be
>> tight to evaluator. If any service is at task level, then it should be
>> tight to task I guess.
>
> That's what the Context stacking is for, and it covers the three cases you mentioned:
>
> * Singleton in the Evaluator: Put the service into the root context,
> which is the first context created on an Evaluator. All other
> contexts inherit its objects.
> * Per-Context: Just put it into that Context.
> * Per-Task: Launch a Context just for that Task.
>
> Markus
RE: REEF Services
Posted by "Julia Wang (QIUHE)" <Qi...@microsoft.com>.
> * Singleton in the Evaluator: Put the service into the root context,
> which is the first context created on an Evaluator. All other
> contexts inherit its objects.
> * Per-Context: Just put it into that Context.
> * Per-Task: Launch a Context just for that Task.
This is pretty clear! Thanks for the summary. In current .Net Evaluator, as we only use one active context, so the first and second cases are treated in the same way.
'
-----Original Message-----
From: Markus Weimer [mailto:markus@weimo.de]
Sent: Tuesday, February 2, 2016 10:09 AM
To: dev@reef.apache.org
Subject: Re: REEF Services
On 2016-02-02 10:02, Julia Wang (QIUHE) wrote:
> If we need one service per context, then it should be tight to
> context. If we only need singleton in entire evaluator, it should be
> tight to evaluator. If any service is at task level, then it should be
> tight to task I guess.
That's what the Context stacking is for, and it covers the three cases you mentioned:
* Singleton in the Evaluator: Put the service into the root context,
which is the first context created on an Evaluator. All other
contexts inherit its objects.
* Per-Context: Just put it into that Context.
* Per-Task: Launch a Context just for that Task.
Markus
Re: REEF Services
Posted by Markus Weimer <ma...@weimo.de>.
On 2016-02-02 10:02, Julia Wang (QIUHE) wrote:
> If we need one service per context, then it should be tight to
> context. If we only need singleton in entire evaluator, it should be
> tight to evaluator. If any service is at task level, then it should
> be tight to task I guess.
That's what the Context stacking is for, and it covers the three cases
you mentioned:
* Singleton in the Evaluator: Put the service into the root context,
which is the first context created on an Evaluator. All other
contexts inherit its objects.
* Per-Context: Just put it into that Context.
* Per-Task: Launch a Context just for that Task.
Markus
RE: REEF Services
Posted by "Julia Wang (QIUHE)" <Qi...@microsoft.com>.
> I haven't seen a case where a Service is not tied to a Context yet.
> But the term "Service" is a bit confusing though since it can be
> interpreted in a few ways.
> When we say Slider can be a service in REEF, the service does not mean
> a REEF Service component.
>
This is a good point. It really depends on what services. Examples are Naming Service, Network Service. If we need one service per context, then it should be tight to context. If we only need singleton in entire evaluator, it should be tight to evaluator. If any service is at task level, then it should be tight to task I guess.
Thanks,
Julia
-----Original Message-----
From: Andrew Chung [mailto:afchung90@gmail.com]
Sent: Tuesday, February 2, 2016 9:43 AM
To: dev@reef.apache.org
Subject: Re: REEF Services
Thanks for the comments Gon! Replies inline.
On Tue, Feb 2, 2016 at 2:13 AM, Byung-Gon Chun <bg...@gmail.com> wrote:
> Thanks, Andrew!
>
> On Tue, Feb 2, 2016 at 9:25 AM, Andrew Chung <af...@gmail.com> wrote:
>
>> Hi All,
>>
>> Recently, I have been going over the .NET Evaluator code to implement
>> various missing features and have been cross comparing the code with
>> Java code.
>>
>> I have come across a few potential issues and questions with regards
>> to REEF Services:
>> 1. What is the scope of a REEF Service? Are they tied to a Context?
>> According to the way the code is currently structured (see
>> https://na01.safelinks.protection.outlook.com/?url=ContextRuntime.java&data=01%7c01%7cQiuhe.Wang%40microsoft.com%7c034bee95cec24eb3304208d32bf86dc9%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=AnXjkvcWKBGkF7%2bT6U0TAoEDLYfO8tdnEOqPd0d6U90%3d), I believe this is the case.
>>
>
> I haven't seen a case where a Service is not tied to a Context yet.
> But the term "Service" is a bit confusing though since it can be
> interpreted in a few ways.
> When we say Slider can be a service in REEF, the service does not mean
> a REEF Service component.
>
I agree, though changing the term at this point may require significant effort... Seeing how `ServiceConfiguration` submission is tied to submission on top of an `ActiveContext`, I think we are safe to make the assumption that a REEF Service is tied to a Context.
>
>> 2. Currently, the Java code does not use the variable `services` from
>> `serviceInjector.GetInstance`. I believe it is possible for certain
>> implementations of JVM JIT compilers to optimize away services, so
>> users may end up not having the Services created at all.
>>
>
> Should we be concerned about item 2?
>
I believe so. Local variable Object construction without using the Object and expecting it to be created or stick around is never good. A solution is to tie services as a private variable to `ContextRuntime`, if Services are indeed tied to Contexts.
See: https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fstackoverflow.com%2fquestions%2f15205016%2fcan-unused-private-variables-be-gced-before-their-holding-instance&data=01%7c01%7cQiuhe.Wang%40microsoft.com%7c034bee95cec24eb3304208d32bf86dc9%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=UvglLSts8Y36RsJcSQFa7D8KrcxpAMimUBYrJHeL%2fs0%3d
>
>>
>> If a REEF Service is tied to a Context and is only visible by it and
>> its children Contexts, I believe we should do the following:
>> 1. Create a `Service` interface that extends `AutoCloseable` and
>> expose the method `Start()`.
>> 2. Make `ContextRuntime` implement `AutoCloseable`.
>> 3. Close `Service` only when `ContextRuntime` is closed.
>>
>> How does this sound? Note that this may require modifications to the
>> `ServiceConfiguration` API we expose now.
>>
>> Thanks,
>> Andrew
>>
>
>
>
> --
> Byung-Gon Chun
Re: REEF Services
Posted by Andrew Chung <af...@gmail.com>.
Thanks for the comments Gon! Replies inline.
On Tue, Feb 2, 2016 at 2:13 AM, Byung-Gon Chun <bg...@gmail.com> wrote:
> Thanks, Andrew!
>
> On Tue, Feb 2, 2016 at 9:25 AM, Andrew Chung <af...@gmail.com> wrote:
>
>> Hi All,
>>
>> Recently, I have been going over the .NET Evaluator code to implement
>> various missing features and have been cross comparing the code with
>> Java code.
>>
>> I have come across a few potential issues and questions with regards
>> to REEF Services:
>> 1. What is the scope of a REEF Service? Are they tied to a Context?
>> According to the way the code is currently structured (see
>> ContextRuntime.java), I believe this is the case.
>>
>
> I haven't seen a case where a Service is not tied to a Context yet.
> But the term "Service" is a bit confusing though since it can be
> interpreted in a few ways.
> When we say Slider can be a service in REEF, the service does not mean a
> REEF Service component.
>
I agree, though changing the term at this point may require
significant effort... Seeing how `ServiceConfiguration` submission is
tied to submission on top of an `ActiveContext`, I think we are safe
to make the assumption that a REEF Service is tied to a Context.
>
>> 2. Currently, the Java code does not use the variable `services` from
>> `serviceInjector.GetInstance`. I believe it is possible for certain
>> implementations of JVM JIT compilers to optimize away services, so
>> users may end up not having the Services created at all.
>>
>
> Should we be concerned about item 2?
>
I believe so. Local variable Object construction without using the
Object and expecting it to be created or stick around is never good. A
solution is to tie services as a private variable to `ContextRuntime`,
if Services are indeed tied to Contexts.
See: http://stackoverflow.com/questions/15205016/can-unused-private-variables-be-gced-before-their-holding-instance
>
>>
>> If a REEF Service is tied to a Context and is only visible by it and
>> its children Contexts, I believe we should do the following:
>> 1. Create a `Service` interface that extends `AutoCloseable` and
>> expose the method `Start()`.
>> 2. Make `ContextRuntime` implement `AutoCloseable`.
>> 3. Close `Service` only when `ContextRuntime` is closed.
>>
>> How does this sound? Note that this may require modifications to the
>> `ServiceConfiguration` API we expose now.
>>
>> Thanks,
>> Andrew
>>
>
>
>
> --
> Byung-Gon Chun
Re: REEF Services
Posted by Byung-Gon Chun <bg...@gmail.com>.
Thanks, Andrew!
On Tue, Feb 2, 2016 at 9:25 AM, Andrew Chung <af...@gmail.com> wrote:
> Hi All,
>
> Recently, I have been going over the .NET Evaluator code to implement
> various missing features and have been cross comparing the code with
> Java code.
>
> I have come across a few potential issues and questions with regards
> to REEF Services:
> 1. What is the scope of a REEF Service? Are they tied to a Context?
> According to the way the code is currently structured (see
> ContextRuntime.java), I believe this is the case.
>
I haven't seen a case where a Service is not tied to a Context yet.
But the term "Service" is a bit confusing though since it can be
interpreted in a few ways.
When we say Slider can be a service in REEF, the service does not mean a
REEF Service component.
> 2. Currently, the Java code does not use the variable `services` from
> `serviceInjector.GetInstance`. I believe it is possible for certain
> implementations of JVM JIT compilers to optimize away services, so
> users may end up not having the Services created at all.
>
Should we be concerned about item 2?
>
> If a REEF Service is tied to a Context and is only visible by it and
> its children Contexts, I believe we should do the following:
> 1. Create a `Service` interface that extends `AutoCloseable` and
> expose the method `Start()`.
> 2. Make `ContextRuntime` implement `AutoCloseable`.
> 3. Close `Service` only when `ContextRuntime` is closed.
>
> How does this sound? Note that this may require modifications to the
> `ServiceConfiguration` API we expose now.
>
> Thanks,
> Andrew
>
--
Byung-Gon Chun
Re: REEF Services
Posted by Andrew Chung <af...@gmail.com>.
Hi Julia,
There was no link in the previous email, it was simply `ContextRuntime
dot java`. Sorry about the confusion!
Services are not cloned, but a `ServiceConfiguration` can be submitted
on top of another context, thus it seems like a Service is related to
a Context and will not be `closed` until the related Context is
closed.
Thanks,
Andrew
On Mon, Feb 1, 2016 at 4:51 PM, Julia Wang (QIUHE)
<Qi...@microsoft.com> wrote:
> The link below doesn't work.
>
> Can you describe the clone scenarios below? Do you mean attach a service to a context, when closing one context, clone it for next context in the stack? What's the difference for sharing the service?
>
> Thanks,
> Julia
>
> -----Original Message-----
> From: Andrew Chung [mailto:afchung90@gmail.com]
> Sent: Monday, February 1, 2016 4:25 PM
> To: dev@reef.apache.org
> Subject: REEF Services
>
> Hi All,
>
> Recently, I have been going over the .NET Evaluator code to implement various missing features and have been cross comparing the code with Java code.
>
> I have come across a few potential issues and questions with regards to REEF Services:
> 1. What is the scope of a REEF Service? Are they tied to a Context?
> According to the way the code is currently structured (see https://na01.safelinks.protection.outlook.com/?url=ContextRuntime.java&data=01%7c01%7cQiuhe.Wang%40microsoft.com%7c38f6984e74c04f76f9e408d32b676778%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=4xcfRtqd4O7bSk9bqRjrDGghRQEWAPq88QFABupfdLQ%3d), I believe this is the case.
> 2. Currently, the Java code does not use the variable `services` from `serviceInjector.GetInstance`. I believe it is possible for certain implementations of JVM JIT compilers to optimize away services, so users may end up not having the Services created at all.
>
> If a REEF Service is tied to a Context and is only visible by it and its children Contexts, I believe we should do the following:
> 1. Create a `Service` interface that extends `AutoCloseable` and expose the method `Start()`.
> 2. Make `ContextRuntime` implement `AutoCloseable`.
> 3. Close `Service` only when `ContextRuntime` is closed.
>
> How does this sound? Note that this may require modifications to the `ServiceConfiguration` API we expose now.
>
> Thanks,
> Andrew
RE: REEF Services
Posted by "Julia Wang (QIUHE)" <Qi...@microsoft.com>.
The link below doesn't work.
Can you describe the clone scenarios below? Do you mean attach a service to a context, when closing one context, clone it for next context in the stack? What's the difference for sharing the service?
Thanks,
Julia
-----Original Message-----
From: Andrew Chung [mailto:afchung90@gmail.com]
Sent: Monday, February 1, 2016 4:25 PM
To: dev@reef.apache.org
Subject: REEF Services
Hi All,
Recently, I have been going over the .NET Evaluator code to implement various missing features and have been cross comparing the code with Java code.
I have come across a few potential issues and questions with regards to REEF Services:
1. What is the scope of a REEF Service? Are they tied to a Context?
According to the way the code is currently structured (see https://na01.safelinks.protection.outlook.com/?url=ContextRuntime.java&data=01%7c01%7cQiuhe.Wang%40microsoft.com%7c38f6984e74c04f76f9e408d32b676778%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=4xcfRtqd4O7bSk9bqRjrDGghRQEWAPq88QFABupfdLQ%3d), I believe this is the case.
2. Currently, the Java code does not use the variable `services` from `serviceInjector.GetInstance`. I believe it is possible for certain implementations of JVM JIT compilers to optimize away services, so users may end up not having the Services created at all.
If a REEF Service is tied to a Context and is only visible by it and its children Contexts, I believe we should do the following:
1. Create a `Service` interface that extends `AutoCloseable` and expose the method `Start()`.
2. Make `ContextRuntime` implement `AutoCloseable`.
3. Close `Service` only when `ContextRuntime` is closed.
How does this sound? Note that this may require modifications to the `ServiceConfiguration` API we expose now.
Thanks,
Andrew