You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@openwhisk.apache.org by Tyson Norris <tn...@adobe.com.INVALID> on 2017/06/09 23:54:34 UTC
Pluggable components in OpenWhisk
Hi -
I’ve seen the need for portions of OpenWhisk to be more “pluggable”, and seen similar comments in GitHub and dev list.
Some areas I think are worthy of discussion are:
- tracing
- logging
- container management (docker, runc, mesos, swarm, kubernets)
- persistence (couchdb)
- messaging (Kafka) (I haven’t heard any suggestions here, but it wouldn’t surprise me)
To make any progress on anything being pluggable, one approach is to provide a convention for doing the plugging.
Toward this, I put together a simple SPI style wrapper around akka extensions and ServiceLoader, so that:
- its easy to consume, for the app (adding well defined SPIs), and the impl builders (providing additional SPI impls
- its easy (and clean) to configure: no unused jars in your classpath, no guessing at which impl is loaded at runtime, etc
Here is a stripped down example:
https://github.com/tysonnorris/akka-spi
My suggested way forward would be:
- if this approach seems useful, I’ll create a PR to merge the utility classes (see spi-core) into OW master
- once the utilities for this approach are in master, create another PR for a specific proposed SPI, which would 1) separate the SPI to a new directory and 2) provide the default impl of that SPI.
- one an SPI is defined, additional impls can be added without disruption (and enabled optionally in different envs).
- Once there are additional SPI impls available for a specific SPI, it MIGHT become important to define an approach for building classpathes for the OW runtimes where only specific SPI impls arrive on the classpath; I haven’t considered how to do this at this point.
Thanks
Tyson
Re: Pluggable components in OpenWhisk
Posted by Tyson Norris <tn...@adobe.com.INVALID>.
I would phrase things slightly different
"there's no runtime DI framework for Scala out the shelf “ => I don’t think that’s exactly true, some discussion is here: http://letitcrash.com/post/55958814293/akka-dependency-injection
But providing an SPI is different than adopting dependency injection in some ways - SPI is more of a code organization decision vs a binding approach. I am currently more concerned about the code organization, and if we want a more complex binding approach in the future (guice etc), that will certainly be possible. For example, what I proposed with akka-spi does not facilitate dependency injection EXCEPT for the impl choice of a specific SPI. If its not an SPI, it cannot be injected (unless through some other way). I’m not sure if that is a real problem, but I’m interested in feedback to find out.
"need to agree on what gets packaged by default in the docker containers” => this is not really critical, other than streamlining the classpath (ignoring things like duplicate classes with different versions etc), since
- regardless of the classpath, the active impl will have to be explicitly enabled (if you don’t want to use the default).
- a default will always be available with the standard build(s)
in short: no change to existing behavior
We could start with 2 builds of each container:
- includes only default SPI impls
- includes default impls + all alternate impls (each SPI impl enabled via runtime flags e.g. whisk.spi.invoker=whisk.invoker.Reactive)
And additional container variants would have to be custom builds (at least with build flags, or at most a separate gradle script), possibly with customized application.conf (to avoid runtime flags for enabling alternate SPI impls).
I think an example will help, like DefaultInvoker vs InvokerReactive (a theoretical alternate impl to Invoker SPI):
- docker run invoker:latest (runs invoker with default impl DefaultInvoker)
- docker run invoker-all:latest (runs invoker with default impl DefaultInvoker; all impls are on classpath)
- docker run invoker-all:latest -Dwhisk.spi.invoker=whisk.invoker.Reactive (runs invoker with InvokerReactive impl, all impls are on classpath)
- docker run invoker-custom:latest (runs invoker with whatever impl is configured in custom application.conf)
Tyson
On Jun 13, 2017, at 1:31 PM, Dragos Dascalita Haut <dd...@adobe.com.INVALID>> wrote:
"...To make any progress on anything being pluggable, one approach is to provide a convention for doing the plugging...."
So we're basically saying there's no runtime DI framework for Scala out the shelf and we need to come up with one ? It's fine if we do, but it would be great to list our assumptions on the things we're interested to support. i.e. do we want to factor in versioning with semantics for when 2 module versions are compatible, dependency chains, composition, etc.
We also need to agree on what gets packaged by default in the docker containers for the controller and invoker, as this decision can make it easier or harder for users to switch implementations.
Other than that, I +1 Tyson's idea which looks simple, easy to understand, and if it helps us initialize components dynamically, at runtime, it would be great to proceed with it sooner than later.
There's also some thoughts and implementations into the Play Framework [1], but I'm not sure how adaptable those are for OW.
dragos
[1] - https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.playframework.com%2Fdocumentation%2F2.5.x%2FScalaDependencyInjection&data=02%7C01%7C%7C4b5d8175c091499de41e08d4b29b26ab%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636329826823508107&sdata=vRNiiadJluEX0fVcRr1R93XyREdIe522QwFr0LgDdJw%3D&reserved=0
________________________________
From: Rodric Rabbah <ro...@gmail.com>>
Sent: Friday, June 9, 2017 5:45:52 PM
To: dev@openwhisk.apache.org<ma...@openwhisk.apache.org>
Subject: Re: Pluggable components in OpenWhisk
Thanks for taking a shot at this! - at first glance it looks like this may be a good fit. I'll go through it more slowly to provide more informed feedback.
-r
Re: Pluggable components in OpenWhisk
Posted by Dragos Dascalita Haut <dd...@adobe.com.INVALID>.
"...To make any progress on anything being pluggable, one approach is to provide a convention for doing the plugging...."
So we're basically saying there's no runtime DI framework for Scala out the shelf and we need to come up with one ? It's fine if we do, but it would be great to list our assumptions on the things we're interested to support. i.e. do we want to factor in versioning with semantics for when 2 module versions are compatible, dependency chains, composition, etc.
We also need to agree on what gets packaged by default in the docker containers for the controller and invoker, as this decision can make it easier or harder for users to switch implementations.
Other than that, I +1 Tyson's idea which looks simple, easy to understand, and if it helps us initialize components dynamically, at runtime, it would be great to proceed with it sooner than later.
There's also some thoughts and implementations into the Play Framework [1], but I'm not sure how adaptable those are for OW.
dragos
[1] - https://www.playframework.com/documentation/2.5.x/ScalaDependencyInjection
________________________________
From: Rodric Rabbah <ro...@gmail.com>
Sent: Friday, June 9, 2017 5:45:52 PM
To: dev@openwhisk.apache.org
Subject: Re: Pluggable components in OpenWhisk
Thanks for taking a shot at this! - at first glance it looks like this may be a good fit. I'll go through it more slowly to provide more informed feedback.
-r
Re: Pluggable components in OpenWhisk
Posted by Rodric Rabbah <ro...@gmail.com>.
Thanks for taking a shot at this! - at first glance it looks like this may be a good fit. I'll go through it more slowly to provide more informed feedback.
-r
Re: Pluggable components in OpenWhisk
Posted by Tyson Norris <tn...@adobe.com.INVALID>.
Following up on this thread, I just created this PR to propose an SPI approach for making OW internal components pluggable, with examples of db + messaging (using existing impls as defaults so no change in functionality). This does NOT address any potential class path changes to include additional/alternate impls, it is ONLY the support for loading the impls, so that classpath can be changed in the future. Some description of the model and discussion of classpath modification options is included in the docs/spi.md file.
https://github.com/apache/incubator-openwhisk/pull/2414
It has evolved in various ways from my original proposal, but hopefully a usable example will be a better demonstration of the approach.
Thanks in advance for feedback.
Tyson
On Jun 14, 2017, at 6:43 AM, Rodric Rabbah <ro...@gmail.com>> wrote:
I saw that class and I just wanted to check if a contribution would be
accepted eventually.
We are generally tending toward a model where the components are all
deployment time configurable so that one can tailor the architecture to use
or experiment with different technology. The datastore has an abstract
interface, the message bus producer and consumer also has an abstract
interface, we recently refactor the interface to the container pool
management, and there's ongoing discussion about logging and tracing in the
same way. These abstractions were intended to allow for experimentation and
with the right model for plugins, I think we can address the increasing
interest we are observing in making the architecture adaptable in several
ways.
On the message bus, Kafka makes sense because it offers low latency
messaging and offers persistence of the activation requests. That said, I
expect this is will be an area of continued interest as it has implication
for the load balancing and scheduling.
-r
Re: Pluggable components in OpenWhisk
Posted by Rodric Rabbah <ro...@gmail.com>.
> I saw that class and I just wanted to check if a contribution would be
accepted eventually.
We are generally tending toward a model where the components are all
deployment time configurable so that one can tailor the architecture to use
or experiment with different technology. The datastore has an abstract
interface, the message bus producer and consumer also has an abstract
interface, we recently refactor the interface to the container pool
management, and there's ongoing discussion about logging and tracing in the
same way. These abstractions were intended to allow for experimentation and
with the right model for plugins, I think we can address the increasing
interest we are observing in making the architecture adaptable in several
ways.
On the message bus, Kafka makes sense because it offers low latency
messaging and offers persistence of the activation requests. That said, I
expect this is will be an area of continued interest as it has implication
for the load balancing and scheduling.
-r
Re: Pluggable components in OpenWhisk
Posted by Clebert Suconic <cl...@gmail.com>.
I haven't. I saw that class and I just wanted to check if a contribution
would be accepted eventually.
I think this would make sense. Think not only of other messaging providers
but the possibility of integrating with existing messaging solutions on
cloud.
On Wed, Jun 14, 2017 at 12:51 AM Tyson Norris <tn...@adobe.com.invalid>
wrote:
> I haven’t heard anyone ask for this, but there is no reason it should not
> be possible to extract an interface for messaging; in fact it is mostly
> there already (see whisk.core.connector.MessageProducer), except for the
> binding aspect. Have you already implemented an alternate MessageProducer
> impl?
>
>
> > On Jun 13, 2017, at 6:45 PM, Clebert Suconic <cl...@gmail.com>
> wrote:
> >
> > On Fri, Jun 9, 2017 at 7:54 PM, Tyson Norris <tn...@adobe.com.invalid>
> wrote:
> >> Hi -
> >> I’ve seen the need for portions of OpenWhisk to be more “pluggable”,
> and seen similar comments in GitHub and dev list.
> >>
> >> Some areas I think are worthy of discussion are:
> >> - tracing
> >> - logging
> >> - container management (docker, runc, mesos, swarm, kubernets)
> >> - persistence (couchdb)
> >> - messaging (Kafka) (I haven’t heard any suggestions here, but it
> wouldn’t surprise me)
> >
> >
> > Any plans to make messaging pluggable to other providers beyond Kafka?
>
> --
Clebert Suconic
Re: Pluggable components in OpenWhisk
Posted by Tyson Norris <tn...@adobe.com.INVALID>.
I haven’t heard anyone ask for this, but there is no reason it should not be possible to extract an interface for messaging; in fact it is mostly there already (see whisk.core.connector.MessageProducer), except for the binding aspect. Have you already implemented an alternate MessageProducer impl?
> On Jun 13, 2017, at 6:45 PM, Clebert Suconic <cl...@gmail.com> wrote:
>
> On Fri, Jun 9, 2017 at 7:54 PM, Tyson Norris <tn...@adobe.com.invalid> wrote:
>> Hi -
>> I’ve seen the need for portions of OpenWhisk to be more “pluggable”, and seen similar comments in GitHub and dev list.
>>
>> Some areas I think are worthy of discussion are:
>> - tracing
>> - logging
>> - container management (docker, runc, mesos, swarm, kubernets)
>> - persistence (couchdb)
>> - messaging (Kafka) (I haven’t heard any suggestions here, but it wouldn’t surprise me)
>
>
> Any plans to make messaging pluggable to other providers beyond Kafka?
Re: Pluggable components in OpenWhisk
Posted by Clebert Suconic <cl...@gmail.com>.
On Fri, Jun 9, 2017 at 7:54 PM, Tyson Norris <tn...@adobe.com.invalid> wrote:
> Hi -
> I’ve seen the need for portions of OpenWhisk to be more “pluggable”, and seen similar comments in GitHub and dev list.
>
> Some areas I think are worthy of discussion are:
> - tracing
> - logging
> - container management (docker, runc, mesos, swarm, kubernets)
> - persistence (couchdb)
> - messaging (Kafka) (I haven’t heard any suggestions here, but it wouldn’t surprise me)
Any plans to make messaging pluggable to other providers beyond Kafka?