You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@openwhisk.apache.org by Martin Henke <ma...@web.de> on 2019/07/03 14:29:22 UTC
Re: A plan to (re) implement OpenWhisk on top of Knative
Michele,
FYI: Sugandha and myself published a Medium blog article which describes to build Nodejs10 images using Tekton and run it on Knative
(based on the work from Priti).
It might be on interest in the context of your Actionloop related Knative work.
Link:
https://medium.com/@sugandha.agrawal18/build-knative-service-with-tekton-and-apache-openwhisk-node-js-runtime-f660bbc3a11e
Regards,
Martin
On 2019/05/20 15:00:02, "Michele Sciabarra" <m....@sciabarra.com> wrote:
> Ok great, I see the discussion is starting to bring ideas.>
>
> Yes my goal is basically to run existing actions in Knative, create and invoke. And possibile retain the ability of an action to invoke another action.>
>
> I understand the different way they expose services, so I am rethinking the idea of using a "work-alike" path. >
>
> If it is needed we can add it with an ingress but it may be not necessary in the initial implementation.>
>
> Also I checked a bit ML and discussions and I see this Tekton thing that should be the preferred way.>
>
> Not sure if I understand the relation with the current Build API documented in the website. Is Tekton "compatible" or it has a different API?>
>
>
> -- >
> Michele Sciabarra>
> michele@sciabarra.com>
>
> ----- Original message ----->
> From: "Markus Thömmes" <ma...@apache.org>>
> To: dev@openwhisk.apache.org>
> Subject: Re: A plan to (re) implement OpenWhisk on top of Knative>
> Date: Monday, May 20, 2019 4:50 PM>
>
> Good discussion, thanks!>
>
> Can we try to define what the desired end-goal is here? I'm a bit unclear>
> what resembling the OpenWhisk API actually buys us.>
>
> To me, the desired end-state would be to run OpenWhisk actions as-is on a>
> Knative cluster (similar to OpenFaaS' and Azure's integration). There's no>
> good way for us to provide the full API without spinning up a control plane>
> and we can only handle so much via the CLI. So to me, the end-goal looks>
> like:>
>
> 1. *wsk action create* actually doing all the pieces necessary to run a>
> piece of code on Knative.>
> 2. *wsk action invoke* doing some HTTP call under the hood to "invoke" that>
> action. The action should be reachable via a sensible URL. If we really>
> want to keep the API surface (as I said, I'm dubious here) we can also do>
> that via ingress level abstractions (like VirtualService).>
>
> Cheers,>
> Markus>
>
> Am Mo., 20. Mai 2019 um 15:33 Uhr schrieb Martin Henke <ma...@web.de>
> >:>
>
> >>
> > > On 20. May 2019, at 14:55, Michele Sciabarra <mi...@sciabarra.com>>
> > wrote:>
> > >>
> > >> Michele,>
> > >>
> > >> I like the idea to make the ActionLoop based runtimes to be runnable on>
> > Knative.>
> > >>>
> > >> My thoughts on this:>
> > >> - I second Markus concern to implement the invocation API onto Knative>
> > instead of just using Knative service syntax.>
> > > Can you elaborate this? I do not understand.>
> >>
> > Knative service syntax: https://<service(in our case =>
> > action)>.<namespace>.<host>/>
> > OW invocation https://>
> > <api-host>/api/v1/namespaces/<namespace>/actions/<action>>
> >>
> > (I personally so no worth in inventing a distinct API for OW images, but>
> > as said I would see that as a valid optional feature)>
> >>
> > >>
> > >> - I would have concerns to make it dependent on Gloo which is kind of a>
> > minority choice for Knative load balancing>
> > > I do not think it will be hard to setup a test also using Istio, I do>
> > not want to be limited to Gloo.>
> >>
> > I just wanted to prevent that Gloo gets a “official” prerequisite for an>
> > “official” OW on Knative flow.>
> > It is of course free to you to use what ever you want to do in your>
> > prototype.>
> >>
> > > - In my opinion the goal should be to have some uniform behaviour for>
> > ActionLoop based runtimes>
> > >> and other ones like the adapted NodeJS runtimes demonstrated by Matt>
> > and Priti>
> > > As much as I can tell the current implementation is just the building>
> > and exposing the "/init" and "/run" but I can be wrong.>
> > > The build can be of course reused, so it continues the effort. For the>
> > frontend, from the documentation I think Matt wants to add a proxy, while I>
> > would like to implemeent the "invocation" straight in the runtime. This is>
> > open to discussion, but of course it is better to reach an agreement.>
> >>
> > Also in the work of Priti and Matt the invocation goes directly to the>
> > runtime. The action code is either passed with the call (not yet tested by>
> > me) or set via environment variable in the docker build.>
> >>
> > >>
> > >> - As Knative Build seems be on a dead end I would propose to target>
> > Tekton as the build system (which developed as kind of >successor out of>
> > Knative)>
> > >>
> > > If Knative build is dead then it would be a bit unfair that they change>
> > it as the scope of the Knative project!>
> > > It looks like the goal is to setup some standards! And I would be very>
> > disappointed to know that.>
> >>
> > Tekton evolved out of Knative Build (or more correct out of Knative>
> > Pipelines) but is very similar to the Knative build.>
> > Flows can easily be ported from one to the other,>
> > If we target Tekton build we would target the platform were the Knative>
> > build team is focusing on.>
> > But again feel free to use whatever platform for your prototype work.>
> >>
> > > At this stage the build is the more interesting thing, and it could be>
> > even imported in main openwhisk to speed up deployment.>
> > > I have already baked it in the ActionLoop runtimes (with precompilation).>
> > > Also if we use Tekton, where is the Knative standard then? What is the>
> > point? We can build our own system instead of "Knativizing" it...>
> > >>
> > >> Maybe it would be a good solution to tackle two things independently.>
> > >> 1) Design and implement a common protocol of building, running and>
> > calling OW runtimes on Knative>
> > >> 2) Implement the OW invocation API on top of Knative as an additional>
> > option for those who have the need to expose it.>
> > >>
> > > On this, for my personal approach at building things, I want something>
> > that works and it is complete and useful. A "MVP”.>
> >>
> > Cool. Just go on.>
> >>
> > > So I do not plan to split the effort. Version 0.1 must be a minimal>
> > working subset of OpenWhisk on Knative.>
> > > Because otherwise there will be incomplete useless inusable pieces>
> > around (see for example kwsk).>
> > >>
> > > It does not mean that things cannot be modular, nor that everyone must>
> > but to me "openwhisk-knative" must be a single repo with all the pieces to>
> > make something where you can download is and deploy in a kubernetes cluster>
> > and be able to deploy simple actions. When this works, we can improve>
> > incrementally and split it but keeping it working.>
> > >>
> > >> I would looking forward to work with you on the first work item.>
> > > Great but I see now more details to discuss before we can start. Most>
> > notably I need to understand how I can build on top of Mark and Priti work>
> > and continue their work. ANd I can even probably recover some of the code>
> > of kwsk as they implemented some openwhisk api, that I want now in the>
> > runtime.>
> > >>
> >>
> > I do not want to stop you in any way. My hope is that the action loop>
> > runtimes and the “other ones” do expose the same behaviour when being>
> > called. So that the users is not surprised when calling different actions>
> > in different languages.>
> > And behaving the same way might also mean to adapt the “other languages”>
> > to the same behaviour as the action loop based ones.>
> > They just should be uniform to be used.>
> >>
> > When your prototype is accessible it would be a good point of time to>
> > discuss this.>
> >>
> > As said I very much like your effort.>
> >>
> > >>
> > >> On 20. May 2019, at 08:55, Michele Sciabarra <mi...@sciabarra.com>>
> > wrote:>
> > >>>
> > >>>
> > >>>> I have an idea for implementing a prototype of OpenWhisk on top of>
> > Knative.>
> > >>>> My basic ideas are: do not use any proxy, forwarding or adapter:>
> > extend>
> > >>>> the runtime to support the REST call and expose them as ingress. And>
> > use a>
> > >>>> wrapper on top of `kubectl` to generate all the needed components.>
> > >>>
> > >>> Does this tie into the work that Matt was doing to the runtimes to make>
> > >>> them runnable on Knative? Is this lined up with that at all?>
> > >> Actually yes. He suggested I can investigate how to migrate ActionLoop>
> > to port many other languages to Knative.>
> > >> Also he recommended I add my idea and this is what I am doing. Current>
> > code is, if I am not wrong, a Knative build of the nodejs runtime.>
> > >>>
> > >> There has been a number of attempts and proposal to move forward>
> > OpenWhisk. My idea that to succeed we need something small but that just>
> > works. This is my idea to be able to implement in the shorter time frame>
> > possible an actual subset of OpenWhisk that works and it is truly built on>
> > top of Knative. So I am putting the thing a bit further than Matt work.>
> > >>>
> > >>>
> > >>>> My goal is to have a functional work-alike of OpenWhisk built on top>
> > of>
> > >>>> Knative, using ActionLoop as a foundation. I will extend ActionLoop to>
> > >>>> support the required REST calls of OpenWhisk.>
> > >>>>
> > >>>> I also want to create tool, I will call `wskn`. This tool will>
> > initially>
> > >>>> just a python script, a wrapper on top of `kubectl` as it will>
> > generate>
> > >>>> kubernetes descriptors.>
> > >>> Why not build this into "wsk" itself? The Azure Functions CLI as an>
> > example>
> > >>> supports multiple deployment types like this in one CLI.>
> > >>>
> > >> When it will works, yes, of course. But to start, what I really need is>
> > a prototype that can generate kubernetes descripttors to feed to kubectl,>
> > so a simplee, quick and ditry, separate tool (that I will keep together>
> > the runtime) is all I need for now.>
> > >>>
> > >>>>>
> > >>>> It will support initially just the the action creation and>
> > invocation, and>
> > >>>> only synchronous (blocking) behaviour, as all the request will go>
> > straight>
> > >>>> to the runtimes. Hopefully also a subset of `package` and>
> > `activation`.>
> > >>>> Again triggers, rules, asynchronous for later.>
> > >>>>>
> > >>>> The idea is that you will be able to create actions and web actions>
> > that>
> > >>>> can run existing OpenWhisk actions, at least those with blocking>
> > behaviour>
> > >>>> that run with ActionLoop (Go, Java, Python, PHP, Swift, Rust, Ruby,>
> > >>>> Crystal...)>
> > >>>>>
> > >>>> Implementation.>
> > >>>> ==============>
> > >>>>>
> > >>>> This is how I plan to implement it.>
> > >>>>>
> > >>>> At this stage I want to use just Knative Serving and Knative Build,>
> > using>
> > >>>> Gloo for the ingress part. I also plan to install a local Docker>
> > registry>
> > >>>> Kubernetes registry, so we do not have to use DockerHub for>
> > everything. All>
> > >>>> of this can be done with existing command line tools in a few minutes>
> > in>
> > >>>> any running Kubernetes deployment.>
> > >>>>>
> > >>>
> > >>> Why specifying Gloo here? Do you need anything specific from Gloo>
> > itself?>
> > >>> If not I'd propose to just keep it on a Knative Serving API surface>
> > level.>
> > >> I want to build it on top of Knative serving, full stop. Currently,>
> > installing Gloo is pretty easy and is more lightweight than Istio so I>
> > will use it for my first implementation.>
> > >>>
> > >>>>>
> > >>>> When I create an action, it will use Knative build that will work>
> > roughly>
> > >>>> in this way:>
> > >>>>>
> > >>>> - create a configmap with the action code>
> > >>>> - build the actin using ActionLoop precompilation feature that will>
> > return>
> > >>>> a zip file including all the needed to run the action>
> > >>>> - create a new docker image extending the runtime with the new zip,>
> > using>
> > >>>> Kaanico>
> > >>>> - push the image in the local registry>
> > >>> This feels like a fairly heavyweight process, we should be able to>
> > come up>
> > >>> with a way to circumvent zipping entirely. Maybe the runtime can detect>
> > >>> that the unzipped content is already there and skip the unzip step?>
> > >>>
> > >> Actually this is my first idea of how to use Knative build. And is not>
> > complicated: when I create the action, a run a build that includes Kanico.>
> > I generate a Dockerfile on the fly. The docker file uses the action runtime>
> > that already know how to compile a script. And then I save an image. I>
> > already implemented un "autoinit" so just launching the image will give a>
> > runtime ready to run that execute an action already compiled.>
> > >>>
> > >>>
> > >>> I'm fairly hesitant on the usage of a ConfigMap for storing the action>
> > >>> code. It's all stored in the in-cluster etcd instance and it has a>
> > limit of>
> > >>> 1M. This is at most a stop-gap solution to provide a PoC I think. Any>
> > ideas>
> > >>> on how to "productize" this?>
> > >>>
> > >> ConfigMap can be mounted as files, so it is an easy way to feed an>
> > action to a build. It is just an easy way to feed the action code to the>
> > Build.>
> > >>>
> > >> My initial constraint is that I want just to generate Kubernetes>
> > descriptors to feed to kubectl.>
> > >> Of course in the long run I can add some "file upload" storage.>
> > >>>
> > >> If I could to this file upload when invoking a build it could ideal as>
> > I do not have to store anything anywhere, just process the code and>
> > generate a single layer to execute actions to be store in the registry.>
> > >> I will investigate better this area, I understand your concern.>
> > >>>
> > >>>>
> > >>>> At this point you can run the action. ActionLoop will be extended to>
> > >>>> support invocations in the format>
> > >>>> "/v1/namespaces/namespace/actions/package/action".>
> > >>> Why bother reimplementing this exact path? To obtain API compatibility>
> > with>
> > >>> OpenWhisk as it is today?>
> > >>>
> > >> I want to implement a subset of the OpenWhisk API on top of Knative>
> > serving.>
> > >> Knative serving already does the scaling and routing, so what we need>
> > are the "endpoints" to invoke actions.>
> > >>>
> > >> Since I do not want to add additional components, not at the first>
> > stage. Just knative serve and build, the runtime and a controller script,>
> > the runtime is the natural place where to "handle" the API invocations,>
> > since Knative only generates the URL but not anything else. If I>
> > understood well, Matt is adding a proxy. I do not want to add a proxy, just>
> > add to the runtime the ability to respond to "API like" calls, at least>
> > those regarding action invocation.>
> > >>>
> > >>>> It will do all the decoding required to invoke the action with the>
> > >>>> expected paramenters (straight invocation thrhoug the actinloop>
> > protocol,>
> > >>>> not proxies).>
> > >>> Does this mean moving all of the Controller's "smartness" about>
> > incoming>
> > >>> and outgoing HTTP requests (see the whole WebActions for example)?>
> > >>>
> > >> At least decoding web actions in the runtime, yes. Knative serving>
> > already has routing and proxying.>
> > >> So a true implementation on top of Knative requires IHMO this>
> > sacrifice. Unless there is a way to keep the controller in a "Knative">
> > compatible way. Open to suggestions here.>
> > >>>
> > >>> Each action will then be exposed using an ingress with its specific>
> > >>> invocation path.>
> > >>>>
> > >>> If the community agrees with this plan, I would create a repo>
> > >>> `incubator-openwhisk-knative` to work on it.>
> > >>>>
> > >>> Thoughts?>
> >>
> >>
>
Re: A plan to (re) implement OpenWhisk on top of Knative
Posted by Sugandha Agrawal <su...@gmail.com>.
Cool!!!
Regards
Sugandha Agrawal
On Wed, Jul 3, 2019 at 4:37 PM Michele Sciabarra <mi...@sciabarra.com>
wrote:
> Thanks! Indeed it was my next step since I am now trying to build actions
> with the actionloop based runtime with tekton. I thought to use Priti work
> that has to be updated to use Tekton so your article is very welcome and
> useful.
>
> --
> Michele Sciabarra
> michele@sciabarra.com
>
> ----- Original message -----
> From: Martin Henke <ma...@web.de>
> To: dev@openwhisk.apache.org
> Subject: Re: A plan to (re) implement OpenWhisk on top of Knative
> Date: Wednesday, July 03, 2019 4:29 PM
>
> Michele,
>
> FYI: Sugandha and myself published a Medium blog article which describes
> to build Nodejs10 images using Tekton and run it on Knative
> (based on the work from Priti).
> It might be on interest in the context of your Actionloop related Knative
> work.
>
> Link:
>
> https://medium.com/@sugandha.agrawal18/build-knative-service-with-tekton-and-apache-openwhisk-node-js-runtime-f660bbc3a11e
>
> Regards,
> Martin
>
>
> On 2019/05/20 15:00:02, "Michele Sciabarra" <m....@sciabarra.com> wrote:
> > Ok great, I see the discussion is starting to bring ideas.>
> >
> > Yes my goal is basically to run existing actions in Knative, create and
> invoke. And possibile retain the ability of an action to invoke another
> action.>
> >
> > I understand the different way they expose services, so I am rethinking
> the idea of using a "work-alike" path. >
> >
> > If it is needed we can add it with an ingress but it may be not
> necessary in the initial implementation.>
> >
> > Also I checked a bit ML and discussions and I see this Tekton thing that
> should be the preferred way.>
> >
> > Not sure if I understand the relation with the current Build API
> documented in the website. Is Tekton "compatible" or it has a different
> API?>
> >
> >
> > -- >
> > Michele Sciabarra>
> > michele@sciabarra.com>
> >
> > ----- Original message ----->
> > From: "Markus Thömmes" <ma...@apache.org>>
> > To: dev@openwhisk.apache.org>
> > Subject: Re: A plan to (re) implement OpenWhisk on top of Knative>
> > Date: Monday, May 20, 2019 4:50 PM>
> >
> > Good discussion, thanks!>
> >
> > Can we try to define what the desired end-goal is here? I'm a bit
> unclear>
> > what resembling the OpenWhisk API actually buys us.>
> >
> > To me, the desired end-state would be to run OpenWhisk actions as-is on
> a>
> > Knative cluster (similar to OpenFaaS' and Azure's integration). There's
> no>
> > good way for us to provide the full API without spinning up a control
> plane>
> > and we can only handle so much via the CLI. So to me, the end-goal
> looks>
> > like:>
> >
> > 1. *wsk action create* actually doing all the pieces necessary to run a>
> > piece of code on Knative.>
> > 2. *wsk action invoke* doing some HTTP call under the hood to "invoke"
> that>
> > action. The action should be reachable via a sensible URL. If we really>
> > want to keep the API surface (as I said, I'm dubious here) we can also
> do>
> > that via ingress level abstractions (like VirtualService).>
> >
> > Cheers,>
> > Markus>
> >
> > Am Mo., 20. Mai 2019 um 15:33 Uhr schrieb Martin Henke <
> martin.henke@web.de>
> > >:>
> >
> > >>
> > > > On 20. May 2019, at 14:55, Michele Sciabarra <mi...@sciabarra.com>>
> > > wrote:>
> > > >>
> > > >> Michele,>
> > > >>
> > > >> I like the idea to make the ActionLoop based runtimes to be
> runnable on>
> > > Knative.>
> > > >>>
> > > >> My thoughts on this:>
> > > >> - I second Markus concern to implement the invocation API onto
> Knative>
> > > instead of just using Knative service syntax.>
> > > > Can you elaborate this? I do not understand.>
> > >>
> > > Knative service syntax: https://<service(in our case =>
> > > action)>.<namespace>.<host>/>
> > > OW invocation https://>
> > > <api-host>/api/v1/namespaces/<namespace>/actions/<action>>
> > >>
> > > (I personally so no worth in inventing a distinct API for OW images,
> but>
> > > as said I would see that as a valid optional feature)>
> > >>
> > > >>
> > > >> - I would have concerns to make it dependent on Gloo which is kind
> of a>
> > > minority choice for Knative load balancing>
> > > > I do not think it will be hard to setup a test also using Istio, I
> do>
> > > not want to be limited to Gloo.>
> > >>
> > > I just wanted to prevent that Gloo gets a “official” prerequisite for
> an>
> > > “official” OW on Knative flow.>
> > > It is of course free to you to use what ever you want to do in your>
> > > prototype.>
> > >>
> > > > - In my opinion the goal should be to have some uniform behaviour
> for>
> > > ActionLoop based runtimes>
> > > >> and other ones like the adapted NodeJS runtimes demonstrated by
> Matt>
> > > and Priti>
> > > > As much as I can tell the current implementation is just the
> building>
> > > and exposing the "/init" and "/run" but I can be wrong.>
> > > > The build can be of course reused, so it continues the effort. For
> the>
> > > frontend, from the documentation I think Matt wants to add a proxy,
> while I>
> > > would like to implemeent the "invocation" straight in the runtime.
> This is>
> > > open to discussion, but of course it is better to reach an agreement.>
> > >>
> > > Also in the work of Priti and Matt the invocation goes directly to
> the>
> > > runtime. The action code is either passed with the call (not yet
> tested by>
> > > me) or set via environment variable in the docker build.>
> > >>
> > > >>
> > > >> - As Knative Build seems be on a dead end I would propose to
> target>
> > > Tekton as the build system (which developed as kind of >successor out
> of>
> > > Knative)>
> > > >>
> > > > If Knative build is dead then it would be a bit unfair that they
> change>
> > > it as the scope of the Knative project!>
> > > > It looks like the goal is to setup some standards! And I would be
> very>
> > > disappointed to know that.>
> > >>
> > > Tekton evolved out of Knative Build (or more correct out of Knative>
> > > Pipelines) but is very similar to the Knative build.>
> > > Flows can easily be ported from one to the other,>
> > > If we target Tekton build we would target the platform were the
> Knative>
> > > build team is focusing on.>
> > > But again feel free to use whatever platform for your prototype work.>
> > >>
> > > > At this stage the build is the more interesting thing, and it could
> be>
> > > even imported in main openwhisk to speed up deployment.>
> > > > I have already baked it in the ActionLoop runtimes (with
> precompilation).>
> > > > Also if we use Tekton, where is the Knative standard then? What is
> the>
> > > point? We can build our own system instead of "Knativizing" it...>
> > > >>
> > > >> Maybe it would be a good solution to tackle two things
> independently.>
> > > >> 1) Design and implement a common protocol of building, running and>
> > > calling OW runtimes on Knative>
> > > >> 2) Implement the OW invocation API on top of Knative as an
> additional>
> > > option for those who have the need to expose it.>
> > > >>
> > > > On this, for my personal approach at building things, I want
> something>
> > > that works and it is complete and useful. A "MVP”.>
> > >>
> > > Cool. Just go on.>
> > >>
> > > > So I do not plan to split the effort. Version 0.1 must be a minimal>
> > > working subset of OpenWhisk on Knative.>
> > > > Because otherwise there will be incomplete useless inusable pieces>
> > > around (see for example kwsk).>
> > > >>
> > > > It does not mean that things cannot be modular, nor that everyone
> must>
> > > but to me "openwhisk-knative" must be a single repo with all the
> pieces to>
> > > make something where you can download is and deploy in a kubernetes
> cluster>
> > > and be able to deploy simple actions. When this works, we can improve>
> > > incrementally and split it but keeping it working.>
> > > >>
> > > >> I would looking forward to work with you on the first work item.>
> > > > Great but I see now more details to discuss before we can start.
> Most>
> > > notably I need to understand how I can build on top of Mark and Priti
> work>
> > > and continue their work. ANd I can even probably recover some of the
> code>
> > > of kwsk as they implemented some openwhisk api, that I want now in
> the>
> > > runtime.>
> > > >>
> > >>
> > > I do not want to stop you in any way. My hope is that the action loop>
> > > runtimes and the “other ones” do expose the same behaviour when being>
> > > called. So that the users is not surprised when calling different
> actions>
> > > in different languages.>
> > > And behaving the same way might also mean to adapt the “other
> languages”>
> > > to the same behaviour as the action loop based ones.>
> > > They just should be uniform to be used.>
> > >>
> > > When your prototype is accessible it would be a good point of time to>
> > > discuss this.>
> > >>
> > > As said I very much like your effort.>
> > >>
> > > >>
> > > >> On 20. May 2019, at 08:55, Michele Sciabarra <mi...@sciabarra.com>>
>
> > > wrote:>
> > > >>>
> > > >>>
> > > >>>> I have an idea for implementing a prototype of OpenWhisk on top
> of>
> > > Knative.>
> > > >>>> My basic ideas are: do not use any proxy, forwarding or adapter:>
> > > extend>
> > > >>>> the runtime to support the REST call and expose them as ingress.
> And>
> > > use a>
> > > >>>> wrapper on top of `kubectl` to generate all the needed
> components.>
> > > >>>
> > > >>> Does this tie into the work that Matt was doing to the runtimes to
> make>
> > > >>> them runnable on Knative? Is this lined up with that at all?>
> > > >> Actually yes. He suggested I can investigate how to migrate
> ActionLoop>
> > > to port many other languages to Knative.>
> > > >> Also he recommended I add my idea and this is what I am doing.
> Current>
> > > code is, if I am not wrong, a Knative build of the nodejs runtime.>
> > > >>>
> > > >> There has been a number of attempts and proposal to move forward>
> > > OpenWhisk. My idea that to succeed we need something small but that
> just>
> > > works. This is my idea to be able to implement in the shorter time
> frame>
> > > possible an actual subset of OpenWhisk that works and it is truly
> built on>
> > > top of Knative. So I am putting the thing a bit further than Matt
> work.>
> > > >>>
> > > >>>
> > > >>>> My goal is to have a functional work-alike of OpenWhisk built on
> top>
> > > of>
> > > >>>> Knative, using ActionLoop as a foundation. I will extend
> ActionLoop to>
> > > >>>> support the required REST calls of OpenWhisk.>
> > > >>>>
> > > >>>> I also want to create tool, I will call `wskn`. This tool will>
> > > initially>
> > > >>>> just a python script, a wrapper on top of `kubectl` as it will>
> > > generate>
> > > >>>> kubernetes descriptors.>
> > > >>> Why not build this into "wsk" itself? The Azure Functions CLI as
> an>
> > > example>
> > > >>> supports multiple deployment types like this in one CLI.>
> > > >>>
> > > >> When it will works, yes, of course. But to start, what I really
> need is>
> > > a prototype that can generate kubernetes descripttors to feed to
> kubectl,>
> > > so a simplee, quick and ditry, separate tool (that I will keep
> together>
> > > the runtime) is all I need for now.>
> > > >>>
> > > >>>>>
> > > >>>> It will support initially just the the action creation and>
> > > invocation, and>
> > > >>>> only synchronous (blocking) behaviour, as all the request will
> go>
> > > straight>
> > > >>>> to the runtimes. Hopefully also a subset of `package` and>
> > > `activation`.>
> > > >>>> Again triggers, rules, asynchronous for later.>
> > > >>>>>
> > > >>>> The idea is that you will be able to create actions and web
> actions>
> > > that>
> > > >>>> can run existing OpenWhisk actions, at least those with blocking>
> > > behaviour>
> > > >>>> that run with ActionLoop (Go, Java, Python, PHP, Swift, Rust,
> Ruby,>
> > > >>>> Crystal...)>
> > > >>>>>
> > > >>>> Implementation.>
> > > >>>> ==============>
> > > >>>>>
> > > >>>> This is how I plan to implement it.>
> > > >>>>>
> > > >>>> At this stage I want to use just Knative Serving and Knative
> Build,>
> > > using>
> > > >>>> Gloo for the ingress part. I also plan to install a local Docker>
> > > registry>
> > > >>>> Kubernetes registry, so we do not have to use DockerHub for>
> > > everything. All>
> > > >>>> of this can be done with existing command line tools in a few
> minutes>
> > > in>
> > > >>>> any running Kubernetes deployment.>
> > > >>>>>
> > > >>>
> > > >>> Why specifying Gloo here? Do you need anything specific from Gloo>
> > > itself?>
> > > >>> If not I'd propose to just keep it on a Knative Serving API
> surface>
> > > level.>
> > > >> I want to build it on top of Knative serving, full stop.
> Currently,>
> > > installing Gloo is pretty easy and is more lightweight than Istio so
> I>
> > > will use it for my first implementation.>
> > > >>>
> > > >>>>>
> > > >>>> When I create an action, it will use Knative build that will
> work>
> > > roughly>
> > > >>>> in this way:>
> > > >>>>>
> > > >>>> - create a configmap with the action code>
> > > >>>> - build the actin using ActionLoop precompilation feature that
> will>
> > > return>
> > > >>>> a zip file including all the needed to run the action>
> > > >>>> - create a new docker image extending the runtime with the new
> zip,>
> > > using>
> > > >>>> Kaanico>
> > > >>>> - push the image in the local registry>
> > > >>> This feels like a fairly heavyweight process, we should be able
> to>
> > > come up>
> > > >>> with a way to circumvent zipping entirely. Maybe the runtime can
> detect>
> > > >>> that the unzipped content is already there and skip the unzip
> step?>
> > > >>>
> > > >> Actually this is my first idea of how to use Knative build. And is
> not>
> > > complicated: when I create the action, a run a build that includes
> Kanico.>
> > > I generate a Dockerfile on the fly. The docker file uses the action
> runtime>
> > > that already know how to compile a script. And then I save an image.
> I>
> > > already implemented un "autoinit" so just launching the image will
> give a>
> > > runtime ready to run that execute an action already compiled.>
> > > >>>
> > > >>>
> > > >>> I'm fairly hesitant on the usage of a ConfigMap for storing the
> action>
> > > >>> code. It's all stored in the in-cluster etcd instance and it has
> a>
> > > limit of>
> > > >>> 1M. This is at most a stop-gap solution to provide a PoC I think.
> Any>
> > > ideas>
> > > >>> on how to "productize" this?>
> > > >>>
> > > >> ConfigMap can be mounted as files, so it is an easy way to feed
> an>
> > > action to a build. It is just an easy way to feed the action code to
> the>
> > > Build.>
> > > >>>
> > > >> My initial constraint is that I want just to generate Kubernetes>
> > > descriptors to feed to kubectl.>
> > > >> Of course in the long run I can add some "file upload" storage.>
> > > >>>
> > > >> If I could to this file upload when invoking a build it could ideal
> as>
> > > I do not have to store anything anywhere, just process the code and>
> > > generate a single layer to execute actions to be store in the
> registry.>
> > > >> I will investigate better this area, I understand your concern.>
> > > >>>
> > > >>>>
> > > >>>> At this point you can run the action. ActionLoop will be extended
> to>
> > > >>>> support invocations in the format>
> > > >>>> "/v1/namespaces/namespace/actions/package/action".>
> > > >>> Why bother reimplementing this exact path? To obtain API
> compatibility>
> > > with>
> > > >>> OpenWhisk as it is today?>
> > > >>>
> > > >> I want to implement a subset of the OpenWhisk API on top of
> Knative>
> > > serving.>
> > > >> Knative serving already does the scaling and routing, so what we
> need>
> > > are the "endpoints" to invoke actions.>
> > > >>>
> > > >> Since I do not want to add additional components, not at the first>
> > > stage. Just knative serve and build, the runtime and a controller
> script,>
> > > the runtime is the natural place where to "handle" the API
> invocations,>
> > > since Knative only generates the URL but not anything else. If I>
> > > understood well, Matt is adding a proxy. I do not want to add a proxy,
> just>
> > > add to the runtime the ability to respond to "API like" calls, at
> least>
> > > those regarding action invocation.>
> > > >>>
> > > >>>> It will do all the decoding required to invoke the action with
> the>
> > > >>>> expected paramenters (straight invocation thrhoug the actinloop>
> > > protocol,>
> > > >>>> not proxies).>
> > > >>> Does this mean moving all of the Controller's "smartness" about>
> > > incoming>
> > > >>> and outgoing HTTP requests (see the whole WebActions for
> example)?>
> > > >>>
> > > >> At least decoding web actions in the runtime, yes. Knative serving>
> > > already has routing and proxying.>
> > > >> So a true implementation on top of Knative requires IHMO this>
> > > sacrifice. Unless there is a way to keep the controller in a
> "Knative">
> > > compatible way. Open to suggestions here.>
> > > >>>
> > > >>> Each action will then be exposed using an ingress with its
> specific>
> > > >>> invocation path.>
> > > >>>>
> > > >>> If the community agrees with this plan, I would create a repo>
> > > >>> `incubator-openwhisk-knative` to work on it.>
> > > >>>>
> > > >>> Thoughts?>
> > >>
> > >>
> >
>
Re: A plan to (re) implement OpenWhisk on top of Knative
Posted by Michele Sciabarra <mi...@sciabarra.com>.
Thanks! Indeed it was my next step since I am now trying to build actions with the actionloop based runtime with tekton. I thought to use Priti work that has to be updated to use Tekton so your article is very welcome and useful.
--
Michele Sciabarra
michele@sciabarra.com
----- Original message -----
From: Martin Henke <ma...@web.de>
To: dev@openwhisk.apache.org
Subject: Re: A plan to (re) implement OpenWhisk on top of Knative
Date: Wednesday, July 03, 2019 4:29 PM
Michele,
FYI: Sugandha and myself published a Medium blog article which describes to build Nodejs10 images using Tekton and run it on Knative
(based on the work from Priti).
It might be on interest in the context of your Actionloop related Knative work.
Link:
https://medium.com/@sugandha.agrawal18/build-knative-service-with-tekton-and-apache-openwhisk-node-js-runtime-f660bbc3a11e
Regards,
Martin
On 2019/05/20 15:00:02, "Michele Sciabarra" <m....@sciabarra.com> wrote:
> Ok great, I see the discussion is starting to bring ideas.>
>
> Yes my goal is basically to run existing actions in Knative, create and invoke. And possibile retain the ability of an action to invoke another action.>
>
> I understand the different way they expose services, so I am rethinking the idea of using a "work-alike" path. >
>
> If it is needed we can add it with an ingress but it may be not necessary in the initial implementation.>
>
> Also I checked a bit ML and discussions and I see this Tekton thing that should be the preferred way.>
>
> Not sure if I understand the relation with the current Build API documented in the website. Is Tekton "compatible" or it has a different API?>
>
>
> -- >
> Michele Sciabarra>
> michele@sciabarra.com>
>
> ----- Original message ----->
> From: "Markus Thömmes" <ma...@apache.org>>
> To: dev@openwhisk.apache.org>
> Subject: Re: A plan to (re) implement OpenWhisk on top of Knative>
> Date: Monday, May 20, 2019 4:50 PM>
>
> Good discussion, thanks!>
>
> Can we try to define what the desired end-goal is here? I'm a bit unclear>
> what resembling the OpenWhisk API actually buys us.>
>
> To me, the desired end-state would be to run OpenWhisk actions as-is on a>
> Knative cluster (similar to OpenFaaS' and Azure's integration). There's no>
> good way for us to provide the full API without spinning up a control plane>
> and we can only handle so much via the CLI. So to me, the end-goal looks>
> like:>
>
> 1. *wsk action create* actually doing all the pieces necessary to run a>
> piece of code on Knative.>
> 2. *wsk action invoke* doing some HTTP call under the hood to "invoke" that>
> action. The action should be reachable via a sensible URL. If we really>
> want to keep the API surface (as I said, I'm dubious here) we can also do>
> that via ingress level abstractions (like VirtualService).>
>
> Cheers,>
> Markus>
>
> Am Mo., 20. Mai 2019 um 15:33 Uhr schrieb Martin Henke <ma...@web.de>
> >:>
>
> >>
> > > On 20. May 2019, at 14:55, Michele Sciabarra <mi...@sciabarra.com>>
> > wrote:>
> > >>
> > >> Michele,>
> > >>
> > >> I like the idea to make the ActionLoop based runtimes to be runnable on>
> > Knative.>
> > >>>
> > >> My thoughts on this:>
> > >> - I second Markus concern to implement the invocation API onto Knative>
> > instead of just using Knative service syntax.>
> > > Can you elaborate this? I do not understand.>
> >>
> > Knative service syntax: https://<service(in our case =>
> > action)>.<namespace>.<host>/>
> > OW invocation https://>
> > <api-host>/api/v1/namespaces/<namespace>/actions/<action>>
> >>
> > (I personally so no worth in inventing a distinct API for OW images, but>
> > as said I would see that as a valid optional feature)>
> >>
> > >>
> > >> - I would have concerns to make it dependent on Gloo which is kind of a>
> > minority choice for Knative load balancing>
> > > I do not think it will be hard to setup a test also using Istio, I do>
> > not want to be limited to Gloo.>
> >>
> > I just wanted to prevent that Gloo gets a “official” prerequisite for an>
> > “official” OW on Knative flow.>
> > It is of course free to you to use what ever you want to do in your>
> > prototype.>
> >>
> > > - In my opinion the goal should be to have some uniform behaviour for>
> > ActionLoop based runtimes>
> > >> and other ones like the adapted NodeJS runtimes demonstrated by Matt>
> > and Priti>
> > > As much as I can tell the current implementation is just the building>
> > and exposing the "/init" and "/run" but I can be wrong.>
> > > The build can be of course reused, so it continues the effort. For the>
> > frontend, from the documentation I think Matt wants to add a proxy, while I>
> > would like to implemeent the "invocation" straight in the runtime. This is>
> > open to discussion, but of course it is better to reach an agreement.>
> >>
> > Also in the work of Priti and Matt the invocation goes directly to the>
> > runtime. The action code is either passed with the call (not yet tested by>
> > me) or set via environment variable in the docker build.>
> >>
> > >>
> > >> - As Knative Build seems be on a dead end I would propose to target>
> > Tekton as the build system (which developed as kind of >successor out of>
> > Knative)>
> > >>
> > > If Knative build is dead then it would be a bit unfair that they change>
> > it as the scope of the Knative project!>
> > > It looks like the goal is to setup some standards! And I would be very>
> > disappointed to know that.>
> >>
> > Tekton evolved out of Knative Build (or more correct out of Knative>
> > Pipelines) but is very similar to the Knative build.>
> > Flows can easily be ported from one to the other,>
> > If we target Tekton build we would target the platform were the Knative>
> > build team is focusing on.>
> > But again feel free to use whatever platform for your prototype work.>
> >>
> > > At this stage the build is the more interesting thing, and it could be>
> > even imported in main openwhisk to speed up deployment.>
> > > I have already baked it in the ActionLoop runtimes (with precompilation).>
> > > Also if we use Tekton, where is the Knative standard then? What is the>
> > point? We can build our own system instead of "Knativizing" it...>
> > >>
> > >> Maybe it would be a good solution to tackle two things independently.>
> > >> 1) Design and implement a common protocol of building, running and>
> > calling OW runtimes on Knative>
> > >> 2) Implement the OW invocation API on top of Knative as an additional>
> > option for those who have the need to expose it.>
> > >>
> > > On this, for my personal approach at building things, I want something>
> > that works and it is complete and useful. A "MVP”.>
> >>
> > Cool. Just go on.>
> >>
> > > So I do not plan to split the effort. Version 0.1 must be a minimal>
> > working subset of OpenWhisk on Knative.>
> > > Because otherwise there will be incomplete useless inusable pieces>
> > around (see for example kwsk).>
> > >>
> > > It does not mean that things cannot be modular, nor that everyone must>
> > but to me "openwhisk-knative" must be a single repo with all the pieces to>
> > make something where you can download is and deploy in a kubernetes cluster>
> > and be able to deploy simple actions. When this works, we can improve>
> > incrementally and split it but keeping it working.>
> > >>
> > >> I would looking forward to work with you on the first work item.>
> > > Great but I see now more details to discuss before we can start. Most>
> > notably I need to understand how I can build on top of Mark and Priti work>
> > and continue their work. ANd I can even probably recover some of the code>
> > of kwsk as they implemented some openwhisk api, that I want now in the>
> > runtime.>
> > >>
> >>
> > I do not want to stop you in any way. My hope is that the action loop>
> > runtimes and the “other ones” do expose the same behaviour when being>
> > called. So that the users is not surprised when calling different actions>
> > in different languages.>
> > And behaving the same way might also mean to adapt the “other languages”>
> > to the same behaviour as the action loop based ones.>
> > They just should be uniform to be used.>
> >>
> > When your prototype is accessible it would be a good point of time to>
> > discuss this.>
> >>
> > As said I very much like your effort.>
> >>
> > >>
> > >> On 20. May 2019, at 08:55, Michele Sciabarra <mi...@sciabarra.com>>
> > wrote:>
> > >>>
> > >>>
> > >>>> I have an idea for implementing a prototype of OpenWhisk on top of>
> > Knative.>
> > >>>> My basic ideas are: do not use any proxy, forwarding or adapter:>
> > extend>
> > >>>> the runtime to support the REST call and expose them as ingress. And>
> > use a>
> > >>>> wrapper on top of `kubectl` to generate all the needed components.>
> > >>>
> > >>> Does this tie into the work that Matt was doing to the runtimes to make>
> > >>> them runnable on Knative? Is this lined up with that at all?>
> > >> Actually yes. He suggested I can investigate how to migrate ActionLoop>
> > to port many other languages to Knative.>
> > >> Also he recommended I add my idea and this is what I am doing. Current>
> > code is, if I am not wrong, a Knative build of the nodejs runtime.>
> > >>>
> > >> There has been a number of attempts and proposal to move forward>
> > OpenWhisk. My idea that to succeed we need something small but that just>
> > works. This is my idea to be able to implement in the shorter time frame>
> > possible an actual subset of OpenWhisk that works and it is truly built on>
> > top of Knative. So I am putting the thing a bit further than Matt work.>
> > >>>
> > >>>
> > >>>> My goal is to have a functional work-alike of OpenWhisk built on top>
> > of>
> > >>>> Knative, using ActionLoop as a foundation. I will extend ActionLoop to>
> > >>>> support the required REST calls of OpenWhisk.>
> > >>>>
> > >>>> I also want to create tool, I will call `wskn`. This tool will>
> > initially>
> > >>>> just a python script, a wrapper on top of `kubectl` as it will>
> > generate>
> > >>>> kubernetes descriptors.>
> > >>> Why not build this into "wsk" itself? The Azure Functions CLI as an>
> > example>
> > >>> supports multiple deployment types like this in one CLI.>
> > >>>
> > >> When it will works, yes, of course. But to start, what I really need is>
> > a prototype that can generate kubernetes descripttors to feed to kubectl,>
> > so a simplee, quick and ditry, separate tool (that I will keep together>
> > the runtime) is all I need for now.>
> > >>>
> > >>>>>
> > >>>> It will support initially just the the action creation and>
> > invocation, and>
> > >>>> only synchronous (blocking) behaviour, as all the request will go>
> > straight>
> > >>>> to the runtimes. Hopefully also a subset of `package` and>
> > `activation`.>
> > >>>> Again triggers, rules, asynchronous for later.>
> > >>>>>
> > >>>> The idea is that you will be able to create actions and web actions>
> > that>
> > >>>> can run existing OpenWhisk actions, at least those with blocking>
> > behaviour>
> > >>>> that run with ActionLoop (Go, Java, Python, PHP, Swift, Rust, Ruby,>
> > >>>> Crystal...)>
> > >>>>>
> > >>>> Implementation.>
> > >>>> ==============>
> > >>>>>
> > >>>> This is how I plan to implement it.>
> > >>>>>
> > >>>> At this stage I want to use just Knative Serving and Knative Build,>
> > using>
> > >>>> Gloo for the ingress part. I also plan to install a local Docker>
> > registry>
> > >>>> Kubernetes registry, so we do not have to use DockerHub for>
> > everything. All>
> > >>>> of this can be done with existing command line tools in a few minutes>
> > in>
> > >>>> any running Kubernetes deployment.>
> > >>>>>
> > >>>
> > >>> Why specifying Gloo here? Do you need anything specific from Gloo>
> > itself?>
> > >>> If not I'd propose to just keep it on a Knative Serving API surface>
> > level.>
> > >> I want to build it on top of Knative serving, full stop. Currently,>
> > installing Gloo is pretty easy and is more lightweight than Istio so I>
> > will use it for my first implementation.>
> > >>>
> > >>>>>
> > >>>> When I create an action, it will use Knative build that will work>
> > roughly>
> > >>>> in this way:>
> > >>>>>
> > >>>> - create a configmap with the action code>
> > >>>> - build the actin using ActionLoop precompilation feature that will>
> > return>
> > >>>> a zip file including all the needed to run the action>
> > >>>> - create a new docker image extending the runtime with the new zip,>
> > using>
> > >>>> Kaanico>
> > >>>> - push the image in the local registry>
> > >>> This feels like a fairly heavyweight process, we should be able to>
> > come up>
> > >>> with a way to circumvent zipping entirely. Maybe the runtime can detect>
> > >>> that the unzipped content is already there and skip the unzip step?>
> > >>>
> > >> Actually this is my first idea of how to use Knative build. And is not>
> > complicated: when I create the action, a run a build that includes Kanico.>
> > I generate a Dockerfile on the fly. The docker file uses the action runtime>
> > that already know how to compile a script. And then I save an image. I>
> > already implemented un "autoinit" so just launching the image will give a>
> > runtime ready to run that execute an action already compiled.>
> > >>>
> > >>>
> > >>> I'm fairly hesitant on the usage of a ConfigMap for storing the action>
> > >>> code. It's all stored in the in-cluster etcd instance and it has a>
> > limit of>
> > >>> 1M. This is at most a stop-gap solution to provide a PoC I think. Any>
> > ideas>
> > >>> on how to "productize" this?>
> > >>>
> > >> ConfigMap can be mounted as files, so it is an easy way to feed an>
> > action to a build. It is just an easy way to feed the action code to the>
> > Build.>
> > >>>
> > >> My initial constraint is that I want just to generate Kubernetes>
> > descriptors to feed to kubectl.>
> > >> Of course in the long run I can add some "file upload" storage.>
> > >>>
> > >> If I could to this file upload when invoking a build it could ideal as>
> > I do not have to store anything anywhere, just process the code and>
> > generate a single layer to execute actions to be store in the registry.>
> > >> I will investigate better this area, I understand your concern.>
> > >>>
> > >>>>
> > >>>> At this point you can run the action. ActionLoop will be extended to>
> > >>>> support invocations in the format>
> > >>>> "/v1/namespaces/namespace/actions/package/action".>
> > >>> Why bother reimplementing this exact path? To obtain API compatibility>
> > with>
> > >>> OpenWhisk as it is today?>
> > >>>
> > >> I want to implement a subset of the OpenWhisk API on top of Knative>
> > serving.>
> > >> Knative serving already does the scaling and routing, so what we need>
> > are the "endpoints" to invoke actions.>
> > >>>
> > >> Since I do not want to add additional components, not at the first>
> > stage. Just knative serve and build, the runtime and a controller script,>
> > the runtime is the natural place where to "handle" the API invocations,>
> > since Knative only generates the URL but not anything else. If I>
> > understood well, Matt is adding a proxy. I do not want to add a proxy, just>
> > add to the runtime the ability to respond to "API like" calls, at least>
> > those regarding action invocation.>
> > >>>
> > >>>> It will do all the decoding required to invoke the action with the>
> > >>>> expected paramenters (straight invocation thrhoug the actinloop>
> > protocol,>
> > >>>> not proxies).>
> > >>> Does this mean moving all of the Controller's "smartness" about>
> > incoming>
> > >>> and outgoing HTTP requests (see the whole WebActions for example)?>
> > >>>
> > >> At least decoding web actions in the runtime, yes. Knative serving>
> > already has routing and proxying.>
> > >> So a true implementation on top of Knative requires IHMO this>
> > sacrifice. Unless there is a way to keep the controller in a "Knative">
> > compatible way. Open to suggestions here.>
> > >>>
> > >>> Each action will then be exposed using an ingress with its specific>
> > >>> invocation path.>
> > >>>>
> > >>> If the community agrees with this plan, I would create a repo>
> > >>> `incubator-openwhisk-knative` to work on it.>
> > >>>>
> > >>> Thoughts?>
> >>
> >>
>
Re: Re: Re: A plan to (re) implement OpenWhisk on top of Knative
Posted by Matt Rutkowski <mr...@us.ibm.com>.
FYI, Priti and I spoke earlier this week about exploring the addition of
new "build" command for wskdeploy that could utilize Tekton.
Kind regards,
Matt
From: "Michele Sciabarra" <mi...@sciabarra.com>
To: dev@openwhisk.apache.org
Date: 07/03/2019 12:06 PM
Subject: [EXTERNAL] Re: Re: A plan to (re) implement OpenWhisk on
top of Knative
Well.... it is actually a bit more complex than this.
If we use Tekton for building, it is Tekton the standard, so as long as we
define a standard for input and output, we delegate to a tekton build the
preparation of an image ready for being served by Knative Serving,
However the action loop based runtimes rely on a "compilation" script
already written in python that performs the steps, so all I need to do is
to adapt the compilation to the Tekton standard. This is something I
already did, at lest for Go, modifying the actionloop codee and I am now
trying to complete the pipeline.
The way I did for Go was not getting rid of the "init" step but providing
an "autoinit" step that happens at runtime start, executing a binary
executable already embedded in the image and produced by the compilation
done by the runtime itself.
All of this is more complex to say than to do so I hope I can show
something soon... hopefully pretty interesting.
---
Michele Sciabarra
michele@sciabarra.com
PS the book "learning Apache OpenWhisk" is now on printing!
----- Original message -----
From: Matt Rutkowski <mr...@us.ibm.com>
To: dev@openwhisk.apache.org
Subject: Re: Re: A plan to (re) implement OpenWhisk on top of Knative
Date: Wednesday, July 03, 2019 4:43 PM
Hi Martin,
It is my belief that Michele, now freshly returned from book editing
(congrats), was going to implement the same interface for pre/post
processing requests that we implemented for NodeJS. This would allow us a
path to support this across all language runtimes as well as formalize our
runtime contract that aligns with a Knative Service approach and then also
allows us to better explore some of the use cases being discussed on
another thread from Rodric. I believe that we should look at smaller
steps towards that alignment like removing the need for the "init"
entrypoint and allowing access to parameters either by env. vars. or
function arguments allowing both compat. with 12-factor app approach, as
well as attempting to encourage a functional programming model where you
should ideally be unaware of any system environment.
Kind regards,
Matt
From: Martin Henke <ma...@web.de>
To: dev@openwhisk.apache.org
Date: 07/03/2019 09:29 AM
Subject: [EXTERNAL] Re: A plan to (re) implement OpenWhisk on top
of Knative
Michele,
FYI: Sugandha and myself published a Medium blog article which describes
to build Nodejs10 images using Tekton and run it on Knative
(based on the work from Priti).
It might be on interest in the context of your Actionloop related Knative
work.
Link:
https://urldefense.proofpoint.com/v2/url?u=https-3A__medium.com_-40sugandha.agrawal18_build-2Dknative-2Dservice-2Dwith-2Dtekton-2Dand-2Dapache-2Dopenwhisk-2Dnode-2Djs-2Druntime-2Df660bbc3a11e&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=6zQLM7Gc0Sv1iwayKOKa4_SFxRIxS478q2gZlAJj4Zw&m=bl7bwPaoiSE6fi0fyVHNC9bNUnh1ZUUlfl3lwUZbcH0&s=IQBGPfSTPiAjKhIlsqKDpAjoJBNd7By1YnjxOIx2454&e=
Regards,
Martin
On 2019/05/20 15:00:02, "Michele Sciabarra" <m....@sciabarra.com> wrote:
> Ok great, I see the discussion is starting to bring ideas.>
>
> Yes my goal is basically to run existing actions in Knative, create and
invoke. And possibile retain the ability of an action to invoke another
action.>
>
> I understand the different way they expose services, so I am rethinking
the idea of using a "work-alike" path. >
>
> If it is needed we can add it with an ingress but it may be not
necessary in the initial implementation.>
>
> Also I checked a bit ML and discussions and I see this Tekton thing that
should be the preferred way.>
>
> Not sure if I understand the relation with the current Build API
documented in the website. Is Tekton "compatible" or it has a different
API?>
>
>
> -- >
> Michele Sciabarra>
> michele@sciabarra.com>
>
> ----- Original message ----->
> From: "Markus Thömmes" <ma...@apache.org>>
> To: dev@openwhisk.apache.org>
> Subject: Re: A plan to (re) implement OpenWhisk on top of Knative>
> Date: Monday, May 20, 2019 4:50 PM>
>
> Good discussion, thanks!>
>
> Can we try to define what the desired end-goal is here? I'm a bit
unclear>
> what resembling the OpenWhisk API actually buys us.>
>
> To me, the desired end-state would be to run OpenWhisk actions as-is on
a>
> Knative cluster (similar to OpenFaaS' and Azure's integration). There's
no>
> good way for us to provide the full API without spinning up a control
plane>
> and we can only handle so much via the CLI. So to me, the end-goal
looks>
> like:>
>
> 1. *wsk action create* actually doing all the pieces necessary to run a>
> piece of code on Knative.>
> 2. *wsk action invoke* doing some HTTP call under the hood to "invoke"
that>
> action. The action should be reachable via a sensible URL. If we really>
> want to keep the API surface (as I said, I'm dubious here) we can also
do>
> that via ingress level abstractions (like VirtualService).>
>
> Cheers,>
> Markus>
>
> Am Mo., 20. Mai 2019 um 15:33 Uhr schrieb Martin Henke
<ma...@web.de>
> >:>
>
> >>
> > > On 20. May 2019, at 14:55, Michele Sciabarra <mi...@sciabarra.com>>
> > wrote:>
> > >>
> > >> Michele,>
> > >>
> > >> I like the idea to make the ActionLoop based runtimes to be
runnable on>
> > Knative.>
> > >>>
> > >> My thoughts on this:>
> > >> - I second Markus concern to implement the invocation API onto
Knative>
> > instead of just using Knative service syntax.>
> > > Can you elaborate this? I do not understand.>
> >>
> > Knative service syntax: https://<service(in our case =>
> > action)>.<namespace>.<host>/>
> > OW invocation https://>
> > <api-host>/api/v1/namespaces/<namespace>/actions/<action>>
> >>
> > (I personally so no worth in inventing a distinct API for OW images,
but>
> > as said I would see that as a valid optional feature)>
> >>
> > >>
> > >> - I would have concerns to make it dependent on Gloo which is kind
of a>
> > minority choice for Knative load balancing>
> > > I do not think it will be hard to setup a test also using Istio, I
do>
> > not want to be limited to Gloo.>
> >>
> > I just wanted to prevent that Gloo gets a “official” prerequisite for
an>
> > “official” OW on Knative flow.>
> > It is of course free to you to use what ever you want to do in your>
> > prototype.>
> >>
> > > - In my opinion the goal should be to have some uniform behaviour
for>
> > ActionLoop based runtimes>
> > >> and other ones like the adapted NodeJS runtimes demonstrated by
Matt>
> > and Priti>
> > > As much as I can tell the current implementation is just the
building>
> > and exposing the "/init" and "/run" but I can be wrong.>
> > > The build can be of course reused, so it continues the effort. For
the>
> > frontend, from the documentation I think Matt wants to add a proxy,
while I>
> > would like to implemeent the "invocation" straight in the runtime.
This is>
> > open to discussion, but of course it is better to reach an agreement.>
> >>
> > Also in the work of Priti and Matt the invocation goes directly to
the>
> > runtime. The action code is either passed with the call (not yet
tested by>
> > me) or set via environment variable in the docker build.>
> >>
> > >>
> > >> - As Knative Build seems be on a dead end I would propose to
target>
> > Tekton as the build system (which developed as kind of >successor out
of>
> > Knative)>
> > >>
> > > If Knative build is dead then it would be a bit unfair that they
change>
> > it as the scope of the Knative project!>
> > > It looks like the goal is to setup some standards! And I would be
very>
> > disappointed to know that.>
> >>
> > Tekton evolved out of Knative Build (or more correct out of Knative>
> > Pipelines) but is very similar to the Knative build.>
> > Flows can easily be ported from one to the other,>
> > If we target Tekton build we would target the platform were the
Knative>
> > build team is focusing on.>
> > But again feel free to use whatever platform for your prototype work.>
> >>
> > > At this stage the build is the more interesting thing, and it could
be>
> > even imported in main openwhisk to speed up deployment.>
> > > I have already baked it in the ActionLoop runtimes (with
precompilation).>
> > > Also if we use Tekton, where is the Knative standard then? What is
the>
> > point? We can build our own system instead of "Knativizing" it...>
> > >>
> > >> Maybe it would be a good solution to tackle two things
independently.>
> > >> 1) Design and implement a common protocol of building, running and>
> > calling OW runtimes on Knative>
> > >> 2) Implement the OW invocation API on top of Knative as an
additional>
> > option for those who have the need to expose it.>
> > >>
> > > On this, for my personal approach at building things, I want
something>
> > that works and it is complete and useful. A "MVP”.>
> >>
> > Cool. Just go on.>
> >>
> > > So I do not plan to split the effort. Version 0.1 must be a minimal>
> > working subset of OpenWhisk on Knative.>
> > > Because otherwise there will be incomplete useless inusable pieces>
> > around (see for example kwsk).>
> > >>
> > > It does not mean that things cannot be modular, nor that everyone
must>
> > but to me "openwhisk-knative" must be a single repo with all the
pieces to>
> > make something where you can download is and deploy in a kubernetes
cluster>
> > and be able to deploy simple actions. When this works, we can improve>
> > incrementally and split it but keeping it working.>
> > >>
> > >> I would looking forward to work with you on the first work item.>
> > > Great but I see now more details to discuss before we can start.
Most>
> > notably I need to understand how I can build on top of Mark and Priti
work>
> > and continue their work. ANd I can even probably recover some of the
code>
> > of kwsk as they implemented some openwhisk api, that I want now in
the>
> > runtime.>
> > >>
> >>
> > I do not want to stop you in any way. My hope is that the action loop>
> > runtimes and the “other ones” do expose the same behaviour when being>
> > called. So that the users is not surprised when calling different
actions>
> > in different languages.>
> > And behaving the same way might also mean to adapt the “other
languages”>
> > to the same behaviour as the action loop based ones.>
> > They just should be uniform to be used.>
> >>
> > When your prototype is accessible it would be a good point of time to>
> > discuss this.>
> >>
> > As said I very much like your effort.>
> >>
> > >>
> > >> On 20. May 2019, at 08:55, Michele Sciabarra <mi...@sciabarra.com>>
> > wrote:>
> > >>>
> > >>>
> > >>>> I have an idea for implementing a prototype of OpenWhisk on top
of>
> > Knative.>
> > >>>> My basic ideas are: do not use any proxy, forwarding or adapter:>
> > extend>
> > >>>> the runtime to support the REST call and expose them as ingress.
And>
> > use a>
> > >>>> wrapper on top of `kubectl` to generate all the needed
components.>
> > >>>
> > >>> Does this tie into the work that Matt was doing to the runtimes to
make>
> > >>> them runnable on Knative? Is this lined up with that at all?>
> > >> Actually yes. He suggested I can investigate how to migrate
ActionLoop>
> > to port many other languages to Knative.>
> > >> Also he recommended I add my idea and this is what I am doing.
Current>
> > code is, if I am not wrong, a Knative build of the nodejs runtime.>
> > >>>
> > >> There has been a number of attempts and proposal to move forward>
> > OpenWhisk. My idea that to succeed we need something small but that
just>
> > works. This is my idea to be able to implement in the shorter time
frame>
> > possible an actual subset of OpenWhisk that works and it is truly
built on>
> > top of Knative. So I am putting the thing a bit further than Matt
work.>
> > >>>
> > >>>
> > >>>> My goal is to have a functional work-alike of OpenWhisk built on
top>
> > of>
> > >>>> Knative, using ActionLoop as a foundation. I will extend
ActionLoop to>
> > >>>> support the required REST calls of OpenWhisk.>
> > >>>>
> > >>>> I also want to create tool, I will call `wskn`. This tool will>
> > initially>
> > >>>> just a python script, a wrapper on top of `kubectl` as it will>
> > generate>
> > >>>> kubernetes descriptors.>
> > >>> Why not build this into "wsk" itself? The Azure Functions CLI as
an>
> > example>
> > >>> supports multiple deployment types like this in one CLI.>
> > >>>
> > >> When it will works, yes, of course. But to start, what I really
need is>
> > a prototype that can generate kubernetes descripttors to feed to
kubectl,>
> > so a simplee, quick and ditry, separate tool (that I will keep
together>
> > the runtime) is all I need for now.>
> > >>>
> > >>>>>
> > >>>> It will support initially just the the action creation and>
> > invocation, and>
> > >>>> only synchronous (blocking) behaviour, as all the request will
go>
> > straight>
> > >>>> to the runtimes. Hopefully also a subset of `package` and>
> > `activation`.>
> > >>>> Again triggers, rules, asynchronous for later.>
> > >>>>>
> > >>>> The idea is that you will be able to create actions and web
actions>
> > that>
> > >>>> can run existing OpenWhisk actions, at least those with blocking>
> > behaviour>
> > >>>> that run with ActionLoop (Go, Java, Python, PHP, Swift, Rust,
Ruby,>
> > >>>> Crystal...)>
> > >>>>>
> > >>>> Implementation.>
> > >>>> ==============>
> > >>>>>
> > >>>> This is how I plan to implement it.>
> > >>>>>
> > >>>> At this stage I want to use just Knative Serving and Knative
Build,>
> > using>
> > >>>> Gloo for the ingress part. I also plan to install a local Docker>
> > registry>
> > >>>> Kubernetes registry, so we do not have to use DockerHub for>
> > everything. All>
> > >>>> of this can be done with existing command line tools in a few
minutes>
> > in>
> > >>>> any running Kubernetes deployment.>
> > >>>>>
> > >>>
> > >>> Why specifying Gloo here? Do you need anything specific from Gloo>
> > itself?>
> > >>> If not I'd propose to just keep it on a Knative Serving API
surface>
> > level.>
> > >> I want to build it on top of Knative serving, full stop.
Currently,>
> > installing Gloo is pretty easy and is more lightweight than Istio so
I>
> > will use it for my first implementation.>
> > >>>
> > >>>>>
> > >>>> When I create an action, it will use Knative build that will
work>
> > roughly>
> > >>>> in this way:>
> > >>>>>
> > >>>> - create a configmap with the action code>
> > >>>> - build the actin using ActionLoop precompilation feature that
will>
> > return>
> > >>>> a zip file including all the needed to run the action>
> > >>>> - create a new docker image extending the runtime with the new
zip,>
> > using>
> > >>>> Kaanico>
> > >>>> - push the image in the local registry>
> > >>> This feels like a fairly heavyweight process, we should be able
to>
> > come up>
> > >>> with a way to circumvent zipping entirely. Maybe the runtime can
detect>
> > >>> that the unzipped content is already there and skip the unzip
step?>
> > >>>
> > >> Actually this is my first idea of how to use Knative build. And is
not>
> > complicated: when I create the action, a run a build that includes
Kanico.>
> > I generate a Dockerfile on the fly. The docker file uses the action
runtime>
> > that already know how to compile a script. And then I save an image.
I>
> > already implemented un "autoinit" so just launching the image will
give a>
> > runtime ready to run that execute an action already compiled.>
> > >>>
> > >>>
> > >>> I'm fairly hesitant on the usage of a ConfigMap for storing the
action>
> > >>> code. It's all stored in the in-cluster etcd instance and it has
a>
> > limit of>
> > >>> 1M. This is at most a stop-gap solution to provide a PoC I think.
Any>
> > ideas>
> > >>> on how to "productize" this?>
> > >>>
> > >> ConfigMap can be mounted as files, so it is an easy way to feed
an>
> > action to a build. It is just an easy way to feed the action code to
the>
> > Build.>
> > >>>
> > >> My initial constraint is that I want just to generate Kubernetes>
> > descriptors to feed to kubectl.>
> > >> Of course in the long run I can add some "file upload" storage.>
> > >>>
> > >> If I could to this file upload when invoking a build it could ideal
as>
> > I do not have to store anything anywhere, just process the code and>
> > generate a single layer to execute actions to be store in the
registry.>
> > >> I will investigate better this area, I understand your concern.>
> > >>>
> > >>>>
> > >>>> At this point you can run the action. ActionLoop will be extended
to>
> > >>>> support invocations in the format>
> > >>>> "/v1/namespaces/namespace/actions/package/action".>
> > >>> Why bother reimplementing this exact path? To obtain API
compatibility>
> > with>
> > >>> OpenWhisk as it is today?>
> > >>>
> > >> I want to implement a subset of the OpenWhisk API on top of
Knative>
> > serving.>
> > >> Knative serving already does the scaling and routing, so what we
need>
> > are the "endpoints" to invoke actions.>
> > >>>
> > >> Since I do not want to add additional components, not at the first>
> > stage. Just knative serve and build, the runtime and a controller
script,>
> > the runtime is the natural place where to "handle" the API
invocations,>
> > since Knative only generates the URL but not anything else. If I>
> > understood well, Matt is adding a proxy. I do not want to add a proxy,
just>
> > add to the runtime the ability to respond to "API like" calls, at
least>
> > those regarding action invocation.>
> > >>>
> > >>>> It will do all the decoding required to invoke the action with
the>
> > >>>> expected paramenters (straight invocation thrhoug the actinloop>
> > protocol,>
> > >>>> not proxies).>
> > >>> Does this mean moving all of the Controller's "smartness" about>
> > incoming>
> > >>> and outgoing HTTP requests (see the whole WebActions for
example)?>
> > >>>
> > >> At least decoding web actions in the runtime, yes. Knative serving>
> > already has routing and proxying.>
> > >> So a true implementation on top of Knative requires IHMO this>
> > sacrifice. Unless there is a way to keep the controller in a
"Knative">
> > compatible way. Open to suggestions here.>
> > >>>
> > >>> Each action will then be exposed using an ingress with its
specific>
> > >>> invocation path.>
> > >>>>
> > >>> If the community agrees with this plan, I would create a repo>
> > >>> `incubator-openwhisk-knative` to work on it.>
> > >>>>
> > >>> Thoughts?>
> >>
> >>
>
Re: Re: A plan to (re) implement OpenWhisk on top of Knative
Posted by Michele Sciabarra <mi...@sciabarra.com>.
Well.... it is actually a bit more complex than this.
If we use Tekton for building, it is Tekton the standard, so as long as we define a standard for input and output, we delegate to a tekton build the preparation of an image ready for being served by Knative Serving,
However the action loop based runtimes rely on a "compilation" script already written in python that performs the steps, so all I need to do is to adapt the compilation to the Tekton standard. This is something I already did, at lest for Go, modifying the actionloop codee and I am now trying to complete the pipeline.
The way I did for Go was not getting rid of the "init" step but providing an "autoinit" step that happens at runtime start, executing a binary executable already embedded in the image and produced by the compilation done by the runtime itself.
All of this is more complex to say than to do so I hope I can show something soon... hopefully pretty interesting.
---
Michele Sciabarra
michele@sciabarra.com
PS the book "learning Apache OpenWhisk" is now on printing!
----- Original message -----
From: Matt Rutkowski <mr...@us.ibm.com>
To: dev@openwhisk.apache.org
Subject: Re: Re: A plan to (re) implement OpenWhisk on top of Knative
Date: Wednesday, July 03, 2019 4:43 PM
Hi Martin,
It is my belief that Michele, now freshly returned from book editing
(congrats), was going to implement the same interface for pre/post
processing requests that we implemented for NodeJS. This would allow us a
path to support this across all language runtimes as well as formalize our
runtime contract that aligns with a Knative Service approach and then also
allows us to better explore some of the use cases being discussed on
another thread from Rodric. I believe that we should look at smaller
steps towards that alignment like removing the need for the "init"
entrypoint and allowing access to parameters either by env. vars. or
function arguments allowing both compat. with 12-factor app approach, as
well as attempting to encourage a functional programming model where you
should ideally be unaware of any system environment.
Kind regards,
Matt
From: Martin Henke <ma...@web.de>
To: dev@openwhisk.apache.org
Date: 07/03/2019 09:29 AM
Subject: [EXTERNAL] Re: A plan to (re) implement OpenWhisk on top
of Knative
Michele,
FYI: Sugandha and myself published a Medium blog article which describes
to build Nodejs10 images using Tekton and run it on Knative
(based on the work from Priti).
It might be on interest in the context of your Actionloop related Knative
work.
Link:
https://urldefense.proofpoint.com/v2/url?u=https-3A__medium.com_-40sugandha.agrawal18_build-2Dknative-2Dservice-2Dwith-2Dtekton-2Dand-2Dapache-2Dopenwhisk-2Dnode-2Djs-2Druntime-2Df660bbc3a11e&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=6zQLM7Gc0Sv1iwayKOKa4_SFxRIxS478q2gZlAJj4Zw&m=bl7bwPaoiSE6fi0fyVHNC9bNUnh1ZUUlfl3lwUZbcH0&s=IQBGPfSTPiAjKhIlsqKDpAjoJBNd7By1YnjxOIx2454&e=
Regards,
Martin
On 2019/05/20 15:00:02, "Michele Sciabarra" <m....@sciabarra.com> wrote:
> Ok great, I see the discussion is starting to bring ideas.>
>
> Yes my goal is basically to run existing actions in Knative, create and
invoke. And possibile retain the ability of an action to invoke another
action.>
>
> I understand the different way they expose services, so I am rethinking
the idea of using a "work-alike" path. >
>
> If it is needed we can add it with an ingress but it may be not
necessary in the initial implementation.>
>
> Also I checked a bit ML and discussions and I see this Tekton thing that
should be the preferred way.>
>
> Not sure if I understand the relation with the current Build API
documented in the website. Is Tekton "compatible" or it has a different
API?>
>
>
> -- >
> Michele Sciabarra>
> michele@sciabarra.com>
>
> ----- Original message ----->
> From: "Markus Thömmes" <ma...@apache.org>>
> To: dev@openwhisk.apache.org>
> Subject: Re: A plan to (re) implement OpenWhisk on top of Knative>
> Date: Monday, May 20, 2019 4:50 PM>
>
> Good discussion, thanks!>
>
> Can we try to define what the desired end-goal is here? I'm a bit
unclear>
> what resembling the OpenWhisk API actually buys us.>
>
> To me, the desired end-state would be to run OpenWhisk actions as-is on
a>
> Knative cluster (similar to OpenFaaS' and Azure's integration). There's
no>
> good way for us to provide the full API without spinning up a control
plane>
> and we can only handle so much via the CLI. So to me, the end-goal
looks>
> like:>
>
> 1. *wsk action create* actually doing all the pieces necessary to run a>
> piece of code on Knative.>
> 2. *wsk action invoke* doing some HTTP call under the hood to "invoke"
that>
> action. The action should be reachable via a sensible URL. If we really>
> want to keep the API surface (as I said, I'm dubious here) we can also
do>
> that via ingress level abstractions (like VirtualService).>
>
> Cheers,>
> Markus>
>
> Am Mo., 20. Mai 2019 um 15:33 Uhr schrieb Martin Henke
<ma...@web.de>
> >:>
>
> >>
> > > On 20. May 2019, at 14:55, Michele Sciabarra <mi...@sciabarra.com>>
> > wrote:>
> > >>
> > >> Michele,>
> > >>
> > >> I like the idea to make the ActionLoop based runtimes to be
runnable on>
> > Knative.>
> > >>>
> > >> My thoughts on this:>
> > >> - I second Markus concern to implement the invocation API onto
Knative>
> > instead of just using Knative service syntax.>
> > > Can you elaborate this? I do not understand.>
> >>
> > Knative service syntax: https://<service(in our case =>
> > action)>.<namespace>.<host>/>
> > OW invocation https://>
> > <api-host>/api/v1/namespaces/<namespace>/actions/<action>>
> >>
> > (I personally so no worth in inventing a distinct API for OW images,
but>
> > as said I would see that as a valid optional feature)>
> >>
> > >>
> > >> - I would have concerns to make it dependent on Gloo which is kind
of a>
> > minority choice for Knative load balancing>
> > > I do not think it will be hard to setup a test also using Istio, I
do>
> > not want to be limited to Gloo.>
> >>
> > I just wanted to prevent that Gloo gets a “official” prerequisite for
an>
> > “official” OW on Knative flow.>
> > It is of course free to you to use what ever you want to do in your>
> > prototype.>
> >>
> > > - In my opinion the goal should be to have some uniform behaviour
for>
> > ActionLoop based runtimes>
> > >> and other ones like the adapted NodeJS runtimes demonstrated by
Matt>
> > and Priti>
> > > As much as I can tell the current implementation is just the
building>
> > and exposing the "/init" and "/run" but I can be wrong.>
> > > The build can be of course reused, so it continues the effort. For
the>
> > frontend, from the documentation I think Matt wants to add a proxy,
while I>
> > would like to implemeent the "invocation" straight in the runtime.
This is>
> > open to discussion, but of course it is better to reach an agreement.>
> >>
> > Also in the work of Priti and Matt the invocation goes directly to
the>
> > runtime. The action code is either passed with the call (not yet
tested by>
> > me) or set via environment variable in the docker build.>
> >>
> > >>
> > >> - As Knative Build seems be on a dead end I would propose to
target>
> > Tekton as the build system (which developed as kind of >successor out
of>
> > Knative)>
> > >>
> > > If Knative build is dead then it would be a bit unfair that they
change>
> > it as the scope of the Knative project!>
> > > It looks like the goal is to setup some standards! And I would be
very>
> > disappointed to know that.>
> >>
> > Tekton evolved out of Knative Build (or more correct out of Knative>
> > Pipelines) but is very similar to the Knative build.>
> > Flows can easily be ported from one to the other,>
> > If we target Tekton build we would target the platform were the
Knative>
> > build team is focusing on.>
> > But again feel free to use whatever platform for your prototype work.>
> >>
> > > At this stage the build is the more interesting thing, and it could
be>
> > even imported in main openwhisk to speed up deployment.>
> > > I have already baked it in the ActionLoop runtimes (with
precompilation).>
> > > Also if we use Tekton, where is the Knative standard then? What is
the>
> > point? We can build our own system instead of "Knativizing" it...>
> > >>
> > >> Maybe it would be a good solution to tackle two things
independently.>
> > >> 1) Design and implement a common protocol of building, running and>
> > calling OW runtimes on Knative>
> > >> 2) Implement the OW invocation API on top of Knative as an
additional>
> > option for those who have the need to expose it.>
> > >>
> > > On this, for my personal approach at building things, I want
something>
> > that works and it is complete and useful. A "MVP”.>
> >>
> > Cool. Just go on.>
> >>
> > > So I do not plan to split the effort. Version 0.1 must be a minimal>
> > working subset of OpenWhisk on Knative.>
> > > Because otherwise there will be incomplete useless inusable pieces>
> > around (see for example kwsk).>
> > >>
> > > It does not mean that things cannot be modular, nor that everyone
must>
> > but to me "openwhisk-knative" must be a single repo with all the
pieces to>
> > make something where you can download is and deploy in a kubernetes
cluster>
> > and be able to deploy simple actions. When this works, we can improve>
> > incrementally and split it but keeping it working.>
> > >>
> > >> I would looking forward to work with you on the first work item.>
> > > Great but I see now more details to discuss before we can start.
Most>
> > notably I need to understand how I can build on top of Mark and Priti
work>
> > and continue their work. ANd I can even probably recover some of the
code>
> > of kwsk as they implemented some openwhisk api, that I want now in
the>
> > runtime.>
> > >>
> >>
> > I do not want to stop you in any way. My hope is that the action loop>
> > runtimes and the “other ones” do expose the same behaviour when being>
> > called. So that the users is not surprised when calling different
actions>
> > in different languages.>
> > And behaving the same way might also mean to adapt the “other
languages”>
> > to the same behaviour as the action loop based ones.>
> > They just should be uniform to be used.>
> >>
> > When your prototype is accessible it would be a good point of time to>
> > discuss this.>
> >>
> > As said I very much like your effort.>
> >>
> > >>
> > >> On 20. May 2019, at 08:55, Michele Sciabarra <mi...@sciabarra.com>>
> > wrote:>
> > >>>
> > >>>
> > >>>> I have an idea for implementing a prototype of OpenWhisk on top
of>
> > Knative.>
> > >>>> My basic ideas are: do not use any proxy, forwarding or adapter:>
> > extend>
> > >>>> the runtime to support the REST call and expose them as ingress.
And>
> > use a>
> > >>>> wrapper on top of `kubectl` to generate all the needed
components.>
> > >>>
> > >>> Does this tie into the work that Matt was doing to the runtimes to
make>
> > >>> them runnable on Knative? Is this lined up with that at all?>
> > >> Actually yes. He suggested I can investigate how to migrate
ActionLoop>
> > to port many other languages to Knative.>
> > >> Also he recommended I add my idea and this is what I am doing.
Current>
> > code is, if I am not wrong, a Knative build of the nodejs runtime.>
> > >>>
> > >> There has been a number of attempts and proposal to move forward>
> > OpenWhisk. My idea that to succeed we need something small but that
just>
> > works. This is my idea to be able to implement in the shorter time
frame>
> > possible an actual subset of OpenWhisk that works and it is truly
built on>
> > top of Knative. So I am putting the thing a bit further than Matt
work.>
> > >>>
> > >>>
> > >>>> My goal is to have a functional work-alike of OpenWhisk built on
top>
> > of>
> > >>>> Knative, using ActionLoop as a foundation. I will extend
ActionLoop to>
> > >>>> support the required REST calls of OpenWhisk.>
> > >>>>
> > >>>> I also want to create tool, I will call `wskn`. This tool will>
> > initially>
> > >>>> just a python script, a wrapper on top of `kubectl` as it will>
> > generate>
> > >>>> kubernetes descriptors.>
> > >>> Why not build this into "wsk" itself? The Azure Functions CLI as
an>
> > example>
> > >>> supports multiple deployment types like this in one CLI.>
> > >>>
> > >> When it will works, yes, of course. But to start, what I really
need is>
> > a prototype that can generate kubernetes descripttors to feed to
kubectl,>
> > so a simplee, quick and ditry, separate tool (that I will keep
together>
> > the runtime) is all I need for now.>
> > >>>
> > >>>>>
> > >>>> It will support initially just the the action creation and>
> > invocation, and>
> > >>>> only synchronous (blocking) behaviour, as all the request will
go>
> > straight>
> > >>>> to the runtimes. Hopefully also a subset of `package` and>
> > `activation`.>
> > >>>> Again triggers, rules, asynchronous for later.>
> > >>>>>
> > >>>> The idea is that you will be able to create actions and web
actions>
> > that>
> > >>>> can run existing OpenWhisk actions, at least those with blocking>
> > behaviour>
> > >>>> that run with ActionLoop (Go, Java, Python, PHP, Swift, Rust,
Ruby,>
> > >>>> Crystal...)>
> > >>>>>
> > >>>> Implementation.>
> > >>>> ==============>
> > >>>>>
> > >>>> This is how I plan to implement it.>
> > >>>>>
> > >>>> At this stage I want to use just Knative Serving and Knative
Build,>
> > using>
> > >>>> Gloo for the ingress part. I also plan to install a local Docker>
> > registry>
> > >>>> Kubernetes registry, so we do not have to use DockerHub for>
> > everything. All>
> > >>>> of this can be done with existing command line tools in a few
minutes>
> > in>
> > >>>> any running Kubernetes deployment.>
> > >>>>>
> > >>>
> > >>> Why specifying Gloo here? Do you need anything specific from Gloo>
> > itself?>
> > >>> If not I'd propose to just keep it on a Knative Serving API
surface>
> > level.>
> > >> I want to build it on top of Knative serving, full stop.
Currently,>
> > installing Gloo is pretty easy and is more lightweight than Istio so
I>
> > will use it for my first implementation.>
> > >>>
> > >>>>>
> > >>>> When I create an action, it will use Knative build that will
work>
> > roughly>
> > >>>> in this way:>
> > >>>>>
> > >>>> - create a configmap with the action code>
> > >>>> - build the actin using ActionLoop precompilation feature that
will>
> > return>
> > >>>> a zip file including all the needed to run the action>
> > >>>> - create a new docker image extending the runtime with the new
zip,>
> > using>
> > >>>> Kaanico>
> > >>>> - push the image in the local registry>
> > >>> This feels like a fairly heavyweight process, we should be able
to>
> > come up>
> > >>> with a way to circumvent zipping entirely. Maybe the runtime can
detect>
> > >>> that the unzipped content is already there and skip the unzip
step?>
> > >>>
> > >> Actually this is my first idea of how to use Knative build. And is
not>
> > complicated: when I create the action, a run a build that includes
Kanico.>
> > I generate a Dockerfile on the fly. The docker file uses the action
runtime>
> > that already know how to compile a script. And then I save an image.
I>
> > already implemented un "autoinit" so just launching the image will
give a>
> > runtime ready to run that execute an action already compiled.>
> > >>>
> > >>>
> > >>> I'm fairly hesitant on the usage of a ConfigMap for storing the
action>
> > >>> code. It's all stored in the in-cluster etcd instance and it has
a>
> > limit of>
> > >>> 1M. This is at most a stop-gap solution to provide a PoC I think.
Any>
> > ideas>
> > >>> on how to "productize" this?>
> > >>>
> > >> ConfigMap can be mounted as files, so it is an easy way to feed
an>
> > action to a build. It is just an easy way to feed the action code to
the>
> > Build.>
> > >>>
> > >> My initial constraint is that I want just to generate Kubernetes>
> > descriptors to feed to kubectl.>
> > >> Of course in the long run I can add some "file upload" storage.>
> > >>>
> > >> If I could to this file upload when invoking a build it could ideal
as>
> > I do not have to store anything anywhere, just process the code and>
> > generate a single layer to execute actions to be store in the
registry.>
> > >> I will investigate better this area, I understand your concern.>
> > >>>
> > >>>>
> > >>>> At this point you can run the action. ActionLoop will be extended
to>
> > >>>> support invocations in the format>
> > >>>> "/v1/namespaces/namespace/actions/package/action".>
> > >>> Why bother reimplementing this exact path? To obtain API
compatibility>
> > with>
> > >>> OpenWhisk as it is today?>
> > >>>
> > >> I want to implement a subset of the OpenWhisk API on top of
Knative>
> > serving.>
> > >> Knative serving already does the scaling and routing, so what we
need>
> > are the "endpoints" to invoke actions.>
> > >>>
> > >> Since I do not want to add additional components, not at the first>
> > stage. Just knative serve and build, the runtime and a controller
script,>
> > the runtime is the natural place where to "handle" the API
invocations,>
> > since Knative only generates the URL but not anything else. If I>
> > understood well, Matt is adding a proxy. I do not want to add a proxy,
just>
> > add to the runtime the ability to respond to "API like" calls, at
least>
> > those regarding action invocation.>
> > >>>
> > >>>> It will do all the decoding required to invoke the action with
the>
> > >>>> expected paramenters (straight invocation thrhoug the actinloop>
> > protocol,>
> > >>>> not proxies).>
> > >>> Does this mean moving all of the Controller's "smartness" about>
> > incoming>
> > >>> and outgoing HTTP requests (see the whole WebActions for
example)?>
> > >>>
> > >> At least decoding web actions in the runtime, yes. Knative serving>
> > already has routing and proxying.>
> > >> So a true implementation on top of Knative requires IHMO this>
> > sacrifice. Unless there is a way to keep the controller in a
"Knative">
> > compatible way. Open to suggestions here.>
> > >>>
> > >>> Each action will then be exposed using an ingress with its
specific>
> > >>> invocation path.>
> > >>>>
> > >>> If the community agrees with this plan, I would create a repo>
> > >>> `incubator-openwhisk-knative` to work on it.>
> > >>>>
> > >>> Thoughts?>
> >>
> >>
>
Re: Re: A plan to (re) implement OpenWhisk on top of Knative
Posted by Martin Henke <ma...@web.de>.
Hi Matt,
I fully agree with your thoughts. We should definitely (stepwise) aim for an unified model for all runtimes.
Regards,
Martin
> On 3. Jul 2019, at 16:43, Matt Rutkowski <mr...@us.ibm.com> wrote:
>
> Hi Martin,
>
> It is my belief that Michele, now freshly returned from book editing
> (congrats), was going to implement the same interface for pre/post
> processing requests that we implemented for NodeJS. This would allow us a
> path to support this across all language runtimes as well as formalize our
> runtime contract that aligns with a Knative Service approach and then also
> allows us to better explore some of the use cases being discussed on
> another thread from Rodric. I believe that we should look at smaller
> steps towards that alignment like removing the need for the "init"
> entrypoint and allowing access to parameters either by env. vars. or
> function arguments allowing both compat. with 12-factor app approach, as
> well as attempting to encourage a functional programming model where you
> should ideally be unaware of any system environment.
>
> Kind regards,
> Matt
>
>
>
>
> From: Martin Henke <ma...@web.de>
> To: dev@openwhisk.apache.org
> Date: 07/03/2019 09:29 AM
> Subject: [EXTERNAL] Re: A plan to (re) implement OpenWhisk on top
> of Knative
>
>
>
> Michele,
>
> FYI: Sugandha and myself published a Medium blog article which describes
> to build Nodejs10 images using Tekton and run it on Knative
> (based on the work from Priti).
> It might be on interest in the context of your Actionloop related Knative
> work.
>
> Link:
> https://urldefense.proofpoint.com/v2/url?u=https-3A__medium.com_-40sugandha.agrawal18_build-2Dknative-2Dservice-2Dwith-2Dtekton-2Dand-2Dapache-2Dopenwhisk-2Dnode-2Djs-2Druntime-2Df660bbc3a11e&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=6zQLM7Gc0Sv1iwayKOKa4_SFxRIxS478q2gZlAJj4Zw&m=bl7bwPaoiSE6fi0fyVHNC9bNUnh1ZUUlfl3lwUZbcH0&s=IQBGPfSTPiAjKhIlsqKDpAjoJBNd7By1YnjxOIx2454&e=
>
>
> Regards,
> Martin
>
>
> On 2019/05/20 15:00:02, "Michele Sciabarra" <m....@sciabarra.com> wrote:
>> Ok great, I see the discussion is starting to bring ideas.>
>>
>> Yes my goal is basically to run existing actions in Knative, create and
> invoke. And possibile retain the ability of an action to invoke another
> action.>
>>
>> I understand the different way they expose services, so I am rethinking
> the idea of using a "work-alike" path. >
>>
>> If it is needed we can add it with an ingress but it may be not
> necessary in the initial implementation.>
>>
>> Also I checked a bit ML and discussions and I see this Tekton thing that
> should be the preferred way.>
>>
>> Not sure if I understand the relation with the current Build API
> documented in the website. Is Tekton "compatible" or it has a different
> API?>
>>
>>
>> -- >
>> Michele Sciabarra>
>> michele@sciabarra.com>
>>
>> ----- Original message ----->
>> From: "Markus Thömmes" <ma...@apache.org>>
>> To: dev@openwhisk.apache.org>
>> Subject: Re: A plan to (re) implement OpenWhisk on top of Knative>
>> Date: Monday, May 20, 2019 4:50 PM>
>>
>> Good discussion, thanks!>
>>
>> Can we try to define what the desired end-goal is here? I'm a bit
> unclear>
>> what resembling the OpenWhisk API actually buys us.>
>>
>> To me, the desired end-state would be to run OpenWhisk actions as-is on
> a>
>> Knative cluster (similar to OpenFaaS' and Azure's integration). There's
> no>
>> good way for us to provide the full API without spinning up a control
> plane>
>> and we can only handle so much via the CLI. So to me, the end-goal
> looks>
>> like:>
>>
>> 1. *wsk action create* actually doing all the pieces necessary to run a>
>
>> piece of code on Knative.>
>> 2. *wsk action invoke* doing some HTTP call under the hood to "invoke"
> that>
>> action. The action should be reachable via a sensible URL. If we really>
>
>> want to keep the API surface (as I said, I'm dubious here) we can also
> do>
>> that via ingress level abstractions (like VirtualService).>
>>
>> Cheers,>
>> Markus>
>>
>> Am Mo., 20. Mai 2019 um 15:33 Uhr schrieb Martin Henke
> <ma...@web.de>
>>> :>
>>
>>>>
>>>> On 20. May 2019, at 14:55, Michele Sciabarra <mi...@sciabarra.com>>
>>> wrote:>
>>>>>
>>>>> Michele,>
>>>>>
>>>>> I like the idea to make the ActionLoop based runtimes to be
> runnable on>
>>> Knative.>
>>>>>>
>>>>> My thoughts on this:>
>>>>> - I second Markus concern to implement the invocation API onto
> Knative>
>>> instead of just using Knative service syntax.>
>>>> Can you elaborate this? I do not understand.>
>>>>
>>> Knative service syntax: https://<service(in our case =>
>>> action)>.<namespace>.<host>/>
>>> OW invocation https://>
>>> <api-host>/api/v1/namespaces/<namespace>/actions/<action>>
>>>>
>>> (I personally so no worth in inventing a distinct API for OW images,
> but>
>>> as said I would see that as a valid optional feature)>
>>>>
>>>>>
>>>>> - I would have concerns to make it dependent on Gloo which is kind
> of a>
>>> minority choice for Knative load balancing>
>>>> I do not think it will be hard to setup a test also using Istio, I
> do>
>>> not want to be limited to Gloo.>
>>>>
>>> I just wanted to prevent that Gloo gets a “official” prerequisite for
> an>
>>> “official” OW on Knative flow.>
>>> It is of course free to you to use what ever you want to do in your>
>>> prototype.>
>>>>
>>>> - In my opinion the goal should be to have some uniform behaviour
> for>
>>> ActionLoop based runtimes>
>>>>> and other ones like the adapted NodeJS runtimes demonstrated by
> Matt>
>>> and Priti>
>>>> As much as I can tell the current implementation is just the
> building>
>>> and exposing the "/init" and "/run" but I can be wrong.>
>>>> The build can be of course reused, so it continues the effort. For
> the>
>>> frontend, from the documentation I think Matt wants to add a proxy,
> while I>
>>> would like to implemeent the "invocation" straight in the runtime.
> This is>
>>> open to discussion, but of course it is better to reach an agreement.>
>
>>>>
>>> Also in the work of Priti and Matt the invocation goes directly to
> the>
>>> runtime. The action code is either passed with the call (not yet
> tested by>
>>> me) or set via environment variable in the docker build.>
>>>>
>>>>>
>>>>> - As Knative Build seems be on a dead end I would propose to
> target>
>>> Tekton as the build system (which developed as kind of >successor out
> of>
>>> Knative)>
>>>>>
>>>> If Knative build is dead then it would be a bit unfair that they
> change>
>>> it as the scope of the Knative project!>
>>>> It looks like the goal is to setup some standards! And I would be
> very>
>>> disappointed to know that.>
>>>>
>>> Tekton evolved out of Knative Build (or more correct out of Knative>
>>> Pipelines) but is very similar to the Knative build.>
>>> Flows can easily be ported from one to the other,>
>>> If we target Tekton build we would target the platform were the
> Knative>
>>> build team is focusing on.>
>>> But again feel free to use whatever platform for your prototype work.>
>
>>>>
>>>> At this stage the build is the more interesting thing, and it could
> be>
>>> even imported in main openwhisk to speed up deployment.>
>>>> I have already baked it in the ActionLoop runtimes (with
> precompilation).>
>>>> Also if we use Tekton, where is the Knative standard then? What is
> the>
>>> point? We can build our own system instead of "Knativizing" it...>
>>>>>
>>>>> Maybe it would be a good solution to tackle two things
> independently.>
>>>>> 1) Design and implement a common protocol of building, running and>
>
>>> calling OW runtimes on Knative>
>>>>> 2) Implement the OW invocation API on top of Knative as an
> additional>
>>> option for those who have the need to expose it.>
>>>>>
>>>> On this, for my personal approach at building things, I want
> something>
>>> that works and it is complete and useful. A "MVP”.>
>>>>
>>> Cool. Just go on.>
>>>>
>>>> So I do not plan to split the effort. Version 0.1 must be a minimal>
>
>>> working subset of OpenWhisk on Knative.>
>>>> Because otherwise there will be incomplete useless inusable pieces>
>>> around (see for example kwsk).>
>>>>>
>>>> It does not mean that things cannot be modular, nor that everyone
> must>
>>> but to me "openwhisk-knative" must be a single repo with all the
> pieces to>
>>> make something where you can download is and deploy in a kubernetes
> cluster>
>>> and be able to deploy simple actions. When this works, we can improve>
>
>>> incrementally and split it but keeping it working.>
>>>>>
>>>>> I would looking forward to work with you on the first work item.>
>>>> Great but I see now more details to discuss before we can start.
> Most>
>>> notably I need to understand how I can build on top of Mark and Priti
> work>
>>> and continue their work. ANd I can even probably recover some of the
> code>
>>> of kwsk as they implemented some openwhisk api, that I want now in
> the>
>>> runtime.>
>>>>>
>>>>
>>> I do not want to stop you in any way. My hope is that the action loop>
>
>>> runtimes and the “other ones” do expose the same behaviour when being>
>
>>> called. So that the users is not surprised when calling different
> actions>
>>> in different languages.>
>>> And behaving the same way might also mean to adapt the “other
> languages”>
>>> to the same behaviour as the action loop based ones.>
>>> They just should be uniform to be used.>
>>>>
>>> When your prototype is accessible it would be a good point of time to>
>
>>> discuss this.>
>>>>
>>> As said I very much like your effort.>
>>>>
>>>>>
>>>>> On 20. May 2019, at 08:55, Michele Sciabarra <mi...@sciabarra.com>>
>
>>> wrote:>
>>>>>>
>>>>>>
>>>>>>> I have an idea for implementing a prototype of OpenWhisk on top
> of>
>>> Knative.>
>>>>>>> My basic ideas are: do not use any proxy, forwarding or adapter:>
>
>>> extend>
>>>>>>> the runtime to support the REST call and expose them as ingress.
> And>
>>> use a>
>>>>>>> wrapper on top of `kubectl` to generate all the needed
> components.>
>>>>>>
>>>>>> Does this tie into the work that Matt was doing to the runtimes to
> make>
>>>>>> them runnable on Knative? Is this lined up with that at all?>
>>>>> Actually yes. He suggested I can investigate how to migrate
> ActionLoop>
>>> to port many other languages to Knative.>
>>>>> Also he recommended I add my idea and this is what I am doing.
> Current>
>>> code is, if I am not wrong, a Knative build of the nodejs runtime.>
>>>>>>
>>>>> There has been a number of attempts and proposal to move forward>
>>> OpenWhisk. My idea that to succeed we need something small but that
> just>
>>> works. This is my idea to be able to implement in the shorter time
> frame>
>>> possible an actual subset of OpenWhisk that works and it is truly
> built on>
>>> top of Knative. So I am putting the thing a bit further than Matt
> work.>
>>>>>>
>>>>>>
>>>>>>> My goal is to have a functional work-alike of OpenWhisk built on
> top>
>>> of>
>>>>>>> Knative, using ActionLoop as a foundation. I will extend
> ActionLoop to>
>>>>>>> support the required REST calls of OpenWhisk.>
>>>>>>>
>>>>>>> I also want to create tool, I will call `wskn`. This tool will>
>>> initially>
>>>>>>> just a python script, a wrapper on top of `kubectl` as it will>
>>> generate>
>>>>>>> kubernetes descriptors.>
>>>>>> Why not build this into "wsk" itself? The Azure Functions CLI as
> an>
>>> example>
>>>>>> supports multiple deployment types like this in one CLI.>
>>>>>>
>>>>> When it will works, yes, of course. But to start, what I really
> need is>
>>> a prototype that can generate kubernetes descripttors to feed to
> kubectl,>
>>> so a simplee, quick and ditry, separate tool (that I will keep
> together>
>>> the runtime) is all I need for now.>
>>>>>>
>>>>>>>>
>>>>>>> It will support initially just the the action creation and>
>>> invocation, and>
>>>>>>> only synchronous (blocking) behaviour, as all the request will
> go>
>>> straight>
>>>>>>> to the runtimes. Hopefully also a subset of `package` and>
>>> `activation`.>
>>>>>>> Again triggers, rules, asynchronous for later.>
>>>>>>>>
>>>>>>> The idea is that you will be able to create actions and web
> actions>
>>> that>
>>>>>>> can run existing OpenWhisk actions, at least those with blocking>
>
>>> behaviour>
>>>>>>> that run with ActionLoop (Go, Java, Python, PHP, Swift, Rust,
> Ruby,>
>>>>>>> Crystal...)>
>>>>>>>>
>>>>>>> Implementation.>
>>>>>>> ==============>
>>>>>>>>
>>>>>>> This is how I plan to implement it.>
>>>>>>>>
>>>>>>> At this stage I want to use just Knative Serving and Knative
> Build,>
>>> using>
>>>>>>> Gloo for the ingress part. I also plan to install a local Docker>
>
>>> registry>
>>>>>>> Kubernetes registry, so we do not have to use DockerHub for>
>>> everything. All>
>>>>>>> of this can be done with existing command line tools in a few
> minutes>
>>> in>
>>>>>>> any running Kubernetes deployment.>
>>>>>>>>
>>>>>>
>>>>>> Why specifying Gloo here? Do you need anything specific from Gloo>
>
>>> itself?>
>>>>>> If not I'd propose to just keep it on a Knative Serving API
> surface>
>>> level.>
>>>>> I want to build it on top of Knative serving, full stop.
> Currently,>
>>> installing Gloo is pretty easy and is more lightweight than Istio so
> I>
>>> will use it for my first implementation.>
>>>>>>
>>>>>>>>
>>>>>>> When I create an action, it will use Knative build that will
> work>
>>> roughly>
>>>>>>> in this way:>
>>>>>>>>
>>>>>>> - create a configmap with the action code>
>>>>>>> - build the actin using ActionLoop precompilation feature that
> will>
>>> return>
>>>>>>> a zip file including all the needed to run the action>
>>>>>>> - create a new docker image extending the runtime with the new
> zip,>
>>> using>
>>>>>>> Kaanico>
>>>>>>> - push the image in the local registry>
>>>>>> This feels like a fairly heavyweight process, we should be able
> to>
>>> come up>
>>>>>> with a way to circumvent zipping entirely. Maybe the runtime can
> detect>
>>>>>> that the unzipped content is already there and skip the unzip
> step?>
>>>>>>
>>>>> Actually this is my first idea of how to use Knative build. And is
> not>
>>> complicated: when I create the action, a run a build that includes
> Kanico.>
>>> I generate a Dockerfile on the fly. The docker file uses the action
> runtime>
>>> that already know how to compile a script. And then I save an image.
> I>
>>> already implemented un "autoinit" so just launching the image will
> give a>
>>> runtime ready to run that execute an action already compiled.>
>>>>>>
>>>>>>
>>>>>> I'm fairly hesitant on the usage of a ConfigMap for storing the
> action>
>>>>>> code. It's all stored in the in-cluster etcd instance and it has
> a>
>>> limit of>
>>>>>> 1M. This is at most a stop-gap solution to provide a PoC I think.
> Any>
>>> ideas>
>>>>>> on how to "productize" this?>
>>>>>>
>>>>> ConfigMap can be mounted as files, so it is an easy way to feed
> an>
>>> action to a build. It is just an easy way to feed the action code to
> the>
>>> Build.>
>>>>>>
>>>>> My initial constraint is that I want just to generate Kubernetes>
>>> descriptors to feed to kubectl.>
>>>>> Of course in the long run I can add some "file upload" storage.>
>>>>>>
>>>>> If I could to this file upload when invoking a build it could ideal
> as>
>>> I do not have to store anything anywhere, just process the code and>
>>> generate a single layer to execute actions to be store in the
> registry.>
>>>>> I will investigate better this area, I understand your concern.>
>>>>>>
>>>>>>>
>>>>>>> At this point you can run the action. ActionLoop will be extended
> to>
>>>>>>> support invocations in the format>
>>>>>>> "/v1/namespaces/namespace/actions/package/action".>
>>>>>> Why bother reimplementing this exact path? To obtain API
> compatibility>
>>> with>
>>>>>> OpenWhisk as it is today?>
>>>>>>
>>>>> I want to implement a subset of the OpenWhisk API on top of
> Knative>
>>> serving.>
>>>>> Knative serving already does the scaling and routing, so what we
> need>
>>> are the "endpoints" to invoke actions.>
>>>>>>
>>>>> Since I do not want to add additional components, not at the first>
>
>>> stage. Just knative serve and build, the runtime and a controller
> script,>
>>> the runtime is the natural place where to "handle" the API
> invocations,>
>>> since Knative only generates the URL but not anything else. If I>
>>> understood well, Matt is adding a proxy. I do not want to add a proxy,
> just>
>>> add to the runtime the ability to respond to "API like" calls, at
> least>
>>> those regarding action invocation.>
>>>>>>
>>>>>>> It will do all the decoding required to invoke the action with
> the>
>>>>>>> expected paramenters (straight invocation thrhoug the actinloop>
>>> protocol,>
>>>>>>> not proxies).>
>>>>>> Does this mean moving all of the Controller's "smartness" about>
>>> incoming>
>>>>>> and outgoing HTTP requests (see the whole WebActions for
> example)?>
>>>>>>
>>>>> At least decoding web actions in the runtime, yes. Knative serving>
>
>>> already has routing and proxying.>
>>>>> So a true implementation on top of Knative requires IHMO this>
>>> sacrifice. Unless there is a way to keep the controller in a
> "Knative">
>>> compatible way. Open to suggestions here.>
>>>>>>
>>>>>> Each action will then be exposed using an ingress with its
> specific>
>>>>>> invocation path.>
>>>>>>>
>>>>>> If the community agrees with this plan, I would create a repo>
>>>>>> `incubator-openwhisk-knative` to work on it.>
>>>>>>>
>>>>>> Thoughts?>
>>>>
>>>>
>>
>
>
>
>
Re: Re: A plan to (re) implement OpenWhisk on top of Knative
Posted by Matt Rutkowski <mr...@us.ibm.com>.
Hi Martin,
It is my belief that Michele, now freshly returned from book editing
(congrats), was going to implement the same interface for pre/post
processing requests that we implemented for NodeJS. This would allow us a
path to support this across all language runtimes as well as formalize our
runtime contract that aligns with a Knative Service approach and then also
allows us to better explore some of the use cases being discussed on
another thread from Rodric. I believe that we should look at smaller
steps towards that alignment like removing the need for the "init"
entrypoint and allowing access to parameters either by env. vars. or
function arguments allowing both compat. with 12-factor app approach, as
well as attempting to encourage a functional programming model where you
should ideally be unaware of any system environment.
Kind regards,
Matt
From: Martin Henke <ma...@web.de>
To: dev@openwhisk.apache.org
Date: 07/03/2019 09:29 AM
Subject: [EXTERNAL] Re: A plan to (re) implement OpenWhisk on top
of Knative
Michele,
FYI: Sugandha and myself published a Medium blog article which describes
to build Nodejs10 images using Tekton and run it on Knative
(based on the work from Priti).
It might be on interest in the context of your Actionloop related Knative
work.
Link:
https://urldefense.proofpoint.com/v2/url?u=https-3A__medium.com_-40sugandha.agrawal18_build-2Dknative-2Dservice-2Dwith-2Dtekton-2Dand-2Dapache-2Dopenwhisk-2Dnode-2Djs-2Druntime-2Df660bbc3a11e&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=6zQLM7Gc0Sv1iwayKOKa4_SFxRIxS478q2gZlAJj4Zw&m=bl7bwPaoiSE6fi0fyVHNC9bNUnh1ZUUlfl3lwUZbcH0&s=IQBGPfSTPiAjKhIlsqKDpAjoJBNd7By1YnjxOIx2454&e=
Regards,
Martin
On 2019/05/20 15:00:02, "Michele Sciabarra" <m....@sciabarra.com> wrote:
> Ok great, I see the discussion is starting to bring ideas.>
>
> Yes my goal is basically to run existing actions in Knative, create and
invoke. And possibile retain the ability of an action to invoke another
action.>
>
> I understand the different way they expose services, so I am rethinking
the idea of using a "work-alike" path. >
>
> If it is needed we can add it with an ingress but it may be not
necessary in the initial implementation.>
>
> Also I checked a bit ML and discussions and I see this Tekton thing that
should be the preferred way.>
>
> Not sure if I understand the relation with the current Build API
documented in the website. Is Tekton "compatible" or it has a different
API?>
>
>
> -- >
> Michele Sciabarra>
> michele@sciabarra.com>
>
> ----- Original message ----->
> From: "Markus Thömmes" <ma...@apache.org>>
> To: dev@openwhisk.apache.org>
> Subject: Re: A plan to (re) implement OpenWhisk on top of Knative>
> Date: Monday, May 20, 2019 4:50 PM>
>
> Good discussion, thanks!>
>
> Can we try to define what the desired end-goal is here? I'm a bit
unclear>
> what resembling the OpenWhisk API actually buys us.>
>
> To me, the desired end-state would be to run OpenWhisk actions as-is on
a>
> Knative cluster (similar to OpenFaaS' and Azure's integration). There's
no>
> good way for us to provide the full API without spinning up a control
plane>
> and we can only handle so much via the CLI. So to me, the end-goal
looks>
> like:>
>
> 1. *wsk action create* actually doing all the pieces necessary to run a>
> piece of code on Knative.>
> 2. *wsk action invoke* doing some HTTP call under the hood to "invoke"
that>
> action. The action should be reachable via a sensible URL. If we really>
> want to keep the API surface (as I said, I'm dubious here) we can also
do>
> that via ingress level abstractions (like VirtualService).>
>
> Cheers,>
> Markus>
>
> Am Mo., 20. Mai 2019 um 15:33 Uhr schrieb Martin Henke
<ma...@web.de>
> >:>
>
> >>
> > > On 20. May 2019, at 14:55, Michele Sciabarra <mi...@sciabarra.com>>
> > wrote:>
> > >>
> > >> Michele,>
> > >>
> > >> I like the idea to make the ActionLoop based runtimes to be
runnable on>
> > Knative.>
> > >>>
> > >> My thoughts on this:>
> > >> - I second Markus concern to implement the invocation API onto
Knative>
> > instead of just using Knative service syntax.>
> > > Can you elaborate this? I do not understand.>
> >>
> > Knative service syntax: https://<service(in our case =>
> > action)>.<namespace>.<host>/>
> > OW invocation https://>
> > <api-host>/api/v1/namespaces/<namespace>/actions/<action>>
> >>
> > (I personally so no worth in inventing a distinct API for OW images,
but>
> > as said I would see that as a valid optional feature)>
> >>
> > >>
> > >> - I would have concerns to make it dependent on Gloo which is kind
of a>
> > minority choice for Knative load balancing>
> > > I do not think it will be hard to setup a test also using Istio, I
do>
> > not want to be limited to Gloo.>
> >>
> > I just wanted to prevent that Gloo gets a “official” prerequisite for
an>
> > “official” OW on Knative flow.>
> > It is of course free to you to use what ever you want to do in your>
> > prototype.>
> >>
> > > - In my opinion the goal should be to have some uniform behaviour
for>
> > ActionLoop based runtimes>
> > >> and other ones like the adapted NodeJS runtimes demonstrated by
Matt>
> > and Priti>
> > > As much as I can tell the current implementation is just the
building>
> > and exposing the "/init" and "/run" but I can be wrong.>
> > > The build can be of course reused, so it continues the effort. For
the>
> > frontend, from the documentation I think Matt wants to add a proxy,
while I>
> > would like to implemeent the "invocation" straight in the runtime.
This is>
> > open to discussion, but of course it is better to reach an agreement.>
> >>
> > Also in the work of Priti and Matt the invocation goes directly to
the>
> > runtime. The action code is either passed with the call (not yet
tested by>
> > me) or set via environment variable in the docker build.>
> >>
> > >>
> > >> - As Knative Build seems be on a dead end I would propose to
target>
> > Tekton as the build system (which developed as kind of >successor out
of>
> > Knative)>
> > >>
> > > If Knative build is dead then it would be a bit unfair that they
change>
> > it as the scope of the Knative project!>
> > > It looks like the goal is to setup some standards! And I would be
very>
> > disappointed to know that.>
> >>
> > Tekton evolved out of Knative Build (or more correct out of Knative>
> > Pipelines) but is very similar to the Knative build.>
> > Flows can easily be ported from one to the other,>
> > If we target Tekton build we would target the platform were the
Knative>
> > build team is focusing on.>
> > But again feel free to use whatever platform for your prototype work.>
> >>
> > > At this stage the build is the more interesting thing, and it could
be>
> > even imported in main openwhisk to speed up deployment.>
> > > I have already baked it in the ActionLoop runtimes (with
precompilation).>
> > > Also if we use Tekton, where is the Knative standard then? What is
the>
> > point? We can build our own system instead of "Knativizing" it...>
> > >>
> > >> Maybe it would be a good solution to tackle two things
independently.>
> > >> 1) Design and implement a common protocol of building, running and>
> > calling OW runtimes on Knative>
> > >> 2) Implement the OW invocation API on top of Knative as an
additional>
> > option for those who have the need to expose it.>
> > >>
> > > On this, for my personal approach at building things, I want
something>
> > that works and it is complete and useful. A "MVP”.>
> >>
> > Cool. Just go on.>
> >>
> > > So I do not plan to split the effort. Version 0.1 must be a minimal>
> > working subset of OpenWhisk on Knative.>
> > > Because otherwise there will be incomplete useless inusable pieces>
> > around (see for example kwsk).>
> > >>
> > > It does not mean that things cannot be modular, nor that everyone
must>
> > but to me "openwhisk-knative" must be a single repo with all the
pieces to>
> > make something where you can download is and deploy in a kubernetes
cluster>
> > and be able to deploy simple actions. When this works, we can improve>
> > incrementally and split it but keeping it working.>
> > >>
> > >> I would looking forward to work with you on the first work item.>
> > > Great but I see now more details to discuss before we can start.
Most>
> > notably I need to understand how I can build on top of Mark and Priti
work>
> > and continue their work. ANd I can even probably recover some of the
code>
> > of kwsk as they implemented some openwhisk api, that I want now in
the>
> > runtime.>
> > >>
> >>
> > I do not want to stop you in any way. My hope is that the action loop>
> > runtimes and the “other ones” do expose the same behaviour when being>
> > called. So that the users is not surprised when calling different
actions>
> > in different languages.>
> > And behaving the same way might also mean to adapt the “other
languages”>
> > to the same behaviour as the action loop based ones.>
> > They just should be uniform to be used.>
> >>
> > When your prototype is accessible it would be a good point of time to>
> > discuss this.>
> >>
> > As said I very much like your effort.>
> >>
> > >>
> > >> On 20. May 2019, at 08:55, Michele Sciabarra <mi...@sciabarra.com>>
> > wrote:>
> > >>>
> > >>>
> > >>>> I have an idea for implementing a prototype of OpenWhisk on top
of>
> > Knative.>
> > >>>> My basic ideas are: do not use any proxy, forwarding or adapter:>
> > extend>
> > >>>> the runtime to support the REST call and expose them as ingress.
And>
> > use a>
> > >>>> wrapper on top of `kubectl` to generate all the needed
components.>
> > >>>
> > >>> Does this tie into the work that Matt was doing to the runtimes to
make>
> > >>> them runnable on Knative? Is this lined up with that at all?>
> > >> Actually yes. He suggested I can investigate how to migrate
ActionLoop>
> > to port many other languages to Knative.>
> > >> Also he recommended I add my idea and this is what I am doing.
Current>
> > code is, if I am not wrong, a Knative build of the nodejs runtime.>
> > >>>
> > >> There has been a number of attempts and proposal to move forward>
> > OpenWhisk. My idea that to succeed we need something small but that
just>
> > works. This is my idea to be able to implement in the shorter time
frame>
> > possible an actual subset of OpenWhisk that works and it is truly
built on>
> > top of Knative. So I am putting the thing a bit further than Matt
work.>
> > >>>
> > >>>
> > >>>> My goal is to have a functional work-alike of OpenWhisk built on
top>
> > of>
> > >>>> Knative, using ActionLoop as a foundation. I will extend
ActionLoop to>
> > >>>> support the required REST calls of OpenWhisk.>
> > >>>>
> > >>>> I also want to create tool, I will call `wskn`. This tool will>
> > initially>
> > >>>> just a python script, a wrapper on top of `kubectl` as it will>
> > generate>
> > >>>> kubernetes descriptors.>
> > >>> Why not build this into "wsk" itself? The Azure Functions CLI as
an>
> > example>
> > >>> supports multiple deployment types like this in one CLI.>
> > >>>
> > >> When it will works, yes, of course. But to start, what I really
need is>
> > a prototype that can generate kubernetes descripttors to feed to
kubectl,>
> > so a simplee, quick and ditry, separate tool (that I will keep
together>
> > the runtime) is all I need for now.>
> > >>>
> > >>>>>
> > >>>> It will support initially just the the action creation and>
> > invocation, and>
> > >>>> only synchronous (blocking) behaviour, as all the request will
go>
> > straight>
> > >>>> to the runtimes. Hopefully also a subset of `package` and>
> > `activation`.>
> > >>>> Again triggers, rules, asynchronous for later.>
> > >>>>>
> > >>>> The idea is that you will be able to create actions and web
actions>
> > that>
> > >>>> can run existing OpenWhisk actions, at least those with blocking>
> > behaviour>
> > >>>> that run with ActionLoop (Go, Java, Python, PHP, Swift, Rust,
Ruby,>
> > >>>> Crystal...)>
> > >>>>>
> > >>>> Implementation.>
> > >>>> ==============>
> > >>>>>
> > >>>> This is how I plan to implement it.>
> > >>>>>
> > >>>> At this stage I want to use just Knative Serving and Knative
Build,>
> > using>
> > >>>> Gloo for the ingress part. I also plan to install a local Docker>
> > registry>
> > >>>> Kubernetes registry, so we do not have to use DockerHub for>
> > everything. All>
> > >>>> of this can be done with existing command line tools in a few
minutes>
> > in>
> > >>>> any running Kubernetes deployment.>
> > >>>>>
> > >>>
> > >>> Why specifying Gloo here? Do you need anything specific from Gloo>
> > itself?>
> > >>> If not I'd propose to just keep it on a Knative Serving API
surface>
> > level.>
> > >> I want to build it on top of Knative serving, full stop.
Currently,>
> > installing Gloo is pretty easy and is more lightweight than Istio so
I>
> > will use it for my first implementation.>
> > >>>
> > >>>>>
> > >>>> When I create an action, it will use Knative build that will
work>
> > roughly>
> > >>>> in this way:>
> > >>>>>
> > >>>> - create a configmap with the action code>
> > >>>> - build the actin using ActionLoop precompilation feature that
will>
> > return>
> > >>>> a zip file including all the needed to run the action>
> > >>>> - create a new docker image extending the runtime with the new
zip,>
> > using>
> > >>>> Kaanico>
> > >>>> - push the image in the local registry>
> > >>> This feels like a fairly heavyweight process, we should be able
to>
> > come up>
> > >>> with a way to circumvent zipping entirely. Maybe the runtime can
detect>
> > >>> that the unzipped content is already there and skip the unzip
step?>
> > >>>
> > >> Actually this is my first idea of how to use Knative build. And is
not>
> > complicated: when I create the action, a run a build that includes
Kanico.>
> > I generate a Dockerfile on the fly. The docker file uses the action
runtime>
> > that already know how to compile a script. And then I save an image.
I>
> > already implemented un "autoinit" so just launching the image will
give a>
> > runtime ready to run that execute an action already compiled.>
> > >>>
> > >>>
> > >>> I'm fairly hesitant on the usage of a ConfigMap for storing the
action>
> > >>> code. It's all stored in the in-cluster etcd instance and it has
a>
> > limit of>
> > >>> 1M. This is at most a stop-gap solution to provide a PoC I think.
Any>
> > ideas>
> > >>> on how to "productize" this?>
> > >>>
> > >> ConfigMap can be mounted as files, so it is an easy way to feed
an>
> > action to a build. It is just an easy way to feed the action code to
the>
> > Build.>
> > >>>
> > >> My initial constraint is that I want just to generate Kubernetes>
> > descriptors to feed to kubectl.>
> > >> Of course in the long run I can add some "file upload" storage.>
> > >>>
> > >> If I could to this file upload when invoking a build it could ideal
as>
> > I do not have to store anything anywhere, just process the code and>
> > generate a single layer to execute actions to be store in the
registry.>
> > >> I will investigate better this area, I understand your concern.>
> > >>>
> > >>>>
> > >>>> At this point you can run the action. ActionLoop will be extended
to>
> > >>>> support invocations in the format>
> > >>>> "/v1/namespaces/namespace/actions/package/action".>
> > >>> Why bother reimplementing this exact path? To obtain API
compatibility>
> > with>
> > >>> OpenWhisk as it is today?>
> > >>>
> > >> I want to implement a subset of the OpenWhisk API on top of
Knative>
> > serving.>
> > >> Knative serving already does the scaling and routing, so what we
need>
> > are the "endpoints" to invoke actions.>
> > >>>
> > >> Since I do not want to add additional components, not at the first>
> > stage. Just knative serve and build, the runtime and a controller
script,>
> > the runtime is the natural place where to "handle" the API
invocations,>
> > since Knative only generates the URL but not anything else. If I>
> > understood well, Matt is adding a proxy. I do not want to add a proxy,
just>
> > add to the runtime the ability to respond to "API like" calls, at
least>
> > those regarding action invocation.>
> > >>>
> > >>>> It will do all the decoding required to invoke the action with
the>
> > >>>> expected paramenters (straight invocation thrhoug the actinloop>
> > protocol,>
> > >>>> not proxies).>
> > >>> Does this mean moving all of the Controller's "smartness" about>
> > incoming>
> > >>> and outgoing HTTP requests (see the whole WebActions for
example)?>
> > >>>
> > >> At least decoding web actions in the runtime, yes. Knative serving>
> > already has routing and proxying.>
> > >> So a true implementation on top of Knative requires IHMO this>
> > sacrifice. Unless there is a way to keep the controller in a
"Knative">
> > compatible way. Open to suggestions here.>
> > >>>
> > >>> Each action will then be exposed using an ingress with its
specific>
> > >>> invocation path.>
> > >>>>
> > >>> If the community agrees with this plan, I would create a repo>
> > >>> `incubator-openwhisk-knative` to work on it.>
> > >>>>
> > >>> Thoughts?>
> >>
> >>
>