You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@openwhisk.apache.org by Michele Sciabarra <op...@sciabarra.com> on 2018/07/25 14:56:22 UTC
MiniWhisk: what you think?
Hello, in the process of developing some examples for the goproxy, I realized I want a tool to make easier developing go actions locally. While it is generally acceptable to deploy your actions straight to the IBM Cloud for example when you code in Javascript (or Python) it is less desiderable for Go because compilation time in the cloud is not so fast as it is when compiled locally, and you have the additional time of uploading a binary that is generally bigger than javascript actions.
SO I ended up with this idea of the "miniwhisk". I am posting here to see if it is acceptable or... there are better solutions.
My idea of the miniwhisk is a "single action " executor. It should work more or less this way:
$ miniwhisk /path/of/action -runtime openwhisk/actionloop-go-v1.10:master -watch *.go -build make -action demo
This command will launch the runtime "openwhisk/actionloop-go-v1.10:master" using docker run, then will watch the files specified with "-w". When a file changes, it will execute the build command (-build) and then execute and "init" of the action runtime, post the action to the runtime as an init.
Additional (and most importantly) it starts a webserver that will listen to /path/of/action for GET and POST and will then translate requests in appropriate /run posts for the runtime.
Basically it is a tool to develop an action in go locally simulating what would happen when run in the real OpenWhisk.
How does sound the idea? Is it worth the effort?
--
Michele Sciabarra
openwhisk@sciabarra.com
Re: MiniWhisk: what you think?
Posted by Michele Sciabarra <mi...@sciabarra.com>.
Share the code and let’s collaborate. If you remember I started the
Openwhisk runtime from your code, there are still a few lines of
code left :)
--
Michele Sciabarra
michele@sciabarra.com
On Thu, Jul 26, 2018, at 7:38 PM, James Thomas wrote:
> Michele,
>
> I'd also started a project this month called "miniwsk" around local
> development - great minds! :)
>
> I had a similar idea about an instance of the platform that
> would spin up> containers locally. My approach was to implement the OpenWhisk
> platform API> using a stub server that would execute the actions using Docker in
> the host> system. I also wanted to use Go lang for the mock server so that
> it would> be a simple binary I could start on demand. Implementing the
> API means I> can just use all the normal OpenWhisk tools without modification by
> pointing them to localhost.
>
> It's still in the super-early stages of development so I'm not
> sure when> I'll have it ready for publishing... I wasn't going to handle re-
> build go> based stuff as your idea does. It was purely a stub server to invoke
> containers on demand.
>
> On 25 July 2018 at 15:56, Michele Sciabarra
> <op...@sciabarra.com> wrote:>
> > Hello, in the process of developing some examples for the
> > goproxy, I> > realized I want a tool to make easier developing go actions
> > locally. While> > it is generally acceptable to deploy your actions straight to the
> > IBM Cloud> > for example when you code in Javascript (or Python) it is less
> > desiderable> > for Go because compilation time in the cloud is not so fast as it
> > is when> > compiled locally, and you have the additional time of uploading a
> > binary> > that is generally bigger than javascript actions.
> >
> > SO I ended up with this idea of the "miniwhisk". I am posting here
> > to see> > if it is acceptable or... there are better solutions.
> >
> > My idea of the miniwhisk is a "single action " executor. It
> > should work> > more or less this way:
> >
> > $ miniwhisk /path/of/action -runtime openwhisk/actionloop-go-
> > v1.10:master> > -watch *.go -build make -action demo
> >
> > This command will launch the runtime "openwhisk/actionloop-go-
> > v1.10:master"> > using docker run, then will watch the files specified with "-w".
> > When a> > file changes, it will execute the build command (-build) and then
> > execute> > and "init" of the action runtime, post the action to the runtime as
> > an init.> >
> > Additional (and most importantly) it starts a webserver that will
> > listen> > to /path/of/action for GET and POST and will then translate
> > requests in> > appropriate /run posts for the runtime.
> >
> > Basically it is a tool to develop an action in go locally
> > simulating what> > would happen when run in the real OpenWhisk.
> >
> >
> > How does sound the idea? Is it worth the effort?
> >
> >
> >
> >
> >
> >
> >
> > --
> > Michele Sciabarra
> > openwhisk@sciabarra.com
> >
>
>
>
> --
> Regards,
> James Thomas
Re: MiniWhisk: what you think?
Posted by Michele Sciabarra <mi...@sciabarra.com>.
Great! It is a good start, indeed I want to prototype my idea with
existing tools. Then see what is possible to do to improve them. Waiting
for James ‘ miniwhisk’. Is the ‘light whisk’ for Iot available
somewhere?
--
Michele Sciabarra
michele@sciabarra.com
On Fri, Jul 27, 2018, at 7:39 AM, Michael Marth wrote:
> Michele,
>
> Fyi
> https://github.com/apache/incubator-openwhisk-devtools/tree/master/node-local> (obviously does not cover your Go use case - but since you asked what> else is around...)
>
> Cheers
> Michael
>
> On 27.07.18, 00:46, "Michele Sciabarra"
> <mi...@sciabarra.com> wrote:>
> Indeed after thinking about my idea was to try to extend the
> invoker.py
> to serve urls, and add a —watch feature so if files are modified
> then
> they are zipped and sent to the runtime as /init. Since the go
> runtime
> compiles already I think it is better to leave the compilation to> the
> runtime. Another interesting feature of the goproxy is that it
> supports
> the “unnecessary” multiple initializations so it should ne able to> do
> incremental compilation at least for go code.Another interesting
> feature that I am ‘stealing’ from the openwhisk
> shell is support for debuggers. I will try to make available delve> if I
> can. I suppose it is just enough to be able to run the client
> executable with delve and expose the debugger port. I think the
> openwhisk shell is awesome for node but I want something specific> for go
> that works with vscode, and let me run tests on the code as I
> develop
> it. And also the go debugger now.I plan to write the miniwhisk for> now in Python, as part of the examples
> for go. Then eventually rewrite it in go if it proves interesting> enough
> to became a standalone tool.
> Thoughts?
>
> --
> Michele Sciabarra
> michele@sciabarra.com
>
>
>
> On Thu, Jul 26, 2018, at 8:54 PM, Rodric Rabbah wrote:
> >
> > > My approach was to implement the OpenWhisk platform API
> > > using a stub server that would execute the actions using
> > > Docker> in
> > > the host> > system.
> >
> > You really don’t need this though - look at the invoker.py
> > script. That> is enough IMO, either extending that or copying
> > that> into a new
> > executable (go or node).
> >
> > -r
>
>
>
Re: MiniWhisk: what you think?
Posted by Michael Marth <mm...@adobe.com.INVALID>.
Michele,
Fyi https://github.com/apache/incubator-openwhisk-devtools/tree/master/node-local
(obviously does not cover your Go use case - but since you asked what else is around...)
Cheers
Michael
On 27.07.18, 00:46, "Michele Sciabarra" <mi...@sciabarra.com> wrote:
Indeed after thinking about my idea was to try to extend the invoker.py
to serve urls, and add a —watch feature so if files are modified then
they are zipped and sent to the runtime as /init. Since the go runtime
compiles already I think it is better to leave the compilation to the
runtime. Another interesting feature of the goproxy is that it supports
the “unnecessary” multiple initializations so it should ne able to do
incremental compilation at least for go code.Another interesting feature that I am ‘stealing’ from the openwhisk
shell is support for debuggers. I will try to make available delve if I
can. I suppose it is just enough to be able to run the client
executable with delve and expose the debugger port. I think the
openwhisk shell is awesome for node but I want something specific for go
that works with vscode, and let me run tests on the code as I develop
it. And also the go debugger now.I plan to write the miniwhisk for now in Python, as part of the examples
for go. Then eventually rewrite it in go if it proves interesting enough
to became a standalone tool.
Thoughts?
--
Michele Sciabarra
michele@sciabarra.com
On Thu, Jul 26, 2018, at 8:54 PM, Rodric Rabbah wrote:
>
> > My approach was to implement the OpenWhisk platform API
> > using a stub server that would execute the actions using Docker in
> > the host> > system.
>
> You really don’t need this though - look at the invoker.py
> script. That> is enough IMO, either extending that or copying that into a new
> executable (go or node).
>
> -r
Re: MiniWhisk: what you think?
Posted by Michele Sciabarra <mi...@sciabarra.com>.
Indeed after thinking about my idea was to try to extend the invoker.py
to serve urls, and add a —watch feature so if files are modified then
they are zipped and sent to the runtime as /init. Since the go runtime
compiles already I think it is better to leave the compilation to the
runtime. Another interesting feature of the goproxy is that it supports
the “unnecessary” multiple initializations so it should ne able to do
incremental compilation at least for go code.Another interesting feature that I am ‘stealing’ from the openwhisk
shell is support for debuggers. I will try to make available delve if I
can. I suppose it is just enough to be able to run the client
executable with delve and expose the debugger port. I think the
openwhisk shell is awesome for node but I want something specific for go
that works with vscode, and let me run tests on the code as I develop
it. And also the go debugger now.I plan to write the miniwhisk for now in Python, as part of the examples
for go. Then eventually rewrite it in go if it proves interesting enough
to became a standalone tool.
Thoughts?
--
Michele Sciabarra
michele@sciabarra.com
On Thu, Jul 26, 2018, at 8:54 PM, Rodric Rabbah wrote:
>
> > My approach was to implement the OpenWhisk platform API
> > using a stub server that would execute the actions using Docker in
> > the host> > system.
>
> You really don’t need this though - look at the invoker.py
> script. That> is enough IMO, either extending that or copying that into a new
> executable (go or node).
>
> -r
Re: MiniWhisk: what you think?
Posted by Rodric Rabbah <ro...@gmail.com>.
> My approach was to implement the OpenWhisk platform API
> using a stub server that would execute the actions using Docker in the host
> system.
You really don’t need this though - look at the invoker.py script. That is enough IMO, either extending that or copying that into a new executable (go or node).
-r
Re: MiniWhisk: what you think?
Posted by James Thomas <jt...@gmail.com>.
Michele,
I'd also started a project this month called "miniwsk" around local
development - great minds! :)
I had a similar idea about an instance of the platform that would spin up
containers locally. My approach was to implement the OpenWhisk platform API
using a stub server that would execute the actions using Docker in the host
system. I also wanted to use Go lang for the mock server so that it would
be a simple binary I could start on demand. Implementing the API means I
can just use all the normal OpenWhisk tools without modification by
pointing them to localhost.
It's still in the super-early stages of development so I'm not sure when
I'll have it ready for publishing... I wasn't going to handle re-build go
based stuff as your idea does. It was purely a stub server to invoke
containers on demand.
On 25 July 2018 at 15:56, Michele Sciabarra <op...@sciabarra.com> wrote:
> Hello, in the process of developing some examples for the goproxy, I
> realized I want a tool to make easier developing go actions locally. While
> it is generally acceptable to deploy your actions straight to the IBM Cloud
> for example when you code in Javascript (or Python) it is less desiderable
> for Go because compilation time in the cloud is not so fast as it is when
> compiled locally, and you have the additional time of uploading a binary
> that is generally bigger than javascript actions.
>
> SO I ended up with this idea of the "miniwhisk". I am posting here to see
> if it is acceptable or... there are better solutions.
>
> My idea of the miniwhisk is a "single action " executor. It should work
> more or less this way:
>
> $ miniwhisk /path/of/action -runtime openwhisk/actionloop-go-v1.10:master
> -watch *.go -build make -action demo
>
> This command will launch the runtime "openwhisk/actionloop-go-v1.10:master"
> using docker run, then will watch the files specified with "-w". When a
> file changes, it will execute the build command (-build) and then execute
> and "init" of the action runtime, post the action to the runtime as an init.
>
> Additional (and most importantly) it starts a webserver that will listen
> to /path/of/action for GET and POST and will then translate requests in
> appropriate /run posts for the runtime.
>
> Basically it is a tool to develop an action in go locally simulating what
> would happen when run in the real OpenWhisk.
>
>
> How does sound the idea? Is it worth the effort?
>
>
>
>
>
>
>
> --
> Michele Sciabarra
> openwhisk@sciabarra.com
>
--
Regards,
James Thomas
Re: MiniWhisk: what you think?
Posted by Michele Sciabarra <mi...@sciabarra.com>.
>Michele, this is a useful idea to ease local development. When it's not
>enough to test using the native runtime (nodejs. python, go) directly, I
>use the invoke.py script [1] to minic running inside the container.
Thats is a great information because I was wondering how to know exactly the invocation parameters.
I know a few, but not all. Of course I could rewrite it in Go :)
>There was a plugin for the cloud shell from IBM that did what you're
>suggesting as well and there may be a youtube video for this (@nick
>mitchell might know where it is, I couldn't find it easily).
There "was" or there "is"? No need to redo it if it is there. This is why I am asking here.
Some links to see the state and decide if it is worth to extend or it is good enough.
>The thought was to push this even further and instead of replicating the
>openwhisk API more generally, use a very light weight openwhisk like the
>one David and Pavel are alluding to.
Yes but I am not sure it is the same think. My tool would also compile so it would act like the typical development tools, I do not think the Light Whisk would do that as the purpose is different
>I do think this is a generally useful tool for local development and I
>think we have several parts to make it possible today. +1 from me.
Links please, duplicating is not my goal , if I can improve what is there it is better.
[1]
https://github.com/apache/incubator-openwhisk/blob/master/tools/actionProxy/invoke.py
-r
On Wed, Jul 25, 2018 at 5:17 PM, David Breitgand <DA...@il.ibm.com> wrote:
> Hi Michele,
>
> There is a PR we've submitted on Lean OpenWhisk (
> https://github.com/apache/incubator-openwhisk/pull/3886)
> There is also a blog here:
> https://medium.com/openwhisk/lean-openwhisk-open-source-
> faas-for-edge-computing-fb823c6bbb9b
>
> Would you like to look at it and see if it will be good enough for what
> you want to achieve.
> The idea is to have OW (an actual one based on the Controller and Invoker
> upstream), but with a very small memory footprint.
>
> Thanks.
>
> -- david
>
>
>
>
> From: Michele Sciabarra <op...@sciabarra.com>
> To: dev@openwhisk.apache.org
> Date: 25/07/2018 08:02 PM
> Subject: MiniWhisk: what you think?
>
>
>
> Hello, in the process of developing some examples for the goproxy, I
> realized I want a tool to make easier developing go actions locally. While
> it is generally acceptable to deploy your actions straight to the IBM
> Cloud for example when you code in Javascript (or Python) it is less
> desiderable for Go because compilation time in the cloud is not so fast as
> it is when compiled locally, and you have the additional time of uploading
> a binary that is generally bigger than javascript actions.
>
> SO I ended up with this idea of the "miniwhisk". I am posting here to see
> if it is acceptable or... there are better solutions.
>
> My idea of the miniwhisk is a "single action " executor. It should work
> more or less this way:
>
> $ miniwhisk /path/of/action -runtime openwhisk/actionloop-go-v1.10:master
>
> -watch *.go -build make -action demo
>
> This command will launch the runtime
> "openwhisk/actionloop-go-v1.10:master" using docker run, then will watch
> the files specified with "-w". When a file changes, it will execute the
> build command (-build) and then execute and "init" of the action runtime,
> post the action to the runtime as an init.
>
> Additional (and most importantly) it starts a webserver that will listen
> to /path/of/action for GET and POST and will then translate requests in
> appropriate /run posts for the runtime.
>
> Basically it is a tool to develop an action in go locally simulating what
> would happen when run in the real OpenWhisk.
>
>
> How does sound the idea? Is it worth the effort?
>
>
>
>
>
>
>
> --
> Michele Sciabarra
> openwhisk@sciabarra.com
>
>
>
>
>
>
Re: MiniWhisk: what you think?
Posted by Rodric Rabbah <ro...@gmail.com>.
Michele, this is a useful idea to ease local development. When it's not
enough to test using the native runtime (nodejs. python, go) directly, I
use the invoke.py script [1] to minic running inside the container.
There was a plugin for the cloud shell from IBM that did what you're
suggesting as well and there may be a youtube video for this (@nick
mitchell might know where it is, I couldn't find it easily).
The thought was to push this even further and instead of replicating the
openwhisk API more generally, use a very light weight openwhisk like the
one David and Pavel are alluding to.
I do think this is a generally useful tool for local development and I
think we have several parts to make it possible today. +1 from me.
[1]
https://github.com/apache/incubator-openwhisk/blob/master/tools/actionProxy/invoke.py
-r
On Wed, Jul 25, 2018 at 5:17 PM, David Breitgand <DA...@il.ibm.com> wrote:
> Hi Michele,
>
> There is a PR we've submitted on Lean OpenWhisk (
> https://github.com/apache/incubator-openwhisk/pull/3886)
> There is also a blog here:
> https://medium.com/openwhisk/lean-openwhisk-open-source-
> faas-for-edge-computing-fb823c6bbb9b
>
> Would you like to look at it and see if it will be good enough for what
> you want to achieve.
> The idea is to have OW (an actual one based on the Controller and Invoker
> upstream), but with a very small memory footprint.
>
> Thanks.
>
> -- david
>
>
>
>
> From: Michele Sciabarra <op...@sciabarra.com>
> To: dev@openwhisk.apache.org
> Date: 25/07/2018 08:02 PM
> Subject: MiniWhisk: what you think?
>
>
>
> Hello, in the process of developing some examples for the goproxy, I
> realized I want a tool to make easier developing go actions locally. While
> it is generally acceptable to deploy your actions straight to the IBM
> Cloud for example when you code in Javascript (or Python) it is less
> desiderable for Go because compilation time in the cloud is not so fast as
> it is when compiled locally, and you have the additional time of uploading
> a binary that is generally bigger than javascript actions.
>
> SO I ended up with this idea of the "miniwhisk". I am posting here to see
> if it is acceptable or... there are better solutions.
>
> My idea of the miniwhisk is a "single action " executor. It should work
> more or less this way:
>
> $ miniwhisk /path/of/action -runtime openwhisk/actionloop-go-v1.10:master
>
> -watch *.go -build make -action demo
>
> This command will launch the runtime
> "openwhisk/actionloop-go-v1.10:master" using docker run, then will watch
> the files specified with "-w". When a file changes, it will execute the
> build command (-build) and then execute and "init" of the action runtime,
> post the action to the runtime as an init.
>
> Additional (and most importantly) it starts a webserver that will listen
> to /path/of/action for GET and POST and will then translate requests in
> appropriate /run posts for the runtime.
>
> Basically it is a tool to develop an action in go locally simulating what
> would happen when run in the real OpenWhisk.
>
>
> How does sound the idea? Is it worth the effort?
>
>
>
>
>
>
>
> --
> Michele Sciabarra
> openwhisk@sciabarra.com
>
>
>
>
>
>
Re: MiniWhisk: what you think?
Posted by Michele Sciabarra <mi...@sciabarra.com>.
Sorry for the stupid question but which shell? bx ? wsk ?
Because the build part could be implemented using existing solutions (there are many)
--
Michele Sciabarra
michele@sciabarra.com
----- Original message -----
From: Nick Mitchell <mo...@gmail.com>
To: dev@openwhisk.apache.org
Subject: Re: MiniWhisk: what you think?
Date: Wed, 25 Jul 2018 14:35:37 -0400
the shell indeed has a `local` plugin that does most of this, though it
does not support the `go build` part; but it does take care of pulling the
base images, injecting code, and routes invokes to and from the container.
for nodejs, we can also optionally hand the local invocation off to the
chrome debugger.
Re: MiniWhisk: what you think?
Posted by Nick Mitchell <mo...@gmail.com>.
the shell indeed has a `local` plugin that does most of this, though it
does not support the `go build` part; but it does take care of pulling the
base images, injecting code, and routes invokes to and from the container.
for nodejs, we can also optionally hand the local invocation off to the
chrome debugger.
Re: MiniWhisk: what you think?
Posted by Michele Sciabarra <mi...@sciabarra.com>.
I have seen the Lean Whisk project and indeed it is very interesting.
What I am proposing here is not however a "lean" version of Whisk, it is a pure development tool.
A simple "runner" for runtimes as is, to be useful when you develop, especially with compiled languages like go, where even if is is possible to compile a go action on the fly, it takes too much time anyway to be practical.
--
Michele Sciabarra
michele@sciabarra.com
----- Original message -----
From: David Breitgand <DA...@il.ibm.com>
To: dev@openwhisk.apache.org
Cc: Pavel Kravchenco <KP...@il.ibm.com>
Subject: Re: MiniWhisk: what you think?
Date: Wed, 25 Jul 2018 20:17:00 +0300
Hi Michele,
There is a PR we've submitted on Lean OpenWhisk (
https://github.com/apache/incubator-openwhisk/pull/3886)
There is also a blog here:
https://medium.com/openwhisk/lean-openwhisk-open-source-faas-for-edge-computing-fb823c6bbb9b
Would you like to look at it and see if it will be good enough for what
you want to achieve.
The idea is to have OW (an actual one based on the Controller and Invoker
upstream), but with a very small memory footprint.
Thanks.
-- david
From: Michele Sciabarra <op...@sciabarra.com>
To: dev@openwhisk.apache.org
Date: 25/07/2018 08:02 PM
Subject: MiniWhisk: what you think?
Hello, in the process of developing some examples for the goproxy, I
realized I want a tool to make easier developing go actions locally. While
it is generally acceptable to deploy your actions straight to the IBM
Cloud for example when you code in Javascript (or Python) it is less
desiderable for Go because compilation time in the cloud is not so fast as
it is when compiled locally, and you have the additional time of uploading
a binary that is generally bigger than javascript actions.
SO I ended up with this idea of the "miniwhisk". I am posting here to see
if it is acceptable or... there are better solutions.
My idea of the miniwhisk is a "single action " executor. It should work
more or less this way:
$ miniwhisk /path/of/action -runtime openwhisk/actionloop-go-v1.10:master
-watch *.go -build make -action demo
This command will launch the runtime
"openwhisk/actionloop-go-v1.10:master" using docker run, then will watch
the files specified with "-w". When a file changes, it will execute the
build command (-build) and then execute and "init" of the action runtime,
post the action to the runtime as an init.
Additional (and most importantly) it starts a webserver that will listen
to /path/of/action for GET and POST and will then translate requests in
appropriate /run posts for the runtime.
Basically it is a tool to develop an action in go locally simulating what
would happen when run in the real OpenWhisk.
How does sound the idea? Is it worth the effort?
--
Michele Sciabarra
openwhisk@sciabarra.com
Re: MiniWhisk: what you think?
Posted by David Breitgand <DA...@il.ibm.com>.
Hi Michele,
There is a PR we've submitted on Lean OpenWhisk (
https://github.com/apache/incubator-openwhisk/pull/3886)
There is also a blog here:
https://medium.com/openwhisk/lean-openwhisk-open-source-faas-for-edge-computing-fb823c6bbb9b
Would you like to look at it and see if it will be good enough for what
you want to achieve.
The idea is to have OW (an actual one based on the Controller and Invoker
upstream), but with a very small memory footprint.
Thanks.
-- david
From: Michele Sciabarra <op...@sciabarra.com>
To: dev@openwhisk.apache.org
Date: 25/07/2018 08:02 PM
Subject: MiniWhisk: what you think?
Hello, in the process of developing some examples for the goproxy, I
realized I want a tool to make easier developing go actions locally. While
it is generally acceptable to deploy your actions straight to the IBM
Cloud for example when you code in Javascript (or Python) it is less
desiderable for Go because compilation time in the cloud is not so fast as
it is when compiled locally, and you have the additional time of uploading
a binary that is generally bigger than javascript actions.
SO I ended up with this idea of the "miniwhisk". I am posting here to see
if it is acceptable or... there are better solutions.
My idea of the miniwhisk is a "single action " executor. It should work
more or less this way:
$ miniwhisk /path/of/action -runtime openwhisk/actionloop-go-v1.10:master
-watch *.go -build make -action demo
This command will launch the runtime
"openwhisk/actionloop-go-v1.10:master" using docker run, then will watch
the files specified with "-w". When a file changes, it will execute the
build command (-build) and then execute and "init" of the action runtime,
post the action to the runtime as an init.
Additional (and most importantly) it starts a webserver that will listen
to /path/of/action for GET and POST and will then translate requests in
appropriate /run posts for the runtime.
Basically it is a tool to develop an action in go locally simulating what
would happen when run in the real OpenWhisk.
How does sound the idea? Is it worth the effort?
--
Michele Sciabarra
openwhisk@sciabarra.com