You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@openwhisk.apache.org by David Breitgand <DA...@il.ibm.com> on 2018/07/16 11:17:00 UTC
Proposing Lean OpenWhisk
Hi,
Pavel and I are working on lean OpenWhisk. The idea is to allow an
efficient use of OpenWhisk with small form factor compute nodes (e.g., IoT
gateways). The proposition is to get rid of Kafka, have controller and
invoker compiled together into a "lean" controller-invoker (have a Gradle
project for that) with controller calling the invoker's method directly
via an in-memory object (a queue, which is also loaded via SPI) via a new
pluggable load balancer that is selected via SPI using configuration file
property setting. Here is a blog
https://medium.com/@davidbr_9022/lean-openwhisk-open-source-faas-for-edge-computing-fb823c6bbb9b
providing some more detail and points to our fork.
An important thing is that we are _not_ proposing a new project. Rather we
want to submit a PR to allow for a lean distro of OpenWhisk that uses the
same OpenWhisk code base and consumes everything downstream by virtue of
exploiting SPI.
Thoughts?
Thanks.
-- david
===============================================================
David Breitgand, Ph. D.
Senior Researcher, IBM Research -- Haifa, Israel
Tel: +972-4-829-1007 | Mobile: +972 54 7277-881
"Ambition is the path to success, persistence is the vehicle you arrive
in", William Eardley IV
==============================================================
Re: Proposing Lean OpenWhisk
Posted by Rodric Rabbah <ro...@gmail.com>.
Glad to see this work reaching maturity. As Markus noted I think there could be more convergence down the line too.
Looking forward to the PR.
-r
> On Jul 16, 2018, at 4:17 AM, David Breitgand <DA...@il.ibm.com> wrote:
>
> Hi,
>
> Pavel and I are working on lean OpenWhisk. The idea is to allow an efficient use of OpenWhisk with small form factor compute nodes (e.g., IoT gateways). The proposition is to get rid of Kafka, have controller and invoker compiled together into a "lean" controller-invoker (have a Gradle project for that) with controller calling the invoker's method directly via an in-memory object (a queue, which is also loaded via SPI) via a new pluggable load balancer that is selected via SPI using configuration file property setting. Here is a blog https://medium.com/@davidbr_9022/lean-openwhisk-open-source-faas-for-edge-computing-fb823c6bbb9bproviding some more detail and points to our fork.
>
> An important thing is that we are _not_ proposing a new project. Rather we want to submit a PR to allow for a lean distro of OpenWhisk that uses the same OpenWhisk code base and consumes everything downstream by virtue of exploiting SPI.
>
> Thoughts?
>
> Thanks.
>
> -- david
> ===============================================================
> David Breitgand, Ph. D.
> Senior Researcher, IBM Research -- Haifa, Israel
> Tel: +972-4-829-1007 | Mobile: +972 54 7277-881
> "Ambition is the path to success, persistence is the vehicle you arrive in", William Eardley IV
> ==============================================================
>
Re: Proposing Lean OpenWhisk
Posted by Dominic Kim <st...@gmail.com>.
Dear David.
Pretty much make sense to me!
Thanks for the reply.
Best regards
Dominic
2018-07-17 22:39 GMT+09:00 david.breitgand@gmail.com <
david.breitgand@gmail.com>:
> Hi Dominic,
>
> Lean OpenWhisk is not supposed to run on the IoT devices such as sensors
> and actuators directly. It's supposed to run on a Gateway node that
> controls the sensors and actuators connected to it. Think AWS GreenGrass,
> Azure Functions on IoT Edge. This is the same use case. The data from a
> sensor, say a temperature sensor reading, will be sent to the Gateway via
> MQTT or HTTP or whatever and there will be a provider at the Gateway (say,
> an MQTT feed, which is outside of the OW core and this proposal) that can
> trigger an action on a trigger previously created via a feed action for
> this type of feed.
>
> This proposal is strictly about making OW a better fit for small Gateway
> form factors.
>
> It's true that there are some other tools we need to provide to make
> OW@Edge a feasible option for developers, but they are outside of the
> core and this specific proposal and merit a separate discussion.
>
> Cheers.
>
> -- david
>
> On 2018/07/16 11:40:35, Dominic Kim <st...@gmail.com> wrote:
> > Dear David.
> >
> > This is an awesome idea!!
> >
> > Is this to control IoT devices programmatically?
> > If yes, there would be many different types of IoT devices especially in
> > terms of their capabilities such as lighting sensors, thermometer, robot
> > cleaner, and so on.
> >
> > Then do you have anything in mind to take care of heterogeneous sets of
> > edge nodes?
> > There is a possibility that some actions should only run on thermometers,
> > while the others should run on lighting sensors.
> >
> > If you are trying to install "one-for-all" OpenWhisk cluster rather than
> > having separate OpenWhisk clusters for each device types, how will you
> > manage heterogenous container pools and properly assign relevant actions
> to
> > them?
> >
> >
> > Best regards,
> > Dominic
> >
> >
> > 2018-07-16 20:24 GMT+09:00 Markus Thoemmes <ma...@de.ibm.com>:
> >
> > > Hi David,
> > >
> > > please send your PR for sure! IIRC we made the Loadbalancer pluggable
> > > specifically for this use-case. Sounds like a great addition to our
> > > possible deployment topologies.
> > >
> > > Shameless plug: Would you review the architecture I proposed here:
> > > https://lists.apache.org/thread.html/29289006d190b2c68451f7625c13bb
> > > 8020cc8e9928db66f1b0def18e@%3Cdev.openwhisk.apache.org%3E
> > >
> > > In theory, this could make your proposal even leaner in the future.
> Don't
> > > hear me say though we should hold this back, we can absolutely go
> forward
> > > with your implementation first. Just wanted to quickly verify this
> use-case
> > > will also work with what we might plan for in the future.
> > >
> > > Cheers,
> > > Markus
> > >
> > >
> >
>
Re: Proposing Lean OpenWhisk
Posted by da...@gmail.com,
da...@gmail.com.
Hi Dominic,
Lean OpenWhisk is not supposed to run on the IoT devices such as sensors and actuators directly. It's supposed to run on a Gateway node that controls the sensors and actuators connected to it. Think AWS GreenGrass, Azure Functions on IoT Edge. This is the same use case. The data from a sensor, say a temperature sensor reading, will be sent to the Gateway via MQTT or HTTP or whatever and there will be a provider at the Gateway (say, an MQTT feed, which is outside of the OW core and this proposal) that can trigger an action on a trigger previously created via a feed action for this type of feed.
This proposal is strictly about making OW a better fit for small Gateway form factors.
It's true that there are some other tools we need to provide to make OW@Edge a feasible option for developers, but they are outside of the core and this specific proposal and merit a separate discussion.
Cheers.
-- david
On 2018/07/16 11:40:35, Dominic Kim <st...@gmail.com> wrote:
> Dear David.
>
> This is an awesome idea!!
>
> Is this to control IoT devices programmatically?
> If yes, there would be many different types of IoT devices especially in
> terms of their capabilities such as lighting sensors, thermometer, robot
> cleaner, and so on.
>
> Then do you have anything in mind to take care of heterogeneous sets of
> edge nodes?
> There is a possibility that some actions should only run on thermometers,
> while the others should run on lighting sensors.
>
> If you are trying to install "one-for-all" OpenWhisk cluster rather than
> having separate OpenWhisk clusters for each device types, how will you
> manage heterogenous container pools and properly assign relevant actions to
> them?
>
>
> Best regards,
> Dominic
>
>
> 2018-07-16 20:24 GMT+09:00 Markus Thoemmes <ma...@de.ibm.com>:
>
> > Hi David,
> >
> > please send your PR for sure! IIRC we made the Loadbalancer pluggable
> > specifically for this use-case. Sounds like a great addition to our
> > possible deployment topologies.
> >
> > Shameless plug: Would you review the architecture I proposed here:
> > https://lists.apache.org/thread.html/29289006d190b2c68451f7625c13bb
> > 8020cc8e9928db66f1b0def18e@%3Cdev.openwhisk.apache.org%3E
> >
> > In theory, this could make your proposal even leaner in the future. Don't
> > hear me say though we should hold this back, we can absolutely go forward
> > with your implementation first. Just wanted to quickly verify this use-case
> > will also work with what we might plan for in the future.
> >
> > Cheers,
> > Markus
> >
> >
>
Re: Proposing Lean OpenWhisk
Posted by Dominic Kim <st...@gmail.com>.
Dear David.
This is an awesome idea!!
Is this to control IoT devices programmatically?
If yes, there would be many different types of IoT devices especially in
terms of their capabilities such as lighting sensors, thermometer, robot
cleaner, and so on.
Then do you have anything in mind to take care of heterogeneous sets of
edge nodes?
There is a possibility that some actions should only run on thermometers,
while the others should run on lighting sensors.
If you are trying to install "one-for-all" OpenWhisk cluster rather than
having separate OpenWhisk clusters for each device types, how will you
manage heterogenous container pools and properly assign relevant actions to
them?
Best regards,
Dominic
2018-07-16 20:24 GMT+09:00 Markus Thoemmes <ma...@de.ibm.com>:
> Hi David,
>
> please send your PR for sure! IIRC we made the Loadbalancer pluggable
> specifically for this use-case. Sounds like a great addition to our
> possible deployment topologies.
>
> Shameless plug: Would you review the architecture I proposed here:
> https://lists.apache.org/thread.html/29289006d190b2c68451f7625c13bb
> 8020cc8e9928db66f1b0def18e@%3Cdev.openwhisk.apache.org%3E
>
> In theory, this could make your proposal even leaner in the future. Don't
> hear me say though we should hold this back, we can absolutely go forward
> with your implementation first. Just wanted to quickly verify this use-case
> will also work with what we might plan for in the future.
>
> Cheers,
> Markus
>
>
Re: Proposing Lean OpenWhisk
Posted by Markus Thoemmes <ma...@de.ibm.com>.
Hi David,
please send your PR for sure! IIRC we made the Loadbalancer pluggable specifically for this use-case. Sounds like a great addition to our possible deployment topologies.
Shameless plug: Would you review the architecture I proposed here: https://lists.apache.org/thread.html/29289006d190b2c68451f7625c13bb8020cc8e9928db66f1b0def18e@%3Cdev.openwhisk.apache.org%3E
In theory, this could make your proposal even leaner in the future. Don't hear me say though we should hold this back, we can absolutely go forward with your implementation first. Just wanted to quickly verify this use-case will also work with what we might plan for in the future.
Cheers,
Markus