You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@openwhisk.apache.org by Michael Marth <mm...@adobe.com> on 2017/05/03 18:57:32 UTC
Re: Performance tests for OpenWhisk
Markus,
Quick update: sent the below to users@infra. So far no reaction. The archive is here [1] but Bertrand tells me only ASF member have access - for whatever reason.
Michael
[1] https://lists.apache.org/thread.html/70999f9233dac9b416ef9dedc97c0ef196a938c05d6a407b94ba3479@%3Cusers.infra.apache.org%3E
On Fri, Apr 28, 2017 at 2:23 PM, Michael Marth <mm...@adobe.com>> wrote:
Dear Infra team,
I am enquiring on behalf of the OpenWhisk project (currently in Incubator)
[1].
We would like to periodically run performance tests on a distributed
environment (OpenWhisk typically runs on more than 1 machine). So we are
basically looking for an ability to spin up/tear down a number of (virtual)
machines and exclusively use them for a certain amount of time (so that the
VMs are not shared and the performance test results are comparable over
time).
The order of magnitude would be ~5-10 VMs for 1 hour 3 times a week.
I would like to find out if there is an ASF-supported mechanism to do that.
For example, can Infra provide such infrastructure? Or is there a cloud
provider (like Azure) that might sponsor such efforts with VMs? Or maybe
there is an established way for commercial companies that are interested in
an ASF project to sponsor (fund) such tests?
If none of the above exists, then it would also be helpful for us to get to
know how other projects run such sort of tests.
Thanks a lot!
Michael
[1]
https://lists.apache.org/thread.html/b66ab5b438f2db5cdc8c5f5eabece201b4ad090058fa3a9a3bd09d12@%3Cdev.openwhisk.apache.org%3E
From: Markus Thömmes <ma...@me.com>>
Reply-To: "dev@openwhisk.apache.org<ma...@openwhisk.apache.org>" <de...@openwhisk.apache.org>>
Date: Wednesday 26 April 2017 12:59
To: "dev@openwhisk.apache.org<ma...@openwhisk.apache.org>" <de...@openwhisk.apache.org>>
Subject: Re: Performance tests for OpenWhisk
Hi Michael,
yeah that sounds pretty much spot on. I'd like to have at least 2 VMs with 4+ cores and 8GB memory. One VM would host the management stack while one would be dedicated to an Invoker only. That way we could assert single-invoker performance the easiest.
Thanks for helping!
Cheers,
Markus
Am 26. April 2017 um 11:36 schrieb Michael Marth <mm...@adobe.com>>:
Markus,
Does what I describe reflect what you are looking for?
If yes, I am happy to ask on infra.
Let me know
Michael
On 26/04/17 07:52, "Bertrand Delacretaz" <bd...@apache.org>> wrote:
Hi Michael,
On Tue, Apr 25, 2017 at 6:52 PM, Michael Marth <mm...@adobe.com>> wrote:
...Maybe our mentors can chime in. Has this been discussed in the ASF board or so?...
Best would be to ask the ASF infrastructure team via
users@infra.apache.org<ma...@infra.apache.org> - briefly describe what you need to see what's
possible.
-Bertrand
Re: Performance tests for OpenWhisk
Posted by Dascalita Dragos <dd...@gmail.com>.
+1 to the idea of having 1 machine dedicated to running action containers
only.
We've also taken Markus' repo a step further with a distributed testing
tool named Locust.io [1] . The tests are at [2].
I've also used Tsung [3] on other projects (it's based on Erlang). Tsung
provides the best performance I've seen out there. It scales to millions of
concurrent users with only a few machines. But b/c of Erlang, I've been
having issues running a cluster in docker containers. Locust.io is much
easier to setup and unlike Tsung, which provides an XML to describe the
tests, Locust.io supports Python and Go scripts.
For running performance tests in a single machine we don't need a
distributed testing tool, but I would still build on such a tool b/c I find
it valuable to reuse the same scripts to generate more load in bigger
OpenWhisk clusters.
[1] - http://locust.io/
[2] - https://github.com/adobe-apiplatform/openwhisk-performance-tests
[3] - http://tsung.erlang-projects.org/
On Thu, Aug 3, 2017 at 1:04 AM Carlos Santana <cs...@gmail.com> wrote:
> Following up here I requested a new repo incubator-openwhisk-performance
> INFRA ticket here: https://issues.apache.org/jira/browse/INFRA-14778
>
>
> On Wed, May 3, 2017 at 2:57 PM Michael Marth <mm...@adobe.com> wrote:
>
> > Markus,
> >
> > Quick update: sent the below to users@infra. So far no reaction. The
> > archive is here [1] but Bertrand tells me only ASF member have access -
> > for whatever reason.
> >
> > Michael
> >
> > [1]
> >
> https://lists.apache.org/thread.html/70999f9233dac9b416ef9dedc97c0ef196a938c05d6a407b94ba3479@%3Cusers.infra.apache.org%3E
> >
> >
> > On Fri, Apr 28, 2017 at 2:23 PM, Michael Marth <mmarth@adobe.com<mailto:
> > mmarth@adobe.com>> wrote:
> > Dear Infra team,
> >
> > I am enquiring on behalf of the OpenWhisk project (currently in
> Incubator)
> > [1].
> >
> > We would like to periodically run performance tests on a distributed
> > environment (OpenWhisk typically runs on more than 1 machine). So we are
> > basically looking for an ability to spin up/tear down a number of
> (virtual)
> > machines and exclusively use them for a certain amount of time (so that
> the
> > VMs are not shared and the performance test results are comparable over
> > time).
> > The order of magnitude would be ~5-10 VMs for 1 hour 3 times a week.
> >
> > I would like to find out if there is an ASF-supported mechanism to do
> that.
> > For example, can Infra provide such infrastructure? Or is there a cloud
> > provider (like Azure) that might sponsor such efforts with VMs? Or maybe
> > there is an established way for commercial companies that are interested
> in
> > an ASF project to sponsor (fund) such tests?
> >
> > If none of the above exists, then it would also be helpful for us to get
> to
> > know how other projects run such sort of tests.
> >
> > Thanks a lot!
> > Michael
> >
> >
> > [1]
> >
> >
> https://lists.apache.org/thread.html/b66ab5b438f2db5cdc8c5f5eabece201b4ad090058fa3a9a3bd09d12@%3Cdev.openwhisk.apache.org%3E
> >
> >
> >
> >
> > From: Markus Thömmes <markusthoemmes@me.com<mailto:markusthoemmes@me.com
> >>
> > Reply-To: "dev@openwhisk.apache.org<ma...@openwhisk.apache.org>" <
> > dev@openwhisk.apache.org<ma...@openwhisk.apache.org>>
> > Date: Wednesday 26 April 2017 12:59
> > To: "dev@openwhisk.apache.org<ma...@openwhisk.apache.org>" <
> > dev@openwhisk.apache.org<ma...@openwhisk.apache.org>>
> > Subject: Re: Performance tests for OpenWhisk
> >
> > Hi Michael,
> >
> > yeah that sounds pretty much spot on. I'd like to have at least 2 VMs
> with
> > 4+ cores and 8GB memory. One VM would host the management stack while one
> > would be dedicated to an Invoker only. That way we could assert
> > single-invoker performance the easiest.
> >
> > Thanks for helping!
> >
> > Cheers,
> > Markus
> >
> > Am 26. April 2017 um 11:36 schrieb Michael Marth <mmarth@adobe.com
> <mailto:
> > mmarth@adobe.com>>:
> >
> > Markus,
> >
> > Does what I describe reflect what you are looking for?
> > If yes, I am happy to ask on infra.
> >
> > Let me know
> > Michael
> >
> >
> >
> > On 26/04/17 07:52, "Bertrand Delacretaz" <bdelacretaz@apache.org<mailto:
> > bdelacretaz@apache.org>> wrote:
> >
> > Hi Michael,
> >
> > On Tue, Apr 25, 2017 at 6:52 PM, Michael Marth <mmarth@adobe.com<mailto:
> > mmarth@adobe.com>> wrote:
> > ...Maybe our mentors can chime in. Has this been discussed in the ASF
> > board or so?...
> >
> > Best would be to ask the ASF infrastructure team via
> > users@infra.apache.org<ma...@infra.apache.org> - briefly describe
> > what you need to see what's
> > possible.
> >
> > -Bertrand
> >
>
Re: Performance tests for OpenWhisk
Posted by Carlos Santana <cs...@gmail.com>.
Following up here I requested a new repo incubator-openwhisk-performance
INFRA ticket here: https://issues.apache.org/jira/browse/INFRA-14778
On Wed, May 3, 2017 at 2:57 PM Michael Marth <mm...@adobe.com> wrote:
> Markus,
>
> Quick update: sent the below to users@infra. So far no reaction. The
> archive is here [1] but Bertrand tells me only ASF member have access -
> for whatever reason.
>
> Michael
>
> [1]
> https://lists.apache.org/thread.html/70999f9233dac9b416ef9dedc97c0ef196a938c05d6a407b94ba3479@%3Cusers.infra.apache.org%3E
>
>
> On Fri, Apr 28, 2017 at 2:23 PM, Michael Marth <mmarth@adobe.com<mailto:
> mmarth@adobe.com>> wrote:
> Dear Infra team,
>
> I am enquiring on behalf of the OpenWhisk project (currently in Incubator)
> [1].
>
> We would like to periodically run performance tests on a distributed
> environment (OpenWhisk typically runs on more than 1 machine). So we are
> basically looking for an ability to spin up/tear down a number of (virtual)
> machines and exclusively use them for a certain amount of time (so that the
> VMs are not shared and the performance test results are comparable over
> time).
> The order of magnitude would be ~5-10 VMs for 1 hour 3 times a week.
>
> I would like to find out if there is an ASF-supported mechanism to do that.
> For example, can Infra provide such infrastructure? Or is there a cloud
> provider (like Azure) that might sponsor such efforts with VMs? Or maybe
> there is an established way for commercial companies that are interested in
> an ASF project to sponsor (fund) such tests?
>
> If none of the above exists, then it would also be helpful for us to get to
> know how other projects run such sort of tests.
>
> Thanks a lot!
> Michael
>
>
> [1]
>
> https://lists.apache.org/thread.html/b66ab5b438f2db5cdc8c5f5eabece201b4ad090058fa3a9a3bd09d12@%3Cdev.openwhisk.apache.org%3E
>
>
>
>
> From: Markus Thömmes <ma...@me.com>>
> Reply-To: "dev@openwhisk.apache.org<ma...@openwhisk.apache.org>" <
> dev@openwhisk.apache.org<ma...@openwhisk.apache.org>>
> Date: Wednesday 26 April 2017 12:59
> To: "dev@openwhisk.apache.org<ma...@openwhisk.apache.org>" <
> dev@openwhisk.apache.org<ma...@openwhisk.apache.org>>
> Subject: Re: Performance tests for OpenWhisk
>
> Hi Michael,
>
> yeah that sounds pretty much spot on. I'd like to have at least 2 VMs with
> 4+ cores and 8GB memory. One VM would host the management stack while one
> would be dedicated to an Invoker only. That way we could assert
> single-invoker performance the easiest.
>
> Thanks for helping!
>
> Cheers,
> Markus
>
> Am 26. April 2017 um 11:36 schrieb Michael Marth <mmarth@adobe.com<mailto:
> mmarth@adobe.com>>:
>
> Markus,
>
> Does what I describe reflect what you are looking for?
> If yes, I am happy to ask on infra.
>
> Let me know
> Michael
>
>
>
> On 26/04/17 07:52, "Bertrand Delacretaz" <bdelacretaz@apache.org<mailto:
> bdelacretaz@apache.org>> wrote:
>
> Hi Michael,
>
> On Tue, Apr 25, 2017 at 6:52 PM, Michael Marth <mmarth@adobe.com<mailto:
> mmarth@adobe.com>> wrote:
> ...Maybe our mentors can chime in. Has this been discussed in the ASF
> board or so?...
>
> Best would be to ask the ASF infrastructure team via
> users@infra.apache.org<ma...@infra.apache.org> - briefly describe
> what you need to see what's
> possible.
>
> -Bertrand
>