Bennchmarking all the runtimes, both init and run

[Michele Sciabarra <mi...@sciabarra.com>]

Hello Whiskers, 

without any further ado here is the report with a nice html graph.

https://sciabarracom.github.io/incubator-openwhisk-runtime-actionloop/

Test Methodology: I wrote a "multipost" Go application able to run multiple init and run in sequence and I measured the execution time with the unix "time" utility. I tested runs executing one init and 1000 runs in sequence. I tested init starting 100 containers in different ports then executing 100 init AND run in sequence on the various containers.

A few notes. 

I tested go and swift both precompiled (main_go.zip and main_swift.zip) and with a compilation on-the-fly.  Go init time looks slower than swift, but the Go is statically compiled and it is huge, while the swift is dynamically compiled and it is small. I believe most of the time is spent uploading and unpacking the zip. I should consider creating dynamically linked go execution. 

Python runtimes are slow, and this is not a surprise, I will try to migrate them to actionloop. Ruby also is pretty slow and a migration should be considered. Actionloop is the faster on execution for go, very close to the fastest nodejs. 

I can contribute the benchmarks to the repo devtools if interested.

-- 
  Michele Sciabarra
  michele@sciabarra.com

Re: Bennchmarking all the runtimes, both init and run

[Carlos Santana <cs...@gmail.com>]

This is super awesome Michele +1


- Carlos Santana
@csantanapr

> On Dec 9, 2018, at 1:14 PM, Michele Sciabarra <mi...@sciabarra.com> wrote:
> 
> Hello Whiskers, 
> 
> without any further ado here is the report with a nice html graph.
> 
> https://sciabarracom.github.io/incubator-openwhisk-runtime-actionloop/
> 
> Test Methodology: I wrote a "multipost" Go application able to run multiple init and run in sequence and I measured the execution time with the unix "time" utility. I tested runs executing one init and 1000 runs in sequence. I tested init starting 100 containers in different ports then executing 100 init AND run in sequence on the various containers.
> 
> A few notes. 
> 
> I tested go and swift both precompiled (main_go.zip and main_swift.zip) and with a compilation on-the-fly.  Go init time looks slower than swift, but the Go is statically compiled and it is huge, while the swift is dynamically compiled and it is small. I believe most of the time is spent uploading and unpacking the zip. I should consider creating dynamically linked go execution. 
> 
> Python runtimes are slow, and this is not a surprise, I will try to migrate them to actionloop. Ruby also is pretty slow and a migration should be considered. Actionloop is the faster on execution for go, very close to the fastest nodejs. 
> 
> I can contribute the benchmarks to the repo devtools if interested.
> 
> -- 
>  Michele Sciabarra
>  michele@sciabarra.com