You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@openwhisk.apache.org by ma...@apache.org on 2017/08/16 09:23:40 UTC
[incubator-openwhisk-performance] branch master created (now
02d1eed)
This is an automated email from the ASF dual-hosted git repository.
markusthoemmes pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk-performance.git.
at 02d1eed Fix travis build
This branch includes the following new commits:
new bcecff9 Initial commit
new 64f2447 Base travis.yml, checking out openwhisk
new e325661 Pull openwhisk images from docker
new 15f9cf5 Clone shallowly and hook up deployment
new 482c7a7 Refactor deployment
new 088d1f4 Common docker setup
new 608c3f2 Minimize deployment effort
new 8423020 Run a basic test
new c092f1a Actually use a performance test
new d7e1dbd Use loadtest to benchmark
new c32108a Override default limits to effectively disable them
new b29486a Increase sample size for the test
new 9a0cf49 Refactor test setup
new d10646e More refactoring
new d1162b7 Add documentation
new 1be1191 Add throughput test with concurrency switch
new c58f77e Documentation for the throughput test
new d2ffa1f Bumping concurrency to see if throuhput increases
new cd838d3 Documentation on using the testsuites
new da0a881 Refactor the repository to make each test independent
new 7b6dccf Fix shell include issue
new 02d1eed Fix travis build
The 22 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
--
To stop receiving notification emails like this one, please contact
['"commits@openwhisk.apache.org" <co...@openwhisk.apache.org>'].
[incubator-openwhisk-performance] 09/22: Actually use a performance
test
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markusthoemmes pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk-performance.git
commit c092f1ac15ee0796645b5fe4c82ad648e28ef74c
Author: Markus Thoemmes <ma...@de.ibm.com>
AuthorDate: Mon Apr 24 09:49:20 2017 +0200
Actually use a performance test
---
test.sh | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/test.sh b/test.sh
index 2d8cd2e..6be3ff6 100755
--- a/test.sh
+++ b/test.sh
@@ -1,7 +1,14 @@
#!/bin/sh
+action="noop"
+
# create a noop action
-curl -u "$(cat openwhisk/ansible/files/auth.guest)" "172.17.0.1:10001/api/v1/namespaces/_/actions/curlTest" -XPUT -d '{"namespace":"_","name":"test","exec":{"kind":"nodejs:default","code":"function main(){return {};}"}}' -H "Content-Type: application/json"
+echo "Creating noop action"
+curl -u "$(cat openwhisk/ansible/files/auth.guest)" "172.17.0.1:10001/api/v1/namespaces/_/actions/$action" -XPUT -d '{"namespace":"_","name":"test","exec":{"kind":"nodejs:default","code":"function main(){return {};}"}}' -H "Content-Type: application/json"
# run the noop action
-curl -u "$(cat openwhisk/ansible/files/auth.guest)" "172.17.0.1:10001/api/v1/namespaces/_/actions/curlTest?blocking=true" -XPOST
\ No newline at end of file
+echo "Running noop action once to assert an intact system"
+curl -u "$(cat openwhisk/ansible/files/auth.guest)" "172.17.0.1:10001/api/v1/namespaces/_/actions/$action?blocking=true" -XPOST
+
+# run performance harness
+ab -n 1000 -m POST -k -A "$(cat openwhisk/ansible/files/auth.guest)" "172.17.0.1:10001/api/v1/namespaces/_/actions/$action?blocking=true"
\ No newline at end of file
--
To stop receiving notification emails like this one, please contact
"commits@openwhisk.apache.org" <co...@openwhisk.apache.org>.
[incubator-openwhisk-performance] 16/22: Add throughput test with
concurrency switch
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markusthoemmes pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk-performance.git
commit 1be1191260c7caca66afa996427ff9b2233ec4d8
Author: Markus Thoemmes <ma...@de.ibm.com>
AuthorDate: Mon Apr 24 14:56:19 2017 +0200
Add throughput test with concurrency switch
---
.travis.yml | 2 +-
test.sh | 8 ++++++--
2 files changed, 7 insertions(+), 3 deletions(-)
diff --git a/.travis.yml b/.travis.yml
index 8231114..9dd9bd2 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -10,4 +10,4 @@ install:
- ./deploy.sh
script:
- - ./test.sh "http://172.17.0.1:10001" "$(cat openwhisk/ansible/files/auth.guest)"
\ No newline at end of file
+ - ./test.sh "http://172.17.0.1:10001" "$(cat openwhisk/ansible/files/auth.guest)" 4
\ No newline at end of file
diff --git a/test.sh b/test.sh
index e2d0510..2dbf029 100755
--- a/test.sh
+++ b/test.sh
@@ -3,6 +3,7 @@ set -e
host=$1
credentials=$2
+concurrency=$3
action="noop"
@@ -14,5 +15,8 @@ curl -u "$credentials" "$host/api/v1/namespaces/_/actions/$action" -XPUT -d '{"n
echo "Running noop action once to assert an intact system"
curl -u "$credentials" "$host/api/v1/namespaces/_/actions/$action?blocking=true" -XPOST
-# run performance harness
-docker run --rm markusthoemmes/loadtest loadtest -n 10000 -k -m POST -H "Authorization: basic $(echo $credentials | base64 -w 0)" "$host/api/v1/namespaces/_/actions/$action?blocking=true"
\ No newline at end of file
+# run latency tests
+docker run --rm markusthoemmes/loadtest loadtest -n 10000 -k -m POST -H "Authorization: basic $(echo $credentials | base64 -w 0)" "$host/api/v1/namespaces/_/actions/$action?blocking=true"
+
+# run maximum throughput tests
+docker run --rm markusthoemmes/loadtest loadtest -n 10000 -c "$concurrency" -k -m POST -H "Authorization: basic $(echo $credentials | base64 -w 0)" "$host/api/v1/namespaces/_/actions/$action?blocking=true"
\ No newline at end of file
--
To stop receiving notification emails like this one, please contact
"commits@openwhisk.apache.org" <co...@openwhisk.apache.org>.
[incubator-openwhisk-performance] 07/22: Minimize deployment effort
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markusthoemmes pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk-performance.git
commit 608c3f293acb69c5c03390482e79238a985ef6bc
Author: Markus Thoemmes <ma...@de.ibm.com>
AuthorDate: Mon Apr 24 08:43:58 2017 +0200
Minimize deployment effort
---
deploy.sh | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/deploy.sh b/deploy.sh
index fd3ac97..7083193 100755
--- a/deploy.sh
+++ b/deploy.sh
@@ -12,12 +12,15 @@ git clone --depth 1 https://github.com/openwhisk/openwhisk.git
pip install --user ansible==2.1.2.0
cd openwhisk/ansible
-ANSIBLE_CMD="ansible-playbook -i environments/local -e docker_image_prefix=openwhisk"
+ANSIBLE_CMD="ansible-playbook -i environments/local -e docker_image_prefix=openwhisk -e docker_registry=docker.io/"
$ANSIBLE_CMD setup.yml
$ANSIBLE_CMD prereq.yml
$ANSIBLE_CMD couchdb.yml
$ANSIBLE_CMD initdb.yml
-$ANSIBLE_CMD apigateway.yml
$ANSIBLE_CMD wipe.yml
-$ANSIBLE_CMD openwhisk.yml
\ No newline at end of file
+
+$ANSIBLE_CMD consul.yml
+$ANSIBLE_CMD kafka.yml
+$ANSIBLE_CMD controller.yml
+$ANSIBLE_CMD invoker.yml
\ No newline at end of file
--
To stop receiving notification emails like this one, please contact
"commits@openwhisk.apache.org" <co...@openwhisk.apache.org>.
[incubator-openwhisk-performance] 22/22: Fix travis build
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markusthoemmes pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk-performance.git
commit 02d1eedb380c1257e19499344655b06b93e2676e
Author: Markus Thoemmes <ma...@de.ibm.com>
AuthorDate: Mon May 15 20:20:11 2017 +0200
Fix travis build
---
deploy.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/deploy.sh b/deploy.sh
index b9ab8b7..9ca82ce 100755
--- a/deploy.sh
+++ b/deploy.sh
@@ -11,7 +11,7 @@ sudo service docker restart
git clone --depth 1 https://github.com/openwhisk/openwhisk.git
# install ansible
-pip install --user ansible==2.1.2.0
+pip install --user ansible==2.3.0.0
cd openwhisk/ansible
LIMITS='{"limits":{"actions":{"invokes":{"perMinute":999999,"concurrent":999999,"concurrentInSystem":999999}},"triggers":{"fires":{"perMinute":999999}}}}'
--
To stop receiving notification emails like this one, please contact
"commits@openwhisk.apache.org" <co...@openwhisk.apache.org>.
[incubator-openwhisk-performance] 14/22: More refactoring
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markusthoemmes pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk-performance.git
commit d10646e70fc600b37feda3830b6511a9b08265fb
Author: Markus Thoemmes <ma...@de.ibm.com>
AuthorDate: Mon Apr 24 14:05:14 2017 +0200
More refactoring
---
.travis.yml | 2 +-
deploy.sh | 2 ++
test.sh | 7 ++++---
3 files changed, 7 insertions(+), 4 deletions(-)
diff --git a/.travis.yml b/.travis.yml
index 8b65bda..8231114 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -10,4 +10,4 @@ install:
- ./deploy.sh
script:
- - ./test.sh
\ No newline at end of file
+ - ./test.sh "http://172.17.0.1:10001" "$(cat openwhisk/ansible/files/auth.guest)"
\ No newline at end of file
diff --git a/deploy.sh b/deploy.sh
index 56f755d..b9ab8b7 100755
--- a/deploy.sh
+++ b/deploy.sh
@@ -1,5 +1,7 @@
#!/bin/sh
+set -e
+
# common docker setup
sudo gpasswd -a travis docker
sudo -E bash -c 'echo '\''DOCKER_OPTS="-H tcp://0.0.0.0:4243 -H unix:///var/run/docker.sock --storage-driver=overlay --userns-remap=default"'\'' > /etc/default/docker'
diff --git a/test.sh b/test.sh
index 4b464fa..e2d0510 100755
--- a/test.sh
+++ b/test.sh
@@ -1,9 +1,10 @@
#!/bin/sh
set -e
+host=$1
+credentials=$2
+
action="noop"
-credentials="$(cat openwhisk/ansible/files/auth.guest)"
-host="172.17.0.1:10001"
# create a noop action
echo "Creating noop action"
@@ -14,4 +15,4 @@ echo "Running noop action once to assert an intact system"
curl -u "$credentials" "$host/api/v1/namespaces/_/actions/$action?blocking=true" -XPOST
# run performance harness
-docker run --rm markusthoemmes/loadtest loadtest -n 10000 -H "Authorization: basic $(echo $credentials | base64 -w 0)" "http://$host/api/v1/namespaces/_/actions/$action?blocking=true" -m POST
\ No newline at end of file
+docker run --rm markusthoemmes/loadtest loadtest -n 10000 -k -m POST -H "Authorization: basic $(echo $credentials | base64 -w 0)" "$host/api/v1/namespaces/_/actions/$action?blocking=true"
\ No newline at end of file
--
To stop receiving notification emails like this one, please contact
"commits@openwhisk.apache.org" <co...@openwhisk.apache.org>.
[incubator-openwhisk-performance] 11/22: Override default limits to
effectively disable them
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markusthoemmes pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk-performance.git
commit c32108ad0c7de124711a7e52dd706658e8c3b72a
Author: Markus Thoemmes <ma...@de.ibm.com>
AuthorDate: Mon Apr 24 11:01:16 2017 +0200
Override default limits to effectively disable them
---
deploy.sh | 3 ++-
test.sh | 2 +-
2 files changed, 3 insertions(+), 2 deletions(-)
diff --git a/deploy.sh b/deploy.sh
index 7083193..56f755d 100755
--- a/deploy.sh
+++ b/deploy.sh
@@ -12,7 +12,8 @@ git clone --depth 1 https://github.com/openwhisk/openwhisk.git
pip install --user ansible==2.1.2.0
cd openwhisk/ansible
-ANSIBLE_CMD="ansible-playbook -i environments/local -e docker_image_prefix=openwhisk -e docker_registry=docker.io/"
+LIMITS='{"limits":{"actions":{"invokes":{"perMinute":999999,"concurrent":999999,"concurrentInSystem":999999}},"triggers":{"fires":{"perMinute":999999}}}}'
+ANSIBLE_CMD="ansible-playbook -i environments/local -e docker_image_prefix=openwhisk -e docker_registry=docker.io/ -e $LIMITS"
$ANSIBLE_CMD setup.yml
$ANSIBLE_CMD prereq.yml
diff --git a/test.sh b/test.sh
index 305e891..30cbba7 100755
--- a/test.sh
+++ b/test.sh
@@ -11,4 +11,4 @@ echo "Running noop action once to assert an intact system"
curl -u "$(cat openwhisk/ansible/files/auth.guest)" "172.17.0.1:10001/api/v1/namespaces/_/actions/$action?blocking=true" -XPOST
# run performance harness
-docker run --rm markusthoemmes/loadtest loadtest -H "Authorization: basic MjNiYzQ2YjEtNzFmNi00ZWQ1LThjNTQtODE2YWE0ZjhjNTAyOjEyM3pPM3haQ0xyTU42djJCS0sxZFhZRnBYbFBrY2NPRnFtMTJDZEFzTWdSVTRWck5aOWx5R1ZDR3VNREdJd1A=" "http://172.17.0.1:10001/api/v1/namespaces/_/actions/$action?blocking=true" -m POST -n 1000
\ No newline at end of file
+docker run --rm markusthoemmes/loadtest loadtest -n 1000 -H "Authorization: basic MjNiYzQ2YjEtNzFmNi00ZWQ1LThjNTQtODE2YWE0ZjhjNTAyOjEyM3pPM3haQ0xyTU42djJCS0sxZFhZRnBYbFBrY2NPRnFtMTJDZEFzTWdSVTRWck5aOWx5R1ZDR3VNREdJd1A=" "http://172.17.0.1:10001/api/v1/namespaces/_/actions/$action?blocking=true" -m POST
\ No newline at end of file
--
To stop receiving notification emails like this one, please contact
"commits@openwhisk.apache.org" <co...@openwhisk.apache.org>.
[incubator-openwhisk-performance] 01/22: Initial commit
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markusthoemmes pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk-performance.git
commit bcecff99c0ea2c0f886cd9ef7340f1d62ec02b89
Author: Markus Thömmes <ma...@me.com>
AuthorDate: Mon Apr 24 07:50:30 2017 +0200
Initial commit
---
LICENSE | 201 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
README.md | 2 +
2 files changed, 203 insertions(+)
diff --git a/LICENSE b/LICENSE
new file mode 100644
index 0000000..8dada3e
--- /dev/null
+++ b/LICENSE
@@ -0,0 +1,201 @@
+ Apache License
+ Version 2.0, January 2004
+ http://www.apache.org/licenses/
+
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+ 1. Definitions.
+
+ "License" shall mean the terms and conditions for use, reproduction,
+ and distribution as defined by Sections 1 through 9 of this document.
+
+ "Licensor" shall mean the copyright owner or entity authorized by
+ the copyright owner that is granting the License.
+
+ "Legal Entity" shall mean the union of the acting entity and all
+ other entities that control, are controlled by, or are under common
+ control with that entity. For the purposes of this definition,
+ "control" means (i) the power, direct or indirect, to cause the
+ direction or management of such entity, whether by contract or
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
+ outstanding shares, or (iii) beneficial ownership of such entity.
+
+ "You" (or "Your") shall mean an individual or Legal Entity
+ exercising permissions granted by this License.
+
+ "Source" form shall mean the preferred form for making modifications,
+ including but not limited to software source code, documentation
+ source, and configuration files.
+
+ "Object" form shall mean any form resulting from mechanical
+ transformation or translation of a Source form, including but
+ not limited to compiled object code, generated documentation,
+ and conversions to other media types.
+
+ "Work" shall mean the work of authorship, whether in Source or
+ Object form, made available under the License, as indicated by a
+ copyright notice that is included in or attached to the work
+ (an example is provided in the Appendix below).
+
+ "Derivative Works" shall mean any work, whether in Source or Object
+ form, that is based on (or derived from) the Work and for which the
+ editorial revisions, annotations, elaborations, or other modifications
+ represent, as a whole, an original work of authorship. For the purposes
+ of this License, Derivative Works shall not include works that remain
+ separable from, or merely link (or bind by name) to the interfaces of,
+ the Work and Derivative Works thereof.
+
+ "Contribution" shall mean any work of authorship, including
+ the original version of the Work and any modifications or additions
+ to that Work or Derivative Works thereof, that is intentionally
+ submitted to Licensor for inclusion in the Work by the copyright owner
+ or by an individual or Legal Entity authorized to submit on behalf of
+ the copyright owner. For the purposes of this definition, "submitted"
+ means any form of electronic, verbal, or written communication sent
+ to the Licensor or its representatives, including but not limited to
+ communication on electronic mailing lists, source code control systems,
+ and issue tracking systems that are managed by, or on behalf of, the
+ Licensor for the purpose of discussing and improving the Work, but
+ excluding communication that is conspicuously marked or otherwise
+ designated in writing by the copyright owner as "Not a Contribution."
+
+ "Contributor" shall mean Licensor and any individual or Legal Entity
+ on behalf of whom a Contribution has been received by Licensor and
+ subsequently incorporated within the Work.
+
+ 2. Grant of Copyright License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ copyright license to reproduce, prepare Derivative Works of,
+ publicly display, publicly perform, sublicense, and distribute the
+ Work and such Derivative Works in Source or Object form.
+
+ 3. Grant of Patent License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ (except as stated in this section) patent license to make, have made,
+ use, offer to sell, sell, import, and otherwise transfer the Work,
+ where such license applies only to those patent claims licensable
+ by such Contributor that are necessarily infringed by their
+ Contribution(s) alone or by combination of their Contribution(s)
+ with the Work to which such Contribution(s) was submitted. If You
+ institute patent litigation against any entity (including a
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
+ or a Contribution incorporated within the Work constitutes direct
+ or contributory patent infringement, then any patent licenses
+ granted to You under this License for that Work shall terminate
+ as of the date such litigation is filed.
+
+ 4. Redistribution. You may reproduce and distribute copies of the
+ Work or Derivative Works thereof in any medium, with or without
+ modifications, and in Source or Object form, provided that You
+ meet the following conditions:
+
+ (a) You must give any other recipients of the Work or
+ Derivative Works a copy of this License; and
+
+ (b) You must cause any modified files to carry prominent notices
+ stating that You changed the files; and
+
+ (c) You must retain, in the Source form of any Derivative Works
+ that You distribute, all copyright, patent, trademark, and
+ attribution notices from the Source form of the Work,
+ excluding those notices that do not pertain to any part of
+ the Derivative Works; and
+
+ (d) If the Work includes a "NOTICE" text file as part of its
+ distribution, then any Derivative Works that You distribute must
+ include a readable copy of the attribution notices contained
+ within such NOTICE file, excluding those notices that do not
+ pertain to any part of the Derivative Works, in at least one
+ of the following places: within a NOTICE text file distributed
+ as part of the Derivative Works; within the Source form or
+ documentation, if provided along with the Derivative Works; or,
+ within a display generated by the Derivative Works, if and
+ wherever such third-party notices normally appear. The contents
+ of the NOTICE file are for informational purposes only and
+ do not modify the License. You may add Your own attribution
+ notices within Derivative Works that You distribute, alongside
+ or as an addendum to the NOTICE text from the Work, provided
+ that such additional attribution notices cannot be construed
+ as modifying the License.
+
+ You may add Your own copyright statement to Your modifications and
+ may provide additional or different license terms and conditions
+ for use, reproduction, or distribution of Your modifications, or
+ for any such Derivative Works as a whole, provided Your use,
+ reproduction, and distribution of the Work otherwise complies with
+ the conditions stated in this License.
+
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
+ any Contribution intentionally submitted for inclusion in the Work
+ by You to the Licensor shall be under the terms and conditions of
+ this License, without any additional terms or conditions.
+ Notwithstanding the above, nothing herein shall supersede or modify
+ the terms of any separate license agreement you may have executed
+ with Licensor regarding such Contributions.
+
+ 6. Trademarks. This License does not grant permission to use the trade
+ names, trademarks, service marks, or product names of the Licensor,
+ except as required for reasonable and customary use in describing the
+ origin of the Work and reproducing the content of the NOTICE file.
+
+ 7. Disclaimer of Warranty. Unless required by applicable law or
+ agreed to in writing, Licensor provides the Work (and each
+ Contributor provides its Contributions) on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ implied, including, without limitation, any warranties or conditions
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+ PARTICULAR PURPOSE. You are solely responsible for determining the
+ appropriateness of using or redistributing the Work and assume any
+ risks associated with Your exercise of permissions under this License.
+
+ 8. Limitation of Liability. In no event and under no legal theory,
+ whether in tort (including negligence), contract, or otherwise,
+ unless required by applicable law (such as deliberate and grossly
+ negligent acts) or agreed to in writing, shall any Contributor be
+ liable to You for damages, including any direct, indirect, special,
+ incidental, or consequential damages of any character arising as a
+ result of this License or out of the use or inability to use the
+ Work (including but not limited to damages for loss of goodwill,
+ work stoppage, computer failure or malfunction, or any and all
+ other commercial damages or losses), even if such Contributor
+ has been advised of the possibility of such damages.
+
+ 9. Accepting Warranty or Additional Liability. While redistributing
+ the Work or Derivative Works thereof, You may choose to offer,
+ and charge a fee for, acceptance of support, warranty, indemnity,
+ or other liability obligations and/or rights consistent with this
+ License. However, in accepting such obligations, You may act only
+ on Your own behalf and on Your sole responsibility, not on behalf
+ of any other Contributor, and only if You agree to indemnify,
+ defend, and hold each Contributor harmless for any liability
+ incurred by, or claims asserted against, such Contributor by reason
+ of your accepting any such warranty or additional liability.
+
+ END OF TERMS AND CONDITIONS
+
+ APPENDIX: How to apply the Apache License to your work.
+
+ To apply the Apache License to your work, attach the following
+ boilerplate notice, with the fields enclosed by brackets "{}"
+ replaced with your own identifying information. (Don't include
+ the brackets!) The text should be enclosed in the appropriate
+ comment syntax for the file format. We also recommend that a
+ file or class name and description of purpose be included on the
+ same "printed page" as the copyright notice for easier
+ identification within third-party archives.
+
+ Copyright {yyyy} {name of copyright owner}
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
diff --git a/README.md b/README.md
new file mode 100644
index 0000000..305ddbb
--- /dev/null
+++ b/README.md
@@ -0,0 +1,2 @@
+# openwhisk-performance
+Performance harness for Apache OpenWhisk
--
To stop receiving notification emails like this one, please contact
"commits@openwhisk.apache.org" <co...@openwhisk.apache.org>.
[incubator-openwhisk-performance] 19/22: Documentation on using the
testsuites
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markusthoemmes pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk-performance.git
commit cd838d38834e0ccaf160a46816c59710eedacaf8
Author: Markus Thoemmes <ma...@de.ibm.com>
AuthorDate: Tue Apr 25 08:10:09 2017 +0200
Documentation on using the testsuites
---
README.md | 25 +++++++++++++++++++++----
1 file changed, 21 insertions(+), 4 deletions(-)
diff --git a/README.md b/README.md
index 2d208f9..7b4c214 100644
--- a/README.md
+++ b/README.md
@@ -2,14 +2,18 @@
A few simple but efficient performance test suites for Apache OpenWhisk. Determines the maximum throughput and end-user latency of the system.
## Test setup
-- A standard OpenWhisk system is deployed. Note that the edge nginx router and API Gateway are left out currently.
+- A standard OpenWhisk system is deployed. Note that the edge nginx router and API Gateway are left out currently. As a consequence, the tests talk directly to the controller.
- Limits are set to 999999 each, for the test's load that means: No throttling at all.
- The deployment uses a docker setup as proposed by the OpenWhisk development team: `overlay` driver and HTTP API enabled via a UNIX port.
+The load is driven by the beautiful [`loadtest`](https://www.npmjs.com/package/loadtest) module.
+
### Travis' machine setup
The [machine provided by Travis](https://docs.travis-ci.com/user/ci-environment/#Virtualization-environments) has ~2 CPU cores (likely shared through virtualization) and 7.5GB memory.
-## Latency test
+## Suites
+
+### Latency test
The latency test determines the end-to-end latency a user experiences when doing a blocking invocation. The action used is a noop so the numbers returned are plain overhead of the OpenWhisk system.
- 1 HTTP request at a time (concurrency: 1)
@@ -18,9 +22,22 @@ The latency test determines the end-to-end latency a user experiences when doing
**Note:** The throughput number has a 100% correlation with the latency in this case. This test does not serve to determine the maximum throughput of the system.
-## Throughput test
+### Throughput test
The throughput test determines the maximum throughput a user can get out of the system while using a single action. The action used again is a noop, so the numbers are plain OpenWhisk overhead. Note that the throughput does not directly correlate to end-to-end latency here, as the system does more processing in the background as it shows to the user in a blocking invocation.
- 4 HTTP requests at a time (concurrency: 4) (basically CPU cores * 2 to exploit some buffering)
- 10.000 samples with a single user
-- noop action
\ No newline at end of file
+- noop action
+
+## Running the suites yourself
+To run the suites against your own system (or any other OpenWhisk deployment really), simply use the `test.sh` script.
+
+It takes a couple of parameters, defined as follows:
+
+```
+> test.sh HOST CREDENTIALS CONCURRENCY_LEVEL
+```
+
+- **HOST**: The host to reach OpenWhisk. Should include the protocol (`http`/`https`) and the port if applicable.
+- **CREDENTIALS**: Credentials for OpenWhisk in USER:PASS form.
+- **CONCURRENCY_LEVEL**: Concurrency level for the loadtest. This number should be a multiple of the number of CPU cores all invoker machines have in total.
\ No newline at end of file
--
To stop receiving notification emails like this one, please contact
"commits@openwhisk.apache.org" <co...@openwhisk.apache.org>.
[incubator-openwhisk-performance] 05/22: Refactor deployment
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markusthoemmes pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk-performance.git
commit 482c7a720287a8e6a7591d440170f8c4b2a911b4
Author: Markus Thoemmes <ma...@de.ibm.com>
AuthorDate: Mon Apr 24 08:22:47 2017 +0200
Refactor deployment
---
.travis.yml | 1 -
deploy.sh | 22 ++++++++++++++++------
2 files changed, 16 insertions(+), 7 deletions(-)
diff --git a/.travis.yml b/.travis.yml
index a847366..15ae264 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -7,7 +7,6 @@ env:
- TERM=dumb
install:
- - git clone --depth 1 https://github.com/openwhisk/openwhisk.git
- ./deploy.sh
script:
diff --git a/deploy.sh b/deploy.sh
index 4cd9415..15bf1f6 100755
--- a/deploy.sh
+++ b/deploy.sh
@@ -1,8 +1,18 @@
#!/bin/sh
-images="controller invoker"
-for i in $images
-do
- docker pull "openwhisk/$i"
- docker tag "openwhisk/$i" $i
-done
\ No newline at end of file
+# checkout OpenWhisk latest
+git clone --depth 1 https://github.com/openwhisk/openwhisk.git
+
+# install ansible
+pip install --user ansible==2.1.2.0
+
+cd openwhisk/ansible
+ANSIBLE_CMD="ansible-playbook -i environments/local -e docker_image_prefix=openwhisk"
+
+$ANSIBLE_CMD setup.yml
+$ANSIBLE_CMD prereq.yml
+$ANSIBLE_CMD couchdb.yml
+$ANSIBLE_CMD initdb.yml
+$ANSIBLE_CMD apigateway.yml
+$ANSIBLE_CMD wipe.yml
+$ANSIBLE_CMD openwhisk.yml
\ No newline at end of file
--
To stop receiving notification emails like this one, please contact
"commits@openwhisk.apache.org" <co...@openwhisk.apache.org>.
[incubator-openwhisk-performance] 21/22: Fix shell include issue
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markusthoemmes pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk-performance.git
commit 7b6dccf5b020bedecbc9381be0020af69b412776
Author: Markus Thoemmes <ma...@de.ibm.com>
AuthorDate: Tue Apr 25 11:06:28 2017 +0200
Fix shell include issue
---
tests/latency.sh | 5 +++--
tests/throughput.sh | 3 ++-
2 files changed, 5 insertions(+), 3 deletions(-)
diff --git a/tests/latency.sh b/tests/latency.sh
index 3078be9..41f40ea 100755
--- a/tests/latency.sh
+++ b/tests/latency.sh
@@ -1,15 +1,16 @@
#!/bin/sh
set -e
+currentDir="$(dirname "$0")"
# Host to use. Needs to include the protocol.
host=$1
# Credentials to use for the test. USER:PASS format.
credentials=$2
# How many samples to create by the test. Default: 10000
-samples=${3:-10000} # default value of 10000
+samples=${3:-10000}
action="noopLatency"
-./create.sh "$host" "$credentials" "$action"
+"$currentDir/create.sh" "$host" "$credentials" "$action"
# run latency tests
encodedAuth=$(echo "$credentials" | base64 -w 0)
diff --git a/tests/throughput.sh b/tests/throughput.sh
index bf002c3..743e8a7 100755
--- a/tests/throughput.sh
+++ b/tests/throughput.sh
@@ -1,5 +1,6 @@
#!/bin/sh
set -e
+currentDir="$(dirname "$0")"
# Host to use. Needs to include the protocol.
host=$1
@@ -12,7 +13,7 @@ concurrency=$3
samples=${4:-10000}
action="noopThroughput"
-./create.sh "$host" "$credentials" "$action"
+"$currentDir/create.sh" "$host" "$credentials" "$action"
# run throughput tests
encodedAuth=$(echo "$credentials" | base64 -w 0)
--
To stop receiving notification emails like this one, please contact
"commits@openwhisk.apache.org" <co...@openwhisk.apache.org>.
[incubator-openwhisk-performance] 10/22: Use loadtest to benchmark
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markusthoemmes pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk-performance.git
commit d7e1dbd0aed95f3e1d355c54398db7da89301f4e
Author: Markus Thoemmes <ma...@de.ibm.com>
AuthorDate: Mon Apr 24 10:06:33 2017 +0200
Use loadtest to benchmark
---
test.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/test.sh b/test.sh
index 6be3ff6..305e891 100755
--- a/test.sh
+++ b/test.sh
@@ -11,4 +11,4 @@ echo "Running noop action once to assert an intact system"
curl -u "$(cat openwhisk/ansible/files/auth.guest)" "172.17.0.1:10001/api/v1/namespaces/_/actions/$action?blocking=true" -XPOST
# run performance harness
-ab -n 1000 -m POST -k -A "$(cat openwhisk/ansible/files/auth.guest)" "172.17.0.1:10001/api/v1/namespaces/_/actions/$action?blocking=true"
\ No newline at end of file
+docker run --rm markusthoemmes/loadtest loadtest -H "Authorization: basic MjNiYzQ2YjEtNzFmNi00ZWQ1LThjNTQtODE2YWE0ZjhjNTAyOjEyM3pPM3haQ0xyTU42djJCS0sxZFhZRnBYbFBrY2NPRnFtMTJDZEFzTWdSVTRWck5aOWx5R1ZDR3VNREdJd1A=" "http://172.17.0.1:10001/api/v1/namespaces/_/actions/$action?blocking=true" -m POST -n 1000
\ No newline at end of file
--
To stop receiving notification emails like this one, please contact
"commits@openwhisk.apache.org" <co...@openwhisk.apache.org>.
[incubator-openwhisk-performance] 20/22: Refactor the repository to
make each test independent
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markusthoemmes pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk-performance.git
commit da0a8819b0f6848e920d155c4fefd9cb0fbba057
Author: Markus Thoemmes <ma...@de.ibm.com>
AuthorDate: Tue Apr 25 10:50:21 2017 +0200
Refactor the repository to make each test independent
---
.travis.yml | 3 ++-
README.md | 12 +-----------
test.sh | 22 ----------------------
tests/create.sh | 17 +++++++++++++++++
tests/latency.sh | 16 ++++++++++++++++
tests/throughput.sh | 19 +++++++++++++++++++
6 files changed, 55 insertions(+), 34 deletions(-)
diff --git a/.travis.yml b/.travis.yml
index 845c8a6..6f256d3 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -10,4 +10,5 @@ install:
- ./deploy.sh
script:
- - ./test.sh "http://172.17.0.1:10001" "$(cat openwhisk/ansible/files/auth.guest)" 8
\ No newline at end of file
+ - ./tests/latency.sh "http://172.17.0.1:10001" "$(cat openwhisk/ansible/files/auth.guest)"
+ - ./tests/throughput.sh "http://172.17.0.1:10001" "$(cat openwhisk/ansible/files/auth.guest)" 4
\ No newline at end of file
diff --git a/README.md b/README.md
index 7b4c214..5e35629 100644
--- a/README.md
+++ b/README.md
@@ -30,14 +30,4 @@ The throughput test determines the maximum throughput a user can get out of the
- noop action
## Running the suites yourself
-To run the suites against your own system (or any other OpenWhisk deployment really), simply use the `test.sh` script.
-
-It takes a couple of parameters, defined as follows:
-
-```
-> test.sh HOST CREDENTIALS CONCURRENCY_LEVEL
-```
-
-- **HOST**: The host to reach OpenWhisk. Should include the protocol (`http`/`https`) and the port if applicable.
-- **CREDENTIALS**: Credentials for OpenWhisk in USER:PASS form.
-- **CONCURRENCY_LEVEL**: Concurrency level for the loadtest. This number should be a multiple of the number of CPU cores all invoker machines have in total.
\ No newline at end of file
+To run the suites against your own system (or any other OpenWhisk deployment really), simply use the repsective script in `/tests` script. Their parameters are defined inline in the respective file.
\ No newline at end of file
diff --git a/test.sh b/test.sh
deleted file mode 100755
index 2dbf029..0000000
--- a/test.sh
+++ /dev/null
@@ -1,22 +0,0 @@
-#!/bin/sh
-set -e
-
-host=$1
-credentials=$2
-concurrency=$3
-
-action="noop"
-
-# create a noop action
-echo "Creating noop action"
-curl -u "$credentials" "$host/api/v1/namespaces/_/actions/$action" -XPUT -d '{"namespace":"_","name":"test","exec":{"kind":"nodejs:default","code":"function main(){return {};}"}}' -H "Content-Type: application/json"
-
-# run the noop action
-echo "Running noop action once to assert an intact system"
-curl -u "$credentials" "$host/api/v1/namespaces/_/actions/$action?blocking=true" -XPOST
-
-# run latency tests
-docker run --rm markusthoemmes/loadtest loadtest -n 10000 -k -m POST -H "Authorization: basic $(echo $credentials | base64 -w 0)" "$host/api/v1/namespaces/_/actions/$action?blocking=true"
-
-# run maximum throughput tests
-docker run --rm markusthoemmes/loadtest loadtest -n 10000 -c "$concurrency" -k -m POST -H "Authorization: basic $(echo $credentials | base64 -w 0)" "$host/api/v1/namespaces/_/actions/$action?blocking=true"
\ No newline at end of file
diff --git a/tests/create.sh b/tests/create.sh
new file mode 100755
index 0000000..9198a9a
--- /dev/null
+++ b/tests/create.sh
@@ -0,0 +1,17 @@
+#!/bin/sh
+set -e
+
+# Host to use. Needs to include the protocol.
+host=$1
+# Credentials to use for the test. USER:PASS format.
+credentials=$2
+# Name of the action to create and test.
+action=$3
+
+# create a noop action
+echo "Creating action $action"
+curl -u "$credentials" "$host/api/v1/namespaces/_/actions/$action" -XPUT -d '{"namespace":"_","name":"test","exec":{"kind":"nodejs:default","code":"function main(){return {};}"}}' -H "Content-Type: application/json"
+
+# run the noop action
+echo "Running $action once to assert an intact system"
+curl -u "$credentials" "$host/api/v1/namespaces/_/actions/$action?blocking=true" -XPOST
\ No newline at end of file
diff --git a/tests/latency.sh b/tests/latency.sh
new file mode 100755
index 0000000..3078be9
--- /dev/null
+++ b/tests/latency.sh
@@ -0,0 +1,16 @@
+#!/bin/sh
+set -e
+
+# Host to use. Needs to include the protocol.
+host=$1
+# Credentials to use for the test. USER:PASS format.
+credentials=$2
+# How many samples to create by the test. Default: 10000
+samples=${3:-10000} # default value of 10000
+
+action="noopLatency"
+./create.sh "$host" "$credentials" "$action"
+
+# run latency tests
+encodedAuth=$(echo "$credentials" | base64 -w 0)
+docker run --rm markusthoemmes/loadtest loadtest -n "$samples" -k -m POST -H "Authorization: basic $encodedAuth" "$host/api/v1/namespaces/_/actions/$action?blocking=true"
\ No newline at end of file
diff --git a/tests/throughput.sh b/tests/throughput.sh
new file mode 100755
index 0000000..bf002c3
--- /dev/null
+++ b/tests/throughput.sh
@@ -0,0 +1,19 @@
+#!/bin/sh
+set -e
+
+# Host to use. Needs to include the protocol.
+host=$1
+# Credentials to use for the test. USER:PASS format.
+credentials=$2
+# concurrency level of the throughput test: How many requests should
+# open in parallel.
+concurrency=$3
+# How many samples to create by the test. Default: 10000
+samples=${4:-10000}
+
+action="noopThroughput"
+./create.sh "$host" "$credentials" "$action"
+
+# run throughput tests
+encodedAuth=$(echo "$credentials" | base64 -w 0)
+docker run --rm markusthoemmes/loadtest loadtest -n "$samples" -c "$concurrency" -k -m POST -H "Authorization: basic $encodedAuth" "$host/api/v1/namespaces/_/actions/$action?blocking=true"
\ No newline at end of file
--
To stop receiving notification emails like this one, please contact
"commits@openwhisk.apache.org" <co...@openwhisk.apache.org>.
[incubator-openwhisk-performance] 13/22: Refactor test setup
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markusthoemmes pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk-performance.git
commit 9a0cf499a1f9f46955b09cc2ccce7e1ac2b67a3e
Author: Markus Thoemmes <ma...@de.ibm.com>
AuthorDate: Mon Apr 24 13:47:44 2017 +0200
Refactor test setup
---
test.sh | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/test.sh b/test.sh
index e67ae9f..4b464fa 100755
--- a/test.sh
+++ b/test.sh
@@ -1,14 +1,17 @@
#!/bin/sh
+set -e
action="noop"
+credentials="$(cat openwhisk/ansible/files/auth.guest)"
+host="172.17.0.1:10001"
# create a noop action
echo "Creating noop action"
-curl -u "$(cat openwhisk/ansible/files/auth.guest)" "172.17.0.1:10001/api/v1/namespaces/_/actions/$action" -XPUT -d '{"namespace":"_","name":"test","exec":{"kind":"nodejs:default","code":"function main(){return {};}"}}' -H "Content-Type: application/json"
+curl -u "$credentials" "$host/api/v1/namespaces/_/actions/$action" -XPUT -d '{"namespace":"_","name":"test","exec":{"kind":"nodejs:default","code":"function main(){return {};}"}}' -H "Content-Type: application/json"
# run the noop action
echo "Running noop action once to assert an intact system"
-curl -u "$(cat openwhisk/ansible/files/auth.guest)" "172.17.0.1:10001/api/v1/namespaces/_/actions/$action?blocking=true" -XPOST
+curl -u "$credentials" "$host/api/v1/namespaces/_/actions/$action?blocking=true" -XPOST
# run performance harness
-docker run --rm markusthoemmes/loadtest loadtest -n 10000 -H "Authorization: basic MjNiYzQ2YjEtNzFmNi00ZWQ1LThjNTQtODE2YWE0ZjhjNTAyOjEyM3pPM3haQ0xyTU42djJCS0sxZFhZRnBYbFBrY2NPRnFtMTJDZEFzTWdSVTRWck5aOWx5R1ZDR3VNREdJd1A=" "http://172.17.0.1:10001/api/v1/namespaces/_/actions/$action?blocking=true" -m POST
\ No newline at end of file
+docker run --rm markusthoemmes/loadtest loadtest -n 10000 -H "Authorization: basic $(echo $credentials | base64 -w 0)" "http://$host/api/v1/namespaces/_/actions/$action?blocking=true" -m POST
\ No newline at end of file
--
To stop receiving notification emails like this one, please contact
"commits@openwhisk.apache.org" <co...@openwhisk.apache.org>.
[incubator-openwhisk-performance] 06/22: Common docker setup
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markusthoemmes pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk-performance.git
commit 088d1f46a49b2a8f2c5af5af9693d7c142a91193
Author: Markus Thoemmes <ma...@de.ibm.com>
AuthorDate: Mon Apr 24 08:33:23 2017 +0200
Common docker setup
---
deploy.sh | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/deploy.sh b/deploy.sh
index 15bf1f6..fd3ac97 100755
--- a/deploy.sh
+++ b/deploy.sh
@@ -1,6 +1,11 @@
#!/bin/sh
-# checkout OpenWhisk latest
+# common docker setup
+sudo gpasswd -a travis docker
+sudo -E bash -c 'echo '\''DOCKER_OPTS="-H tcp://0.0.0.0:4243 -H unix:///var/run/docker.sock --storage-driver=overlay --userns-remap=default"'\'' > /etc/default/docker'
+sudo service docker restart
+
+# checkout openwhisk latest
git clone --depth 1 https://github.com/openwhisk/openwhisk.git
# install ansible
--
To stop receiving notification emails like this one, please contact
"commits@openwhisk.apache.org" <co...@openwhisk.apache.org>.
[incubator-openwhisk-performance] 15/22: Add documentation
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markusthoemmes pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk-performance.git
commit d1162b702673f3794de41a4a37726b052290676c
Author: Markus Thoemmes <ma...@de.ibm.com>
AuthorDate: Mon Apr 24 14:29:25 2017 +0200
Add documentation
---
README.md | 19 +++++++++++++++++--
1 file changed, 17 insertions(+), 2 deletions(-)
diff --git a/README.md b/README.md
index 305ddbb..8ced6d6 100644
--- a/README.md
+++ b/README.md
@@ -1,2 +1,17 @@
-# openwhisk-performance
-Performance harness for Apache OpenWhisk
+# Performance tests for Apache OpenWhisk
+A few simple but efficient performance test suites for Apache OpenWhisk. Determines the maximum throughput and end-user latency of the system.
+
+## Test setup
+- A standard OpenWhisk system is deployed. Note that the edge nginx router and API Gateway are left out currently.
+- Limits are set to 999999 each, for the test's load that means: No throttling at all.
+- The deployment uses a docker setup as proposed by the OpenWhisk development team: `overlay` driver and HTTP API enabled via a UNIX port.
+
+### Travis' machine setup
+The [machine provided by Travis](https://docs.travis-ci.com/user/ci-environment/#Virtualization-environments) has ~2 CPU cores (likely shared through virtualization) and 7.5GB memory.
+
+## Latency test
+The latency test determines the end-to-end latency a user experiences when doing a blocking invocation. The action used is a noop so the numbers returned are plain overhead of the OpenWhisk system.
+
+- 1 HTTP request at a time (concurrency: 1)
+- 10.000 samples with a single user
+- noop action
--
To stop receiving notification emails like this one, please contact
"commits@openwhisk.apache.org" <co...@openwhisk.apache.org>.
[incubator-openwhisk-performance] 03/22: Pull openwhisk images from
docker
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markusthoemmes pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk-performance.git
commit e32566130df540a1d7018b78c90135d05fd1d59f
Author: Markus Thoemmes <ma...@de.ibm.com>
AuthorDate: Mon Apr 24 08:08:55 2017 +0200
Pull openwhisk images from docker
---
deploy.sh | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/deploy.sh b/deploy.sh
new file mode 100755
index 0000000..4cd9415
--- /dev/null
+++ b/deploy.sh
@@ -0,0 +1,8 @@
+#!/bin/sh
+
+images="controller invoker"
+for i in $images
+do
+ docker pull "openwhisk/$i"
+ docker tag "openwhisk/$i" $i
+done
\ No newline at end of file
--
To stop receiving notification emails like this one, please contact
"commits@openwhisk.apache.org" <co...@openwhisk.apache.org>.
[incubator-openwhisk-performance] 08/22: Run a basic test
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markusthoemmes pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk-performance.git
commit 8423020576accdb96f3ac7aa61f3d4f63f4d0f95
Author: Markus Thoemmes <ma...@de.ibm.com>
AuthorDate: Mon Apr 24 09:27:02 2017 +0200
Run a basic test
---
.travis.yml | 2 +-
test.sh | 7 +++++++
2 files changed, 8 insertions(+), 1 deletion(-)
diff --git a/.travis.yml b/.travis.yml
index 15ae264..8b65bda 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -10,4 +10,4 @@ install:
- ./deploy.sh
script:
- - true
\ No newline at end of file
+ - ./test.sh
\ No newline at end of file
diff --git a/test.sh b/test.sh
new file mode 100755
index 0000000..2d8cd2e
--- /dev/null
+++ b/test.sh
@@ -0,0 +1,7 @@
+#!/bin/sh
+
+# create a noop action
+curl -u "$(cat openwhisk/ansible/files/auth.guest)" "172.17.0.1:10001/api/v1/namespaces/_/actions/curlTest" -XPUT -d '{"namespace":"_","name":"test","exec":{"kind":"nodejs:default","code":"function main(){return {};}"}}' -H "Content-Type: application/json"
+
+# run the noop action
+curl -u "$(cat openwhisk/ansible/files/auth.guest)" "172.17.0.1:10001/api/v1/namespaces/_/actions/curlTest?blocking=true" -XPOST
\ No newline at end of file
--
To stop receiving notification emails like this one, please contact
"commits@openwhisk.apache.org" <co...@openwhisk.apache.org>.
[incubator-openwhisk-performance] 12/22: Increase sample size for
the test
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markusthoemmes pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk-performance.git
commit b29486a9ce53acbe599e3d8479ffc3365ad639a6
Author: Markus Thoemmes <ma...@de.ibm.com>
AuthorDate: Mon Apr 24 11:12:52 2017 +0200
Increase sample size for the test
---
test.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/test.sh b/test.sh
index 30cbba7..e67ae9f 100755
--- a/test.sh
+++ b/test.sh
@@ -11,4 +11,4 @@ echo "Running noop action once to assert an intact system"
curl -u "$(cat openwhisk/ansible/files/auth.guest)" "172.17.0.1:10001/api/v1/namespaces/_/actions/$action?blocking=true" -XPOST
# run performance harness
-docker run --rm markusthoemmes/loadtest loadtest -n 1000 -H "Authorization: basic MjNiYzQ2YjEtNzFmNi00ZWQ1LThjNTQtODE2YWE0ZjhjNTAyOjEyM3pPM3haQ0xyTU42djJCS0sxZFhZRnBYbFBrY2NPRnFtMTJDZEFzTWdSVTRWck5aOWx5R1ZDR3VNREdJd1A=" "http://172.17.0.1:10001/api/v1/namespaces/_/actions/$action?blocking=true" -m POST
\ No newline at end of file
+docker run --rm markusthoemmes/loadtest loadtest -n 10000 -H "Authorization: basic MjNiYzQ2YjEtNzFmNi00ZWQ1LThjNTQtODE2YWE0ZjhjNTAyOjEyM3pPM3haQ0xyTU42djJCS0sxZFhZRnBYbFBrY2NPRnFtMTJDZEFzTWdSVTRWck5aOWx5R1ZDR3VNREdJd1A=" "http://172.17.0.1:10001/api/v1/namespaces/_/actions/$action?blocking=true" -m POST
\ No newline at end of file
--
To stop receiving notification emails like this one, please contact
"commits@openwhisk.apache.org" <co...@openwhisk.apache.org>.
[incubator-openwhisk-performance] 02/22: Base travis.yml,
checking out openwhisk
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markusthoemmes pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk-performance.git
commit 64f2447e77af0adb13ec670fab56d31bb02fda7e
Author: Markus Thoemmes <ma...@de.ibm.com>
AuthorDate: Mon Apr 24 07:59:20 2017 +0200
Base travis.yml, checking out openwhisk
---
.travis.yml | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/.travis.yml b/.travis.yml
new file mode 100644
index 0000000..f50ee9c
--- /dev/null
+++ b/.travis.yml
@@ -0,0 +1,13 @@
+sudo: required
+language: java
+services:
+ - docker
+
+env:
+ - TERM=dumb
+
+install:
+ - git clone https://github.com/openwhisk/openwhisk.git
+
+script:
+ - true
\ No newline at end of file
--
To stop receiving notification emails like this one, please contact
"commits@openwhisk.apache.org" <co...@openwhisk.apache.org>.
[incubator-openwhisk-performance] 04/22: Clone shallowly and hook
up deployment
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markusthoemmes pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk-performance.git
commit 15f9cf56c3564928d011ad561e9a1655ada0b587
Author: Markus Thoemmes <ma...@de.ibm.com>
AuthorDate: Mon Apr 24 08:13:39 2017 +0200
Clone shallowly and hook up deployment
---
.travis.yml | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/.travis.yml b/.travis.yml
index f50ee9c..a847366 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -7,7 +7,8 @@ env:
- TERM=dumb
install:
- - git clone https://github.com/openwhisk/openwhisk.git
+ - git clone --depth 1 https://github.com/openwhisk/openwhisk.git
+ - ./deploy.sh
script:
- true
\ No newline at end of file
--
To stop receiving notification emails like this one, please contact
"commits@openwhisk.apache.org" <co...@openwhisk.apache.org>.
[incubator-openwhisk-performance] 18/22: Bumping concurrency to see
if throuhput increases
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markusthoemmes pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk-performance.git
commit d2ffa1f195a11842b06bf0b32bb5e9a0509fd62a
Author: Markus Thoemmes <ma...@de.ibm.com>
AuthorDate: Mon Apr 24 15:24:21 2017 +0200
Bumping concurrency to see if throuhput increases
---
.travis.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/.travis.yml b/.travis.yml
index 9dd9bd2..845c8a6 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -10,4 +10,4 @@ install:
- ./deploy.sh
script:
- - ./test.sh "http://172.17.0.1:10001" "$(cat openwhisk/ansible/files/auth.guest)" 4
\ No newline at end of file
+ - ./test.sh "http://172.17.0.1:10001" "$(cat openwhisk/ansible/files/auth.guest)" 8
\ No newline at end of file
--
To stop receiving notification emails like this one, please contact
"commits@openwhisk.apache.org" <co...@openwhisk.apache.org>.
[incubator-openwhisk-performance] 17/22: Documentation for the
throughput test
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markusthoemmes pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk-performance.git
commit c58f77e94328491a7194d870f31685baad181607
Author: Markus Thoemmes <ma...@de.ibm.com>
AuthorDate: Mon Apr 24 15:00:23 2017 +0200
Documentation for the throughput test
---
README.md | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/README.md b/README.md
index 8ced6d6..2d208f9 100644
--- a/README.md
+++ b/README.md
@@ -15,3 +15,12 @@ The latency test determines the end-to-end latency a user experiences when doing
- 1 HTTP request at a time (concurrency: 1)
- 10.000 samples with a single user
- noop action
+
+**Note:** The throughput number has a 100% correlation with the latency in this case. This test does not serve to determine the maximum throughput of the system.
+
+## Throughput test
+The throughput test determines the maximum throughput a user can get out of the system while using a single action. The action used again is a noop, so the numbers are plain OpenWhisk overhead. Note that the throughput does not directly correlate to end-to-end latency here, as the system does more processing in the background as it shows to the user in a blocking invocation.
+
+- 4 HTTP requests at a time (concurrency: 4) (basically CPU cores * 2 to exploit some buffering)
+- 10.000 samples with a single user
+- noop action
\ No newline at end of file
--
To stop receiving notification emails like this one, please contact
"commits@openwhisk.apache.org" <co...@openwhisk.apache.org>.