You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@geode.apache.org by up...@apache.org on 2018/11/27 23:55:06 UTC
[geode-benchmarks] branch develop updated: Scripts to run
benchmarks against a baseline version of geode
This is an automated email from the ASF dual-hosted git repository.
upthewaterspout pushed a commit to branch develop
in repository https://gitbox.apache.org/repos/asf/geode-benchmarks.git
The following commit(s) were added to refs/heads/develop by this push:
new b536d69 Scripts to run benchmarks against a baseline version of geode
b536d69 is described below
commit b536d69ddb15471f189da5bdba7eef128121fca4
Author: Dan Smith <up...@apache.org>
AuthorDate: Tue Nov 27 13:50:08 2018 -0800
Scripts to run benchmarks against a baseline version of geode
Script to run the benchmarks with two different versions of geode for
comparision purposes.
---
geode-benchmarks/build.gradle | 2 +-
harness/build.gradle | 2 +-
infrastructure/google_cloud/README.md | 45 ++++++++++++++++++++++
.../{run_tests.sh => run_against_baseline.sh} | 27 +++----------
infrastructure/google_cloud/run_tests.sh | 17 ++++----
5 files changed, 63 insertions(+), 30 deletions(-)
diff --git a/geode-benchmarks/build.gradle b/geode-benchmarks/build.gradle
index 9b90312..69c8352 100644
--- a/geode-benchmarks/build.gradle
+++ b/geode-benchmarks/build.gradle
@@ -19,7 +19,7 @@ plugins {
id 'java'
}
-group 'org.upthewaterspout.geode-performance'
+group 'org.apache.geode-benchmark'
version '1.0-SNAPSHOT'
sourceCompatibility = 1.8
diff --git a/harness/build.gradle b/harness/build.gradle
index 7fc72e9..bc33562 100644
--- a/harness/build.gradle
+++ b/harness/build.gradle
@@ -19,7 +19,7 @@ plugins {
id 'java'
}
-group 'org.upthewaterspout.geode-performance'
+group 'org.apache.geode-benchmark'
version '1.0-SNAPSHOT'
sourceCompatibility = 1.8
diff --git a/infrastructure/google_cloud/README.md b/infrastructure/google_cloud/README.md
new file mode 100644
index 0000000..85830a5
--- /dev/null
+++ b/infrastructure/google_cloud/README.md
@@ -0,0 +1,45 @@
+# Benchmark Utilities for Google Cloud Platform
+
+
+These utilities create instances and run tests in your google cloud account. They assume that
+you have installed the gcloud utility and set your default project, region, and zone.
+
+# build_image.sh
+`build_image.sh` creates a compute image suitable for launching benchmarks on. It takes no
+arguments. Run this before running any other scripts if you are running benchmarks in a GCP project
+you've never run benchmarks in before.
+
+# launch_cluster.sh
+`launch_cluster.sh` creates an instance group in GCP based on an image created by `build_image.sh`.
+
+It takes three arguments. First, a tag to identify the cluster for use with other utilities. The
+second argument is the number of instances to create. The third and final argument is the subnet to
+create the instances in. In general this should be specified as
+`projects/<gcp-project>/regions/<region>/subnetworks/<subnet-name>`
+
+# run_tests.sh
+Runs benchmark tests against a single branch of `geode`. Arguments are (in order)
+
+* tag (the same as the cluster launched via `launch_cluster.sh`)
+* branch of geode (must exist in the apache geode repository)
+* output directory for results
+* branch of benchmark code to use (must exist in the apache geode-benchmarks repository)
+
+# run_against_baseline.sh
+Runs benchmark tests against two branches of geode for comparison purposes. Arguments are (in order)
+
+* tag (the same as the cluster launched via `launch_cluster.sh`)
+* branch of geode (must exist in the apache geode repository)
+* branch of benchmark code to use (must exist in the apache geode-benchmarks repository)
+
+
+# destroy_cluster.sh
+Destroys a cluster that you created. Arguments are the tag that you passed to launch_cluster.sh
+
+#Example
+```bash
+./build_image.sh
+./launch_cluster.sh mycluster 4
+./run-tests mycluster
+./destroy_cluster.sh
+```
\ No newline at end of file
diff --git a/infrastructure/google_cloud/run_tests.sh b/infrastructure/google_cloud/run_against_baseline.sh
similarity index 54%
copy from infrastructure/google_cloud/run_tests.sh
copy to infrastructure/google_cloud/run_against_baseline.sh
index 86f058b..63063cb 100755
--- a/infrastructure/google_cloud/run_tests.sh
+++ b/infrastructure/google_cloud/run_against_baseline.sh
@@ -17,27 +17,12 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-set -e
-
+DATE=$(date '+%m-%d-%Y-%H-%M-%S')
TAG=${1}
BRANCH=${2:-develop}
-PREFIX="geode-performance-${TAG}"
-DATE=$(date '+%m-%d-%Y-%H-%M-%S')
-OUTPUT=output-${DATE}
-
-INSTANCES=$(gcloud compute instance-groups list-instances ${PREFIX} | grep "${TAG}" | awk '{print $1}')
-
-
-HOSTS=$(echo ${INSTANCES} | tr ' ' ',')
-
-FIRST_INSTANCE=$(echo ${INSTANCES} | awk '{print $1}' )
-
-gcloud compute ssh geode@$FIRST_INSTANCE --command="\
- rm -rf geode-benchmarks && \
- git clone --depth=1 https://github.com/apache/geode --branch ${BRANCH} && \
- git clone https://github.com/apache/geode-benchmarks && \
- cd geode-benchmarks && \
- ./gradlew --include-build ../geode benchmark -Phosts=${HOSTS}"
+BASELINE=${3:-"rel/v1.7.0"}
+BENCHMARK_BRANCH=${4:-develop}
+OUTPUT=output-${DATE}-${TAG}
-mkdir -p ${OUTPUT}
-gcloud compute scp --recurse geode@${FIRST_INSTANCE}:geode-benchmarks/geode-benchmarks/build/benchmarks ${OUTPUT}
+./run_tests.sh ${TAG} ${BRANCH} ${OUTPUT}/branch ${BENCHMARK_BRANCH}
+./run_tests.sh ${TAG} ${BASELINE} ${OUTPUT}/baseline ${BENCHMARK_BRANCH}
diff --git a/infrastructure/google_cloud/run_tests.sh b/infrastructure/google_cloud/run_tests.sh
index 86f058b..477a75a 100755
--- a/infrastructure/google_cloud/run_tests.sh
+++ b/infrastructure/google_cloud/run_tests.sh
@@ -16,14 +16,13 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
-
-set -e
+DATE=$(date '+%m-%d-%Y-%H-%M-%S')
TAG=${1}
BRANCH=${2:-develop}
+OUTPUT=${3:-output-${DATE}-${TAG}}
+BENCHMARK_BRANCH=${4:-develop}
PREFIX="geode-performance-${TAG}"
-DATE=$(date '+%m-%d-%Y-%H-%M-%S')
-OUTPUT=output-${DATE}
INSTANCES=$(gcloud compute instance-groups list-instances ${PREFIX} | grep "${TAG}" | awk '{print $1}')
@@ -33,11 +32,15 @@ HOSTS=$(echo ${INSTANCES} | tr ' ' ',')
FIRST_INSTANCE=$(echo ${INSTANCES} | awk '{print $1}' )
gcloud compute ssh geode@$FIRST_INSTANCE --command="\
- rm -rf geode-benchmarks && \
- git clone --depth=1 https://github.com/apache/geode --branch ${BRANCH} && \
- git clone https://github.com/apache/geode-benchmarks && \
+ rm -rf geode-benchmarks geode && \
+ git clone --depth=1 https://github.com/apache/geode --branch ${BRANCH} geode && \
+ git clone https://github.com/apache/geode-benchmarks --branch ${BENCHMARK_BRANCH} && \
cd geode-benchmarks && \
./gradlew --include-build ../geode benchmark -Phosts=${HOSTS}"
+
mkdir -p ${OUTPUT}
+
+gcloud compute scp --recurse geode@${FIRST_INSTANCE}:geode-benchmarks/geode-benchmarks/build/reports ${OUTPUT}/reports
+
gcloud compute scp --recurse geode@${FIRST_INSTANCE}:geode-benchmarks/geode-benchmarks/build/benchmarks ${OUTPUT}