You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@avro.apache.org by dk...@apache.org on 2018/11/26 18:27:41 UTC
[avro] branch master updated (f86da24 -> 2950a10)
This is an automated email from the ASF dual-hosted git repository.
dkulp pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/avro.git.
from f86da24 C: Namespace "" means space = NULL.
new fe58b5a AVRO-2269 Make Perf.java more usable.
new 008b190 Added build.sh flag to pass extra docker-run args, updated perf-doc to explain how to use.
new 2950a10 AVRO-2269 More documentation for using Perf.java
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
build.sh | 12 +-
doc/src/content/htmldocs/performance-testing.html | 173 +++++++++
.../ipc/src/test/java/org/apache/avro/io/Perf.java | 187 ++++++++--
share/test/run-perf.sh | 389 +++++++++++++++++++++
4 files changed, 725 insertions(+), 36 deletions(-)
create mode 100644 doc/src/content/htmldocs/performance-testing.html
create mode 100755 share/test/run-perf.sh
[avro] 02/03: Added build.sh flag to pass extra docker-run args,
updated perf-doc to explain how to use.
Posted by dk...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dkulp pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/avro.git
commit 008b19051b0de6b81a57b93442b341916a6acf8a
Author: rstata <rs...@yahoo.com>
AuthorDate: Sun Nov 25 16:06:20 2018 -0800
Added build.sh flag to pass extra docker-run args, updated perf-doc to explain how to use.
---
build.sh | 12 ++++++--
doc/src/content/htmldocs/performance-testing.html | 34 +++++++++++++++++++++++
2 files changed, 44 insertions(+), 2 deletions(-)
diff --git a/build.sh b/build.sh
index 0dc4788..10544df 100755
--- a/build.sh
+++ b/build.sh
@@ -20,9 +20,10 @@ set -e # exit on error
cd `dirname "$0"` # connect to root
VERSION=`cat share/VERSION.txt`
+DOCKER_XTRA_ARGS=""
function usage {
- echo "Usage: $0 {test|dist|sign|clean|docker|rat|githooks|docker-test}"
+ echo "Usage: $0 {test|dist|sign|clean|docker [--args \"docker-args\"]|rat|githooks|docker-test}"
exit 1
}
@@ -33,8 +34,10 @@ fi
set -x # echo commands
-for target in "$@"
+while (( "$#" ))
do
+ target="$1"
+ shift
case "$target" in
test)
@@ -200,6 +203,10 @@ do
;;
docker)
+ if [[ $1 =~ ^--args ]]; then
+ DOCKER_XTRA_ARGS=$2
+ shift 2
+ fi
docker build -t avro-build -f share/docker/Dockerfile .
if [ "$(uname -s)" == "Linux" ]; then
USER_NAME=${SUDO_USER:=$USER}
@@ -226,6 +233,7 @@ UserSpecificDocker
-v ${HOME}/.m2:/home/${USER_NAME}/.m2 \
-v ${HOME}/.gnupg:/home/${USER_NAME}/.gnupg \
-u ${USER_NAME} \
+ ${DOCKER_XTRA_ARGS} \
avro-build-${USER_NAME} bash
;;
diff --git a/doc/src/content/htmldocs/performance-testing.html b/doc/src/content/htmldocs/performance-testing.html
index f01c36a..5cd8026 100644
--- a/doc/src/content/htmldocs/performance-testing.html
+++ b/doc/src/content/htmldocs/performance-testing.html
@@ -81,10 +81,44 @@ As mentioned in the introduction, we tried a number of different mechanisms to r
<p> <li> Modified the code slightly, for example: starting the timer of a cycle after, rather than before, encoders or decoders are constructed; cacheing encoders and decoders; and reusing record objects during read tests rather than construct new ones for each record being read.
+<p> <li> Using Docker's <code>--cpuset-cpus</code> flag to force the tests onto a single core.
+
<p> <li> Using a dedicated EC2 instance (<code>c5d.2xlarge</code>).
</ul>
Of the above, the only change that made a significant difference was the last: in going from a laptop and desktop computer to a dedicated EC2 instances, we went from over 70 tests (out of 200) with a variance of 5% or more between runs to 35. As mentioned in the introduction, we should switch to a framework like <a href="https://java-performance.info/jmh/">JMH</a> to attack this problem more fundamentally.
+<p> If you want to setup your own EC2 instance for testing, here's how we did it. We launched a dedicated EC2 <code>c5d.2xlarge</code> instance from the AWS console, using the "Amazon Linux 64-bit HVM GP2" AMI. We logged into this instance and ran the following commands to install Docker and Git (we did all our Avro build and testing inside the Docker image):
+<pre>
+ sudo yum update
+ sudo yum install -y git-all
+ git config --global user.name "Your Name"
+ git config --global user.email email-address-used@github.com
+ git config --global core.editor emacs
+ sudo install -y docker
+ sudo usermod -aG docker ec2-user ## Need to log back in for this to take effect
+ sudo service docker start
+</pre>
+At this point you can checkout Avro and launch your Docker container:
+<pre>
+ git clone https://github.com/apache/avro.git
+ cd avro
+ ./build.sh docker --args "--cpuset-cpus 2,6"
+</pre>
+The <code>--args</code> flag in the last command deserves some explanation. In general, the <code>--args</code> allows you to pass additional arguments to the <code>docker run</code> command executed inside <code>build.sh</code>. In this case, the <code>--cpuset-cpus</code> flag for <code>docker</code> tells docker to schedule the contianer exclusively on the listed (virtual) CPUs. We identified vCPUs 2 and 6 using the <code>lscpu</code> Linux command:
+<pre>
+ [ec2-user@ip-0-0-0-0 avro]$ lscpu --extended
+ CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE
+ 0 0 0 0 0:0:0:0 yes
+ 1 0 0 1 1:1:1:0 yes
+ 2 0 0 2 2:2:2:0 yes
+ 3 0 0 3 3:3:3:0 yes
+ 4 0 0 0 0:0:0:0 yes
+ 5 0 0 1 1:1:1:0 yes
+ 6 0 0 2 2:2:2:0 yes
+ 7 0 0 3 3:3:3:0 yes
+</pre>
+Notice that (v)CPUs 2 and 6 are both on core 2: it's sufficient to schedule the container on the same core, vs a single vCPU.
+
<h1>Appendix A: Sample uses of run-perf.sh</h1>
[avro] 01/03: AVRO-2269 Make Perf.java more usable.
Posted by dk...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dkulp pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/avro.git
commit fe58b5a5d434c60bf3971a5dce4079bde33363d7
Author: rstata <rs...@yahoo.com>
AuthorDate: Fri Nov 2 19:31:34 2018 -0700
AVRO-2269 Make Perf.java more usable.
---
doc/src/content/htmldocs/performance-testing.html | 136 +++++++
.../ipc/src/test/java/org/apache/avro/io/Perf.java | 187 ++++++++--
share/test/run-perf.sh | 389 +++++++++++++++++++++
3 files changed, 678 insertions(+), 34 deletions(-)
diff --git a/doc/src/content/htmldocs/performance-testing.html b/doc/src/content/htmldocs/performance-testing.html
new file mode 100644
index 0000000..f01c36a
--- /dev/null
+++ b/doc/src/content/htmldocs/performance-testing.html
@@ -0,0 +1,136 @@
+<html>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<head>
+<title>Testing performance improvements</title>
+</head>
+
+<body>
+
+(Note: This document pertains only to the Java implementation Avro.)
+
+
+<h1>1.0 Introduction</h1>
+
+<p>Recent work on improving the performance of "specific record" (<a href="https://issues.apache.org/jira/browse/AVRO-2090">AVRO-2090</a> and <a href="https://issues.apache.org/jira/browse/AVRO-2247">AVRO-2247</a> has highlighted the need for a benchmark that can be used to test the validity of alleged performance "improvements."</p>
+
+<p> As a starting point, the Avro project has class called <code>Perf</code> (in the test source of the <code>ipc</code> subproject). <code>Perf</code> is a command-line tool contains close to 70 performance individual performance tests. These tests include tests for reading and writing primitive values, arrays and maps, plus tests for reading and writing records through all of the APIs (generic, specific, reflect).</p>
+
+<p> When using <code>Perf</code> for some recent performance work, we encountered two problems. First, because it depends on build artifacts from across the Avro project, it can be tricky to invoke. Second, and more seriously, independent runs of the tests in <code>Perf</code> can vary in performance by as much as 40%. While typical variance is less than that, the variance is high enough that it makes it impossible to tell if a change in performance is simply this noise, or can be pro [...]
+
+<p> This document addresses both problems, the usability problem in Section 2 and the variability issue in Section 3. Regarding the variability issue, as you will see, we haven't really been able to manage it in a fundamental manner. As <a href="https://issues.apache.org/jira/browse/AVRO-2269?focusedCommentId=16688925&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16688925">suggested by Zoltan Frakas</a>, we should look into porting <code>Perf</code> ov [...]
+
+
+<h1>2.0 Invoking <code>Perf</code></h1>
+
+<h2>2.1 Simple invocation</h2>
+
+<p>Here is the easiest way we found to directly invoke <code>Perf</code>.</p>
+
+<p>As mentioned in the Introduction, <code>Perf</code> is dependent upon build artifacts from some of the other Avro subprojects. When you invoke <code>Perf</code>, it should be invoked with your most recent build of those artifacts (assuming you're performance-testing your current work). We have found that the easiest way to ensure the proper artifacts are used is to use Maven to invoke <code>Perf</code>. </p>
+
+<p>The recipe for using Maven in this way is simple. First, from the <code>lang/java</code> directoy, you need to build <em>and install</em> Avro:</p>
+
+<p><code> mvn clean install</code></p>
+
+<p>(You can add <code>-DskipTests</code> to the above command line if you don't need to run test suite.) When this is done, you need to change your working directory to the <code>lang/java/ipc</code> directory. From there, you can invoke <code>Perf</code> with the following command line:</p>
+
+<p><code>
+ mvn exec:java -Dexec.classpathScope=test -Dexec.mainClass=org.apache.avro.io.Perf -Dexec.args="..."
+</code></p>
+
+<p>The <code>exec.args</code> string contains the arguments you want to pass through to the <code>Perf.main</code> function.</p>
+
+<p>To speed up your edit-compile-test loop, you can do a selective build of Avro in addition to skipping tests:
+
+<p><code> mvn clean && mvn -pl "avro,compiler,maven-plugin,ipc" install -DskipTests</code></p>
+
+
+
+<h2>2.2 Using the run-perf.sh script</h2>
+
+<p>If you're using <code>Perf</code>, chances are that you want to compare the performance of a proposed optimization against the performance of a baseline (that baseline most likely being the current master branch of Avro). Generating this comparative data can be tedious if you're running <code>Perf</code> by hand. To relieve this tedium, you can use the <code>run-perf.sh</code> script instead (found in the <code>share/test</code> directory from the Avro top-level directory).</p>
+
+<p>To use this script, you put different implementations of Avro onto different branches of your Avro git repository. One of these branches is designated the "baseline" branch and the others are the "treatment" branches. The script will run the baseline and all the treatments, and will compare generate a CSV file containing a comparison of the treatments against the baseline.</p>
+
+<p>Running <code>run-perf.sh --help</code> will output a detailed manual-page for this script. Appendix A of this document contains sample invocations of this test script for different use cases.</p>
+
+<p>NOTE: as mentioned in <code>run-perf.sh --help</code>, <b>this script is designed to be run from the <code>lang/java/ipc</code> directory</b>, which is the Maven project containing the <code>Perf</code> program.</p>
+
+
+
+<h1>3.0 Managing variance</h1>
+
+As mentioned in the introduction, we tried a number of different mechanisms to reduce variance, including:
+<ul>
+<li> Varying <code>org.apache.avro.io.perf.count</code>, <code>org.apache.io.perf.cycles</code>, and <code>org.apache.avro.io.perf.use-direct</code>, as well as the number of times we run <code>Perf.java</code> within a single "run" of a test.
+
+<p> <li> Taking the minimum times across runs, rather than the maximum times, using the second or third run as a baseline rather than the first, using statistical methods to eliminate outlying values.
+
+<p> <li> Modified the code slightly, for example: starting the timer of a cycle after, rather than before, encoders or decoders are constructed; cacheing encoders and decoders; and reusing record objects during read tests rather than construct new ones for each record being read.
+
+<p> <li> Using a dedicated EC2 instance (<code>c5d.2xlarge</code>).
+</ul>
+Of the above, the only change that made a significant difference was the last: in going from a laptop and desktop computer to a dedicated EC2 instances, we went from over 70 tests (out of 200) with a variance of 5% or more between runs to 35. As mentioned in the introduction, we should switch to a framework like <a href="https://java-performance.info/jmh/">JMH</a> to attack this problem more fundamentally.
+
+
+<h1>Appendix A: Sample uses of run-perf.sh</h1>
+
+<p>A detailed explanation of <code>run-perf.sh</code> is printed when you give it the <code>--help</code> flag. To help you more quickly understand how to use <code>run-perf.sh</code> we present here a few examples of how we used it in our recent testing efforts.
+
+<p> To summarize, you invoke it as follows:
+<pre>
+ ../../../share/test/run-perf.sh [--out-dir D] \
+ [--perf-args STRING] [-Dkey=value]* [--] \
+ [-Dkey=value]* branch_baseline[:name_baseline_run] \
+ [-Dkey=value]* branch_1[:name_treatment_run_1] \
+ ... <br>
+ [-Dkey=value]* branch_n[:name_treatment_run_n] <br>
+</pre>
+The path given here is relative to the <code>lang/java/ipc</code> directory, which needs to be the current working directory when calling this script. The script executes multiple <em>runs</em> of testing. The first run is called the <em>baseline run</em>, the subsequent runs are the <em>treatment runs</em>. Each run consists of four identical executions of <code>Perf.java</code>. The running times for each <code>Perf.java</code> test are averaged to obtain the final running time for [...]
+
+<p>The following invocation is what we used to measure the variance of <code>Perf.java</code>:
+<pre>
+../../../share/test/run-perf.sh --out-dir ~/calibration \
+ -Dorg.apache.avro.specific.use_custom_coders=true \
+ AVRO-2269:baseline AVRO-2269:run1 AVRO-2269:run2 AVRO-2269:run3
+</pre>
+In this invocation, the baseline run and all three treatment runs come from the same Git branch: <code>AVRO-2269</code>. We need to give a name to each run: in this case runs have been named "baseline"--the baseline run--and "run1", "run2", and "run3"--the treatment runs. Note that the name of the Git branch to be used for a run must always be provided, but the name for the run itself (e.g., "baseline") is optional. If a name for the run is not provided, then the name of the Git branc [...]
+
+<p><code>run-perf.sh</code> uses Maven to invoke <code>Perf.java</code>. The <code>-D</code> flag is used to pass system properties to Maven, which in turn will pass them through to <code>Perf.java</code>. In the example above, we use this flag to turn on the custom-coders feature recently checked into Avro. Note that initial <code>-D</code> flags will be passed to <em>all</em> runs, while <code>-D</code> switches that come just before the name of Git branch of a run apply to only tha [...]
+
+<p>Finally, note that <code>run-perf.sh</code> generates a lot of intermediate files as well as the final <code>summary.csv</code> file. Thus, it is recommended that the output of each execution of <code>run-pref.sh</code> is sent to a dedicated directory, provided by the <code>--out-dir</code> flag. If that directory does not exist, it will be created. (Observe that <code>run-perf.sh</code> outputs a file called <code>command.txt</code> containing the full command-line used to invoke [...]
+
+<p>The next invocation is what we used to ensure that the new "custom coders" optimization for specific records does indeed improve performance:
+<pre>
+../../../share/test/run-perf.sh --out-dir ~/retest-codegen \
+ --perf-args "-Sf" \
+ AVRO-2269:baseline \
+ -Dorg.apache.avro.specific.use_custom_coders=true AVRO-2269:custom-coders
+</pre>
+In this case, unlike the previous one, the <code>-D</code> flag that turns on the use of custom coders is applied specifically to the treatment run, and not globally. Also, since this flag only affects the Specific Record case, we use the <code>--perf-args</code> flag to pass additional arguments to <code>Perf.java</code>; in this case, the <code>-Sf</code> flag tells <code>Perf.java</code> to run just the specific-record related tests and not the entire test suite.
+
+<p>This last example shows how we checked the performance impact of two new feature-branches we've been developing:
+<pre>
+../../../share/test/run-perf.sh --out-dir ~/new-branches \
+ -Dorg.apache.avro.specific.use_custom_coders=true \
+ AVRO-2269:baseline combined-opts full-refactor
+</pre>
+In this case, once again, we turn on custom-coders for all runs. In this case, again, the Git branch <code>AVRO-2269</code> is used for our baseline run. However, in this case, the treatment runs come from two other Git branches: <code>combined-opts</code> and <code>full-refactor</code>. We didn't provide run-names for these runs because the Git branch-name were fine to be used as run names (we explicitly named the first run "baseline" not because we had to, but because we like the co [...]
+
+<p>Although we didn't state it before, in preparing for a run, <code>run-perf.sh</code> will checkout the Git branch to be used for the run and use <code>mvn install</code> to build and install it. It does this for each branch, so the invocation just given will checkout and build three different branches during its overall execution. (As an optimization, if one run uses the same branch as the previous run, then the branch is <em>not</em> checked-out or rebuilt between runs.)
+
+</body>
+</html>
diff --git a/lang/java/ipc/src/test/java/org/apache/avro/io/Perf.java b/lang/java/ipc/src/test/java/org/apache/avro/io/Perf.java
index 8606694..df4be24 100644
--- a/lang/java/ipc/src/test/java/org/apache/avro/io/Perf.java
+++ b/lang/java/ipc/src/test/java/org/apache/avro/io/Perf.java
@@ -102,6 +102,8 @@ public class Perf {
new TestDescriptor(StringTest.class, "-s").add(BASIC);
new TestDescriptor(ArrayTest.class, "-a").add(BASIC);
new TestDescriptor(MapTest.class, "-m").add(BASIC);
+ new TestDescriptor(ExtendedEnumResolveTest.class, "-ee").add(BASIC);
+ new TestDescriptor(UnchangedUnionResolveTest.class, "-uu").add(BASIC);
BATCHES.put("-record", RECORD);
new TestDescriptor(RecordTest.class, "-R").add(RECORD);
new TestDescriptor(ValidatingRecord.class, "-Rv").add(RECORD);
@@ -141,13 +143,14 @@ public class Perf {
private static final int BYTES_PS_FIELD = 2;
private static final int ENTRIES_PS_FIELD = 3;
private static final int BYTES_PC_FIELD = 4;
- private static final int MAX_FIELD = 4;
+ private static final int MIN_TIME_FIELD = 5;
+ private static final int MAX_FIELD_TAG = 5;
private static void usage() {
- StringBuilder usage = new StringBuilder("Usage: Perf [-o <file>] [-c <spec>] { -nowrite | -noread | ");
+ StringBuilder usage = new StringBuilder("Usage: Perf [-o <file>] [-c <spec>] { -nowrite | -noread }");
StringBuilder details = new StringBuilder();
details.append(" -o file (send output to a file)\n");
- details.append(" -c [n][t][e][b][c] (format as no-header CSV; include Name, Time, Entries/sec, Bytes/sec, and/or bytes/Cycle; no spec=all fields)\n");
+ details.append(" -c [n][t][e][b][c][m] (format as no-header CSV; include Name, Time, Entries/sec, Bytes/sec, bytes/Cycle, and/or min time/op; no spec=all fields)\n");
details.append(" -nowrite (do not execute write tests)\n");
details.append(" -noread (do not execute write tests)\n");
for (Map.Entry<String, List<TestDescriptor>> entry : BATCHES.entrySet()) {
@@ -179,6 +182,7 @@ public class Perf {
String outputfilename = null;
PrintStream out = System.out;
boolean[] csvFormat = null;
+ String csvFormatString = null;
for (int i = 0; i < args.length; i++) {
String a = args[i];
@@ -200,17 +204,20 @@ public class Perf {
continue;
}
if ("-c".equals(a)) {
- if (i == args.length-1 || args[i+1].startsWith("-"))
- csvFormat = new boolean[] { true, true, true, true, true };
- else {
- csvFormat = new boolean[5];
- for (char c : args[++i].toCharArray())
+ if (i == args.length-1 || args[i+1].startsWith("-")) {
+ csvFormatString = "ntebcm"; // For diagnostics
+ csvFormat = new boolean[] { true, true, true, true, true, true };
+ } else {
+ csvFormatString = args[++i];
+ csvFormat = new boolean[MAX_FIELD_TAG+1];
+ for (char c : csvFormatString.toCharArray())
switch (c) {
case 'n': csvFormat[NAME_FIELD] = true; break;
case 't': csvFormat[TIME_FIELD] = true; break;
case 'e': csvFormat[BYTES_PS_FIELD] = true; break;
case 'b': csvFormat[ENTRIES_PS_FIELD] = true; break;
case 'c': csvFormat[BYTES_PC_FIELD] = true; break;
+ case 'm': csvFormat[MIN_TIME_FIELD] = true; break;
default:
usage();
System.exit(1);
@@ -237,9 +244,12 @@ public class Perf {
}
}
System.out.println("Executing tests: \n" + tests + "\n readTests:" +
- readTests + "\n writeTests:" + writeTests + "\n cycles=" + CYCLES);
+ readTests + "\n writeTests:" + writeTests + "\n cycles=" + CYCLES +
+ "\n count=" + (COUNT / 1000) + "K");
if (out != System.out) System.out.println(" Writing to: " + outputfilename);
- if (csvFormat != null) System.out.println(" in CSV format.");
+ if (csvFormat != null) System.out.println(" CSV format: " + csvFormatString);
+
+ TestResult tr = new TestResult();
for (int k = 0; k < tests.size(); k++) {
Test t = tests.get(k);
@@ -275,28 +285,41 @@ public class Perf {
t.writeTest();
}
}
- t.reset();
+
// test
- long s = 0;
System.gc();
- t.init();
if (t.isReadTest() && readTests) {
+ tr.reset();
for (int i = 0; i < t.cycles; i++) {
- s += t.readTest();
+ tr.update(t.readTest());
}
- printResult(out, csvFormat, s, t, t.name + "Read");
+ printResult(out, csvFormat, tr, t, t.name + "Read");
}
- s = 0;
if (t.isWriteTest() && writeTests) {
+ tr.reset();
for (int i = 0; i < t.cycles; i++) {
- s += t.writeTest();
+ tr.update(t.writeTest());
}
- printResult(out, csvFormat, s, t, t.name + "Write");
+ printResult(out, csvFormat, tr, t, t.name + "Write");
}
t.reset();
}
}
+ private static class TestResult {
+ public long totalTime;
+ public long minTime;
+ public void reset() {
+ totalTime = 0L;
+ minTime = Long.MAX_VALUE;
+ }
+ public long update(long t) {
+ totalTime += t;
+ minTime = Math.min(t, minTime);
+ return t;
+ }
+ }
+
private static final void printHeader() {
String header = String.format(
"%60s time M entries/sec M bytes/sec bytes/cycle",
@@ -305,23 +328,25 @@ public class Perf {
}
private static final void printResult(PrintStream o, boolean[] csv,
- long s, Test t, String name)
+ TestResult tr, Test t, String name)
{
- s /= 1000;
+ long s = tr.totalTime / 1000;
double entries = (t.cycles * (double) t.count);
double bytes = t.cycles * (double) t.encodedSize;
StringBuilder result = new StringBuilder();
if (csv != null) {
boolean commaneeded = false;
- for (int i = 0; i <= MAX_FIELD; i++) {
+ for (int i = 0; i <= MAX_FIELD_TAG; i++) {
+ if (! csv[i]) continue;
if (commaneeded) result.append(",");
else commaneeded = true;
switch (i) {
case NAME_FIELD: result.append(name); break;
case TIME_FIELD: result.append(String.format("%d", (s/1000))); break;
case BYTES_PS_FIELD: result.append(String.format("%.3f", (entries / s))); break;
- case ENTRIES_PS_FIELD: result.append(String.format(".3%f", (bytes / s))); break;
+ case ENTRIES_PS_FIELD: result.append(String.format("%.3f", (bytes / s))); break;
case BYTES_PC_FIELD: result.append(String.format("%d", t.encodedSize)); break;
+ case MIN_TIME_FIELD: result.append(String.format("%d", tr.minTime)); break;
}
}
} else {
@@ -388,6 +413,13 @@ public class Perf {
* higher level constructs, just manual serialization.
*/
private static abstract class BasicTest extends Test {
+ /** Switch to using a DirectBinaryEncoder rather than a BufferedBinaryEncoder
+ * for writing tests. DirectBinaryEncoders are noticably slower than Buffered
+ * ones, but they can be more consistent in their performance, which can make
+ * it easier to detect small performance improvements. */
+ public static boolean USE_DIRECT_ENCODER
+ = Boolean.parseBoolean(System.getProperty("org.apache.avro.io.perf.use-direct","false"));
+
protected final Schema schema;
protected byte[] data;
BasicTest(String name, String json) throws IOException {
@@ -400,16 +432,16 @@ public class Perf {
@Override
public final long readTest() throws IOException {
- long t = System.nanoTime();
Decoder d = getDecoder();
+ long t = System.nanoTime();
readInternal(d);
return (System.nanoTime() - t);
}
@Override
public final long writeTest() throws IOException {
- long t = System.nanoTime();
Encoder e = getEncoder();
+ long t = System.nanoTime();
writeInternal(e);
e.flush();
return (System.nanoTime() - t);
@@ -428,8 +460,8 @@ public class Perf {
}
protected Encoder newEncoder(ByteArrayOutputStream out) throws IOException {
- Encoder e = encoder_factory.binaryEncoder(out, null);
-// Encoder e = encoder_factory.directBinaryEncoder(out, null);
+ Encoder e = (USE_DIRECT_ENCODER ? encoder_factory.directBinaryEncoder(out, null)
+ : encoder_factory.binaryEncoder(out, null));
// Encoder e = encoder_factory.blockingBinaryEncoder(out, null);
// Encoder e = new LegacyBinaryEncoder(out);
return e;
@@ -1419,18 +1451,13 @@ public class Perf {
protected final SpecificDatumReader<T> reader;
protected final SpecificDatumWriter<T> writer;
private Object[] sourceData;
+ private T reuse;
protected SpecificTest(String name, String writerSchema) throws IOException {
super(name, writerSchema, 48);
reader = newReader();
writer = newWriter();
}
- protected SpecificDatumReader<T> getReader() {
- return reader;
- }
- protected SpecificDatumWriter<T> getWriter() {
- return writer;
- }
protected SpecificDatumReader<T> newReader() {
return new SpecificDatumReader<>(schema);
}
@@ -1444,6 +1471,7 @@ public class Perf {
for (int i = 0; i < sourceData.length; i++) {
sourceData[i] = genSingleRecord(r);
}
+ reuse = genSingleRecord(r);
}
protected abstract T genSingleRecord(Random r);
@@ -1451,7 +1479,7 @@ public class Perf {
@Override
void readInternal(Decoder d) throws IOException {
for (int i = 0; i < count; i++) {
- getReader().read(null, d);
+ reader.read(reuse, d);
}
}
@Override
@@ -1459,7 +1487,7 @@ public class Perf {
for (int i = 0; i < sourceData.length; i++) {
@SuppressWarnings("unchecked")
T rec = (T) sourceData[i];
- getWriter().write(rec, e);
+ writer.write(rec, e);
}
}
@Override
@@ -1815,4 +1843,95 @@ public class Perf {
return new Rec1(r);
}
}
+
+ static abstract class ResolvingTest extends BasicTest {
+ GenericRecord[] sourceData = null;
+ Schema writeSchema;
+
+ private static String mkSchema(String subschema) {
+ return ("{ \"type\": \"record\", \"name\": \"R\", \"fields\": [\n"
+ + "{ \"name\": \"f\", \"type\": " + subschema + "}\n"
+ + "] }");
+ }
+
+ public ResolvingTest(String name, String r, String w) throws IOException {
+ super(name, mkSchema(r));
+ isWriteTest = false;
+ this.writeSchema = new Schema.Parser().parse(mkSchema(w));
+ }
+
+ @Override
+ protected Decoder getDecoder() throws IOException {
+ return new ResolvingDecoder(writeSchema, schema, super.getDecoder());
+ }
+
+ @Override
+ void readInternal(Decoder d) throws IOException {
+ GenericDatumReader<Object> reader = new GenericDatumReader<>(schema);
+ for (int i = 0; i < count; i++) {
+ reader.read(null, d);
+ }
+ }
+
+ @Override
+ void writeInternal(Encoder e) throws IOException {
+ GenericDatumWriter<Object> writer = new GenericDatumWriter<>(writeSchema);
+ for (int i = 0; i < sourceData.length; i++) {
+ writer.write(sourceData[i], e);
+ }
+ }
+
+ @Override
+ void reset() {
+ sourceData = null;
+ data = null;
+ }
+ }
+
+ static class ExtendedEnumResolveTest extends ResolvingTest {
+ private static final String ENUM_WRITER =
+ "{ \"type\": \"enum\", \"name\":\"E\", \"symbols\": [\"A\", \"B\"] }";
+ private static final String ENUM_READER =
+ "{ \"type\": \"enum\", \"name\":\"E\", \"symbols\": [\"A\",\"B\",\"C\",\"D\",\"E\"] }";
+
+ public ExtendedEnumResolveTest() throws IOException {
+ super("ExtendedEnum", ENUM_READER, ENUM_WRITER);
+ }
+
+ @Override
+ void genSourceData() {
+ Random r = newRandom();
+ Schema eSchema = writeSchema.getField("f").schema();
+ sourceData = new GenericRecord[count];
+ for (int i = 0; i < sourceData.length; i++) {
+ GenericRecord rec = new GenericData.Record(writeSchema);
+ int tag = r.nextInt(2);
+ rec.put("f", GenericData.get().createEnum(eSchema.getEnumSymbols().get(tag), eSchema));
+ sourceData[i] = rec;
+ }
+ }
+ }
+
+ static class UnchangedUnionResolveTest extends ResolvingTest {
+ private static final String UNCHANGED_UNION =
+ "[ \"null\", \"int\" ]";
+
+ public UnchangedUnionResolveTest() throws IOException {
+ super("UnchangedUnion", UNCHANGED_UNION, UNCHANGED_UNION);
+ }
+
+ @Override
+ void genSourceData() {
+ Random r = newRandom();
+ Schema uSchema = writeSchema.getField("f").schema();
+ sourceData = new GenericRecord[count];
+ for (int i = 0; i < sourceData.length; i++) {
+ GenericRecord rec = new GenericData.Record(writeSchema);
+ int val = r.nextInt(1000000);
+ Integer v = (val < 750000 ? new Integer(val) : null);
+ rec.put("f", v);
+ sourceData[i] = rec;
+ }
+ }
+ }
}
diff --git a/share/test/run-perf.sh b/share/test/run-perf.sh
new file mode 100755
index 0000000..7aa8b0a
--- /dev/null
+++ b/share/test/run-perf.sh
@@ -0,0 +1,389 @@
+#!/bin/bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+set -ex
+
+function usage {
+ echo "`basename $0` --help"
+ echo "`basename $0` [--min] [--out-dir D] [--iters N] [--skip-one]\\"
+ echo " [--only-combine] [--perf-args STRING] [-Dkey=value]* \\"
+ echo " [--] [-Dkey=value]* branch_1[:name_1] .. [-Dkey=value]* branch_n[:name_n]"
+ echo
+ echo "Run a set of trials Perf.java trials can compare results."
+ echo "A 'trial' is N runs of Perf.java against the code as it"
+ echo "exists on a branch in git. By comparing Perf.java output"
+ echo "generated by different branches in git, we can understand"
+ echo "the relative performance of those branches."
+ echo
+ echo "This script must be run in the lang/java/ipc directory of"
+ echo "the Avro source code, on a computer where Maven is installed"
+ echo "and the other build-prerequisites of Avro are in place. This"
+ echo "script will do a 'mvn clean install' of Avro from withing"
+ echo "the lang/java directory, before running tests."
+ echo
+ echo "The way Perf.java works is that it times an 'inner loop' that"
+ echo "reads or writes a large number of records (the exact number can"
+ echo "be controlled by a system property as described below). This"
+ echo "inner loop is called a 'cycle.' Perf.java runs a medium number of"
+ echo "these cycles, and outputs either the average or the minimum"
+ echo "of their running times. This script runs Perf.java a small number"
+ echo "of times (controllable by the --iters flag), and takes either the"
+ echo "average or minimum of those. The result of all this is the results"
+ echo "of a single 'trial.'"
+ echo
+ echo "The basic model is that there is a 'baseline' trial plus any"
+ echo "number of 'treatment' trials. The goal is to compare the"
+ echo "performance of each treatment against the baseline. The main"
+ echo "output is written to the file 'summary.csv'. This file contains"
+ echo "one line per performance test run by Perf.java. Each row contains"
+ echo "a 'results' column for each trial, followed by a 'comparison' column"
+ echo "for each treatment trial. The results column contains the average"
+ echo "(or minimum) of the runtimes of all cycles over all iterations of"
+ echo "the trial. The comparison columns contains the difference between"
+ echo "the performance of the treatment and the baseline, as a percent"
+ echo "of the baseline. Specifically, it countains"
+ echo " 100*(treatment-baseline)/baseline, i.e., positive numbers mean"
+ echo "we've seen a speedup."
+ echo
+ echo "By default, the running times of cycles are averaged together."
+ echo "The --min flag changes that to taking the minimum."
+ echo
+ echo "By default, output is written to the current working directory."
+ echo "However, lots of intermediate files are generated, so it's recommended"
+ echo "that the --out-dir argument is used to redirect the output to"
+ echo "a different working directory."
+ echo
+ echo "By default, the number of iterations in a trial is 4, but this can"
+ echo "be changed with the --iters flag."
+ echo
+ echo "Perf.java takes a number of command-line arguments, and can be"
+ echo "influenced by system properties. Command-line arguments can be"
+ echo "passed using the --perf-args flag. When using this switch, pass"
+ echo "your Perf.java command-line arguments in a single string, even if"
+ echo "there are more than one of them. You can set system properties"
+ echo " using -Dkey=value switch, just as you would with Maven. System"
+ echo "properties that come before the the '--' switch and the first"
+ echo "branch are passed to all trials. System properties that come after"
+ echo "the '--' switch and/or first branch are passed to the branch that"
+ echo "follows them. Commonly used system properties include:"
+ echo
+ echo " org.apache.avro.io.perf.count -- the number of elements"
+ echo "generated for the inner-most loop of the performance test. Defaults"
+ echo "to 250K. Must be a multple of 4."
+ echo
+ echo " org.apache.avro.io.perf.cycles -- the number of times the inner-"
+ echo "most loop is called within an invocation of Perf.java. Defaults"
+ echo " to 800."
+ echo
+ echo " org.apache.avro.io.perf.use-direct -- use DirectBinaryEncoder instead"
+ echo "of BufferedBinaryEncoder for write tests. It is slower, but performance-wise"
+ echo "it can be more consistent, which helps when trying to detect small performance"
+ echo "improvements."
+ echo
+ echo " org.apache.avro.specific.use_custom_coders -- flag that turns on"
+ echo " the use of the custom-coder optimization in the SpecificRecord tests."
+ echo "Defaults to 'false;' set to 'true' to turn them on."
+ echo
+ echo "Trials, as indicated, are branches in git. The branch_i arguments"
+ echo " indicate which what branches make up a trial. The first of these"
+ echo "(branch_1) is considered the \"baseline\" trial: it's the trial"
+ echo "that all the others are compared against. (However, if the --skip-one"
+ echo "is provided, the result from the first trial is ignored and the second"
+ echo "becomes the baseline.)"
+ echo
+ echo "Each trial has a name as well as a branch. By default, the name of"
+ echo "the branch is the name of the trial, but an explicit name can be given"
+ echo "by suffixing the branch name with a trial name (e.g., 'foo:bar' will"
+ echo "use the branch 'foo' for a trial, but the trial will be named 'bar')."
+ echo "Trials must have unique names, so when multiple trials are run off the"
+ echo "same branch, explicit trial names must be used."
+ echo
+ echo "In addition to writing 'summary.csv', this script outputs other files,"
+ echo "allowing you to analyze the granular results of a test run. The file"
+ echo "results.csv contains a row per test in Perf.java. Each column"
+ echo "contains the result of a single run of Perf.java. If N is the"
+ echo "number of iterations in a trial, then the first N columns are the"
+ echo "results from the individual iterations of the first trial, the"
+ echo "next N are the results from the second trial, and so forth. In"
+ echo "addition, for each branch B being tested, there are multiple"
+ echo "files 'B_i.csv' for each iteration i in the trial. These per-trial"
+ echo "files have two columns, the first being the name of the test, the"
+ echo "second being the result of that test. Thus, 'result.csv' is the"
+ echo "result of joining these per-trial files on the trial-name, and"
+ echo "summary.csv averages (or takes the minimum) of these per-trial"
+ echo "results, and adds the comparison column."
+ echo
+ echo "If the --only-combine flag is given, then the script will assume"
+ echo "that the B_i files have been generated, and will simply join them"
+ echo "to compute results.csv and summary.csv. This allows you to debug"
+ echo "the code that combines these files without having to wait around"
+ echo "for Perf.java to be run a bunch of times."
+}
+
+if [[ "$1" == "--help" ]]; then
+ usage
+ exit 0
+fi
+
+if [[ ! `pwd` =~ java/ipc ]]; then
+ echo "Must be run from lang/java/ipc"
+ echo "Type `basename $0` --help for help"
+ exit 1
+fi
+
+TEST="-c nt"
+EXTRA_CLI=""
+OUT="."
+SKIP_ONE="false"
+STATIC_SYSPROPS=()
+ITERS=4
+
+# DBG=echo
+
+function Perf_java {
+ local fname=$1
+ shift
+
+ if [[ "$DBG" != "" ]]; then
+ $DBG MAVEN_OPTS=-server mvn exec:java -Dexec.classpathScope=test \
+ -Dexec.mainClass=org.apache.avro.io.Perf ${STATIC_SYSPROPS[@]} \
+ -Dexec.args="${TEST} -o ${fname} ${EXTRA_CLI}" \
+ $@
+ else
+ mvn exec:java -Dexec.classpathScope=test \
+ -Dexec.mainClass=org.apache.avro.io.Perf ${STATIC_SYSPROPS[@]} \
+ -Dexec.args="${TEST} -o ${fname} ${EXTRA_CLI}" \
+ $@
+ fi
+}
+
+function run_trial {
+ local lastbranch=$1
+ local thisbranch=$2
+ local thistrialname=$3
+ shift 3
+
+ if [[ "$thisbranch" != "$lastbranch" ]]; then
+ $DBG git checkout $thisbranch
+ (cd ..; $DBG mvn clean && $DBG mvn -pl "avro,compiler,maven-plugin,ipc" install -DskipTests)
+ fi
+ for i in $(seq 1 ${ITERS}); do Perf_java ${OUT}/${thistrialname}${i}.csv $@; done
+}
+
+function run_trials {
+ local -a allprops=( )
+
+ while (( "$#" )); do
+ case "$1" in
+ --)
+ break;
+ ;;
+ *)
+ allprops+=( $1 )
+ shift
+ ;;
+ esac
+ done
+
+ local -a thisprops=( )
+ local lastbranch=""
+ local thisbranch
+ local thistrialname
+
+ while (( "$#" )); do
+ case "$1" in
+ --) # Ignore these
+ shift
+ ;;
+ -D*)
+ thisprops+=( $1 )
+ shift
+ ;;
+ *)
+ thisbranch=$1
+ thistrialname=$2
+ git rev-parse --verify $thisbranch
+ run_trial "$lastbranch" $thisbranch $thistrialname ${allprops[@]} ${thisprops[@]}
+ lastbranch=$thisbranch
+ thisprops=( )
+ shift 2
+ ;;
+ esac
+ done
+}
+
+function join_results {
+ pushd ${OUT}
+ local header="TestName"
+ for b in $@; do
+ for i in $(seq 1 ${ITERS}); do
+ header="${header},${b}${i}"
+ done
+ done
+# echo $header > results.csv
+ if [[ "$SKIP_ONE" == "true" ]]; then shift; fi
+ cut -d , -f 1,2 ${1}1.csv | sort >> results.csv
+ if [[ 1 < "${ITERS}" ]]; then
+ for i in $(seq 2 ${ITERS}); do
+ cut -d , -f 1,2 ${1}$i.csv | sort | join -t , results.csv - > tmp.csv
+ mv tmp.csv results.csv
+ done
+ fi
+ shift
+ for b in $@; do
+ for i in $(seq 1 ${ITERS}); do
+ cut -d , -f 1,2 ${b}$i.csv | sort | join -t , results.csv - > tmp.csv
+ mv tmp.csv results.csv
+ done
+ done
+ popd
+}
+
+AVG='BEGIN { RS=" "; } { s += $1; n += 1; } END { printf "%f", s/n; }'
+MIN='BEGIN { RS=" "; m = 10000000000; } { if ($1 < m) m = $1; } END { printf "%f", m; }'
+PERCENT='{ printf "%f", 100*($1-$2)/$1; }'
+
+function print_line {
+ local line=$1
+ shift
+ local awks
+ if [[ "$TEST" == "-c nt" ]]; then awks="$AVG"; else awks="$MIN"; fi
+
+ local -a results=( )
+ for t in ${trials[*]}; do
+ local result=""
+ for i in $(seq 1 $ITERS); do
+ result="$result $1"
+ shift
+ done
+ result=$(echo $result | awk "$awks")
+ results+=( $result )
+ line="${line},${result}"
+ done
+
+ local baseline=0
+ if [[ "$SKIP_ONE" == "true" ]]; then start=1; fi
+ for i in $(seq `expr ${baseline} + 1` `expr ${#trials[*]} - 1`); do
+ result=$(echo "${results[$baseline]} ${results[$i]}" | awk "$PERCENT")
+ line="${line},${result}"
+ done
+ echo "$line"
+}
+
+
+
+###
+### ACTUAL SCRIPT STARTS HERE
+###
+
+declare command="$0 $*"
+declare onlycombine="false"
+declare -a run_trials_args=( )
+declare -a trials=( )
+
+while (( "$#" )); do
+ case "$1" in
+ --help)
+ usage
+ exit
+ ;;
+ --min)
+ TEST="-c nm"
+ shift
+ ;;
+ --out-dir)
+ if [[ $OUT != "." ]]; then
+ echo "Cannot use --out-dir twice."
+ echo "Type `basename $0` --help for help"
+ exit 1
+ fi
+ OUT=$2
+ mkdir -p $OUT
+ shift 2
+ ;;
+ --iters)
+ ITERS=$2
+ shift 2
+ ;;
+ --only-combine)
+ onlycombine="true"
+ shift
+ ;;
+ --skip-one)
+ SKIP_ONE="true"
+ shift
+ ;;
+ --perf-args)
+ EXTRA_CLI=$2
+ shift 2
+ ;;
+ -D*)
+ if [[ ! $1 =~ ^-D[^\ =]+= ]]; then
+ echo "Bad system property: $1"
+ echo "Type `basename $0` --help for help"
+ exit 1
+ fi
+ run_trials_args+=( $1 )
+ shift
+ ;;
+ --)
+ run_trials_args+=( $1 )
+ shift
+ ;;
+ --*)
+ echo "Unknown switch: $1"
+ echo "Type `basename $0` --help for help"
+ exit 1
+ ;;
+ *)
+ if [[ "$1" =~ ^([^:]*):(.*) ]]; then
+ thisbranch=${BASH_REMATCH[1]}
+ thistrialname=${BASH_REMATCH[2]}
+ else
+ thisbranch=$1
+ thistrialname=$1
+ fi
+ if [[ "$thisbranch" == "" || "$thistrialname" == "" ]]; then
+ echo "Neither branch ($thisbranch) nor trial ($thistrialname) names may be empty"
+ echo "Type `basename $0` --help for help"
+ exit 1
+ fi
+ if [[ "${trials[@]}" =~ $thistrialname ]]; then
+ echo "Trial named '$thistrialname' is not unique"
+ echo "Type `basename $0` --help for help"
+ exit 1
+ fi
+ trials+=( "$thistrialname" )
+ run_trials_args+=( "--" "$thisbranch" "$thistrialname" )
+ shift
+ ;;
+ esac
+done
+
+# Document how the outputs were generated
+echo "$command" > $OUT/command.txt
+
+if [[ ${onlycombine} == "false" ]]; then
+ run_trials ${run_trials_args[@]}
+fi
+
+join_results ${trials[@]}
+
+cat $OUT/results.csv | while read line; do
+ fields=( $(echo $line | tr "," " ") )
+ print_line "${fields[@]}"
+done > $OUT/summary.csv
[avro] 03/03: AVRO-2269 More documentation for using Perf.java
Posted by dk...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dkulp pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/avro.git
commit 2950a10eb9bbaf4f6a52f4692f6a151e00997cd1
Author: rstata <rs...@yahoo.com>
AuthorDate: Sun Nov 25 16:30:00 2018 -0800
AVRO-2269 More documentation for using Perf.java
---
doc/src/content/htmldocs/performance-testing.html | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/doc/src/content/htmldocs/performance-testing.html b/doc/src/content/htmldocs/performance-testing.html
index 5cd8026..fcab40d 100644
--- a/doc/src/content/htmldocs/performance-testing.html
+++ b/doc/src/content/htmldocs/performance-testing.html
@@ -102,9 +102,12 @@ At this point you can checkout Avro and launch your Docker container:
<pre>
git clone https://github.com/apache/avro.git
cd avro
+ screen
./build.sh docker --args "--cpuset-cpus 2,6"
</pre>
-The <code>--args</code> flag in the last command deserves some explanation. In general, the <code>--args</code> allows you to pass additional arguments to the <code>docker run</code> command executed inside <code>build.sh</code>. In this case, the <code>--cpuset-cpus</code> flag for <code>docker</code> tells docker to schedule the contianer exclusively on the listed (virtual) CPUs. We identified vCPUs 2 and 6 using the <code>lscpu</code> Linux command:
+Note the use of <code>screen</code> here: executions of <code>run-perf.sh</code> can take a few hours, depending on the configuration. By running it inside of <code>screen</code>, you are protected from an SSH disconnection causing <code>run-perf.sh</code> to prematurely terminate.
+
+<p>The <code>--args</code> flag in the last command deserves some explanation. In general, the <code>--args</code> allows you to pass additional arguments to the <code>docker run</code> command executed inside <code>build.sh</code>. In this case, the <code>--cpuset-cpus</code> flag for <code>docker</code> tells docker to schedule the contianer exclusively on the listed (virtual) CPUs. We identified vCPUs 2 and 6 using the <code>lscpu</code> Linux command:
<pre>
[ec2-user@ip-0-0-0-0 avro]$ lscpu --extended
CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE
@@ -117,7 +120,7 @@ The <code>--args</code> flag in the last command deserves some explanation. In
6 0 0 2 2:2:2:0 yes
7 0 0 3 3:3:3:0 yes
</pre>
-Notice that (v)CPUs 2 and 6 are both on core 2: it's sufficient to schedule the container on the same core, vs a single vCPU.
+Notice that (v)CPUs 2 and 6 are both on core 2: it's sufficient to schedule the container on the same core, vs a single vCPU. One final tip: to confirm that your container is running on the expected CPUs, run <code>top</code> and then press the <code>1</code> key -- this will show you the load on each individual CPU.
<h1>Appendix A: Sample uses of run-perf.sh</h1>