You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ignite.apache.org by yz...@apache.org on 2017/02/16 17:36:22 UTC
[21/50] [abbrv] ignite git commit: IGNITE-4212 Ignite Benchmarking
Simplification and Automation / IGNITE-4478 Fixing documentation and
resolving lack of usability
IGNITE-4212 Ignite Benchmarking Simplification and Automation / IGNITE-4478 Fixing documentation and resolving lack of usability
Project: http://git-wip-us.apache.org/repos/asf/ignite/repo
Commit: http://git-wip-us.apache.org/repos/asf/ignite/commit/605d946a
Tree: http://git-wip-us.apache.org/repos/asf/ignite/tree/605d946a
Diff: http://git-wip-us.apache.org/repos/asf/ignite/diff/605d946a
Branch: refs/heads/ignite-comm-balance-master
Commit: 605d946ada5ca4e562e256fa0edb29cc25dea7dc
Parents: e1c3dda
Author: oleg-ostanin <oo...@gridgain.com>
Authored: Thu Feb 9 20:08:57 2017 +0300
Committer: Anton Vinogradov <av...@apache.org>
Committed: Thu Feb 9 20:08:57 2017 +0300
----------------------------------------------------------------------
modules/yardstick/DEVNOTES-standalone.txt | 15 +++
modules/yardstick/DEVNOTES.txt | 20 ++--
modules/yardstick/README.txt | 119 +++++++++++++------
.../config/benchmark-sample.properties | 62 ++++++++++
modules/yardstick/pom.xml | 4 +-
pom.xml | 1 +
6 files changed, 172 insertions(+), 49 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/ignite/blob/605d946a/modules/yardstick/DEVNOTES-standalone.txt
----------------------------------------------------------------------
diff --git a/modules/yardstick/DEVNOTES-standalone.txt b/modules/yardstick/DEVNOTES-standalone.txt
new file mode 100644
index 0000000..d14dde2
--- /dev/null
+++ b/modules/yardstick/DEVNOTES-standalone.txt
@@ -0,0 +1,15 @@
+Building from standalone sources
+================================
+
+Run
+
+mvn clean package
+
+Artifacts can be found in `/target/assembly directory.`
+
+
+Writing Ignite Benchmarks
+=========================
+
+All benchmarks extend `AbstractBenchmark` class. A new benchmark should also extend this abstract class
+and implement `test` method. This is the method which actually tests performance.
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/ignite/blob/605d946a/modules/yardstick/DEVNOTES.txt
----------------------------------------------------------------------
diff --git a/modules/yardstick/DEVNOTES.txt b/modules/yardstick/DEVNOTES.txt
index 6921243..b8b1f53 100644
--- a/modules/yardstick/DEVNOTES.txt
+++ b/modules/yardstick/DEVNOTES.txt
@@ -1,16 +1,20 @@
-Yardstick Ignite Maven Build Instructions
+Building from Ignite sources
=========================================
-Yardstick can be build from standalone sources using following maven command:
+Run
-mvn clean package
+mvn clean package -Pyardstick -pl modules/yardstick -am -DskipTests
-Artifacts can be found in /target/assembly directory.
+in the Apache Ignite root directory.
-To build yardstick from Apache Ignite sources use:
+This command will compile the project and also unpack scripts from `yardstick-resources.zip` file to
+`modules/yardstick/target/assembly/bin` directory.
-mvn clean package -Pyardstick -pl modules/yardstick -am -DskipTests
+Artifacts can be found in `modules/yardstick/target/assembly` directory.
+
+Writing Ignite Benchmarks
+=========================
-in Apache Ignite root directory
+All benchmarks extend `AbstractBenchmark` class. A new benchmark should also extend this abstract class
+and implement `test` method. This is the method which actually tests performance.
-Artifacts can be found in modules/yardstick/target/assembly directory.
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/ignite/blob/605d946a/modules/yardstick/README.txt
----------------------------------------------------------------------
diff --git a/modules/yardstick/README.txt b/modules/yardstick/README.txt
index 353ddd8..dcfe0e6 100644
--- a/modules/yardstick/README.txt
+++ b/modules/yardstick/README.txt
@@ -1,55 +1,92 @@
Yardstick Ignite Benchmarks
===========================
-Yardstick Ignite is a set of Ignite Grid (http://ignite.apache.org/) benchmarks written on top of Yardstick framework.
+Apache Ignite benchmarks are written on top of Yardstick Framework (https://github.com/gridgain/yardstick) and allow
+you to measure performance of various Apache Ignite components and modules.
-Yardstick Framework
-===================
-Visit Yardstick Repository (https://github.com/gridgain/yardstick) for detailed information on how to run Yardstick benchmarks and how to generate graphs.
+The documentation below describes how to execute and configure already assembled benchmarks. If you need to add new
+benchmarks or build existing one then refer to instruction from `DEVNOTES.txt` in source directory.
-The documentation below describes configuration parameters in addition to standard Yardstick parameters.
+Visit Yardstick Repository (https://github.com/gridgain/yardstick) for detailed information in regards resulting graphs
+generation and how the frameworks works.
-Building from Ignite sources
-============
-1. Create a local clone of Ignite repository
-2. Run
-mvn package
+Running Ignite Benchmarks Locally
+==========================================
-command for Yardstick Ignite POM
+The simplest way to start with benchmarking is to use one of the executable scripts available under `benchmarks\bin` directory:
-Building from standalone sources
-=====================
-Run
+./bin/benchmark-run-all.sh config/benchmark-atomic-put.properties
-mvn package
+The command above will benchmark the cache put operation for a distributed atomic cache. The results of the benchmark will be added
+to auto-generated `results-{DATE-TIME}` directory.
-command for Yardstick Ignite POM
+If `./bin/benchmark-run-all.sh` command is executed as is without any parameters and modifications in configurations files then
+all the available benchmarks will be executed on a local machine using `config/benchmark.properties` configuration.
-Provided Benchmarks
-===================
-The following benchmarks are provided:
+To get more information about available benchmarks and configuration parameters refer to \u201cProvided Benchmarks\u201d and
+\u201cProperties And Command Line Arguments\u201d sections below.
+
+
+Running Ignite Benchmarks Remotely
+=========================================
+
+For running Ignite benchmarks on remote hosts you need to upload Ignite-Yardstick to each one of your remote hosts.
+
+NOTE: The path to the uploaded Ignite-Yardstick should be exactly the same on each host.
+
+Then you need to make some changes in config files:
-1. `GetBenchmark` - benchmarks atomic distributed cache get operation
-2. `PutBenchmark` - benchmarks atomic distributed cache put operation
-3. `PutGetBenchmark` - benchmarks atomic distributed cache put and get operations together
-4. `PutTxBenchmark` - benchmarks transactional distributed cache put operation
-5. `PutGetTxBenchmark` - benchmarks transactional distributed cache put and get operations together
-6. `SqlQueryBenchmark` - benchmarks distributed SQL query over cached data
-7. `SqlQueryJoinBenchmark` - benchmarks distributed SQL query with a Join over cached data
-8. `SqlQueryPutBenchmark` - benchmarks distributed SQL query with simultaneous cache updates
+1. You need to comment or delete the
+ <property name="localHost" value="127.0.0.1"/>
+line in `config/ignite-localhost-config.xml` file.
-Writing Ignite Benchmarks
-=========================
-All benchmarks extend `AbstractBenchmark` class. A new benchmark should also extend this abstract class
-and implement `test` method. This is the method that is actually benchmarked.
+2. You need to replace all the `127.0.0.1` addresses in `ignite-localhost-config.xml` file by actual IPs of your remote
+servers. You can add or delete lines with IP addresses if you want to run benchmarks on different number of hosts.
+There must be at least one IP address in the list.
-Running Ignite Benchmarks
-=========================
-Before running Ignite benchmarks, run:
+3. You need to replace the `localhost` strings by actual IP of your servers in the lines
+SERVERS='localhost,localhost'
+DRIVERS='localhost'
+in `config/benchmark-atomic-put.properties` file.
+
+Then use the following command:
+
+./bin/benchmark-run-all.sh config/benchmark-atomic-put.properties
+
+
+It is recommended to create some copies of original config files, edit these copies and then use as a
+parameter for `./bin/benchmark-run-all.sh` script.
+
+
+Provided Benchmarks
+===================
+The following benchmarks are provided:
-mvn package
+1. `GetBenchmark` - benchmarks atomic distributed cache get operation
+2. `PutBenchmark` - benchmarks atomic distributed cache put operation
+3. `PutGetBenchmark` - benchmarks atomic distributed cache put and get operations together
+4. `PutTxBenchmark` - benchmarks transactional distributed cache put operation
+5. `PutGetTxBenchmark` - benchmarks transactional distributed cache put and get operations together
+6. `SqlQueryBenchmark` - benchmarks distributed SQL query over cached data
+7. `SqlQueryJoinBenchmark` - benchmarks distributed SQL query with a Join over cached data
+8. `SqlQueryPutBenchmark` - benchmarks distributed SQL query with simultaneous cache updates
+9. `AffinityCallBenchmark` - benchmarks affinity call operation
+10. `ApplyBenchmark` - benchmarks apply operation
+11. `BroadcastBenchmark` - benchmarks broadcast operations
+12. `ExecuteBenchmark` - benchmarks execute operations
+13. `RunBenchmark` - benchmarks running task operations
+14. `PutGetOffHeapBenchmark` - benchmarks atomic distributed cache put and get operations together off heap
+15. `PutGetOffHeapValuesBenchmark` - benchmarks atomic distributed cache put value operations off heap
+16. `PutOffHeapBenchmark` - benchmarks atomic distributed cache put operations off heap
+17. `PutOffHeapValuesBenchmark` - benchmarks atomic distributed cache get value operations off heap
+18. `PutTxOffHeapBenchmark` - benchmarks transactional distributed cache put operation off heap
+19. `PutTxOffHeapValuesBenchmark` - benchmarks transactional distributed cache put value operation off heap
+20. `SqlQueryOffHeapBenchmark` -benchmarks distributed SQL query over cached data off heap
+21. `SqlQueryJoinOffHeapBenchmark` - benchmarks distributed SQL query with a Join over cached data off heap
+22. `SqlQueryPutOffHeapBenchmark` - benchmarks distributed SQL query with simultaneous cache updates off heap
+23. `PutAllBenchmark` - benchmarks atomic distributed cache batch put operation
+24. `PutAllTxBenchmark` - benchmarks transactional distributed cache batch put operation
-command. This command will compile the project and also will unpack scripts from `yardstick-resources.zip` file to `bin` directory.
Properties And Command Line Arguments
=====================================
@@ -67,15 +104,19 @@ The following Ignite benchmark properties can be defined in the benchmark config
* `-cs` or `--cacheStore` - Enable or disable cache store readThrough, writeThrough
* `-cl` or `--client` - Client flag
* `-nc` or `--nearCache` - Near cache flag
-* `-nn <num>` or `--nodeNumber <num>` - Number of nodes (automatically set in `benchmark.properties`), used to wait for the specified number of nodes to start
+* `-nn <num>` or `--nodeNumber <num>` - Number of nodes (automatically set in `benchmark.properties`), used to wait for
+ the specified number of nodes to start
* `-sm <mode>` or `-syncMode <mode>` - Synchronization mode (defined in `CacheWriteSynchronizationMode`)
* `-ot` or `--offheapTiered` - Flag indicating whether tiered off-heap mode is on
* `-ov` or `--offheapValuesOnly` - Flag indicating whether off-heap mode is on and only cache values are stored off-heap
* `-r <num>` or `--range` - Range of keys that are randomly generated for cache operations
+* `-rd or --restartdelay` - Restart delay in seconds
+* `-rs or --restartsleep` - Restart sleep in seconds
* `-rth <host>` or `--restHost <host>` - REST TCP host
* `-rtp <num>` or `--restPort <num>` - REST TCP port, indicates that a Ignite node is ready to process Ignite Clients
* `-ss` or `--syncSend` - Flag indicating whether synchronous send is used in `TcpCommunicationSpi`
-* `-txc <value>` or `--txConcurrency <value>` - Cache transaction concurrency control, either `OPTIMISTIC` or `PESSIMISTIC` (defined in `CacheTxConcurrency`)
+* `-txc <value>` or `--txConcurrency <value>` - Cache transaction concurrency control, either `OPTIMISTIC` or
+ `PESSIMISTIC` (defined in `CacheTxConcurrency`)
* `-txi <value>` or `--txIsolation <value>` - Cache transaction isolation (defined in `CacheTxIsolation`)
* `-wb` or `--writeBehind` - Enable or disable writeBehind for cache store
* `-wom <mode>` or `--writeOrderMode <mode>` - Write order mode for ATOMIC caches (defined in `CacheAtomicWriteOrderMode`)
@@ -91,7 +132,7 @@ SERVER_HOSTS=localhost,localhost
Note that -dn and -sn, which stand for data node and server node, are native Yardstick parameters and are documented in Yardstick framework.
===========================================================================================================================================
-CONFIGS="-b 1 -sm PRIMARY_SYNC -dn PutBenchmark -sn IgniteNode"
+CONFIGS="-b 1 -sm PRIMARY_SYNC -dn PutBenchmark`IgniteNode"
```
Issues
http://git-wip-us.apache.org/repos/asf/ignite/blob/605d946a/modules/yardstick/config/benchmark-sample.properties
----------------------------------------------------------------------
diff --git a/modules/yardstick/config/benchmark-sample.properties b/modules/yardstick/config/benchmark-sample.properties
new file mode 100644
index 0000000..fee7cbb
--- /dev/null
+++ b/modules/yardstick/config/benchmark-sample.properties
@@ -0,0 +1,62 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# Contains benchmarks for ATOMIC cache.
+#
+
+# JVM options.
+JVM_OPTS=${JVM_OPTS}" -DIGNITE_QUIET=false"
+
+# Uncomment to enable concurrent garbage collection (GC) if you encounter long GC pauses.
+# JVM_OPTS=${JVM_OPTS}" \
+# -XX:+UseParNewGC \
+# -XX:+UseConcMarkSweepGC \
+# -XX:+UseTLAB \
+# -XX:NewSize=128m \
+# -XX:MaxNewSize=128m \
+# -XX:MaxTenuringThreshold=0 \
+# -XX:SurvivorRatio=1024 \
+# -XX:+UseCMSInitiatingOccupancyOnly \
+# -XX:CMSInitiatingOccupancyFraction=60 \
+#"
+
+# List of default probes.
+# Add DStatProbe or VmStatProbe if your OS supports it (e.g. if running on Linux).
+BENCHMARK_DEFAULT_PROBES=ThroughputLatencyProbe,PercentileProbe
+
+# Packages where the specified benchmark is searched by reflection mechanism.
+BENCHMARK_PACKAGES=org.yardstickframework,org.apache.ignite.yardstick
+
+# Probe point writer class name.
+# BENCHMARK_WRITER=
+
+# Comma-separated list of the hosts to run BenchmarkServers on. 2 nodes on local host are enabled by default.
+SERVER_HOSTS=localhost,localhost
+
+# Comma-separated list of the hosts to run BenchmarkDrivers on. 1 node on local host is enabled by default.
+DRIVER_HOSTS=localhost
+
+# Remote username.
+# REMOTE_USER=
+
+# Number of nodes, used to wait for the specified number of nodes to start.
+nodesNum=$((`echo ${SERVER_HOSTS} | tr ',' '\n' | wc -l` + `echo ${DRIVER_HOSTS} | tr ',' '\n' | wc -l`))
+
+# Run configuration.
+# Note that each benchmark is set to run for 300 seconds (5 mins) with warm-up set to 60 seconds (1 minute).
+CONFIGS="\
+-cfg ${SCRIPT_DIR}/../config/ignite-localhost-config.xml -nn ${nodesNum} -b 1 -w 60 -d 300 -t 64 -sm PRIMARY_SYNC -dn IgnitePutBenchmark -sn IgniteNode -ds atomic-put-1-backup,\
+"
http://git-wip-us.apache.org/repos/asf/ignite/blob/605d946a/modules/yardstick/pom.xml
----------------------------------------------------------------------
diff --git a/modules/yardstick/pom.xml b/modules/yardstick/pom.xml
index 4c4d138..d96fcc6 100644
--- a/modules/yardstick/pom.xml
+++ b/modules/yardstick/pom.xml
@@ -35,7 +35,7 @@
<url>http://ignite.apache.org</url>
<properties>
- <yardstick.version>0.8.0</yardstick.version>
+ <yardstick.version>0.8.2</yardstick.version>
<spring.version>4.1.0.RELEASE</spring.version>
</properties>
@@ -56,7 +56,7 @@
<groupId>org.apache.ignite</groupId>
<artifactId>ignite-indexing</artifactId>
<version>${project.version}</version>
- </dependency>
+ </dependency>
<dependency>
<groupId>org.apache.ignite</groupId>
http://git-wip-us.apache.org/repos/asf/ignite/blob/605d946a/pom.xml
----------------------------------------------------------------------
diff --git a/pom.xml b/pom.xml
index 790cd4b..ea76053 100644
--- a/pom.xml
+++ b/pom.xml
@@ -506,6 +506,7 @@
<fileset dir="${basedir}/target/release-package/benchmarks/config/">
<include name="*.*" />
<exclude name="benchmark.properties"/>
+ <exclude name="benchmark-sample.properties"/>
<exclude name="benchmark-multicast.properties"/>
<exclude name="ignite-base-config.xml"/>
<exclude name="ignite-localhost-config.xml"/>