You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tinkerpop.apache.org by pi...@apache.org on 2016/03/30 08:39:17 UTC
[23/40] incubator-tinkerpop git commit: This commit includes the
following: * added new gremlin-benchmark module and initial benchmark classes
* deprecated the JUnit benchmark classes,
noted this deprecation in upgrade doc * expanded scope of LoadGraphWi
This commit includes the following:
* added new gremlin-benchmark module and initial benchmark classes
* deprecated the JUnit benchmark classes, noted this deprecation in upgrade doc
* expanded scope of LoadGraphWith annotation to allow for class level annotation
* added developer benchmarking documentation
Project: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/commit/682d313e
Tree: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/tree/682d313e
Diff: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/diff/682d313e
Branch: refs/heads/TINKERPOP-968
Commit: 682d313e8632c10470f0376f38c169acb6e047c3
Parents: d5a7856
Author: Ted Wilmes <tw...@gmail.com>
Authored: Mon Mar 28 21:12:57 2016 -0500
Committer: Ted Wilmes <tw...@gmail.com>
Committed: Mon Mar 28 21:12:57 2016 -0500
----------------------------------------------------------------------
.gitignore | 1 +
CHANGELOG.asciidoc | 1 +
docs/src/dev/developer/contributing.asciidoc | 31 +++++
.../upgrade/release-3.2.x-incubating.asciidoc | 5 +
gremlin-benchmark/pom.xml | 130 +++++++++++++++++++
.../benchmark/util/AbstractBenchmarkBase.java | 117 +++++++++++++++++
.../benchmark/util/AbstractGraphBenchmark.java | 85 ++++++++++++
.../util/AbstractGraphMutateBenchmark.java | 48 +++++++
.../gremlin/process/GraphMutateBenchmark.java | 92 +++++++++++++
.../process/GraphTraversalBenchmark.java | 101 ++++++++++++++
.../apache/tinkerpop/gremlin/LoadGraphWith.java | 2 +-
.../process/ProcessPerformanceSuite.java | 3 +
.../process/TraversalPerformanceTest.java | 3 +
pom.xml | 8 +-
14 files changed, 625 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/682d313e/.gitignore
----------------------------------------------------------------------
diff --git a/.gitignore b/.gitignore
index f4779c5..11ca63e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -14,3 +14,4 @@ _bsp/
doc/out
docs/*.asciidoc
ext/
+benchmarks/
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/682d313e/CHANGELOG.asciidoc
----------------------------------------------------------------------
diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc
index ea0d52b..87a7ba6 100644
--- a/CHANGELOG.asciidoc
+++ b/CHANGELOG.asciidoc
@@ -26,6 +26,7 @@ image::https://raw.githubusercontent.com/apache/incubator-tinkerpop/master/docs/
TinkerPop 3.2.0 (NOT OFFICIALLY RELEASED YET)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+* Added `gremlin-benchmark` module with JMH benchmarking base classes that can be used for further benchmark development.
* `TraversalStrategies.GlobalCache` supports both `Graph` and `GraphComputer` strategy registrations.
* `select("a","b").by("name").by("age")`-style traversals now work in OLAP with new `PathProcessorStrategy`.
* `DedupGlobalStep` can now handle star-bound `by()`-modulators and scoped keys on `GraphComputer`.
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/682d313e/docs/src/dev/developer/contributing.asciidoc
----------------------------------------------------------------------
diff --git a/docs/src/dev/developer/contributing.asciidoc b/docs/src/dev/developer/contributing.asciidoc
index 4a76184..86a90ba 100644
--- a/docs/src/dev/developer/contributing.asciidoc
+++ b/docs/src/dev/developer/contributing.asciidoc
@@ -89,6 +89,7 @@ mvn -Dmaven.javadoc.skip=true --projects tinkergraph-gremlin test
** Execute with the `-DincludeNeo4j` option to include transactional tests.
** Execute with the `-DuseEpoll` option to try to use Netty native transport (works on Linux, but will fallback to Java NIO on other OS).
* Performance Tests: `mvn verify -DskipPerformanceTests=false`
+* Benchmarks: `mvn verify -DskipBenchmarks=false`
IDE Setup with Intellij
-----------------------
@@ -301,6 +302,36 @@ When writing a test case for a Gremlin step, be sure to use the following conven
** `checkResults(Arrays.asList("marko","josh"), traversal)`
** `checkMap(new HashMap<String,Long>() {{ put("marko",1l); }}, traversal.next())`
+Developing Benchmarks
+~~~~~~~~~~~~~~~~~~~~~
+
+Benchmarks are a useful tool to track performance between TinkerPop versions and also as tools to aid development
+decision making. TinkerPop uses link:http://openjdk.java.net/projects/code-tools/jmh/[OpenJDK JMH] for benchmark development.
+The JMH framework provides tools for writing robust benchmarking code that avoid many of the pitfalls inherent in benchmarking
+JIT compiled code on the JVM. Example JMH benchmarks can be found
+link:http://hg.openjdk.java.net/code-tools/jmh/file/tip/jmh-samples/src/main/java/org/openjdk/jmh/samples/[here].
+
+TinkerPop benchmarks live in the `gremlin-benchmark` module and can either be run from within your IDE or as a standalone
+uber-jar. The uber-jar is the JMH recommended approach and also makes it easy to distribute artifacts to various environments
+to gather benchmarking numbers. Having said that, in most cases it should be sufficient to run it from within the IDE.
+
+Benchmarks will not run by default because they are time consuming. To enable benchmarks during the test phase do
+`-DskipBenchmarks=false`. To change the number of warmup iterations, measurement iterations, and forks you can do
+`mvn clean test -DskipBenchmarks=false -DdefaultForks=5 -DmeasureIterations=20 -DwarmupIterations=20`. Benchmark results
+will be output by default to the `benchmarks` directory in JSON format.
+
+Benchmarks may also be run from the command line using the JMH runner. Build the uber-jar and simply run
+`java -jar gremlin-benchmark-TP-VERSION.jar`. To see a list of JMH runner options, add the `-h` flag.
+
+The JUnit/JMH integration was inspired by the Netty projects microbenchmarking suite. Please refer to the Netty
+link:http://netty.io/wiki/microbenchmarks.html[docs] for more details. Presently there are 3 abstract benchmark classes
+that may be used as building blocks for your benchmarks; `AbstractBenchmarkBase`, `AbstractGraphBenchmark`, and
+`AbstractGraphMutateBenchmark`.
+
+* `AbstractBenchmarkBase` - extend when your benchmark does not require a graph instance
+* `AbstractGraphBenchmark` - extend when you are benchmarking read operations against a graph
+* `AbstractGraphMutateBenchmark` - extend when you are benchmarking graph mutation operations eg. `g.addV()`, `graph.addVertex()`
+
[[rtc]]
Review then Commit
~~~~~~~~~~~~~~~~~~
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/682d313e/docs/src/upgrade/release-3.2.x-incubating.asciidoc
----------------------------------------------------------------------
diff --git a/docs/src/upgrade/release-3.2.x-incubating.asciidoc b/docs/src/upgrade/release-3.2.x-incubating.asciidoc
index d28669e..8d53287 100644
--- a/docs/src/upgrade/release-3.2.x-incubating.asciidoc
+++ b/docs/src/upgrade/release-3.2.x-incubating.asciidoc
@@ -322,6 +322,11 @@ no longer exists and such post-reduction processing is handled by the reducing s
See: link:https://issues.apache.org/jira/browse/TINKERPOP-1164[TINKERPOP-1164]
+Performance Tests
++++++++++++++++++
+The `ProcessPerformanceSuite` and `TraversalPerformanceTest` have been deprecated. They are still available, but going forward,
+providers should implement their own performance tests and not rely on the built-in JUnit benchmark-based performance test suit.
+
Graph Processor Providers
^^^^^^^^^^^^^^^^^^^^^^^^^
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/682d313e/gremlin-benchmark/pom.xml
----------------------------------------------------------------------
diff --git a/gremlin-benchmark/pom.xml b/gremlin-benchmark/pom.xml
new file mode 100644
index 0000000..092e6d5
--- /dev/null
+++ b/gremlin-benchmark/pom.xml
@@ -0,0 +1,130 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <artifactId>tinkerpop</artifactId>
+ <groupId>org.apache.tinkerpop</groupId>
+ <version>3.2.0-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>gremlin-benchmark</artifactId>
+ <name>Apache TinkerPop :: Gremlin Benchmark</name>
+ <properties>
+ <jmh.version>1.11.3</jmh.version>
+ <!-- Skip benchmarks by default because they are time consuming. -->
+ <skipBenchmarks>true</skipBenchmarks>
+ <skipTests>${skipBenchmarks}</skipTests>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>${junit.version}</version>
+ <scope>compile</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.tinkerpop</groupId>
+ <artifactId>gremlin-core</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.tinkerpop</groupId>
+ <artifactId>gremlin-test</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.tinkerpop</groupId>
+ <artifactId>tinkergraph-gremlin</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.openjdk.jmh</groupId>
+ <artifactId>jmh-core</artifactId>
+ <version>${jmh.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.openjdk.jmh</groupId>
+ <artifactId>jmh-generator-annprocess</artifactId>
+ <version>${jmh.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ </dependencies>
+ <build>
+ <plugins>
+ <plugin>
+ <artifactId>maven-deploy-plugin</artifactId>
+ <configuration>
+ <skip>true</skip>
+ </configuration>
+ </plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-surefire-plugin</artifactId>
+ <configuration>
+ <testSourceDirectory>${project.build.sourceDirectory}</testSourceDirectory>
+ <testClassesDirectory>${project.build.outputDirectory}</testClassesDirectory>
+ <includes>
+ <include>**/*Benchmark*.java</include>
+ </includes>
+ <excludes>
+ <exclude>**/*$*.class</exclude>
+ <exclude>**/Abstract*</exclude>
+ <exclude>**/*_jmhType*</exclude>
+ </excludes>
+ <systemPropertyVariables>
+ <benchmarkReportDir>${project.build.directory}/reports/benchmark/</benchmarkReportDir>
+ </systemPropertyVariables>
+ </configuration>
+ </plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-shade-plugin</artifactId>
+ <version>2.2</version>
+ <executions>
+ <execution>
+ <phase>package</phase>
+ <goals>
+ <goal>shade</goal>
+ </goals>
+ <configuration>
+ <createDependencyReducedPom>false</createDependencyReducedPom>
+ <transformers>
+ <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
+ <mainClass>org.openjdk.jmh.Main</mainClass>
+ </transformer>
+ </transformers>
+ <filters>
+ <filter>
+ <artifact>*:*</artifact>
+ <excludes>
+ <exclude>META-INF/*.SF</exclude>
+ <exclude>META-INF/*.DSA</exclude>
+ <exclude>META-INF/*.RSA</exclude>
+ </excludes>
+ </filter>
+ </filters>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+</project>
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/682d313e/gremlin-benchmark/src/main/java/org/apache/tinkerpop/benchmark/util/AbstractBenchmarkBase.java
----------------------------------------------------------------------
diff --git a/gremlin-benchmark/src/main/java/org/apache/tinkerpop/benchmark/util/AbstractBenchmarkBase.java b/gremlin-benchmark/src/main/java/org/apache/tinkerpop/benchmark/util/AbstractBenchmarkBase.java
new file mode 100644
index 0000000..5b73045
--- /dev/null
+++ b/gremlin-benchmark/src/main/java/org/apache/tinkerpop/benchmark/util/AbstractBenchmarkBase.java
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.benchmark.util;
+
+import org.junit.Test;
+import org.openjdk.jmh.annotations.Fork;
+import org.openjdk.jmh.annotations.Measurement;
+import org.openjdk.jmh.annotations.Warmup;
+import org.openjdk.jmh.results.format.ResultFormatType;
+import org.openjdk.jmh.runner.Runner;
+import org.openjdk.jmh.runner.options.ChainedOptionsBuilder;
+import org.openjdk.jmh.runner.options.OptionsBuilder;
+
+import java.io.File;
+import java.text.SimpleDateFormat;
+import java.util.Date;
+
+/**
+ * Base class for all TinkerPop OpenJDK JMH benchmarks. Based upon Netty's approach to running JMH benchmarks
+ * from JUnit.
+ *
+ * @see <a href="http://netty.io/wiki/microbenchmarks.html"</a>
+ *
+ * @author Ted Wilmes (http://twilmes.org)
+ */
+@Warmup(iterations = AbstractBenchmarkBase.DEFAULT_WARMUP_ITERATIONS)
+@Measurement(iterations = AbstractBenchmarkBase.DEFAULT_MEASURE_ITERATIONS)
+@Fork(AbstractBenchmarkBase.DEFAULT_FORKS)
+public abstract class AbstractBenchmarkBase {
+
+ protected static final int DEFAULT_WARMUP_ITERATIONS = 10;
+ protected static final int DEFAULT_MEASURE_ITERATIONS = 10;
+ protected static final int DEFAULT_FORKS = 2;
+ protected static final String DEFAULT_BENCHMARK_DIRECTORY = "./benchmarks/";
+
+ protected static final String[] JVM_ARGS = {
+ "-server", "-Xms1g", "-Xmx1g"
+ };
+
+ @Test
+ public void run() throws Exception {
+ final String className = getClass().getSimpleName();
+
+ final ChainedOptionsBuilder runnerOptions = new OptionsBuilder()
+ .include(".*" + className + ".*")
+ .jvmArgs(JVM_ARGS);
+
+ if (getWarmupIterations() > 0) {
+ runnerOptions.warmupIterations(getWarmupIterations());
+ }
+
+ if (getMeasureIterations() > 0) {
+ runnerOptions.measurementIterations(getMeasureIterations());
+ }
+
+ if (getForks() > 0) {
+ runnerOptions.forks(getForks());
+ }
+
+ if (getReportDir() != null) {
+ final String dtmStr = new SimpleDateFormat("yyyyMMddHHmmss").format(new Date());
+ final String filePath = getReportDir() + className + "-" + dtmStr + ".json";
+ final File file = new File(filePath);
+ if (file.exists()) {
+ file.delete();
+ } else {
+ file.getParentFile().mkdirs();
+ file.createNewFile();
+ }
+
+ runnerOptions.resultFormat(ResultFormatType.JSON);
+ runnerOptions.result(filePath);
+ }
+
+ new Runner(runnerOptions.build()).run();
+ }
+
+ protected int getWarmupIterations() {
+ return getIntProperty("warmupIterations", DEFAULT_WARMUP_ITERATIONS);
+ }
+
+ protected int getMeasureIterations() {
+ return getIntProperty("measureIterations", DEFAULT_MEASURE_ITERATIONS);
+ }
+
+ protected int getForks() {
+ return getIntProperty("forks", DEFAULT_FORKS);
+ }
+
+ protected String getReportDir() {
+ return System.getProperty("benchmark.dir", DEFAULT_BENCHMARK_DIRECTORY);
+ }
+
+ private int getIntProperty(final String propertyName, final int defaultValue) {
+ final String propertyValue = System.getProperty(propertyName);
+ if(propertyValue == null) {
+ return defaultValue;
+ }
+ return Integer.valueOf(propertyValue);
+ }
+}
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/682d313e/gremlin-benchmark/src/main/java/org/apache/tinkerpop/benchmark/util/AbstractGraphBenchmark.java
----------------------------------------------------------------------
diff --git a/gremlin-benchmark/src/main/java/org/apache/tinkerpop/benchmark/util/AbstractGraphBenchmark.java b/gremlin-benchmark/src/main/java/org/apache/tinkerpop/benchmark/util/AbstractGraphBenchmark.java
new file mode 100644
index 0000000..ba0105c
--- /dev/null
+++ b/gremlin-benchmark/src/main/java/org/apache/tinkerpop/benchmark/util/AbstractGraphBenchmark.java
@@ -0,0 +1,85 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.benchmark.util;
+
+import org.apache.tinkerpop.gremlin.LoadGraphWith;
+import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource;
+import org.apache.tinkerpop.gremlin.structure.Graph;
+import org.apache.tinkerpop.gremlin.structure.io.GraphReader;
+import org.apache.tinkerpop.gremlin.structure.io.gryo.GryoReader;
+import org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerGraph;
+
+import org.openjdk.jmh.annotations.Scope;
+import org.openjdk.jmh.annotations.Setup;
+import org.openjdk.jmh.annotations.State;
+
+import java.io.IOException;
+import java.io.InputStream;
+
+/**
+ * Read-only graph benchmarks extend {@code AbstractGraphBenchmark}. Annotating your benchmark class with {@link LoadGraphWith}
+ * will load the {@link TinkerGraph} instance with the desired data set.
+ *
+ * @author Ted Wilmes (http://twilmes.org)
+ */
+@State(Scope.Thread)
+public class AbstractGraphBenchmark extends AbstractBenchmarkBase {
+
+ private final String PATH = "/org/apache/tinkerpop/gremlin/structure/io/gryo/";
+
+ protected Graph graph;
+ protected GraphTraversalSource g;
+
+ /**
+ * Opens a new {@link TinkerGraph} instance and optionally preloads it with one of the test data sets enumerated
+ * in {@link LoadGraphWith}.
+ *
+ * @throws IOException on failure to load graph
+ */
+ @Setup
+ public void prepare() throws IOException {
+ graph = TinkerGraph.open();
+ g = graph.traversal();
+
+ final LoadGraphWith[] loadGraphWiths = this.getClass().getAnnotationsByType(LoadGraphWith.class);
+ final LoadGraphWith loadGraphWith = loadGraphWiths.length == 0 ? null : loadGraphWiths[0];
+ final LoadGraphWith.GraphData loadGraphWithData = null == loadGraphWith ? null : loadGraphWith.value();
+
+ String graphFile;
+ if(loadGraphWithData != null) {
+ if (loadGraphWithData.equals(LoadGraphWith.GraphData.GRATEFUL)) {
+ graphFile = "grateful-dead.kryo";
+ } else if (loadGraphWithData.equals(LoadGraphWith.GraphData.MODERN)) {
+ graphFile = "tinkerpop-modern.kryo";
+ } else if (loadGraphWithData.equals(LoadGraphWith.GraphData.CLASSIC)) {
+ graphFile = "tinkerpop-classic.kryo";
+ } else if (loadGraphWithData.equals(LoadGraphWith.GraphData.CREW)) {
+ graphFile = "tinkerpop-crew.kryo";
+ } else {
+ throw new RuntimeException("Could not load graph with " + loadGraphWithData);
+ }
+
+ final GraphReader reader = GryoReader.build().create();
+ try (final InputStream stream = AbstractGraphBenchmark.class.
+ getResourceAsStream(PATH + graphFile)) {
+ reader.readGraph(stream, graph);
+ }
+ }
+ }
+}
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/682d313e/gremlin-benchmark/src/main/java/org/apache/tinkerpop/benchmark/util/AbstractGraphMutateBenchmark.java
----------------------------------------------------------------------
diff --git a/gremlin-benchmark/src/main/java/org/apache/tinkerpop/benchmark/util/AbstractGraphMutateBenchmark.java b/gremlin-benchmark/src/main/java/org/apache/tinkerpop/benchmark/util/AbstractGraphMutateBenchmark.java
new file mode 100644
index 0000000..79d9916
--- /dev/null
+++ b/gremlin-benchmark/src/main/java/org/apache/tinkerpop/benchmark/util/AbstractGraphMutateBenchmark.java
@@ -0,0 +1,48 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.benchmark.util;
+
+import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource;
+import org.apache.tinkerpop.gremlin.structure.Graph;
+import org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerGraph;
+
+import org.openjdk.jmh.annotations.Level;
+import org.openjdk.jmh.annotations.Scope;
+import org.openjdk.jmh.annotations.Setup;
+import org.openjdk.jmh.annotations.State;
+
+/**
+ * Graph write and update benchmarks extend {@code AbstractGraphMutateBenchmark}. {@code AbstractGraphMutateBenchmark}
+ * runs setup once per invocation so that benchmark measurements are made on an empty {@link TinkerGraph}. This approach
+ * was taken to isolate the tested method from the performance side effects of unbounded graph growth.
+ *
+ * @author Ted Wilmes (http://twilmes.org)
+ */
+@State(Scope.Thread)
+public abstract class AbstractGraphMutateBenchmark extends AbstractBenchmarkBase {
+
+ protected Graph graph;
+ protected GraphTraversalSource g;
+
+ @Setup(Level.Invocation)
+ public void prepare() {
+ graph = TinkerGraph.open();
+ g = graph.traversal();
+ }
+}
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/682d313e/gremlin-benchmark/src/main/java/org/apache/tinkerpop/gremlin/process/GraphMutateBenchmark.java
----------------------------------------------------------------------
diff --git a/gremlin-benchmark/src/main/java/org/apache/tinkerpop/gremlin/process/GraphMutateBenchmark.java b/gremlin-benchmark/src/main/java/org/apache/tinkerpop/gremlin/process/GraphMutateBenchmark.java
new file mode 100644
index 0000000..04d7138
--- /dev/null
+++ b/gremlin-benchmark/src/main/java/org/apache/tinkerpop/gremlin/process/GraphMutateBenchmark.java
@@ -0,0 +1,92 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.gremlin.process;
+
+import org.apache.tinkerpop.benchmark.util.AbstractGraphMutateBenchmark;
+import org.apache.tinkerpop.gremlin.structure.Edge;
+import org.apache.tinkerpop.gremlin.structure.Property;
+import org.apache.tinkerpop.gremlin.structure.Vertex;
+import org.apache.tinkerpop.gremlin.structure.VertexProperty;
+
+import org.openjdk.jmh.annotations.Benchmark;
+import org.openjdk.jmh.annotations.Setup;
+
+/**
+ * {@code GraphMutateBenchmark} benchmarks {@link org.apache.tinkerpop.gremlin.process.traversal.Traversal} and
+ * {@link org.apache.tinkerpop.gremlin.structure.Graph} mutation methods.
+ *
+ * @author Ted Wilmes (http://twilmes.org)
+ */
+public class GraphMutateBenchmark extends AbstractGraphMutateBenchmark {
+
+ private Vertex a;
+ private Vertex b;
+ private Vertex c;
+ private Edge e;
+
+ @Setup
+ @Override
+ public void prepare() {
+ super.prepare();
+ a = g.addV().next();
+ b = g.addV().next();
+ c = g.addV().next();
+ e = b.addEdge("knows", c);
+ }
+
+ @Benchmark
+ public Vertex testAddVertex() {
+ return graph.addVertex("test");
+ }
+
+ @Benchmark
+ public VertexProperty testVertexProperty() {
+ return a.property("name", "Susan");
+ }
+
+ @Benchmark
+ public Edge testAddEdge() {
+ return a.addEdge("knows", b);
+ }
+
+ @Benchmark
+ public Property testEdgeProperty() {
+ return e.property("met", 1967);
+ }
+
+ @Benchmark
+ public Vertex testAddV() {
+ return g.addV("test").next();
+ }
+
+ @Benchmark
+ public Vertex testVertexPropertyStep() {
+ return g.V(a).property("name", "Susan").next();
+ }
+
+ @Benchmark
+ public Edge testAddE() {
+ return g.V(a).as("a").V(b).as("b").addE("knows").from("a").to("b").next();
+ }
+
+ @Benchmark
+ public Edge testEdgePropertyStep() {
+ return g.E(e).property("met", 1967).next();
+ }
+}
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/682d313e/gremlin-benchmark/src/main/java/org/apache/tinkerpop/gremlin/process/GraphTraversalBenchmark.java
----------------------------------------------------------------------
diff --git a/gremlin-benchmark/src/main/java/org/apache/tinkerpop/gremlin/process/GraphTraversalBenchmark.java b/gremlin-benchmark/src/main/java/org/apache/tinkerpop/gremlin/process/GraphTraversalBenchmark.java
new file mode 100644
index 0000000..191f5e7
--- /dev/null
+++ b/gremlin-benchmark/src/main/java/org/apache/tinkerpop/gremlin/process/GraphTraversalBenchmark.java
@@ -0,0 +1,101 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.gremlin.process;
+
+import org.apache.tinkerpop.benchmark.util.AbstractGraphBenchmark;
+import org.apache.tinkerpop.gremlin.LoadGraphWith;
+import org.apache.tinkerpop.gremlin.process.traversal.Path;
+import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__;
+import org.apache.tinkerpop.gremlin.structure.Edge;
+import org.apache.tinkerpop.gremlin.structure.Vertex;
+
+import org.openjdk.jmh.annotations.Benchmark;
+
+import java.util.List;
+import java.util.Map;
+
+import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.out;
+
+/**
+ * Runs a traversal benchmarks against a {@link org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerGraph} loaded
+ * with the Grateful Dead data set.
+ *
+ * @author Ted Wilmes (http://twilmes.org)
+ */
+@LoadGraphWith(LoadGraphWith.GraphData.GRATEFUL)
+public class GraphTraversalBenchmark extends AbstractGraphBenchmark {
+
+ @Benchmark
+ public List<Vertex> g_V_outE_inV_outE_inV_outE_inV() throws Exception {
+ return g.V().outE().inV().outE().inV().outE().inV().toList();
+ }
+
+ @Benchmark
+ public List<Vertex> g_V_out_out_out() throws Exception {
+ return g.V().out().out().out().toList();
+ }
+
+ @Benchmark
+ public List<Path> g_V_out_out_out_path() throws Exception {
+ return g.V().out().out().out().path().toList();
+ }
+
+ @Benchmark
+ public List<Vertex> g_V_repeatXoutX_timesX2X() throws Exception {
+ return g.V().repeat(out()).times(2).toList();
+ }
+
+ @Benchmark
+ public List<Vertex> g_V_repeatXoutX_timesX3X() throws Exception {
+ return g.V().repeat(out()).times(3).toList();
+ }
+
+ @Benchmark
+ public List<List<Object>> g_V_localXout_out_valuesXnameX_foldX() throws Exception {
+ return g.V().local(out().out().values("name").fold()).toList();
+ }
+
+ @Benchmark
+ public List<List<Object>> g_V_out_localXout_out_valuesXnameX_foldX() throws Exception {
+ return g.V().out().local(out().out().values("name").fold()).toList();
+ }
+
+ @Benchmark
+ public List<List<Object>> g_V_out_mapXout_out_valuesXnameX_toListX() throws Exception {
+ return g.V().out().map(v -> g.V(v.get()).out().out().values("name").toList()).toList();
+ }
+
+ @Benchmark
+ public List<Map<Object, Long>> g_V_label_groupCount() throws Exception {
+ return g.V().label().groupCount().toList();
+ }
+
+ @Benchmark
+ public List<Object> g_V_match_selectXbX_valuesXnameX() throws Exception {
+ return g.V().match(
+ __.as("a").has("name", "Garcia"),
+ __.as("a").in("writtenBy").as("b"),
+ __.as("a").in("sungBy").as("b")).select("b").values("name").toList();
+ }
+
+ @Benchmark
+ public List<Edge> g_E_hasLabelXwrittenByX_whereXinV_inEXsungByX_count_isX0XX_subgraphXsgX() throws Exception {
+ return g.E().hasLabel("writtenBy").where(__.inV().inE("sungBy").count().is(0)).subgraph("sg").toList();
+ }
+}
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/682d313e/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/LoadGraphWith.java
----------------------------------------------------------------------
diff --git a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/LoadGraphWith.java b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/LoadGraphWith.java
index 17893e5..071d579 100644
--- a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/LoadGraphWith.java
+++ b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/LoadGraphWith.java
@@ -52,7 +52,7 @@ import static org.apache.tinkerpop.gremlin.structure.Graph.Features.VertexFeatur
*/
@Inherited
@Retention(RetentionPolicy.RUNTIME)
-@Target(ElementType.METHOD)
+@Target({ElementType.TYPE, ElementType.METHOD})
public @interface LoadGraphWith {
public enum GraphData {
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/682d313e/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/ProcessPerformanceSuite.java
----------------------------------------------------------------------
diff --git a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/ProcessPerformanceSuite.java b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/ProcessPerformanceSuite.java
index f787450..1b2c36c 100644
--- a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/ProcessPerformanceSuite.java
+++ b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/ProcessPerformanceSuite.java
@@ -36,7 +36,10 @@ import org.junit.runners.model.RunnerBuilder;
* For more information on the usage of this suite, please see {@link StructureStandardSuite}.
*
* @author Stephen Mallette (http://stephen.genoprime.com)
+ *
+ * @deprecated As of release 3.2.0. Provider performance tests may be implemented as needed by providers and will not be included as part of the TinkerPop distribution.
*/
+@Deprecated
public class ProcessPerformanceSuite extends AbstractGremlinSuite {
/**
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/682d313e/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/TraversalPerformanceTest.java
----------------------------------------------------------------------
diff --git a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/TraversalPerformanceTest.java b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/TraversalPerformanceTest.java
index b78d4bf..be750f4 100644
--- a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/TraversalPerformanceTest.java
+++ b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/TraversalPerformanceTest.java
@@ -36,10 +36,13 @@ import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.out;
/**
* @author Stephen Mallette (http://stephen.genoprime.com)
* @author Daniel Kuppitz (http://gremlin.guru)
+ *
+ * @deprecated As of release 3.2.0. Provider performance tests may be implemented as needed by providers and will not be included as part of the TinkerPop distribution.
*/
@AxisRange(min = 0, max = 1)
@BenchmarkMethodChart(filePrefix = "gremlin-traversal")
@BenchmarkHistoryChart(labelWith = LabelType.CUSTOM_KEY, maxRuns = 20, filePrefix = "hx-gremlin-traversal")
+@Deprecated
public class TraversalPerformanceTest extends AbstractGremlinTest {
public final static int DEFAULT_BENCHMARK_ROUNDS = 10;
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/682d313e/pom.xml
----------------------------------------------------------------------
diff --git a/pom.xml b/pom.xml
index 98e7553..4937bdd 100644
--- a/pom.xml
+++ b/pom.xml
@@ -103,11 +103,17 @@ limitations under the License.
<email>dylan.millikin@gmail.com</email>
<url>https://github.com/PommeVerte</url>
</contributor>
+ <contributor>
+ <name>Ted Wilmes</name>
+ <email>twilmes@gmail.com</email>
+ <url>https://github.com/twilmes</url>
+ </contributor>
</contributors>
<modules>
<module>gremlin-shaded</module>
<module>gremlin-core</module>
<module>gremlin-test</module>
+ <module>gremlin-benchmark</module>
<module>gremlin-groovy</module>
<module>gremlin-groovy-test</module>
<module>tinkergraph-gremlin</module>
@@ -874,7 +880,7 @@ limitations under the License.
<overview>${basedir}/docs/javadoc/overview.html</overview>
<quiet>true</quiet>
<sourcepath>
- giraph-gremlin/src/main/java:gremlin-core/src/main/java:gremlin-driver/src/main/java:gremlin-groovy/src/main/java:gremlin-groovy-test/src/main/java:gremlin-server/src/main/java:gremlin-test/src/main/java:hadoop-gremlin/src/main/java:neo4j-gremlin/src/main/java:spark-gremlin/src/main/java:tinkergraph-gremlin/src/main/java
+ giraph-gremlin/src/main/java:gremlin-core/src/main/java:gremlin-driver/src/main/java:gremlin-groovy/src/main/java:gremlin-groovy-test/src/main/java:gremlin-server/src/main/java:gremlin-test/src/main/java:hadoop-gremlin/src/main/java:neo4j-gremlin/src/main/java:spark-gremlin/src/main/java:tinkergraph-gremlin/src/main/java:gremlin-benchmark/src/main/java
</sourcepath>
</configuration>
</execution>