You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tinkerpop.apache.org by sp...@apache.org on 2015/11/24 01:13:42 UTC
[01/11] incubator-tinkerpop git commit: TINKERPOP3-886: Allow any
GraphReader/Writer to be persistence engine for TinkerGraph
Repository: incubator-tinkerpop
Updated Branches:
refs/heads/master f0d52e6a6 -> 46bacb3af
TINKERPOP3-886: Allow any GraphReader/Writer to be persistence engine for TinkerGraph
Project: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/commit/5c4e9660
Tree: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/tree/5c4e9660
Diff: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/diff/5c4e9660
Branch: refs/heads/master
Commit: 5c4e9660d7fdf3c6d3aa71dd3742a4f0d549cf01
Parents: c2c58a7
Author: Kushal <ku...@foodinhood.com>
Authored: Sat Nov 7 16:44:43 2015 -0800
Committer: Kushal <ku...@foodinhood.com>
Committed: Sat Nov 7 16:44:43 2015 -0800
----------------------------------------------------------------------
docs/src/implementations.asciidoc | 3 ++-
.../tinkerpop/gremlin/tinkergraph/structure/TinkerGraph.java | 6 +++++-
2 files changed, 7 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/5c4e9660/docs/src/implementations.asciidoc
----------------------------------------------------------------------
diff --git a/docs/src/implementations.asciidoc b/docs/src/implementations.asciidoc
old mode 100644
new mode 100755
index b5c2c4d..66e2738
--- a/docs/src/implementations.asciidoc
+++ b/docs/src/implementations.asciidoc
@@ -630,7 +630,8 @@ TinkerGraph has several settings that can be provided on creation via `Configura
value is specified here, the the `gremlin.tinkergraph.graphFormat` should also be specified. If this value is not
included (default), then the graph will stay in-memory and not be loaded/persisted to disk.
|gremlin.tinkergraph.graphFormat |The format to use to serialize the graph which may be one of the following:
-`graphml`, `graphson`, or `gryo`. If a value is specified here, the the `gremlin.tinkergraph.graphLocation` should
+`graphml`, `graphson`, gryo`, or a fully qualified class name that implements Io.Builder interface.
+If a value is specified here, the the `gremlin.tinkergraph.graphLocation` should
also be specified. If this value is not included (default), then the graph will stay in-memory and not be
loaded/persisted to disk.
|=========================================================
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/5c4e9660/tinkergraph-gremlin/src/main/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/TinkerGraph.java
----------------------------------------------------------------------
diff --git a/tinkergraph-gremlin/src/main/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/TinkerGraph.java b/tinkergraph-gremlin/src/main/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/TinkerGraph.java
old mode 100644
new mode 100755
index 9fc1d3e..5c17ac6
--- a/tinkergraph-gremlin/src/main/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/TinkerGraph.java
+++ b/tinkergraph-gremlin/src/main/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/TinkerGraph.java
@@ -282,8 +282,12 @@ public final class TinkerGraph implements Graph {
io(IoCore.graphson()).readGraph(graphLocation);
} else if (graphFormat.equals("gryo")) {
io(IoCore.gryo()).readGraph(graphLocation);
+ } else {
+ Class<Io.Builder> ioBuilderClass = (Class<Io.Builder>) Class.forName(graphFormat); //If graphFormat is not a fully qualified class, get ClassNotFoundException. Will this be clear enough to user? currently they will just RuntimeException at end of method
+ Io.Builder ioBuilder = ioBuilderClass.newInstance(); //If graphFormat is class not derived from Io.Builder, get ClassCastdException. Will this be clear enough to user?
+ io(ioBuilder).readGraph(graphLocation);
}
- } catch (IOException ioe) {
+ } catch (Exception exc) {
throw new RuntimeException(String.format("Could not load graph at %s with %s", graphLocation, graphFormat));
}
}
[10/11] incubator-tinkerpop git commit: Merge from TINKERPOP3-886
brought some file permission changes for some reason.
Posted by sp...@apache.org.
Merge from TINKERPOP3-886 brought some file permission changes for some reason.
Converted from rwx to rw-
Project: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/commit/5f1a354e
Tree: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/tree/5f1a354e
Diff: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/diff/5f1a354e
Branch: refs/heads/master
Commit: 5f1a354e577b7c31f00469c91acae61a6144a85f
Parents: 94a4e48
Author: Stephen Mallette <sp...@genoprime.com>
Authored: Mon Nov 23 19:11:52 2015 -0500
Committer: Stephen Mallette <sp...@genoprime.com>
Committed: Mon Nov 23 19:11:52 2015 -0500
----------------------------------------------------------------------
CHANGELOG.asciidoc | 0
docs/src/reference/implementations.asciidoc | 0
.../main/java/org/apache/tinkerpop/gremlin/structure/io/IoCore.java | 0
.../apache/tinkerpop/gremlin/tinkergraph/structure/TinkerGraph.java | 0
.../tinkerpop/gremlin/tinkergraph/structure/TinkerGraphTest.java | 0
5 files changed, 0 insertions(+), 0 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/5f1a354e/CHANGELOG.asciidoc
----------------------------------------------------------------------
diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc
old mode 100755
new mode 100644
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/5f1a354e/docs/src/reference/implementations.asciidoc
----------------------------------------------------------------------
diff --git a/docs/src/reference/implementations.asciidoc b/docs/src/reference/implementations.asciidoc
old mode 100755
new mode 100644
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/5f1a354e/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/IoCore.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/IoCore.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/IoCore.java
old mode 100755
new mode 100644
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/5f1a354e/tinkergraph-gremlin/src/main/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/TinkerGraph.java
----------------------------------------------------------------------
diff --git a/tinkergraph-gremlin/src/main/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/TinkerGraph.java b/tinkergraph-gremlin/src/main/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/TinkerGraph.java
old mode 100755
new mode 100644
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/5f1a354e/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/TinkerGraphTest.java
----------------------------------------------------------------------
diff --git a/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/TinkerGraphTest.java b/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/TinkerGraphTest.java
old mode 100755
new mode 100644
[09/11] incubator-tinkerpop git commit: Merge branch 'TINKERPOP3-886'
Posted by sp...@apache.org.
Merge branch 'TINKERPOP3-886'
Project: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/commit/94a4e489
Tree: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/tree/94a4e489
Diff: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/diff/94a4e489
Branch: refs/heads/master
Commit: 94a4e48906c1a631b7b7ec2223d9cdc4962609e7
Parents: 3251482 be12d3e
Author: Stephen Mallette <sp...@genoprime.com>
Authored: Mon Nov 23 19:04:11 2015 -0500
Committer: Stephen Mallette <sp...@genoprime.com>
Committed: Mon Nov 23 19:04:11 2015 -0500
----------------------------------------------------------------------
CHANGELOG.asciidoc | 1 +
docs/src/reference/implementations.asciidoc | 4 +-
.../tinkerpop/gremlin/structure/io/IoCore.java | 6 ++
.../tinkergraph/structure/TinkerGraph.java | 14 ++--
.../tinkergraph/structure/TinkerGraphTest.java | 67 ++++++++++++++++++++
5 files changed, 87 insertions(+), 5 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/94a4e489/CHANGELOG.asciidoc
----------------------------------------------------------------------
diff --cc CHANGELOG.asciidoc
index 5122f74,e6fce8e..e179533
mode 100644,100755..100755
--- a/CHANGELOG.asciidoc
+++ b/CHANGELOG.asciidoc
[02/11] incubator-tinkerpop git commit: Merge remote-tracking branch
'upstream/master'
Posted by sp...@apache.org.
Merge remote-tracking branch 'upstream/master'
Project: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/commit/16faa852
Tree: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/tree/16faa852
Diff: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/diff/16faa852
Branch: refs/heads/master
Commit: 16faa852e350b4d06a1f600078b6604b568438a7
Parents: 5c4e966 5003fad
Author: Kushal <ku...@foodinhood.com>
Authored: Sat Nov 7 17:02:08 2015 -0800
Committer: Kushal <ku...@foodinhood.com>
Committed: Sat Nov 7 17:02:08 2015 -0800
----------------------------------------------------------------------
docs/src/developer-administration.asciidoc | 91 ++++++++++++++++++++
docs/src/developer.asciidoc | 2 +
docs/src/gremlin-applications.asciidoc | 6 ++
.../PartitionStrategyProcessTest.java | 13 +--
4 files changed, 106 insertions(+), 6 deletions(-)
----------------------------------------------------------------------
[06/11] incubator-tinkerpop git commit: updates
Posted by sp...@apache.org.
updates
Project: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/commit/450a70ca
Tree: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/tree/450a70ca
Diff: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/diff/450a70ca
Branch: refs/heads/master
Commit: 450a70ca745d325175a60cac9791701e6f3ada02
Parents: b71b733
Author: Kushal <ku...@foodinhood.com>
Authored: Fri Nov 20 22:33:01 2015 -0800
Committer: Kushal <ku...@foodinhood.com>
Committed: Fri Nov 20 22:33:01 2015 -0800
----------------------------------------------------------------------
CHANGELOG.asciidoc | 1 +
.../java/org/apache/tinkerpop/gremlin/structure/io/IoCore.java | 6 +++---
.../gremlin/tinkergraph/structure/TinkerGraphTest.java | 6 +++++-
3 files changed, 9 insertions(+), 4 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/450a70ca/CHANGELOG.asciidoc
----------------------------------------------------------------------
diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc
old mode 100644
new mode 100755
index 5f3018a..e6fce8e
--- a/CHANGELOG.asciidoc
+++ b/CHANGELOG.asciidoc
@@ -29,6 +29,7 @@ TinkerPop 3.1.1 (NOT OFFICIALLY RELEASED YET)
* Added `explain()`-step which yields a `TraversalExplanation` with a pretty `toString()` detailing the compilation process.
* Fixed a traversal strategy ordering bug in `AdjacentToIncidentStrategy` and `IncidentToAdjacentStrategy`.
* Made a number of changes to improve traversal startup and execution performance.
+* Added support for 'gremlin.tinkergraph.graphLocation' to accept a fully qualified class name that implements Io.Builder interface
[[release-3.1.0-incubating]]
TinkerPop 3.1.0 (Release Date: November 16, 2015)
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/450a70ca/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/IoCore.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/IoCore.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/IoCore.java
index 9295269..9d9ad60 100755
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/IoCore.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/IoCore.java
@@ -53,9 +53,9 @@ public final class IoCore {
return GryoIo.build();
}
- public static Io.Builder createIoBuilder(String graphFormat) throws ClassNotFoundException, IllegalAccessException, InstantiationException {
- Class<Io.Builder> ioBuilderClass = (Class<Io.Builder>) Class.forName(graphFormat);
- Io.Builder ioBuilder = ioBuilderClass.newInstance();
+ public static Io.Builder createIoBuilder(final String graphFormat) throws ClassNotFoundException, IllegalAccessException, InstantiationException {
+ final Class<Io.Builder> ioBuilderClass = (Class<Io.Builder>) Class.forName(graphFormat);
+ final Io.Builder ioBuilder = ioBuilderClass.newInstance();
return ioBuilder;
}
}
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/450a70ca/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/TinkerGraphTest.java
----------------------------------------------------------------------
diff --git a/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/TinkerGraphTest.java b/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/TinkerGraphTest.java
index 1f627fc..7ff797b 100755
--- a/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/TinkerGraphTest.java
+++ b/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/TinkerGraphTest.java
@@ -32,7 +32,11 @@ import org.apache.tinkerpop.gremlin.structure.io.IoTest;
import org.junit.Ignore;
import org.junit.Test;
-import java.io.*;
+import java.io.BufferedOutputStream;
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.File;
+import java.io.FileOutputStream;
import java.util.Set;
import static org.junit.Assert.assertEquals;
[05/11] incubator-tinkerpop git commit: Merge remote-tracking branch
'upstream/master'
Posted by sp...@apache.org.
Merge remote-tracking branch 'upstream/master'
Project: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/commit/b71b733c
Tree: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/tree/b71b733c
Diff: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/diff/b71b733c
Branch: refs/heads/master
Commit: b71b733cc0fa26d98d15ec628298b6cd58610c27
Parents: c6626da 4a995c2
Author: Kushal <ku...@foodinhood.com>
Authored: Fri Nov 20 22:20:31 2015 -0800
Committer: Kushal <ku...@foodinhood.com>
Committed: Fri Nov 20 22:20:31 2015 -0800
----------------------------------------------------------------------
CHANGELOG.asciidoc | 100 +++-
docs/src/developer-contributing.asciidoc | 70 ++-
docs/src/developer-meetings.asciidoc | 2 +-
docs/src/developer-release.asciidoc | 45 +-
docs/src/gremlin-applications.asciidoc | 27 +
docs/src/the-graph.asciidoc | 3 +
docs/src/the-traversal.asciidoc | 35 +-
docs/src/tutorials-getting-started.asciidoc | 571 +++++++++++++++++++
.../upgrade-release-3.0.x-incubating.asciidoc | 91 ++-
.../upgrade-release-3.1.x-incubating.asciidoc | 217 +++++--
.../images/gremlin-on-software-vertex.png | Bin 0 -> 39573 bytes
docs/static/images/gremlin-to-the-6.png | Bin 0 -> 71084 bytes
docs/static/images/gremlin-to-the-7.png | Bin 0 -> 80114 bytes
docs/static/images/gremlin-to-the-8.png | Bin 0 -> 107598 bytes
docs/static/images/gremlin-to-the-9.png | Bin 0 -> 135099 bytes
.../images/modern-edge-1-to-3-1-gremlin.png | Bin 0 -> 11607 bytes
docs/static/images/modern-edge-1-to-3-1.png | Bin 0 -> 3210 bytes
.../images/modern-edge-1-to-3-2-gremlin.png | Bin 0 -> 17029 bytes
docs/static/images/modern-edge-1-to-3-2.png | Bin 0 -> 4420 bytes
.../images/modern-edge-1-to-3-3-gremlin.png | Bin 0 -> 20948 bytes
docs/static/images/modern-edge-1-to-3-3.png | Bin 0 -> 7372 bytes
docs/static/images/quantum-gremlin-atom.png | Bin 0 -> 816113 bytes
giraph-gremlin/pom.xml | 2 +-
gremlin-console/pom.xml | 2 +-
.../groovy/plugin/DriverRemoteAcceptor.java | 38 +-
.../DriverRemoteAcceptorIntegrateTest.java | 8 +
.../groovy/plugin/DriverRemoteAcceptorTest.java | 40 ++
gremlin-core/pom.xml | 2 +-
.../gremlin/process/traversal/Traversal.java | 12 +
.../process/traversal/step/util/Parameters.java | 15 +-
.../finalization/EngineDependentStrategy.java | 8 +-
.../AdjacentToIncidentStrategy.java | 5 +-
.../IncidentToAdjacentStrategy.java | 6 +
.../traversal/util/DefaultTraversal.java | 13 +-
.../util/DefaultTraversalStrategies.java | 2 +-
.../traversal/util/TraversalExplanation.java | 117 ++++
.../process/traversal/util/TraversalHelper.java | 8 +-
.../structure/io/graphson/GraphSONMapper.java | 2 +-
.../structure/io/graphson/GraphSONModule.java | 3 +
.../io/graphson/GraphSONSerializers.java | 47 ++
.../gremlin/structure/io/gryo/GryoMapper.java | 19 +-
.../util/AbstractThreadLocalTransaction.java | 40 +-
.../util/AbstractThreadedTransaction.java | 44 +-
.../structure/util/AbstractTransaction.java | 54 +-
gremlin-driver/pom.xml | 2 +-
.../apache/tinkerpop/gremlin/driver/Client.java | 24 +
.../tinkerpop/gremlin/driver/Cluster.java | 11 +-
...raphSONMessageSerializerGremlinV1d0Test.java | 46 +-
.../ser/GraphSONMessageSerializerV1d0Test.java | 58 +-
.../ser/GryoMessageSerializerV1d0Test.java | 31 +
gremlin-groovy-test/pom.xml | 2 +-
.../step/sideEffect/GroovyExplainTest.groovy | 35 ++
.../step/sideEffect/GroovyGroupCountTest.groovy | 5 +
.../step/sideEffect/GroovyGroupTestV3d0.groovy | 8 +-
.../process/GroovyProcessComputerSuite.java | 2 +
.../process/GroovyProcessStandardSuite.java | 2 +
gremlin-groovy/pom.xml | 2 +-
.../gremlin/groovy/plugin/Artifact.java | 10 +-
.../gremlin/groovy/plugin/ArtifactTest.java | 87 +++
.../groovy/util/DependencyGrabberTest.java | 2 +
gremlin-server/pom.xml | 2 +-
gremlin-server/scripts/generate-modern.groovy | 2 +-
.../server/GremlinDriverIntegrateTest.java | 7 +-
.../server/GremlinResultSetIntegrateTest.java | 29 +
gremlin-shaded/pom.xml | 2 +-
gremlin-test/pom.xml | 2 +-
.../process/AbstractGremlinProcessTest.java | 14 +-
.../gremlin/process/ProcessComputerSuite.java | 2 +
.../gremlin/process/ProcessStandardSuite.java | 2 +
.../traversal/step/sideEffect/ExplainTest.java | 85 +++
.../step/sideEffect/GroupCountTest.java | 32 +-
.../step/sideEffect/GroupTestV3d0.java | 16 +-
.../gremlin/structure/TransactionTest.java | 84 +++
hadoop-gremlin/pom.xml | 26 +-
.../gremlin/hadoop/structure/HadoopGraph.java | 17 +-
neo4j-gremlin/pom.xml | 2 +-
pom.xml | 26 +-
spark-gremlin/pom.xml | 2 +-
tinkergraph-gremlin/pom.xml | 2 +-
.../structure/TinkerGraphPlayTest.java | 4 +-
80 files changed, 2079 insertions(+), 254 deletions(-)
----------------------------------------------------------------------
[03/11] incubator-tinkerpop git commit: updates
Posted by sp...@apache.org.
updates
Project: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/commit/c6626dac
Tree: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/tree/c6626dac
Diff: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/diff/c6626dac
Branch: refs/heads/master
Commit: c6626dac29c4d26503c13da2b7d12b662fd13908
Parents: 16faa85
Author: Kushal <ku...@foodinhood.com>
Authored: Tue Nov 10 17:23:31 2015 -0800
Committer: Kushal <ku...@foodinhood.com>
Committed: Tue Nov 10 17:23:31 2015 -0800
----------------------------------------------------------------------
docs/src/implementations.asciidoc | 5 +-
.../tinkerpop/gremlin/structure/io/IoCore.java | 6 ++
.../tinkergraph/structure/TinkerGraph.java | 16 +++--
.../tinkergraph/structure/TinkerGraphTest.java | 69 +++++++++++++++++++-
4 files changed, 84 insertions(+), 12 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/c6626dac/docs/src/implementations.asciidoc
----------------------------------------------------------------------
diff --git a/docs/src/implementations.asciidoc b/docs/src/implementations.asciidoc
index 66e2738..1a58ef9 100755
--- a/docs/src/implementations.asciidoc
+++ b/docs/src/implementations.asciidoc
@@ -630,8 +630,9 @@ TinkerGraph has several settings that can be provided on creation via `Configura
value is specified here, the the `gremlin.tinkergraph.graphFormat` should also be specified. If this value is not
included (default), then the graph will stay in-memory and not be loaded/persisted to disk.
|gremlin.tinkergraph.graphFormat |The format to use to serialize the graph which may be one of the following:
-`graphml`, `graphson`, gryo`, or a fully qualified class name that implements Io.Builder interface.
-If a value is specified here, the the `gremlin.tinkergraph.graphLocation` should
+`graphml`, `graphson`, `gryo`, or a fully qualified class name that implements Io.Builder interface (which allows for
+external third party graph reader/writer formats to be used for persistence).
+If a value is specified here, then the `gremlin.tinkergraph.graphLocation` should
also be specified. If this value is not included (default), then the graph will stay in-memory and not be
loaded/persisted to disk.
|=========================================================
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/c6626dac/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/IoCore.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/IoCore.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/IoCore.java
old mode 100644
new mode 100755
index a22dd28..9295269
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/IoCore.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/IoCore.java
@@ -52,4 +52,10 @@ public final class IoCore {
public static Io.Builder<GryoIo> gryo() {
return GryoIo.build();
}
+
+ public static Io.Builder createIoBuilder(String graphFormat) throws ClassNotFoundException, IllegalAccessException, InstantiationException {
+ Class<Io.Builder> ioBuilderClass = (Class<Io.Builder>) Class.forName(graphFormat);
+ Io.Builder ioBuilder = ioBuilderClass.newInstance();
+ return ioBuilder;
+ }
}
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/c6626dac/tinkergraph-gremlin/src/main/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/TinkerGraph.java
----------------------------------------------------------------------
diff --git a/tinkergraph-gremlin/src/main/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/TinkerGraph.java b/tinkergraph-gremlin/src/main/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/TinkerGraph.java
index 5c17ac6..35e268e 100755
--- a/tinkergraph-gremlin/src/main/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/TinkerGraph.java
+++ b/tinkergraph-gremlin/src/main/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/TinkerGraph.java
@@ -283,12 +283,10 @@ public final class TinkerGraph implements Graph {
} else if (graphFormat.equals("gryo")) {
io(IoCore.gryo()).readGraph(graphLocation);
} else {
- Class<Io.Builder> ioBuilderClass = (Class<Io.Builder>) Class.forName(graphFormat); //If graphFormat is not a fully qualified class, get ClassNotFoundException. Will this be clear enough to user? currently they will just RuntimeException at end of method
- Io.Builder ioBuilder = ioBuilderClass.newInstance(); //If graphFormat is class not derived from Io.Builder, get ClassCastdException. Will this be clear enough to user?
- io(ioBuilder).readGraph(graphLocation);
+ io(IoCore.createIoBuilder(graphFormat)).readGraph(graphLocation);
}
- } catch (Exception exc) {
- throw new RuntimeException(String.format("Could not load graph at %s with %s", graphLocation, graphFormat));
+ } catch (Exception ex) {
+ throw new RuntimeException(String.format("Could not load graph at %s with %s", graphLocation, graphFormat), ex);
}
}
}
@@ -311,12 +309,16 @@ public final class TinkerGraph implements Graph {
io(IoCore.graphson()).writeGraph(graphLocation);
} else if (graphFormat.equals("gryo")) {
io(IoCore.gryo()).writeGraph(graphLocation);
+ } else {
+ io(IoCore.createIoBuilder(graphFormat)).writeGraph(graphLocation);
}
- } catch (IOException ioe) {
- throw new RuntimeException(String.format("Could not save graph at %s with %s", graphLocation, graphFormat));
+ } catch (Exception ex) {
+ throw new RuntimeException(String.format("Could not save graph at %s with %s", graphLocation, graphFormat), ex);
}
}
+
+
private <T extends Element> Iterator<T> createElementIterator(final Class<T> clazz, final Map<Object, T> elements,
final IdManager idManager,
final Object... ids) {
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/c6626dac/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/TinkerGraphTest.java
----------------------------------------------------------------------
diff --git a/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/TinkerGraphTest.java b/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/TinkerGraphTest.java
old mode 100644
new mode 100755
index b3dc64e..1f627fc
--- a/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/TinkerGraphTest.java
+++ b/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/TinkerGraphTest.java
@@ -23,19 +23,21 @@ import org.apache.commons.configuration.Configuration;
import org.apache.tinkerpop.gremlin.TestHelper;
import org.apache.tinkerpop.gremlin.process.traversal.P;
import org.apache.tinkerpop.gremlin.structure.Edge;
+import org.apache.tinkerpop.gremlin.structure.Graph;
import org.apache.tinkerpop.gremlin.structure.Vertex;
+import org.apache.tinkerpop.gremlin.structure.io.Io;
import org.apache.tinkerpop.gremlin.structure.io.IoCore;
+import org.apache.tinkerpop.gremlin.structure.io.IoRegistry;
import org.apache.tinkerpop.gremlin.structure.io.IoTest;
import org.junit.Ignore;
import org.junit.Test;
-import java.io.ByteArrayInputStream;
-import java.io.ByteArrayOutputStream;
-import java.io.File;
+import java.io.*;
import java.util.Set;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;
+import static org.mockito.Mockito.mock;
/**
* @author Marko A. Rodriguez (http://markorodriguez.com)
@@ -382,4 +384,65 @@ public class TinkerGraphTest {
IoTest.assertModernGraph(reloadedGraph, true, false);
reloadedGraph.close();
}
+
+ @Test
+ public void shouldPersistToAnyGraphFormat() {
+ final String graphLocation = TestHelper.makeTestDataPath(TinkerGraphTest.class, "temp").getAbsolutePath() + "shouldPersistToAnyGraphFormat.dat";
+ final File f = new File(graphLocation);
+ if (f.exists() && f.isFile()) f.delete();
+
+ final Configuration conf = new BaseConfiguration();
+ conf.setProperty(TinkerGraph.GREMLIN_TINKERGRAPH_GRAPH_FORMAT, TestIoBuilder.class.getName());
+ conf.setProperty(TinkerGraph.GREMLIN_TINKERGRAPH_GRAPH_LOCATION, graphLocation);
+ final TinkerGraph graph = TinkerGraph.open(conf);
+ TinkerFactory.generateModern(graph);
+
+ //Test write graph
+ graph.close();
+ assertEquals(TestIoBuilder.calledRegistry, 1);
+ assertEquals(TestIoBuilder.calledGraph, 1);
+ assertEquals(TestIoBuilder.calledCreate, 1);
+
+ try (BufferedOutputStream os = new BufferedOutputStream(new FileOutputStream(f))){
+ os.write("dummy string".getBytes());
+ } catch (Exception e) {
+ e.printStackTrace();
+ }
+
+ //Test read graph
+ final TinkerGraph readGraph = TinkerGraph.open(conf);
+ assertEquals(TestIoBuilder.calledRegistry, 1);
+ assertEquals(TestIoBuilder.calledGraph, 1);
+ assertEquals(TestIoBuilder.calledCreate, 1);
+ }
+
+ public static class TestIoBuilder implements Io.Builder{
+
+ static int calledRegistry, calledGraph, calledCreate;
+
+ public TestIoBuilder(){
+ //Looks awkward to reset static vars inside a constructor, but makes sense from testing perspective
+ calledRegistry=0;
+ calledGraph=0;
+ calledCreate=0;
+ }
+
+ @Override
+ public Io.Builder<? extends Io> registry(IoRegistry registry) {
+ calledRegistry++;
+ return this;
+ }
+
+ @Override
+ public Io.Builder<? extends Io> graph(Graph graph) {
+ calledGraph++;
+ return this;
+ }
+
+ @Override
+ public Io create() {
+ calledCreate++;
+ return mock(Io.class);
+ }
+ }
}
[11/11] incubator-tinkerpop git commit: Merge remote-tracking branch
'origin/master'
Posted by sp...@apache.org.
Merge remote-tracking branch 'origin/master'
Project: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/commit/46bacb3a
Tree: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/tree/46bacb3a
Diff: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/diff/46bacb3a
Branch: refs/heads/master
Commit: 46bacb3afb92788623004fe9dfd14f5dbdc1dcdc
Parents: 5f1a354 f0d52e6
Author: Stephen Mallette <sp...@genoprime.com>
Authored: Mon Nov 23 19:13:26 2015 -0500
Committer: Stephen Mallette <sp...@genoprime.com>
Committed: Mon Nov 23 19:13:26 2015 -0500
----------------------------------------------------------------------
docs/src/reference/the-traversal.asciidoc | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
[08/11] incubator-tinkerpop git commit: Merge branch 'TINKERPOP3-886'
Posted by sp...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/94a4e489/docs/src/reference/implementations.asciidoc
----------------------------------------------------------------------
diff --cc docs/src/reference/implementations.asciidoc
index b5c2c4d,0000000..1a58ef9
mode 100644,000000..100755
--- a/docs/src/reference/implementations.asciidoc
+++ b/docs/src/reference/implementations.asciidoc
@@@ -1,1764 -1,0 +1,1766 @@@
+////
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+////
+[[implementations]]
+Implementations
+===============
+
+image::gremlin-racecar.png[width=325]
+
+[[graph-system-provider-requirements]]
+Graph System Provider Requirements
+----------------------------------
+
+image:tinkerpop-enabled.png[width=140,float=left] At the core of TinkerPop3 is a Java8 API. The implementation of this
+core API and its validation via the `gremlin-test` suite is all that is required of a graph system provider wishing to
+provide a TinkerPop3-enabled graph engine. Once a graph system has a valid implementation, then all the applications
+provided by TinkerPop (e.g. Gremlin Console, Gremlin Server, etc.) and 3rd-party developers (e.g. Gremlin-Scala,
+Gremlin-JS, etc.) will integrate properly. Finally, please feel free to use the logo on the left to promote your
+TinkerPop3 implementation.
+
+Implementing Gremlin-Core
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The classes that a graph system provider should focus on implementing are itemized below. It is a good idea to study
+the <<tinkergraph-gremlin,TinkerGraph>> (in-memory OLTP and OLAP in `tinkergraph-gremlin`), <<neo4j-gremlin,Neo4jGraph>>
+(OTLP w/ transactions in `neo4j-gremlin`) and/or <<hadoop-gremlin,HadoopGraph>> (OLAP in `hadoop-gremlin`)
+implementations for ideas and patterns.
+
+. Online Transactional Processing Graph Systems (*OLTP*)
+ .. Structure API: `Graph`, `Element`, `Vertex`, `Edge`, `Property` and `Transaction` (if transactions are supported).
+ .. Process API: `TraversalStrategy` instances for optimizing Gremlin traversals to the provider's graph system (i.e. `TinkerGraphStepStrategy`).
+. Online Analytics Processing Graph Systems (*OLAP*)
+ .. Everything required of OTLP is required of OLAP (but not vice versa).
+ .. GraphComputer API: `GraphComputer`, `Messenger`, `Memory`.
+
+Please consider the following implementation notes:
+
+* Be sure your `Graph` implementation is named as `XXXGraph` (e.g. TinkerGraph, Neo4jGraph, HadoopGraph, etc.).
+* Use `StringHelper` to ensuring that the `toString()` representation of classes are consistent with other implementations.
+* Ensure that your implementation's `Features` (Graph, Vertex, etc.) are correct so that test cases handle particulars accordingly.
+* Use the numerous static method helper classes such as `ElementHelper`, `GraphComputerHelper`, `VertexProgramHelper`, etc.
+* There are a number of default methods on the provided interfaces that are semantically correct. However, if they are
+not efficient for the implementation, override them.
+* Implement the `structure/` package interfaces first and then, if desired, interfaces in the `process/` package interfaces.
+* `ComputerGraph` is a `Wrapper` system that ensure proper semantics during a GraphComputer computation.
+
+[[oltp-implementations]]
+OLTP Implementations
+^^^^^^^^^^^^^^^^^^^^
+
+image:pipes-character-1.png[width=110,float=right] The most important interfaces to implement are in the `structure/`
+package. These include interfaces like Graph, Vertex, Edge, Property, Transaction, etc. The `StructureStandardSuite`
+will ensure that the semantics of the methods implemented are correct. Moreover, there are numerous `Exceptions`
+classes with static exceptions that should be thrown by the graph system so that all the exceptions and their
+messages are consistent amongst all TinkerPop3 implementations.
+
+[[olap-implementations]]
+OLAP Implementations
+^^^^^^^^^^^^^^^^^^^^
+
+image:furnace-character-1.png[width=110,float=right] Implementing the OLAP interfaces may be a bit more complicated.
+Note that before OLAP interfaces are implemented, it is necessary for the OLTP interfaces to be, at minimal,
+implemented as specified in <<oltp-implementations,OLTP Implementations>>. A summary of each required interface
+implementation is presented below:
+
+. `GraphComputer`: A fluent builder for specifying an isolation level, a VertexProgram, and any number of MapReduce jobs to be submitted.
+. `Memory`: A global blackboard for ANDing, ORing, INCRing, and SETing values for specified keys.
+. `Messenger`: The system that collects and distributes messages being propagated by vertices executing the VertexProgram application.
+. `MapReduce.MapEmitter`: The system that collects key/value pairs being emitted by the MapReduce applications map-phase.
+. `MapReduce.ReduceEmitter`: The system that collects key/value pairs being emitted by the MapReduce applications combine- and reduce-phases.
+
+NOTE: The VertexProgram and MapReduce interfaces in the `process/computer/` package are not required by the graph
+system. Instead, these are interfaces to be implemented by application developers writing VertexPrograms and MapReduce jobs.
+
+IMPORTANT: TinkerPop3 provides three OLAP implementations: <<tinkergraph-gremlin,TinkerGraphComputer>> (TinkerGraph),
+<<giraphgraphcomputer,GiraphGraphComputer>> (HadoopGraph), and <<sparkgraphcomputer,`SparkGraphComputer`>> (Hadoop).
+Given the complexity of the OLAP system, it is good to study and copy many of the patterns used in these reference
+implementations.
+
+Implementing GraphComputer
+++++++++++++++++++++++++++
+
+image:furnace-character-3.png[width=150,float=right] The most complex method in GraphComputer is the `submit()`-method. The method must do the following:
+
+. Ensure the the GraphComputer has not already been executed.
+. Ensure that at least there is a VertexProgram or 1 MapReduce job.
+. If there is a VertexProgram, validate that it can execute on the GraphComputer given the respectively defined features.
+. Create the Memory to be used for the computation.
+. Execute the VertexProgram.setup() method once and only once.
+. Execute the VertexProgram.execute() method for each vertex.
+. Execute the VertexProgram.terminate() method once and if true, repeat VertexProgram.execute().
+. When VertexProgram.terminate() returns true, move to MapReduce job execution.
+. MapReduce jobs are not required to be executed in any specified order.
+. For each Vertex, execute MapReduce.map(). Then (if defined) execute MapReduce.combine() and MapReduce.reduce().
+. Update Memory with runtime information.
+. Construct a new `ComputerResult` containing the compute Graph and Memory.
+
+Implementing Memory
++++++++++++++++++++
+
+image:gremlin-brain.png[width=175,float=left] The Memory object is initially defined by `VertexProgram.setup()`.
+The memory data is available in the first round of the `VertexProgram.execute()` method. Each Vertex, when executing
+the VertexProgram, can update the Memory in its round. However, the update is not seen by the other vertices until
+the next round. At the end of the first round, all the updates are aggregated and the new memory data is available
+on the second round. This process repeats until the VertexProgram terminates.
+
+Implementing Messenger
+++++++++++++++++++++++
+
+The Messenger object is similar to the Memory object in that a vertex can read and write to the Messenger. However,
+the data it reads are the messages sent to the vertex in the previous step and the data it writes are the messages
+that will be readable by the receiving vertices in the subsequent round.
+
+Implementing MapReduce Emitters
++++++++++++++++++++++++++++++++
+
+image:hadoop-logo-notext.png[width=150,float=left] The MapReduce framework in TinkerPop3 is similar to the model
+popularized by link:http://apache.hadoop.org[Hadoop]. The primary difference is that all Mappers process the vertices
+of the graph, not an arbitrary key/value pair. However, the vertices' edges can not be accessed -- only their
+properties. This greatly reduces the amount of data needed to be pushed through the MapReduce engine as any edge
+information required, can be computed in the VertexProgram.execute() method. Moreover, at this stage, vertices can
+not be mutated, only their token and property data read. A Gremlin OLAP system needs to provide implementations for
+to particular classes: `MapReduce.MapEmitter` and `MapReduce.ReduceEmitter`. TinkerGraph's implementation is provided
+below which demonstrates the simplicity of the algorithm (especially when the data is all within the same JVM).
+
+[source,java]
+----
+public class TinkerMapEmitter<K, V> implements MapReduce.MapEmitter<K, V> {
+
+ public Map<K, Queue<V>> reduceMap;
+ public Queue<KeyValue<K, V>> mapQueue;
+ private final boolean doReduce;
+
+ public TinkerMapEmitter(final boolean doReduce) { <1>
+ this.doReduce = doReduce;
+ if (this.doReduce)
+ this.reduceMap = new ConcurrentHashMap<>();
+ else
+ this.mapQueue = new ConcurrentLinkedQueue<>();
+ }
+
+ @Override
+ public void emit(K key, V value) {
+ if (this.doReduce)
+ this.reduceMap.computeIfAbsent(key, k -> new ConcurrentLinkedQueue<>()).add(value); <2>
+ else
+ this.mapQueue.add(new KeyValue<>(key, value)); <3>
+ }
+
+ protected void complete(final MapReduce<K, V, ?, ?, ?> mapReduce) {
+ if (!this.doReduce && mapReduce.getMapKeySort().isPresent()) { <4>
+ final Comparator<K> comparator = mapReduce.getMapKeySort().get();
+ final List<KeyValue<K, V>> list = new ArrayList<>(this.mapQueue);
+ Collections.sort(list, Comparator.comparing(KeyValue::getKey, comparator));
+ this.mapQueue.clear();
+ this.mapQueue.addAll(list);
+ } else if (mapReduce.getMapKeySort().isPresent()) {
+ final Comparator<K> comparator = mapReduce.getMapKeySort().get();
+ final List<Map.Entry<K, Queue<V>>> list = new ArrayList<>();
+ list.addAll(this.reduceMap.entrySet());
+ Collections.sort(list, Comparator.comparing(Map.Entry::getKey, comparator));
+ this.reduceMap = new LinkedHashMap<>();
+ list.forEach(entry -> this.reduceMap.put(entry.getKey(), entry.getValue()));
+ }
+ }
+}
+----
+
+<1> If the MapReduce job has a reduce, then use one data structure (`reduceMap`), else use another (`mapList`). The
+difference being that a reduction requires a grouping by key and therefore, the `Map<K,Queue<V>>` definition. If no
+reduction/grouping is required, then a simple `Queue<KeyValue<K,V>>` can be leveraged.
+<2> If reduce is to follow, then increment the Map with a new value for the key. `MapHelper` is a TinkerPop3 class
+with static methods for adding data to a Map.
+<3> If no reduce is to follow, then simply append a KeyValue to the queue.
+<4> When the map phase is complete, any map-result sorting required can be executed at this point.
+
+[source,java]
+----
+public class TinkerReduceEmitter<OK, OV> implements MapReduce.ReduceEmitter<OK, OV> {
+
+ protected Queue<KeyValue<OK, OV>> reduceQueue = new ConcurrentLinkedQueue<>();
+
+ @Override
+ public void emit(final OK key, final OV value) {
+ this.reduceQueue.add(new KeyValue<>(key, value));
+ }
+
+ protected void complete(final MapReduce<?, ?, OK, OV, ?> mapReduce) {
+ if (mapReduce.getReduceKeySort().isPresent()) {
+ final Comparator<OK> comparator = mapReduce.getReduceKeySort().get();
+ final List<KeyValue<OK, OV>> list = new ArrayList<>(this.reduceQueue);
+ Collections.sort(list, Comparator.comparing(KeyValue::getKey, comparator));
+ this.reduceQueue.clear();
+ this.reduceQueue.addAll(list);
+ }
+ }
+}
+----
+
+The method `MapReduce.reduce()` is defined as:
+
+[source,java]
+public void reduce(final OK key, final Iterator<OV> values, final ReduceEmitter<OK, OV> emitter) { ... }
+
+In other words, for the TinkerGraph implementation, iterate through the entrySet of the `reduceMap` and call the
+`reduce()` method on each entry. The `reduce()` method can emit key/value pairs which are simply aggregated into a
+`Queue<KeyValue<OK,OV>>` in an analogous fashion to `TinkerMapEmitter` when no reduce is to follow. These two emitters
+are tied together in `TinkerGraphComputer.submit()`.
+
+[source,java]
+----
+...
+for (final MapReduce mapReduce : mapReducers) {
+ if (mapReduce.doStage(MapReduce.Stage.MAP)) {
+ final TinkerMapEmitter<?, ?> mapEmitter = new TinkerMapEmitter<>(mapReduce.doStage(MapReduce.Stage.REDUCE));
+ final SynchronizedIterator<Vertex> vertices = new SynchronizedIterator<>(this.graph.vertices());
+ workers.setMapReduce(mapReduce);
+ workers.mapReduceWorkerStart(MapReduce.Stage.MAP);
+ workers.executeMapReduce(workerMapReduce -> {
+ while (true) {
+ final Vertex vertex = vertices.next();
+ if (null == vertex) return;
+ workerMapReduce.map(ComputerGraph.mapReduce(vertex), mapEmitter);
+ }
+ });
+ workers.mapReduceWorkerEnd(MapReduce.Stage.MAP);
+
+ // sort results if a map output sort is defined
+ mapEmitter.complete(mapReduce);
+
+ // no need to run combiners as this is single machine
+ if (mapReduce.doStage(MapReduce.Stage.REDUCE)) {
+ final TinkerReduceEmitter<?, ?> reduceEmitter = new TinkerReduceEmitter<>();
+ final SynchronizedIterator<Map.Entry<?, Queue<?>>> keyValues = new SynchronizedIterator((Iterator) mapEmitter.reduceMap.entrySet().iterator());
+ workers.mapReduceWorkerStart(MapReduce.Stage.REDUCE);
+ workers.executeMapReduce(workerMapReduce -> {
+ while (true) {
+ final Map.Entry<?, Queue<?>> entry = keyValues.next();
+ if (null == entry) return;
+ workerMapReduce.reduce(entry.getKey(), entry.getValue().iterator(), reduceEmitter);
+ }
+ });
+ workers.mapReduceWorkerEnd(MapReduce.Stage.REDUCE);
+ reduceEmitter.complete(mapReduce); // sort results if a reduce output sort is defined
+ mapReduce.addResultToMemory(this.memory, reduceEmitter.reduceQueue.iterator()); <1>
+ } else {
+ mapReduce.addResultToMemory(this.memory, mapEmitter.mapQueue.iterator()); <2>
+ }
+ }
+}
+...
+----
+
+<1> Note that the final results of the reducer are provided to the Memory as specified by the application developer's
+`MapReduce.addResultToMemory()` implementation.
+<2> If there is no reduce stage, the the map-stage results are inserted into Memory as specified by the application
+developer's `MapReduce.addResultToMemory()` implementation.
+
+[[io-implementations]]
+IO Implementations
+^^^^^^^^^^^^^^^^^^
+
+If a `Graph` requires custom serializers for IO to work properly, implement the `Graph.io` method. A typical example
+of where a `Graph` would require such a custom serializers is if their identifier system uses non-primitive values,
+such as OrientDB's `Rid` class. From basic serialization of a single `Vertex` all the way up the stack to Gremlin
+Server, the need to know how to handle these complex identifiers is an important requirement.
+
+The first step to implementing custom serializers is to first implement the `IoRegistry` interface and register the
+custom classes and serializers to it. Each `Io` implementation has different requirements for what it expects from the
+`IoRegistry`:
+
+* *GraphML* - No custom serializers expected/allowed.
+* *GraphSON* - Register a Jackson `SimpleModule`. The `SimpleModule` encapsulates specific classes to be serialized,
+so it does not need to be registered to a specific class in the `IoRegistry` (use `null`).
+* *Gryo* - Expects registration of one of three objects:
+** Register just the custom class with a `null` Kryo `Serializer` implementation - this class will use default "field-level" Kryo serialization.
+** Register the custom class with a specific Kryo `Serializer' implementation.
+** Register the custom class with a `Function<Kryo, Serializer>` for those cases where the Kryo `Serializer` requires the `Kryo` instance to get constructed.
+
+This implementation should provide a zero-arg constructor as the stack may require instantiation via reflection.
+Consider extending `AbstractIoRegistry` for convenience as follows:
+
+[source,java]
+----
+public class MyGraphIoRegistry extends AbstractIoRegistry {
+ public MyGraphIoRegistry() {
+ register(GraphSONIo.class, null, new MyGraphSimpleModule());
+ register(GryoIo.class, MyGraphIdClass.class, new MyGraphIdSerializer());
+ }
+}
+----
+
+In the `Graph.io` method, provide the `IoRegistry` object to the supplied `Builder` and call the `create` method to
+return that `Io` instance as follows:
+
+[source,java]
+----
+public <I extends Io> I io(final Io.Builder<I> builder) {
+ return (I) builder.graph(this).registry(myGraphIoRegistry).create();
+}}
+----
+
+In this way, `Graph` implementations can pre-configure custom serializers for IO interactions and users will not need
+to know about those details. Following this pattern will ensure proper execution of the test suite as well as
+simplified usage for end-users.
+
+IMPORTANT: Proper implementation of IO is critical to successful `Graph` operations in Gremlin Server. The Test Suite
+does have "serialization" tests that provide some assurance that an implementation is working properly, but those
+tests cannot make assertions against any specifics of a custom serializer. It is the responsibility of the
+implementer to test the specifics of their custom serializers.
+
+TIP: Consider separating serializer code into its own module, if possible, so that clients that use the `Graph`
+implementation remotely don't need a full dependency on the entire `Graph` - just the IO components and related
+classes being serialized.
+
+[[validating-with-gremlin-test]]
+Validating with Gremlin-Test
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+image:gremlin-edumacated.png[width=225]
+
+[source,xml]
+<dependency>
+ <groupId>org.apache.tinkerpop</groupId>
+ <artifactId>gremlin-test</artifactId>
+ <version>x.y.z</version>
+</dependency>
+<dependency>
+ <groupId>org.apache.tinkerpop</groupId>
+ <artifactId>gremlin-groovy-test</artifactId>
+ <version>x.y.z</version>
+</dependency>
+
+The operational semantics of any OLTP or OLAP implementation are validated by `gremlin-test` and functional
+interoperability with the Groovy environment is ensured by `gremlin-groovy-test`. To implement these tests, provide
+test case implementations as shown below, where `XXX` below denotes the name of the graph implementation (e.g.
+TinkerGraph, Neo4jGraph, HadoopGraph, etc.).
+
+[source,java]
+----
+// Structure API tests
+@RunWith(StructureStandardSuite.class)
+@GraphProviderClass(provider = XXXGraphProvider.class, graph = XXXGraph.class)
+public class XXXStructureStandardTest {}
+
+// Process API tests
+@RunWith(ProcessComputerSuite.class)
+@GraphProviderClass(provider = XXXGraphProvider.class, graph = XXXGraph.class)
+public class XXXProcessComputerTest {}
+
+@RunWith(ProcessStandardSuite.class)
+@GraphProviderClass(provider = XXXGraphProvider.class, graph = XXXGraph.class)
+public class XXXProcessStandardTest {}
+
+@RunWith(GroovyEnvironmentSuite.class)
+@GraphProviderClass(provider = XXXProvider.class, graph = TinkerGraph.class)
+public class XXXGroovyEnvironmentTest {}
+
+@RunWith(GroovyProcessStandardSuite.class)
+@GraphProviderClass(provider = XXXGraphProvider.class, graph = TinkerGraph.class)
+public class XXXGroovyProcessStandardTest {}
+
+@RunWith(GroovyProcessComputerSuite.class)
+@GraphProviderClass(provider = XXXGraphComputerProvider.class, graph = TinkerGraph.class)
+public class XXXGroovyProcessComputerTest {}
+----
+
+The above set of tests represent the minimum test suite set to implement. There are other "integration" and
+"performance" tests that should be considered optional. Implementing those tests requires the same pattern as shown above.
+
+IMPORTANT: It is as important to look at "ignored" tests as it is to look at ones that fail. The `gremlin-test`
+suite utilizes the `Feature` implementation exposed by the `Graph` to determine which tests to execute. If a test
+utilizes features that are not supported by the graph, it will ignore them. While that may be fine, implementers
+should validate that the ignored tests are appropriately bypassed and that there are no mistakes in their feature
+definitions. Moreover, implementers should consider filling gaps in their own test suites, especially when
+IO-related tests are being ignored.
+
+The only test-class that requires any code investment is the `GraphProvider` implementation class. This class is a
+used by the test suite to construct `Graph` configurations and instances and provides information about the
+implementation itself. In most cases, it is best to simply extend `AbstractGraphProvider` as it provides many
+default implementations of the `GraphProvider` interface.
+
+Finally, specify the test suites that will be supported by the `Graph` implementation using the `@Graph.OptIn`
+annotation. See the `TinkerGraph` implementation below as an example:
+
+[source,java]
+----
+@Graph.OptIn(Graph.OptIn.SUITE_STRUCTURE_STANDARD)
+@Graph.OptIn(Graph.OptIn.SUITE_PROCESS_STANDARD)
+@Graph.OptIn(Graph.OptIn.SUITE_PROCESS_COMPUTER)
+@Graph.OptIn(Graph.OptIn.SUITE_GROOVY_PROCESS_STANDARD)
+@Graph.OptIn(Graph.OptIn.SUITE_GROOVY_PROCESS_COMPUTER)
+@Graph.OptIn(Graph.OptIn.SUITE_GROOVY_ENVIRONMENT)
+public class TinkerGraph implements Graph {
+----
+
+Only include annotations for the suites the implementation will support. Note that implementing the suite, but
+not specifying the appropriate annotation will prevent the suite from running (an obvious error message will appear
+in this case when running the mis-configured suite).
+
+There are times when there may be a specific test in the suite that the implementation cannot support (despite the
+features it implements) or should not otherwise be executed. It is possible for implementers to "opt-out" of a test
+by using the `@Graph.OptOut` annotation. The following is an example of this annotation usage as taken from
+`HadoopGraph`:
+
+[source, java]
+----
+@Graph.OptIn(Graph.OptIn.SUITE_PROCESS_STANDARD)
+@Graph.OptIn(Graph.OptIn.SUITE_PROCESS_COMPUTER)
+@Graph.OptOut(
+ test = "org.apache.tinkerpop.gremlin.process.graph.step.map.MatchTest$Traversals",
+ method = "g_V_matchXa_hasXname_GarciaX__a_inXwrittenByX_b__a_inXsungByX_bX",
+ reason = "Hadoop-Gremlin is OLAP-oriented and for OLTP operations, linear-scan joins are required. This particular tests takes many minutes to execute.")
+@Graph.OptOut(
+ test = "org.apache.tinkerpop.gremlin.process.graph.step.map.MatchTest$Traversals",
+ method = "g_V_matchXa_inXsungByX_b__a_inXsungByX_c__b_outXwrittenByX_d__c_outXwrittenByX_e__d_hasXname_George_HarisonX__e_hasXname_Bob_MarleyXX",
+ reason = "Hadoop-Gremlin is OLAP-oriented and for OLTP operations, linear-scan joins are required. This particular tests takes many minutes to execute.")
+@Graph.OptOut(
+ test = "org.apache.tinkerpop.gremlin.process.computer.GraphComputerTest",
+ method = "shouldNotAllowBadMemoryKeys",
+ reason = "Hadoop does a hard kill on failure and stops threads which stops test cases. Exception handling semantics are correct though.")
+@Graph.OptOut(
+ test = "org.apache.tinkerpop.gremlin.process.computer.GraphComputerTest",
+ method = "shouldRequireRegisteringMemoryKeys",
+ reason = "Hadoop does a hard kill on failure and stops threads which stops test cases. Exception handling semantics are correct though.")
+public class HadoopGraph implements Graph {
+----
+
+The above examples show how to ignore individual tests. It is also possible to:
+
+* Ignore an entire test case (i.e. all the methods within the test) by setting the `method` to "*".
+* Ignore a "base" test class such that test that extend from those classes will all be ignored. This style of
+ignoring is useful for Gremlin "process" tests that have bases classes that are extended by various Gremlin flavors (e.g. groovy).
+* Ignore a `GraphComputer` test based on the type of `GraphComputer` being used. Specify the "computer" attribute on
+the `OptOut` (which is an array specification) which should have a value of the `GraphComputer` implementation class
+that should ignore that test. This attribute should be left empty for "standard" execution and by default all
+`GraphComputer` implementations will be included in the `OptOut` so if there are multiple implementations, explicitly
+specify the ones that should be excluded.
+
+Also note that some of the tests in the Gremlin Test Suite are parameterized tests and require an additional level of
+specificity to be properly ignored. To ignore these types of tests, examine the name template of the parameterized
+tests. It is defined by a Java annotation that looks like this:
+
+[source, java]
+@Parameterized.Parameters(name = "expect({0})")
+
+The annotation above shows that the name of each parameterized test will be prefixed with "expect" and have
+parentheses wrapped around the first parameter (at index 0) value supplied to each test. This information can
+only be garnered by studying the test set up itself. Once the pattern is determined and the specific unique name of
+the parameterized test is identified, add it to the `specific` property on the `OptOut` annotation in addition to
+the other arguments.
+
+These annotations help provide users a level of transparency into test suite compliance (via the
+xref:describe-graph[describeGraph()] utility function). It also allows implementers to have a lot of flexibility in
+terms of how they wish to support TinkerPop. For example, maybe there is a single test case that prevents an
+implementer from claiming support of a `Feature`. The implementer could choose to either not support the `Feature`
+or to support it but "opt-out" of the test with a "reason" as to why so that users understand the limitation.
+
+IMPORTANT: Before using `OptOut` be sure that the reason for using it is sound and it is more of a last resort.
+It is possible that a test from the suite doesn't properly represent the expectations of a feature, is too broad or
+narrow for the semantics it is trying to enforce or simply contains a bug. Please consider raising issues in the
+developer mailing list with such concerns before assuming `OptOut` is the only answer.
+
+IMPORTANT: There are no tests that specifically validate complete compliance with Gremlin Server. Generally speaking,
+a `Graph` that passes the full Test Suite, should be compliant with Gremlin Server. The one area where problems can
+occur is in serialization. Always ensure that IO is properly implemented, that custom serializers are tested fully
+and ultimately integration test the `Graph` with an actual Gremlin Server instance.
+
+CAUTION: Configuring tests to run in parallel might result in errors that are difficult to debug as there is some
+shared state in test execution around graph configuration. It is therefore recommended that parallelism be turned
+off for the test suite (the Maven SureFire Plugin is configured this way by default). It may also be important to
+include this setting, `<reuseForks>false</reuseForks>`, in the SureFire configuration if tests are failing in an
+unexplainable way.
+
+Accessibility via GremlinPlugin
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+image:gremlin-plugin.png[width=100,float=left] The applications distributed with TinkerPop3 do not distribute with
+any graph system implementations besides TinkerGraph. If your implementation is stored in a Maven repository (e.g.
+Maven Central Repository), then it is best to provide a `GremlinPlugin` implementation so the respective jars can be
+downloaded according and when required by the user. Neo4j's GremlinPlugin is provided below for reference.
+
+[source,java]
+----
+public class Neo4jGremlinPlugin implements GremlinPlugin {
+
+ private static final String IMPORT = "import ";
+ private static final String DOT_STAR = ".*";
+
+ private static final Set<String> IMPORTS = new HashSet<String>() {{
+ add(IMPORT + Neo4jGraph.class.getPackage().getName() + DOT_STAR);
+ }};
+
+ @Override
+ public String getName() {
+ return "neo4j";
+ }
+
+ @Override
+ public void pluginTo(final PluginAcceptor pluginAcceptor) {
+ pluginAcceptor.addImports(IMPORTS);
+ }
+}
+----
+
+With the above plugin implementations, users can now download respective binaries for Gremlin Console, Gremlin Server, etc.
+
+[source,groovy]
+gremlin> g = Neo4jGraph.open('/tmp/neo4j')
+No such property: Neo4jGraph for class: groovysh_evaluate
+Display stack trace? [yN]
+gremlin> :install org.apache.tinkerpop neo4j-gremlin x.y.z
+==>loaded: [org.apache.tinkerpop, neo4j-gremlin, …]
+gremlin> :plugin use tinkerpop.neo4j
+==>tinkerpop.neo4j activated
+gremlin> g = Neo4jGraph.open('/tmp/neo4j')
+==>neo4jgraph[EmbeddedGraphDatabase [/tmp/neo4j]]
+
+In-Depth Implementations
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+image:gremlin-painting.png[width=200,float=right] The graph system implementation details presented thus far are
+minimum requirements necessary to yield a valid TinkerPop3 implementation. However, there are other areas that a
+graph system provider can tweak to provide an implementation more optimized for their underlying graph engine. Typical
+areas of focus include:
+
+* Traversal Strategies: A <<traversalstrategy,TraversalStrategy>> can be used to alter a traversal prior to its
+execution. A typical example is converting a pattern of `g.V().has('name','marko')` into a global index lookup for
+all vertices with name "marko". In this way, a `O(|V|)` lookup becomes an `O(log(|V|))`. Please review
+`TinkerGraphStepStrategy` for ideas.
+* Step Implementations: Every <<graph-traversal-steps,step>> is ultimately referenced by the `GraphTraversal`
+interface. It is possible to extend `GraphTraversal` to use a graph system specific step implementation.
+
+
+[[tinkergraph-gremlin]]
+TinkerGraph-Gremlin
+-------------------
+
+[source,xml]
+----
+<dependency>
+ <groupId>org.apache.tinkerpop</groupId>
+ <artifactId>tinkergraph-gremlin</artifactId>
+ <version>x.y.z</version>
+</dependency>
+----
+
+image:tinkerpop-character.png[width=100,float=left] TinkerGraph is a single machine, in-memory (with optional
+persistence), non-transactional graph engine that provides both OLTP and OLAP functionality. It is deployed with
+TinkerPop3 and serves as the reference implementation for other providers to study in order to understand the
+semantics of the various methods of the TinkerPop3 API. Constructing a simple graph in Java8 is presented below.
+
+[source,java]
+Graph g = TinkerGraph.open();
+Vertex marko = g.addVertex("name","marko","age",29);
+Vertex lop = g.addVertex("name","lop","lang","java");
+marko.addEdge("created",lop,"weight",0.6d);
+
+The above graph creates two vertices named "marko" and "lop" and connects them via a created-edge with a weight=0.6
+property. Next, the graph can be queried as such.
+
+[source,java]
+g.V().has("name","marko").out("created").values("name")
+
+The `g.V().has("name","marko")` part of the query can be executed in two ways.
+
+ * A linear scan of all vertices filtering out those vertices that don't have the name "marko"
+ * A `O(log(|V|))` index lookup for all vertices with the name "marko"
+
+Given the initial graph construction in the first code block, no index was defined and thus, a linear scan is executed.
+However, if the graph was constructed as such, then an index lookup would be used.
+
+[source,java]
+Graph g = TinkerGraph.open();
+g.createIndex("name",Vertex.class)
+
+The execution times for a vertex lookup by property is provided below for both no-index and indexed version of
+TinkerGraph over the Grateful Dead graph.
+
+[gremlin-groovy]
+----
+graph = TinkerGraph.open()
+g = graph.traversal()
+graph.io(graphml()).readGraph('data/grateful-dead.xml')
+clock(1000) {g.V().has('name','Garcia').iterate()} <1>
+graph = TinkerGraph.open()
+g = graph.traversal()
+graph.createIndex('name',Vertex.class)
+graph.io(graphml()).readGraph('data/grateful-dead.xml')
+clock(1000){g.V().has('name','Garcia').iterate()} <2>
+----
+
+<1> Determine the average runtime of 1000 vertex lookups when no `name`-index is defined.
+<2> Determine the average runtime of 1000 vertex lookups when a `name`-index is defined.
+
+IMPORTANT: Each graph system will have different mechanism by which indices and schemas are defined. TinkerPop3
+does not require any conformance in this area. In TinkerGraph, the only definitions are around indices. With other
+graph systems, property value types, indices, edge labels, etc. may be required to be defined _a priori_ to adding
+data to the graph.
+
+NOTE: TinkerGraph is distributed with Gremlin Server and is therefore automatically available to it for configuration.
+
+Configuration
+~~~~~~~~~~~~~
+
+TinkerGraph has several settings that can be provided on creation via `Configuration` object:
+
+[width="100%",cols="2,10",options="header"]
+|=========================================================
+|Property |Description
+|gremlin.graph |`org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerGraph`
+|gremlin.tinkergraph.vertexIdManager |The `IdManager` implementation to use for vertices.
+|gremlin.tinkergraph.edgeIdManager |The `IdManager` implementation to use for edges.
+|gremlin.tinkergraph.vertexPropertyIdManager |The `IdManager` implementation to use for vertex properties.
+|gremlin.tinkergraph.defaultVertexPropertyCardinality |The default `VertexProperty.Cardinality` to use when `Vertex.property(k,v)` is called.
+|gremlin.tinkergraph.graphLocation |The path and file name for where TinkerGraph should persist the graph data. If a
+value is specified here, the the `gremlin.tinkergraph.graphFormat` should also be specified. If this value is not
+included (default), then the graph will stay in-memory and not be loaded/persisted to disk.
+|gremlin.tinkergraph.graphFormat |The format to use to serialize the graph which may be one of the following:
- `graphml`, `graphson`, or `gryo`. If a value is specified here, the the `gremlin.tinkergraph.graphLocation` should
++`graphml`, `graphson`, `gryo`, or a fully qualified class name that implements Io.Builder interface (which allows for
++external third party graph reader/writer formats to be used for persistence).
++If a value is specified here, then the `gremlin.tinkergraph.graphLocation` should
+also be specified. If this value is not included (default), then the graph will stay in-memory and not be
+loaded/persisted to disk.
+|=========================================================
+
+The `IdManager` settings above refer to how TinkerGraph will control identifiers for vertices, edges and vertex
+properties. There are several options for each of these settings: `ANY`, `LONG`, `INTEGER`, `UUID`, or the fully
+qualified class name of an `IdManager` implementation on the classpath. When not specified, the default values
+for all settings is `ANY`, meaning that the graph will work with any object on the JVM as the identifier and will
+generate new identifiers from `Long` when the identifier is not user supplied. TinkerGraph will also expect the
+user to understand the types used for identifiers when querying, meaning that `g.V(1)` and `g.V(1L)` could return
+two different vertices. `LONG`, `INTEGER` and `UUID` settings will try to coerce identifier values to the expected
+type as well as generate new identifiers with that specified type.
+
+If the TinkerGraph is configured for persistence with `gremlin.tinkergraph.graphLocation` and
+`gremlin.tinkergraph.graphFormat`, then the graph will be written to the specified location with the specified
+format when `Graph.close()` is called. In addition, if these settings are present, TinkerGraph will attempt to
+load the graph from the specified location.
+
+It is important to consider the data being imported to TinkerGraph with respect to `defaultVertexPropertyCardinality`
+setting. For example, if a `.gryo` file is known to contain multi-property data, be sure to set the default
+cardinality to `list` or else the data will import as `single`. Consider the following:
+
+[gremlin-groovy]
+----
+graph = TinkerGraph.open()
+graph.io(gryo()).readGraph("data/tinkerpop-crew.kryo")
+g = graph.traversal()
+g.V().properties()
+conf = new BaseConfiguration()
+conf.setProperty("gremlin.tinkergraph.defaultVertexPropertyCardinality","list")
+graph = TinkerGraph.open(conf)
+graph.io(gryo()).readGraph("data/tinkerpop-crew.kryo")
+g = graph.traversal()
+g.V().properties()
+----
+
+[[neo4j-gremlin]]
+Neo4j-Gremlin
+-------------
+
+[source,xml]
+----
+<dependency>
+ <groupId>org.apache.tinkerpop</groupId>
+ <artifactId>neo4j-gremlin</artifactId>
+ <version>x.y.z</version>
+</dependency>
+<!-- neo4j-tinkerpop-api-impl is NOT Apache 2 licensed - more information below -->
+<dependency>
+ <groupId>org.neo4j</groupId>
+ <artifactId>neo4j-tinkerpop-api-impl</artifactId>
+ <version>0.1-2.2</version>
+</dependency>
+----
+
+link:http://neotechnology.com[Neo Technology] are the developers of the OLTP-based link:http://neo4j.org[Neo4j graph database].
+
+CAUTION: Unless under a commercial agreement with Neo Technology, Neo4j is licensed
+link:http://en.wikipedia.org/wiki/Affero_General_Public_License[AGPL]. The `neo4j-gremlin` module is licensed Apache2
+because it only references the Apache2-licensed Neo4j API (not its implementation). Note that neither the
+<<gremlin-console,Gremlin Console>> nor <<gremlin-server,Gremlin Server>> distribute with the Neo4j implementation
+binaries. To access the binaries, use the `:install` command to download binaries from
+link:http://search.maven.org/[Maven Central Repository].
+
+[source,groovy]
+----
+gremlin> :install org.apache.tinkerpop neo4j-gremlin x.y.z
+==>Loaded: [org.apache.tinkerpop, neo4j-gremlin, x.y.z] - restart the console to use [tinkerpop.neo4j]
+gremlin> :q
+...
+gremlin> :plugin use tinkerpop.neo4j
+==>tinkerpop.neo4j activated
+gremlin> graph = Neo4jGraph.open('/tmp/neo4j')
+==>neo4jgraph[EmbeddedGraphDatabase [/tmp/neo4j]]
+----
+
+NOTE: Neo4j link:http://docs.neo4j.org/chunked/stable/ha.html[High Availability] is currently not supported by
+Neo4j-Gremlin.
+
+TIP: To host Neo4j in <<gremlin-server,Gremlin Server>>, the dependencies must first be "installed" or otherwise
+copied to the Gremlin Server path. The automated method for doing this would be to execute
+`bin/gremlin-server.sh -i org.apache.tinkerpop neo4j-gremlin x.y.z`.
+
+Indices
+~~~~~~~
+
+Neo4j 2.x indices leverage vertex labels to partition the index space. TinkerPop3 does not provide method interfaces
+for defining schemas/indices for the underlying graph system. Thus, in order to create indices, it is important to
+call the Neo4j API directly.
+
+NOTE: `Neo4jGraphStep` will attempt to discern which indices to use when executing a traversal of the form `g.V().has()`.
+
+The Gremlin-Console session below demonstrates Neo4j indices. For more information, please refer to the Neo4j documentation:
+
+* Manipulating indices with link:http://docs.neo4j.org/chunked/stable/query-schema-index.html[Cypher].
+* Manipulating indices with the Neo4j link:http://docs.neo4j.org/chunked/stable/tutorials-java-embedded-new-index.html[Java API].
+
+[gremlin-groovy]
+----
+graph = Neo4jGraph.open('/tmp/neo4j')
+graph.cypher("CREATE INDEX ON :person(name)")
+graph.tx().commit() <1>
+graph.addVertex(label,'person','name','marko')
+graph.addVertex(label,'dog','name','puppy')
+g = graph.traversal()
+g.V().hasLabel('person').has('name','marko').values('name')
+graph.close()
+----
+
+<1> Schema mutations must happen in a different transaction than graph mutations
+
+Below demonstrates the runtime benefits of indices and demonstrates how if there is no defined index (only vertex
+labels), a linear scan of the vertex-label partition is still faster than a linear scan of all vertices.
+
+[gremlin-groovy]
+----
+graph = Neo4jGraph.open('/tmp/neo4j')
+graph.io(graphml()).readGraph('data/grateful-dead.xml')
+g = graph.traversal()
+g.tx().commit()
+clock(1000) {g.V().hasLabel('artist').has('name','Garcia').iterate()} <1>
+graph.cypher("CREATE INDEX ON :artist(name)") <2>
+g.tx().commit()
+Thread.sleep(5000) <3>
+clock(1000) {g.V().hasLabel('artist').has('name','Garcia').iterate()} <4>
+clock(1000) {g.V().has('name','Garcia').iterate()} <5>
+graph.cypher("DROP INDEX ON :artist(name)") <6>
+g.tx().commit()
+graph.close()
+----
+
+<1> Find all artists whose name is Garcia which does a linear scan of the artist vertex-label partition.
+<2> Create an index for all artist vertices on their name property.
+<3> Neo4j indices are eventually consistent so this stalls to give the index time to populate itself.
+<4> Find all artists whose name is Garcia which uses the pre-defined schema index.
+<5> Find all vertices whose name is Garcia which requires a linear scan of all the data in the graph.
+<6> Drop the created index.
+
+Multi/Meta-Properties
+~~~~~~~~~~~~~~~~~~~~~
+
+`Neo4jGraph` supports both multi- and meta-properties (see <<_vertex_properties,vertex properties>>). These features
+are not native to Neo4j and are implemented using "hidden" Neo4j nodes. For example, when a vertex has multiple
+"name" properties, each property is a new node (multi-properties) which can have properties attached to it
+(meta-properties). As such, the native, underlying representation may become difficult to query directly using
+another graph language such as <<_cypher,Cypher>>. The default setting is to disable multi- and meta-properties.
+However, if this feature is desired, then it can be activated via `gremlin.neo4j.metaProperties` and
+`gremlin.neo4j.multiProperties` configurations being set to `true`. Once the configuration is set, it can not be
+changed for the lifetime of the graph.
+
+[gremlin-groovy]
+----
+conf = new BaseConfiguration()
+conf.setProperty('gremlin.neo4j.directory','/tmp/neo4j')
+conf.setProperty('gremlin.neo4j.multiProperties',true)
+conf.setProperty('gremlin.neo4j.metaProperties',true)
+graph = Neo4jGraph.open(conf)
+g = graph.traversal()
+g.addV('name','michael','name','michael hunger','name','mhunger')
+g.V().properties('name').property('acl', 'public')
+g.V(0).valueMap()
+g.V(0).properties()
+g.V(0).properties().valueMap()
+graph.close()
+----
+
+WARNING: `Neo4jGraph` without multi- and meta-properties is in 1-to-1 correspondence with the native, underlying Neo4j
+representation. It is recommended that if the user does not require multi/meta-properties, then they should not
+enable them. Without multi- and meta-properties enabled, Neo4j can be interacted with with other tools and technologies
+that do not leverage TinkerPop.
+
+IMPORTANT: When using a multi-property enabled `Neo4jGraph`, vertices may represent their properties on "hidden
+nodes" adjacent to the vertex. If a vertex property key/value is required for indexing, then two indices are
+required -- e.g. `CREATE INDEX ON :person(name)` and `CREATE INDEX ON :vertexProperty(name)`
+(see <<_indices,Neo4j indices>>).
+
+Cypher
+~~~~~~
+
+image::gremlin-loves-cypher.png[width=400]
+
+NeoTechnology are the creators of the graph pattern-match query language link:http://www.neo4j.org/learn/cypher[Cypher].
+It is possible to leverage Cypher from within Gremlin by using the `Neo4jGraph.cypher()` graph traversal method.
+
+[gremlin-groovy]
+----
+graph = Neo4jGraph.open('/tmp/neo4j')
+graph.io(gryo()).readGraph('data/tinkerpop-modern.kryo')
+graph.cypher('MATCH (a {name:"marko"}) RETURN a')
+graph.cypher('MATCH (a {name:"marko"}) RETURN a').select('a').out('knows').values('name')
+graph.close()
+----
+
+Thus, like <<match-step,`match()`>>-step in Gremlin, it is possible to do a declarative pattern match and then move
+back into imperative Gremlin.
+
+TIP: For those developers using <<gremlin-server,Gremlin Server>> against Neo4j, it is possible to do Cypher queries
+by simply placing the Cypher string in `graph.cypher(...)` before submission to the server.
+
+Multi-Label
+~~~~~~~~~~~
+
+TinkerPop3 requires every `Element` to have a single, immutable string label (i.e. a `Vertex`, `Edge`, and
+`VertexProperty`). In Neo4j, a `Node` (vertex) can have an
+link:http://neo4j.com/docs/stable/graphdb-neo4j-labels.html[arbitrary number of labels] while a `Relationship`
+(edge) can have one and only one. Furthermore, in Neo4j, `Node` labels are mutable while `Relationship` labels are
+not. In order to handle this mismatch, three `Neo4jVertex` specific methods exist in Neo4j-Gremlin.
+
+[source,java]
+public Set<String> labels() // get all the labels of the vertex
+public void addLabel(String label) // add a label to the vertex
+public void removeLabel(String label) // remove a label from the vertex
+
+An example use case is presented below.
+
+[gremlin-groovy]
+----
+graph = Neo4jGraph.open('/tmp/neo4j')
+vertex = (Neo4jVertex) graph.addVertex('human::animal') <1>
+vertex.label() <2>
+vertex.labels() <3>
+vertex.addLabel('organism') <4>
+vertex.label()
+vertex.removeLabel('human') <5>
+vertex.labels()
+vertex.addLabel('organism') <6>
+vertex.labels()
+vertex.removeLabel('human') <7>
+vertex.label()
+g = graph.traversal()
+g.V().has(label,'organism') <8>
+g.V().has(label,of('organism')) <9>
+g.V().has(label,of('organism')).has(label,of('animal'))
+g.V().has(label,of('organism').and(of('animal')))
+graph.close()
+----
+
+<1> Typecasting to a `Neo4jVertex` is only required in Java.
+<2> The standard `Vertex.label()` method returns all the labels in alphabetical order concatenated using `::`.
+<3> `Neo4jVertex.labels()` method returns the individual labels as a set.
+<4> `Neo4jVertex.addLabel()` method adds a single label.
+<5> `Neo4jVertex.removeLabel()` method removes a single label.
+<6> Labels are unique and thus duplicate labels don't exist.
+<7> If a label that does not exist is removed, nothing happens.
+<8> `P.eq()` does a full string match and should only be used if multi-labels are not leveraged.
+<9> `LabelP.of()` is specific to `Neo4jGraph` and used for multi-label matching.
+
+IMPORTANT: `LabelP.of()` is only required if multi-labels are leveraged. `LabelP.of()` is used when
+filtering/looking-up vertices by their label(s) as the standard `P.eq()` does a direct match on the `::`-representation
+of `vertex.label()`
+
+Loading with BulkLoaderVertexProgram
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The <<bulkloadervertexprogram, BulkLoaderVertexProgram>> is a generalized bulk loader that can be used to load
+large amounts of data to and from Neo4j. The following code demonstrates how to load the modern graph from TinkerGraph
+into Neo4j:
+
+[gremlin-groovy]
+----
+wgConf = 'conf/neo4j-standalone.properties'
+modern = TinkerFactory.createModern()
+blvp = BulkLoaderVertexProgram.build().
+ keepOriginalIds(false).
+ writeGraph(wgConf).create(modern)
+modern.compute().workers(1).program(blvp).submit().get()
+graph = GraphFactory.open(wgConf)
+g = graph.traversal()
+g.V().valueMap()
+graph.close()
+----
+
+[source,properties]
+----
+# neo4j-standalone.properties
+
+gremlin.graph=org.apache.tinkerpop.gremlin.neo4j.structure.Neo4jGraph
+gremlin.neo4j.directory=/tmp/neo4j
+gremlin.neo4j.conf.node_auto_indexing=true
+gremlin.neo4j.conf.relationship_auto_indexing=true
+----
+
+[[hadoop-gremlin]]
+Hadoop-Gremlin
+--------------
+
+[source,xml]
+----
+<dependency>
+ <groupId>org.apache.tinkerpop</groupId>
+ <artifactId>hadoop-gremlin</artifactId>
+ <version>x.y.z</version>
+</dependency>
+----
+
+image:hadoop-logo-notext.png[width=100,float=left] link:http://hadoop.apache.org/[Hadoop] is a distributed
+computing framework that is used to process data represented across a multi-machine compute cluster. When the
+data in the Hadoop cluster represents a TinkerPop3 graph, then Hadoop-Gremlin can be used to process the graph
+using both TinkerPop3's OLTP and OLAP graph computing models.
+
+IMPORTANT: This section assumes that the user has a Hadoop 2.x cluster functioning. For more information on getting
+started with Hadoop, please see the
+link:http://hadoop.apache.org/docs/r2.7.1/hadoop-project-dist/hadoop-common/SingleCluster.html[Single Node Setup]
+tutorial. Moreover, if using `GiraphGraphComputer` or `SparkGraphComputer` it is advisable that the reader also
+familiarize their self with Giraph (link:http://giraph.apache.org/quick_start.html[Getting Started]) and Spark
+(link:http://spark.apache.org/docs/latest/quick-start.html[Quick Start]).
+
+Installing Hadoop-Gremlin
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The `HADOOP_GREMLIN_LIBS` references locations that contains jars that should be uploaded to a respective
+distributed cache (link:http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html[YARN] or SparkServer).
+Note that the locations in `HADOOP_GREMLIN_LIBS` can be a colon-separated (`:`) and all jars from all locations will
+be loaded into the cluster. Typically, only the jars of the respective GraphComputer are required to be loaded (e.g.
+`GiraphGraphComputer` plugin lib directory).
+
+[source,shell]
+export HADOOP_GREMLIN_LIBS=/usr/local/gremlin-console/ext/giraph-gremlin/lib
+
+If using <<gremlin-console,Gremlin Console>>, it is important to install the Hadoop-Gremlin plugin. Note that
+Hadoop-Gremlin requires a Gremlin Console restart after installing.
+
+[source,text]
+----
+$ bin/gremlin.sh
+
+ \,,,/
+ (o o)
+-----oOOo-(3)-oOOo-----
+plugin activated: tinkerpop.server
+plugin activated: tinkerpop.utilities
+plugin activated: tinkerpop.tinkergraph
+gremlin> :install org.apache.tinkerpop hadoop-gremlin x.y.z
+==>loaded: [org.apache.tinkerpop, hadoop-gremlin, x.y.z] - restart the console to use [tinkerpop.hadoop]
+gremlin> :q
+$ bin/gremlin.sh
+
+ \,,,/
+ (o o)
+-----oOOo-(3)-oOOo-----
+plugin activated: tinkerpop.server
+plugin activated: tinkerpop.utilities
+plugin activated: tinkerpop.tinkergraph
+gremlin> :plugin use tinkerpop.hadoop
+==>tinkerpop.hadoop activated
+gremlin>
+----
+
+Properties Files
+~~~~~~~~~~~~~~~~
+
+`HadoopGraph` makes use of properties files which ultimately get turned into Apache configurations and/or
+Hadoop configurations. The example properties file presented below is located at `conf/hadoop/hadoop-gryo.properties`.
+
+[source,text]
+gremlin.graph=org.apache.tinkerpop.gremlin.hadoop.structure.HadoopGraph
+gremlin.hadoop.inputLocation=tinkerpop-modern.kryo
+gremlin.hadoop.graphInputFormat=org.apache.tinkerpop.gremlin.hadoop.structure.io.gryo.GryoInputFormat
+gremlin.hadoop.outputLocation=output
+gremlin.hadoop.graphOutputFormat=org.apache.tinkerpop.gremlin.hadoop.structure.io.gryo.GryoOutputFormat
+gremlin.hadoop.jarsInDistributedCache=true
+####################################
+# Spark Configuration #
+####################################
+spark.master=local[4]
+spark.executor.memory=1g
+spark.serializer=org.apache.tinkerpop.gremlin.spark.structure.io.gryo.GryoSerializer
+####################################
+# SparkGraphComputer Configuration #
+####################################
+gremlin.spark.graphInputRDD=org.apache.tinkerpop.gremlin.spark.structure.io.InputRDDFormat
+gremlin.spark.graphOutputRDD=org.apache.tinkerpop.gremlin.spark.structure.io.OutputRDDFormat
+gremlin.spark.persistContext=true
+#####################################
+# GiraphGraphComputer Configuration #
+#####################################
+giraph.minWorkers=2
+giraph.maxWorkers=2
+giraph.useOutOfCoreGraph=true
+giraph.useOutOfCoreMessages=true
+mapreduce.map.java.opts=-Xmx1024m
+mapreduce.reduce.java.opts=-Xmx1024m
+giraph.numInputThreads=2
+giraph.numComputeThreads=2
+
+A review of the Hadoop-Gremlin specific properties are provided in the table below. For the respective OLAP
+engines (<<sparkgraphcomputer,`SparkGraphComputer`>> or <<giraphgraphcomputer,`GiraphGraphComputer`>>) refer
+to their respective documentation for configuration options.
+
+[width="100%",cols="2,10",options="header"]
+|=========================================================
+|Property |Description
+|gremlin.graph |The class of the graph to construct using GraphFactory.
+|gremlin.hadoop.inputLocation |The location of the input file(s) for Hadoop-Gremlin to read the graph from.
+|gremlin.hadoop.graphInputFormat |The format that the graph input file(s) are represented in.
+|gremlin.hadoop.outputLocation |The location to write the computed HadoopGraph to.
+|gremlin.hadoop.graphOutputFormat |The format that the output file(s) should be represented in.
+|gremlin.hadoop.jarsInDistributedCache |Whether to upload the Hadoop-Gremlin jars to a distributed cache (necessary if jars are not on the machines' classpaths).
+|=========================================================
+
+
+
+Along with the properties above, the numerous link:http://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/core-default.xml[Hadoop specific properties]
+can be added as needed to tune and parameterize the executed Hadoop-Gremlin job on the respective Hadoop cluster.
+
+IMPORTANT: As the size of the graphs being processed becomes large, it is important to fully understand how the
+underlying OLAP engine (e.g. Spark, Giraph, etc.) works and understand the numerous parameterizations offered by
+these systems. Such knowledge can help alleviate out of memory exceptions, slow load times, slow processing times,
+garbage collection issues, etc.
+
+OLTP Hadoop-Gremlin
+~~~~~~~~~~~~~~~~~~~
+
+image:hadoop-pipes.png[width=180,float=left] It is possible to execute OLTP operations over a `HadoopGraph`.
+However, realize that the underlying HDFS files are not random access and thus, to retrieve a vertex, a linear scan
+is required. OLTP operations are useful for peeking into the graph prior to executing a long running OLAP job -- e.g.
+`g.V().valueMap().limit(10)`.
+
+CAUTION: OLTP operations on `HadoopGraph` are not efficient. They require linear scans to execute and are unreasonable
+for large graphs. In such large graph situations, make use of <<traversalvertexprogram,TraversalVertexProgram>>
+which is the OLAP Gremlin machine.
+
+[gremlin-groovy]
+----
+hdfs.copyFromLocal('data/tinkerpop-modern.kryo', 'tinkerpop-modern.kryo')
+hdfs.ls()
+graph = GraphFactory.open('conf/hadoop/hadoop-gryo.properties')
+g = graph.traversal()
+g.V().count()
+g.V().out().out().values('name')
+g.V().group().by{it.value('name')[1]}.by('name').next()
+----
+
+OLAP Hadoop-Gremlin
+~~~~~~~~~~~~~~~~~~~
+
+image:hadoop-furnace.png[width=180,float=left] Hadoop-Gremlin was designed to execute OLAP operations via
+`GraphComputer`. The OLTP examples presented previously are reproduced below, but using `TraversalVertexProgram`
+for the execution of the Gremlin traversal.
+
+A `Graph` in TinkerPop3 can support any number of `GraphComputer` implementations. Out of the box, Hadoop-Gremlin
+supports the following three implementations.
+
+* <<mapreducegraphcomputer,`MapReduceGraphComputer`>>: Leverages Hadoop's MapReduce engine to execute TinkerPop3 OLAP
+computations. (*coming soon*)
+** The graph must fit within the total disk space of the Hadoop cluster (supports massive graphs). Message passing is
+coordinated via MapReduce jobs over the on-disk graph (slow traversals).
+* <<sparkgraphcomputer,`SparkGraphComputer`>>: Leverages Apache Spark to execute TinkerPop3 OLAP computations.
+** The graph may fit within the total RAM of the cluster (supports larger graphs). Message passing is coordinated via
+Spark map/reduce/join operations on in-memory and disk-cached data (average speed traversals).
+* <<giraphgraphcomputer,`GiraphGraphComputer`>>: Leverages Apache Giraph to execute TinkerPop3 OLAP computations.
+** The graph should fit within the total RAM of the Hadoop cluster (graph size restriction), though "out-of-core"
+processing is possible. Message passing is coordinated via ZooKeeper for the in-memory graph (speedy traversals).
+
+TIP: image:gremlin-sugar.png[width=50,float=left] For those wanting to use the <<sugar-plugin,SugarPlugin>> with
+their submitted traversal, do `:remote config useSugar true` as well as `:plugin use tinkerpop.sugar` at the start of
+the Gremlin Console session if it is not already activated.
+
+Note that `SparkGraphComputer` and `GiraphGraphComputer` are loaded via their respective plugins. Typically only
+one plugin or the other is loaded depending on the desired `GraphComputer` to use.
+
+[source,text]
+----
+$ bin/gremlin.sh
+
+ \,,,/
+ (o o)
+-----oOOo-(3)-oOOo-----
+plugin activated: tinkerpop.server
+plugin activated: tinkerpop.utilities
+plugin activated: tinkerpop.tinkergraph
+plugin activated: tinkerpop.hadoop
+gremlin> :install org.apache.tinkerpop giraph-gremlin x.y.z
+==>loaded: [org.apache.tinkerpop, giraph-gremlin, x.y.z] - restart the console to use [tinkerpop.giraph]
+gremlin> :install org.apache.tinkerpop spark-gremlin x.y.z
+==>loaded: [org.apache.tinkerpop, spark-gremlin, x.y.z] - restart the console to use [tinkerpop.spark]
+gremlin> :q
+$ bin/gremlin.sh
+
+ \,,,/
+ (o o)
+-----oOOo-(3)-oOOo-----
+plugin activated: tinkerpop.server
+plugin activated: tinkerpop.utilities
+plugin activated: tinkerpop.tinkergraph
+plugin activated: tinkerpop.hadoop
+gremlin> :plugin use tinkerpop.giraph
+==>tinkerpop.giraph activated
+gremlin> :plugin use tinkerpop.spark
+==>tinkerpop.spark activated
+----
+
+WARNING: Hadoop, Spark, and Giraph all depend on many of the same libraries (e.g. ZooKeeper, Snappy, Netty, Guava,
+etc.). Unfortunately, typically these dependencies are not to the same versions of the respective libraries. As such,
+it is best to *not* have both Spark and Giraph plugins loaded in the same console session nor in the same Java
+project (though intelligent `<exclusion>`-usage can help alleviate conflicts in a Java project).
+
+[[mapreducegraphcomputer]]
+MapReduceGraphComputer
+^^^^^^^^^^^^^^^^^^^^^^
+
+*COMING SOON*
+
+[[sparkgraphcomputer]]
+SparkGraphComputer
+^^^^^^^^^^^^^^^^^^
+
+[source,xml]
+----
+<dependency>
+ <groupId>org.apache.tinkerpop</groupId>
+ <artifactId>spark-gremlin</artifactId>
+ <version>x.y.z</version>
+</dependency>
+----
+
+image:spark-logo.png[width=175,float=left] link:http://spark.apache.org[Spark] is an Apache Software Foundation
+project focused on general-purpose OLAP data processing. Spark provides a hybrid in-memory/disk-based distributed
+computing model that is similar to Hadoop's MapReduce model. Spark maintains a fluent function chaining DSL that is
+arguably easier for developers to work with than native Hadoop MapReduce. Spark-Gremlin provides an implementation of
+the bulk-synchronous parallel, distributed message passing algorithm within Spark and thus, any `VertexProgram` can be
+executed over `SparkGraphComputer`.
+
+[gremlin-groovy]
+----
+graph = GraphFactory.open('conf/hadoop/hadoop-gryo.properties')
+g = graph.traversal(computer(SparkGraphComputer))
+g.V().count()
+g.V().out().out().values('name')
+----
+
+For using lambdas in Gremlin-Groovy, simply provide `:remote connect` a `TraversalSource` which leverages SparkGraphComputer.
+
+[gremlin-groovy]
+----
+graph = GraphFactory.open('conf/hadoop/hadoop-gryo.properties')
+g = graph.traversal(computer(SparkGraphComputer))
+:remote connect tinkerpop.hadoop graph g
+:> g.V().group().by{it.value('name')[1]}.by('name')
+----
+
+The `SparkGraphComputer` algorithm leverages Spark's caching abilities to reduce the amount of data shuffled across
+the wire on each iteration of the <<vertexprogram,`VertexProgram`>>. When the graph is loaded as a Spark RDD
+(Resilient Distributed Dataset) it is immediately cached as `graphRDD`. The `graphRDD` is a distributed adjacency
+list which encodes the vertex, its properties, and all its incident edges. On the first iteration, each vertex
+(in parallel) is passed through `VertexProgram.execute()`. This yields an output of the vertex's mutated state
+(i.e. updated compute keys -- `propertyX`) and its outgoing messages. This `viewOutgoingRDD` is then reduced to
+`viewIncomingRDD` where the outgoing messages are sent to their respective vertices. If a `MessageCombiner` exists
+for the vertex program, then messages are aggregated locally and globally to ultimately yield one incoming message
+for the vertex. This reduce sequence is the "message pass." If the vertex program does not terminate on this
+iteration, then the `viewIncomingRDD` is joined with the cached `graphRDD` and the process continues. When there
+are no more iterations, there is a final join and the resultant RDD is stripped of its edges and messages. This
+`mapReduceRDD` is cached and is processed by each <<mapreduce,`MapReduce`>> job in the
+<<graphcomputer,`GraphComputer`>> computation.
+
+image::spark-algorithm.png[width=775]
+
+[width="100%",cols="2,10",options="header"]
+|========================================================
+|Property |Description
+|gremlin.spark.graphInputRDD |A class for creating RDD's from underlying graph data, defaults to Hadoop `InputFormat`.
+|gremlin.spark.graphOutputRDD |A class for output RDD's, defaults to Hadoop `OutputFormat`.
+|gremlin.spark.persistContext |Whether to create a new `SparkContext` for every `SparkGraphComputer` or to reuse an existing one.
+|========================================================
+
+If the provider/user wishes to not use Hadoop `InputFormats`, it is possible to leverage Spark's RDD
+constructs directly. There is a `gremlin.spark.graphInputRDD` configuration that references a `Class<? extends
+InputRDD>`. An `InputRDD` provides a read method that takes a `SparkContext` and returns a graphRDD. Likewise, use
+`gremlin.spark.graphOutputRDD` and the respective `OutputRDD`.
+
+It is possible to persist the graph RDD between jobs within the `SparkContext` (e.g. SparkServer) by leveraging `PersistedOutputRDD`.
+Note that `gremlin.spark.persistContext` should be set to `true` or else the persisted RDD will be destroyed when the `SparkContext` closes.
+The persisted RDD is named by the `gremlin.hadoop.outputLocation` configuration (i.e. named in `SparkContext.getPersistedRDDs()`).
+Finally, `PersistedInputRDD` is used with respective `gremlin.hadoop.inputLocation` to retrieve the persisted RDD from the `SparkContext`.
+
+When using a persistent `Spark Context` the configuration used by the original Spark Configuration will be inherited by all threaded
+references to that Spark Context. The exception to this rule are those properties which have a specific thread local effect.
+
+.Thread Local Properties
+. spark.jobGroup.id
+. spark.job.description
+. spark.job.interruptOnCancel
+. spark.scheduler.pool
+
+Loading with BulkLoaderVertexProgram
+++++++++++++++++++++++++++++++++++++
+
+The <<bulkloadervertexprogram, BulkLoaderVertexProgram>> is a generalized bulk loader that can be used to load large
+amounts of data to and from different `Graph` implementations. The following code demonstrates how to load the
+Grateful Dead graph from HadoopGraph into TinkerGraph over Spark:
+
+[gremlin-groovy]
+----
+hdfs.copyFromLocal('data/grateful-dead.kryo', 'data/grateful-dead.kryo')
+readGraph = GraphFactory.open('conf/hadoop/hadoop-grateful-gryo.properties')
+writeGraph = 'conf/tinkergraph-gryo.properties'
+blvp = BulkLoaderVertexProgram.build().
+ keepOriginalIds(false).
+ writeGraph(writeGraph).create(readGraph)
+readGraph.compute(SparkGraphComputer).workers(1).program(blvp).submit().get()
+:set max-iteration 10
+graph = GraphFactory.open(writeGraph)
+g = graph.traversal()
+g.V().valueMap()
+graph.close()
+----
+
+[source,properties]
+----
+# hadoop-grateful-gryo.properties
+
+#
+# Hadoop Graph Configuration
+#
+gremlin.graph=org.apache.tinkerpop.gremlin.hadoop.structure.HadoopGraph
+gremlin.hadoop.graphInputFormat=org.apache.tinkerpop.gremlin.hadoop.structure.io.gryo.GryoInputFormat
+gremlin.hadoop.memoryOutputFormat=org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat
+gremlin.hadoop.inputLocation=data/grateful-dead.kryo
+gremlin.hadoop.outputLocation=output
+gremlin.hadoop.deriveMemory=false
+gremlin.hadoop.jarsInDistributedCache=true
+
+#
+# SparkGraphComputer Configuration
+#
+spark.master=local[1]
+spark.executor.memory=1g
+spark.serializer=org.apache.tinkerpop.gremlin.spark.structure.io.gryo.GryoSerializer
+----
+
+[source,properties]
+----
+# tinkergraph-gryo.properties
+
+gremlin.graph=org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerGraph
+gremlin.tinkergraph.graphFormat=gryo
+gremlin.tinkergraph.graphLocation=/tmp/tinkergraph.kryo
+----
+
+IMPORTANT: The path to TinkerGraph jars needs to be included in the `HADOOP_GREMLIN_LIBS` for the above example to work.
+
+[[giraphgraphcomputer]]
+GiraphGraphComputer
+^^^^^^^^^^^^^^^^^^^
+
+[source,xml]
+----
+<dependency>
+ <groupId>org.apache.tinkerpop</groupId>
+ <artifactId>giraph-gremlin</artifactId>
+ <version>x.y.z</version>
+</dependency>
+----
+
+image:giraph-logo.png[width=100,float=left] link:http://giraph.apache.org[Giraph] is an Apache Software Foundation
+project focused on OLAP-based graph processing. Giraph makes use of the distributed graph computing paradigm made
+popular by Google's Pregel. In Giraph, developers write "vertex programs" that get executed at each vertex in
+parallel. These programs communicate with one another in a bulk synchronous parallel (BSP) manner. This model aligns
+with TinkerPop3's `GraphComputer` API. TinkerPop3 provides an implementation of `GraphComputer` that works for Giraph
+called `GiraphGraphComputer`. Moreover, with TinkerPop3's <<mapreduce,MapReduce>>-framework, the standard
+Giraph/Pregel model is extended to support an arbitrary number of MapReduce phases to aggregate and yield results
+from the graph. Below are examples using `GiraphGraphComputer` from the <<gremlin-console,Gremlin-Console>>.
+
+WARNING: Giraph uses a large number of Hadoop counters. The default for Hadoop is 120. In `mapred-site.xml` it is
+possible to increase the limit it via the `mapreduce.job.counters.max` property. A good value to use is 1000. This
+is a cluster-wide property so be sure to restart the cluster after updating.
+
+WARNING: The maximum number of workers can be no larger than the number of map-slots in the Hadoop cluster minus 1.
+For example, if the Hadoop cluster has 4 map slots, then `giraph.maxWorkers` can not be larger than 3. One map-slot
+is reserved for the master compute node and all other slots can be allocated as workers to execute the VertexPrograms
+on the vertices of the graph.
+
+If `GiraphGraphComputer` will be used as the `GraphComputer` for `HadoopGraph` then its `lib` directory should be
+specified in `HADOOP_GREMLIN_LIBS`.
+
+[source,shell]
+export HADOOP_GREMLIN_LIBS=$HADOOP_GREMLIN_LIBS:/usr/local/gremlin-console/ext/giraph-gremlin/lib
+
+Or, the user can specify the directory in the Gremlin Console.
+
+[source,groovy]
+System.setProperty('HADOOP_GREMLIN_LIBS',System.getProperty('HADOOP_GREMLIN_LIBS') + ':' + '/usr/local/gremlin-console/ext/giraph-gremlin/lib')
+
+[gremlin-groovy]
+----
+graph = GraphFactory.open('conf/hadoop/hadoop-gryo.properties')
+g = graph.traversal(computer(GiraphGraphComputer))
+g.V().count()
+g.V().out().out().values('name')
+----
+
+IMPORTANT: The examples above do not use lambdas (i.e. closures in Gremlin-Groovy). This makes the traversal
+serializable and thus, able to be distributed to all machines in the Hadoop cluster. If a lambda is required in a
+traversal, then the traversal must be sent as a `String` and compiled locally at each machine in the cluster. The
+following example demonstrates the `:remote` command which allows for submitting Gremlin traversals as a `String`.
+
+[gremlin-groovy]
+----
+graph = GraphFactory.open('conf/hadoop/hadoop-gryo.properties')
+g = graph.traversal(computer(GiraphGraphComputer))
+:remote connect tinkerpop.hadoop graph g
+:> g.V().group().by{it.value('name')[1]}.by('name')
+result
+result.memory.runtime
+result.memory.keys()
+result.memory.get('~reducing')
+----
+
+NOTE: If the user explicitly specifies `giraph.maxWorkers` and/or `giraph.numComputeThreads` in the configuration,
+then these values will be used by Giraph. However, if these are not specified and the user never calls
+`GraphComputer.workers()` then `GiraphGraphComputer` will try to compute the number of workers/threads to use based
+on the cluster's profile.
+
+Loading with BulkLoaderVertexProgram
+++++++++++++++++++++++++++++++++++++
+
+The <<bulkloadervertexprogram, BulkLoaderVertexProgram>> is a generalized bulk loader that can be used to load
+large amounts of data to and from different `Graph` implementations. The following code demonstrates how to load
+the Grateful Dead graph from HadoopGraph into TinkerGraph over Giraph:
+
+[gremlin-groovy]
+----
+hdfs.copyFromLocal('data/grateful-dead.kryo', 'data/grateful-dead.kryo')
+readGraph = GraphFactory.open('conf/hadoop/hadoop-grateful-gryo.properties')
+writeGraph = 'conf/tinkergraph-gryo.properties'
+blvp = BulkLoaderVertexProgram.build().
+ keepOriginalIds(false).
+ writeGraph(writeGraph).create(readGraph)
+readGraph.compute(GiraphGraphComputer).workers(1).program(blvp).submit().get()
+:set max-iteration 10
+graph = GraphFactory.open(writeGraph)
+g = graph.traversal()
+g.V().valueMap()
+graph.close()
+----
+
+[source,properties]
+----
+# hadoop-grateful-gryo.properties
+
+#
+# Hadoop Graph Configuration
+#
+gremlin.graph=org.apache.tinkerpop.gremlin.hadoop.structure.HadoopGraph
+gremlin.hadoop.graphInputFormat=org.apache.tinkerpop.gremlin.hadoop.structure.io.gryo.GryoInputFormat
+gremlin.hadoop.graphOutputFormat=org.apache.hadoop.mapreduce.lib.output.NullOutputFormat
+gremlin.hadoop.memoryOutputFormat=org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat
+gremlin.hadoop.inputLocation=data/grateful-dead.kryo
+gremlin.hadoop.outputLocation=output
+gremlin.hadoop.deriveMemory=false
+gremlin.hadoop.jarsInDistributedCache=true
+
+#
+# GiraphGraphComputer Configuration
+#
+giraph.minWorkers=1
+giraph.maxWorkers=1
+giraph.useOutOfCoreGraph=true
+giraph.useOutOfCoreMessages=true
+mapred.map.child.java.opts=-Xmx1024m
+mapred.reduce.child.java.opts=-Xmx1024m
+giraph.numInputThreads=4
+giraph.numComputeThreads=4
+giraph.maxMessagesInMemory=100000
+----
+
+[source,properties]
+----
+# tinkergraph-gryo.properties
+
+gremlin.graph=org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerGraph
+gremlin.tinkergraph.graphFormat=gryo
+gremlin.tinkergraph.graphLocation=/tmp/tinkergraph.kryo
+----
+
+NOTE: The path to TinkerGraph needs to be included in the `HADOOP_GREMLIN_LIBS` for the above example to work.
+
+Input/Output Formats
+~~~~~~~~~~~~~~~~~~~~
+
+image:adjacency-list.png[width=300,float=right] Hadoop-Gremlin provides various I/O formats -- i.e. Hadoop
+`InputFormat` and `OutputFormat`. All of the formats make use of an link:http://en.wikipedia.org/wiki/Adjacency_list[adjacency list]
+representation of the graph where each "row" represents a single vertex, its properties, and its incoming and
+outgoing edges.
+
+{empty} +
+
+[[gryo-io-format]]
+Gryo I/O Format
+^^^^^^^^^^^^^^^
+
+* **InputFormat**: `org.apache.tinkerpop.gremlin.hadoop.structure.io.gryo.GryoInputFormat`
+* **OutputFormat**: `org.apache.tinkerpop.gremlin.hadoop.structure.io.gryo.GryoOutputFormat`
+
+<<gryo-reader-writer,Gryo>> is a binary graph format that leverages link:https://github.com/EsotericSoftware/kryo[Kryo]
+to make a compact, binary representation of a vertex. It is recommended that users leverage Gryo given its space/time
+savings over text-based representations.
+
+NOTE: The `GryoInputFormat` is splittable.
+
+[[graphson-io-format]]
+GraphSON I/O Format
+^^^^^^^^^^^^^^^^^^^
+
+* **InputFormat**: `org.apache.tinkerpop.gremlin.hadoop.structure.io.graphson.GraphSONInputFormat`
+* **OutputFormat**: `org.apache.tinkerpop.gremlin.hadoop.structure.io.graphson.GraphSONOutputFormat`
+
+<<graphson-reader-writer,GraphSON>> is a JSON based graph format. GraphSON is a space-expensive graph format in that
+it is a text-based markup language. However, it is convenient for many developers to work with as its structure is
+simple (easy to create and parse).
+
+The data below represents an adjacency list representation of the classic TinkerGraph toy graph in GraphSON format.
+
+[source,json]
+{"id":1,"label":"person","outE":{"created":[{"id":9,"inV":3,"properties":{"weight":0.4}}],"knows":[{"id":7,"inV":2,"properties":{"weight":0.5}},{"id":8,"inV":4,"properties":{"weight":1.0}}]},"properties":{"name":[{"id":0,"value":"marko"}],"age":[{"id":1,"value":29}]}}
+{"id":2,"label":"person","inE":{"knows":[{"id":7,"outV":1,"properties":{"weight":0.5}}]},"properties":{"name":[{"id":2,"value":"vadas"}],"age":[{"id":3,"value":27}]}}
+{"id":3,"label":"software","inE":{"created":[{"id":9,"outV":1,"properties":{"weight":0.4}},{"id":11,"outV":4,"properties":{"weight":0.4}},{"id":12,"outV":6,"properties":{"weight":0.2}}]},"properties":{"name":[{"id":4,"value":"lop"}],"lang":[{"id":5,"value":"java"}]}}
+{"id":4,"label":"person","inE":{"knows":[{"id":8,"outV":1,"properties":{"weight":1.0}}]},"outE":{"created":[{"id":10,"inV":5,"properties":{"weight":1.0}},{"id":11,"inV":3,"properties":{"weight":0.4}}]},"properties":{"name":[{"id":6,"value":"josh"}],"age":[{"id":7,"value":32}]}}
+{"id":5,"label":"software","inE":{"created":[{"id":10,"outV":4,"properties":{"weight":1.0}}]},"properties":{"name":[{"id":8,"value":"ripple"}],"lang":[{"id":9,"value":"java"}]}}
+{"id":6,"label":"person","outE":{"created":[{"id":12,"inV":3,"properties":{"weight":0.2}}]},"properties":{"name":[{"id":10,"value":"peter"}],"age":[{"id":11,"value":35}]}}
+
+[[script-io-format]]
+Script I/O Format
+^^^^^^^^^^^^^^^^^
+
+* **InputFormat**: `org.apache.tinkerpop.gremlin.hadoop.structure.io.script.ScriptInputFormat`
+* **OutputFormat**: `org.apache.tinkerpop.gremlin.hadoop.structure.io.script.ScriptOutputFormat`
+
+`ScriptInputFormat` and `ScriptOutputFormat` take an arbitrary script and use that script to either read or write
+`Vertex` objects, respectively. This can be considered the most general `InputFormat`/`OutputFormat` possible in that
+Hadoop-Gremlin uses the user provided script for all reading/writing.
+
+ScriptInputFormat
++++++++++++++++++
+
+The data below represents an adjacency list representation of the classic TinkerGraph toy graph. First line reads,
+"vertex `1`, labeled `person` having 2 property values (`marko` and `29`) has 3 outgoing edges; the first edge is
+labeled `knows`, connects the current vertex `1` with vertex `2` and has a property value `0.4`, and so on."
+
+[source]
+1:person:marko:29 knows:2:0.5,knows:4:1.0,created:3:0.4
+2:person:vadas:27
+3:project:lop:java
+4:person:josh:32 created:3:0.4,created:5:1.0
+5:project:ripple:java
+6:person:peter:35 created:3:0.2
+
+There is no corresponding `InputFormat` that can parse this particular file (or some adjacency list variant of it).
+As such, `ScriptInputFormat` can be used. With `ScriptInputFormat` a script is stored in HDFS and leveraged by each
+mapper in the Hadoop job. The script must have the following method defined:
+
+[source,groovy]
+def parse(String line, ScriptElementFactory factory) { ... }
+
+`ScriptElementFactory` provides the following 4 methods:
+
+[source,java]
+Vertex vertex(Object id); // get or create the vertex with the given id
+Vertex vertex(Object id, String label); // get or create the vertex with the given id and label
+Edge edge(Vertex out, Vertex in); // create an edge between the two given vertices
+Edge edge(Vertex out, Vertex in, String label); // create an edge between the two given vertices using the given label
+
+An appropriate `parse()` for the above adjacency list file is:
+
+[source,groovy]
+def parse(line, factory) {
+ def parts = line.split(/ /)
+ def (id, label, name, x) = parts[0].split(/:/).toList()
+ def v1 = factory.vertex(id, label)
+ if (name != null) v1.property('name', name) // first value is always the name
+ if (x != null) {
+ // second value depends on the vertex label; it's either
+ // the age of a person or the language of a project
+ if (label.equals('project')) v1.property('lang', x)
+ else v1.property('age', Integer.valueOf(x))
+ }
+ if (parts.length == 2) {
+ parts[1].split(/,/).grep { !it.isEmpty() }.each {
+ def (eLabel, refId, weight) = it.split(/:/).toList()
+ def v2 = factory.vertex(refId)
+ def edge = factory.edge(v1, v2, eLabel)
+ edge.property('weight', Double.valueOf(weight))
+ }
+ }
+ return v1
+}
+
+The resultant `Vertex` denotes whether the line parsed yielded a valid Vertex. As such, if the line is not valid
+(e.g. a comment line, a skip line, etc.), then simply return `null`.
+
+ScriptOutputFormat Support
+++++++++++++++++++++++++++
+
+The principle above can also be used to convert a vertex to an arbitrary `String` representation that is ultimately
+streamed back to a file in HDFS. This is the role of `ScriptOutputFormat`. `ScriptOutputFormat` requires that the
+provided script maintains a method with the following signature:
+
+[source,groovy]
+def stringify(Vertex vertex) { ... }
+
+An appropriate `stringify()` to produce output in the same format that was shown in the `ScriptInputFormat` sample is:
+
+[source,groovy]
+def stringify(vertex) {
+ def v = vertex.values('name', 'age', 'lang').inject(vertex.id(), vertex.label()).join(':')
+ def outE = vertex.outE().map {
+ def e = it.get()
+ e.values('weight').inject(e.label(), e.inV().next().id()).join(':')
+ }.join(',')
+ return [v, outE].join('\t')
+}
+
+Interacting with HDFS
+~~~~~~~~~~~~~~~~~~~~~
+
+The distributed file system of Hadoop is called link:http://en.wikipedia.org/wiki/Apache_Hadoop#Hadoop_distributed_file_system[HDFS].
+The results of any OLAP operation are stored in HDFS accessible via `hdfs`.
+
+[gremlin-groovy]
+----
+graph = GraphFactory.open('conf/hadoop/hadoop-gryo.properties')
+g = graph.traversal(computer(SparkGraphComputer))
+:remote connect tinkerpop.hadoop graph g
+:> g.V().group().by{it.value('name')[1]}.by('name')
+hdfs.ls()
+hdfs.ls('output')
+hdfs.ls('output/~reducing')
+hdfs.head('output/~reducing', ObjectWritable)
+----
+
+A list of the HDFS methods available are itemized below. Note that these methods are also available for the 'local' variable:
+
+[width="100%",cols="13,10",options="header"]
+|=========================================================
+| Method| Description
+|hdfs.ls(String path)| List the contents of the supplied directory.
+|hdfs.cp(String from, String to)| Copy the specified path to the specified path.
+|hdfs.exists(String path)| Whether the specified path exists.
+|hdfs.rm(String path)| Remove the specified path.
+|hdfs.rmr(String path)| Remove the specified path and its contents recurssively.
+|hdfs.copyToLocal(String from, String to)| Copy the specified HDFS path to the specified local path.
+|hdfs.copyFromLocal(String from, String to)| Copy the specified local path to the specified HDFS path.
+|hdfs.mergeToLocal(String from, String to)| Merge the files in path to the specified local path.
+|hdfs.head(String path)| Display the data in the path as text.
+|hdfs.head(String path, int lineCount)| Text display only the first `lineCount`-number of lines in the path.
+|hdfs.head(String path, int totalKeyValues, Class<Writable> writableClass)| Display the path interpreting the key values as respective writable.
+|=========================================================
+
+A Command Line Example
+~~~~~~~~~~~~~~~~~~~~~~
+
+image::pagerank-logo.png[width=300]
+
+The classic link:http://en.wikipedia.org/wiki/PageRank[PageRank] centrality algorithm can be executed over the
+TinkerPop graph from the command line using `GiraphGraphComputer`.
+
+WARNING: Be sure that the `HADOOP_GREMLIN_LIBS` references the location `lib` directory of the respective
+`GraphComputer` engine being used or else the requisite dependencies will not be uploaded to the Hadoop cluster.
+
+[source,text]
+----
+$ hdfs dfs -copyFromLocal data/tinkerpop-modern.json tinkerpop-modern.json
+$ hdfs dfs -ls
+Found 2 items
+-rw-r--r-- 1 marko supergroup 2356 2014-07-28 13:00 /user/marko/tinkerpop-modern.json
+$ hadoop jar target/giraph-gremlin-x.y.z-job.jar org.apache.tinkerpop.gremlin.giraph.process.computer.GiraphGraphComputer ../hadoop-gremlin/conf/hadoop-graphson.properties
+15/09/11 08:02:08 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
+15/09/11 08:02:11 INFO computer.GiraphGraphComputer: HadoopGremlin(Giraph): PageRankVertexProgram[alpha=0.85,iterations=30]
+15/09/11 08:02:12 INFO mapreduce.JobSubmitter: number of splits:3
+15/09/11 08:02:12 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1441915907347_0028
+15/09/11 08:02:12 INFO impl.YarnClientImpl: Submitted application application_1441915907347_0028
+15/09/11 08:02:12 INFO job.GiraphJob: Tracking URL: http://markos-macbook:8088/proxy/application_1441915907347_0028/
+15/09/11 08:02:12 INFO job.GiraphJob: Waiting for resources... Job will start only when it gets all 3 mappers
+15/09/11 08:03:54 INFO mapreduce.Job: Running job: job_1441915907347_0028
+15/09/11 08:03:55 INFO mapreduce.Job: Job job_1441915907347_0028 running in uber mode : false
+15/09/11 08:03:55 INFO mapreduce.Job: map 33% reduce 0%
+15/09/11 08:03:57 INFO mapreduce.Job: map 67% reduce 0%
+15/09/11 08:04:01 INFO mapreduce.Job: map 100% reduce 0%
+15/09/11 08:06:17 INFO mapreduce.Job: Job job_1441915907347_0028 completed successfully
+15/09/11 08:06:17 INFO mapreduce.Job: Counters: 80
+ File System Counters
+ FILE: Number of bytes read=0
+ FILE: Number of bytes written=483918
+ FILE: Number of read operations=0
+ FILE: Number of large read operations=0
+ FILE: Number of write operations=0
+ HDFS: Number of bytes read=1465
+ HDFS: Number of bytes written=1760
+ HDFS: Number of read operations=39
+ HDFS: Number of large read operations=0
+ HDFS: Number of write operations=20
+ Job Counters
+ Launched map tasks=3
+ Other local map tasks=3
+ Total time spent by all maps in occupied slots (ms)=458105
+ Total time spent by all reduces in occupied slots (ms)=0
+ Total time spent by all map tasks (ms)=458105
+ Total vcore-seconds taken by all map tasks=458105
+ Total megabyte-seconds taken by all map tasks=469099520
+ Map-Reduce Framework
+ Map input records=3
+ Map output records=0
+ Input split bytes=132
+ Spilled Records=0
+ Failed Shuffles=0
+ Merged Map outputs=0
+ GC time elapsed (ms)=1594
+ CPU time spent (ms)=0
+ Physical memory (bytes) snapshot=0
+ Virtual memory (bytes) snapshot=0
+ Total committed heap usage (bytes)=527958016
+ Giraph Stats
+ Aggregate edges=0
+ Aggregate finished vertices=0
+ Aggregate sent message message bytes=13535
+ Aggregate sent messages=186
+ Aggregate vertices=6
+ Current master task partition=0
+ Current workers=2
+ Last checkpointed superstep=0
+ Sent message bytes=438
+ Sent messages=6
+ Superstep=31
+ Giraph Timers
+ Initialize (ms)=2996
+ Input superstep (ms)=5209
+ Setup (ms)=59
+ Shutdown (ms)=9324
+ Superstep 0 GiraphComputation (ms)=3861
+ Superstep 1 GiraphComputation (ms)=4027
+ Superstep 10 GiraphComputation (ms)=4000
+ Superstep 11 GiraphComputation (ms)=4004
+ Superstep 12 GiraphComputation (ms)=3999
+ Superstep 13 GiraphComputation (ms)=4000
+ Superstep 14 GiraphComputation (ms)=4005
+ Superstep 15 GiraphComputation (ms)=4003
+ Superstep 16 GiraphComputation (ms)=4001
+ Superstep 17 GiraphComputation (ms)=4007
+ Superstep 18 GiraphComputation (ms)=3998
+ Superstep 19 GiraphComputation (ms)=4006
+ Superstep 2 GiraphComputation (ms)=4007
+ Superstep 20 GiraphComputation (ms)=3996
+ Superstep 21 GiraphComputation (ms)=4006
+ Superstep 22 GiraphComputation (ms)=4002
+ Superstep 23 GiraphComputation (ms)=3998
+ Superstep 24 GiraphComputation (ms)=4003
+ Superstep 25 GiraphComputation (ms)=4001
+ Superstep 26 GiraphComputation (ms)=4003
+ Superstep 27 GiraphComputation (ms)=4005
+ Superstep 28 GiraphComputation (ms)=4002
+ Superstep 29 GiraphComputation (ms)=4001
+ Superstep 3 GiraphComputation (ms)=3988
+ Superstep 30 GiraphComputation (ms)=4248
+ Superstep 4 GiraphComputation (ms)=4010
+ Superstep 5 GiraphComputation (ms)=3998
+ Superstep 6 GiraphComputation (ms)=3996
+ Superstep 7 GiraphComputation (ms)=4005
+ Superstep 8 GiraphComputation (ms)=4009
+ Superstep 9 GiraphComputation (ms)=3994
+ Total (ms)=138788
+ File Input Format Counters
+ Bytes Read=0
+ File Output Format Counters
+ Bytes Written=0
+$ hdfs dfs -cat output/~g/*
+{"id":1,"label":"person","properties":{"gremlin.pageRankVertexProgram.pageRank":[{"id":39,"value":0.15000000000000002}],"name":[{"id":0,"value":"marko"}],"gremlin.pageRankVertexProgram.edgeCount":[{"id":10,"value":3.0}],"age":[{"id":1,"value":29}]}}
+{"id":5,"label":"software","properties":{"gremlin.pageRankVertexProgram.pageRank":[{"id":35,"value":0.23181250000000003}],"name":[{"id":8,"value":"ripple"}],"gremlin.pageRankVertexProgram.edgeCount":[{"id":6,"value":0.0}],"lang":[{"id":9,"value":"java"}]}}
+{"id":3,"label":"software","properties":{"gremlin.pageRankVertexProgram.pageRank":[{"id":39,"value":0.4018125}],"name":[{"id":4,"value":"lop"}],"gremlin.pageRankVertexProgram.edgeCount":[{"id":10,"value":0.0}],"lang":[{"id":5,"value":"java"}]}}
+{"id":4,"label":"person","properties":{"gremlin.pageRankVertexProgram.pageRank":[{"id":39,"value":0.19250000000000003}],"name":[{"id":6,"value":"josh"}],"gremlin.pageRankVertexProgram.edgeCount":[{"id":10,"value":2.0}],"age":[{"id":7,"value":32}]}}
+{"id":2,"label":"person","properties":{"gremlin.pageRankVertexProgram.pageRank":[{"id":35,"value":0.19250000000000003}],"name":[{"id":2,"value":"vadas"}],"gremlin.pageRankVertexProgram.edgeCount":[{"id":6,"value":0.0}],"age":[{"id":3,"value":27}]}}
+{
<TRUNCATED>
[07/11] incubator-tinkerpop git commit: Merge branch 'master' of
https://github.com/kushal256/incubator-tinkerpop into TINKERPOP3-886
Posted by sp...@apache.org.
Merge branch 'master' of https://github.com/kushal256/incubator-tinkerpop into TINKERPOP3-886
Project: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/commit/be12d3e5
Tree: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/tree/be12d3e5
Diff: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/diff/be12d3e5
Branch: refs/heads/master
Commit: be12d3e592ca0da83f300fc6b94d735cb1bd07ee
Parents: 321044d 450a70c
Author: Stephen Mallette <sp...@genoprime.com>
Authored: Mon Nov 23 17:26:02 2015 -0500
Committer: Stephen Mallette <sp...@genoprime.com>
Committed: Mon Nov 23 17:26:02 2015 -0500
----------------------------------------------------------------------
CHANGELOG.asciidoc | 1 +
docs/src/tutorials-getting-started.asciidoc | 7 ++++++
.../tinkerpop/gremlin/structure/io/IoCore.java | 6 ++---
gremlin-server/scripts/generate-modern.groovy | 2 +-
hadoop-gremlin/pom.xml | 24 ++++++++------------
.../tinkergraph/structure/TinkerGraphTest.java | 6 ++++-
6 files changed, 27 insertions(+), 19 deletions(-)
----------------------------------------------------------------------
[04/11] incubator-tinkerpop git commit: Merge branch 'master' of
https://github.com/kushal256/incubator-tinkerpop into TINKERPOP3-886
Posted by sp...@apache.org.
Merge branch 'master' of https://github.com/kushal256/incubator-tinkerpop into TINKERPOP3-886
Project: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/commit/321044dd
Tree: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/tree/321044dd
Diff: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/diff/321044dd
Branch: refs/heads/master
Commit: 321044dd20b8830d8d4d2aaca2f45f5b4845e2ff
Parents: d3b927a c6626da
Author: Stephen Mallette <sp...@genoprime.com>
Authored: Thu Nov 19 07:39:32 2015 -0500
Committer: Stephen Mallette <sp...@genoprime.com>
Committed: Thu Nov 19 07:39:32 2015 -0500
----------------------------------------------------------------------
docs/src/implementations.asciidoc | 4 +-
.../tinkerpop/gremlin/structure/io/IoCore.java | 6 ++
.../tinkergraph/structure/TinkerGraph.java | 14 ++--
.../tinkergraph/structure/TinkerGraphTest.java | 69 +++++++++++++++++++-
4 files changed, 85 insertions(+), 8 deletions(-)
----------------------------------------------------------------------