You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tinkerpop.apache.org by sp...@apache.org on 2015/11/20 12:55:29 UTC
[05/15] incubator-tinkerpop git commit: Made subdirectories for
various "books" in the docs.
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/bc46d649/docs/src/the-graph.asciidoc
----------------------------------------------------------------------
diff --git a/docs/src/the-graph.asciidoc b/docs/src/the-graph.asciidoc
deleted file mode 100644
index eceec21..0000000
--- a/docs/src/the-graph.asciidoc
+++ /dev/null
@@ -1,771 +0,0 @@
-////
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements. See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-////
-[[graph]]
-The Graph
-=========
-
-image::gremlin-standing.png[width=125]
-
-Features
---------
-
-A `Feature` implementation describes the capabilities of a `Graph` instance. This interface is implemented by graph
-system providers for two purposes:
-
-. It tells users the capabilities of their `Graph` instance.
-. It allows the features they do comply with to be tested against the Gremlin Test Suite - tests that do not comply are "ignored").
-
-The following example in the Gremlin Console shows how to print all the features of a `Graph`:
-
-[gremlin-groovy]
-----
-graph = TinkerGraph.open()
-graph.features()
-----
-
-A common pattern for using features is to check their support prior to performing an operation:
-
-[gremlin-groovy]
-----
-graph.features().graph().supportsTransactions()
-graph.features().graph().supportsTransactions() ? g.tx().commit() : "no tx"
-----
-
-TIP: To ensure provider agnostic code, always check feature support prior to usage of a particular function. In that
-way, the application can behave gracefully in case a particular implementation is provided at runtime that does not
-support a function being accessed.
-
-WARNING: Assignments of a `GraphStrategy` can alter the base features of a `Graph` in dynamic ways, such that checks
-against a `Feature` may not always reflect the behavior exhibited when the `GraphStrategy` is in use.
-
-[[vertex-properties]]
-Vertex Properties
------------------
-
-image:vertex-properties.png[width=215,float=left] TinkerPop3 introduces the concept of a `VertexProperty<V>`. All the
-properties of a `Vertex` are a `VertexProperty`. A `VertexProperty` implements `Property` and as such, it has a
-key/value pair. However, `VertexProperty` also implements `Element` and thus, can have a collection of key/value
-pairs. Moreover, while an `Edge` can only have one property of key "name" (for example), a `Vertex` can have multiple
-"name" properties. With the inclusion of vertex properties, two features are introduced which ultimately advance the
-graph modelers toolkit:
-
-. Multiple properties (*multi-properties*): a vertex property key can have multiple values (i.e. a vertex can have
-multiple "name" properties).
-. Properties on properties (*meta-properties*): a vertex property can have properties (i.e. a vertex property can
-have key/value data associated with it).
-
-A collection of use cases are itemized below:
-
-. *Permissions*: Vertex properties can have key/value ACL-type permission information associated with them.
-. *Auditing*: When a vertex property is manipulated, it can have key/value information attached to it saying who the
-creator, deletor, etc. are.
-. *Provenance*: The "name" of a vertex can be declared by multiple users.
-
-A running example using vertex properties is provided below to demonstrate and explain the API.
-
-[gremlin-groovy]
-----
-graph = TinkerGraph.open()
-g = graph.traversal(standard())
-v = g.addV('name','marko','name','marko a. rodriguez').next()
-g.V(v).properties().count()
-g.V(v).properties('name').count() <1>
-g.V(v).properties()
-g.V(v).properties('name')
-g.V(v).properties('name').hasValue('marko')
-g.V(v).properties('name').hasValue('marko').property('acl','private') <2>
-g.V(v).properties('name').hasValue('marko a. rodriguez')
-g.V(v).properties('name').hasValue('marko a. rodriguez').property('acl','public')
-g.V(v).properties('name').has('acl','public').value()
-g.V(v).properties('name').has('acl','public').drop() <3>
-g.V(v).properties('name').has('acl','public').value()
-g.V(v).properties('name').has('acl','private').value()
-g.V(v).properties()
-g.V(v).properties().properties() <4>
-g.V(v).properties().property('date',2014) <5>
-g.V(v).properties().property('creator','stephen')
-g.V(v).properties().properties()
-g.V(v).properties('name').valueMap()
-g.V(v).property('name','okram') <6>
-g.V(v).properties('name')
-g.V(v).values('name') <7>
-----
-
-<1> A vertex can have zero or more properties with the same key associated with it.
-<2> A vertex property can have standard key/value properties attached to it.
-<3> Vertex property removal is identical to property removal.
-<4> It is property to get the properties of a vertex property.
-<5> A vertex property can have any number of key/value properties attached to it.
-<6> `property(...)` will remove all existing key'd properties before adding the new single property (see `VertexProperty.Cardinality`).
-<7> If only the value of a property is needed, then `values()` can be used.
-
-If the concept of vertex properties is difficult to grasp, then it may be best to think of vertex properties in terms
-of "literal vertices." A vertex can have an edge to a "literal vertex" that has a single value key/value -- e.g.
-"value=okram." The edge that points to that literal vertex has an edge-label of "name." The properties on the edge
-represent the literal vertex's properties. The "literal vertex" can not have any other edges to it (only one from the
-associated vertex).
-
-[[the-crew-toy-graph]]
-TIP: A toy graph demonstrating all of the new TinkerPop3 graph structure features is available at
-`TinkerFactory.createTheCrew()` and `data/tinkerpop-crew*`. This graph demonstrates multi-properties and meta-properties.
-
-.TinkerPop Crew
-image::the-crew-graph.png[width=685]
-
-[gremlin-groovy,theCrew]
-----
-g.V().as('a').
- properties('location').as('b').
- hasNot('endTime').as('c').
- select('a','b','c').by('name').by(value).by('startTime') // determine the current location of each person
-g.V().has('name','gremlin').inE('uses').
- order().by('skill',incr).as('a').
- outV().as('b').
- select('a','b').by('skill').by('name') // rank the users of gremlin by their skill level
-----
-
-Graph Variables
----------------
-
-TinkerPop3 introduces the concept of `Graph.Variables`. Variables are key/value pairs associated with the graph
-itself -- in essence, a `Map<String,Object>`. These variables are intended to store metadata about the graph. Example
-use cases include:
-
- * *Schema information*: What do the namespace prefixes resolve to and when was the schema last modified?
- * *Global permissions*: What are the access rights for particular groups?
- * *System user information*: Who are the admins of the system?
-
-An example of graph variables in use is presented below:
-
-[gremlin-groovy]
-----
-graph = TinkerGraph.open()
-graph.variables()
-graph.variables().set('systemAdmins',['stephen','peter','pavel'])
-graph.variables().set('systemUsers',['matthias','marko','josh'])
-graph.variables().keys()
-graph.variables().get('systemUsers')
-graph.variables().get('systemUsers').get()
-graph.variables().remove('systemAdmins')
-graph.variables().keys()
-----
-
-IMPORTANT: Graph variables are not intended to be subject to heavy, concurrent mutation nor to be used in complex
-computations. The intention is to have a location to store data about the graph for administrative purposes.
-
-[[transactions]]
-Graph Transactions
-------------------
-
-image:gremlin-coins.png[width=100,float=right] A link:http://en.wikipedia.org/wiki/Database_transaction[database transaction]
-represents a unit of work to execute against the database. Transactions are controlled by an implementation of the
-`Transaction` interface and that object can be obtained from the `Graph` interface using the `tx()` method. It is
-important to note that the `Transaction` object does not represent a "transaction" itself. It merely exposes the
-methods for working with transactions (e.g. committing, rolling back, etc).
-
-Most `Graph` implementations that `supportsTransactions` will implement an "automatic" `ThreadLocal` transaction,
-which means that when a read or write occurs after the `Graph` is instantiated a transaction is automatically
-started within that thread. There is no need to manually call a method to "create" or "start" a transaction. Simply
-modify the graph as required and call `graph.tx().commit()` to apply changes or `graph.tx().rollback()` to undo them.
-When the next read or write action occurs against the graph, a new transaction will be started within that current
-thread of execution.
-
-When using transactions in this fashion, especially in web application (e.g. REST server), it is important to ensure
-that transaction do not leak from one request to the next. In other words, unless a client is somehow bound via
-session to process every request on the same server thread, ever request must be committed or rolled back at the end
-of the request. By ensuring that the request encapsulates a transaction, it ensures that a future request processed
-on a server thread is starting in a fresh transactional state and will not have access to the remains of one from an
-earlier request. A good strategy is to rollback a transaction at the start of a request, so that if it so happens that
-a transactional leak does occur between requests somehow, a fresh transaction is assured by the fresh request.
-
-TIP: The `tx()` method is on the `Graph` interface, but it is also available on the `TraversalSource` spawned from a
-`Graph`. Calls to `TraversalSource.tx()` are proxied through to the underlying `Graph` as a convenience.
-
-Configuring
-~~~~~~~~~~~
-
-Determining when a transaction starts is dependent upon the behavior assigned to the `Transaction`. It is up to the
-`Graph` implementation to determine the default behavior and unless the implementation doesn't allow it, the behavior
-itself can be altered via these `Transaction` methods:
-
-[source,java]
-----
-public Transaction onReadWrite(final Consumer<Transaction> consumer);
-
-public Transaction onClose(final Consumer<Transaction> consumer);
-----
-
-Providing a `Consumer` function to `onReadWrite` allows definition of how a transaction starts when a read or a write
-occurs. `Transaction.READ_WRITE_BEHAVIOR` contains pre-defined `Consumer` functions to supply to the `onReadWrite`
-method. It has two options:
-
-* `AUTO` - automatic transactions where the transaction is started implicitly to the read or write operation
-* `MANUAL` - manual transactions where it is up to the user to explicitly open a transaction, throwing an exception
-if the transaction is not open
-
-Providing a `Consumer` function to `onClose` allows configuration of how a transaction is handled when
-`Transaction.close()` is called. `Transaction.CLOSE_BEHAVIOR` has several pre-defined options that can be supplied to
-this method:
-
-* `COMMIT` - automatically commit an open transaction
-* `ROLLBACK` - automatically rollback an open transaction
-* `MANUAL` - throw an exception if a transaction is open, forcing the user to explicitly close the transaction
-
-IMPORTANT: As transactions are `ThreadLocal` in nature, so are the transaction configurations for `onReadWrite` and
-`onClose`.
-
-Once there is an understanding for how transactions are configured, most of the rest of the `Transaction` interface
-is self-explanatory. Note that <<neo4j-gremlin,Neo4j-Gremlin>> is used for the examples to follow as TinkerGraph does
-not support transactions.
-
-[source,groovy]
-----
-gremlin> graph = Neo4jGraph.open('/tmp/neo4j')
-==>neo4jgraph[EmbeddedGraphDatabase [/tmp/neo4j]]
-gremlin> graph.features()
-==>FEATURES
-> GraphFeatures
->-- Transactions: true <1>
->-- Computer: false
->-- Persistence: true
-...
-gremlin> graph.tx().onReadWrite(Transaction.READ_WRITE_BEHAVIOR.AUTO) <2>
-==>org.apache.tinkerpop.gremlin.neo4j.structure.Neo4jGraph$Neo4jTransaction@1c067c0d
-gremlin> graph.addVertex("name","stephen") <3>
-==>v[0]
-gremlin> graph.tx().commit() <4>
-==>null
-gremlin> graph.tx().onReadWrite(Transaction.READ_WRITE_BEHAVIOR.MANUAL) <5>
-==>org.apache.tinkerpop.gremlin.neo4j.structure.Neo4jGraph$Neo4jTransaction@1c067c0d
-gremlin> graph.tx().isOpen()
-==>false
-gremlin> graph.addVertex("name","marko") <6>
-Open a transaction before attempting to read/write the transaction
-gremlin> graph.tx().open() <7>
-==>null
-gremlin> graph.addVertex("name","marko") <8>
-==>v[1]
-gremlin> graph.tx().commit()
-==>null
-----
-
-<1> Check `features` to ensure that the graph supports transactions.
-<2> By default, `Neo4jGraph` is configured with "automatic" transactions, so it is set here for demonstration purposes only.
-<3> When the vertex is added, the transaction is automatically started. From this point, more mutations can be staged
-or other read operations executed in the context of that open transaction.
-<4> Calling `commit` finalizes the transaction.
-<5> Change transaction behavior to require manual control.
-<6> Adding a vertex now results in failure because the transaction was not explicitly opened.
-<7> Explicitly open a transaction.
-<8> Adding a vertex now succeeds as the transaction was manually opened.
-
-NOTE: It may be important to consult the documentation of the `Graph` implementation when it comes to the specifics of
-how transactions will behave. TinkerPop allows some latitude in this area and implementations may not have the exact
-same behaviors and link:https://en.wikipedia.org/wiki/ACID[ACID] guarantees.
-
-Retries
-~~~~~~~
-
-There are times when transactions fail. Failure may be indicative of some permanent condition, but other failures
-might simply require the transaction to be retried for possible future success. The `Transaction` object also exposes
-a method for executing automatic transaction retries:
-
-[gremlin-groovy]
-----
-graph = Neo4jGraph.open('/tmp/neo4j')
-graph.tx().submit {it.addVertex("name","josh")}.retry(10)
-graph.tx().submit {it.addVertex("name","daniel")}.exponentialBackoff(10)
-graph.close()
-----
-
-As shown above, the `submit` method takes a `Function<Graph, R>` which is the unit of work to execute and possibly
-retry on failure. The method returns a `Transaction.Workload` object which has a number of default methods for common
-retry strategies. It is also possible to supply a custom retry function if a default one does not suit the required
-purpose.
-
-Threaded Transactions
-~~~~~~~~~~~~~~~~~~~~~
-
-Most `Graph` implementations that support transactions do so in a `ThreadLocal` manner, where the current transaction
-is bound to the current thread of execution. Consider the following example to demonstrate:
-
-[source,java]
-----
-graph.addVertex("name","stephen");
-
-Thread t1 = new Thread(() -> {
- graph.addVertex("name","josh");
-});
-
-Thread t2 = new Thread(() -> {
- graph.addVertex("name","marko");
-});
-
-t1.start()
-t2.start()
-
-t1.join()
-t2.join()
-
-graph.tx().commit();
-----
-
-The above code shows three vertices added to `graph` in three different threads: the current thread, `t1` and
-`t2`. One might expect that by the time this body of code finished executing, that there would be three vertices
-persisted to the `Graph`. However, given the `ThreadLocal` nature of transactions, there really were three separate
-transactions created in that body of code (i.e. one for each thread of execution) and the only one committed was the
-first call to `addVertex` in the primary thread of execution. The other two calls to that method within `t1` and `t2`
-were never committed and thus orphaned.
-
-A `Graph` that `supportsThreadedTransactions` is one that allows for a `Graph` to operate outside of that constraint,
-thus allowing multiple threads to operate within the same transaction. Therefore, if there was a need to have three
-different threads operating within the same transaction, the above code could be re-written as follows:
-
-[source,java]
-----
-Graph threaded = graph.tx().newThreadedTx();
-threaded.addVertex("name","stephen");
-
-Thread t1 = new Thread(() -> {
- threaded.addVertex("name","josh");
-});
-
-Thread t2 = new Thread(() -> {
- threaded.addVertex("name","marko");
-});
-
-t1.start()
-t2.start()
-
-t1.join()
-t2.join()
-
-threaded.tx().commit();
-----
-
-In the above case, the call to `graph.tx().newThreadedTx()` creates a new `Graph` instance that is unbound from the
-`ThreadLocal` transaction, thus allowing each thread to operate on it in the same context. In this case, there would
-be three separate vertices persisted to the `Graph`.
-
-Gremlin I/O
------------
-
-image:gremlin-io.png[width=250,float=right] The task of getting data in and out of `Graph` instances is the job of
-the Gremlin I/O packages. Gremlin I/O provides two interfaces for reading and writing `Graph` instances: `GraphReader`
-and `GraphWriter`. These interfaces expose methods that support:
-
-* Reading and writing an entire `Graph`
-* Reading and writing a `Traversal<Vertex>` as adjacency list format
-* Reading and writing a single `Vertex` (with and without associated `Edge` objects)
-* Reading and writing a single `Edge`
-* Reading and writing a single `VertexProperty`
-* Reading and writing a single `Property`
-* Reading and writing an arbitrary `Object`
-
-In all cases, these methods operate in the currency of `InputStream` and `OutputStream` objects, allowing graphs and
-their related elements to be written to and read from files, byte arrays, etc. The `Graph` interface offers the `io`
-method, which provides access to "reader/writer builder" objects that are pre-configured with serializers provided by
-the `Graph`, as well as helper methods for the various I/O capabilities. Unless there are very advanced requirements
-for the serialization process, it is always best to utilize the methods on the `Io` interface to construct
-`GraphReader` and `GraphWriter` instances, as the implementation may provide some custom settings that would otherwise
-have to be configured manually by the user to do the serialization.
-
-It is up to the implementations of the `GraphReader` and `GraphWriter` interfaces to choose the methods they
-implement and the manner in which they work together. The only semantics enforced and expected is that the write
-methods should produce output that is compatible with the corresponding read method (e.g. the output of
-`writeVertices` should be readable as input to `readVertices` and the output of `writeProperty` should be readable as
-input to `readProperty`).
-
-GraphML Reader/Writer
-~~~~~~~~~~~~~~~~~~~~~
-
-image:gremlin-graphml.png[width=350,float=left] The link:http://graphml.graphdrawing.org/[GraphML] file format is a
-common XML-based representation of a graph. It is widely supported by graph-related tools and libraries making it a
-solid interchange format for TinkerPop. In other words, if the intent is to work with graph data in conjunction with
-applications outside of TinkerPop, GraphML may be the best choice to do that. Common use cases might be:
-
-* Generate a graph using link:https://networkx.github.io/[NetworkX], export it with GraphML and import it to TinkerPop.
-* Produce a subgraph and export it to GraphML to be consumed by and visualized in link:https://gephi.org/[Gephi].
-* Migrate the data of an entire graph to a different graph database not supported by TinkerPop.
-
-As GraphML is a specification for the serialization of an entire graph and not the individual elements of a graph,
-methods that support input and output of single vertices, edges, etc. are not supported.
-
-CAUTION: GraphML is a "lossy" format in that it only supports primitive values for properties and does not have
-support for `Graph` variables. It will use `toString` to serialize property values outside of those primitives.
-
-CAUTION: GraphML, as a specification, allows for `<edge>` and `<node>` elements to appear in any order. The
-`GraphMLReader` will support that, however, that capability comes with a limitation. TinkerPop does not allow the
-vertex label to be changed after the vertex has been created. Therefore, if an `<edge>` element comes before the
-`<node>` the label on the vertex will be ignored. It is thus better to order `<node>` elements in the GraphML to
-appear before all `<edge>` elements if vertex labels are important to the graph.
-
-The following code shows how to write a `Graph` instance to file called `tinkerpop-modern.xml` and then how to read
-that file back into a different instance:
-
-[source,java]
-----
-final Graph graph = TinkerFactory.createModern();
-graph.io(IoCore.graphml()).writeGraph("tinkerpop-modern.xml");
-final Graph newGraph = TinkerGraph.open();
-newGraph.io(IoCore.graphml()).readGraph("tinkerpop-modern.xml");
-----
-
-If a custom configuration is required, then have the `Graph` generate a `GraphReader` or `GraphWriter` "builder" instance:
-
-[source,java]
-----
-final Graph graph = TinkerFactory.createModern();
-try (final OutputStream os = new FileOutputStream("tinkerpop-modern.xml")) {
- graph.io(IoCore.graphml()).writer().normalize(true).create().writeGraph(os, graph);
-}
-
-final Graph newGraph = TinkerGraph.open();
-try (final InputStream stream = new FileInputStream("tinkerpop-modern.xml")) {
- newGraph.io(IoCore.graphml()).reader().vertexIdKey("name").create().readGraph(stream, newGraph);
-}
-----
-
-[[graphson-reader-writer]]
-GraphSON Reader/Writer
-~~~~~~~~~~~~~~~~~~~~~~
-
-image:gremlin-graphson.png[width=350,float=left] GraphSON is a link:http://json.org/[JSON]-based format extended
-from earlier versions of TinkerPop. It is important to note that TinkerPop3's GraphSON is not backwards compatible
-with prior TinkerPop GraphSON versions. GraphSON has some support from graph-related application outside of TinkerPop,
-but it is generally best used in two cases:
-
-* A text format of the graph or its elements is desired (e.g. debugging, usage in source control, etc.)
-* The graph or its elements need to be consumed by code that is not JVM-based (e.g. JavaScript, Python, .NET, etc.)
-
-GraphSON supports all of the `GraphReader` and `GraphWriter` interface methods and can therefore read or write an
-entire `Graph`, vertices, arbitrary objects, etc. The following code shows how to write a `Graph` instance to file
-called `tinkerpop-modern.json` and then how to read that file back into a different instance:
-
-[source,java]
-----
-final Graph graph = TinkerFactory.createModern();
-graph.io(IoCore.graphson()).writeGraph("tinkerpop-modern.json");
-
-final Graph newGraph = TinkerGraph.open();
-newGraph.io(IoCore.graphson()).readGraph("tinkerpop-modern.json");
-----
-
-If a custom configuration is required, then have the `Graph` generate a `GraphReader` or `GraphWriter` "builder" instance:
-
-[source,java]
-----
-final Graph graph = TinkerFactory.createModern();
-try (final OutputStream os = new FileOutputStream("tinkerpop-modern.json")) {
- final GraphSONMapper mapper = graph.io(IoCore.graphson()).mapper().normalize(true).create()
- graph.io(IoCore.graphson()).writer().mapper(mapper).create().writeGraph(os, graph)
-}
-
-final Graph newGraph = TinkerGraph.open();
-try (final InputStream stream = new FileInputStream("tinkerpop-modern.json")) {
- newGraph.io(IoCore.graphson()).reader().vertexIdKey("name").create().readGraph(stream, newGraph);
-}
-----
-
-One of the important configuration options of the `GraphSONReader` and `GraphSONWriter` is the ability to embed type
-information into the output. By embedding the types, it becomes possible to serialize a graph without losing type
-information that might be important when being consumed by another source. The importance of this concept is
-demonstrated in the following example where a single `Vertex` is written to GraphSON using the Gremlin Console:
-
-[gremlin-groovy]
-----
-graph = TinkerFactory.createModern()
-g = graph.traversal()
-f = new FileOutputStream("vertex-1.json")
-graph.io(graphson()).writer().create().writeVertex(f, g.V(1).next(), BOTH)
-f.close()
-----
-
-The following GraphSON example shows the output of `GraphSonWriter.writeVertex()` with associated edges:
-
-[source,json]
-----
-{
- "id": 1,
- "label": "person",
- "outE": {
- "created": [
- {
- "id": 9,
- "inV": 3,
- "properties": {
- "weight": 0.4
- }
- }
- ],
- "knows": [
- {
- "id": 7,
- "inV": 2,
- "properties": {
- "weight": 0.5
- }
- },
- {
- "id": 8,
- "inV": 4,
- "properties": {
- "weight": 1
- }
- }
- ]
- },
- "properties": {
- "name": [
- {
- "id": 0,
- "value": "marko"
- }
- ],
- "age": [
- {
- "id": 1,
- "value": 29
- }
- ]
- }
-}
-----
-
-The vertex properly serializes to valid JSON but note that a consuming application will not automatically know how to
-interpret the numeric values. In coercing those Java values to JSON, such information is lost.
-
-With a minor change to the construction of the `GraphSONWriter` the lossy nature of GraphSON can be avoided:
-
-[gremlin-groovy]
-----
-graph = TinkerFactory.createModern()
-g = graph.traversal()
-f = new FileOutputStream("vertex-1.json")
-mapper = graph.io(graphson()).mapper().embedTypes(true).create()
-graph.io(graphson()).writer().mapper(mapper).create().writeVertex(f, g.V(1).next(), BOTH)
-f.close()
-----
-
-In the above code, the `embedTypes` option is set to `true` and the output below shows the difference in the output:
-
-[source,json]
-----
-{
- "@class": "java.util.HashMap",
- "id": 1,
- "label": "person",
- "outE": {
- "@class": "java.util.HashMap",
- "created": [
- "java.util.ArrayList",
- [
- {
- "@class": "java.util.HashMap",
- "id": 9,
- "inV": 3,
- "properties": {
- "@class": "java.util.HashMap",
- "weight": 0.4
- }
- }
- ]
- ],
- "knows": [
- "java.util.ArrayList",
- [
- {
- "@class": "java.util.HashMap",
- "id": 7,
- "inV": 2,
- "properties": {
- "@class": "java.util.HashMap",
- "weight": 0.5
- }
- },
- {
- "@class": "java.util.HashMap",
- "id": 8,
- "inV": 4,
- "properties": {
- "@class": "java.util.HashMap",
- "weight": 1
- }
- }
- ]
- ]
- },
- "properties": {
- "@class": "java.util.HashMap",
- "name": [
- "java.util.ArrayList",
- [
- {
- "@class": "java.util.HashMap",
- "id": [
- "java.lang.Long",
- 0
- ],
- "value": "marko"
- }
- ]
- ],
- "age": [
- "java.util.ArrayList",
- [
- {
- "@class": "java.util.HashMap",
- "id": [
- "java.lang.Long",
- 1
- ],
- "value": 29
- }
- ]
- ]
- }
-}
-----
-
-The ambiguity of components of the GraphSON is now removed by the `@class` property, which contains Java class
-information for the data it is associated with. The `@class` property is used for all non-final types, with the
-exception of a small number of "natural" types (String, Boolean, Integer, and Double) which can be correctly inferred
-from JSON typing. While the output is more verbose, it comes with the security of not losing type information. While
-non-JVM languages won't be able to consume this information automatically, at least there is a hint as to how the
-values should be coerced back into the correct types in the target language.
-
-[[gryo-reader-writer]]
-Gryo Reader/Writer
-~~~~~~~~~~~~~~~~~~
-
-image:gremlin-kryo.png[width=400,float=left] link:https://github.com/EsotericSoftware/kryo[Kryo] is a popular
-serialization package for the JVM. Gremlin-Kryo is a binary `Graph` serialization format for use on the JVM by JVM
-languages. It is designed to be space efficient, non-lossy and is promoted as the standard format to use when working
-with graph data inside of the TinkerPop stack. A list of common use cases is presented below:
-
-* Migration from one Gremlin Structure implementation to another (e.g. `TinkerGraph` to `Neo4jGraph`)
-* Serialization of individual graph elements to be sent over the network to another JVM.
-* Backups of in-memory graphs or subgraphs.
-
-CAUTION: When migrating between Gremlin Structure implementations, Kryo may not lose data, but it is important to
-consider the features of each `Graph` and whether or not the data types supported in one will be supported in the
-other. Failure to do so, may result in errors.
-
-Kryo supports all of the `GraphReader` and `GraphWriter` interface methods and can therefore read or write an entire
-`Graph`, vertices, edges, etc. The following code shows how to write a `Graph` instance to file called
-`tinkerpop-modern.kryo` and then how to read that file back into a different instance:
-
-[source,java]
-----
-final Graph graph = TinkerFactory.createModern();
-graph.io(IoCore.gryo()).writeGraph("tinkerpop-modern.kryo");
-
-final Graph newGraph = TinkerGraph.open();
-newGraph.io(IoCore.gryo()).readGraph("tinkerpop-modern.kryo")'
-----
-
-If a custom configuration is required, then have the `Graph` generate a `GraphReader` or `GraphWriter` "builder" instance:
-
-[source,java]
-----
-final Graph graph = TinkerFactory.createModern();
-try (final OutputStream os = new FileOutputStream("tinkerpop-modern.kryo")) {
- graph.io(IoCore.gryo()).writer().create().writeGraph(os, graph);
-}
-
-final Graph newGraph = TinkerGraph.open();
-try (final InputStream stream = new FileInputStream("tinkerpop-modern.kryo")) {
- newGraph.io(IoCore.gryo()).reader().vertexIdKey("name").create().readGraph(stream, newGraph);
-}
-----
-
-NOTE: The preferred extension for files names produced by Gryo is `.kryo`.
-
-TinkerPop2 Data Migration
-~~~~~~~~~~~~~~~~~~~~~~~~~
-
-image:data-migration.png[width=300,float=right] For those using TinkerPop2, migrating to TinkerPop3 will mean a number
-of programming changes, but may also require a migration of the data depending on the graph implementation. For
-example, trying to open `TinkerGraph` data from TinkerPop2 with TinkerPop3 code will not work, however opening a
-TinkerPop2 `Neo4jGraph` with a TinkerPop3 `Neo4jGraph` should work provided there aren't Neo4j version compatibility
-mismatches preventing the read.
-
-If such a situation arises that a particular TinkerPop2 `Graph` can not be read by TinkerPop3, a "legacy" data
-migration approach exists. The migration involves writing the TinkerPop2 `Graph` to GraphSON, then reading it to
-TinkerPop3 with the `LegacyGraphSONReader` (a limited implementation of the `GraphReader` interface).
-
-The following represents an example migration of the "classic" toy graph. In this example, the "classic" graph is
-saved to GraphSON using TinkerPop2.
-
-[source,groovy]
-----
-gremlin> Gremlin.version()
-==>2.5.z
-gremlin> graph = TinkerGraphFactory.createTinkerGraph()
-==>tinkergraph[vertices:6 edges:6]
-gremlin> GraphSONWriter.outputGraph(graph,'/tmp/tp2.json',GraphSONMode.EXTENDED)
-==>null
-----
-
-The above console session uses the `gremlin-groovy` distribution from TinkerPop2. It is important to generate the
-`tp2.json` file using the `EXTENDED` mode as it will include data types when necessary which will help limit
-"lossiness" on the TinkerPop3 side when imported. Once `tp2.json` is created, it can then be imported to a TinkerPop3
-`Graph`.
-
-[source,groovy]
-----
-gremlin> Gremlin.version()
-==>x.y.z
-gremlin> graph = TinkerGraph.open()
-==>tinkergraph[vertices:0 edges:0]
-gremlin> r = LegacyGraphSONReader.build().create()
-==>org.apache.tinkerpop.gremlin.structure.io.graphson.LegacyGraphSONReader@64337702
-gremlin> r.readGraph(new FileInputStream('/tmp/tp2.json'), graph)
-==>null
-gremlin> g = graph.traversal(standard())
-==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard]
-gremlin> g.E()
-==>e[11][4-created->3]
-==>e[12][6-created->3]
-==>e[7][1-knows->2]
-==>e[8][1-knows->4]
-==>e[9][1-created->3]
-==>e[10][4-created->5]
-----
-
-Namespace Conventions
----------------------
-
-End users, <<implementations,graph system providers>>, <<graphcomputer,`GraphComputer`>> algorithm designers,
-<<gremlin-plugins,GremlinPlugin>> creators, etc. all leverage properties on elements to store information. There are
-a few conventions that should be respected when naming property keys to ensure that conflicts between these
-stakeholders do not conflict.
-
-* End users are granted the _flat namespace_ (e.g. `name`, `age`, `location`) to key their properties and label their elements.
-* Graph system providers are granted the _hidden namespace_ (e.g. `~metadata`) to key their properties and labels.
-Data keyed as such is only accessible via the graph system implementation and no other stakeholders are granted read
-nor write access to data prefixed with "~" (see `Graph.Hidden`). Test coverage and exceptions exist to ensure that
-graph systems respect this hard boundary.
-* <<vertexprogram,`VertexProgram`>> and <<mapreduce,`MapReduce`>> developers should, like `GraphStrategy` developers,
-leverage _qualified namespaces_ particular to their domain (e.g. `mydomain.myvertexprogram.computedata`).
-* `GremlinPlugin` creators should prefix their plugin name with their domain (e.g. `mydomain.myplugin`).
-
-IMPORTANT: TinkerPop uses `tinkerpop.` and `gremlin.` as the prefixes for provided strategies, vertex programs, map
-reduce implementations, and plugins.
-
-The only truly protected namespace is the _hidden namespace_ provided to graph systems. From there, its up to
-engineers to respect the namespacing conventions presented.
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/bc46d649/docs/src/the-graphcomputer.asciidoc
----------------------------------------------------------------------
diff --git a/docs/src/the-graphcomputer.asciidoc b/docs/src/the-graphcomputer.asciidoc
deleted file mode 100644
index af65f07..0000000
--- a/docs/src/the-graphcomputer.asciidoc
+++ /dev/null
@@ -1,475 +0,0 @@
-////
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements. See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-////
-[[graphcomputer]]
-The GraphComputer
-=================
-
-image:graphcomputer-puffers.png[width=350,float=right] TinkerPop3 provides two primary means of interacting with a
-graph: link:http://en.wikipedia.org/wiki/Online_transaction_processing[online transaction processing] (OLTP) and
-link:http://en.wikipedia.org/wiki/Online_analytical_processing[online analytical processing] (OLAP). OTLP-based
-graph systems allow the user to query the graph in real-time. However, typically, real-time performance is only
-possible when a local traversal is enacted. A local traversal is one that starts at a particular vertex (or small set
-of vertices) and touches a small set of connected vertices (by any arbitrary path of arbitrary length). In short, OLTP
-queries interact with a limited set of data and respond on the order of milliseconds or seconds. On the other hand,
-with OLAP graph processing, the entire graph is processed and thus, every vertex and edge is analyzed (some times
-more than once for iterative, recursive algorithms). Due to the amount of data being processed, the results are
-typically not returned in real-time and for massive graphs (i.e. graphs represented across a cluster of machines),
-results can take on the order of minutes or hours.
-
- * *OLTP*: real-time, limited data accessed, random data access, sequential processing, querying
- * *OLAP*: long running, entire data set accessed, sequential data access, parallel processing, batch processing
-
-image::oltp-vs-olap.png[width=600]
-
-The image above demonstrates the difference between Gremlin OLTP and Gremlin OLAP. With Gremlin OLTP, the graph is
-walked by moving from vertex-to-vertex via incident edges. With Gremlin OLAP, all vertices are provided a
-`VertexProgram`. The programs send messages to one another with the topological structure of the graph acting as the
-communication network (though random message passing possible). In many respects, the messages passed are like
-the OLTP traversers moving from vertex-to-vertex. However, all messages are moving independent of one another, in
-parallel. Once a vertex program is finished computing, TinkerPop3's OLAP engine supports any number
-link:http://en.wikipedia.org/wiki/MapReduce[`MapReduce`] jobs over the resultant graph.
-
-IMPORTANT: `GraphComputer` was designed from the start to be used within a multi-JVM, distributed environment --
-in other words, a multi-machine compute cluster. As such, all the computing objects must be able to be migrated
-between JVMs. The pattern promoted is to store state information in a `Configuration` object to later be regenerated
-by a loading process. It is important to realize that `VertexProgram`, `MapReduce`, and numerous particular instances
-rely heavily on the state of the computing classes (not the structure, but the processes) to be stored in a
-`Configuration`.
-
-[[vertexprogram]]
-VertexProgram
--------------
-
-image:bsp-diagram.png[width=400,float=right] GraphComputer takes a `VertexProgram`. A VertexProgram can be thought of
-as a piece of code that is executed at each vertex in logically parallel manner until some termination condition is
-met (e.g. a number of iterations have occurred, no more data is changing in the graph, etc.). A submitted
-`VertexProgram` is copied to all the workers in the graph. A worker is not an explicit concept in the API, but is
-assumed of all `GraphComputer` implementations. At minimum each vertex is a worker (though this would be inefficient
-due to the fact that each vertex would maintain a VertexProgram). In practice, the workers partition the vertex set
-and and are responsible for the execution of the VertexProgram over all the vertices within their sphere of influence.
-The workers orchestrate the execution of the `VertexProgram.execute()` method on all their vertices in an
-link:http://en.wikipedia.org/wiki/Bulk_synchronous_parallel[bulk synchronous parallel] (BSP) fashion. The vertices
-are able to communicate with one another via messages. There are two kinds of messages in Gremlin OLAP:
-`MessageScope.Local` and `MessageScope.Global`. A local message is a message to an adjacent vertex. A global
-message is a message to any arbitrary vertex in the graph. Once the VertexProgram has completed its execution,
-any number of `MapReduce` jobs are evaluated. MapReduce jobs are provided by the user via `GraphComputer.mapReduce()`
- or by the `VertexProgram` via `VertexProgram.getMapReducers()`.
-
-image::graphcomputer.png[width=500]
-
-The example below demonstrates how to submit a VertexProgram to a graph's GraphComputer. `GraphComputer.submit()`
-yields a `Future<ComputerResult>`. The `ComputerResult` has the resultant computed graph which can be a full copy
-of the original graph (see <<hadoop-gremlin,Hadoop-Gremlin>>) or a view over the original graph (see
-<<tinkergraph,TinkerGraph>>). The ComputerResult also provides access to computational side-effects called `Memory`
-(which includes, for example, runtime, number of iterations, results of MapReduce jobs, and VertexProgram-specific
-memory manipulations).
-
-[gremlin-groovy,modern]
-----
-result = graph.compute().program(PageRankVertexProgram.build().create()).submit().get()
-result.memory().runtime
-g = result.graph().traversal(standard())
-g.V().valueMap('name',PageRankVertexProgram.PAGE_RANK)
-----
-
-NOTE: This model of "vertex-centric graph computing" was made popular by Google's
-link:http://googleresearch.blogspot.com/2009/06/large-scale-graph-computing-at-google.html[Pregel] graph engine.
-In the open source world, this model is found in OLAP graph computing systems such as link:https://giraph.apache.org/[Giraph],
-link:https://hama.apache.org/[Hama], and link:http://faunus.thinkaurelius.com[Faunus]. TinkerPop3 extends the
-popularized model with integrated post-processing <<mapreduce,MapReduce>> jobs over the vertex set.
-
-[[mapreduce]]
-MapReduce
----------
-
-The BSP model proposed by Pregel stores the results of the computation in a distributed manner as properties on the
-elements in the graph. In many situations, it is necessary to aggregate those resultant properties into a single
-result set (i.e. a statistic). For instance, assume a VertexProgram that computes a nominal cluster for each vertex
-(i.e. link:http://en.wikipedia.org/wiki/Community_structure[a graph clustering algorithm]). At the end of the
-computation, each vertex will have a property denoting the cluster it was assigned to. TinkerPop3 provides the
-ability to answer global questions about the clusters. For instance, in order to answer the following questions,
-`MapReduce` jobs are required:
-
- * How many vertices are in each cluster? (*presented below*)
- * How many unique clusters are there? (*presented below*)
- * What is the average age of each vertex in each cluster?
- * What is the degree distribution of the vertices in each cluster?
-
-A compressed representation of the `MapReduce` API in TinkerPop3 is provided below. The key idea is that the
-`map`-stage processes all vertices to emit key/value pairs. Those values are aggregated on their respective key
-for the `reduce`-stage to do its processing to ultimately yield more key/value pairs.
-
-[source,java]
-public interface MapReduce<MK, MV, RK, RV, R> {
- public void map(final Vertex vertex, final MapEmitter<MK, MV> emitter);
- public void reduce(final MK key, final Iterator<MV> values, final ReduceEmitter<RK, RV> emitter);
- // there are more methods
-}
-
-IMPORTANT: The vertex that is passed into the `MapReduce.map()` method does not contain edges. The vertex only
-contains original and computed vertex properties. This reduces the amount of data required to be loaded and ensures
-that MapReduce is used for post-processing computed results. All edge-based computing should be accomplished in the
-`VertexProgram`.
-
-image::mapreduce.png[width=650]
-
-The `MapReduce` extension to GraphComputer is made explicit when examining the
-<<peerpressurevertexprogram,`PeerPressureVertexProgram`>> and corresponding `ClusterPopulationMapReduce`.
-In the code below, the GraphComputer result returns the computed on `Graph` as well as the `Memory` of the
-computation (`ComputerResult`). The memory maintain the results of any MapReduce jobs. The cluster population
-MapReduce result states that there are 5 vertices in cluster 1 and 1 vertex in cluster 6. This can be verified
-(in a serial manner) by looking at the `PeerPressureVertexProgram.CLUSTER` property of the resultant graph. Notice
-that the property is "hidden" unless it is directly accessed via name.
-
-[gremlin-groovy,modern]
-----
-graph = TinkerFactory.createModern()
-result = graph.compute().program(PeerPressureVertexProgram.build().create()).mapReduce(ClusterPopulationMapReduce.build().create()).submit().get()
-result.memory().get('clusterPopulation')
-g = result.graph().traversal(standard())
-g.V().values(PeerPressureVertexProgram.CLUSTER).groupCount().next()
-g.V().valueMap()
-----
-
-If there are numerous statistics desired, then its possible to register as many MapReduce jobs as needed. For
-instance, the `ClusterCountMapReduce` determines how many unique clusters were created by the peer pressure algorithm.
-Below both `ClusterCountMapReduce` and `ClusterPopulationMapReduce` are computed over the resultant graph.
-
-[gremlin-groovy,modern]
-----
-result = graph.compute().program(PeerPressureVertexProgram.build().create()).
- mapReduce(ClusterPopulationMapReduce.build().create()).
- mapReduce(ClusterCountMapReduce.build().create()).submit().get()
-result.memory().clusterPopulation
-result.memory().clusterCount
-----
-
-IMPORTANT: The MapReduce model of TinkerPop3 does not support MapReduce chaining. Thus, the order in which the
-MapReduce jobs are executed is irrelevant. This is made apparent when realizing that the `map()`-stage takes a
-`Vertex` as its input and the `reduce()`-stage yields key/value pairs. Thus, the results of reduce can not feed back
-into map.
-
-A Collection of VertexPrograms
-------------------------------
-
-TinkerPop3 provides a collection of VertexPrograms that implement common algorithms. This section discusses the various implementations.
-
-IMPORTANT: The vertex programs presented are what are provided as of TinkerPop x.y.z. Over time, with future releases,
-more algorithms will be added.
-
-[[pagerankvertexprogram]]
-PageRankVertexProgram
-~~~~~~~~~~~~~~~~~~~~~
-
-image:gremlin-pagerank.png[width=400,float=right] link:http://en.wikipedia.org/wiki/PageRank[PageRank] is perhaps the
-most popular OLAP-oriented graph algorithm. This link:http://en.wikipedia.org/wiki/Centrality[eigenvector centrality]
-variant was developed by Brin and Page of Google. PageRank defines a centrality value for all vertices in the graph,
-where centrality is defined recursively where a vertex is central if it is connected to central vertices. PageRank is
-an iterative algorithm that converges to a link:http://en.wikipedia.org/wiki/Ergodicity[steady state distribution]. If
-the pageRank values are normalized to 1.0, then the pageRank value of a vertex is the probability that a random walker
-will be seen that that vertex in the graph at any arbitrary moment in time. In order to help developers understand the
-methods of a `VertexProgram`, the PageRankVertexProgram code is analyzed below.
-
-[source,java]
-----
-public class PageRankVertexProgram implements VertexProgram<Double> { <1>
-
- private MessageScope.Local<Double> incidentMessageScope = MessageScope.Local.of(__::outE); <2>
- private MessageScope.Local<Double> countMessageScope = MessageScope.Local.of(new MessageScope.Local.ReverseTraversalSupplier(this.incidentMessageScope));
-
- public static final String PAGE_RANK = "gremlin.pageRankVertexProgram.pageRank"; <3>
- public static final String EDGE_COUNT = "gremlin.pageRankVertexProgram.edgeCount";
-
- private static final String VERTEX_COUNT = "gremlin.pageRankVertexProgram.vertexCount";
- private static final String ALPHA = "gremlin.pageRankVertexProgram.alpha";
- private static final String TOTAL_ITERATIONS = "gremlin.pageRankVertexProgram.totalIterations";
- private static final String INCIDENT_TRAVERSAL_SUPPLIER = "gremlin.pageRankVertexProgram.incidentTraversalSupplier";
-
- private ConfigurationTraversal<Vertex, Edge> configurationTraversal;
- private double vertexCountAsDouble = 1.0d;
- private double alpha = 0.85d;
- private int totalIterations = 30;
-
- private static final Set<String> COMPUTE_KEYS = new HashSet<>(Arrays.asList(PAGE_RANK, EDGE_COUNT));
-
- private PageRankVertexProgram() {}
-
- @Override
- public void loadState(final Graph graph, final Configuration configuration) { <4>
- if (configuration.containsKey(TRAVERSAL_SUPPLIER)) {
- this.configurationTraversal = ConfigurationTraversal.loadState(graph, configuration, TRAVERSAL_SUPPLIER);
- this.incidentMessageScope = MessageScope.Local.of(this.configurationTraversal);
- this.countMessageScope = MessageScope.Local.of(new MessageScope.Local.ReverseTraversalSupplier(this.incidentMessageScope));
- }
- this.vertexCountAsDouble = configuration.getDouble(VERTEX_COUNT, 1.0d);
- this.alpha = configuration.getDouble(ALPHA, 0.85d);
- this.totalIterations = configuration.getInt(TOTAL_ITERATIONS, 30);
- }
-
- @Override
- public void storeState(final Configuration configuration) {
- configuration.setProperty(VERTEX_PROGRAM, PageRankVertexProgram.class.getName());
- configuration.setProperty(VERTEX_COUNT, this.vertexCountAsDouble);
- configuration.setProperty(ALPHA, this.alpha);
- configuration.setProperty(TOTAL_ITERATIONS, this.totalIterations);
- if (null != this.traversalSupplier) {
- this.traversalSupplier.storeState(configuration);
- }
- }
-
- @Override
- public Set<String> getElementComputeKeys() { <5>
- return COMPUTE_KEYS;
- }
-
- @Override
- public Optional<MessageCombiner<Double>> getMessageCombiner() {
- return (Optional) PageRankMessageCombiner.instance();
- }
-
- @Override
- public Set<MessageScope> getMessageScopes(final int iteration) {
- final Set<MessageScope> set = new HashSet<>();
- set.add(0 == iteration ? this.countMessageScope : this.incidentMessageScope);
- return set;
- }
-
- @Override
- public void setup(final Memory memory) {
-
- }
-
- @Override
- public void execute(final Vertex vertex, Messenger<Double> messenger, final Memory memory) { <6>
- if (memory.isInitialIteration()) { <7>
- messenger.sendMessage(this.countMessageScope, 1.0d);
- } else if (1 == memory.getIteration()) { <8>
- double initialPageRank = 1.0d / this.vertexCountAsDouble;
- double edgeCount = IteratorUtils.reduce(messenger.receiveMessages(), 0.0d, (a, b) -> a + b);
- vertex.property(PAGE_RANK, initialPageRank);
- vertex.property(EDGE_COUNT, edgeCount);
- messenger.sendMessage(this.incidentMessageScope, initialPageRank / edgeCount);
- } else { <9>
- double newPageRank = IteratorUtils.reduce(messenger.receiveMessages(), 0.0d, (a, b) -> a + b);
- newPageRank = (this.alpha * newPageRank) + ((1.0d - this.alpha) / this.vertexCountAsDouble);
- vertex.property(PAGE_RANK, newPageRank);
- messenger.sendMessage(this.incidentMessageScope, newPageRank / vertex.<Double>value(EDGE_COUNT));
- }
- }
-
- @Override
- public boolean terminate(final Memory memory) { <10>
- return memory.getIteration() >= this.totalIterations;
- }
-
- @Override
- public String toString() {
- return StringFactory.vertexProgramString(this, "alpha=" + this.alpha + ",iterations=" + this.totalIterations);
- }
-}
-----
-
-<1> `PageRankVertexProgram` implements `VertexProgram<Double>` because the messages it sends are Java doubles.
-<2> The default path of energy propagation is via outgoing edges from the current vertex.
-<3> The resulting PageRank values for the vertices are stored as a hidden property.
-<4> A vertex program is constructed using an Apache `Configuration` to ensure easy dissemination across a cluster of JVMs.
-<5> A vertex program must define the "compute keys" that are the properties being operated on during the computation.
-<6> The "while"-loop of the vertex program.
-<7> In order to determine how to distribute the energy to neighbors, a "1"-count is used to determine how many incident vertices exist for the `MessageScope`.
-<8> Initially, each vertex is provided an equal amount of energy represented as a double.
-<9> Energy is aggregated, computed on according to the PageRank algorithm, and then disseminated according to the defined `MessageScope.Local`.
-<10> The computation is terminated after a pre-defined number of iterations.
-
-The above `PageRankVertexProgram` is used as follows.
-
-[gremlin-groovy,modern]
-----
-result = graph.compute().program(PageRankVertexProgram.build().create()).submit().get()
-result.memory().runtime
-g = result.graph().traversal(standard())
-g.V().valueMap('name',PageRankVertexProgram.PAGE_RANK)
-----
-
-
-[[peerpressurevertexprogram]]
-PeerPressureVertexProgram
-~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The `PeerPressureVertexProgram` is a clustering algorithm that assigns a nominal value to each vertex in the graph.
-The nominal value represents the vertex's cluster. If two vertices have the same nominal value, then they are in the
-same cluster. The algorithm proceeds in the following manner.
-
- . Every vertex assigns itself to a unique cluster ID (initially, its vertex ID).
- . Every vertex determines its per neighbor vote strength as 1.0d / incident edges count.
- . Every vertex sends its cluster ID and vote strength to its adjacent vertices as a `Pair<Serializable,Double>`
- . Every vertex generates a vote energy distribution of received cluster IDs and changes its current cluster ID to the most frequent cluster ID.
- .. If there is a tie, then the cluster with the lowest `toString()` comparison is selected.
- . Steps 3 and 4 repeat until either a max number of iterations has occurred or no vertex has adjusted its cluster anymore.
-
-[[bulkloadervertexprogram]]
-BulkLoaderVertexProgram
-~~~~~~~~~~~~~~~~~~~~~~~
-
-The `BulkLoaderVertexProgram` provides a generalized way for loading graphs of any size (preferably large sized graphs)
-into a persistent `Graph`. The input can be any existing `Graph` database supporting TinkerPop3 or any of the Hadoop
-GraphInputFormats (e.g. `GraphSONInputFormat`, `GryoInputFormat` or `ScriptInputFormat`). The following example
-demonstrates how to load data from one TinkerGraph to another:
-
-[gremlin-groovy]
-----
-writeGraphConf = new BaseConfiguration()
-writeGraphConf.setProperty("gremlin.graph", "org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerGraph")
-writeGraphConf.setProperty("gremlin.tinkergraph.graphFormat", "gryo")
-writeGraphConf.setProperty("gremlin.tinkergraph.graphLocation", "/tmp/tinkergraph.kryo")
-modern = TinkerFactory.createModern()
-blvp = BulkLoaderVertexProgram.build().
- keepOriginalIds(false).
- writeGraph(writeGraphConf).create(modern)
-modern.compute().workers(1).program(blvp).submit().get()
-graph = GraphFactory.open(writeGraphConf)
-g = graph.traversal()
-g.V().valueMap()
-graph.close()
-----
-
-.Available configuration options
-[width="800px",options="header"]
-|========================================
-|Builder Method |Purpose | Default Value
-|`bulkLoader(Class\|String)` | Sets the class of the bulk loader implementation. | `IncrementalBulkLoader`
-|`vertexIdProperty(String)` | Sets the name of the property in the target graph that holds the vertex id from the
-source graph. | `bulkLoader.vertex.id`
-|`keepOriginalIds(boolean)` |Whether to keep the id's from the source graph in the target graph or not. It's
-recommended to keep them if it's planned to do further bulk loads using the same datasources. | `true`
-|`userSuppliedIds(boolean)` |Whether to use the id's from the source graph as id's in the target graph. If set to
-`true`, `vertexIdProperty` will be ignored. Note, that the target graph must support user supplied identifiers. | `false`
-|`intermediateBatchSize(int)` |Sets the batch size for intermediate transactions. This is per thread in a
-multi-threaded environment. +0+ means that transactions will only be committed at the end of an iteration cycle.
-It's recommended to tune this property for the target graph and not use the default value of +0+. | `0`
-|`writeGraph(String)` | Sets the path to a `GraphFactory` compatible configuration file for the target graph. | _none_
-|========================================
-
-NOTE: `BulkLoaderVertexProgram` comes with a default `BulkLoader` implementation, namely `IncrementalBulkLoader`. It
-will work for the most use-cases, but has one limitation though: It doesn't support multi-valued properties.
-`IncrementalBulkLoader` will handle every property as a single-valued property. A custom `BulkLoader` implementation
-has to be used if the default behavior is not sufficient.
-
-NOTE: A custom `BulkLoader` implementation for incremental loading should use `GraphTraversal` methods to create/update
-elements (e.g. `g.addV()` instead of `graph.addVertex()`). This way the `BulkLoaderVertexProgram` is able to efficiently
-track changes in the underlying graph and can apply several optimization techniques.
-
-[[traversalvertexprogram]]
-TraversalVertexProgram
-~~~~~~~~~~~~~~~~~~~~~~
-
-image:traversal-vertex-program.png[width=250,float=left] The `TraversalVertexProgram` is a "special" VertexProgram in
-that it can be executed via a `GraphTraversal` with a `ComputerTraversalEngine`. In Gremlin, it is possible to have
-the same traversal executed using either the standard OTLP-engine or the `GraphComputer` OLAP-engine. The difference
-being where the traversal is submitted.
-
-NOTE: This model of graph traversal in a BSP system was first implemented by the
-link:http://faunus.thinkaurelius.com[Faunus] graph analytics engine and originally described in
-link:http://markorodriguez.com/2011/04/19/local-and-distributed-traversal-engines/[Local and Distributed Traversal Engines].
-
-[gremlin-groovy,modern]
-----
-g = graph.traversal(standard())
-g.V().both().hasLabel('person').values('age').groupCount().next() // OLTP
-g = graph.traversal(computer())
-g.V().both().hasLabel('person').values('age').groupCount().next() // OLAP
-----
-
-image::olap-traversal.png[width=650]
-
-In the OLAP example above, a `TraversalVertexProgram` is (logically) sent to each vertex in the graph. Each instance
-evaluation requires (logically) 5 BSP iterations and each iteration is interpreted as such:
-
- . `g.V()`: Put a traverser on each vertex in the graph.
- . `both()`: Propagate each traverser to the vertices `both`-adjacent to its current vertex.
- . `hasLabel('person')`: If the vertex is not a person, kill the traversers at that vertex.
- . `values('age')`: Have all the traversers reference the integer age of their current vertex.
- . `groupCount()`: Count how many times a particular age has been seen.
-
-While 5 iterations were presented, in fact, `TraversalVertexProgram` will execute the traversal in only
-3 iterations. The reason being is that `hasLabel('person').values('age').groupCount()` can all be executed in a
-single iteration as any message sent would simply be to the current executing vertex. Thus, a simple optimization
-exists in Gremlin OLAP called "reflexive message passing" which simulates non-message-passing BSP iterations within a
-single BSP iteration.
-
-When the computation is complete a <<mapreduce,MapReduce>> job executes which aggregates all the `groupCount()`
-sideEffect Map (i.e. "`HashMap`") objects on each vertex into a single local representation (thus, turning the
-distributed Map representation into a local Map representation).
-
-////
-The same OLAP traversal can be executed using the standard `g.compute()` model, though at the expense of verbosity.
-`TraversalVertexProgram` provides a fluent `Builder` for constructing a `TraversalVertexProgram`. The specified
-`traversal()` can be either a `Supplier<Traversal>` object, a `Supplier<Traversal>` class, or a
-link:http://en.wikipedia.org/wiki/Scripting_for_the_Java_Platform[JSR-223] script that will generate (i.e. supply) a
-`Traversal`. If `traversal()` is supplied a single string, it is assumed that "gremlin-groovy" is the `ScriptEngine`
-to use. If two strings are supplied, then the first string denotes the `ScriptEngine` to evaluate the second string
-script with in order to generate (i.e. supply) the `Traversal`.
-
-[gremlin-groovy,modern]
-----
-//g.engine(computer())
-//result = g.compute().program(TraversalVertexProgram.build().traversal(g.V().both().hasLabel('person').values('age').groupCount('a')).create()).submit().get()
-//result.memory().a
-//result.memory().iteration
-//result.memory().runtime
-----
-////
-
-[[distributed-gremlin-gotchas]]
-Distributed Gremlin Gotchas
-^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Gremlin OLTP is not identical to Gremlin OLAP.
-
-IMPORTANT: There are two primary theoretical differences between Gremlin OLTP and Gremlin OLAP. First, Gremlin OLTP
-(via `Traversal`) leverages a link:http://en.wikipedia.org/wiki/Depth-first_search[depth-first] execution engine.
-Depth-first execution has a limited memory footprint due to link:http://en.wikipedia.org/wiki/Lazy_evaluation[lazy evaluation].
-On the other hand, Gremlin OLAP (via `TraversalVertexProgram`) leverages a
-link:http://en.wikipedia.org/wiki/Breadth-first_search[breadth-first] execution engine which maintains a larger memory
-footprint, but a better time complexity due to vertex-local traversers being able to be merged. The second difference
-is that Gremlin OLTP is executed in a serial fashion, while Gremlin OLAP is executed in a parallel fashion. These two
-fundamental differences lead to the behaviors enumerated below.
-
-image::gremlin-without-a-cause.png[width=200,float=right]
-
-. Traversal sideEffects are represented as a distributed data structure across the graph's vertex set. It is not
-possible to get a global view of a sideEffect until it is aggregated via a <<mapreduce,MapReduce>> job. In some
-situations, the local vertex representation of the sideEffect is sufficient to ensure the intended semantics of the
-traversal are respected. However, this is not generally true so be wary of traversals that require global views of a
-sideEffect.
-. When evaluating traversals that rely on path information (i.e. the history of the traversal), practical
-computational limits can easily be reached due the link:http://en.wikipedia.org/wiki/Combinatorial_explosion[combinatoric explosion]
-of data. With path computing enabled, every traverser is unique and thus, must be enumerated as opposed to being
-counted/merged. The difference being a collection of paths vs. a single 64-bit long at a single vertex. For more
-information on this concept, please see link:http://thinkaurelius.com/2012/11/11/faunus-provides-big-graph-data-analytics/[Faunus Provides Big Graph Data].
-. When traversals of the form `x.as('a').y.someSideEffectStep('a').z` are evaluated, the `a` object is stored in the
-path information of the traverser and thus, such traversals (may) turn on path calculations when executed on a
-`GraphComputer`
-. Steps that are concerned with the global ordering of traversers do not have a meaningful representation in
-OLAP. For example, what does <<order-step,`order()`>>-step mean when all traversers are being processed in parallel?
-Even if the traversers were aggregated and ordered, then at the next step they would return to being executed in
-parallel and thus, in an unpredictable order. When `order()`-like steps are executed at the end of a traversal (i.e
-the final step), the `TraverserMapReduce` job ensures the resultant serial representation is ordered accordingly.
-. Steps that are concerned with providing a global aggregate to the next step of computation do not have a correlate
-in OLAP. For example, <<fold-step,`fold()`>>-step can only fold up the objects at each executing vertex. Next, even
-if a global fold was possible, where would it go? Which vertex would be the host of the data structure? The
-`fold()`-step only makes sense as an end-step whereby a MapReduce job can generate the proper global-to-local data
-reduction.