You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tinkerpop.apache.org by dk...@apache.org on 2016/01/22 19:58:47 UTC
[1/3] incubator-tinkerpop git commit: added note for init-tp-spark.sh
script in reference docs
Repository: incubator-tinkerpop
Updated Branches:
refs/heads/master 08c421b0d -> 6f3fd00e6
added note for init-tp-spark.sh script in reference docs
Project: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/commit/610877af
Tree: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/tree/610877af
Diff: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/diff/610877af
Branch: refs/heads/master
Commit: 610877af9766d11a2c94b377255756a2abc5114e
Parents: f8354fc
Author: Daniel Kuppitz <da...@hotmail.com>
Authored: Fri Jan 22 17:56:47 2016 +0100
Committer: Daniel Kuppitz <da...@hotmail.com>
Committed: Fri Jan 22 17:56:47 2016 +0100
----------------------------------------------------------------------
docs/src/reference/implementations.asciidoc | 14 ++++++++++++++
1 file changed, 14 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/610877af/docs/src/reference/implementations.asciidoc
----------------------------------------------------------------------
diff --git a/docs/src/reference/implementations.asciidoc b/docs/src/reference/implementations.asciidoc
index 9b39d65..d713cf5 100644
--- a/docs/src/reference/implementations.asciidoc
+++ b/docs/src/reference/implementations.asciidoc
@@ -1166,6 +1166,20 @@ arguably easier for developers to work with than native Hadoop MapReduce. Spark-
the bulk-synchronous parallel, distributed message passing algorithm within Spark and thus, any `VertexProgram` can be
executed over `SparkGraphComputer`.
+If `SparkGraphComputer` will be used as the `GraphComputer` for `HadoopGraph` then its `lib` directory should be
+specified in `HADOOP_GREMLIN_LIBS`.
+
+[source,shell]
+export HADOOP_GREMLIN_LIBS=$HADOOP_GREMLIN_LIBS:/usr/local/gremlin-console/ext/spark-gremlin/lib
+
+Furthermore the lib directory should be distributed across all servers in the Spark cluster. For this purpose TinkerPop
+provides a helper script, which takes the Spark installation directory and the the spark servers as input:
+
+[source,shell]
+bin/init-tp-spark.sh /usr/local/spark spark@10.0.0.1 spark@10.0.0.2 spark@10.0.0.3
+
+Once the `lib` directory is distributed, `SparkGraphComputer` can be used as follows.
+
[gremlin-groovy]
----
graph = GraphFactory.open('conf/hadoop/hadoop-gryo.properties')
[2/3] incubator-tinkerpop git commit: Merge branch 'master' of
https://git-wip-us.apache.org/repos/asf/incubator-tinkerpop
Posted by dk...@apache.org.
Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/incubator-tinkerpop
Project: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/commit/c167108e
Tree: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/tree/c167108e
Diff: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/diff/c167108e
Branch: refs/heads/master
Commit: c167108e2d9fbd061286c750b9376bb31995c0fa
Parents: 610877a 08c421b
Author: Daniel Kuppitz <da...@hotmail.com>
Authored: Fri Jan 22 19:57:36 2016 +0100
Committer: Daniel Kuppitz <da...@hotmail.com>
Committed: Fri Jan 22 19:57:36 2016 +0100
----------------------------------------------------------------------
CHANGELOG.asciidoc | 1 +
docs/preprocessor/preprocess.sh | 19 ++++++++++---------
docs/static/images/tinkerpop-splash.png | Bin 142445 -> 653463 bytes
docs/static/images/tinkerpop3-sketch.png | Bin 0 -> 68203 bytes
neo4j-gremlin/pom.xml | 6 +++++-
5 files changed, 16 insertions(+), 10 deletions(-)
----------------------------------------------------------------------
[3/3] incubator-tinkerpop git commit: fixed typo
Posted by dk...@apache.org.
fixed typo
Project: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/commit/6f3fd00e
Tree: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/tree/6f3fd00e
Diff: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/diff/6f3fd00e
Branch: refs/heads/master
Commit: 6f3fd00e66972aa27816c565e157c4302e602061
Parents: c167108
Author: Daniel Kuppitz <da...@hotmail.com>
Authored: Fri Jan 22 19:58:20 2016 +0100
Committer: Daniel Kuppitz <da...@hotmail.com>
Committed: Fri Jan 22 19:58:20 2016 +0100
----------------------------------------------------------------------
CHANGELOG.asciidoc | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/6f3fd00e/CHANGELOG.asciidoc
----------------------------------------------------------------------
diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc
index a68a359..b72bfa0 100644
--- a/CHANGELOG.asciidoc
+++ b/CHANGELOG.asciidoc
@@ -35,7 +35,7 @@ TinkerPop 3.1.1 (NOT OFFICIALLY RELEASED YET)
* Integrated `NumberHelper` in `SackFunctions`.
* The Spark persistence `StorageLevel` can now be set for both job graphs and `PersistedOutputRDD` data.
* Added to the list of "invalid binding keys" allowed by Gremlin Server to cover the private fields of `T` which get exposed in the `ScriptEngine` on static imports.
-* Added `BulkDumperVertex` that allows to dump a whole graph in any of the supported IO formats (GraphSON, Gryo, Script).
+* Added `BulkDumperVertexProgram` that allows to dump a whole graph in any of the supported IO formats (GraphSON, Gryo, Script).
* Fixed a bug around duration calculations of `cap()`-step during profiling.
* It is possible to completely avoid using HDFS with Spark if `PersistedInputRDD` and `PersistedOutpuRDD` are leveraged.
* `InputRDD` and `OutputRDD` can now process both graphs and memory (i.e. sideEffects).