You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tinkerpop.apache.org by rd...@apache.org on 2017/01/27 21:29:52 UTC
[01/29] tinkerpop git commit: Updated the closeness and betweeness
centrality recipes. [Forced Update!]
Repository: tinkerpop
Updated Branches:
refs/heads/TINKERPOP-1602 b18f59cd7 -> 33253dc6b (forced update)
Updated the closeness and betweeness centrality recipes.
1) added a disclaimer that both recipes should be used with care (only on small (sub)graphs)
2) optimized the execution plan
3) took multiple shortest paths between a pair of vertices into account
4) updated the sample graph to one that has multiple shortest paths between certain pairs of vertices
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/5577ea56
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/5577ea56
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/5577ea56
Branch: refs/heads/TINKERPOP-1602
Commit: 5577ea564e8a9634d3b0699cbddee59b039db90f
Parents: 8ad2911
Author: Daniel Kuppitz <da...@hotmail.com>
Authored: Fri Jan 20 16:36:42 2017 +0100
Committer: Daniel Kuppitz <da...@hotmail.com>
Committed: Fri Jan 20 16:41:09 2017 +0100
----------------------------------------------------------------------
docs/src/recipes/centrality.asciidoc | 121 +++++++++++++++++-------------
1 file changed, 67 insertions(+), 54 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/5577ea56/docs/src/recipes/centrality.asciidoc
----------------------------------------------------------------------
diff --git a/docs/src/recipes/centrality.asciidoc b/docs/src/recipes/centrality.asciidoc
index cbce418..f6fba30 100644
--- a/docs/src/recipes/centrality.asciidoc
+++ b/docs/src/recipes/centrality.asciidoc
@@ -67,43 +67,48 @@ image:betweeness-example.png[width=600]
[gremlin-groovy ]
----
-g.addV('name','a').as('a').
- addV('name','b').as('b').
- addV('name','c').as('c').
- addV('name','d').as('d').
- addV('name','e').as('e').
+g.addV(id,'A').as('a').
+ addV(id,'B').as('b').
+ addV(id,'C').as('c').
+ addV(id,'D').as('d').
+ addV(id,'E').as('e').
+ addV(id,'F').as('f').
addE('next').from('a').to('b').
addE('next').from('b').to('c').
- addE('next').from('c').to('d').
- addE('next').from('d').to('e').iterate()
-g.withSack(0).V().store("x").repeat(both().simplePath()).emit().path(). <1>
- group().by(project("a","b").by(limit(local, 1)). <2>
- by(tail(local, 1))).
- by(order().by(count(local))). <3>
- select(values).as("shortestPaths"). <4>
- select("x").unfold().as("v"). <5>
- select("shortestPaths"). <6>
- map(unfold().filter(unfold().where(eq("v"))).count()). <7>
- sack(sum).sack().as("betweeness"). <8>
- select("v","betweeness")
+ addE('next').from('b').to('d').
+ addE('next').from('c').to('e').
+ addE('next').from('d').to('e').
+ addE('next').from('e').to('f').iterate()
+g.V().as("v"). <1>
+ repeat(both().simplePath().as("v")).emit(). <2>
+ filter(project("x","y","z").by(select(first, "v")). <3>
+ by(select(last, "v")).
+ by(select(all, "v").count(local)).as("triple").
+ coalesce(select("x","y").as("a"). <4>
+ select("triples").unfold().as("t").
+ select("x","y").where(eq("a")).
+ select("t"),
+ store("triples")). <5>
+ select("z").as("length").
+ select("triple").select("z").where(eq("length"))). <6>
+ select(all, "v").unfold(). <7>
+ groupCount().next() <8>
----
-<1> Defines a Gremlin link:http://tinkerpop.apache.org/docs/x.y.z/reference/#sack-step[sack] with a value of zero,
-which represents the initial betweeness score for each vertex, and traverses on both incoming and outgoing edges
-avoiding <<cycle-detection, cyclic paths>>.
-<2> Group each path by the first and last vertex.
-<3> Reduce the list of paths to the shortest path between the first and last vertex by ordering on their lengths.
-<4> Recall that at this point, there is a `Map` keyed by first and last vertex and with a value of just the shortest
-path. Extract the shortest path with `select(values)`, since that's the only portion required for the remainder of
-the traversal.
-<5> The "x" key contains the list of vertices stored from step 1 - unfold that list into "v" for later use. This step
-will unwrap the vertex that is stored in the `Traverser` as
-link:http://tinkerpop.apache.org/javadocs/x.y.z/full/org/apache/tinkerpop/gremlin/process/traversal/step/util/BulkSet.html[BulkSet]
-so that it can be used directly in the `Traversal`.
-<6> Iterate the set of shortest paths. At this point, it is worth noting that the traversal is iterating each vertex
-in "v" and for each vertex in "v" it is iterating each `Path` in "shortestpaths".
-<7> For each path, transform it to a count of the number of times that "v" from step 5 is encountered.
-<8> Sum the counts for each vertex using `sack()`, normalize the value and label it as the "betweeness" to be the score.
+<1> Starting from each vertex in the graph...
+<2> ...traverse on both - incoming and outgoing - edges, avoiding <<cycle-detection, cyclic paths>>.
+<3> Create a triple consisting of the first vertex, the last vertex and the length of the path between them.
+<4> Determine whether a path between those two vertices was already found.
+<5> If this is the first path between the two vertices, store the triple in an internal collection named "triples".
+<6> Keep only those paths between a pair of vertices that have the same length as the first path that was found between them.
+<7> Select all shortest paths and unfold them.
+<8> Count the number of occurrences of each vertex, which is ultimately its betweeness score.
+
+WARNING: Since the betweeness centrality algorithm requires the shortest path between any pair of vertices in the graph,
+its practical applications are very limited. It's recommended to use this algorithm only on small subgraphs (graphs like
+the link:http://tinkerpop.apache.org/docs/current/reference/#grateful-dead[Grateful Dead graph] with only 808 vertices
+and 8049 edges already require a massive amount of compute resources to determine the shortest paths between all vertex
+pairs).
[[closeness-centrality]]
Closeness Centrality
@@ -114,28 +119,36 @@ other reachable vertices in the graph. The following examples use the modern toy
[gremlin-groovy,modern]
----
-g.withSack(1f).V().repeat(both().simplePath()).emit().path(). <1>
- group().by(project("a","b").by(limit(local, 1)). <2>
- by(tail(local, 1))).
- by(order().by(count(local))). <3>
- select(values).unfold(). <4>
- project("v","length").
- by(limit(local, 1)). <5>
- by(count(local).sack(div).sack()). <6>
- group().by(select("v")).by(select("length").sum()) <7>
+g = TinkerFactory.createModern().traversal()
+g.withSack(1f).V().as("v"). <1>
+ repeat(both().simplePath().as("v")).emit(). <2>
+ filter(project("x","y","z").by(select(first, "v")). <3>
+ by(select(last, "v")).
+ by(select(all, "v").count(local)).as("triple").
+ coalesce(select("x","y").as("a"). <4>
+ select("triples").unfold().as("t").
+ select("x","y").where(eq("a")).
+ select("t"),
+ store("triples")). <5>
+ select("z").as("length").
+ select("triple").select("z").where(eq("length"))). <6>
+ group().by(select(first, "v")). <7>
+ by(select(all, "v").count(local).sack(div).sack().sum()).next()
----
-<1> Defines a Gremlin link:http://tinkerpop.apache.org/docs/x.y.z/reference/#sack-step[sack] with a value of one,
-and traverses on both incoming and outgoing edges avoiding <<cycle-detection, cyclic paths>>.
-<2> Group each path by the first and last vertex.
-<3> Reduce the list of paths to the shortest path between the first and last vertex by ordering on their lengths.
-<4> Recall that at this point, there is a `Map` keyed by first and last vertex and with a value of just the shortest
-path. Extract the shortest path with `select(values)`, since that's the only portion required for the remainder of
-the traversal.
-<5> The first `by()` modulator for `project()` extracts the first vertex in the path.
-<6> The second `by()` modulator for `project()` extracts the path length and divides that distance by the value of
-the `sack()` which was initialized to 1 at the start of the traversal.
-<7> Group the resulting `Map` objects on "v" and sum their lengths to get the centrality score for each.
+<1> Defines a Gremlin link:http://tinkerpop.apache.org/docs/x.y.z/reference/#sack-step[sack] with a value of one.
+<2> Traverses on both - incoming and outgoing - edges, avoiding <<cycle-detection, cyclic paths>>.
+<3> Create a triple consisting of the first vertex, the last vertex and the length of the path between them.
+<4> Determine whether a path between those two vertices was already found.
+<5> If this is the first path between the two vertices, store the triple in an internal collection named "triples".
+<6> Keep only those paths between a pair of vertices that have the same length as the first path that was found between them.
+<7> For each vertex divide 1 by the product of the lengths of all shortest paths that start with this particular vertex.
+
+WARNING: Since the closeness centrality algorithm requires the shortest path between any pair of vertices in the graph,
+its practical applications are very limited. It's recommended to use this algorithm only on small subgraphs (graphs like
+the link:http://tinkerpop.apache.org/docs/current/reference/#grateful-dead[Grateful Dead graph] with only 808 vertices
+and 8049 edges already require a massive amount of compute resources to determine the shortest paths between all vertex
+pairs).
[[eigenvector-centrality]]
Eigenvector Centrality
@@ -165,4 +178,4 @@ link:http://tinkerpop.apache.org/docs/current/reference/#timelimit-step[time lim
traversal from taking longer than one hundred milliseconds to execute (the previous example takes considerably longer
than that). While the answer provided with the `timeLimit()` is not the absolute ranking, it does provide a relative
ranking that closely matches the absolute one. The use of `timeLimit()` in certain algorithms (e.g. recommendations)
-can shorten the time required to get a reasonable and usable result.
\ No newline at end of file
+can shorten the time required to get a reasonable and usable result.
[11/29] tinkerpop git commit: Fixed how plugins are bound to a
scriptengine.
Posted by rd...@apache.org.
Fixed how plugins are bound to a scriptengine.
Didn't have the reflection code right for finding the appliesTo method CTR
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/28dffda6
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/28dffda6
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/28dffda6
Branch: refs/heads/TINKERPOP-1602
Commit: 28dffda6b55dae775215ed08d5bcd2cfa280b07b
Parents: 956f350
Author: Stephen Mallette <sp...@genoprime.com>
Authored: Wed Jan 25 13:20:53 2017 -0500
Committer: Stephen Mallette <sp...@genoprime.com>
Committed: Wed Jan 25 13:20:53 2017 -0500
----------------------------------------------------------------------
.../apache/tinkerpop/gremlin/groovy/engine/GremlinExecutor.java | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/28dffda6/gremlin-groovy/src/main/java/org/apache/tinkerpop/gremlin/groovy/engine/GremlinExecutor.java
----------------------------------------------------------------------
diff --git a/gremlin-groovy/src/main/java/org/apache/tinkerpop/gremlin/groovy/engine/GremlinExecutor.java b/gremlin-groovy/src/main/java/org/apache/tinkerpop/gremlin/groovy/engine/GremlinExecutor.java
index 349af8e..8fdb86e 100644
--- a/gremlin-groovy/src/main/java/org/apache/tinkerpop/gremlin/groovy/engine/GremlinExecutor.java
+++ b/gremlin-groovy/src/main/java/org/apache/tinkerpop/gremlin/groovy/engine/GremlinExecutor.java
@@ -40,6 +40,7 @@ import java.io.FileReader;
import java.io.IOException;
import java.lang.reflect.Method;
import java.util.ArrayList;
+import java.util.Collection;
import java.util.Collections;
import java.util.HashMap;
import java.util.HashSet;
@@ -479,8 +480,8 @@ public class GremlinExecutor implements AutoCloseable {
}
try {
- final Method appliesTo = builderClazz.getMethod("appliesTo");
- pluginBuilder = appliesTo.invoke(pluginBuilder, language);
+ final Method appliesTo = builderClazz.getMethod("appliesTo", Collection.class);
+ pluginBuilder = appliesTo.invoke(pluginBuilder, Collections.singletonList(language));
} catch (NoSuchMethodException ignored) {
}
[05/29] tinkerpop git commit: Merge branch 'centrality-recipes' into
tp32
Posted by rd...@apache.org.
Merge branch 'centrality-recipes' into tp32
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/9d883043
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/9d883043
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/9d883043
Branch: refs/heads/TINKERPOP-1602
Commit: 9d883043ed3123af5b1c614b343cbbdd0dc9013f
Parents: aa262d6 1756461
Author: Daniel Kuppitz <da...@hotmail.com>
Authored: Tue Jan 24 11:00:26 2017 +0100
Committer: Daniel Kuppitz <da...@hotmail.com>
Committed: Tue Jan 24 11:00:26 2017 +0100
----------------------------------------------------------------------
docs/src/recipes/centrality.asciidoc | 121 ++++++++++++++-----------
docs/static/images/betweeness-example.png | Bin 8465 -> 21283 bytes
2 files changed, 67 insertions(+), 54 deletions(-)
----------------------------------------------------------------------
[09/29] tinkerpop git commit: Fixed a bug where keepLabels were not
being pushed down into child traversals by PathRetractionStrategy.
Posted by rd...@apache.org.
Fixed a bug where keepLabels were not being pushed down into child traversals by PathRetractionStrategy.
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/da762dfe
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/da762dfe
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/da762dfe
Branch: refs/heads/TINKERPOP-1602
Commit: da762dfee9b0ed05fd1185d80403e1be41873b58
Parents: e3889bf
Author: Ted Wilmes <tw...@gmail.com>
Authored: Tue Jan 24 14:57:07 2017 -0600
Committer: Ted Wilmes <tw...@gmail.com>
Committed: Tue Jan 24 14:57:07 2017 -0600
----------------------------------------------------------------------
CHANGELOG.asciidoc | 1 +
.../optimization/PathRetractionStrategy.java | 58 +++++++++++++++++---
.../PathRetractionStrategyTest.java | 19 +++----
3 files changed, 60 insertions(+), 18 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/da762dfe/CHANGELOG.asciidoc
----------------------------------------------------------------------
diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc
index 9453158..95cfb71 100644
--- a/CHANGELOG.asciidoc
+++ b/CHANGELOG.asciidoc
@@ -26,6 +26,7 @@ image::https://raw.githubusercontent.com/apache/tinkerpop/master/docs/static/ima
TinkerPop 3.2.4 (Release Date: NOT OFFICIALLY RELEASED YET)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+* Fixed a bug where `keepLabels` were not being pushed down into child traversals by `PathRetractionStrategy`.
* Fixed a bug associated with user-provided maps and `GroupSideEffectStep`.
* `GroupBiOperator` no longer maintains a detached traversal and thus, no more side-effect related OLAP inconsistencies.
* Added `ProjectedTraverser` which wraps a traverser with a `List<Object>` of projected data.
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/da762dfe/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategy.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategy.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategy.java
index fcc22a4..fc7eb8a 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategy.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategy.java
@@ -24,6 +24,7 @@ import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategy;
import org.apache.tinkerpop.gremlin.process.traversal.step.Barrier;
import org.apache.tinkerpop.gremlin.process.traversal.step.LambdaHolder;
import org.apache.tinkerpop.gremlin.process.traversal.step.PathProcessor;
+import org.apache.tinkerpop.gremlin.process.traversal.step.TraversalParent;
import org.apache.tinkerpop.gremlin.process.traversal.step.branch.RepeatStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.filter.DedupGlobalStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.MatchStep;
@@ -33,6 +34,7 @@ import org.apache.tinkerpop.gremlin.process.traversal.strategy.AbstractTraversal
import org.apache.tinkerpop.gremlin.process.traversal.traverser.TraverserRequirement;
import org.apache.tinkerpop.gremlin.process.traversal.util.PathUtil;
import org.apache.tinkerpop.gremlin.process.traversal.util.TraversalHelper;
+import org.apache.tinkerpop.gremlin.util.iterator.IteratorUtils;
import org.javatuples.Pair;
import java.util.ArrayList;
@@ -96,8 +98,12 @@ public final class PathRetractionStrategy extends AbstractTraversalStrategy<Trav
if (currentStep instanceof MatchStep && (currentStep.getNextStep().equals(EmptyStep.instance()) || currentStep.getNextStep() instanceof DedupGlobalStep)) {
pathProcessor.setKeepLabels(((MatchStep) currentStep).getMatchStartLabels());
pathProcessor.getKeepLabels().addAll(((MatchStep) currentStep).getMatchEndLabels());
- } else
- pathProcessor.setKeepLabels(new HashSet<>(keepLabels));
+ } else {
+ if (pathProcessor.getKeepLabels() == null)
+ pathProcessor.setKeepLabels(new HashSet<>(keepLabels));
+ else
+ pathProcessor.getKeepLabels().addAll(new HashSet<>(keepLabels));
+ }
if (currentStep.getTraversal().getParent() instanceof MatchStep) {
pathProcessor.setKeepLabels(((MatchStep) currentStep.getTraversal().getParent().asStep()).getMatchStartLabels());
@@ -111,6 +117,7 @@ public final class PathRetractionStrategy extends AbstractTraversalStrategy<Trav
!(currentStep.getNextStep() instanceof Barrier) &&
!(currentStep.getTraversal().getParent() instanceof MatchStep) &&
(!(currentStep.getNextStep() instanceof EmptyStep) || TraversalHelper.isGlobalChild(currentStep.getTraversal())))
+
TraversalHelper.insertAfterStep(new NoOpBarrierStep<>(traversal, this.standardBarrierSize), currentStep, traversal);
}
}
@@ -141,16 +148,29 @@ public final class PathRetractionStrategy extends AbstractTraversalStrategy<Trav
hasRepeat = true;
}
+ // if parent step is a TraversalParent itself and it has more than 1 child traversal
+ // propagate labels to children
+ if (step instanceof TraversalParent) {
+ final List<Traversal.Admin<Object, Object>> children = new ArrayList<>();
+ children.addAll(((TraversalParent) step).getGlobalChildren());
+ children.addAll(((TraversalParent) step).getLocalChildren());
+ // if this is the only child traversal, do not re-push labels
+ if (children.size() > 1)
+ applyToChildren(keepLabels, children);
+ }
+
// propagate requirements of keep labels back through the traversal's previous steps
// to ensure that the label is not dropped before it reaches the step(s) that require it
step = step.getPreviousStep();
while (!(step.equals(EmptyStep.instance()))) {
if (step instanceof PathProcessor) {
- if (((PathProcessor) step).getKeepLabels() == null) {
- ((PathProcessor) step).setKeepLabels(new HashSet<>(keepLabels));
- } else {
- ((PathProcessor) step).getKeepLabels().addAll(new HashSet<>(keepLabels));
- }
+ addLabels((PathProcessor) step, keepLabels);
+ }
+ if (step instanceof TraversalParent) {
+ final List<Traversal.Admin<Object, Object>> children = new ArrayList<>();
+ children.addAll(((TraversalParent) step).getGlobalChildren());
+ children.addAll(((TraversalParent) step).getLocalChildren());
+ applyToChildren(keepLabels, children);
}
step = step.getPreviousStep();
}
@@ -190,8 +210,30 @@ public final class PathRetractionStrategy extends AbstractTraversalStrategy<Trav
}
}
+ private void applyToChildren(Set<String> keepLabels, List<Traversal.Admin<Object, Object>> children) {
+ for (Traversal.Admin<Object, Object> child : children) {
+ TraversalHelper.applyTraversalRecursively(trav -> addLabels(trav, keepLabels), child);
+ }
+ }
+
+ private void addLabels(final Traversal.Admin traversal, final Set<String> keepLabels) {
+ for (Object s : traversal.getSteps()) {
+ if (s instanceof PathProcessor) {
+ addLabels((PathProcessor) s, keepLabels);
+ }
+ }
+ }
+
+ private void addLabels(PathProcessor s, Set<String> keepLabels) {
+ if (s.getKeepLabels() == null) {
+ s.setKeepLabels(new HashSet<>(keepLabels));
+ } else {
+ s.getKeepLabels().addAll(new HashSet<>(keepLabels));
+ }
+ }
+
@Override
public Set<Class<? extends OptimizationStrategy>> applyPrior() {
return PRIORS;
}
-}
+}
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/da762dfe/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategyTest.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategyTest.java b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategyTest.java
index fc1bc10..71b0ad5 100644
--- a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategyTest.java
+++ b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategyTest.java
@@ -37,14 +37,8 @@ import java.util.Arrays;
import java.util.List;
import java.util.Set;
-import static org.apache.tinkerpop.gremlin.process.traversal.P.eq;
-import static org.apache.tinkerpop.gremlin.process.traversal.P.gte;
-import static org.apache.tinkerpop.gremlin.process.traversal.P.neq;
-import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.as;
-import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.limit;
-import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.out;
-import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.select;
-import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.values;
+import static org.apache.tinkerpop.gremlin.process.traversal.P.*;
+import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.*;
import static org.apache.tinkerpop.gremlin.process.traversal.strategy.optimization.PathRetractionStrategy.MAX_BARRIER_SIZE;
import static org.junit.Assert.assertEquals;
@@ -153,7 +147,7 @@ public class PathRetractionStrategyTest {
as("c").out().as("d", "e").select("c", "e").out().select("c"),
as("c").out().as("d", "e").select("c", "e").out().select("c")).
out().select("c"),
- "[[b, c, e], [c, e], [[c], [c]], [[c], [c]], []]", null},
+ "[[b, c, e], [c, e], [[c, e], [c, e]], [[c, e], [c, e]], []]", null},
{__.V().as("a").out().as("b").select("a").select("b").
local(as("c").out().as("d", "e").select("c", "e").out().select("c")).
out().select("c"),
@@ -180,7 +174,12 @@ public class PathRetractionStrategyTest {
{__.V().out("created").project("a", "b").by("name").by(__.in("created").count()).order().by(select("b")).select("a"), "[[[a]], []]", null},
{__.order().by("weight", Order.decr).store("w").by("weight").filter(values("weight").as("cw").
select("w").by(limit(Scope.local, 1)).as("mw").where("cw", eq("mw"))).project("from", "to", "weight").by(__.outV()).by(__.inV()).by("weight"),
- "[[[cw, mw], []]]", null}
+ "[[[cw, mw], []]]", null},
+ {__.V().limit(1).as("z").out().repeat(store("seen").out().where(without("seen"))).until(where(eq("z"))),
+ "[[[z, seen]], [[z, seen]]]", null},
+ {__.V().as("a").optional(bothE().dedup().as("b")).
+ choose(select("b"), select("a","b"), project("a").by(select("a"))),
+ "[[[a, b]], [[a, b]], [[a, b]], [[[a, b]]], [[a, b]]]", null}
});
}
}
[23/29] tinkerpop git commit: done for day. minor clean up push.
nothing major.
Posted by rd...@apache.org.
done for day. minor clean up push. nothing major.
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/9dee11d6
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/9dee11d6
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/9dee11d6
Branch: refs/heads/TINKERPOP-1602
Commit: 9dee11d6534943e1953016a3e0a92549609d7252
Parents: 989237f
Author: Marko A. Rodriguez <ok...@gmail.com>
Authored: Thu Jan 26 15:10:49 2017 -0700
Committer: Marko A. Rodriguez <ok...@gmail.com>
Committed: Fri Jan 27 14:24:18 2017 -0700
----------------------------------------------------------------------
.../optimization/SingleIterationStrategy.java | 56 +++++++++-----------
1 file changed, 25 insertions(+), 31 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/9dee11d6/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategy.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategy.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategy.java
index 19d9854..eed1104 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategy.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategy.java
@@ -28,7 +28,6 @@ import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategy;
import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.DefaultGraphTraversal;
import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__;
import org.apache.tinkerpop.gremlin.process.traversal.step.Barrier;
-import org.apache.tinkerpop.gremlin.process.traversal.step.LambdaHolder;
import org.apache.tinkerpop.gremlin.process.traversal.step.SideEffectCapable;
import org.apache.tinkerpop.gremlin.process.traversal.step.TraversalParent;
import org.apache.tinkerpop.gremlin.process.traversal.step.branch.LocalStep;
@@ -91,36 +90,31 @@ public final class SingleIterationStrategy extends AbstractTraversalStrategy<Tra
break;
}
}
- } // if the traversal doesn't message pass, then don't try and localize it as its just wasted computation
- if (doesMessagePass) {
- final boolean beyondStarGraph =
- TraversalHelper.hasStepOfAssignableClassRecursively(Scope.global, LambdaHolder.class, computerTraversal) ||
- !TraversalHelper.isLocalStarGraph(computerTraversal);
- if (!beyondStarGraph && // if we move beyond the star graph, then localization is not possible.
- (computerTraversal.getStartStep() instanceof GraphStep) && // while GraphComputer requires GraphStep starts, this is just a precaution when inject() starts are supported
- !(computerTraversal.getStartStep().getNextStep() instanceof EmptyStep) && // if its just a g.V()/E(), then don't localize
- !(computerTraversal.getStartStep().getNextStep() instanceof LocalStep) && // removes the potential for the infinite recursive application of the traversal
- !(computerTraversal.getStartStep().getNextStep() instanceof Barrier) && // if the second step is a barrier, no point in trying to localize anything
- !computerTraversal.getTraverserRequirements().contains(TraverserRequirement.LABELED_PATH) && // this is to alleviate issues with DetachedElement in paths (TODO: when detachment is dynamic, remove this)
- !computerTraversal.getTraverserRequirements().contains(TraverserRequirement.PATH) && // this is to alleviate issues with DetachedElement in paths (TODO: when detachment is dynamic, remove this)
- TraversalHelper.getStepsOfAssignableClassRecursively(SideEffectCapable.class, computerTraversal).isEmpty() && // this is to alleviate issues with DetachedElement in paths (TODO: when detachment is dynamic, remove this)
- !(TraversalHelper.getStepsOfAssignableClass(TraversalParent.class, computerTraversal). // this is a strict precaution that could be loosed with deeper logic on barriers in global children
- stream().
- filter(parent -> !parent.getGlobalChildren().isEmpty()).findAny().isPresent())) {
-
- final Traversal.Admin<?, ?> newComputerTraversal = step.computerTraversal.getPure();
- final Traversal.Admin localTraversal = new DefaultGraphTraversal<>();
- final Step barrier = (Step) TraversalHelper.getFirstStepOfAssignableClass(Barrier.class, newComputerTraversal).orElse(null);
- if (null == barrier || !(barrier instanceof TraversalParent && (barrier.getPreviousStep() instanceof VertexStep || barrier.getPreviousStep() instanceof EdgeVertexStep))) {
- TraversalHelper.removeToTraversal(newComputerTraversal.getStartStep().getNextStep(), null == barrier ? EmptyStep.instance() : barrier, localTraversal);
- assert !localTraversal.getSteps().isEmpty(); // given the if() constraints, this is impossible
- if (localTraversal.getSteps().size() > 1) { // if its just a single step, a local wrap will not alter its locus of computation
- if (null == barrier)
- TraversalHelper.insertTraversal(0, (Traversal.Admin) __.local(localTraversal), newComputerTraversal);
- else
- TraversalHelper.insertTraversal(barrier.getPreviousStep(), (Traversal.Admin) __.local(localTraversal), newComputerTraversal);
- step.setComputerTraversal(newComputerTraversal);
- }
+ }
+ if (doesMessagePass && // if the traversal doesn't message pass, then don't try and localize it as its just wasted computation
+ TraversalHelper.isLocalStarGraph(computerTraversal) && // if we move beyond the star graph, then localization is not possible.
+ (computerTraversal.getStartStep() instanceof GraphStep) && // while GraphComputer requires GraphStep starts, this is just a precaution when inject() starts are supported
+ !(computerTraversal.getStartStep().getNextStep() instanceof EmptyStep) && // if its just a g.V()/E(), then don't localize
+ !(computerTraversal.getStartStep().getNextStep() instanceof LocalStep) && // removes the potential for the infinite recursive application of the traversal
+ !(computerTraversal.getStartStep().getNextStep() instanceof Barrier) && // if the second step is a barrier, no point in trying to localize anything
+ !computerTraversal.getTraverserRequirements().contains(TraverserRequirement.LABELED_PATH) && // this is to alleviate issues with DetachedElement in paths (TODO: when detachment is dynamic, remove this)
+ !computerTraversal.getTraverserRequirements().contains(TraverserRequirement.PATH) && // this is to alleviate issues with DetachedElement in paths (TODO: when detachment is dynamic, remove this)
+ TraversalHelper.getStepsOfAssignableClassRecursively(SideEffectCapable.class, computerTraversal).isEmpty() && // this is to alleviate issues with DetachedElement in paths (TODO: when detachment is dynamic, remove this)
+ !(TraversalHelper.getStepsOfAssignableClass(TraversalParent.class, computerTraversal). // this is a strict precaution that could be loosed with deeper logic on barriers in global children
+ stream().
+ filter(parent -> !parent.getGlobalChildren().isEmpty()).findAny().isPresent())) {
+ final Traversal.Admin<?, ?> newComputerTraversal = step.computerTraversal.getPure();
+ final Traversal.Admin localTraversal = new DefaultGraphTraversal<>();
+ final Step barrier = (Step) TraversalHelper.getFirstStepOfAssignableClass(Barrier.class, newComputerTraversal).orElse(null);
+ if (null == barrier || !(barrier instanceof TraversalParent && (barrier.getPreviousStep() instanceof VertexStep || barrier.getPreviousStep() instanceof EdgeVertexStep))) {
+ TraversalHelper.removeToTraversal(newComputerTraversal.getStartStep().getNextStep(), null == barrier ? EmptyStep.instance() : barrier, localTraversal);
+ assert !localTraversal.getSteps().isEmpty(); // given the if() constraints, this is impossible
+ if (localTraversal.getSteps().size() > 1) { // if its just a single step, a local wrap will not alter its locus of computation
+ if (null == barrier)
+ TraversalHelper.insertTraversal(0, (Traversal.Admin) __.local(localTraversal), newComputerTraversal);
+ else
+ TraversalHelper.insertTraversal(barrier.getPreviousStep(), (Traversal.Admin) __.local(localTraversal), newComputerTraversal);
+ step.setComputerTraversal(newComputerTraversal);
}
}
}
[03/29] tinkerpop git commit: Fixed up betweeness example graphic -
for real this time.
Posted by rd...@apache.org.
Fixed up betweeness example graphic - for real this time.
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/1756461c
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/1756461c
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/1756461c
Branch: refs/heads/TINKERPOP-1602
Commit: 1756461cef6f51a1610702c813b759b40de2a849
Parents: 841fc30
Author: Stephen Mallette <sp...@genoprime.com>
Authored: Mon Jan 23 11:30:44 2017 -0500
Committer: Stephen Mallette <sp...@genoprime.com>
Committed: Mon Jan 23 11:30:44 2017 -0500
----------------------------------------------------------------------
docs/static/images/betweeness-example.png | Bin 9961 -> 21283 bytes
1 file changed, 0 insertions(+), 0 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/1756461c/docs/static/images/betweeness-example.png
----------------------------------------------------------------------
diff --git a/docs/static/images/betweeness-example.png b/docs/static/images/betweeness-example.png
index a086a65..fa58b5d 100755
Binary files a/docs/static/images/betweeness-example.png and b/docs/static/images/betweeness-example.png differ
[25/29] tinkerpop git commit: updated doc for ssl client auth
Posted by rd...@apache.org.
updated doc for ssl client auth
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/9b279c6b
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/9b279c6b
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/9b279c6b
Branch: refs/heads/TINKERPOP-1602
Commit: 9b279c6b738f1cbe8eceab88f0269ceb971b615a
Parents: c62480b
Author: Robert Dale <ro...@gmail.com>
Authored: Tue Jan 17 14:39:32 2017 -0500
Committer: Robert Dale <ro...@gmail.com>
Committed: Fri Jan 27 16:26:32 2017 -0500
----------------------------------------------------------------------
docs/src/reference/gremlin-applications.asciidoc | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/9b279c6b/docs/src/reference/gremlin-applications.asciidoc
----------------------------------------------------------------------
diff --git a/docs/src/reference/gremlin-applications.asciidoc b/docs/src/reference/gremlin-applications.asciidoc
index e515ef7..2454581 100644
--- a/docs/src/reference/gremlin-applications.asciidoc
+++ b/docs/src/reference/gremlin-applications.asciidoc
@@ -1036,7 +1036,8 @@ The following table describes the various configuration options that Gremlin Ser
|ssl.keyCertChainFile |The X.509 certificate chain file in PEM format. If this value is not present and `ssl.enabled` is `true` a self-signed certificate will be used (not suitable for production). |_none_
|ssl.keyFile |The `PKCS#8` private key file in PEM format. If this value is not present and `ssl.enabled` is `true` a self-signed certificate will be used (not suitable for production). |_none_
|ssl.keyPassword |The password of the `keyFile` if it is password-protected |_none_
-|ssl.trustCertChainFile |Trusted certificates for verifying the remote endpoint's certificate. The file should contain an X.509 certificate chain in PEM format. A system default will be used if this setting is not present. (Not supported) |_none_
+|ssl.needClientAuth | Optional. One of NONE, OPTIONAL, REQUIRE. Enables client certificate authentication at the level specified. Can be used in combination with Authenticator. |_none_
+|ssl.trustCertChainFile | Required when needClientAuth is OPTIONAL or REQUIRE. Trusted certificates for verifying the remote endpoint's certificate. The file should contain an X.509 certificate chain in PEM format. |_none_
|strictTransactionManagement |Set to `true` to require `aliases` to be submitted on every requests, where the `aliases` become the scope of transaction management. |false
|threadPoolBoss |The number of threads available to Gremlin Server for accepting connections. Should always be set to `1`. |1
|threadPoolWorker |The number of threads available to Gremlin Server for processing non-blocking reads and writes. |1
[15/29] tinkerpop git commit: Added check for
MatchStep.Select.FilterStep pattern so that MatchStep start and end keys are
preserved correctly.
Posted by rd...@apache.org.
Added check for MatchStep.Select.FilterStep pattern so that MatchStep start and end keys are preserved correctly.
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/cf7a0777
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/cf7a0777
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/cf7a0777
Branch: refs/heads/TINKERPOP-1602
Commit: cf7a0777d4a1d0053e5b6c5a8e7fa6ca5166d05a
Parents: da762df
Author: Ted Wilmes <tw...@gmail.com>
Authored: Fri Jan 27 08:21:34 2017 -0600
Committer: Ted Wilmes <tw...@gmail.com>
Committed: Fri Jan 27 08:21:34 2017 -0600
----------------------------------------------------------------------
.../strategy/optimization/PathRetractionStrategy.java | 9 ++++++++-
.../strategy/optimization/PathRetractionStrategyTest.java | 2 +-
2 files changed, 9 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/cf7a0777/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategy.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategy.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategy.java
index fc7eb8a..da6fd0e 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategy.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategy.java
@@ -27,8 +27,12 @@ import org.apache.tinkerpop.gremlin.process.traversal.step.PathProcessor;
import org.apache.tinkerpop.gremlin.process.traversal.step.TraversalParent;
import org.apache.tinkerpop.gremlin.process.traversal.step.branch.RepeatStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.filter.DedupGlobalStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.filter.FilterStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.filter.TraversalFilterStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.filter.WhereTraversalStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.MatchStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.NoOpBarrierStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.map.SelectOneStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.util.EmptyStep;
import org.apache.tinkerpop.gremlin.process.traversal.strategy.AbstractTraversalStrategy;
import org.apache.tinkerpop.gremlin.process.traversal.traverser.TraverserRequirement;
@@ -95,7 +99,10 @@ public final class PathRetractionStrategy extends AbstractTraversalStrategy<Trav
// add the keep labels to the path processor
if (currentStep instanceof PathProcessor) {
final PathProcessor pathProcessor = (PathProcessor) currentStep;
- if (currentStep instanceof MatchStep && (currentStep.getNextStep().equals(EmptyStep.instance()) || currentStep.getNextStep() instanceof DedupGlobalStep)) {
+ if (currentStep instanceof MatchStep &&
+ (currentStep.getNextStep().equals(EmptyStep.instance()) ||
+ currentStep.getNextStep() instanceof DedupGlobalStep ||
+ currentStep.getNextStep() instanceof SelectOneStep && currentStep.getNextStep().getNextStep() instanceof FilterStep)) {
pathProcessor.setKeepLabels(((MatchStep) currentStep).getMatchStartLabels());
pathProcessor.getKeepLabels().addAll(((MatchStep) currentStep).getMatchEndLabels());
} else {
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/cf7a0777/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategyTest.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategyTest.java b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategyTest.java
index 71b0ad5..d12b621 100644
--- a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategyTest.java
+++ b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategyTest.java
@@ -179,7 +179,7 @@ public class PathRetractionStrategyTest {
"[[[z, seen]], [[z, seen]]]", null},
{__.V().as("a").optional(bothE().dedup().as("b")).
choose(select("b"), select("a","b"), project("a").by(select("a"))),
- "[[[a, b]], [[a, b]], [[a, b]], [[[a, b]]], [[a, b]]]", null}
+ "[[[a, b]], [[a, b]], [[a, b]], [[[a, b]]], [[a, b]]]", null},
});
}
}
[10/29] tinkerpop git commit: Fixed a gremlin server bug with global
bindings
Posted by rd...@apache.org.
Fixed a gremlin server bug with global bindings
TraversalSources established in init scripts using the new plugin system weren't getting set to the GraphManager given the fact that the GremlinScriptEngineManager does runs the scripts when the GremlinScriptEngine is first initialized. By forcing the GremlinScriptEngine to initialize with a throw-away script the init scripts will execute. This is a bit of a hack - will consider other solutions for 3.3.0. CTR
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/956f3505
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/956f3505
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/956f3505
Branch: refs/heads/TINKERPOP-1602
Commit: 956f35056a967ecf1d8e74dc04bb227ba4823047
Parents: d08a68f
Author: Stephen Mallette <sp...@genoprime.com>
Authored: Wed Jan 25 09:24:49 2017 -0500
Committer: Stephen Mallette <sp...@genoprime.com>
Committed: Wed Jan 25 09:24:49 2017 -0500
----------------------------------------------------------------------
.../tinkerpop/gremlin/server/util/ServerGremlinExecutor.java | 6 ++++++
1 file changed, 6 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/956f3505/gremlin-server/src/main/java/org/apache/tinkerpop/gremlin/server/util/ServerGremlinExecutor.java
----------------------------------------------------------------------
diff --git a/gremlin-server/src/main/java/org/apache/tinkerpop/gremlin/server/util/ServerGremlinExecutor.java b/gremlin-server/src/main/java/org/apache/tinkerpop/gremlin/server/util/ServerGremlinExecutor.java
index 2b2f1f1..199c9f8 100644
--- a/gremlin-server/src/main/java/org/apache/tinkerpop/gremlin/server/util/ServerGremlinExecutor.java
+++ b/gremlin-server/src/main/java/org/apache/tinkerpop/gremlin/server/util/ServerGremlinExecutor.java
@@ -145,6 +145,12 @@ public class ServerGremlinExecutor<T extends ScheduledExecutorService> {
logger.info("Initialized GremlinExecutor and configured ScriptEngines.");
+ // force each scriptengine to process something so that the init scripts will fire (this is necessary if
+ // the GremlinExecutor is using the GremlinScriptEngineManager. this is a bit of hack, but it at least allows
+ // the global bindings to become available after the init scripts are run (DefaultGremlinScriptEngineManager
+ // runs the init scripts when the GremlinScriptEngine is created.
+ settings.scriptEngines.keySet().forEach(engineName -> gremlinExecutor.eval("1+1", engineName, Collections.emptyMap()).join());
+
// script engine init may have altered the graph bindings or maybe even created new ones - need to
// re-apply those references back
gremlinExecutor.getGlobalBindings().entrySet().stream()
[04/29] tinkerpop git commit: TINKERPOP-1433 CTR: added release
manager key setup docs.
Posted by rd...@apache.org.
TINKERPOP-1433 CTR: added release manager key setup docs.
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/aa262d6b
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/aa262d6b
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/aa262d6b
Branch: refs/heads/TINKERPOP-1602
Commit: aa262d6bdfc6177b24bcced4346e1cc2f2cb505f
Parents: fe5f557
Author: Ted Wilmes <tw...@gmail.com>
Authored: Mon Jan 23 16:47:00 2017 -0600
Committer: Ted Wilmes <tw...@gmail.com>
Committed: Mon Jan 23 16:47:00 2017 -0600
----------------------------------------------------------------------
docs/src/dev/developer/release.asciidoc | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/aa262d6b/docs/src/dev/developer/release.asciidoc
----------------------------------------------------------------------
diff --git a/docs/src/dev/developer/release.asciidoc b/docs/src/dev/developer/release.asciidoc
index ec1fdf6..b78e860 100644
--- a/docs/src/dev/developer/release.asciidoc
+++ b/docs/src/dev/developer/release.asciidoc
@@ -22,6 +22,7 @@ manager" (a committer fulfills this role), who ensures that the steps in this do
multi-phased and can therefore take several weeks to complete given the time needed for Apache voting and community
feedback. Once a release point has been identified, the following phases represent the flow of "release":
+* Release manager key setup.
* Pre-flight check.
* Optionally, produce a release candidate for community feedback.
* Submit the official release for PMC vote.
@@ -66,6 +67,28 @@ IMPORTANT: The `clean` in the above commands is more important to the pypi deplo
anything found in the `target/python-packaged/dist` directory. Since the names of the artifacts are based on
timestamps, they will not overwrite one another and multiple artifacts will get uploaded.
+Release Manager Requirements
+----------------------------
+
+If this is your first time as release manager, you will need to setup keys for signing purposes per the Apache
+release process. Generally speaking, this will mean that you will need to generate a key-pair and then upload your
+public key to a public keyserver.
+
+For a general overview of key basics, refer to link:https://www.apache.org/dev/release-signing.html#key-basics[this]. For detailed
+step-by-step instructions, please follow the instructions link:https://www.apache.org/dev/openpgp.html#generate-key[here].
+
+After completing the key-pair setup instructions, be sure to add yourself to the `PGP signature` section of `bin/validate-distribution.sh`.
+
+[source,text]
+----
+echo -n " * PGP signature ... "
+[ `gpg ${ZIP_FILENAME}.asc 2>&1 | grep -c '^gpg: Good signature from "Stephen Mallette <sp...@apache.org>"$'` -eq 1 ] || \
+[ `gpg ${ZIP_FILENAME}.asc 2>&1 | grep -c '^gpg: Good signature from "Marko Rodriguez <ok...@apache.org>"$'` -eq 1 ] || \
+[ `gpg ${ZIP_FILENAME}.asc 2>&1 | grep -c '^gpg: Good signature from "Theodore Ratte Wilmes (CODE SIGNING KEY) <tw...@apache.org>"'` -eq 1 ] || \
+{ echo "failed"; exit 1; }
+echo "OK"
+----
+
Pre-flight Check
----------------
[02/29] tinkerpop git commit: Fixed up betweeness graphic based on
the revised example.
Posted by rd...@apache.org.
Fixed up betweeness graphic based on the revised example.
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/841fc30a
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/841fc30a
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/841fc30a
Branch: refs/heads/TINKERPOP-1602
Commit: 841fc30a5e8f4b9c1b98ca5056c26d0d26b67115
Parents: 5577ea5
Author: Stephen Mallette <sp...@genoprime.com>
Authored: Mon Jan 23 11:15:30 2017 -0500
Committer: Stephen Mallette <sp...@genoprime.com>
Committed: Mon Jan 23 11:15:30 2017 -0500
----------------------------------------------------------------------
docs/static/images/betweeness-example.png | Bin 8465 -> 9961 bytes
1 file changed, 0 insertions(+), 0 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/841fc30a/docs/static/images/betweeness-example.png
----------------------------------------------------------------------
diff --git a/docs/static/images/betweeness-example.png b/docs/static/images/betweeness-example.png
old mode 100644
new mode 100755
index 650ee53..a086a65
Binary files a/docs/static/images/betweeness-example.png and b/docs/static/images/betweeness-example.png differ
[06/29] tinkerpop git commit: Some minor fixes to plugin
infrastructure
Posted by rd...@apache.org.
Some minor fixes to plugin infrastructure
Graph plugins needed instance() methods to be properly instantiated at runtime. Hooked up ScriptCustomizer in the GremlinScriptEngineManager. Added BindingsGremlinPlugin to expose the various "bindings" customizers. None of these changes are breaking and really aren't meant for usage alongs the 3.2.x line. CTR
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/e3889bf2
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/e3889bf2
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/e3889bf2
Branch: refs/heads/TINKERPOP-1602
Commit: e3889bf2401b42c3afbc85eabc2fbcebf2588974
Parents: 9d88304
Author: Stephen Mallette <sp...@genoprime.com>
Authored: Tue Jan 24 11:05:13 2017 -0500
Committer: Stephen Mallette <sp...@genoprime.com>
Committed: Tue Jan 24 11:09:12 2017 -0500
----------------------------------------------------------------------
.../gremlin/jsr223/BindingsGremlinPlugin.java | 63 ++++++++++++++++++++
.../DefaultGremlinScriptEngineManager.java | 20 +++++++
.../gremlin/jsr223/ScriptCustomizer.java | 4 ++
.../gremlin/groovy/engine/GremlinExecutor.java | 11 +++-
.../neo4j/jsr223/Neo4jGremlinPlugin.java | 6 ++
.../spark/jsr223/SparkGremlinPlugin.java | 6 ++
.../jsr223/TinkerGraphGremlinPlugin.java | 42 +++++++------
7 files changed, 132 insertions(+), 20 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/e3889bf2/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/BindingsGremlinPlugin.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/BindingsGremlinPlugin.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/BindingsGremlinPlugin.java
new file mode 100644
index 0000000..feb501d
--- /dev/null
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/BindingsGremlinPlugin.java
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.gremlin.jsr223;
+
+import javax.script.Bindings;
+import javax.script.SimpleBindings;
+import java.util.Map;
+import java.util.function.Supplier;
+
+/**
+ * @author Stephen Mallette (http://stephen.genoprime.com)
+ */
+public class BindingsGremlinPlugin extends AbstractGremlinPlugin {
+ private static final String NAME = "tinkerpop.bindings";
+
+ private BindingsGremlinPlugin(final Builder builder) {
+ this(builder.bindings);
+ }
+
+ public BindingsGremlinPlugin(final Bindings bindings) {
+ super(NAME, new DefaultBindingsCustomizer(bindings));
+ }
+
+ public BindingsGremlinPlugin(final Supplier<Bindings> bindingsSupplier) {
+ super(NAME, new LazyBindingsCustomizer(bindingsSupplier));
+ }
+
+ public static BindingsGremlinPlugin.Builder build() {
+ return new Builder();
+ }
+
+ public static final class Builder {
+
+ private Bindings bindings = new SimpleBindings();
+
+ private Builder() {}
+
+ public Builder bindings(final Map<String, Object> bindings) {
+ this.bindings = new SimpleBindings(bindings);
+ return this;
+ }
+
+ public BindingsGremlinPlugin create() {
+ return new BindingsGremlinPlugin(this);
+ }
+ }
+}
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/e3889bf2/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/DefaultGremlinScriptEngineManager.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/DefaultGremlinScriptEngineManager.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/DefaultGremlinScriptEngineManager.java
index 86b72f2..436deac 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/DefaultGremlinScriptEngineManager.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/DefaultGremlinScriptEngineManager.java
@@ -23,11 +23,13 @@ import javax.script.ScriptContext;
import java.security.AccessController;
import java.security.PrivilegedAction;
import java.util.ArrayList;
+import java.util.Collection;
import java.util.Collections;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Iterator;
import java.util.List;
+import java.util.Map;
import java.util.Optional;
import java.util.ServiceConfigurationError;
import java.util.ServiceLoader;
@@ -441,6 +443,24 @@ public class DefaultGremlinScriptEngineManager implements GremlinScriptEngineMan
private GremlinScriptEngine createGremlinScriptEngine(final GremlinScriptEngineFactory spi) {
final GremlinScriptEngine engine = spi.getScriptEngine();
engine.setBindings(getBindings(), ScriptContext.GLOBAL_SCOPE);
+
+ final List<ScriptCustomizer> scriptCustomizers = getCustomizers(spi.getEngineName()).stream()
+ .filter(p -> p instanceof ScriptCustomizer)
+ .map(p -> ((ScriptCustomizer) p))
+ .collect(Collectors.toList());
+
+ // since the bindings aren't added until after the ScriptEngine is constructed, running init scripts that
+ // require bindings creates a problem. as a result, init scripts are applied here
+ scriptCustomizers.stream().flatMap(sc -> sc.getScripts().stream()).
+ map(l -> String.join(System.lineSeparator(), l)).forEach(initScript -> {
+ try {
+ final Object initializedBindings = engine.eval(initScript);
+ if (initializedBindings != null && initializedBindings instanceof Map)
+ ((Map<String,Object>) initializedBindings).forEach((k,v) -> put(k,v));
+ } catch (Exception ex) {
+ throw new IllegalStateException(ex);
+ }
+ });
return engine;
}
}
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/e3889bf2/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/ScriptCustomizer.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/ScriptCustomizer.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/ScriptCustomizer.java
index eb2f8bc..5390776 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/ScriptCustomizer.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/ScriptCustomizer.java
@@ -18,11 +18,15 @@
*/
package org.apache.tinkerpop.gremlin.jsr223;
+import javax.script.ScriptEngineManager;
import java.util.Collection;
import java.util.List;
/**
* A {@link Customizer} that executes scripts in a {@link GremlinScriptEngine} instance for purpose of initialization.
+ * Implementors of a {@link GremlinScriptEngine} do not need to be concerned with supporting this {@link Customizer}.
+ * This is work for the {@link ScriptEngineManager} implementation since scripts typically require access to global
+ * bindings and those are not applied to the {@link GremlinScriptEngine} until after construction.
*
* @author Stephen Mallette (http://stephen.genoprime.com)
*/
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/e3889bf2/gremlin-groovy/src/main/java/org/apache/tinkerpop/gremlin/groovy/engine/GremlinExecutor.java
----------------------------------------------------------------------
diff --git a/gremlin-groovy/src/main/java/org/apache/tinkerpop/gremlin/groovy/engine/GremlinExecutor.java b/gremlin-groovy/src/main/java/org/apache/tinkerpop/gremlin/groovy/engine/GremlinExecutor.java
index 486f19e..567a248 100644
--- a/gremlin-groovy/src/main/java/org/apache/tinkerpop/gremlin/groovy/engine/GremlinExecutor.java
+++ b/gremlin-groovy/src/main/java/org/apache/tinkerpop/gremlin/groovy/engine/GremlinExecutor.java
@@ -22,6 +22,7 @@ import org.apache.commons.lang.exception.ExceptionUtils;
import org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine;
import org.apache.commons.lang3.concurrent.BasicThreadFactory;
import org.apache.tinkerpop.gremlin.jsr223.CachedGremlinScriptEngineManager;
+import org.apache.tinkerpop.gremlin.jsr223.DefaultBindingsCustomizer;
import org.apache.tinkerpop.gremlin.jsr223.GremlinPlugin;
import org.apache.tinkerpop.gremlin.jsr223.GremlinScriptEngineManager;
import org.apache.tinkerpop.gremlin.process.traversal.Bytecode;
@@ -121,13 +122,15 @@ public class GremlinExecutor implements AutoCloseable {
this.gremlinScriptEngineManager = new CachedGremlinScriptEngineManager();
initializeGremlinScriptEngineManager();
- // this is temporary so that we can have backward compatibilty to the old plugin system and ScriptEngines
+ // this is temporary so that we can have backward compatibility to the old plugin system and ScriptEngines
// approach to configuring Gremlin Server and GremlinExecutor. This code/check should be removed with the
// deprecated code around this is removed.
if (!useGremlinScriptEngineManager)
this.scriptEngines = createScriptEngines();
- else
+ else {
this.scriptEngines = null;
+ gremlinScriptEngineManager.getEngineByName("gremlin-groovy");
+ }
this.suppliedExecutor = suppliedExecutor;
this.suppliedScheduledExecutor = suppliedScheduledExecutor;
@@ -490,6 +493,10 @@ public class GremlinExecutor implements AutoCloseable {
}
}
}
+
+ if (this.useGremlinScriptEngineManager) {
+ gremlinScriptEngineManager.setBindings(globalBindings);
+ }
}
private ScriptEngines createScriptEngines() {
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/e3889bf2/neo4j-gremlin/src/main/java/org/apache/tinkerpop/gremlin/neo4j/jsr223/Neo4jGremlinPlugin.java
----------------------------------------------------------------------
diff --git a/neo4j-gremlin/src/main/java/org/apache/tinkerpop/gremlin/neo4j/jsr223/Neo4jGremlinPlugin.java b/neo4j-gremlin/src/main/java/org/apache/tinkerpop/gremlin/neo4j/jsr223/Neo4jGremlinPlugin.java
index 82a8d18..839a1d1 100644
--- a/neo4j-gremlin/src/main/java/org/apache/tinkerpop/gremlin/neo4j/jsr223/Neo4jGremlinPlugin.java
+++ b/neo4j-gremlin/src/main/java/org/apache/tinkerpop/gremlin/neo4j/jsr223/Neo4jGremlinPlugin.java
@@ -60,10 +60,16 @@ public final class Neo4jGremlinPlugin extends AbstractGremlinPlugin {
}
}
+ private static final Neo4jGremlinPlugin instance = new Neo4jGremlinPlugin();
+
public Neo4jGremlinPlugin() {
super(NAME, imports);
}
+ public static Neo4jGremlinPlugin instance() {
+ return instance;
+ }
+
@Override
public boolean requireRestart() {
return true;
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/e3889bf2/spark-gremlin/src/main/java/org/apache/tinkerpop/gremlin/spark/jsr223/SparkGremlinPlugin.java
----------------------------------------------------------------------
diff --git a/spark-gremlin/src/main/java/org/apache/tinkerpop/gremlin/spark/jsr223/SparkGremlinPlugin.java b/spark-gremlin/src/main/java/org/apache/tinkerpop/gremlin/spark/jsr223/SparkGremlinPlugin.java
index 9403fa4..f864a37 100644
--- a/spark-gremlin/src/main/java/org/apache/tinkerpop/gremlin/spark/jsr223/SparkGremlinPlugin.java
+++ b/spark-gremlin/src/main/java/org/apache/tinkerpop/gremlin/spark/jsr223/SparkGremlinPlugin.java
@@ -81,10 +81,16 @@ public final class SparkGremlinPlugin extends AbstractGremlinPlugin {
return bindings;
});
+ private static final SparkGremlinPlugin instance = new SparkGremlinPlugin();
+
public SparkGremlinPlugin() {
super(NAME, imports, bindings);
}
+ public static SparkGremlinPlugin instance() {
+ return instance;
+ }
+
@Override
public boolean requireRestart() {
return true;
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/e3889bf2/tinkergraph-gremlin/src/main/java/org/apache/tinkerpop/gremlin/tinkergraph/jsr223/TinkerGraphGremlinPlugin.java
----------------------------------------------------------------------
diff --git a/tinkergraph-gremlin/src/main/java/org/apache/tinkerpop/gremlin/tinkergraph/jsr223/TinkerGraphGremlinPlugin.java b/tinkergraph-gremlin/src/main/java/org/apache/tinkerpop/gremlin/tinkergraph/jsr223/TinkerGraphGremlinPlugin.java
index 19188d2..55e9a4d 100644
--- a/tinkergraph-gremlin/src/main/java/org/apache/tinkerpop/gremlin/tinkergraph/jsr223/TinkerGraphGremlinPlugin.java
+++ b/tinkergraph-gremlin/src/main/java/org/apache/tinkerpop/gremlin/tinkergraph/jsr223/TinkerGraphGremlinPlugin.java
@@ -48,25 +48,31 @@ public final class TinkerGraphGremlinPlugin extends AbstractGremlinPlugin {
private static final ImportCustomizer imports = DefaultImportCustomizer.build()
.addClassImports(TinkerEdge.class,
- TinkerElement.class,
- TinkerFactory.class,
- TinkerGraph.class,
- TinkerGraphVariables.class,
- TinkerHelper.class,
- TinkerIoRegistry.class,
- TinkerIoRegistryV2d0.class,
- TinkerProperty.class,
- TinkerVertex.class,
- TinkerVertexProperty.class,
- TinkerGraphComputer.class,
- TinkerGraphComputerView.class,
- TinkerMapEmitter.class,
- TinkerMemory.class,
- TinkerMessenger.class,
- TinkerReduceEmitter.class,
- TinkerWorkerPool.class).create();
+ TinkerElement.class,
+ TinkerFactory.class,
+ TinkerGraph.class,
+ TinkerGraphVariables.class,
+ TinkerHelper.class,
+ TinkerIoRegistry.class,
+ TinkerIoRegistryV2d0.class,
+ TinkerProperty.class,
+ TinkerVertex.class,
+ TinkerVertexProperty.class,
+ TinkerGraphComputer.class,
+ TinkerGraphComputerView.class,
+ TinkerMapEmitter.class,
+ TinkerMemory.class,
+ TinkerMessenger.class,
+ TinkerReduceEmitter.class,
+ TinkerWorkerPool.class).create();
+
+ private static final TinkerGraphGremlinPlugin instance = new TinkerGraphGremlinPlugin();
public TinkerGraphGremlinPlugin() {
super(NAME, imports);
}
-}
+
+ public static TinkerGraphGremlinPlugin instance() {
+ return instance;
+ }
+}
\ No newline at end of file
[22/29] tinkerpop git commit: dah. my last tweak made a buggy. now
I'm done for the day.
Posted by rd...@apache.org.
dah. my last tweak made a buggy. now I'm done for the day.
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/b2f2a9d6
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/b2f2a9d6
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/b2f2a9d6
Branch: refs/heads/TINKERPOP-1602
Commit: b2f2a9d692392b6eac672c109128ffcb45345150
Parents: 9dee11d
Author: Marko A. Rodriguez <ok...@gmail.com>
Authored: Thu Jan 26 15:18:31 2017 -0700
Committer: Marko A. Rodriguez <ok...@gmail.com>
Committed: Fri Jan 27 14:24:18 2017 -0700
----------------------------------------------------------------------
.../strategy/optimization/SingleIterationStrategy.java | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/b2f2a9d6/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategy.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategy.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategy.java
index eed1104..96a2a0a 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategy.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategy.java
@@ -28,6 +28,7 @@ import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategy;
import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.DefaultGraphTraversal;
import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__;
import org.apache.tinkerpop.gremlin.process.traversal.step.Barrier;
+import org.apache.tinkerpop.gremlin.process.traversal.step.LambdaHolder;
import org.apache.tinkerpop.gremlin.process.traversal.step.SideEffectCapable;
import org.apache.tinkerpop.gremlin.process.traversal.step.TraversalParent;
import org.apache.tinkerpop.gremlin.process.traversal.step.branch.LocalStep;
@@ -93,13 +94,14 @@ public final class SingleIterationStrategy extends AbstractTraversalStrategy<Tra
}
if (doesMessagePass && // if the traversal doesn't message pass, then don't try and localize it as its just wasted computation
TraversalHelper.isLocalStarGraph(computerTraversal) && // if we move beyond the star graph, then localization is not possible.
- (computerTraversal.getStartStep() instanceof GraphStep) && // while GraphComputer requires GraphStep starts, this is just a precaution when inject() starts are supported
+ computerTraversal.getStartStep() instanceof GraphStep && // while GraphComputer requires GraphStep starts, this is just a precaution when inject() starts are supported
!(computerTraversal.getStartStep().getNextStep() instanceof EmptyStep) && // if its just a g.V()/E(), then don't localize
!(computerTraversal.getStartStep().getNextStep() instanceof LocalStep) && // removes the potential for the infinite recursive application of the traversal
!(computerTraversal.getStartStep().getNextStep() instanceof Barrier) && // if the second step is a barrier, no point in trying to localize anything
!computerTraversal.getTraverserRequirements().contains(TraverserRequirement.LABELED_PATH) && // this is to alleviate issues with DetachedElement in paths (TODO: when detachment is dynamic, remove this)
!computerTraversal.getTraverserRequirements().contains(TraverserRequirement.PATH) && // this is to alleviate issues with DetachedElement in paths (TODO: when detachment is dynamic, remove this)
- TraversalHelper.getStepsOfAssignableClassRecursively(SideEffectCapable.class, computerTraversal).isEmpty() && // this is to alleviate issues with DetachedElement in paths (TODO: when detachment is dynamic, remove this)
+ !TraversalHelper.hasStepOfAssignableClassRecursively(LambdaHolder.class, computerTraversal) && // this is because we might be accessing data on an adjacent vertex and that is bad
+ !TraversalHelper.hasStepOfAssignableClassRecursively(SideEffectCapable.class, computerTraversal) && // this is to alleviate issues with DetachedElement in paths (TODO: when detachment is dynamic, remove this)
!(TraversalHelper.getStepsOfAssignableClass(TraversalParent.class, computerTraversal). // this is a strict precaution that could be loosed with deeper logic on barriers in global children
stream().
filter(parent -> !parent.getGlobalChildren().isEmpty()).findAny().isPresent())) {
[24/29] tinkerpop git commit: realize that we can generalized
SingleIterationStrategy to 1) work with GraphActors and 2) work at every
walk-step. Thus,
given that GraphActors isn't about Iterations and we can make this work for
multi-stages, I renamed th
Posted by rd...@apache.org.
realize that we can generalized SingleIterationStrategy to 1) work with GraphActors and 2) work at every walk-step. Thus, given that GraphActors isn't about Iterations and we can make this work for multi-stages, I renamed the strategy MessagePassingReductionStrategy. This way we won't have to name change in the future. Also, I did some more code cleanups and added a bunch more test cases. This is perhaps the most tested strategy.
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/c2a42e27
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/c2a42e27
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/c2a42e27
Branch: refs/heads/TINKERPOP-1602
Commit: c2a42e27f901b95c66e31562940a176b9f932692
Parents: 1819e05
Author: Marko A. Rodriguez <ok...@gmail.com>
Authored: Fri Jan 27 11:30:15 2017 -0700
Committer: Marko A. Rodriguez <ok...@gmail.com>
Committed: Fri Jan 27 14:24:35 2017 -0700
----------------------------------------------------------------------
CHANGELOG.asciidoc | 1 +
.../MessagePassingReductionStrategy.java | 162 +++++++++++++++++++
.../optimization/SingleIterationStrategy.java | 151 -----------------
.../process/traversal/TraversalStrategies.java | 4 +-
.../MessagePassingReductionStrategyTest.java | 135 ++++++++++++++++
.../SingleIterationStrategyTest.java | 128 ---------------
.../SparkSingleIterationStrategy.java | 42 +----
.../SparkSingleIterationStrategyTest.java | 17 +-
8 files changed, 314 insertions(+), 326 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/c2a42e27/CHANGELOG.asciidoc
----------------------------------------------------------------------
diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc
index 860d401..d523ae2 100644
--- a/CHANGELOG.asciidoc
+++ b/CHANGELOG.asciidoc
@@ -28,6 +28,7 @@ TinkerPop 3.2.4 (Release Date: NOT OFFICIALLY RELEASED YET)
* Fixed a bug where `PathProcessor.keepLabels` were not being pushed down into child traversals by `PathRetractionStrategy`.
* Added `SingleIterationStrategy` as a default `GraphComputer` strategy that can re-write some traversals to not require message passing.
+* Added default `MessagePassingReductionStrategy` for `GraphComputer` that can reduce the number of message passing iterations.
* Fixed a bug associated with user-provided maps and `GroupSideEffectStep`.
* `GroupBiOperator` no longer maintains a detached traversal and thus, no more side-effect related OLAP inconsistencies.
* Added `ProjectedTraverser` which wraps a traverser with a `List<Object>` of projected data.
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/c2a42e27/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/MessagePassingReductionStrategy.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/MessagePassingReductionStrategy.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/MessagePassingReductionStrategy.java
new file mode 100644
index 0000000..cff152e
--- /dev/null
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/MessagePassingReductionStrategy.java
@@ -0,0 +1,162 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.tinkerpop.gremlin.process.computer.traversal.strategy.optimization;
+
+import org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.TraversalVertexProgramStep;
+import org.apache.tinkerpop.gremlin.process.computer.util.EmptyMemory;
+import org.apache.tinkerpop.gremlin.process.traversal.Step;
+import org.apache.tinkerpop.gremlin.process.traversal.Traversal;
+import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategy;
+import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.DefaultGraphTraversal;
+import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__;
+import org.apache.tinkerpop.gremlin.process.traversal.step.Barrier;
+import org.apache.tinkerpop.gremlin.process.traversal.step.LambdaHolder;
+import org.apache.tinkerpop.gremlin.process.traversal.step.TraversalParent;
+import org.apache.tinkerpop.gremlin.process.traversal.step.branch.LocalStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.filter.DedupGlobalStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.filter.FilterStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.map.CountGlobalStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.map.EdgeVertexStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.map.GraphStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.map.IdStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.map.TraversalFlatMapStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.map.TraversalMapStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.map.VertexStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.IdentityStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.SideEffectStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.util.EmptyStep;
+import org.apache.tinkerpop.gremlin.process.traversal.strategy.AbstractTraversalStrategy;
+import org.apache.tinkerpop.gremlin.process.traversal.strategy.optimization.AdjacentToIncidentStrategy;
+import org.apache.tinkerpop.gremlin.process.traversal.strategy.optimization.FilterRankingStrategy;
+import org.apache.tinkerpop.gremlin.process.traversal.strategy.optimization.IncidentToAdjacentStrategy;
+import org.apache.tinkerpop.gremlin.process.traversal.strategy.optimization.InlineFilterStrategy;
+import org.apache.tinkerpop.gremlin.process.traversal.traverser.TraverserRequirement;
+import org.apache.tinkerpop.gremlin.process.traversal.util.TraversalHelper;
+import org.apache.tinkerpop.gremlin.structure.Direction;
+import org.apache.tinkerpop.gremlin.structure.Graph;
+import org.apache.tinkerpop.gremlin.structure.util.empty.EmptyGraph;
+
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+/**
+ * @author Marko A. Rodriguez (http://markorodriguez.com)
+ */
+public final class MessagePassingReductionStrategy extends AbstractTraversalStrategy<TraversalStrategy.OptimizationStrategy> implements TraversalStrategy.OptimizationStrategy {
+
+ private static final MessagePassingReductionStrategy INSTANCE = new MessagePassingReductionStrategy();
+
+ private static final Set<Class<? extends OptimizationStrategy>> PRIORS = new HashSet<>(Arrays.asList(
+ IncidentToAdjacentStrategy.class,
+ AdjacentToIncidentStrategy.class,
+ FilterRankingStrategy.class,
+ InlineFilterStrategy.class));
+
+ private MessagePassingReductionStrategy() {
+ }
+
+ @Override
+ public void apply(final Traversal.Admin<?, ?> traversal) {
+ // only process the first traversal step in an OLAP chain
+ TraversalHelper.getFirstStepOfAssignableClass(TraversalVertexProgramStep.class, traversal).ifPresent(step -> {
+ final Graph graph = traversal.getGraph().orElse(EmptyGraph.instance()); // best guess at what the graph will be as its dynamically determined
+ final Traversal.Admin<?, ?> compiledComputerTraversal = step.generateProgram(graph, EmptyMemory.instance()).getTraversal().get().clone();
+ if (!compiledComputerTraversal.isLocked())
+ compiledComputerTraversal.applyStrategies();
+ if (!TraversalHelper.hasStepOfAssignableClassRecursively(Arrays.asList(LocalStep.class, LambdaHolder.class), compiledComputerTraversal) && // don't do anything with lambdas or locals as this leads to unknown adjacencies
+ !compiledComputerTraversal.getTraverserRequirements().contains(TraverserRequirement.PATH) && // when dynamic detachment is provided in 3.3.0, remove this (TODO)
+ !compiledComputerTraversal.getTraverserRequirements().contains(TraverserRequirement.LABELED_PATH) && // when dynamic detachment is provided in 3.3.0, remove this (TODO)
+ !(TraversalHelper.getStepsOfAssignableClass(TraversalParent.class, compiledComputerTraversal). // this is a strict precaution that could be loosed with deeper logic on barriers in global children
+ stream().
+ filter(parent -> !parent.getGlobalChildren().isEmpty()).findAny().isPresent())) {
+ final Traversal.Admin<?, ?> computerTraversal = step.computerTraversal.get().clone();
+ // apply the strategies up to this point
+ final List<TraversalStrategy<?>> strategies = step.getTraversal().getStrategies().toList();
+ final int indexOfStrategy = strategies.indexOf(MessagePassingReductionStrategy.instance());
+ if (indexOfStrategy > 0)
+ TraversalHelper.applySingleLevelStrategies(step.getTraversal(), computerTraversal, strategies.get(indexOfStrategy - 1).getClass());
+ if (computerTraversal.getSteps().size() > 1 &&
+ !(computerTraversal.getStartStep().getNextStep() instanceof Barrier) &&
+ TraversalHelper.hasStepOfAssignableClassRecursively(Arrays.asList(VertexStep.class, EdgeVertexStep.class), computerTraversal) &&
+ TraversalHelper.isLocalStarGraph(computerTraversal)) {
+ final Step barrier = (Step) TraversalHelper.getFirstStepOfAssignableClass(Barrier.class, computerTraversal).orElse(null);
+ if (MessagePassingReductionStrategy.insertElementId(barrier)) // out().count() -> out().id().count()
+ TraversalHelper.insertBeforeStep(new IdStep(computerTraversal), barrier, computerTraversal);
+ if (!(MessagePassingReductionStrategy.endsWithElement(null == barrier ? computerTraversal.getEndStep() : barrier))) {
+ Traversal.Admin newChildTraversal = new DefaultGraphTraversal<>();
+ TraversalHelper.removeToTraversal(computerTraversal.getStartStep() instanceof GraphStep ?
+ computerTraversal.getStartStep().getNextStep() :
+ (Step) computerTraversal.getStartStep(), null == barrier ?
+ EmptyStep.instance() :
+ barrier, newChildTraversal);
+ newChildTraversal = newChildTraversal.getSteps().size() > 1 ? (Traversal.Admin) __.local(newChildTraversal) : newChildTraversal;
+ if (null == barrier)
+ TraversalHelper.insertTraversal(0, newChildTraversal, computerTraversal);
+ else
+ TraversalHelper.insertTraversal(barrier.getPreviousStep(), newChildTraversal, computerTraversal);
+ }
+ }
+ step.setComputerTraversal(computerTraversal);
+ }
+ });
+ }
+
+ private static boolean insertElementId(final Step<?, ?> barrier) {
+ if (!(barrier instanceof Barrier))
+ return false;
+ else if (!endsWithElement(barrier.getPreviousStep()))
+ return false;
+ else if (barrier instanceof CountGlobalStep)
+ return true;
+ else if (barrier instanceof DedupGlobalStep &&
+ ((DedupGlobalStep) barrier).getScopeKeys().isEmpty() &&
+ ((DedupGlobalStep) barrier).getLocalChildren().isEmpty() &&
+ barrier.getNextStep() instanceof CountGlobalStep)
+ return true;
+ else
+ return false;
+ }
+
+ public static final boolean endsWithElement(Step<?, ?> currentStep) {
+ while (!(currentStep instanceof EmptyStep)) {
+ if (currentStep instanceof VertexStep) // only inE, in, and out send messages
+ return (((VertexStep) currentStep).returnsVertex() || !((VertexStep) currentStep).getDirection().equals(Direction.OUT));
+ else if (currentStep instanceof EdgeVertexStep) // TODO: add GraphStep but only if its mid-traversal V()/E()
+ return true;
+ else if (currentStep instanceof TraversalFlatMapStep || currentStep instanceof TraversalMapStep || currentStep instanceof LocalStep)
+ return endsWithElement(((TraversalParent) currentStep).getLocalChildren().get(0).getEndStep());
+ else if (!(currentStep instanceof FilterStep || currentStep instanceof SideEffectStep || currentStep instanceof IdentityStep || currentStep instanceof Barrier))
+ return false;
+ currentStep = currentStep.getPreviousStep();
+ }
+ return false;
+ }
+
+ @Override
+ public Set<Class<? extends OptimizationStrategy>> applyPrior() {
+ return PRIORS;
+ }
+
+ public static MessagePassingReductionStrategy instance() {
+ return INSTANCE;
+ }
+}
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/c2a42e27/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategy.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategy.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategy.java
deleted file mode 100644
index 6b509ef..0000000
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategy.java
+++ /dev/null
@@ -1,151 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-
-package org.apache.tinkerpop.gremlin.process.computer.traversal.strategy.optimization;
-
-import org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.TraversalVertexProgramStep;
-import org.apache.tinkerpop.gremlin.process.computer.util.EmptyMemory;
-import org.apache.tinkerpop.gremlin.process.traversal.Step;
-import org.apache.tinkerpop.gremlin.process.traversal.Traversal;
-import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategy;
-import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.DefaultGraphTraversal;
-import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__;
-import org.apache.tinkerpop.gremlin.process.traversal.step.Barrier;
-import org.apache.tinkerpop.gremlin.process.traversal.step.LambdaHolder;
-import org.apache.tinkerpop.gremlin.process.traversal.step.TraversalParent;
-import org.apache.tinkerpop.gremlin.process.traversal.step.branch.LocalStep;
-import org.apache.tinkerpop.gremlin.process.traversal.step.filter.DedupGlobalStep;
-import org.apache.tinkerpop.gremlin.process.traversal.step.filter.FilterStep;
-import org.apache.tinkerpop.gremlin.process.traversal.step.map.CountGlobalStep;
-import org.apache.tinkerpop.gremlin.process.traversal.step.map.EdgeVertexStep;
-import org.apache.tinkerpop.gremlin.process.traversal.step.map.GraphStep;
-import org.apache.tinkerpop.gremlin.process.traversal.step.map.IdStep;
-import org.apache.tinkerpop.gremlin.process.traversal.step.map.TraversalFlatMapStep;
-import org.apache.tinkerpop.gremlin.process.traversal.step.map.TraversalMapStep;
-import org.apache.tinkerpop.gremlin.process.traversal.step.map.VertexStep;
-import org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.SideEffectStep;
-import org.apache.tinkerpop.gremlin.process.traversal.step.util.EmptyStep;
-import org.apache.tinkerpop.gremlin.process.traversal.strategy.AbstractTraversalStrategy;
-import org.apache.tinkerpop.gremlin.process.traversal.strategy.optimization.AdjacentToIncidentStrategy;
-import org.apache.tinkerpop.gremlin.process.traversal.strategy.optimization.FilterRankingStrategy;
-import org.apache.tinkerpop.gremlin.process.traversal.strategy.optimization.IncidentToAdjacentStrategy;
-import org.apache.tinkerpop.gremlin.process.traversal.strategy.optimization.InlineFilterStrategy;
-import org.apache.tinkerpop.gremlin.process.traversal.traverser.TraverserRequirement;
-import org.apache.tinkerpop.gremlin.process.traversal.util.TraversalHelper;
-import org.apache.tinkerpop.gremlin.structure.Graph;
-import org.apache.tinkerpop.gremlin.structure.util.empty.EmptyGraph;
-
-import java.util.Arrays;
-import java.util.HashSet;
-import java.util.Set;
-
-/**
- * @author Marko A. Rodriguez (http://markorodriguez.com)
- */
-public final class SingleIterationStrategy extends AbstractTraversalStrategy<TraversalStrategy.OptimizationStrategy> implements TraversalStrategy.OptimizationStrategy {
-
- private static final SingleIterationStrategy INSTANCE = new SingleIterationStrategy();
-
- private static final Set<Class<? extends OptimizationStrategy>> PRIORS = new HashSet<>(Arrays.asList(
- IncidentToAdjacentStrategy.class,
- AdjacentToIncidentStrategy.class,
- FilterRankingStrategy.class,
- InlineFilterStrategy.class));
-
- private SingleIterationStrategy() {
- }
-
- @Override
- public void apply(final Traversal.Admin<?, ?> traversal) {
- // only process the first traversal step in an OLAP chain
- TraversalHelper.getFirstStepOfAssignableClass(TraversalVertexProgramStep.class, traversal).ifPresent(step -> {
- final Graph graph = traversal.getGraph().orElse(EmptyGraph.instance()); // best guess at what the graph will be as its dynamically determined
- final Traversal.Admin<?, ?> compiledComputerTraversal = step.generateProgram(graph, EmptyMemory.instance()).getTraversal().get().clone();
- if (!compiledComputerTraversal.isLocked())
- compiledComputerTraversal.applyStrategies();
- if (!TraversalHelper.hasStepOfAssignableClassRecursively(Arrays.asList(LocalStep.class, LambdaHolder.class), compiledComputerTraversal) &&
- !compiledComputerTraversal.getTraverserRequirements().contains(TraverserRequirement.PATH) && // when dynamic detachment is provided in 3.3.0, remove this (TODO)
- !compiledComputerTraversal.getTraverserRequirements().contains(TraverserRequirement.LABELED_PATH) && // when dynamic detachment is provided in 3.3.0, remove this (TODO)
- !(TraversalHelper.getStepsOfAssignableClass(TraversalParent.class, compiledComputerTraversal). // this is a strict precaution that could be loosed with deeper logic on barriers in global children
- stream().
- filter(parent -> !parent.getGlobalChildren().isEmpty()).findAny().isPresent())) {
- final Traversal.Admin<?, ?> computerTraversal = step.computerTraversal.getPure();
- if (computerTraversal.getSteps().size() > 1 &&
- !(computerTraversal.getStartStep().getNextStep() instanceof Barrier) &&
- TraversalHelper.hasStepOfAssignableClassRecursively(Arrays.asList(VertexStep.class, EdgeVertexStep.class), computerTraversal) &&
- TraversalHelper.isLocalStarGraph(computerTraversal)) {
- final Traversal.Admin newChildTraversal = new DefaultGraphTraversal<>();
- final Step barrier = (Step) TraversalHelper.getFirstStepOfAssignableClass(Barrier.class, computerTraversal).orElse(null);
- if (SingleIterationStrategy.insertElementId(barrier)) // out().count() -> out().id().count()
- TraversalHelper.insertBeforeStep(new IdStep(computerTraversal), barrier, computerTraversal);
- if (!(SingleIterationStrategy.endsWithElement(null == barrier ? computerTraversal.getEndStep() : barrier.getPreviousStep()))) {
- TraversalHelper.removeToTraversal(computerTraversal.getStartStep() instanceof GraphStep ?
- computerTraversal.getStartStep().getNextStep() :
- (Step) computerTraversal.getStartStep(), null == barrier ?
- EmptyStep.instance() :
- barrier, newChildTraversal);
- if (null == barrier)
- TraversalHelper.insertTraversal(0, (Traversal.Admin) __.local(newChildTraversal), computerTraversal);
- else
- TraversalHelper.insertTraversal(barrier.getPreviousStep(), (Traversal.Admin) __.local(newChildTraversal), computerTraversal);
- }
- }
- step.setComputerTraversal(computerTraversal);
- }
- });
- }
-
- private static boolean insertElementId(final Step<?, ?> barrier) {
- if (!(barrier instanceof Barrier))
- return false;
- else if (!endsWithElement(barrier.getPreviousStep()))
- return false;
- else if (barrier instanceof CountGlobalStep)
- return true;
- else if (barrier instanceof DedupGlobalStep &&
- ((DedupGlobalStep) barrier).getScopeKeys().isEmpty() &&
- ((DedupGlobalStep) barrier).getLocalChildren().isEmpty() &&
- barrier.getNextStep() instanceof CountGlobalStep)
- return true;
- else
- return false;
- }
-
- private static boolean endsWithElement(Step<?, ?> currentStep) {
- while (!(currentStep instanceof EmptyStep)) {
- if (currentStep instanceof VertexStep || currentStep instanceof EdgeVertexStep) // TODO: add GraphStep but only if its mid-traversal V()/E()
- return true;
- else if (currentStep instanceof TraversalFlatMapStep || currentStep instanceof TraversalMapStep)
- return endsWithElement(((TraversalParent) currentStep).getLocalChildren().get(0).getEndStep());
- else if (!(currentStep instanceof FilterStep || currentStep instanceof SideEffectStep))
- return false;
- currentStep = currentStep.getPreviousStep();
- }
- return false;
- }
-
- @Override
- public Set<Class<? extends OptimizationStrategy>> applyPrior() {
- return PRIORS;
- }
-
- public static SingleIterationStrategy instance() {
- return INSTANCE;
- }
-}
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/c2a42e27/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/TraversalStrategies.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/TraversalStrategies.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/TraversalStrategies.java
index a1cb99c..7f96850 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/TraversalStrategies.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/TraversalStrategies.java
@@ -20,7 +20,7 @@ package org.apache.tinkerpop.gremlin.process.traversal;
import org.apache.tinkerpop.gremlin.process.computer.GraphComputer;
import org.apache.tinkerpop.gremlin.process.computer.traversal.strategy.optimization.GraphFilterStrategy;
-import org.apache.tinkerpop.gremlin.process.computer.traversal.strategy.optimization.SingleIterationStrategy;
+import org.apache.tinkerpop.gremlin.process.computer.traversal.strategy.optimization.MessagePassingReductionStrategy;
import org.apache.tinkerpop.gremlin.process.traversal.strategy.decoration.ConnectiveStrategy;
import org.apache.tinkerpop.gremlin.process.traversal.strategy.finalization.ProfileStrategy;
import org.apache.tinkerpop.gremlin.process.traversal.strategy.optimization.AdjacentToIncidentStrategy;
@@ -223,7 +223,7 @@ public interface TraversalStrategies extends Serializable, Cloneable {
final TraversalStrategies graphComputerStrategies = new DefaultTraversalStrategies();
graphComputerStrategies.addStrategies(
GraphFilterStrategy.instance(),
- SingleIterationStrategy.instance(),
+ MessagePassingReductionStrategy.instance(),
OrderLimitStrategy.instance(),
PathProcessorStrategy.instance(),
ComputerVerificationStrategy.instance());
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/c2a42e27/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/MessagePassingReductionStrategyTest.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/MessagePassingReductionStrategyTest.java b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/MessagePassingReductionStrategyTest.java
new file mode 100644
index 0000000..1171dcd
--- /dev/null
+++ b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/MessagePassingReductionStrategyTest.java
@@ -0,0 +1,135 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.tinkerpop.gremlin.process.computer.traversal.strategy.optimization;
+
+import org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.TraversalVertexProgramStep;
+import org.apache.tinkerpop.gremlin.process.traversal.Traversal;
+import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategies;
+import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategy;
+import org.apache.tinkerpop.gremlin.process.traversal.Traverser;
+import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.DefaultGraphTraversal;
+import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__;
+import org.apache.tinkerpop.gremlin.process.traversal.strategy.optimization.AdjacentToIncidentStrategy;
+import org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversalStrategies;
+import org.apache.tinkerpop.gremlin.structure.Column;
+import org.apache.tinkerpop.gremlin.structure.Direction;
+import org.apache.tinkerpop.gremlin.structure.Vertex;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.function.Function;
+
+import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.bothE;
+import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.in;
+import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.inE;
+import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.out;
+import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.outE;
+import static org.apache.tinkerpop.gremlin.structure.Column.keys;
+import static org.junit.Assert.assertEquals;
+
+/**
+ * @author Marko A. Rodriguez (http://markorodriguez.com)
+ */
+@RunWith(Parameterized.class)
+public class MessagePassingReductionStrategyTest {
+
+ @Parameterized.Parameter(value = 0)
+ public Traversal original;
+
+ @Parameterized.Parameter(value = 1)
+ public Traversal optimized;
+
+ @Parameterized.Parameter(value = 2)
+ public Collection<TraversalStrategy> otherStrategies;
+
+ @Test
+ public void doTest() {
+ final Traversal.Admin<?, ?> rootTraversal = new DefaultGraphTraversal<>();
+ final TraversalVertexProgramStep parent = new TraversalVertexProgramStep(rootTraversal, this.original.asAdmin());
+ rootTraversal.addStep(parent.asStep());
+ parent.setComputerTraversal(this.original.asAdmin());
+ final TraversalStrategies strategies = new DefaultTraversalStrategies();
+ strategies.addStrategies(MessagePassingReductionStrategy.instance());
+ for (final TraversalStrategy strategy : this.otherStrategies) {
+ strategies.addStrategies(strategy);
+ }
+ rootTraversal.setStrategies(strategies);
+ rootTraversal.asAdmin().applyStrategies();
+ assertEquals(this.optimized, parent.computerTraversal.get());
+ }
+
+
+ @Parameterized.Parameters(name = "{0}")
+ public static Iterable<Object[]> generateTestParameters() {
+ final Function<Traverser<Vertex>, Vertex> mapFunction = t -> t.get().vertices(Direction.OUT).next();
+ return Arrays.asList(new Object[][]{
+ {__.V().out().count(), __.V().outE().count(), Collections.singletonList(AdjacentToIncidentStrategy.instance())},
+ {__.V().in().count(), __.V().local(__.inE().id()).count(), Collections.singletonList(AdjacentToIncidentStrategy.instance())},
+ {__.V().id(), __.V().id(), Collections.emptyList()},
+ {__.V().local(out()), __.V().local(out()), Collections.emptyList()},
+ {__.V().local(in()), __.V().local(in()), Collections.emptyList()},
+ {__.V().local(out()).count(), __.V().local(out()).count(), Collections.emptyList()},
+ {__.V().local(in()).count(), __.V().local(in()).count(), Collections.emptyList()},
+ {__.V().out().count(), __.V().local(out().id()).count(), Collections.emptyList()},
+ {__.V().out().label().count(), __.V().out().label().count(), Collections.emptyList()},
+ {__.V().in().id(), __.V().local(in().id()), Collections.emptyList()},
+ {in().id(), __.local(in().id()), Collections.emptyList()}, // test inject
+ {__.V().out().id(), __.V().local(out().id()), Collections.emptyList()},
+ {__.V().both().id(), __.V().local(__.both().id()), Collections.emptyList()},
+ {__.V().outE().inV().id().count(), __.V().local(outE().inV().id()).count(), Collections.emptyList()},
+ {__.V().map(outE().inV()).count(), __.V().local(__.map(outE().inV()).id()).count(), Collections.emptyList()},
+ {__.V().out().map(outE().inV()).count(), __.V().out().map(outE().inV()).count(), Collections.emptyList()},
+ {__.V().outE().map(__.inV()).id().count(), __.V().local(__.outE().map(__.inV()).id()).count(), Collections.emptyList()},
+ {__.V().outE().map(__.inV()).count(), __.V().local(outE().map(__.inV()).id()).count(), Collections.emptyList()},
+ {__.V().outE().map(__.inV()).values("name").count(), __.V().outE().map(__.inV()).values("name").count(), Collections.emptyList()},
+ {__.V().outE().inV().count(), __.V().local(outE().inV().id()).count(), Collections.emptyList()},
+ {__.V().out().id().count(), __.V().local(out().id()).count(), Collections.emptyList()},
+ {__.V().in().id().count(), __.V().local(in().id()).count(), Collections.emptyList()},
+ {__.V().in().id().select("id-map").dedup().count(), __.V().local(in().id().select("id-map")).dedup().count(), Collections.emptyList()},
+ {__.V().in().id().groupCount().select(keys).unfold().dedup().count(), __.V().local(in().id()).groupCount().select(keys).unfold().dedup().count(), Collections.emptyList()},
+ {__.V().outE().hasLabel("knows").values("weight"), __.V().local(outE().hasLabel("knows").values("weight")), Collections.emptyList()},
+ {__.V().outE().values("weight").sum(), __.V().local(outE().values("weight")).sum(), Collections.emptyList()},
+ {__.V().inE().values("weight"), __.V().local(inE().values("weight")), Collections.emptyList()},
+ {__.V().inE().values("weight").sum(), __.V().local(inE().values("weight")).sum(), Collections.emptyList()},
+ {__.V().inE().values("weight").sum().dedup().count(), __.V().local(inE().values("weight")).sum().dedup().count(), Collections.emptyList()},
+ {__.V().as("a").out("knows").as("b").select("a", "b"), __.V().as("a").out("knows").as("b").select("a", "b"), Collections.emptyList()},
+ {__.V().out().groupCount("x").cap("x"), __.V().out().groupCount("x").cap("x"), Collections.emptyList()},
+ {__.V().outE().inV().groupCount().select(Column.values).unfold().dedup().count(), __.V().outE().inV().groupCount().select(Column.values).unfold().dedup().count(), Collections.emptyList()},
+ {__.V().outE().inV().groupCount(), __.V().outE().inV().groupCount(), Collections.emptyList()},
+ {__.V().outE().inV().groupCount().by("name"), __.V().outE().inV().groupCount().by("name"), Collections.emptyList()},
+ {__.V().inE().id().groupCount(), __.V().local(inE().id()).groupCount(), Collections.emptyList()},
+ {__.V().inE().values("weight").groupCount(), __.V().local(inE().values("weight")).groupCount(), Collections.emptyList()},
+ {__.V().outE().inV().tree(), __.V().outE().inV().tree(), Collections.emptyList()},
+ {__.V().in().values("name").groupCount(), __.V().in().values("name").groupCount(), Collections.emptyList()},
+ {__.V().outE().inV().groupCount("x"), __.V().outE().inV().groupCount("x"), Collections.emptyList()},
+ {__.V().in().dedup().count(), __.V().local(in().id()).dedup().count(), Collections.emptyList()},
+ {__.V().bothE().dedup().count(), __.V().local(bothE().id()).dedup().count(), Collections.emptyList()},
+ {__.V().bothE().dedup().by("name").count(), __.V().bothE().dedup().by("name").count(), Collections.emptyList()},
+ {__.V().map(mapFunction).inV().count(), __.V().map(mapFunction).inV().count(), Collections.emptyList()},
+ {__.V().groupCount().by(__.out().count()), __.V().groupCount().by(__.out().count()), Collections.emptyList()},
+ {__.V().identity().out().identity().count(), __.V().local(__.identity().out().identity().id()).count(), Collections.emptyList()},
+ {__.V().identity().out().identity().dedup().count(), __.V().local(__.identity().out().identity().id()).dedup().count(), Collections.emptyList()},
+ });
+ }
+}
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/c2a42e27/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategyTest.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategyTest.java b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategyTest.java
deleted file mode 100644
index 081a541..0000000
--- a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategyTest.java
+++ /dev/null
@@ -1,128 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-
-package org.apache.tinkerpop.gremlin.process.computer.traversal.strategy.optimization;
-
-import org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.TraversalVertexProgramStep;
-import org.apache.tinkerpop.gremlin.process.traversal.Traversal;
-import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategies;
-import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategy;
-import org.apache.tinkerpop.gremlin.process.traversal.Traverser;
-import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.DefaultGraphTraversal;
-import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__;
-import org.apache.tinkerpop.gremlin.process.traversal.strategy.optimization.AdjacentToIncidentStrategy;
-import org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversalStrategies;
-import org.apache.tinkerpop.gremlin.structure.Column;
-import org.apache.tinkerpop.gremlin.structure.Direction;
-import org.apache.tinkerpop.gremlin.structure.Vertex;
-import org.junit.Test;
-import org.junit.runner.RunWith;
-import org.junit.runners.Parameterized;
-
-import java.util.Arrays;
-import java.util.Collection;
-import java.util.Collections;
-import java.util.function.Function;
-
-import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.bothE;
-import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.in;
-import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.inE;
-import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.out;
-import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.outE;
-import static org.apache.tinkerpop.gremlin.structure.Column.keys;
-import static org.junit.Assert.assertEquals;
-
-/**
- * @author Marko A. Rodriguez (http://markorodriguez.com)
- */
-@RunWith(Parameterized.class)
-public class SingleIterationStrategyTest {
-
- @Parameterized.Parameter(value = 0)
- public Traversal original;
-
- @Parameterized.Parameter(value = 1)
- public Traversal optimized;
-
- @Parameterized.Parameter(value = 2)
- public Collection<TraversalStrategy> otherStrategies;
-
- @Test
- public void doTest() {
- final Traversal.Admin<?, ?> rootTraversal = new DefaultGraphTraversal<>();
- final TraversalVertexProgramStep parent = new TraversalVertexProgramStep(rootTraversal, this.original.asAdmin());
- rootTraversal.addStep(parent.asStep());
- parent.setComputerTraversal(this.original.asAdmin());
- final TraversalStrategies strategies = new DefaultTraversalStrategies();
- strategies.addStrategies(SingleIterationStrategy.instance());
- for (final TraversalStrategy strategy : this.otherStrategies) {
- strategies.addStrategies(strategy);
- }
- rootTraversal.setStrategies(strategies);
- rootTraversal.asAdmin().applyStrategies();
- assertEquals(this.optimized, parent.computerTraversal.get());
- }
-
-
- @Parameterized.Parameters(name = "{0}")
- public static Iterable<Object[]> generateTestParameters() {
- final Function<Traverser<Vertex>,Vertex> mapFunction = t -> t.get().vertices(Direction.OUT).next();
- return Arrays.asList(new Object[][]{
- {__.V().out().count(), __.V().local(out().id()).count(), Collections.singletonList(AdjacentToIncidentStrategy.instance())}, // TODO
- {__.V().id(), __.V().id(), Collections.emptyList()},
- {__.V().out().count(), __.V().local(out().id()).count(), Collections.emptyList()},
- {__.V().out().label().count(), __.V().out().label().count(), Collections.emptyList()},
- {__.V().in().id(), __.V().local(in().id()), Collections.emptyList()},
- {in().id(), __.local(in().id()), Collections.emptyList()}, // test inject
- {__.V().out().id(), __.V().local(out().id()), Collections.emptyList()},
- {__.V().both().id(), __.V().local(__.both().id()), Collections.emptyList()},
- {__.V().outE().inV().id().count(), __.V().local(outE().inV().id()).count(), Collections.emptyList()},
- {__.V().map(outE().inV()).count(), __.V().local(__.map(outE().inV()).id()).count(), Collections.emptyList()},
- {__.V().out().map(outE().inV()).count(), __.V().out().map(outE().inV()).count(), Collections.emptyList()},
- {__.V().outE().map(__.inV()).id().count(), __.V().local(__.outE().map(__.inV()).id()).count(), Collections.emptyList()},
- {__.V().outE().map(__.inV()).count(), __.V().local(outE().map(__.inV()).id()).count(), Collections.emptyList()},
- {__.V().outE().map(__.inV()).values("name").count(), __.V().outE().map(__.inV()).values("name").count(), Collections.emptyList()},
- {__.V().outE().inV().count(), __.V().local(outE().inV().id()).count(), Collections.emptyList()},
- {__.V().out().id().count(), __.V().local(out().id()).count(), Collections.emptyList()},
- {__.V().in().id().count(), __.V().local(in().id()).count(), Collections.emptyList()},
- {__.V().in().id().select("id-map").dedup().count(), __.V().local(in().id().select("id-map")).dedup().count(), Collections.emptyList()},
- {__.V().in().id().groupCount().select(keys).unfold().dedup().count(), __.V().local(in().id()).groupCount().select(keys).unfold().dedup().count(), Collections.emptyList()},
- {__.V().outE().hasLabel("knows").values("weight"), __.V().local(outE().hasLabel("knows").values("weight")), Collections.emptyList()},
- {__.V().outE().values("weight").sum(), __.V().local(outE().values("weight")).sum(), Collections.emptyList()},
- {__.V().inE().values("weight"), __.V().local(inE().values("weight")), Collections.emptyList()},
- {__.V().inE().values("weight").sum(), __.V().local(inE().values("weight")).sum(), Collections.emptyList()},
- {__.V().inE().values("weight").sum().dedup().count(), __.V().local(inE().values("weight")).sum().dedup().count(), Collections.emptyList()},
- {__.V().as("a").out("knows").as("b").select("a", "b"), __.V().as("a").out("knows").as("b").select("a", "b"), Collections.emptyList()},
- {__.V().out().groupCount("x").cap("x"), __.V().out().groupCount("x").cap("x"), Collections.emptyList()},
- {__.V().outE().inV().groupCount().select(Column.values).unfold().dedup().count(), __.V().outE().inV().groupCount().select(Column.values).unfold().dedup().count(), Collections.emptyList()},
- {__.V().outE().inV().groupCount(), __.V().outE().inV().groupCount(), Collections.emptyList()},
- {__.V().outE().inV().groupCount().by("name"), __.V().outE().inV().groupCount().by("name"), Collections.emptyList()},
- {__.V().inE().id().groupCount(), __.V().local(inE().id()).groupCount(), Collections.emptyList()},
- {__.V().inE().values("weight").groupCount(), __.V().local(inE().values("weight")).groupCount(), Collections.emptyList()},
- {__.V().outE().inV().tree(), __.V().outE().inV().tree(), Collections.emptyList()},
- {__.V().in().values("name").groupCount(), __.V().in().values("name").groupCount(), Collections.emptyList()},
- {__.V().outE().inV().groupCount("x"), __.V().outE().inV().groupCount("x"), Collections.emptyList()},
- {__.V().in().dedup().count(), __.V().local(in().id()).dedup().count(), Collections.emptyList()},
- {__.V().bothE().dedup().count(), __.V().local(bothE().id()).dedup().count(), Collections.emptyList()},
- {__.V().bothE().dedup().by("name").count(), __.V().bothE().dedup().by("name").count(), Collections.emptyList()},
- {__.V().map(mapFunction).inV().count(), __.V().map(mapFunction).inV().count(), Collections.emptyList()},
- {__.V().groupCount().by(__.out().count()),__.V().groupCount().by(__.out().count()),Collections.emptyList()}
- });
- }
-}
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/c2a42e27/spark-gremlin/src/main/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategy.java
----------------------------------------------------------------------
diff --git a/spark-gremlin/src/main/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategy.java b/spark-gremlin/src/main/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategy.java
index 1288b0d..9e85df6 100644
--- a/spark-gremlin/src/main/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategy.java
+++ b/spark-gremlin/src/main/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategy.java
@@ -21,23 +21,15 @@ package org.apache.tinkerpop.gremlin.spark.process.computer.traversal.strategy.o
import org.apache.tinkerpop.gremlin.hadoop.Constants;
import org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.TraversalVertexProgramStep;
+import org.apache.tinkerpop.gremlin.process.computer.traversal.strategy.optimization.MessagePassingReductionStrategy;
import org.apache.tinkerpop.gremlin.process.computer.util.EmptyMemory;
import org.apache.tinkerpop.gremlin.process.traversal.Scope;
-import org.apache.tinkerpop.gremlin.process.traversal.Step;
import org.apache.tinkerpop.gremlin.process.traversal.Traversal;
import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategy;
-import org.apache.tinkerpop.gremlin.process.traversal.step.Barrier;
-import org.apache.tinkerpop.gremlin.process.traversal.step.TraversalParent;
-import org.apache.tinkerpop.gremlin.process.traversal.step.branch.LocalStep;
-import org.apache.tinkerpop.gremlin.process.traversal.step.filter.FilterStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.EdgeVertexStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.LambdaFlatMapStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.LambdaMapStep;
-import org.apache.tinkerpop.gremlin.process.traversal.step.map.TraversalFlatMapStep;
-import org.apache.tinkerpop.gremlin.process.traversal.step.map.TraversalMapStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.VertexStep;
-import org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.SideEffectStep;
-import org.apache.tinkerpop.gremlin.process.traversal.step.util.EmptyStep;
import org.apache.tinkerpop.gremlin.process.traversal.strategy.AbstractTraversalStrategy;
import org.apache.tinkerpop.gremlin.process.traversal.traverser.TraverserRequirement;
import org.apache.tinkerpop.gremlin.process.traversal.util.TraversalHelper;
@@ -83,9 +75,9 @@ public final class SparkSingleIterationStrategy extends AbstractTraversalStrateg
}
}
if (!doesMessagePass &&
- !SparkSingleIterationStrategy.endsWithInElement(computerTraversal) &&
- !(computerTraversal.getTraverserRequirements().contains(TraverserRequirement.LABELED_PATH) || // todo: remove this when dynamic detachment is available in 3.3.0
- computerTraversal.getTraverserRequirements().contains(TraverserRequirement.PATH))) { // todo: remove this when dynamic detachment is available in 3.3.0
+ !MessagePassingReductionStrategy.endsWithElement(computerTraversal.getEndStep()) &&
+ !(computerTraversal.getTraverserRequirements().contains(TraverserRequirement.LABELED_PATH) || // TODO: remove this when dynamic detachment is available in 3.3.0
+ computerTraversal.getTraverserRequirements().contains(TraverserRequirement.PATH))) { // TODO: remove this when dynamic detachment is available in 3.3.0
step.setComputer(step.getComputer()
// if no message passing, don't partition the loaded graph
.configure(Constants.GREMLIN_SPARK_SKIP_PARTITIONER, true)
@@ -95,34 +87,8 @@ public final class SparkSingleIterationStrategy extends AbstractTraversalStrateg
}
}
- private static final boolean endsWithInElement(final Traversal.Admin<?, ?> traversal) {
- Step<?, ?> current = traversal.getEndStep();
- while (current instanceof Barrier) { // clip off any tail barriers
- current = current.getPreviousStep();
- }
- while (!(current instanceof EmptyStep)) {
- if ((current instanceof VertexStep && (((VertexStep) current).returnsVertex() ||
- !((VertexStep) current).getDirection().equals(Direction.OUT))) ||
- current instanceof EdgeVertexStep) {
- return true;
- } else if (current instanceof TraversalMapStep || current instanceof TraversalFlatMapStep || current instanceof LocalStep) {
- if (endsWithInElement(((TraversalParent) current).getLocalChildren().get(0)))
- return true;
- } else if (current instanceof TraversalParent) {
- if (((TraversalParent) current).getGlobalChildren().stream().filter(SparkSingleIterationStrategy::endsWithInElement).findAny().isPresent())
- return true;
- }
- if (!(current instanceof FilterStep || current instanceof SideEffectStep)) { // side-effects and filters do not alter the mapping and thus, deeper analysis is required
- return false;
- }
- current = current.getPreviousStep();
- }
- return false;
- }
-
public static SparkSingleIterationStrategy instance() {
return INSTANCE;
}
-
}
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/c2a42e27/spark-gremlin/src/test/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategyTest.java
----------------------------------------------------------------------
diff --git a/spark-gremlin/src/test/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategyTest.java b/spark-gremlin/src/test/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategyTest.java
index 4297649..8f97576 100644
--- a/spark-gremlin/src/test/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategyTest.java
+++ b/spark-gremlin/src/test/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategyTest.java
@@ -24,7 +24,7 @@ import org.apache.tinkerpop.gremlin.TestHelper;
import org.apache.tinkerpop.gremlin.hadoop.Constants;
import org.apache.tinkerpop.gremlin.hadoop.structure.io.gryo.GryoInputFormat;
import org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.TraversalVertexProgramStep;
-import org.apache.tinkerpop.gremlin.process.computer.traversal.strategy.optimization.SingleIterationStrategy;
+import org.apache.tinkerpop.gremlin.process.computer.traversal.strategy.optimization.MessagePassingReductionStrategy;
import org.apache.tinkerpop.gremlin.process.traversal.Traversal;
import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource;
import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__;
@@ -67,11 +67,11 @@ public class SparkSingleIterationStrategyTest extends AbstractSparkTest {
/////////// WITHOUT SINGLE-ITERATION STRATEGY LESS SINGLE-PASS OPTIONS ARE AVAILABLE
Graph graph = GraphFactory.open(configuration);
- GraphTraversalSource g = graph.traversal().withComputer().withoutStrategies(SparkInterceptorStrategy.class, SingleIterationStrategy.class);
+ GraphTraversalSource g = graph.traversal().withComputer().withoutStrategies(SparkInterceptorStrategy.class, MessagePassingReductionStrategy.class);
assertFalse(g.getStrategies().toList().contains(SparkInterceptorStrategy.instance()));
assertFalse(g.V().count().explain().getStrategyTraversals().stream().filter(pair -> pair.getValue0() instanceof SparkInterceptorStrategy).findAny().isPresent());
- assertFalse(g.getStrategies().toList().contains(SingleIterationStrategy.instance()));
- assertFalse(g.V().count().explain().getStrategyTraversals().stream().filter(pair -> pair.getValue0() instanceof SingleIterationStrategy).findAny().isPresent());
+ assertFalse(g.getStrategies().toList().contains(MessagePassingReductionStrategy.instance()));
+ assertFalse(g.V().count().explain().getStrategyTraversals().stream().filter(pair -> pair.getValue0() instanceof MessagePassingReductionStrategy).findAny().isPresent());
assertTrue(g.getStrategies().toList().contains(SparkSingleIterationStrategy.instance()));
assertTrue(g.V().count().explain().getStrategyTraversals().stream().filter(pair -> pair.getValue0() instanceof SparkSingleIterationStrategy).findAny().isPresent());
@@ -95,11 +95,11 @@ public class SparkSingleIterationStrategyTest extends AbstractSparkTest {
/////////// WITH SINGLE-ITERATION STRATEGY MORE SINGLE-PASS OPTIONS ARE AVAILABLE
graph = GraphFactory.open(configuration);
- g = graph.traversal().withComputer().withoutStrategies(SparkInterceptorStrategy.class).withStrategies(SingleIterationStrategy.instance());
+ g = graph.traversal().withComputer().withoutStrategies(SparkInterceptorStrategy.class).withStrategies(MessagePassingReductionStrategy.instance());
assertFalse(g.getStrategies().toList().contains(SparkInterceptorStrategy.instance()));
assertFalse(g.V().count().explain().getStrategyTraversals().stream().filter(pair -> pair.getValue0() instanceof SparkInterceptorStrategy).findAny().isPresent());
- assertTrue(g.getStrategies().toList().contains(SingleIterationStrategy.instance()));
- assertTrue(g.V().count().explain().getStrategyTraversals().stream().filter(pair -> pair.getValue0() instanceof SingleIterationStrategy).findAny().isPresent());
+ assertTrue(g.getStrategies().toList().contains(MessagePassingReductionStrategy.instance()));
+ assertTrue(g.V().count().explain().getStrategyTraversals().stream().filter(pair -> pair.getValue0() instanceof MessagePassingReductionStrategy).findAny().isPresent());
assertTrue(g.getStrategies().toList().contains(SparkSingleIterationStrategy.instance()));
assertTrue(g.V().count().explain().getStrategyTraversals().stream().filter(pair -> pair.getValue0() instanceof SparkSingleIterationStrategy).findAny().isPresent());
@@ -114,6 +114,8 @@ public class SparkSingleIterationStrategyTest extends AbstractSparkTest {
test(true, g.V().bothE().values("weight").limit(2));
test(true, 6L, g.V().count());
test(true, 6L, g.V().id().count());
+ test(true, 6L, g.V().identity().outE().identity().count());
+ test(true, 6L, g.V().identity().outE().has("weight").count());
test(true, 6L, g.V().out().count());
test(true, 6L, g.V().outE().inV().count());
test(true, 6L, g.V().outE().inV().id().count());
@@ -167,6 +169,7 @@ public class SparkSingleIterationStrategyTest extends AbstractSparkTest {
test(false, g.V().out().groupCount("x").cap("x"));
test(false, 6L, g.V().both().groupCount("x").cap("x").select(keys).unfold().count());
test(false, g.V().outE().inV().groupCount());
+ test(false, g.V().outE().unfold().inV().groupCount());
test(false, g.V().outE().inV().groupCount().by("name"));
test(false, g.V().outE().inV().tree());
test(false, g.V().outE().inV().id().tree());
[07/29] tinkerpop git commit: GremlinExecutor wasn't constructing
plugins properly
Posted by rd...@apache.org.
GremlinExecutor wasn't constructing plugins properly
Wasn't properly doing type comparisons properly to deal with primitive arguments. CTR
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/470c656a
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/470c656a
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/470c656a
Branch: refs/heads/TINKERPOP-1602
Commit: 470c656ac2ac0822d5b561eadef3b25eaacef558
Parents: e3889bf
Author: Stephen Mallette <sp...@genoprime.com>
Authored: Tue Jan 24 15:08:07 2017 -0500
Committer: Stephen Mallette <sp...@genoprime.com>
Committed: Tue Jan 24 15:08:07 2017 -0500
----------------------------------------------------------------------
.../tinkerpop/gremlin/groovy/engine/GremlinExecutor.java | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/470c656a/gremlin-groovy/src/main/java/org/apache/tinkerpop/gremlin/groovy/engine/GremlinExecutor.java
----------------------------------------------------------------------
diff --git a/gremlin-groovy/src/main/java/org/apache/tinkerpop/gremlin/groovy/engine/GremlinExecutor.java b/gremlin-groovy/src/main/java/org/apache/tinkerpop/gremlin/groovy/engine/GremlinExecutor.java
index 567a248..349af8e 100644
--- a/gremlin-groovy/src/main/java/org/apache/tinkerpop/gremlin/groovy/engine/GremlinExecutor.java
+++ b/gremlin-groovy/src/main/java/org/apache/tinkerpop/gremlin/groovy/engine/GremlinExecutor.java
@@ -18,11 +18,11 @@
*/
package org.apache.tinkerpop.gremlin.groovy.engine;
+import org.apache.commons.lang.ClassUtils;
import org.apache.commons.lang.exception.ExceptionUtils;
import org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine;
import org.apache.commons.lang3.concurrent.BasicThreadFactory;
import org.apache.tinkerpop.gremlin.jsr223.CachedGremlinScriptEngineManager;
-import org.apache.tinkerpop.gremlin.jsr223.DefaultBindingsCustomizer;
import org.apache.tinkerpop.gremlin.jsr223.GremlinPlugin;
import org.apache.tinkerpop.gremlin.jsr223.GremlinScriptEngineManager;
import org.apache.tinkerpop.gremlin.process.traversal.Bytecode;
@@ -469,9 +469,9 @@ public class GremlinExecutor implements AutoCloseable {
final Method configMethod = Stream.of(methods).filter(m -> {
final Class<?> type = customizerConfig.getValue().getClass();
return m.getName().equals(customizerConfig.getKey()) && m.getParameters().length <= 1
- && m.getParameters()[0].getType().isAssignableFrom(type);
+ && ClassUtils.isAssignable(type, m.getParameters()[0].getType(), true);
}).findFirst()
- .orElseThrow(() -> new IllegalStateException("Could not find builder method on " + builderClazz.getCanonicalName()));
+ .orElseThrow(() -> new IllegalStateException("Could not find builder method '" + customizerConfig.getKey() + "' on " + builderClazz.getCanonicalName()));
if (null == customizerConfig.getValue())
pluginBuilder = configMethod.invoke(pluginBuilder);
else
[28/29] tinkerpop git commit: format, organize imports only
Posted by rd...@apache.org.
format, organize imports only
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/33253dc6
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/33253dc6
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/33253dc6
Branch: refs/heads/TINKERPOP-1602
Commit: 33253dc6b76aa5c3d5cb428a35a4379890e4a7a2
Parents: 67f7026
Author: Robert Dale <ro...@gmail.com>
Authored: Wed Jan 25 06:27:43 2017 -0500
Committer: Robert Dale <ro...@gmail.com>
Committed: Fri Jan 27 16:26:33 2017 -0500
----------------------------------------------------------------------
.../gremlin/server/GremlinServerIntegrateTest.java | 15 ++++++---------
1 file changed, 6 insertions(+), 9 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/33253dc6/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinServerIntegrateTest.java
----------------------------------------------------------------------
diff --git a/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinServerIntegrateTest.java b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinServerIntegrateTest.java
index 6e6d1ff..fe00a43 100644
--- a/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinServerIntegrateTest.java
+++ b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinServerIntegrateTest.java
@@ -55,7 +55,6 @@ import org.apache.tinkerpop.gremlin.process.traversal.step.util.BulkSet;
import org.apache.tinkerpop.gremlin.structure.Graph;
import org.apache.tinkerpop.gremlin.structure.T;
import org.apache.tinkerpop.gremlin.server.channel.NioChannelizer;
-import org.apache.tinkerpop.gremlin.structure.Vertex;
import org.apache.tinkerpop.gremlin.structure.util.detached.DetachedVertex;
import org.apache.tinkerpop.gremlin.structure.util.empty.EmptyGraph;
import org.apache.tinkerpop.gremlin.util.Log4jRecordingAppender;
@@ -80,7 +79,6 @@ import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicInteger;
-import java.util.function.Function;
import java.util.function.Supplier;
import java.util.stream.Collectors;
import java.util.stream.IntStream;
@@ -96,7 +94,6 @@ import static org.hamcrest.core.StringStartsWith.startsWith;
import static org.junit.Assert.assertNull;
import static org.junit.Assert.assertTrue;
import static org.junit.Assert.fail;
-import static org.junit.Assume.assumeThat;
import static org.junit.Assert.assertEquals;
/**
@@ -106,13 +103,13 @@ import static org.junit.Assert.assertEquals;
*/
public class GremlinServerIntegrateTest extends AbstractGremlinServerIntegrationTest {
- private static final String SERVER_KEY = "src/test/resources/server.key.pk8";
- private static final String SERVER_CRT = "src/test/resources/server.crt";
- private static final String KEY_PASS = "changeit";
- private static final String CLIENT_KEY = "src/test/resources/client.key.pk8";
- private static final String CLIENT_CRT = "src/test/resources/client.crt";
+ private static final String SERVER_KEY = "src/test/resources/server.key.pk8";
+ private static final String SERVER_CRT = "src/test/resources/server.crt";
+ private static final String KEY_PASS = "changeit";
+ private static final String CLIENT_KEY = "src/test/resources/client.key.pk8";
+ private static final String CLIENT_CRT = "src/test/resources/client.crt";
- private Log4jRecordingAppender recordingAppender = null;
+ private Log4jRecordingAppender recordingAppender = null;
private final Supplier<Graph> graphGetter = () -> server.getServerGremlinExecutor().getGraphManager().getGraphs().get("graph");
private final Configuration conf = new BaseConfiguration() {{
setProperty(Graph.GRAPH, RemoteGraph.class.getName());
[26/29] tinkerpop git commit: Support SSL client auth
Posted by rd...@apache.org.
Support SSL client auth
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/c62480bb
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/c62480bb
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/c62480bb
Branch: refs/heads/TINKERPOP-1602
Commit: c62480bbd563b9904654c23abdc66c4b4828b64e
Parents: c2a42e2
Author: Robert Dale <ro...@gmail.com>
Authored: Tue Jan 17 14:24:00 2017 -0500
Committer: Robert Dale <ro...@gmail.com>
Committed: Fri Jan 27 16:26:32 2017 -0500
----------------------------------------------------------------------
.../apache/tinkerpop/gremlin/server/AbstractChannelizer.java | 4 +++-
.../java/org/apache/tinkerpop/gremlin/server/Settings.java | 6 ++++++
2 files changed, 9 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/c62480bb/gremlin-server/src/main/java/org/apache/tinkerpop/gremlin/server/AbstractChannelizer.java
----------------------------------------------------------------------
diff --git a/gremlin-server/src/main/java/org/apache/tinkerpop/gremlin/server/AbstractChannelizer.java b/gremlin-server/src/main/java/org/apache/tinkerpop/gremlin/server/AbstractChannelizer.java
index 57c6994..d28fd4f 100644
--- a/gremlin-server/src/main/java/org/apache/tinkerpop/gremlin/server/AbstractChannelizer.java
+++ b/gremlin-server/src/main/java/org/apache/tinkerpop/gremlin/server/AbstractChannelizer.java
@@ -242,8 +242,10 @@ public abstract class AbstractChannelizer extends ChannelInitializer<SocketChann
builder = SslContextBuilder.forServer(keyCertChainFile, keyFile, sslSettings.keyPassword)
.trustManager(trustCertChainFile);
}
+
+
- builder.sslProvider(provider);
+ builder.clientAuth(sslSettings.needClientAuth).sslProvider(provider);
try {
return builder.build();
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/c62480bb/gremlin-server/src/main/java/org/apache/tinkerpop/gremlin/server/Settings.java
----------------------------------------------------------------------
diff --git a/gremlin-server/src/main/java/org/apache/tinkerpop/gremlin/server/Settings.java b/gremlin-server/src/main/java/org/apache/tinkerpop/gremlin/server/Settings.java
index 97e2875..a3b9545 100644
--- a/gremlin-server/src/main/java/org/apache/tinkerpop/gremlin/server/Settings.java
+++ b/gremlin-server/src/main/java/org/apache/tinkerpop/gremlin/server/Settings.java
@@ -18,6 +18,7 @@
*/
package org.apache.tinkerpop.gremlin.server;
+import io.netty.handler.ssl.ClientAuth;
import io.netty.handler.ssl.SslContext;
import org.apache.tinkerpop.gremlin.driver.MessageSerializer;
import org.apache.tinkerpop.gremlin.jsr223.GremlinPlugin;
@@ -420,6 +421,11 @@ public class Settings {
* contain an X.509 certificate chain in PEM format. {@code null} uses the system default.
*/
public String trustCertChainFile = null;
+
+ /**
+ * Require client certificate authentication
+ */
+ public ClientAuth needClientAuth = ClientAuth.NONE;
private SslContext sslContext;
[21/29] tinkerpop git commit: added a butt load more test cases
verifying will crazy corner cases.
Posted by rd...@apache.org.
added a butt load more test cases verifying will crazy corner cases.
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/989237fd
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/989237fd
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/989237fd
Branch: refs/heads/TINKERPOP-1602
Commit: 989237fd07a5693121400fe60bb3890c6573543d
Parents: f6b6697
Author: Marko A. Rodriguez <ok...@gmail.com>
Authored: Thu Jan 26 15:02:05 2017 -0700
Committer: Marko A. Rodriguez <ok...@gmail.com>
Committed: Fri Jan 27 14:24:18 2017 -0700
----------------------------------------------------------------------
.../optimization/SingleIterationStrategy.java | 22 ++++++++-----
.../SingleIterationStrategyTest.java | 34 ++++++++++++++------
.../SparkSingleIterationStrategy.java | 17 +++++-----
.../SparkSingleIterationStrategyTest.java | 26 ++++++++++++++-
4 files changed, 73 insertions(+), 26 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/989237fd/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategy.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategy.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategy.java
index efcbe9a..19d9854 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategy.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategy.java
@@ -29,9 +29,11 @@ import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.DefaultGraphTrav
import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__;
import org.apache.tinkerpop.gremlin.process.traversal.step.Barrier;
import org.apache.tinkerpop.gremlin.process.traversal.step.LambdaHolder;
+import org.apache.tinkerpop.gremlin.process.traversal.step.SideEffectCapable;
import org.apache.tinkerpop.gremlin.process.traversal.step.TraversalParent;
import org.apache.tinkerpop.gremlin.process.traversal.step.branch.LocalStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.EdgeVertexStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.map.GraphStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.LambdaFlatMapStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.LambdaMapStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.VertexStep;
@@ -41,6 +43,7 @@ import org.apache.tinkerpop.gremlin.process.traversal.strategy.optimization.Adja
import org.apache.tinkerpop.gremlin.process.traversal.strategy.optimization.FilterRankingStrategy;
import org.apache.tinkerpop.gremlin.process.traversal.strategy.optimization.IncidentToAdjacentStrategy;
import org.apache.tinkerpop.gremlin.process.traversal.strategy.optimization.InlineFilterStrategy;
+import org.apache.tinkerpop.gremlin.process.traversal.traverser.TraverserRequirement;
import org.apache.tinkerpop.gremlin.process.traversal.util.TraversalHelper;
import org.apache.tinkerpop.gremlin.structure.Direction;
import org.apache.tinkerpop.gremlin.structure.Graph;
@@ -93,22 +96,25 @@ public final class SingleIterationStrategy extends AbstractTraversalStrategy<Tra
final boolean beyondStarGraph =
TraversalHelper.hasStepOfAssignableClassRecursively(Scope.global, LambdaHolder.class, computerTraversal) ||
!TraversalHelper.isLocalStarGraph(computerTraversal);
- if (!beyondStarGraph && // if we move beyond the star graph, then localization is not possible.
- !(computerTraversal.getStartStep().getNextStep() instanceof EmptyStep) && // if its just a g.V()/E(), then don't localize
- !(computerTraversal.getStartStep().getNextStep() instanceof LocalStep) && // removes the potential for the infinite recursive application of the traversal
- !(computerTraversal.getStartStep().getNextStep() instanceof Barrier) && // if the second step is a barrier, no point in trying to localize anything
- !(TraversalHelper.getStepsOfAssignableClass(TraversalParent.class, computerTraversal). // this is a strict precaution that could be loosed with deeper logic on barriers in global children
+ if (!beyondStarGraph && // if we move beyond the star graph, then localization is not possible.
+ (computerTraversal.getStartStep() instanceof GraphStep) && // while GraphComputer requires GraphStep starts, this is just a precaution when inject() starts are supported
+ !(computerTraversal.getStartStep().getNextStep() instanceof EmptyStep) && // if its just a g.V()/E(), then don't localize
+ !(computerTraversal.getStartStep().getNextStep() instanceof LocalStep) && // removes the potential for the infinite recursive application of the traversal
+ !(computerTraversal.getStartStep().getNextStep() instanceof Barrier) && // if the second step is a barrier, no point in trying to localize anything
+ !computerTraversal.getTraverserRequirements().contains(TraverserRequirement.LABELED_PATH) && // this is to alleviate issues with DetachedElement in paths (TODO: when detachment is dynamic, remove this)
+ !computerTraversal.getTraverserRequirements().contains(TraverserRequirement.PATH) && // this is to alleviate issues with DetachedElement in paths (TODO: when detachment is dynamic, remove this)
+ TraversalHelper.getStepsOfAssignableClassRecursively(SideEffectCapable.class, computerTraversal).isEmpty() && // this is to alleviate issues with DetachedElement in paths (TODO: when detachment is dynamic, remove this)
+ !(TraversalHelper.getStepsOfAssignableClass(TraversalParent.class, computerTraversal). // this is a strict precaution that could be loosed with deeper logic on barriers in global children
stream().
filter(parent -> !parent.getGlobalChildren().isEmpty()).findAny().isPresent())) {
final Traversal.Admin<?, ?> newComputerTraversal = step.computerTraversal.getPure();
final Traversal.Admin localTraversal = new DefaultGraphTraversal<>();
final Step barrier = (Step) TraversalHelper.getFirstStepOfAssignableClass(Barrier.class, newComputerTraversal).orElse(null);
- final Step endStep = null == barrier ? newComputerTraversal.getEndStep() : barrier.getPreviousStep();
- if (!(endStep instanceof VertexStep || endStep instanceof EdgeVertexStep)) {
+ if (null == barrier || !(barrier instanceof TraversalParent && (barrier.getPreviousStep() instanceof VertexStep || barrier.getPreviousStep() instanceof EdgeVertexStep))) {
TraversalHelper.removeToTraversal(newComputerTraversal.getStartStep().getNextStep(), null == barrier ? EmptyStep.instance() : barrier, localTraversal);
assert !localTraversal.getSteps().isEmpty(); // given the if() constraints, this is impossible
- if (localTraversal.getSteps().size() > 1) { // if its just a single step, a local wrap will not alter its locus of computation
+ if (localTraversal.getSteps().size() > 1) { // if its just a single step, a local wrap will not alter its locus of computation
if (null == barrier)
TraversalHelper.insertTraversal(0, (Traversal.Admin) __.local(localTraversal), newComputerTraversal);
else
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/989237fd/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategyTest.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategyTest.java b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategyTest.java
index 612fb9d..09887f3 100644
--- a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategyTest.java
+++ b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategyTest.java
@@ -27,6 +27,7 @@ import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.DefaultGraphTrav
import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__;
import org.apache.tinkerpop.gremlin.process.traversal.strategy.optimization.AdjacentToIncidentStrategy;
import org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversalStrategies;
+import org.apache.tinkerpop.gremlin.structure.Column;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
@@ -35,6 +36,10 @@ import java.util.Arrays;
import java.util.Collection;
import java.util.Collections;
+import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.in;
+import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.inE;
+import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.out;
+import static org.apache.tinkerpop.gremlin.structure.Column.keys;
import static org.junit.Assert.assertEquals;
/**
@@ -77,8 +82,9 @@ public class SingleIterationStrategyTest {
{__.V().id(), __.V().id(), Collections.emptyList()},
{__.V().out().count(), __.V().out().count(), Collections.emptyList()},
{__.V().out().label().count(), __.V().out().label().count(), Collections.emptyList()},
- {__.V().in().id(), __.V().local(__.in().id()), Collections.emptyList()},
- {__.V().out().id(), __.V().local(__.out().id()), Collections.emptyList()},
+ {__.V().in().id(), __.V().local(in().id()), Collections.emptyList()},
+ {in().id(), in().id(), Collections.emptyList()}, // test inject
+ {__.V().out().id(), __.V().local(out().id()), Collections.emptyList()},
{__.V().both().id(), __.V().local(__.both().id()), Collections.emptyList()},
{__.V().outE().inV().id().count(), __.V().local(__.outE().inV().id()).count(), Collections.emptyList()},
{__.V().map(__.outE().inV()).count(), __.V().map(__.outE().inV()).count(), Collections.emptyList()},
@@ -86,15 +92,25 @@ public class SingleIterationStrategyTest {
{__.V().outE().map(__.inV()).id().count(), __.V().outE().map(__.inV()).id().count(), Collections.emptyList()},
{__.V().outE().map(__.inV()).count(), __.V().outE().map(__.inV()).count(), Collections.emptyList()},
{__.V().outE().map(__.inV()).values("name").count(), __.V().outE().map(__.inV()).values("name").count(), Collections.emptyList()},
- {__.V().outE().inV().count(), __.V().outE().inV().count(), Collections.emptyList()},
- {__.V().out().id().count(), __.V().local(__.out().id()).count(), Collections.emptyList()},
- {__.V().in().id().count(), __.V().local(__.in().id()).count(), Collections.emptyList()},
- {__.V().in().id().select("id-map").dedup().count(), __.V().local(__.in().id().select("id-map")).dedup().count(), Collections.emptyList()},
+ {__.V().outE().inV().count(), __.V().local(__.outE().inV()).count(), Collections.emptyList()},
+ {__.V().out().id().count(), __.V().local(out().id()).count(), Collections.emptyList()},
+ {__.V().in().id().count(), __.V().local(in().id()).count(), Collections.emptyList()},
+ {__.V().in().id().select("id-map").dedup().count(), __.V().local(in().id().select("id-map")).dedup().count(), Collections.emptyList()},
+ {__.V().in().id().groupCount().select(keys).unfold().dedup().count(), __.V().local(in().id()).groupCount().select(keys).unfold().dedup().count(), Collections.emptyList()},
{__.V().outE().values("weight"), __.V().outE().values("weight"), Collections.emptyList()},
{__.V().outE().values("weight").sum(), __.V().outE().values("weight").sum(), Collections.emptyList()},
- {__.V().inE().values("weight"), __.V().local(__.inE().values("weight")), Collections.emptyList()},
- {__.V().inE().values("weight").sum(), __.V().local(__.inE().values("weight")).sum(), Collections.emptyList()},
- {__.V().inE().values("weight").sum().dedup().count(), __.V().local(__.inE().values("weight")).sum().dedup().count(), Collections.emptyList()},
+ {__.V().inE().values("weight"), __.V().local(inE().values("weight")), Collections.emptyList()},
+ {__.V().inE().values("weight").sum(), __.V().local(inE().values("weight")).sum(), Collections.emptyList()},
+ {__.V().inE().values("weight").sum().dedup().count(), __.V().local(inE().values("weight")).sum().dedup().count(), Collections.emptyList()},
+ {__.V().as("a").out("knows").as("b").select("a", "b"), __.V().as("a").out("knows").as("b").select("a", "b"), Collections.emptyList()},
+ {__.V().out().groupCount("x").cap("x"), __.V().out().groupCount("x").cap("x"), Collections.emptyList()},
+ {__.V().outE().inV().groupCount().select(Column.values).unfold().dedup().count(), __.V().outE().inV().groupCount().select(Column.values).unfold().dedup().count(), Collections.emptyList()},
+ {__.V().outE().inV().groupCount(), __.V().outE().inV().groupCount(), Collections.emptyList()},
+ {__.V().outE().inV().groupCount().by("name"), __.V().outE().inV().groupCount().by("name"), Collections.emptyList()},
+ {__.V().inE().id().groupCount(), __.V().local(inE().id()).groupCount(), Collections.emptyList()},
+ {__.V().inE().values("weight").groupCount(), __.V().local(inE().values("weight")).groupCount(), Collections.emptyList()},
+ {__.V().outE().inV().tree(), __.V().outE().inV().tree(), Collections.emptyList()},
+ {__.V().in().values("name").groupCount(), __.V().in().values("name").groupCount(), Collections.emptyList()}
});
}
}
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/989237fd/spark-gremlin/src/main/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategy.java
----------------------------------------------------------------------
diff --git a/spark-gremlin/src/main/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategy.java b/spark-gremlin/src/main/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategy.java
index 2abb9b8..1288b0d 100644
--- a/spark-gremlin/src/main/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategy.java
+++ b/spark-gremlin/src/main/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategy.java
@@ -33,14 +33,13 @@ import org.apache.tinkerpop.gremlin.process.traversal.step.filter.FilterStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.EdgeVertexStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.LambdaFlatMapStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.LambdaMapStep;
-import org.apache.tinkerpop.gremlin.process.traversal.step.map.SelectOneStep;
-import org.apache.tinkerpop.gremlin.process.traversal.step.map.SelectStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.TraversalFlatMapStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.TraversalMapStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.VertexStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.SideEffectStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.util.EmptyStep;
import org.apache.tinkerpop.gremlin.process.traversal.strategy.AbstractTraversalStrategy;
+import org.apache.tinkerpop.gremlin.process.traversal.traverser.TraverserRequirement;
import org.apache.tinkerpop.gremlin.process.traversal.util.TraversalHelper;
import org.apache.tinkerpop.gremlin.structure.Direction;
import org.apache.tinkerpop.gremlin.structure.Graph;
@@ -83,7 +82,10 @@ public final class SparkSingleIterationStrategy extends AbstractTraversalStrateg
}
}
}
- if (!doesMessagePass && !SparkSingleIterationStrategy.endsWithInElement(computerTraversal)) {
+ if (!doesMessagePass &&
+ !SparkSingleIterationStrategy.endsWithInElement(computerTraversal) &&
+ !(computerTraversal.getTraverserRequirements().contains(TraverserRequirement.LABELED_PATH) || // todo: remove this when dynamic detachment is available in 3.3.0
+ computerTraversal.getTraverserRequirements().contains(TraverserRequirement.PATH))) { // todo: remove this when dynamic detachment is available in 3.3.0
step.setComputer(step.getComputer()
// if no message passing, don't partition the loaded graph
.configure(Constants.GREMLIN_SPARK_SKIP_PARTITIONER, true)
@@ -95,6 +97,9 @@ public final class SparkSingleIterationStrategy extends AbstractTraversalStrateg
private static final boolean endsWithInElement(final Traversal.Admin<?, ?> traversal) {
Step<?, ?> current = traversal.getEndStep();
+ while (current instanceof Barrier) { // clip off any tail barriers
+ current = current.getPreviousStep();
+ }
while (!(current instanceof EmptyStep)) {
if ((current instanceof VertexStep && (((VertexStep) current).returnsVertex() ||
!((VertexStep) current).getDirection().equals(Direction.OUT))) ||
@@ -107,11 +112,7 @@ public final class SparkSingleIterationStrategy extends AbstractTraversalStrateg
if (((TraversalParent) current).getGlobalChildren().stream().filter(SparkSingleIterationStrategy::endsWithInElement).findAny().isPresent())
return true;
}
- if (!(current instanceof FilterStep ||
- current instanceof SideEffectStep ||
- current instanceof SelectStep ||
- current instanceof SelectOneStep ||
- current instanceof Barrier)) {
+ if (!(current instanceof FilterStep || current instanceof SideEffectStep)) { // side-effects and filters do not alter the mapping and thus, deeper analysis is required
return false;
}
current = current.getPreviousStep();
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/989237fd/spark-gremlin/src/test/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategyTest.java
----------------------------------------------------------------------
diff --git a/spark-gremlin/src/test/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategyTest.java b/spark-gremlin/src/test/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategyTest.java
index 20596d7..b055eb8 100644
--- a/spark-gremlin/src/test/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategyTest.java
+++ b/spark-gremlin/src/test/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategyTest.java
@@ -43,6 +43,7 @@ import java.util.List;
import java.util.Map;
import java.util.UUID;
+import static org.apache.tinkerpop.gremlin.structure.Column.keys;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertTrue;
@@ -127,12 +128,23 @@ public class SparkSingleIterationStrategyTest extends AbstractSparkTest {
test(true, 6L, g.V().inE().id().count());
test(true, 6L, g.V().outE().count());
test(true, 4L, g.V().outE().inV().id().dedup().count());
- test(true, 6L, g.V().as("a").outE().inV().as("b").id().dedup("a", "b").by(T.id).count());
test(true, 4L, g.V().filter(__.in()).count());
test(true, 6L, g.V().sideEffect(__.in()).count());
+ test(true, 6L, g.V().map(__.constant("hello")).count());
+ test(true, g.V().groupCount());
+ test(true, g.V().groupCount("x"));
+ test(true, g.V().groupCount("x").cap("x"));
+ test(true, g.V().id().groupCount("x").cap("x"));
+ test(true, g.V().outE().groupCount());
+ test(true, g.V().outE().groupCount().by("weight"));
+ test(true, g.V().inE().id().groupCount());
+ test(true, g.V().inE().values("weight").groupCount());
/////
+ test(false, 6L, g.V().as("a").outE().inV().as("b").id().dedup("a", "b").by(T.id).count());
test(false, 6L, g.V().local(__.inE()).count());
test(false, 6L, g.V().outE().outV().count()); // todo: low probability traversal, but none the less could be optimized for
+ test(false, g.V().out().id().groupCount("x")); // todo: low probability traversal, but none the less could be optimized for
+ test(false, g.V().inE().values("weight").groupCount("x")); // todo: low probability traversal, but none the less could be optimized for
test(false, 6L, g.V().in().count());
test(false, 6L, g.V().flatMap(__.in()).count());
test(false, 4L, g.V().map(__.in()).count());
@@ -150,6 +162,18 @@ public class SparkSingleIterationStrategyTest extends AbstractSparkTest {
test(false, g.V().out().valueMap());
test(false, 6L, g.V().as("a").outE().inV().values("name").as("b").dedup("a", "b").count());
test(false, 2L, g.V().outE().inV().groupCount().select(Column.values).unfold().dedup().count());
+ test(false, g.V().out().groupCount("x"));
+ test(false, g.V().out().groupCount("x").cap("x"));
+ test(false, 6L, g.V().both().groupCount("x").cap("x").select(keys).unfold().count());
+ test(false, g.V().outE().inV().groupCount());
+ test(false, g.V().outE().inV().groupCount().by("name"));
+ test(false, g.V().outE().inV().tree());
+ test(false, g.V().inE().groupCount());
+ test(false, g.V().inE().groupCount().by("weight"));
+ test(false, g.V().in().values("name").groupCount());
+ test(false, g.V().out().groupCount("x"));
+ test(false, g.V().in().groupCount("x"));
+ test(false, g.V().both().groupCount("x").cap("x"));
}
private static <R> void test(boolean singleIteration, final Traversal<?, R> traversal) {
[19/29] tinkerpop git commit: I just made SingleIterationStrategy a
whole lot smarter. Not only is the code simpler with less checks,
we now support out().count() -> out().id().count() and out().dedup().count()
as out().id().dedup().count(). This will co
Posted by rd...@apache.org.
I just made SingleIterationStrategy a whole lot smarter. Not only is the code simpler with less checks, we now support out().count() -> out().id().count() and out().dedup().count() as out().id().dedup().count(). This will constrain numerous star local traversers from message passing in all GraphComputer implementations. Again, for Spark, this means we don't have to worry about caching the graph and save lots of clock cycles on many typical queries.
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/1819e05a
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/1819e05a
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/1819e05a
Branch: refs/heads/TINKERPOP-1602
Commit: 1819e05aa1545f14ae97a78c3626d38b732ead9d
Parents: b2f2a9d
Author: Marko A. Rodriguez <ok...@gmail.com>
Authored: Fri Jan 27 06:26:22 2017 -0700
Committer: Marko A. Rodriguez <ok...@gmail.com>
Committed: Fri Jan 27 14:24:18 2017 -0700
----------------------------------------------------------------------
.../optimization/SingleIterationStrategy.java | 105 +++++++++++--------
.../SingleIterationStrategyTest.java | 38 ++++---
.../SparkSingleIterationStrategyTest.java | 20 ++--
3 files changed, 97 insertions(+), 66 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/1819e05a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategy.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategy.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategy.java
index 96a2a0a..6b509ef 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategy.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategy.java
@@ -21,7 +21,6 @@ package org.apache.tinkerpop.gremlin.process.computer.traversal.strategy.optimiz
import org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.TraversalVertexProgramStep;
import org.apache.tinkerpop.gremlin.process.computer.util.EmptyMemory;
-import org.apache.tinkerpop.gremlin.process.traversal.Scope;
import org.apache.tinkerpop.gremlin.process.traversal.Step;
import org.apache.tinkerpop.gremlin.process.traversal.Traversal;
import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategy;
@@ -29,14 +28,18 @@ import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.DefaultGraphTrav
import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__;
import org.apache.tinkerpop.gremlin.process.traversal.step.Barrier;
import org.apache.tinkerpop.gremlin.process.traversal.step.LambdaHolder;
-import org.apache.tinkerpop.gremlin.process.traversal.step.SideEffectCapable;
import org.apache.tinkerpop.gremlin.process.traversal.step.TraversalParent;
import org.apache.tinkerpop.gremlin.process.traversal.step.branch.LocalStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.filter.DedupGlobalStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.filter.FilterStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.map.CountGlobalStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.EdgeVertexStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.GraphStep;
-import org.apache.tinkerpop.gremlin.process.traversal.step.map.LambdaFlatMapStep;
-import org.apache.tinkerpop.gremlin.process.traversal.step.map.LambdaMapStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.map.IdStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.map.TraversalFlatMapStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.map.TraversalMapStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.VertexStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.SideEffectStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.util.EmptyStep;
import org.apache.tinkerpop.gremlin.process.traversal.strategy.AbstractTraversalStrategy;
import org.apache.tinkerpop.gremlin.process.traversal.strategy.optimization.AdjacentToIncidentStrategy;
@@ -45,7 +48,6 @@ import org.apache.tinkerpop.gremlin.process.traversal.strategy.optimization.Inci
import org.apache.tinkerpop.gremlin.process.traversal.strategy.optimization.InlineFilterStrategy;
import org.apache.tinkerpop.gremlin.process.traversal.traverser.TraverserRequirement;
import org.apache.tinkerpop.gremlin.process.traversal.util.TraversalHelper;
-import org.apache.tinkerpop.gremlin.structure.Direction;
import org.apache.tinkerpop.gremlin.structure.Graph;
import org.apache.tinkerpop.gremlin.structure.util.empty.EmptyGraph;
@@ -66,13 +68,6 @@ public final class SingleIterationStrategy extends AbstractTraversalStrategy<Tra
FilterRankingStrategy.class,
InlineFilterStrategy.class));
- private static final Set<Class> MULTI_ITERATION_CLASSES = new HashSet<>(Arrays.asList(
- EdgeVertexStep.class,
- LambdaFlatMapStep.class,
- LambdaMapStep.class
- ));
-
-
private SingleIterationStrategy() {
}
@@ -81,48 +76,70 @@ public final class SingleIterationStrategy extends AbstractTraversalStrategy<Tra
// only process the first traversal step in an OLAP chain
TraversalHelper.getFirstStepOfAssignableClass(TraversalVertexProgramStep.class, traversal).ifPresent(step -> {
final Graph graph = traversal.getGraph().orElse(EmptyGraph.instance()); // best guess at what the graph will be as its dynamically determined
- final Traversal.Admin<?, ?> computerTraversal = step.generateProgram(graph, EmptyMemory.instance()).getTraversal().get().clone();
- // does the traversal as it is message pass?
- boolean doesMessagePass = TraversalHelper.hasStepOfAssignableClassRecursively(Scope.global, MULTI_ITERATION_CLASSES, computerTraversal);
- if (!doesMessagePass) {
- for (final VertexStep vertexStep : TraversalHelper.getStepsOfAssignableClassRecursively(Scope.global, VertexStep.class, computerTraversal)) {
- if (vertexStep.returnsVertex() || !vertexStep.getDirection().equals(Direction.OUT)) { // in-edges require message pass in OLAP
- doesMessagePass = true;
- break;
- }
- }
- }
- if (doesMessagePass && // if the traversal doesn't message pass, then don't try and localize it as its just wasted computation
- TraversalHelper.isLocalStarGraph(computerTraversal) && // if we move beyond the star graph, then localization is not possible.
- computerTraversal.getStartStep() instanceof GraphStep && // while GraphComputer requires GraphStep starts, this is just a precaution when inject() starts are supported
- !(computerTraversal.getStartStep().getNextStep() instanceof EmptyStep) && // if its just a g.V()/E(), then don't localize
- !(computerTraversal.getStartStep().getNextStep() instanceof LocalStep) && // removes the potential for the infinite recursive application of the traversal
- !(computerTraversal.getStartStep().getNextStep() instanceof Barrier) && // if the second step is a barrier, no point in trying to localize anything
- !computerTraversal.getTraverserRequirements().contains(TraverserRequirement.LABELED_PATH) && // this is to alleviate issues with DetachedElement in paths (TODO: when detachment is dynamic, remove this)
- !computerTraversal.getTraverserRequirements().contains(TraverserRequirement.PATH) && // this is to alleviate issues with DetachedElement in paths (TODO: when detachment is dynamic, remove this)
- !TraversalHelper.hasStepOfAssignableClassRecursively(LambdaHolder.class, computerTraversal) && // this is because we might be accessing data on an adjacent vertex and that is bad
- !TraversalHelper.hasStepOfAssignableClassRecursively(SideEffectCapable.class, computerTraversal) && // this is to alleviate issues with DetachedElement in paths (TODO: when detachment is dynamic, remove this)
- !(TraversalHelper.getStepsOfAssignableClass(TraversalParent.class, computerTraversal). // this is a strict precaution that could be loosed with deeper logic on barriers in global children
+ final Traversal.Admin<?, ?> compiledComputerTraversal = step.generateProgram(graph, EmptyMemory.instance()).getTraversal().get().clone();
+ if (!compiledComputerTraversal.isLocked())
+ compiledComputerTraversal.applyStrategies();
+ if (!TraversalHelper.hasStepOfAssignableClassRecursively(Arrays.asList(LocalStep.class, LambdaHolder.class), compiledComputerTraversal) &&
+ !compiledComputerTraversal.getTraverserRequirements().contains(TraverserRequirement.PATH) && // when dynamic detachment is provided in 3.3.0, remove this (TODO)
+ !compiledComputerTraversal.getTraverserRequirements().contains(TraverserRequirement.LABELED_PATH) && // when dynamic detachment is provided in 3.3.0, remove this (TODO)
+ !(TraversalHelper.getStepsOfAssignableClass(TraversalParent.class, compiledComputerTraversal). // this is a strict precaution that could be loosed with deeper logic on barriers in global children
stream().
filter(parent -> !parent.getGlobalChildren().isEmpty()).findAny().isPresent())) {
- final Traversal.Admin<?, ?> newComputerTraversal = step.computerTraversal.getPure();
- final Traversal.Admin localTraversal = new DefaultGraphTraversal<>();
- final Step barrier = (Step) TraversalHelper.getFirstStepOfAssignableClass(Barrier.class, newComputerTraversal).orElse(null);
- if (null == barrier || !(barrier instanceof TraversalParent && (barrier.getPreviousStep() instanceof VertexStep || barrier.getPreviousStep() instanceof EdgeVertexStep))) {
- TraversalHelper.removeToTraversal(newComputerTraversal.getStartStep().getNextStep(), null == barrier ? EmptyStep.instance() : barrier, localTraversal);
- assert !localTraversal.getSteps().isEmpty(); // given the if() constraints, this is impossible
- if (localTraversal.getSteps().size() > 1) { // if its just a single step, a local wrap will not alter its locus of computation
+ final Traversal.Admin<?, ?> computerTraversal = step.computerTraversal.getPure();
+ if (computerTraversal.getSteps().size() > 1 &&
+ !(computerTraversal.getStartStep().getNextStep() instanceof Barrier) &&
+ TraversalHelper.hasStepOfAssignableClassRecursively(Arrays.asList(VertexStep.class, EdgeVertexStep.class), computerTraversal) &&
+ TraversalHelper.isLocalStarGraph(computerTraversal)) {
+ final Traversal.Admin newChildTraversal = new DefaultGraphTraversal<>();
+ final Step barrier = (Step) TraversalHelper.getFirstStepOfAssignableClass(Barrier.class, computerTraversal).orElse(null);
+ if (SingleIterationStrategy.insertElementId(barrier)) // out().count() -> out().id().count()
+ TraversalHelper.insertBeforeStep(new IdStep(computerTraversal), barrier, computerTraversal);
+ if (!(SingleIterationStrategy.endsWithElement(null == barrier ? computerTraversal.getEndStep() : barrier.getPreviousStep()))) {
+ TraversalHelper.removeToTraversal(computerTraversal.getStartStep() instanceof GraphStep ?
+ computerTraversal.getStartStep().getNextStep() :
+ (Step) computerTraversal.getStartStep(), null == barrier ?
+ EmptyStep.instance() :
+ barrier, newChildTraversal);
if (null == barrier)
- TraversalHelper.insertTraversal(0, (Traversal.Admin) __.local(localTraversal), newComputerTraversal);
+ TraversalHelper.insertTraversal(0, (Traversal.Admin) __.local(newChildTraversal), computerTraversal);
else
- TraversalHelper.insertTraversal(barrier.getPreviousStep(), (Traversal.Admin) __.local(localTraversal), newComputerTraversal);
- step.setComputerTraversal(newComputerTraversal);
+ TraversalHelper.insertTraversal(barrier.getPreviousStep(), (Traversal.Admin) __.local(newChildTraversal), computerTraversal);
}
}
+ step.setComputerTraversal(computerTraversal);
}
});
}
+ private static boolean insertElementId(final Step<?, ?> barrier) {
+ if (!(barrier instanceof Barrier))
+ return false;
+ else if (!endsWithElement(barrier.getPreviousStep()))
+ return false;
+ else if (barrier instanceof CountGlobalStep)
+ return true;
+ else if (barrier instanceof DedupGlobalStep &&
+ ((DedupGlobalStep) barrier).getScopeKeys().isEmpty() &&
+ ((DedupGlobalStep) barrier).getLocalChildren().isEmpty() &&
+ barrier.getNextStep() instanceof CountGlobalStep)
+ return true;
+ else
+ return false;
+ }
+
+ private static boolean endsWithElement(Step<?, ?> currentStep) {
+ while (!(currentStep instanceof EmptyStep)) {
+ if (currentStep instanceof VertexStep || currentStep instanceof EdgeVertexStep) // TODO: add GraphStep but only if its mid-traversal V()/E()
+ return true;
+ else if (currentStep instanceof TraversalFlatMapStep || currentStep instanceof TraversalMapStep)
+ return endsWithElement(((TraversalParent) currentStep).getLocalChildren().get(0).getEndStep());
+ else if (!(currentStep instanceof FilterStep || currentStep instanceof SideEffectStep))
+ return false;
+ currentStep = currentStep.getPreviousStep();
+ }
+ return false;
+ }
+
@Override
public Set<Class<? extends OptimizationStrategy>> applyPrior() {
return PRIORS;
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/1819e05a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategyTest.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategyTest.java b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategyTest.java
index 09887f3..081a541 100644
--- a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategyTest.java
+++ b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategyTest.java
@@ -23,11 +23,14 @@ import org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.Traversa
import org.apache.tinkerpop.gremlin.process.traversal.Traversal;
import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategies;
import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategy;
+import org.apache.tinkerpop.gremlin.process.traversal.Traverser;
import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.DefaultGraphTraversal;
import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__;
import org.apache.tinkerpop.gremlin.process.traversal.strategy.optimization.AdjacentToIncidentStrategy;
import org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversalStrategies;
import org.apache.tinkerpop.gremlin.structure.Column;
+import org.apache.tinkerpop.gremlin.structure.Direction;
+import org.apache.tinkerpop.gremlin.structure.Vertex;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
@@ -35,10 +38,13 @@ import org.junit.runners.Parameterized;
import java.util.Arrays;
import java.util.Collection;
import java.util.Collections;
+import java.util.function.Function;
+import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.bothE;
import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.in;
import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.inE;
import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.out;
+import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.outE;
import static org.apache.tinkerpop.gremlin.structure.Column.keys;
import static org.junit.Assert.assertEquals;
@@ -76,29 +82,29 @@ public class SingleIterationStrategyTest {
@Parameterized.Parameters(name = "{0}")
public static Iterable<Object[]> generateTestParameters() {
-
+ final Function<Traverser<Vertex>,Vertex> mapFunction = t -> t.get().vertices(Direction.OUT).next();
return Arrays.asList(new Object[][]{
- {__.V().out().count(), __.V().outE().count(), Collections.singletonList(AdjacentToIncidentStrategy.instance())},
+ {__.V().out().count(), __.V().local(out().id()).count(), Collections.singletonList(AdjacentToIncidentStrategy.instance())}, // TODO
{__.V().id(), __.V().id(), Collections.emptyList()},
- {__.V().out().count(), __.V().out().count(), Collections.emptyList()},
+ {__.V().out().count(), __.V().local(out().id()).count(), Collections.emptyList()},
{__.V().out().label().count(), __.V().out().label().count(), Collections.emptyList()},
{__.V().in().id(), __.V().local(in().id()), Collections.emptyList()},
- {in().id(), in().id(), Collections.emptyList()}, // test inject
+ {in().id(), __.local(in().id()), Collections.emptyList()}, // test inject
{__.V().out().id(), __.V().local(out().id()), Collections.emptyList()},
{__.V().both().id(), __.V().local(__.both().id()), Collections.emptyList()},
- {__.V().outE().inV().id().count(), __.V().local(__.outE().inV().id()).count(), Collections.emptyList()},
- {__.V().map(__.outE().inV()).count(), __.V().map(__.outE().inV()).count(), Collections.emptyList()},
- {__.V().out().map(__.outE().inV()).count(), __.V().out().map(__.outE().inV()).count(), Collections.emptyList()},
- {__.V().outE().map(__.inV()).id().count(), __.V().outE().map(__.inV()).id().count(), Collections.emptyList()},
- {__.V().outE().map(__.inV()).count(), __.V().outE().map(__.inV()).count(), Collections.emptyList()},
+ {__.V().outE().inV().id().count(), __.V().local(outE().inV().id()).count(), Collections.emptyList()},
+ {__.V().map(outE().inV()).count(), __.V().local(__.map(outE().inV()).id()).count(), Collections.emptyList()},
+ {__.V().out().map(outE().inV()).count(), __.V().out().map(outE().inV()).count(), Collections.emptyList()},
+ {__.V().outE().map(__.inV()).id().count(), __.V().local(__.outE().map(__.inV()).id()).count(), Collections.emptyList()},
+ {__.V().outE().map(__.inV()).count(), __.V().local(outE().map(__.inV()).id()).count(), Collections.emptyList()},
{__.V().outE().map(__.inV()).values("name").count(), __.V().outE().map(__.inV()).values("name").count(), Collections.emptyList()},
- {__.V().outE().inV().count(), __.V().local(__.outE().inV()).count(), Collections.emptyList()},
+ {__.V().outE().inV().count(), __.V().local(outE().inV().id()).count(), Collections.emptyList()},
{__.V().out().id().count(), __.V().local(out().id()).count(), Collections.emptyList()},
{__.V().in().id().count(), __.V().local(in().id()).count(), Collections.emptyList()},
{__.V().in().id().select("id-map").dedup().count(), __.V().local(in().id().select("id-map")).dedup().count(), Collections.emptyList()},
{__.V().in().id().groupCount().select(keys).unfold().dedup().count(), __.V().local(in().id()).groupCount().select(keys).unfold().dedup().count(), Collections.emptyList()},
- {__.V().outE().values("weight"), __.V().outE().values("weight"), Collections.emptyList()},
- {__.V().outE().values("weight").sum(), __.V().outE().values("weight").sum(), Collections.emptyList()},
+ {__.V().outE().hasLabel("knows").values("weight"), __.V().local(outE().hasLabel("knows").values("weight")), Collections.emptyList()},
+ {__.V().outE().values("weight").sum(), __.V().local(outE().values("weight")).sum(), Collections.emptyList()},
{__.V().inE().values("weight"), __.V().local(inE().values("weight")), Collections.emptyList()},
{__.V().inE().values("weight").sum(), __.V().local(inE().values("weight")).sum(), Collections.emptyList()},
{__.V().inE().values("weight").sum().dedup().count(), __.V().local(inE().values("weight")).sum().dedup().count(), Collections.emptyList()},
@@ -110,7 +116,13 @@ public class SingleIterationStrategyTest {
{__.V().inE().id().groupCount(), __.V().local(inE().id()).groupCount(), Collections.emptyList()},
{__.V().inE().values("weight").groupCount(), __.V().local(inE().values("weight")).groupCount(), Collections.emptyList()},
{__.V().outE().inV().tree(), __.V().outE().inV().tree(), Collections.emptyList()},
- {__.V().in().values("name").groupCount(), __.V().in().values("name").groupCount(), Collections.emptyList()}
+ {__.V().in().values("name").groupCount(), __.V().in().values("name").groupCount(), Collections.emptyList()},
+ {__.V().outE().inV().groupCount("x"), __.V().outE().inV().groupCount("x"), Collections.emptyList()},
+ {__.V().in().dedup().count(), __.V().local(in().id()).dedup().count(), Collections.emptyList()},
+ {__.V().bothE().dedup().count(), __.V().local(bothE().id()).dedup().count(), Collections.emptyList()},
+ {__.V().bothE().dedup().by("name").count(), __.V().bothE().dedup().by("name").count(), Collections.emptyList()},
+ {__.V().map(mapFunction).inV().count(), __.V().map(mapFunction).inV().count(), Collections.emptyList()},
+ {__.V().groupCount().by(__.out().count()),__.V().groupCount().by(__.out().count()),Collections.emptyList()}
});
}
}
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/1819e05a/spark-gremlin/src/test/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategyTest.java
----------------------------------------------------------------------
diff --git a/spark-gremlin/src/test/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategyTest.java b/spark-gremlin/src/test/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategyTest.java
index b055eb8..4297649 100644
--- a/spark-gremlin/src/test/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategyTest.java
+++ b/spark-gremlin/src/test/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategyTest.java
@@ -139,22 +139,23 @@ public class SparkSingleIterationStrategyTest extends AbstractSparkTest {
test(true, g.V().outE().groupCount().by("weight"));
test(true, g.V().inE().id().groupCount());
test(true, g.V().inE().values("weight").groupCount());
+ test(true, 6L, g.V().outE().outV().count());
+ test(true, g.V().out().id().groupCount("x"));
+ test(true, g.V().inE().values("weight").groupCount("x"));
+ test(true, 6L, g.V().in().count());
+ test(true, 12L, g.V().both().count());
+ test(true, 6L, g.V().flatMap(__.in()).count());
+ test(true, 4L, g.V().map(__.in()).count());
+ test(true, 6L, g.V().inE().count());
+ test(true, 4L, g.V().outE().inV().dedup().count());
/////
test(false, 6L, g.V().as("a").outE().inV().as("b").id().dedup("a", "b").by(T.id).count());
test(false, 6L, g.V().local(__.inE()).count());
- test(false, 6L, g.V().outE().outV().count()); // todo: low probability traversal, but none the less could be optimized for
- test(false, g.V().out().id().groupCount("x")); // todo: low probability traversal, but none the less could be optimized for
- test(false, g.V().inE().values("weight").groupCount("x")); // todo: low probability traversal, but none the less could be optimized for
- test(false, 6L, g.V().in().count());
- test(false, 6L, g.V().flatMap(__.in()).count());
- test(false, 4L, g.V().map(__.in()).count());
+ test(false, 4L, g.V().outE().inV().dedup().by("name").count());
test(false, 6L, g.V().local(__.in()).count());
- test(false, 6L, g.V().inE().count());
test(false, g.V().outE().inV());
test(false, g.V().both());
- test(false, 12L, g.V().both().count());
test(false, g.V().outE().inV().dedup());
- test(false, 4L, g.V().outE().inV().dedup().count());
test(false, 2L, g.V().out().out().count());
test(false, 6L, g.V().as("a").map(__.both()).select("a").count());
test(false, g.V().out().values("name"));
@@ -168,6 +169,7 @@ public class SparkSingleIterationStrategyTest extends AbstractSparkTest {
test(false, g.V().outE().inV().groupCount());
test(false, g.V().outE().inV().groupCount().by("name"));
test(false, g.V().outE().inV().tree());
+ test(false, g.V().outE().inV().id().tree());
test(false, g.V().inE().groupCount());
test(false, g.V().inE().groupCount().by("weight"));
test(false, g.V().in().values("name").groupCount());
[18/29] tinkerpop git commit: Merge branch 'TINKERPOP-1583' into tp32
Posted by rd...@apache.org.
Merge branch 'TINKERPOP-1583' into tp32
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/f0875d7b
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/f0875d7b
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/f0875d7b
Branch: refs/heads/TINKERPOP-1602
Commit: f0875d7be6c0549a091656fa1bd7da22ed3409b0
Parents: c958b22 4dfba77
Author: Ted Wilmes <tw...@gmail.com>
Authored: Fri Jan 27 14:39:17 2017 -0600
Committer: Ted Wilmes <tw...@gmail.com>
Committed: Fri Jan 27 14:39:17 2017 -0600
----------------------------------------------------------------------
CHANGELOG.asciidoc | 1 +
.../optimization/PathRetractionStrategy.java | 66 +++++++++++++++++---
.../PathRetractionStrategyTest.java | 18 ++++--
3 files changed, 71 insertions(+), 14 deletions(-)
----------------------------------------------------------------------
[20/29] tinkerpop git commit: Added SingleIterationStrategy which is
able to rewrite a set of traversals to not use message passing in OLAP. This
is signficant for all GraphComputers as message passing is expense and
furthermore, for SparkGraphComputer a
Posted by rd...@apache.org.
Added SingleIterationStrategy which is able to rewrite a set of traversals to not use message passing in OLAP. This is signficant for all GraphComputers as message passing is expense and furthermore, for SparkGraphComputer as without message-passing, there is no need to cache or partition the graph once loaded.
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/f6b66977
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/f6b66977
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/f6b66977
Branch: refs/heads/TINKERPOP-1602
Commit: f6b669778ec058a555623c6119e0feaaac59c8be
Parents: f0875d7
Author: Marko A. Rodriguez <ok...@gmail.com>
Authored: Thu Jan 26 12:57:58 2017 -0700
Committer: Marko A. Rodriguez <ok...@gmail.com>
Committed: Fri Jan 27 14:24:18 2017 -0700
----------------------------------------------------------------------
CHANGELOG.asciidoc | 3 +-
.../step/map/TraversalVertexProgramStep.java | 5 +
.../optimization/SingleIterationStrategy.java | 132 +++++++++++++++++++
.../process/traversal/TraversalStrategies.java | 2 +
.../SingleIterationStrategyTest.java | 100 ++++++++++++++
.../SparkSingleIterationStrategy.java | 48 ++++++-
.../SparkSingleIterationStrategyTest.java | 89 +++++++++++--
7 files changed, 362 insertions(+), 17 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/f6b66977/CHANGELOG.asciidoc
----------------------------------------------------------------------
diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc
index 95cfb71..860d401 100644
--- a/CHANGELOG.asciidoc
+++ b/CHANGELOG.asciidoc
@@ -26,7 +26,8 @@ image::https://raw.githubusercontent.com/apache/tinkerpop/master/docs/static/ima
TinkerPop 3.2.4 (Release Date: NOT OFFICIALLY RELEASED YET)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-* Fixed a bug where `keepLabels` were not being pushed down into child traversals by `PathRetractionStrategy`.
+* Fixed a bug where `PathProcessor.keepLabels` were not being pushed down into child traversals by `PathRetractionStrategy`.
+* Added `SingleIterationStrategy` as a default `GraphComputer` strategy that can re-write some traversals to not require message passing.
* Fixed a bug associated with user-provided maps and `GroupSideEffectStep`.
* `GroupBiOperator` no longer maintains a detached traversal and thus, no more side-effect related OLAP inconsistencies.
* Added `ProjectedTraverser` which wraps a traverser with a `List<Object>` of projected data.
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/f6b66977/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/step/map/TraversalVertexProgramStep.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/step/map/TraversalVertexProgramStep.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/step/map/TraversalVertexProgramStep.java
index cb7db29..e866ce2 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/step/map/TraversalVertexProgramStep.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/step/map/TraversalVertexProgramStep.java
@@ -54,6 +54,11 @@ public final class TraversalVertexProgramStep extends VertexProgramStep implemen
return Collections.singletonList(this.computerTraversal.get());
}
+ public void setComputerTraversal(final Traversal.Admin<?,?> computerTraversal) {
+ this.computerTraversal = new PureTraversal<>(computerTraversal);
+ this.integrateChild(this.computerTraversal.get());
+ }
+
@Override
public String toString() {
return StringFactory.stepString(this, this.computerTraversal.get(), new GraphFilter(this.computer));
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/f6b66977/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategy.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategy.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategy.java
new file mode 100644
index 0000000..efcbe9a
--- /dev/null
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategy.java
@@ -0,0 +1,132 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.tinkerpop.gremlin.process.computer.traversal.strategy.optimization;
+
+import org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.TraversalVertexProgramStep;
+import org.apache.tinkerpop.gremlin.process.computer.util.EmptyMemory;
+import org.apache.tinkerpop.gremlin.process.traversal.Scope;
+import org.apache.tinkerpop.gremlin.process.traversal.Step;
+import org.apache.tinkerpop.gremlin.process.traversal.Traversal;
+import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategy;
+import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.DefaultGraphTraversal;
+import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__;
+import org.apache.tinkerpop.gremlin.process.traversal.step.Barrier;
+import org.apache.tinkerpop.gremlin.process.traversal.step.LambdaHolder;
+import org.apache.tinkerpop.gremlin.process.traversal.step.TraversalParent;
+import org.apache.tinkerpop.gremlin.process.traversal.step.branch.LocalStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.map.EdgeVertexStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.map.LambdaFlatMapStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.map.LambdaMapStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.map.VertexStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.util.EmptyStep;
+import org.apache.tinkerpop.gremlin.process.traversal.strategy.AbstractTraversalStrategy;
+import org.apache.tinkerpop.gremlin.process.traversal.strategy.optimization.AdjacentToIncidentStrategy;
+import org.apache.tinkerpop.gremlin.process.traversal.strategy.optimization.FilterRankingStrategy;
+import org.apache.tinkerpop.gremlin.process.traversal.strategy.optimization.IncidentToAdjacentStrategy;
+import org.apache.tinkerpop.gremlin.process.traversal.strategy.optimization.InlineFilterStrategy;
+import org.apache.tinkerpop.gremlin.process.traversal.util.TraversalHelper;
+import org.apache.tinkerpop.gremlin.structure.Direction;
+import org.apache.tinkerpop.gremlin.structure.Graph;
+import org.apache.tinkerpop.gremlin.structure.util.empty.EmptyGraph;
+
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.Set;
+
+/**
+ * @author Marko A. Rodriguez (http://markorodriguez.com)
+ */
+public final class SingleIterationStrategy extends AbstractTraversalStrategy<TraversalStrategy.OptimizationStrategy> implements TraversalStrategy.OptimizationStrategy {
+
+ private static final SingleIterationStrategy INSTANCE = new SingleIterationStrategy();
+
+ private static final Set<Class<? extends OptimizationStrategy>> PRIORS = new HashSet<>(Arrays.asList(
+ IncidentToAdjacentStrategy.class,
+ AdjacentToIncidentStrategy.class,
+ FilterRankingStrategy.class,
+ InlineFilterStrategy.class));
+
+ private static final Set<Class> MULTI_ITERATION_CLASSES = new HashSet<>(Arrays.asList(
+ EdgeVertexStep.class,
+ LambdaFlatMapStep.class,
+ LambdaMapStep.class
+ ));
+
+
+ private SingleIterationStrategy() {
+ }
+
+ @Override
+ public void apply(final Traversal.Admin<?, ?> traversal) {
+ // only process the first traversal step in an OLAP chain
+ TraversalHelper.getFirstStepOfAssignableClass(TraversalVertexProgramStep.class, traversal).ifPresent(step -> {
+ final Graph graph = traversal.getGraph().orElse(EmptyGraph.instance()); // best guess at what the graph will be as its dynamically determined
+ final Traversal.Admin<?, ?> computerTraversal = step.generateProgram(graph, EmptyMemory.instance()).getTraversal().get().clone();
+ // does the traversal as it is message pass?
+ boolean doesMessagePass = TraversalHelper.hasStepOfAssignableClassRecursively(Scope.global, MULTI_ITERATION_CLASSES, computerTraversal);
+ if (!doesMessagePass) {
+ for (final VertexStep vertexStep : TraversalHelper.getStepsOfAssignableClassRecursively(Scope.global, VertexStep.class, computerTraversal)) {
+ if (vertexStep.returnsVertex() || !vertexStep.getDirection().equals(Direction.OUT)) { // in-edges require message pass in OLAP
+ doesMessagePass = true;
+ break;
+ }
+ }
+ } // if the traversal doesn't message pass, then don't try and localize it as its just wasted computation
+ if (doesMessagePass) {
+ final boolean beyondStarGraph =
+ TraversalHelper.hasStepOfAssignableClassRecursively(Scope.global, LambdaHolder.class, computerTraversal) ||
+ !TraversalHelper.isLocalStarGraph(computerTraversal);
+ if (!beyondStarGraph && // if we move beyond the star graph, then localization is not possible.
+ !(computerTraversal.getStartStep().getNextStep() instanceof EmptyStep) && // if its just a g.V()/E(), then don't localize
+ !(computerTraversal.getStartStep().getNextStep() instanceof LocalStep) && // removes the potential for the infinite recursive application of the traversal
+ !(computerTraversal.getStartStep().getNextStep() instanceof Barrier) && // if the second step is a barrier, no point in trying to localize anything
+ !(TraversalHelper.getStepsOfAssignableClass(TraversalParent.class, computerTraversal). // this is a strict precaution that could be loosed with deeper logic on barriers in global children
+ stream().
+ filter(parent -> !parent.getGlobalChildren().isEmpty()).findAny().isPresent())) {
+
+ final Traversal.Admin<?, ?> newComputerTraversal = step.computerTraversal.getPure();
+ final Traversal.Admin localTraversal = new DefaultGraphTraversal<>();
+ final Step barrier = (Step) TraversalHelper.getFirstStepOfAssignableClass(Barrier.class, newComputerTraversal).orElse(null);
+ final Step endStep = null == barrier ? newComputerTraversal.getEndStep() : barrier.getPreviousStep();
+ if (!(endStep instanceof VertexStep || endStep instanceof EdgeVertexStep)) {
+ TraversalHelper.removeToTraversal(newComputerTraversal.getStartStep().getNextStep(), null == barrier ? EmptyStep.instance() : barrier, localTraversal);
+ assert !localTraversal.getSteps().isEmpty(); // given the if() constraints, this is impossible
+ if (localTraversal.getSteps().size() > 1) { // if its just a single step, a local wrap will not alter its locus of computation
+ if (null == barrier)
+ TraversalHelper.insertTraversal(0, (Traversal.Admin) __.local(localTraversal), newComputerTraversal);
+ else
+ TraversalHelper.insertTraversal(barrier.getPreviousStep(), (Traversal.Admin) __.local(localTraversal), newComputerTraversal);
+ step.setComputerTraversal(newComputerTraversal);
+ }
+ }
+ }
+ }
+ });
+ }
+
+ @Override
+ public Set<Class<? extends OptimizationStrategy>> applyPrior() {
+ return PRIORS;
+ }
+
+ public static SingleIterationStrategy instance() {
+ return INSTANCE;
+ }
+}
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/f6b66977/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/TraversalStrategies.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/TraversalStrategies.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/TraversalStrategies.java
index 015df70..a1cb99c 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/TraversalStrategies.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/TraversalStrategies.java
@@ -20,6 +20,7 @@ package org.apache.tinkerpop.gremlin.process.traversal;
import org.apache.tinkerpop.gremlin.process.computer.GraphComputer;
import org.apache.tinkerpop.gremlin.process.computer.traversal.strategy.optimization.GraphFilterStrategy;
+import org.apache.tinkerpop.gremlin.process.computer.traversal.strategy.optimization.SingleIterationStrategy;
import org.apache.tinkerpop.gremlin.process.traversal.strategy.decoration.ConnectiveStrategy;
import org.apache.tinkerpop.gremlin.process.traversal.strategy.finalization.ProfileStrategy;
import org.apache.tinkerpop.gremlin.process.traversal.strategy.optimization.AdjacentToIncidentStrategy;
@@ -222,6 +223,7 @@ public interface TraversalStrategies extends Serializable, Cloneable {
final TraversalStrategies graphComputerStrategies = new DefaultTraversalStrategies();
graphComputerStrategies.addStrategies(
GraphFilterStrategy.instance(),
+ SingleIterationStrategy.instance(),
OrderLimitStrategy.instance(),
PathProcessorStrategy.instance(),
ComputerVerificationStrategy.instance());
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/f6b66977/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategyTest.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategyTest.java b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategyTest.java
new file mode 100644
index 0000000..612fb9d
--- /dev/null
+++ b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/optimization/SingleIterationStrategyTest.java
@@ -0,0 +1,100 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.tinkerpop.gremlin.process.computer.traversal.strategy.optimization;
+
+import org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.TraversalVertexProgramStep;
+import org.apache.tinkerpop.gremlin.process.traversal.Traversal;
+import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategies;
+import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategy;
+import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.DefaultGraphTraversal;
+import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__;
+import org.apache.tinkerpop.gremlin.process.traversal.strategy.optimization.AdjacentToIncidentStrategy;
+import org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversalStrategies;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+
+import static org.junit.Assert.assertEquals;
+
+/**
+ * @author Marko A. Rodriguez (http://markorodriguez.com)
+ */
+@RunWith(Parameterized.class)
+public class SingleIterationStrategyTest {
+
+ @Parameterized.Parameter(value = 0)
+ public Traversal original;
+
+ @Parameterized.Parameter(value = 1)
+ public Traversal optimized;
+
+ @Parameterized.Parameter(value = 2)
+ public Collection<TraversalStrategy> otherStrategies;
+
+ @Test
+ public void doTest() {
+ final Traversal.Admin<?, ?> rootTraversal = new DefaultGraphTraversal<>();
+ final TraversalVertexProgramStep parent = new TraversalVertexProgramStep(rootTraversal, this.original.asAdmin());
+ rootTraversal.addStep(parent.asStep());
+ parent.setComputerTraversal(this.original.asAdmin());
+ final TraversalStrategies strategies = new DefaultTraversalStrategies();
+ strategies.addStrategies(SingleIterationStrategy.instance());
+ for (final TraversalStrategy strategy : this.otherStrategies) {
+ strategies.addStrategies(strategy);
+ }
+ rootTraversal.setStrategies(strategies);
+ rootTraversal.asAdmin().applyStrategies();
+ assertEquals(this.optimized, parent.computerTraversal.get());
+ }
+
+
+ @Parameterized.Parameters(name = "{0}")
+ public static Iterable<Object[]> generateTestParameters() {
+
+ return Arrays.asList(new Object[][]{
+ {__.V().out().count(), __.V().outE().count(), Collections.singletonList(AdjacentToIncidentStrategy.instance())},
+ {__.V().id(), __.V().id(), Collections.emptyList()},
+ {__.V().out().count(), __.V().out().count(), Collections.emptyList()},
+ {__.V().out().label().count(), __.V().out().label().count(), Collections.emptyList()},
+ {__.V().in().id(), __.V().local(__.in().id()), Collections.emptyList()},
+ {__.V().out().id(), __.V().local(__.out().id()), Collections.emptyList()},
+ {__.V().both().id(), __.V().local(__.both().id()), Collections.emptyList()},
+ {__.V().outE().inV().id().count(), __.V().local(__.outE().inV().id()).count(), Collections.emptyList()},
+ {__.V().map(__.outE().inV()).count(), __.V().map(__.outE().inV()).count(), Collections.emptyList()},
+ {__.V().out().map(__.outE().inV()).count(), __.V().out().map(__.outE().inV()).count(), Collections.emptyList()},
+ {__.V().outE().map(__.inV()).id().count(), __.V().outE().map(__.inV()).id().count(), Collections.emptyList()},
+ {__.V().outE().map(__.inV()).count(), __.V().outE().map(__.inV()).count(), Collections.emptyList()},
+ {__.V().outE().map(__.inV()).values("name").count(), __.V().outE().map(__.inV()).values("name").count(), Collections.emptyList()},
+ {__.V().outE().inV().count(), __.V().outE().inV().count(), Collections.emptyList()},
+ {__.V().out().id().count(), __.V().local(__.out().id()).count(), Collections.emptyList()},
+ {__.V().in().id().count(), __.V().local(__.in().id()).count(), Collections.emptyList()},
+ {__.V().in().id().select("id-map").dedup().count(), __.V().local(__.in().id().select("id-map")).dedup().count(), Collections.emptyList()},
+ {__.V().outE().values("weight"), __.V().outE().values("weight"), Collections.emptyList()},
+ {__.V().outE().values("weight").sum(), __.V().outE().values("weight").sum(), Collections.emptyList()},
+ {__.V().inE().values("weight"), __.V().local(__.inE().values("weight")), Collections.emptyList()},
+ {__.V().inE().values("weight").sum(), __.V().local(__.inE().values("weight")).sum(), Collections.emptyList()},
+ {__.V().inE().values("weight").sum().dedup().count(), __.V().local(__.inE().values("weight")).sum().dedup().count(), Collections.emptyList()},
+ });
+ }
+}
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/f6b66977/spark-gremlin/src/main/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategy.java
----------------------------------------------------------------------
diff --git a/spark-gremlin/src/main/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategy.java b/spark-gremlin/src/main/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategy.java
index a4acf4c..2abb9b8 100644
--- a/spark-gremlin/src/main/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategy.java
+++ b/spark-gremlin/src/main/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategy.java
@@ -23,12 +23,23 @@ import org.apache.tinkerpop.gremlin.hadoop.Constants;
import org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.TraversalVertexProgramStep;
import org.apache.tinkerpop.gremlin.process.computer.util.EmptyMemory;
import org.apache.tinkerpop.gremlin.process.traversal.Scope;
+import org.apache.tinkerpop.gremlin.process.traversal.Step;
import org.apache.tinkerpop.gremlin.process.traversal.Traversal;
import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategy;
+import org.apache.tinkerpop.gremlin.process.traversal.step.Barrier;
+import org.apache.tinkerpop.gremlin.process.traversal.step.TraversalParent;
+import org.apache.tinkerpop.gremlin.process.traversal.step.branch.LocalStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.filter.FilterStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.EdgeVertexStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.LambdaFlatMapStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.LambdaMapStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.map.SelectOneStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.map.SelectStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.map.TraversalFlatMapStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.map.TraversalMapStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.VertexStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.SideEffectStep;
+import org.apache.tinkerpop.gremlin.process.traversal.step.util.EmptyStep;
import org.apache.tinkerpop.gremlin.process.traversal.strategy.AbstractTraversalStrategy;
import org.apache.tinkerpop.gremlin.process.traversal.util.TraversalHelper;
import org.apache.tinkerpop.gremlin.structure.Direction;
@@ -48,9 +59,8 @@ public final class SparkSingleIterationStrategy extends AbstractTraversalStrateg
private static final Set<Class> MULTI_ITERATION_CLASSES = new HashSet<>(Arrays.asList(
EdgeVertexStep.class,
- LambdaMapStep.class, // maybe?
- LambdaFlatMapStep.class // maybe?
- // VertexStep is special as you need to see if the return class is Edge or Vertex (logic below)
+ LambdaMapStep.class,
+ LambdaFlatMapStep.class
));
private SparkSingleIterationStrategy() {
@@ -63,6 +73,7 @@ public final class SparkSingleIterationStrategy extends AbstractTraversalStrateg
final Traversal.Admin<?, ?> computerTraversal = step.generateProgram(graph, EmptyMemory.instance()).getTraversal().get().clone();
if (!computerTraversal.isLocked())
computerTraversal.applyStrategies();
+ ///
boolean doesMessagePass = TraversalHelper.hasStepOfAssignableClassRecursively(Scope.global, MULTI_ITERATION_CLASSES, computerTraversal);
if (!doesMessagePass) {
for (final VertexStep vertexStep : TraversalHelper.getStepsOfAssignableClassRecursively(Scope.global, VertexStep.class, computerTraversal)) {
@@ -72,18 +83,45 @@ public final class SparkSingleIterationStrategy extends AbstractTraversalStrateg
}
}
}
- if (!doesMessagePass) {
+ if (!doesMessagePass && !SparkSingleIterationStrategy.endsWithInElement(computerTraversal)) {
step.setComputer(step.getComputer()
// if no message passing, don't partition the loaded graph
.configure(Constants.GREMLIN_SPARK_SKIP_PARTITIONER, true)
- // if no message passing, don't cache the loaded graph
+ // if no message passing, don't cache the loaded graph
.configure(Constants.GREMLIN_SPARK_SKIP_GRAPH_CACHE, true));
}
}
}
+ private static final boolean endsWithInElement(final Traversal.Admin<?, ?> traversal) {
+ Step<?, ?> current = traversal.getEndStep();
+ while (!(current instanceof EmptyStep)) {
+ if ((current instanceof VertexStep && (((VertexStep) current).returnsVertex() ||
+ !((VertexStep) current).getDirection().equals(Direction.OUT))) ||
+ current instanceof EdgeVertexStep) {
+ return true;
+ } else if (current instanceof TraversalMapStep || current instanceof TraversalFlatMapStep || current instanceof LocalStep) {
+ if (endsWithInElement(((TraversalParent) current).getLocalChildren().get(0)))
+ return true;
+ } else if (current instanceof TraversalParent) {
+ if (((TraversalParent) current).getGlobalChildren().stream().filter(SparkSingleIterationStrategy::endsWithInElement).findAny().isPresent())
+ return true;
+ }
+ if (!(current instanceof FilterStep ||
+ current instanceof SideEffectStep ||
+ current instanceof SelectStep ||
+ current instanceof SelectOneStep ||
+ current instanceof Barrier)) {
+ return false;
+ }
+ current = current.getPreviousStep();
+ }
+ return false;
+ }
+
public static SparkSingleIterationStrategy instance() {
return INSTANCE;
}
+
}
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/f6b66977/spark-gremlin/src/test/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategyTest.java
----------------------------------------------------------------------
diff --git a/spark-gremlin/src/test/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategyTest.java b/spark-gremlin/src/test/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategyTest.java
index 4e43438..20596d7 100644
--- a/spark-gremlin/src/test/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategyTest.java
+++ b/spark-gremlin/src/test/java/org/apache/tinkerpop/gremlin/spark/process/computer/traversal/strategy/optimization/SparkSingleIterationStrategyTest.java
@@ -24,6 +24,7 @@ import org.apache.tinkerpop.gremlin.TestHelper;
import org.apache.tinkerpop.gremlin.hadoop.Constants;
import org.apache.tinkerpop.gremlin.hadoop.structure.io.gryo.GryoInputFormat;
import org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.TraversalVertexProgramStep;
+import org.apache.tinkerpop.gremlin.process.computer.traversal.strategy.optimization.SingleIterationStrategy;
import org.apache.tinkerpop.gremlin.process.traversal.Traversal;
import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource;
import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__;
@@ -32,7 +33,9 @@ import org.apache.tinkerpop.gremlin.spark.AbstractSparkTest;
import org.apache.tinkerpop.gremlin.spark.process.computer.SparkGraphComputer;
import org.apache.tinkerpop.gremlin.spark.process.computer.SparkHadoopGraphProvider;
import org.apache.tinkerpop.gremlin.spark.structure.io.PersistedOutputRDD;
+import org.apache.tinkerpop.gremlin.structure.Column;
import org.apache.tinkerpop.gremlin.structure.Graph;
+import org.apache.tinkerpop.gremlin.structure.T;
import org.apache.tinkerpop.gremlin.structure.util.GraphFactory;
import org.junit.Test;
@@ -60,29 +63,93 @@ public class SparkSingleIterationStrategyTest extends AbstractSparkTest {
configuration.setProperty(Constants.GREMLIN_HADOOP_DEFAULT_GRAPH_COMPUTER, SparkGraphComputer.class.getCanonicalName());
configuration.setProperty(Constants.GREMLIN_SPARK_PERSIST_CONTEXT, true);
+ /////////// WITHOUT SINGLE-ITERATION STRATEGY LESS SINGLE-PASS OPTIONS ARE AVAILABLE
+
Graph graph = GraphFactory.open(configuration);
- GraphTraversalSource g = graph.traversal().withComputer().withoutStrategies(SparkInterceptorStrategy.class);
+ GraphTraversalSource g = graph.traversal().withComputer().withoutStrategies(SparkInterceptorStrategy.class, SingleIterationStrategy.class);
assertFalse(g.getStrategies().toList().contains(SparkInterceptorStrategy.instance()));
- assertFalse(g.V().count().explain().toString().contains(SparkInterceptorStrategy.class.getSimpleName()));
+ assertFalse(g.V().count().explain().getStrategyTraversals().stream().filter(pair -> pair.getValue0() instanceof SparkInterceptorStrategy).findAny().isPresent());
+ assertFalse(g.getStrategies().toList().contains(SingleIterationStrategy.instance()));
+ assertFalse(g.V().count().explain().getStrategyTraversals().stream().filter(pair -> pair.getValue0() instanceof SingleIterationStrategy).findAny().isPresent());
assertTrue(g.getStrategies().toList().contains(SparkSingleIterationStrategy.instance()));
- assertTrue(g.V().count().explain().toString().contains(SparkSingleIterationStrategy.class.getSimpleName()));
+ assertTrue(g.V().count().explain().getStrategyTraversals().stream().filter(pair -> pair.getValue0() instanceof SparkSingleIterationStrategy).findAny().isPresent());
test(true, g.V().limit(10));
test(true, g.V().values("age").groupCount());
test(true, g.V().groupCount().by(__.out().count()));
test(true, g.V().outE());
- test(true, 6l, g.V().count());
- test(true, 6l, g.V().out().count());
- test(true, 6l, g.V().local(__.inE()).count());
- test(true, 6l, g.V().outE().inV().count());
+ test(true, 6L, g.V().count());
+ test(true, 6L, g.V().out().count());
+ test(true, 6L, g.V().outE().inV().count());
////
+ test(false, 6L, g.V().local(__.inE()).count());
test(false, g.V().outE().inV());
test(false, g.V().both());
- test(false, 12l, g.V().both().count());
+ test(false, 12L, g.V().both().count());
test(false, g.V().out().id());
- test(false, 2l, g.V().out().out().count());
- test(false, 6l, g.V().in().count());
- test(false, 6l, g.V().inE().count());
+ test(false, 2L, g.V().out().out().count());
+ test(false, 6L, g.V().in().count());
+ test(false, 6L, g.V().inE().count());
+
+ /////////// WITH SINGLE-ITERATION STRATEGY MORE SINGLE-PASS OPTIONS ARE AVAILABLE
+
+ graph = GraphFactory.open(configuration);
+ g = graph.traversal().withComputer().withoutStrategies(SparkInterceptorStrategy.class).withStrategies(SingleIterationStrategy.instance());
+ assertFalse(g.getStrategies().toList().contains(SparkInterceptorStrategy.instance()));
+ assertFalse(g.V().count().explain().getStrategyTraversals().stream().filter(pair -> pair.getValue0() instanceof SparkInterceptorStrategy).findAny().isPresent());
+ assertTrue(g.getStrategies().toList().contains(SingleIterationStrategy.instance()));
+ assertTrue(g.V().count().explain().getStrategyTraversals().stream().filter(pair -> pair.getValue0() instanceof SingleIterationStrategy).findAny().isPresent());
+ assertTrue(g.getStrategies().toList().contains(SparkSingleIterationStrategy.instance()));
+ assertTrue(g.V().count().explain().getStrategyTraversals().stream().filter(pair -> pair.getValue0() instanceof SparkSingleIterationStrategy).findAny().isPresent());
+
+ test(true, g.V().limit(10));
+ test(true, g.V().values("age").groupCount());
+ test(true, g.V().groupCount().by(__.out().count()));
+ test(true, g.V().outE());
+ test(true, 6L, g.V().outE().values("weight").count());
+ test(true, 6L, g.V().inE().values("weight").count());
+ test(true, 12L, g.V().bothE().values("weight").count());
+ test(true, g.V().bothE().values("weight"));
+ test(true, g.V().bothE().values("weight").limit(2));
+ test(true, 6L, g.V().count());
+ test(true, 6L, g.V().id().count());
+ test(true, 6L, g.V().out().count());
+ test(true, 6L, g.V().outE().inV().count());
+ test(true, 6L, g.V().outE().inV().id().count());
+ test(true, 2L, g.V().outE().inV().id().groupCount().select(Column.values).unfold().dedup().count());
+ test(true, g.V().out().id());
+ test(true, 6L, g.V().outE().valueMap().count());
+ test(true, g.V().outE().valueMap());
+ test(true, 6L, g.V().inE().valueMap().count());
+ test(true, g.V().inE().valueMap());
+ test(true, 12L, g.V().bothE().valueMap().count());
+ test(true, g.V().bothE().valueMap());
+ test(true, 6L, g.V().inE().id().count());
+ test(true, 6L, g.V().outE().count());
+ test(true, 4L, g.V().outE().inV().id().dedup().count());
+ test(true, 6L, g.V().as("a").outE().inV().as("b").id().dedup("a", "b").by(T.id).count());
+ test(true, 4L, g.V().filter(__.in()).count());
+ test(true, 6L, g.V().sideEffect(__.in()).count());
+ /////
+ test(false, 6L, g.V().local(__.inE()).count());
+ test(false, 6L, g.V().outE().outV().count()); // todo: low probability traversal, but none the less could be optimized for
+ test(false, 6L, g.V().in().count());
+ test(false, 6L, g.V().flatMap(__.in()).count());
+ test(false, 4L, g.V().map(__.in()).count());
+ test(false, 6L, g.V().local(__.in()).count());
+ test(false, 6L, g.V().inE().count());
+ test(false, g.V().outE().inV());
+ test(false, g.V().both());
+ test(false, 12L, g.V().both().count());
+ test(false, g.V().outE().inV().dedup());
+ test(false, 4L, g.V().outE().inV().dedup().count());
+ test(false, 2L, g.V().out().out().count());
+ test(false, 6L, g.V().as("a").map(__.both()).select("a").count());
+ test(false, g.V().out().values("name"));
+ test(false, g.V().out().properties("name"));
+ test(false, g.V().out().valueMap());
+ test(false, 6L, g.V().as("a").outE().inV().values("name").as("b").dedup("a", "b").count());
+ test(false, 2L, g.V().outE().inV().groupCount().select(Column.values).unfold().dedup().count());
}
private static <R> void test(boolean singleIteration, final Traversal<?, R> traversal) {
[12/29] tinkerpop git commit: Had to trap an exception better in
ServerGremlinExecutor during init CTR
Posted by rd...@apache.org.
Had to trap an exception better in ServerGremlinExecutor during init CTR
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/47e5ae2f
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/47e5ae2f
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/47e5ae2f
Branch: refs/heads/TINKERPOP-1602
Commit: 47e5ae2f319fb6eb20fa131a8136ded682b8045b
Parents: 28dffda
Author: Stephen Mallette <sp...@genoprime.com>
Authored: Wed Jan 25 16:39:38 2017 -0500
Committer: Stephen Mallette <sp...@genoprime.com>
Committed: Wed Jan 25 16:39:38 2017 -0500
----------------------------------------------------------------------
.../tinkerpop/gremlin/server/util/ServerGremlinExecutor.java | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/47e5ae2f/gremlin-server/src/main/java/org/apache/tinkerpop/gremlin/server/util/ServerGremlinExecutor.java
----------------------------------------------------------------------
diff --git a/gremlin-server/src/main/java/org/apache/tinkerpop/gremlin/server/util/ServerGremlinExecutor.java b/gremlin-server/src/main/java/org/apache/tinkerpop/gremlin/server/util/ServerGremlinExecutor.java
index 199c9f8..b9b7280 100644
--- a/gremlin-server/src/main/java/org/apache/tinkerpop/gremlin/server/util/ServerGremlinExecutor.java
+++ b/gremlin-server/src/main/java/org/apache/tinkerpop/gremlin/server/util/ServerGremlinExecutor.java
@@ -149,7 +149,13 @@ public class ServerGremlinExecutor<T extends ScheduledExecutorService> {
// the GremlinExecutor is using the GremlinScriptEngineManager. this is a bit of hack, but it at least allows
// the global bindings to become available after the init scripts are run (DefaultGremlinScriptEngineManager
// runs the init scripts when the GremlinScriptEngine is created.
- settings.scriptEngines.keySet().forEach(engineName -> gremlinExecutor.eval("1+1", engineName, Collections.emptyMap()).join());
+ settings.scriptEngines.keySet().forEach(engineName -> {
+ try {
+ gremlinExecutor.eval("1+1", engineName, Collections.emptyMap()).join();
+ } catch (Exception ex) {
+ logger.warn(String.format("Could not initialize {} ScriptEngine as script could not be evaluated - %s", engineName), ex);
+ }
+ });
// script engine init may have altered the graph bindings or maybe even created new ones - need to
// re-apply those references back
[14/29] tinkerpop git commit: Made some improvements to the
BindingsGremlinPlugin
Posted by rd...@apache.org.
Made some improvements to the BindingsGremlinPlugin
Made it possible to specify scopes to apply the bindings to. Added tests. CTR
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/c958b220
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/c958b220
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/c958b220
Branch: refs/heads/TINKERPOP-1602
Commit: c958b220b9559bb5fedbcc6e5c17f2dc142fb430
Parents: 91759a5
Author: Stephen Mallette <sp...@genoprime.com>
Authored: Thu Jan 26 14:46:36 2017 -0500
Committer: Stephen Mallette <sp...@genoprime.com>
Committed: Thu Jan 26 14:46:36 2017 -0500
----------------------------------------------------------------------
.../gremlin/jsr223/BindingsCustomizer.java | 11 ++-
.../gremlin/jsr223/BindingsGremlinPlugin.java | 48 ++++++++--
.../jsr223/DefaultBindingsCustomizer.java | 20 ++++-
.../DefaultGremlinScriptEngineManager.java | 16 +++-
.../gremlin/jsr223/LazyBindingsCustomizer.java | 20 +++++
.../gremlin/jsr223/ScriptFileGremlinPlugin.java | 4 +-
.../jsr223/BindingsScriptEngineTest.java | 95 ++++++++++++++++++++
.../jsr223/GremlinScriptEngineSuite.java | 1 +
8 files changed, 206 insertions(+), 9 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/c958b220/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/BindingsCustomizer.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/BindingsCustomizer.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/BindingsCustomizer.java
index 02c129e..e8dd9f3 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/BindingsCustomizer.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/BindingsCustomizer.java
@@ -21,7 +21,9 @@ package org.apache.tinkerpop.gremlin.jsr223;
import javax.script.Bindings;
/**
- * Provides a way to alter the bindings on a {@link GremlinScriptEngine}.
+ * Provides a way to alter the bindings on a {@link GremlinScriptEngine}. Those implementing {@link GremlinScriptEngine}
+ * instances need to be concerned with accounting for this {@link Customizer}. It is handled automatically by the
+ * {@link DefaultGremlinScriptEngineManager}.
*
* @author Stephen Mallette (http://stephen.genoprime.com)
*/
@@ -30,4 +32,11 @@ public interface BindingsCustomizer extends Customizer {
* Gets the bindings to add to a {@link GremlinScriptEngine}.
*/
public Bindings getBindings();
+
+ /**
+ * Gets the scope to which the bindings apply. The scope is determined by the {@code ScriptContext} values where
+ * "100" is {@code EngineScope} (bindings apply to the current {@link GremlinScriptEngine}) and "200" is
+ * {@code GlobalScope} (bindings apply to the engines created by the current {@link GremlinScriptEngineManager}.
+ */
+ public int getScope();
}
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/c958b220/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/BindingsGremlinPlugin.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/BindingsGremlinPlugin.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/BindingsGremlinPlugin.java
index feb501d..59304df 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/BindingsGremlinPlugin.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/BindingsGremlinPlugin.java
@@ -19,26 +19,48 @@
package org.apache.tinkerpop.gremlin.jsr223;
import javax.script.Bindings;
+import javax.script.ScriptContext;
import javax.script.SimpleBindings;
+import java.util.Collection;
+import java.util.HashSet;
import java.util.Map;
+import java.util.Set;
import java.util.function.Supplier;
/**
+ * A module that allows {@code Bindings} to be applied to a {@link GremlinScriptEngine}. The bindings are controled by
+ * their {@code scope} which are determined by the {@code ScriptContext} values where "100" is {@code ENGINE_SCOPE}
+ * (bindings apply to the current {@link GremlinScriptEngine}) and "200" is {@code GLOBAL_SCOPE} (bindings apply to the
+ * engines created by the current {@link GremlinScriptEngineManager}.
+ * <p/>
+ * Note that bindings are applied in the following order:
+ * <ol>
+ * <li>The order in which the {@link GremlinScriptEngine} is requested from the {@link GremlinScriptEngineManager}</li>
+ * <li>The order in which the {@code BindingsGremlinPlugin} instances are added to the {@link GremlinScriptEngineManager}</li>
+ * </ol>
+ * <p/>
+ * Moreover, they will override one another within a scope and among scopes. For instance, {@code ENGINE_SCOPE} bindings
+ * will override {@code GLOBAL_SCOPE}. Those bindings that are {@code GLOBAL_SCOPE} and applied to a single
+ * {@link GremlinScriptEngine} via an {@link Builder#appliesTo} configuration will still appear present to all other
+ * {@link GremlinScriptEngine} created by the {@link GremlinScriptEngineManager} that the plugin was bound to.
+ * <p/>
+ * This {@link GremlinPlugin} is not enabled for the {@code ServiceLoader}. It is designed to be instantiated manually.
+ *
* @author Stephen Mallette (http://stephen.genoprime.com)
*/
public class BindingsGremlinPlugin extends AbstractGremlinPlugin {
private static final String NAME = "tinkerpop.bindings";
private BindingsGremlinPlugin(final Builder builder) {
- this(builder.bindings);
+ super(NAME, builder.appliesTo, new DefaultBindingsCustomizer(builder.bindings, builder.scope));
}
- public BindingsGremlinPlugin(final Bindings bindings) {
- super(NAME, new DefaultBindingsCustomizer(bindings));
+ public BindingsGremlinPlugin(final Bindings bindings, final int scope) {
+ super(NAME, new DefaultBindingsCustomizer(bindings, scope));
}
- public BindingsGremlinPlugin(final Supplier<Bindings> bindingsSupplier) {
- super(NAME, new LazyBindingsCustomizer(bindingsSupplier));
+ public BindingsGremlinPlugin(final Supplier<Bindings> bindingsSupplier, final int scope) {
+ super(NAME, new LazyBindingsCustomizer(bindingsSupplier, scope));
}
public static BindingsGremlinPlugin.Builder build() {
@@ -48,14 +70,30 @@ public class BindingsGremlinPlugin extends AbstractGremlinPlugin {
public static final class Builder {
private Bindings bindings = new SimpleBindings();
+ private int scope = ScriptContext.ENGINE_SCOPE;
+ private final Set<String> appliesTo = new HashSet<>();
private Builder() {}
+ /**
+ * The name of the {@link GremlinScriptEngine} that this module will apply to. Setting no values here will
+ * make the module available to all the engines.
+ */
+ public BindingsGremlinPlugin.Builder appliesTo(final Collection<String> scriptEngineName) {
+ this.appliesTo.addAll(scriptEngineName);
+ return this;
+ }
+
public Builder bindings(final Map<String, Object> bindings) {
this.bindings = new SimpleBindings(bindings);
return this;
}
+ public Builder scope(final int scope) {
+ this.scope = scope;
+ return this;
+ }
+
public BindingsGremlinPlugin create() {
return new BindingsGremlinPlugin(this);
}
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/c958b220/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/DefaultBindingsCustomizer.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/DefaultBindingsCustomizer.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/DefaultBindingsCustomizer.java
index 0073d39..cee5182 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/DefaultBindingsCustomizer.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/DefaultBindingsCustomizer.java
@@ -19,22 +19,40 @@
package org.apache.tinkerpop.gremlin.jsr223;
import javax.script.Bindings;
+import javax.script.ScriptContext;
/**
- * Default implementation of the {@link BindingsCustomizer}.
+ * Default implementation of the {@link BindingsCustomizer}. If this customizer is applied directly to a
+ * {@link GremlinScriptEngine} it will not apply {@code GLOBAL_SCOPE} bindings. Those can only be applied if the
+ * customizer is applied via the {@link GremlinScriptEngineManager} (which would do so through the
+ * {@link BindingsGremlinPlugin}.
*
* @author Stephen Mallette (http://stephen.genoprime.com)
*/
public class DefaultBindingsCustomizer implements BindingsCustomizer {
private final Bindings bindings;
+ private final int scope;
+ /**
+ * Creates a new object with {@code ScriptContext.ENGINE_SCOPE}.
+ */
public DefaultBindingsCustomizer(final Bindings bindings) {
+ this(bindings, ScriptContext.ENGINE_SCOPE);
+ }
+
+ public DefaultBindingsCustomizer(final Bindings bindings, final int scope) {
this.bindings = bindings;
+ this.scope = scope;
}
@Override
public Bindings getBindings() {
return bindings;
}
+
+ @Override
+ public int getScope() {
+ return scope;
+ }
}
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/c958b220/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/DefaultGremlinScriptEngineManager.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/DefaultGremlinScriptEngineManager.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/DefaultGremlinScriptEngineManager.java
index 436deac..01b4957 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/DefaultGremlinScriptEngineManager.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/DefaultGremlinScriptEngineManager.java
@@ -23,7 +23,6 @@ import javax.script.ScriptContext;
import java.security.AccessController;
import java.security.PrivilegedAction;
import java.util.ArrayList;
-import java.util.Collection;
import java.util.Collections;
import java.util.HashMap;
import java.util.HashSet;
@@ -442,8 +441,23 @@ public class DefaultGremlinScriptEngineManager implements GremlinScriptEngineMan
private GremlinScriptEngine createGremlinScriptEngine(final GremlinScriptEngineFactory spi) {
final GremlinScriptEngine engine = spi.getScriptEngine();
+
+ // merge in bindings that are marked with global scope. these get applied to all GremlinScriptEngine instances
+ getCustomizers(spi.getEngineName()).stream()
+ .filter(p -> p instanceof BindingsCustomizer)
+ .map(p -> ((BindingsCustomizer) p))
+ .filter(bc -> bc.getScope() == ScriptContext.GLOBAL_SCOPE)
+ .forEach(bc -> globalScope.putAll(bc.getBindings()));
engine.setBindings(getBindings(), ScriptContext.GLOBAL_SCOPE);
+ // merge in bindings that are marked with engine scope. these get applied to only those GremlinScriptEngine
+ // instances that are of this gremlin language
+ getCustomizers(spi.getEngineName()).stream()
+ .filter(p -> p instanceof BindingsCustomizer)
+ .map(p -> ((BindingsCustomizer) p))
+ .filter(bc -> bc.getScope() == ScriptContext.ENGINE_SCOPE)
+ .forEach(bc -> engine.setBindings(bc.getBindings(), ScriptContext.ENGINE_SCOPE));
+
final List<ScriptCustomizer> scriptCustomizers = getCustomizers(spi.getEngineName()).stream()
.filter(p -> p instanceof ScriptCustomizer)
.map(p -> ((ScriptCustomizer) p))
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/c958b220/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/LazyBindingsCustomizer.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/LazyBindingsCustomizer.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/LazyBindingsCustomizer.java
index 01ae662..8feb361 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/LazyBindingsCustomizer.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/LazyBindingsCustomizer.java
@@ -19,21 +19,41 @@
package org.apache.tinkerpop.gremlin.jsr223;
import javax.script.Bindings;
+import javax.script.ScriptContext;
import java.util.function.Supplier;
/**
+ * A customizer implementation that provides bindings to a {@link GremlinScriptEngine}. If this customizer is applied
+ * directly to a {@link GremlinScriptEngine} it will not apply {@code GLOBAL_SCOPE} bindings. Those can only be applied
+ * if the customizer is applied via the {@link GremlinScriptEngineManager} (which would do so through the
+ * {@link BindingsGremlinPlugin}.
+ *
* @author Stephen Mallette (http://stephen.genoprime.com)
*/
public class LazyBindingsCustomizer implements BindingsCustomizer {
private final Supplier<Bindings> bindingsSupplier;
+ private final int scope;
+ /**
+ * Creates a new object with {@code ScriptContext.ENGINE_SCOPE}.
+ */
public LazyBindingsCustomizer(final Supplier<Bindings> bindingsSupplier) {
+ this(bindingsSupplier, ScriptContext.ENGINE_SCOPE);
+ }
+
+ public LazyBindingsCustomizer(final Supplier<Bindings> bindingsSupplier, final int scope) {
this.bindingsSupplier = bindingsSupplier;
+ this.scope = scope;
}
@Override
public Bindings getBindings() {
return bindingsSupplier.get();
}
+
+ @Override
+ public int getScope() {
+ return scope;
+ }
}
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/c958b220/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/ScriptFileGremlinPlugin.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/ScriptFileGremlinPlugin.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/ScriptFileGremlinPlugin.java
index 0131ca2..3a3f4be 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/ScriptFileGremlinPlugin.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/ScriptFileGremlinPlugin.java
@@ -28,7 +28,9 @@ import java.util.Set;
/**
* Loads scripts from one or more files into the {@link GremlinScriptEngine} at startup. This {@link GremlinPlugin} is
- * not enabled for the {@code ServiceLoader}. It is designed to be instantiated manually.
+ * not enabled for the {@code ServiceLoader}. It is designed to be instantiated manually. Those implementing
+ * {@link GremlinScriptEngine} instances need to be concerned with accounting for this {@link Customizer}. It is
+ * handled automatically by the {@link DefaultGremlinScriptEngineManager}.
*
* @author Stephen Mallette (http://stephen.genoprime.com)
*/
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/c958b220/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/jsr223/BindingsScriptEngineTest.java
----------------------------------------------------------------------
diff --git a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/jsr223/BindingsScriptEngineTest.java b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/jsr223/BindingsScriptEngineTest.java
new file mode 100644
index 0000000..e02b834
--- /dev/null
+++ b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/jsr223/BindingsScriptEngineTest.java
@@ -0,0 +1,95 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.gremlin.jsr223;
+
+import org.junit.Test;
+
+import javax.script.Bindings;
+import javax.script.ScriptContext;
+import javax.script.ScriptException;
+import javax.script.SimpleBindings;
+import java.util.Collections;
+
+import static org.apache.tinkerpop.gremlin.jsr223.GremlinScriptEngineSuite.ENGINE_TO_TEST;
+import static org.junit.Assert.assertEquals;
+
+/**
+ * @author Stephen Mallette (http://stephen.genoprime.com)
+ */
+public class BindingsScriptEngineTest {
+
+ @Test
+ public void shouldIncludeGlobalBindings() throws ScriptException {
+ final GremlinScriptEngineManager manager = new DefaultGremlinScriptEngineManager();
+ final Bindings b = new SimpleBindings();
+ b.put("x", 1);
+ b.put("y", 2);
+
+ manager.addPlugin(BindingsGremlinPlugin.build().
+ bindings(b).
+ scope(ScriptContext.GLOBAL_SCOPE).
+ appliesTo(Collections.singletonList(ENGINE_TO_TEST)).create());
+
+ final GremlinScriptEngine engine = manager.getEngineByName(ENGINE_TO_TEST);
+ assertEquals(1, engine.eval("x"));
+ assertEquals(2, engine.eval("y"));
+ }
+
+ @Test
+ public void shouldIncludeEngineBindings() throws ScriptException {
+ final GremlinScriptEngineManager manager = new DefaultGremlinScriptEngineManager();
+ final Bindings b = new SimpleBindings();
+ b.put("x", 1);
+ b.put("y", 2);
+
+ manager.addPlugin(BindingsGremlinPlugin.build().
+ bindings(b).
+ scope(ScriptContext.ENGINE_SCOPE).
+ appliesTo(Collections.singletonList(ENGINE_TO_TEST)).create());
+
+ final GremlinScriptEngine engine = manager.getEngineByName(ENGINE_TO_TEST);
+ assertEquals(1, engine.eval("x"));
+ assertEquals(2, engine.eval("y"));
+ }
+
+ @Test
+ public void shouldIncludeEngineBindingsToOverrideGlobalBindings() throws ScriptException {
+ final GremlinScriptEngineManager manager = new DefaultGremlinScriptEngineManager();
+
+ final Bindings b1 = new SimpleBindings();
+ b1.put("x", 1);
+ b1.put("y", 2);
+ manager.addPlugin(BindingsGremlinPlugin.build().
+ bindings(b1).
+ scope(ScriptContext.GLOBAL_SCOPE).
+ appliesTo(Collections.singletonList(ENGINE_TO_TEST)).create());
+
+ final Bindings b2 = new SimpleBindings();
+ b2.put("x", 100);
+ b2.put("y", 200);
+ manager.addPlugin(BindingsGremlinPlugin.build().
+ bindings(b2).
+ scope(ScriptContext.ENGINE_SCOPE).
+ appliesTo(Collections.singletonList(ENGINE_TO_TEST)).create());
+
+ final GremlinScriptEngine engine = manager.getEngineByName(ENGINE_TO_TEST);
+ assertEquals(100, engine.eval("x"));
+ assertEquals(200, engine.eval("y"));
+ }
+}
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/c958b220/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/jsr223/GremlinScriptEngineSuite.java
----------------------------------------------------------------------
diff --git a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/jsr223/GremlinScriptEngineSuite.java b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/jsr223/GremlinScriptEngineSuite.java
index 8ccde91..70c585c 100644
--- a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/jsr223/GremlinScriptEngineSuite.java
+++ b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/jsr223/GremlinScriptEngineSuite.java
@@ -29,6 +29,7 @@ public class GremlinScriptEngineSuite extends Suite {
static String ENGINE_TO_TEST;
private static final Class<?>[] allTests = new Class<?>[]{
+ BindingsScriptEngineTest.class,
CachedGremlinScriptEngineManagerTest.class,
GremlinEnabledScriptEngineTest.class,
ScriptEngineLambdaTest.class };
[16/29] tinkerpop git commit: Removed unused and wildcard imports.
Posted by rd...@apache.org.
Removed unused and wildcard imports.
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/c9999bd1
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/c9999bd1
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/c9999bd1
Branch: refs/heads/TINKERPOP-1602
Commit: c9999bd1343e7b867e0db8c712e2bf5f0392b961
Parents: cf7a077
Author: Ted Wilmes <tw...@gmail.com>
Authored: Fri Jan 27 13:15:02 2017 -0600
Committer: Ted Wilmes <tw...@gmail.com>
Committed: Fri Jan 27 13:15:02 2017 -0600
----------------------------------------------------------------------
.../optimization/PathRetractionStrategy.java | 3 ---
.../optimization/PathRetractionStrategyTest.java | 19 +++++++++++++++----
2 files changed, 15 insertions(+), 7 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/c9999bd1/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategy.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategy.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategy.java
index da6fd0e..439d568 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategy.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategy.java
@@ -28,8 +28,6 @@ import org.apache.tinkerpop.gremlin.process.traversal.step.TraversalParent;
import org.apache.tinkerpop.gremlin.process.traversal.step.branch.RepeatStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.filter.DedupGlobalStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.filter.FilterStep;
-import org.apache.tinkerpop.gremlin.process.traversal.step.filter.TraversalFilterStep;
-import org.apache.tinkerpop.gremlin.process.traversal.step.filter.WhereTraversalStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.MatchStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.NoOpBarrierStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.SelectOneStep;
@@ -38,7 +36,6 @@ import org.apache.tinkerpop.gremlin.process.traversal.strategy.AbstractTraversal
import org.apache.tinkerpop.gremlin.process.traversal.traverser.TraverserRequirement;
import org.apache.tinkerpop.gremlin.process.traversal.util.PathUtil;
import org.apache.tinkerpop.gremlin.process.traversal.util.TraversalHelper;
-import org.apache.tinkerpop.gremlin.util.iterator.IteratorUtils;
import org.javatuples.Pair;
import java.util.ArrayList;
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/c9999bd1/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategyTest.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategyTest.java b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategyTest.java
index d12b621..2f99fd2 100644
--- a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategyTest.java
+++ b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategyTest.java
@@ -37,8 +37,19 @@ import java.util.Arrays;
import java.util.List;
import java.util.Set;
-import static org.apache.tinkerpop.gremlin.process.traversal.P.*;
-import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.*;
+import static org.apache.tinkerpop.gremlin.process.traversal.P.eq;
+import static org.apache.tinkerpop.gremlin.process.traversal.P.gte;
+import static org.apache.tinkerpop.gremlin.process.traversal.P.neq;
+import static org.apache.tinkerpop.gremlin.process.traversal.P.without;
+import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.as;
+import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.bothE;
+import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.limit;
+import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.out;
+import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.project;
+import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.select;
+import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.store;
+import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.values;
+import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.where;
import static org.apache.tinkerpop.gremlin.process.traversal.strategy.optimization.PathRetractionStrategy.MAX_BARRIER_SIZE;
import static org.junit.Assert.assertEquals;
@@ -109,8 +120,8 @@ public class PathRetractionStrategyTest {
{__.V().as("a").out().where(out().where(neq("a"))).out(), "[[[]]]", null},
{__.V().as("a").out().where(neq("a")).out().select("a"), "[[a], []]", null},
{__.V().as("a").out().as("b").where(neq("a")).out().select("a", "b").out().select("b"), "[[a, b], [b], []]", null},
- {__.V().match(__.as("a").out().as("b")), "[[a, b]]", null},
- {__.V().match(__.as("a").out().as("b")).select("a"), "[[a], []]", null},
+ {__.V().match(as("a").out().as("b")), "[[a, b]]", null},
+ {__.V().match(as("a").out().as("b")).select("a"), "[[a], []]", null},
{__.V().out().out().match(
as("a").in("created").as("b"),
as("b").in("knows").as("c")).select("c").out("created").where(neq("a")).values("name"),
[29/29] tinkerpop git commit: added tests
Posted by rd...@apache.org.
added tests
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/67f70260
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/67f70260
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/67f70260
Branch: refs/heads/TINKERPOP-1602
Commit: 67f70260f7062672b49f9b9b8f5d6b82915988ec
Parents: c05e072
Author: Robert Dale <ro...@gmail.com>
Authored: Mon Jan 23 14:27:37 2017 -0500
Committer: Robert Dale <ro...@gmail.com>
Committed: Fri Jan 27 16:26:33 2017 -0500
----------------------------------------------------------------------
.../server/GremlinServerIntegrateTest.java | 87 +++++++++++++++++++-
gremlin-server/src/test/resources/client.crt | 40 +++++++++
.../src/test/resources/client.key.pk8 | 47 +++++++++++
gremlin-server/src/test/resources/server.crt | 41 +++++++++
.../src/test/resources/server.key.pk8 | 47 +++++++++++
5 files changed, 261 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/67f70260/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinServerIntegrateTest.java
----------------------------------------------------------------------
diff --git a/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinServerIntegrateTest.java b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinServerIntegrateTest.java
index 1743e89..6e6d1ff 100644
--- a/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinServerIntegrateTest.java
+++ b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinServerIntegrateTest.java
@@ -18,6 +18,7 @@
*/
package org.apache.tinkerpop.gremlin.server;
+import io.netty.handler.ssl.ClientAuth;
import io.netty.handler.ssl.SslContext;
import io.netty.handler.ssl.SslContextBuilder;
import io.netty.handler.ssl.SslProvider;
@@ -105,7 +106,13 @@ import static org.junit.Assert.assertEquals;
*/
public class GremlinServerIntegrateTest extends AbstractGremlinServerIntegrationTest {
- private Log4jRecordingAppender recordingAppender = null;
+ private static final String SERVER_KEY = "src/test/resources/server.key.pk8";
+ private static final String SERVER_CRT = "src/test/resources/server.crt";
+ private static final String KEY_PASS = "changeit";
+ private static final String CLIENT_KEY = "src/test/resources/client.key.pk8";
+ private static final String CLIENT_CRT = "src/test/resources/client.crt";
+
+ private Log4jRecordingAppender recordingAppender = null;
private final Supplier<Graph> graphGetter = () -> server.getServerGremlinExecutor().getGraphManager().getGraphs().get("graph");
private final Configuration conf = new BaseConfiguration() {{
setProperty(Graph.GRAPH, RemoteGraph.class.getName());
@@ -165,6 +172,36 @@ public class GremlinServerIntegrateTest extends AbstractGremlinServerIntegration
settings.ssl.enabled = true;
settings.ssl.overrideSslContext(createServerSslContext());
break;
+ case "shouldEnableSslAndClientCertificateAuth":
+ settings.ssl = new Settings.SslSettings();
+ settings.ssl.enabled = true;
+ settings.ssl.needClientAuth = ClientAuth.REQUIRE;
+ settings.ssl.keyCertChainFile = SERVER_CRT;
+ settings.ssl.keyFile = SERVER_KEY;
+ settings.ssl.keyPassword =KEY_PASS;
+ // Trust the client
+ settings.ssl.trustCertChainFile = CLIENT_CRT;
+ break;
+ case "shouldEnableSslAndClientCertificateAuthAndFailWithoutCert":
+ settings.ssl = new Settings.SslSettings();
+ settings.ssl.enabled = true;
+ settings.ssl.needClientAuth = ClientAuth.REQUIRE;
+ settings.ssl.keyCertChainFile = SERVER_CRT;
+ settings.ssl.keyFile = SERVER_KEY;
+ settings.ssl.keyPassword =KEY_PASS;
+ // Trust the client
+ settings.ssl.trustCertChainFile = CLIENT_CRT;
+ break;
+ case "shouldEnableSslAndClientCertificateAuthAndFailWithoutTrustedClientCert":
+ settings.ssl = new Settings.SslSettings();
+ settings.ssl.enabled = true;
+ settings.ssl.needClientAuth = ClientAuth.REQUIRE;
+ settings.ssl.keyCertChainFile = SERVER_CRT;
+ settings.ssl.keyFile = SERVER_KEY;
+ settings.ssl.keyPassword =KEY_PASS;
+ // Trust ONLY the server cert
+ settings.ssl.trustCertChainFile = SERVER_CRT;
+ break;
case "shouldStartWithDefaultSettings":
// test with defaults exception for port because we want to keep testing off of 8182
final Settings defaultSettings = new Settings();
@@ -373,7 +410,55 @@ public class GremlinServerIntegrateTest extends AbstractGremlinServerIntegration
cluster.close();
}
}
+
+ @Test
+ public void shouldEnableSslAndClientCertificateAuth() {
+ final Cluster cluster = TestClientFactory.build().enableSsl(true)
+ .keyCertChainFile(CLIENT_CRT).keyFile(CLIENT_KEY)
+ .keyPassword(KEY_PASS).trustCertificateChainFile(SERVER_CRT).create();
+ final Client client = cluster.connect();
+
+ try {
+ assertEquals("test", client.submit("'test'").one().getString());
+ } finally {
+ cluster.close();
+ }
+ }
+
+ @Test
+ public void shouldEnableSslAndClientCertificateAuthAndFailWithoutCert() {
+ final Cluster cluster = TestClientFactory.build().enableSsl(true).create();
+ final Client client = cluster.connect();
+
+ try {
+ client.submit("'test'").one();
+ fail("Should throw exception because ssl client auth is enabled on the server but client does not have a cert");
+ } catch(Exception x) {
+ final Throwable root = ExceptionUtils.getRootCause(x);
+ assertThat(root, instanceOf(TimeoutException.class));
+ } finally {
+ cluster.close();
+ }
+ }
+
+ @Test
+ public void shouldEnableSslAndClientCertificateAuthAndFailWithoutTrustedClientCert() {
+ final Cluster cluster = TestClientFactory.build().enableSsl(true)
+ .keyCertChainFile(CLIENT_CRT).keyFile(CLIENT_KEY)
+ .keyPassword(KEY_PASS).trustCertificateChainFile(SERVER_CRT).create();
+ final Client client = cluster.connect();
+ try {
+ client.submit("'test'").one();
+ fail("Should throw exception because ssl client auth is enabled on the server but does not trust client's cert");
+ } catch(Exception x) {
+ final Throwable root = ExceptionUtils.getRootCause(x);
+ assertThat(root, instanceOf(TimeoutException.class));
+ } finally {
+ cluster.close();
+ }
+ }
+
@Test
public void shouldRespectHighWaterMarkSettingAndSucceed() throws Exception {
// the highwatermark should get exceeded on the server and thus pause the writes, but have no problem catching
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/67f70260/gremlin-server/src/test/resources/client.crt
----------------------------------------------------------------------
diff --git a/gremlin-server/src/test/resources/client.crt b/gremlin-server/src/test/resources/client.crt
new file mode 100644
index 0000000..297335b
--- /dev/null
+++ b/gremlin-server/src/test/resources/client.crt
@@ -0,0 +1,40 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+-----BEGIN CERTIFICATE-----
+MIIDajCCAlICCQCG0QGj7JgLvTANBgkqhkiG9w0BAQsFADB2MQswCQYDVQQGEwJV
+UzELMAkGA1UECAwCTkMxEDAOBgNVBAcMB1JhbGVpZ2gxEzARBgNVBAoMCkNsaWVu
+dCBBcHAxDzANBgNVBAMMBmNsaWVudDEiMCAGCSqGSIb3DQEJARYTZ3JlbWxpbi11
+c2VyQGNsaWVudDAgFw0xNzAxMjMxODQzMjNaGA8yMTE2MTIzMDE4NDMyM1owdjEL
+MAkGA1UEBhMCVVMxCzAJBgNVBAgMAk5DMRAwDgYDVQQHDAdSYWxlaWdoMRMwEQYD
+VQQKDApDbGllbnQgQXBwMQ8wDQYDVQQDDAZjbGllbnQxIjAgBgkqhkiG9w0BCQEW
+E2dyZW1saW4tdXNlckBjbGllbnQwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEK
+AoIBAQDU7U5tNN+0HCKZX31ZKMZwAUI7qGCpYXd05z5SKRWesNsrEeCLnu1DgYhL
+j+hSi2LI+RwTF45DH8zEPIzQ6HEMzuCd2uy7bdDrXv6H/tFUx9Iw0ea5oXGX1qNa
+tzjPTSmw22VXbSo+B5EG0coC5oDy9SpYb2HxeDmegI2OZL6ROFPKbCUTzyJfqTpy
+1mdgnnKTVuQdtWvj/sXDAZzRqtFHwBkHezKCOC4yLNi5+pI01+0V7FbtyCqH7iPS
+VS9VKsLuhPkzAkRh/x9CxaSrwicyzyB3Kyfg7kjmtdrUOrfgxdw+MMPog3JrGoLr
+tfvc9LUh0ImpLPngyWhwc5iNGwmLAgMBAAEwDQYJKoZIhvcNAQELBQADggEBAEi0
+IVUIbDuirNivJXXXT5eFUgfAx5iPjbR1XBjCuHO061d0B8itU7cidKs0y5mDoauy
+vuywNBih6FAsHoYLrqI1gt65fybGxq3wlhnqdLH7GDeHw65e2PB2x+M4NtQlkPTq
+dUgUtQzAo8Hc4DNR5BrvCtLjqT7Knq8QHtGLys8eoDur09894+6WeUjrgUTp88Jl
+uqrZqRHvdMW7sge73cpU1dsDJW0rJqCLZ+qA2V+ZRRCQY1oHuHeK6Dkokabaq3rr
+WrpxfppIPCusJx4nnIwu4d0gZwAKwabOS8lJPjV0frRkA0BuAEpMIbOwZ10Tw7ZM
+2HzamAOiiks4NFDSs94=
+-----END CERTIFICATE-----
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/67f70260/gremlin-server/src/test/resources/client.key.pk8
----------------------------------------------------------------------
diff --git a/gremlin-server/src/test/resources/client.key.pk8 b/gremlin-server/src/test/resources/client.key.pk8
new file mode 100644
index 0000000..213cce3
--- /dev/null
+++ b/gremlin-server/src/test/resources/client.key.pk8
@@ -0,0 +1,47 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+-----BEGIN ENCRYPTED PRIVATE KEY-----
+MIIE6TAbBgkqhkiG9w0BBQMwDgQIuH3rVXHniaECAggABIIEyDzHWny/fttnN6ze
+Q/iNE1z72HBg2anTK3gHdu8W3F9W6BSLhuKaUumxcxX6IfkZmxinqwUxSNrRBswr
+s71EZ7QmX2eaeiWxlo6zP0I898fzqvSBWk6NhntV1SVG++/CMeIMZARs85OxcNGN
+fKEXtBUf5jlo+hy9bVgQRO2M1vHzek/deOHyj1eXwEi2Oxcw1ly3o9lRayIFJFou
+0olp58h8pD5qOsSfIrNG6bHBg00eP1ILE1FUPCdOfDrQEjiKKVqllCXdsUz/KjBY
+0iGYMw3aT2LELnibTy0alFswGaDBRqlzGSIGrejdGTOBj7NmeiW2yOY+aUjCkdXw
+ll862oItlJZO/B16EGgI+czdXZgcL5YRGVOu6loOlGxU60Y681Acjn9wdlxxOqXa
+te7khLGX6qm3ceMJ2DQbMO8JS6UBsGjZqq/ictwfGxNDiTeiWmoyrQvO3qkaqsEt
+iLBJdSsogzxx+hfj+xiBRorf9Gvkk6hDnzeuZl9Voak5qY7RrcmxD8ecdh3g2faO
+VFtxTAxmHJakgGF7fZ7jQoxZJzzRn+rjIhATaEH1/MMME77Z+8rkE74oxFOrizDU
+Eb4AMUV/xgrAtQLGpIFALL7oPk2r1exFgywL40q5/BcXCx65eliXtyoBIc5X+t90
+fTB8lA3K5mT2MXzwMyMPv2n8d8Ta2OjisRHjSue6AqNqifBuniVfSpHP9OUlsKaL
+u3eN8mSuPDhlo7RGG8+B7mSqGDjORR/+BJvwgFreIyurOL22g887uPwjzPXs6lll
+3VFclsrfEqNMP6kBsXlHNAxjT10WoWrF+ONPEyg9PrzTGryTRqW+ptzBqtiQrVag
+/KMee3sdg6i9GN+EGc43DNJzxxlFXvT9kBvOk66tOPEE5963k3ZikjuAX7xBVOvM
+AlqzHHW727fHXqYp0yabWTpr72RuUGQOqPZiMsWd6x65HL2I0WHP9PXTf1vMmnj3
+NgiN1paATl5L8S9Jt8WL8iLskVt2f+CAKJuQfamY5Fg54m/cJzNHV8nAMTTl+0EN
+9vUhvn672wZd9JfB4PMtrIhYFNJElJmmdAwjFrop6goXA/UGgf3M4QNiQetFGxWP
+aJ0tMSdA/ax9nGA5LU6iCwPe51ExrQJAVFw+oE3I8+J1oz1fQOl8zIgyOFwG+bJx
+/Y/JyraEssZ5RLtaGgcm9vZm8Wo8a5TQCbqhoY8x7MwyF7/VpbRZ3bGEUFnWd/yF
+1tCPeZ+q5HN79P3qfZzAcFAU3z3HawXDwQ3XO3Plix04Vjr+QFENeJxhL+3FXtLx
+4nF+Y9Fq6I5x8YQLrPYkIyRz8xWxVBQFZTeNhWx5nYWNcDXgz/S8v3ipY+EPEj9Q
+uOQWNdSJ9XOR+Ju/KSP0151guLkUwpHBCi/CFY4TgD6iFglTyLuZpW2bbim1mmDI
+LSDHwrERQWNmcNznK4PIw227w6EZqUEI0jjkANhXQG6dz1VkaTIZAoEVWHhFNi8S
+gSPkhsWN822QzxZQb+74JoUFBTifP50giaCd6p2HKw8U20FAPyseVIY3IsYatuLU
+8VhfeBQ1GLOOy8/5mRyjL+Gjn/OfNPwps9uuOogx/jGf0JhtjAEY2WNoEEB9DkUv
+AijT0ck1+DM3by4MlA==
+-----END ENCRYPTED PRIVATE KEY-----
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/67f70260/gremlin-server/src/test/resources/server.crt
----------------------------------------------------------------------
diff --git a/gremlin-server/src/test/resources/server.crt b/gremlin-server/src/test/resources/server.crt
new file mode 100644
index 0000000..b777e34
--- /dev/null
+++ b/gremlin-server/src/test/resources/server.crt
@@ -0,0 +1,41 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+-----BEGIN CERTIFICATE-----
+MIIDrjCCApYCCQCg8hxjInTcQzANBgkqhkiG9w0BAQsFADCBlzELMAkGA1UEBhMC
+VVMxCzAJBgNVBAgMAkNBMRQwEgYDVQQHDAtTYW50YSBDbGFyYTEZMBcGA1UECgwQ
+QXBhY2hlIFRpbmtlclBvcDEQMA4GA1UECwwHR3JlbWxpbjESMBAGA1UEAwwJbG9j
+YWxob3N0MSQwIgYJKoZIhvcNAQkBFhVzcG1hbGxldHRlQGFwYWNoZS5vcmcwIBcN
+MTcwMTIzMTg0MTEwWhgPMjExNjEyMzAxODQxMTBaMIGXMQswCQYDVQQGEwJVUzEL
+MAkGA1UECAwCQ0ExFDASBgNVBAcMC1NhbnRhIENsYXJhMRkwFwYDVQQKDBBBcGFj
+aGUgVGlua2VyUG9wMRAwDgYDVQQLDAdHcmVtbGluMRIwEAYDVQQDDAlsb2NhbGhv
+c3QxJDAiBgkqhkiG9w0BCQEWFXNwbWFsbGV0dGVAYXBhY2hlLm9yZzCCASIwDQYJ
+KoZIhvcNAQEBBQADggEPADCCAQoCggEBAK5l+5DDrSGq1n1vYIMbb95buWlo4u5s
+V8wHNmeFCuqNvGfxermHS88SSy6qvdSXISfe+kj9Jkfn1Pjx4czwcnF/q4Au3Gc6
+T9MiAKJUfb4+MwPKERacFobk6LTreKpzvXymAhhM1ktvvA7/opZ+nENDEoIJK/KS
+CY9vRWeHqI1Q+Rf5Nrqw8eySq/ZhPDlppsu5sQr1ghSsuzXhpYNhq8VBMsysL2Zi
+VF5DFMqg9yhBkxrqo70W/pjo4kQJ/iF/4d/3HHicSCVq3/NCkWMVg0HeOZ1eVaZ/
+D6EXv8cYwyaRmf7SldE7RtdbAU0M+Y0Lmuoi2evls0Oiqf8uxEDXeLUCAwEAATAN
+BgkqhkiG9w0BAQsFAAOCAQEAJFAAUxYeUbf6tAGEKCXISted10TybPW/qCxOXEPN
+TTO+xvnzksbcbzOc2X0N+yYIKtkfiuWgD9UJ4QnLSeEKmouMbkTk0ToYJj7SrviI
+f+9R1IiZMiwz+n0igETkEGOK0Ql26Z4g3kc1IueSD07QOLASTvVLtEyoya7LD4S6
+jk1LnbpbHVmgHY4kmtsg6lVQ1zkqrsDQg9goh8dI5AlNCudpd8zLxzsPbm+Q2+DC
+Wd4A2lKdh3rbY2LYpbVhBj9c6E0laaqgyGC7s37XfmyBp4wYlX/30p6RpCR2rcRW
+SKh5NXN52Xx3WuiP42wm9ZC/de8gaODrW8n44xlEaMPJ3A==
+-----END CERTIFICATE-----
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/67f70260/gremlin-server/src/test/resources/server.key.pk8
----------------------------------------------------------------------
diff --git a/gremlin-server/src/test/resources/server.key.pk8 b/gremlin-server/src/test/resources/server.key.pk8
new file mode 100644
index 0000000..9dc6720
--- /dev/null
+++ b/gremlin-server/src/test/resources/server.key.pk8
@@ -0,0 +1,47 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+-----BEGIN ENCRYPTED PRIVATE KEY-----
+MIIE6TAbBgkqhkiG9w0BBQMwDgQIJi2M3dPcx+cCAggABIIEyG/0PwTaqm3ekGkn
+8daHjfcqxCpvwOku0pCCgV7AMsrsj4TvMOHOkId+xKVs4AlXV2grPfTQoMrIFFp0
+26hzpZOHEjAe9XbtzVWgphwLVeWOr5ugovyXPxdCqMBrn251y9Yg6csh5pPrsHEp
+RLTZPQPvEVIhA0tRkPPag5BSlj//nH7PnXYRm4VSRg7WQS15ieHsC1xDM9zYni/b
+ffU++twOGvCdvtGd/lqjJj3dPomKynb6Y4cEc73vQ6cQjeXN1W1Nil4QsDZ53H3U
+KvoWuEVYDf1WDFySgHMFgePHXMMWA8ZkgS6rOrV1TDnoZIhG5BSe7aJNJQdVVMub
+NAxZB4C1unIICuAtsP9IEAhevbfNGySi0jbrnun0pTD22Q95HZVycl1YWamrr1Hl
+17TmHQT7zh4GLlXVI3L/FevoswbuR7misyqE3ketNpxdLcHKs82MVLCwbLzuqm60
+tpSm/jrexhNErKU28kdPzWCwruEHdrlOuMSS8N2YgSZxHIjEId8pZED8dsnPvqhe
+CAEa3F68Js9sngeMvThYirpr0wOsSGmGCDrXTmQvTw+q8C+DJLWpjfpKZA4g624f
+E3CUojwOaPrmaEa25eYm0oYDstY8QncG7nBAuljeXWBYfxARYcLI/bLC/M/q+UeS
+gzjQgoacH4r2Y7rwEA0aeYC+9TWn5rPHPWegdc01A2e7OlYKrlu9C1aKZr//GyBM
+lEVRc0u8RL+RVhmp7ftRyATDG9kJR+zDT26hewaEa5atnPKLjf+37hu7a/6GI+2X
+dpWTzmvWal4eEkFuBg2ekl1lCsuuCUBDWyYrlhsWPYTOSKJ/PTuLq7HVqdGB9kEw
+SavnXV5LPOaY4nAzJTdRk9DHdxSwSrjZ5rvMMD1CAbob/GA9t8aYaGcAWtTwMFs4
+sps40mmab3X0LLvTblHtmRCHWswcACbH2DC6H+0awsaZNI80GxSSutKN+2vH3N0C
+6fLwP1VaYm0qPA3pI1vp9Xu5I//6Hzt3aT/R+6KhS+CH3qbJiHNzg1ywdNqgD3Pp
+bJNEiDKWdLd65bvclVVyWHovWIvvEly1TWUsp4YbdrT5asL/VV82fvo4wCttOq96
+msk9OL5vQADJ32D7vnnYadiT2tpJMjNSug7JKHutJ/cJmm8qkqNwFm/BH+w2t7kh
+elbG0f2P7FpPXVMCJPHhmI/+OBECqBHAAWgRRn91GqeGgY+Xy05orL2R/qeNdBbT
+qmrxqm7LesPgo0IqU82quTkiBERaXMu8qUYaYPGVJIskQuWqd/aArhaFmqZhK3bz
+lepJMDAaeaBT/3ULICHdhsSVAhl6iwid40ow8x1cRFoPEljZ9t6fERN1h2ptNyZT
+LRJ4DUXy4RWTHl3+AVyzwcgMpxkxRTzdjguzAjceOHEMm7UNCvSVhFHVBkG6kXXw
+940iYvhkRnn6HRcPbP5xeJp6GX8RoAl42giRO6OGnkhkepOuAcFKJCC2N5OgdKmq
+mlVGEKndriwyzyiPN8noEJksenL1iMSA7HSnM34uCJoEetTBRuv8+721bTdz8dy0
+s/GghsYztugLjho4yivn2fcxw4gMFiSiliyrfbocJnHAubDMZwEu0fpfY+0fBi9+
+e/Odgw8aTlMUKSo+UA==
+-----END ENCRYPTED PRIVATE KEY-----
[27/29] tinkerpop git commit: updated docs
Posted by rd...@apache.org.
updated docs
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/c05e0729
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/c05e0729
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/c05e0729
Branch: refs/heads/TINKERPOP-1602
Commit: c05e072940a1f4bc55521a7240c0b67640ab2651
Parents: 9b279c6
Author: Robert Dale <ro...@gmail.com>
Authored: Wed Jan 18 09:08:19 2017 -0500
Committer: Robert Dale <ro...@gmail.com>
Committed: Fri Jan 27 16:26:33 2017 -0500
----------------------------------------------------------------------
CHANGELOG.asciidoc | 1 +
docs/src/reference/gremlin-applications.asciidoc | 2 +-
docs/src/upgrade/release-3.2.x-incubating.asciidoc | 9 +++++++++
3 files changed, 11 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/c05e0729/CHANGELOG.asciidoc
----------------------------------------------------------------------
diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc
index d523ae2..76d9ef5 100644
--- a/CHANGELOG.asciidoc
+++ b/CHANGELOG.asciidoc
@@ -103,6 +103,7 @@ TinkerPop 3.2.4 (Release Date: NOT OFFICIALLY RELEASED YET)
* Fixed a severe bug where `GraphComputer` strategies are not being loaded until the second use of the traversal source.
* The root traversal now throws regular `NoSuchElementException` instead of `FastNoSuchElementException`. (*breaking*)
* Added a short sleep to prevent traversal from finishing before it can be interrupted during `TraversalInterruptionComputerTest`.
+* Added support for SSL client authentication
[[release-3-2-3]]
TinkerPop 3.2.3 (Release Date: October 17, 2016)
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/c05e0729/docs/src/reference/gremlin-applications.asciidoc
----------------------------------------------------------------------
diff --git a/docs/src/reference/gremlin-applications.asciidoc b/docs/src/reference/gremlin-applications.asciidoc
index 2454581..5cea36b 100644
--- a/docs/src/reference/gremlin-applications.asciidoc
+++ b/docs/src/reference/gremlin-applications.asciidoc
@@ -1036,7 +1036,7 @@ The following table describes the various configuration options that Gremlin Ser
|ssl.keyCertChainFile |The X.509 certificate chain file in PEM format. If this value is not present and `ssl.enabled` is `true` a self-signed certificate will be used (not suitable for production). |_none_
|ssl.keyFile |The `PKCS#8` private key file in PEM format. If this value is not present and `ssl.enabled` is `true` a self-signed certificate will be used (not suitable for production). |_none_
|ssl.keyPassword |The password of the `keyFile` if it is password-protected |_none_
-|ssl.needClientAuth | Optional. One of NONE, OPTIONAL, REQUIRE. Enables client certificate authentication at the level specified. Can be used in combination with Authenticator. |_none_
+|ssl.needClientAuth | Optional. One of NONE, OPTIONAL, REQUIRE. Enables client certificate authentication at the enforcement level specified. Can be used in combination with Authenticator. |_none_
|ssl.trustCertChainFile | Required when needClientAuth is OPTIONAL or REQUIRE. Trusted certificates for verifying the remote endpoint's certificate. The file should contain an X.509 certificate chain in PEM format. |_none_
|strictTransactionManagement |Set to `true` to require `aliases` to be submitted on every requests, where the `aliases` become the scope of transaction management. |false
|threadPoolBoss |The number of threads available to Gremlin Server for accepting connections. Should always be set to `1`. |1
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/c05e0729/docs/src/upgrade/release-3.2.x-incubating.asciidoc
----------------------------------------------------------------------
diff --git a/docs/src/upgrade/release-3.2.x-incubating.asciidoc b/docs/src/upgrade/release-3.2.x-incubating.asciidoc
index 7b5d638..401d67a 100644
--- a/docs/src/upgrade/release-3.2.x-incubating.asciidoc
+++ b/docs/src/upgrade/release-3.2.x-incubating.asciidoc
@@ -141,6 +141,15 @@ removed.
See: link:https://issues.apache.org/jira/browse/TINKERPOP-1562[TINKERPOP-1562]
+
+SSL Client Authentication
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Added new server configuration option `ssl.needClientAuth`.
+
+See: link:https://issues.apache.org/jira/browse/TINKERPOP-1602[TINKERPOP-1602]
+
+
Upgrading for Providers
~~~~~~~~~~~~~~~~~~~~~~~
[17/29] tinkerpop git commit: Finalized variables and arguments.
Posted by rd...@apache.org.
Finalized variables and arguments.
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/4dfba770
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/4dfba770
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/4dfba770
Branch: refs/heads/TINKERPOP-1602
Commit: 4dfba770877d564637417df668fdb9cddebce36b
Parents: c9999bd
Author: Ted Wilmes <tw...@gmail.com>
Authored: Fri Jan 27 14:12:51 2017 -0600
Committer: Ted Wilmes <tw...@gmail.com>
Committed: Fri Jan 27 14:12:51 2017 -0600
----------------------------------------------------------------------
.../strategy/optimization/PathRetractionStrategy.java | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/4dfba770/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategy.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategy.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategy.java
index 439d568..91f07f9 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategy.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/PathRetractionStrategy.java
@@ -132,7 +132,7 @@ public final class PathRetractionStrategy extends AbstractTraversalStrategy<Trav
Step<?, ?> parent = traversal.getParent().asStep();
final List<Pair<Step, Set<String>>> parentKeeperPairs = new ArrayList<>();
while (!parent.equals(EmptyStep.instance())) {
- Set<String> parentKeepLabels = new HashSet<>(PathUtil.getReferencedLabels(parent));
+ final Set<String> parentKeepLabels = new HashSet<>(PathUtil.getReferencedLabels(parent));
parentKeepLabels.addAll(PathUtil.getReferencedLabelsAfterStep(parent));
parentKeeperPairs.add(new Pair<>(parent, parentKeepLabels));
parent = parent.getTraversal().getParent().asStep();
@@ -214,21 +214,21 @@ public final class PathRetractionStrategy extends AbstractTraversalStrategy<Trav
}
}
- private void applyToChildren(Set<String> keepLabels, List<Traversal.Admin<Object, Object>> children) {
- for (Traversal.Admin<Object, Object> child : children) {
+ private void applyToChildren(final Set<String> keepLabels, final List<Traversal.Admin<Object, Object>> children) {
+ for (final Traversal.Admin<Object, Object> child : children) {
TraversalHelper.applyTraversalRecursively(trav -> addLabels(trav, keepLabels), child);
}
}
private void addLabels(final Traversal.Admin traversal, final Set<String> keepLabels) {
- for (Object s : traversal.getSteps()) {
+ for (final Object s : traversal.getSteps()) {
if (s instanceof PathProcessor) {
addLabels((PathProcessor) s, keepLabels);
}
}
}
- private void addLabels(PathProcessor s, Set<String> keepLabels) {
+ private void addLabels(final PathProcessor s, final Set<String> keepLabels) {
if (s.getKeepLabels() == null) {
s.setKeepLabels(new HashSet<>(keepLabels));
} else {
[08/29] tinkerpop git commit: Sessions were not constucting
GremlinExecutor with new plugin system CTR
Posted by rd...@apache.org.
Sessions were not constucting GremlinExecutor with new plugin system CTR
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/d08a68f3
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/d08a68f3
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/d08a68f3
Branch: refs/heads/TINKERPOP-1602
Commit: d08a68f32c4ad497b6dbbdad5ae916cb659ff019
Parents: 470c656
Author: Stephen Mallette <sp...@genoprime.com>
Authored: Tue Jan 24 15:11:50 2017 -0500
Committer: Stephen Mallette <sp...@genoprime.com>
Committed: Tue Jan 24 15:11:50 2017 -0500
----------------------------------------------------------------------
.../gremlin/server/op/session/Session.java | 17 ++++++++++++-----
1 file changed, 12 insertions(+), 5 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/d08a68f3/gremlin-server/src/main/java/org/apache/tinkerpop/gremlin/server/op/session/Session.java
----------------------------------------------------------------------
diff --git a/gremlin-server/src/main/java/org/apache/tinkerpop/gremlin/server/op/session/Session.java b/gremlin-server/src/main/java/org/apache/tinkerpop/gremlin/server/op/session/Session.java
index 8f4da66..9b16a3b 100644
--- a/gremlin-server/src/main/java/org/apache/tinkerpop/gremlin/server/op/session/Session.java
+++ b/gremlin-server/src/main/java/org/apache/tinkerpop/gremlin/server/op/session/Session.java
@@ -228,11 +228,18 @@ public class Session {
.scheduledExecutorService(scheduledExecutorService);
settings.scriptEngines.forEach((k, v) -> {
- // make sure that server related classes are available at init - not really necessary here because
- // lifecycle hooks are not executed per session, but there should be some consistency .... i guess
- v.imports.add(LifeCycleHook.class.getCanonicalName());
- v.imports.add(LifeCycleHook.Context.class.getCanonicalName());
- gremlinExecutorBuilder.addEngineSettings(k, v.imports, v.staticImports, v.scripts, v.config);
+ // use plugins if they are present and the old approach if not
+ if (v.plugins.isEmpty()) {
+ // make sure that server related classes are available at init - ultimately this body of code will be
+ // deleted when deprecation is removed
+ v.imports.add(LifeCycleHook.class.getCanonicalName());
+ v.imports.add(LifeCycleHook.Context.class.getCanonicalName());
+ gremlinExecutorBuilder.addEngineSettings(k, v.imports, v.staticImports, v.scripts, v.config);
+ } else {
+ // make sure that server related classes are available at init - new approach. the LifeCycleHook stuff
+ // will be added explicitly via configuration using GremlinServerGremlinModule in the yaml
+ gremlinExecutorBuilder.addPlugins(k, v.plugins);
+ }
});
return gremlinExecutorBuilder;
[13/29] tinkerpop git commit: Added resource file for
GiraphGremlinPlugin CTR
Posted by rd...@apache.org.
Added resource file for GiraphGremlinPlugin CTR
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/91759a52
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/91759a52
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/91759a52
Branch: refs/heads/TINKERPOP-1602
Commit: 91759a5236a3f2121bfe24098d9a2f35478dfccf
Parents: 47e5ae2
Author: Stephen Mallette <sp...@genoprime.com>
Authored: Thu Jan 26 10:37:29 2017 -0500
Committer: Stephen Mallette <sp...@genoprime.com>
Committed: Thu Jan 26 10:37:29 2017 -0500
----------------------------------------------------------------------
.../services/org.apache.tinkerpop.gremlin.jsr223.GremlinPlugin | 1 +
1 file changed, 1 insertion(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/91759a52/giraph-gremlin/src/main/resources/META-INF/services/org.apache.tinkerpop.gremlin.jsr223.GremlinPlugin
----------------------------------------------------------------------
diff --git a/giraph-gremlin/src/main/resources/META-INF/services/org.apache.tinkerpop.gremlin.jsr223.GremlinPlugin b/giraph-gremlin/src/main/resources/META-INF/services/org.apache.tinkerpop.gremlin.jsr223.GremlinPlugin
new file mode 100644
index 0000000..9a92905
--- /dev/null
+++ b/giraph-gremlin/src/main/resources/META-INF/services/org.apache.tinkerpop.gremlin.jsr223.GremlinPlugin
@@ -0,0 +1 @@
+org.apache.tinkerpop.gremlin.giraph.jsr223.GiraphGremlinPlugin
\ No newline at end of file