You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tinkerpop.apache.org by sp...@apache.org on 2018/06/15 12:26:39 UTC
[01/12] tinkerpop git commit: TINKERPOP-1831 Refactored EventStrategy
[Forced Update!]
Repository: tinkerpop
Updated Branches:
refs/heads/TINKERPOP-1967 e2ab5d7d6 -> 2469d56f0 (forced update)
TINKERPOP-1831 Refactored EventStrategy
Removed deprecated method. Used VertexProperty.empty() and Property.empty() to represent the "old" value for new property events. Created a enum for configuring detachment on the EventStrategy builder.
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/7963fdfb
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/7963fdfb
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/7963fdfb
Branch: refs/heads/TINKERPOP-1967
Commit: 7963fdfb4407347a37ff35d40f5efdd1ba5f039f
Parents: ae2f304
Author: Stephen Mallette <sp...@genoprime.com>
Authored: Thu Jun 7 09:10:47 2018 -0400
Committer: Stephen Mallette <sp...@genoprime.com>
Committed: Thu Jun 7 09:10:47 2018 -0400
----------------------------------------------------------------------
CHANGELOG.asciidoc | 3 +
docs/src/upgrade/release-3.4.x.asciidoc | 22 +++
.../bulkloading/BulkLoaderVertexProgram.java | 6 -
.../step/sideEffect/AddPropertyStep.java | 6 +-
.../util/event/ConsoleMutationListener.java | 5 -
.../step/util/event/MutationListener.java | 11 +-
.../strategy/decoration/EventStrategy.java | 108 +++++-----
.../strategy/decoration/EventStrategyTest.java | 1 -
.../decoration/EventStrategyProcessTest.java | 197 ++++---------------
9 files changed, 130 insertions(+), 229 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/7963fdfb/CHANGELOG.asciidoc
----------------------------------------------------------------------
diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc
index e886107..7317aeb 100644
--- a/CHANGELOG.asciidoc
+++ b/CHANGELOG.asciidoc
@@ -37,6 +37,9 @@ This release also includes changes from <<release-3-3-3, 3.3.3>>.
* Changed the order of `select()` scopes. The order is now: maps, side-effects, paths.
* Removed previously deprecated Credentials DSL infrastructure.
* Moved `TraversalEngine` to `gremlin-test` as it has long been only used in testing infrastructure.
+* Events from `EventStrategy`raised from "new" mutations will now return a true "empty" property from `VertexProperty.empty()` or `Property.empty()` as is appropriate.
+* `MutationListener#vertexPropertyChanged(Vertex, VertexProperty, Object, Object...)` no longer has a default implementation.
+* Removed previously deprecated `MutationListener#vertexPropertyChanged(Vertex, Property, Object, Object...)`.
* Removed previously deprecated `OpSelectorHandler` constructor.
* Removed previously deprecated `close()` from `GremlinGroovyScriptEngine` which no longer implements `AutoCloseable`.
* Removed previously deprecated `getGraphInputFormat()` and `getGraphOutputFormat()` from `HadoopConfiguration`.
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/7963fdfb/docs/src/upgrade/release-3.4.x.asciidoc
----------------------------------------------------------------------
diff --git a/docs/src/upgrade/release-3.4.x.asciidoc b/docs/src/upgrade/release-3.4.x.asciidoc
index 851d458..68e0c51 100644
--- a/docs/src/upgrade/release-3.4.x.asciidoc
+++ b/docs/src/upgrade/release-3.4.x.asciidoc
@@ -113,6 +113,27 @@ gremlin> g.V().values('name').max()
==>vadas
----
+==== EventStrategy API
+
+There were some minor modifications to how `EventStrategy` is constructed and what can be expected from events raised
+from the addition of new properties.
+
+With respect to the change in terms of `EventStrategy` construction, the `detach()` builder method formerly took a
+`Class` as an argument and that `Class` was meant to be one of the various "detachment factories" or `null`. That
+approach was a bit confusing, so that signature has changed to `detach(EventStrategy.Detachment)` where the argument
+is a more handy enum of detachment options.
+
+As for the changes related to events themselves, it is first worth noting that the previously deprecated
+`vertexPropertyChanged(Vertex, Property, Object, Object...)` on `MutationListener` has been removed for what should
+have originally been the correct signature of `vertexPropertyChanged(Vertex, VertexProperty, Object, Object...)`. In
+prior versions when this method and its related `edgePropertyChanged()` and `vertexPropertyPropertyChanged()` were
+triggered by way of the addition of a new property a "fake" property was included with a `null` value for the
+"oldValue" argument to these methods (as it did not exist prior to this event). That was a bit awkward to reason about
+when dealing with that event. To make this easier, the event now raises with a `VertexProperty.empty()` or
+`Property.empty()` instance, which can be evaluated with `equals()` easily to determine if the property is new or not.
+
+link:https://issues.apache.org/jira/browse/TINKERPOP-1831[TINKERPOP-1831]
+
==== Deprecation Removal
The following deprecated classes, methods or fields have been removed in this version:
@@ -122,6 +143,7 @@ The following deprecated classes, methods or fields have been removed in this ve
** `org.apache.tinkerpop.gremlin.process.traversal.engine.*`
** `org.apache.tinkerpop.gremlin.process.traversal.step.map.LambdaCollectingBarrierStep.Consumers`
** `org.apache.tinkerpop.gremlin.process.traversal.step.util.HasContainer#makeHasContainers(String, P)`
+** `org.apache.tinkerpop.gremlin.process.traversal.step.util.event.MutationListener#vertexPropertyChanged(Vertex, Property, Object, Object...)`
** `org.apache.tinkerpop.gremlin.structure.Element.Exceptions#elementAlreadyRemoved(Class, Object)`
** `org.apache.tinkerpop.gremlin.structure.Graph.Exceptions#elementNotFound(Class, Object)`
** `org.apache.tinkerpop.gremlin.structure.Graph.Exceptions#elementNotFound(Class, Object, Exception)`
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/7963fdfb/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/bulkloading/BulkLoaderVertexProgram.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/bulkloading/BulkLoaderVertexProgram.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/bulkloading/BulkLoaderVertexProgram.java
index 508af3e..c637880 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/bulkloading/BulkLoaderVertexProgram.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/bulkloading/BulkLoaderVertexProgram.java
@@ -460,12 +460,6 @@ public class BulkLoaderVertexProgram implements VertexProgram<Tuple> {
}
@Override
- public void vertexPropertyChanged(final Vertex element, final Property oldValue, final Object setValue,
- final Object... vertexPropertyKeyValues) {
- // do nothing - deprecated
- }
-
- @Override
public void vertexPropertyRemoved(final VertexProperty vertexProperty) {
this.counter++;
}
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/7963fdfb/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/sideEffect/AddPropertyStep.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/sideEffect/AddPropertyStep.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/sideEffect/AddPropertyStep.java
index 04a8414..3589c0c 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/sideEffect/AddPropertyStep.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/sideEffect/AddPropertyStep.java
@@ -97,17 +97,17 @@ public final class AddPropertyStep<S extends Element> extends SideEffectStep<S>
if (element instanceof Vertex)
evt = new Event.VertexPropertyChangedEvent(eventStrategy.detach((Vertex) element),
newProperty ?
- eventStrategy.empty(element, key) :
+ VertexProperty.empty() :
eventStrategy.detach((VertexProperty) currentProperty), value, vertexPropertyKeyValues);
else if (element instanceof Edge)
evt = new Event.EdgePropertyChangedEvent(eventStrategy.detach((Edge) element),
newProperty ?
- eventStrategy.empty(element, key) :
+ Property.empty() :
eventStrategy.detach(currentProperty), value);
else if (element instanceof VertexProperty)
evt = new Event.VertexPropertyPropertyChangedEvent(eventStrategy.detach((VertexProperty) element),
newProperty ?
- eventStrategy.empty(element, key) :
+ Property.empty() :
eventStrategy.detach(currentProperty), value);
else
throw new IllegalStateException(String.format("The incoming object cannot be processed by change eventing in %s: %s", AddPropertyStep.class.getName(), element));
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/7963fdfb/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/util/event/ConsoleMutationListener.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/util/event/ConsoleMutationListener.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/util/event/ConsoleMutationListener.java
index 937f414..1bfdf24 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/util/event/ConsoleMutationListener.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/util/event/ConsoleMutationListener.java
@@ -83,11 +83,6 @@ public class ConsoleMutationListener implements MutationListener {
}
@Override
- public void vertexPropertyChanged(final Vertex element, final Property oldValue, final Object setValue, final Object... vertexPropertyKeyValues) {
- // do nothing - deprecated
- }
-
- @Override
public void vertexPropertyChanged(final Vertex element, final VertexProperty oldValue, final Object setValue, final Object... vertexPropertyKeyValues) {
System.out.println("Vertex [" + element.toString() + "] property [" + oldValue + "] change to [" + setValue + "] in graph [" + graph.toString() + "]");
}
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/7963fdfb/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/util/event/MutationListener.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/util/event/MutationListener.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/util/event/MutationListener.java
index 00e49ed..8525d5d 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/util/event/MutationListener.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/util/event/MutationListener.java
@@ -52,19 +52,10 @@ public interface MutationListener {
/**
* Raised after the property of a {@link Vertex} changed.
*
- * @deprecated As of release 3.2.7, replaced by {@link #vertexPropertyChanged(Vertex, VertexProperty, Object, Object...)}.
- */
- public void vertexPropertyChanged(final Vertex element, final Property oldValue, final Object setValue, final Object... vertexPropertyKeyValues);
-
- /**
- * Raised after the property of a {@link Vertex} changed.
- *
* @param element the {@link Vertex} that changed
* @param setValue the new value of the property
*/
- public default void vertexPropertyChanged(final Vertex element, final VertexProperty oldValue, final Object setValue, final Object... vertexPropertyKeyValues) {
- vertexPropertyChanged(element, (Property) oldValue, setValue, vertexPropertyKeyValues);
- }
+ public void vertexPropertyChanged(final Vertex element, final VertexProperty oldValue, final Object setValue, final Object... vertexPropertyKeyValues);
/**
* Raised after a {@link VertexProperty} was removed from the graph.
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/7963fdfb/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/decoration/EventStrategy.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/decoration/EventStrategy.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/decoration/EventStrategy.java
index b4824c8..094b43d 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/decoration/EventStrategy.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/decoration/EventStrategy.java
@@ -57,53 +57,23 @@ import java.util.List;
*/
public final class EventStrategy extends AbstractTraversalStrategy<TraversalStrategy.DecorationStrategy> implements TraversalStrategy.DecorationStrategy {
private final EventQueue eventQueue;
- private final Class<?> detachmentFactory;
+ private final Detachment detachment;
private EventStrategy(final Builder builder) {
this.eventQueue = builder.eventQueue;
this.eventQueue.setListeners(builder.listeners);
- this.detachmentFactory = builder.detachmentFactory;
+ this.detachment = builder.detachment;
}
- public Class<?> getDetachmentFactory() {
- return this.detachmentFactory;
+ public Detachment getDetachment() {
+ return this.detachment;
}
/**
* Applies the appropriate detach operation to elements that will be raised in mutation events.
*/
public <R> R detach(final R attached) {
- if (null == detachmentFactory)
- return attached;
- else if (detachmentFactory.equals(DetachedFactory.class))
- return DetachedFactory.detach(attached, true);
- else if (detachmentFactory.equals(ReferenceFactory.class))
- return ReferenceFactory.detach(attached);
- else
- throw new IllegalStateException("Unknown detachment option using " + detachmentFactory.getSimpleName());
- }
-
- /**
- * For newly created properties that do not yet exist, an empty {@link Property} is required that just contains
- * a key as a reference.
- */
- public <R extends Property> R empty(final Element element, final String key) {
- // currently the "no detachment" model simply returns a Detached value to maintain consistency with the
- // original API that already existed (where returning "Detached" was the only option). This could probably
- // change in the future to use an "empty" property or perhaps the "change" event API could change all together
- // and have a different return.
- if (null == detachmentFactory || detachmentFactory.equals(DetachedFactory.class)) {
- if (element instanceof Vertex)
- return (R) new DetachedVertexProperty(null, key, null, null);
- else
- return (R) new DetachedProperty(key, null);
- } else if (detachmentFactory.equals(ReferenceFactory.class)) {
- if (element instanceof Vertex)
- return (R) new ReferenceVertexProperty(new DetachedVertexProperty(null, key, null, null));
- else
- return (R) new ReferenceProperty(new DetachedProperty(key, null));
- } else
- throw new IllegalStateException("Unknown empty detachment option using " + detachmentFactory.getSimpleName());
+ return (R) detachment.detach(attached);
}
@Override
@@ -132,7 +102,7 @@ public final class EventStrategy extends AbstractTraversalStrategy<TraversalStra
public final static class Builder {
private final List<MutationListener> listeners = new ArrayList<>();
private EventQueue eventQueue = new DefaultEventQueue();
- private Class<?> detachmentFactory = DetachedFactory.class;
+ private Detachment detachment = Detachment.DETACHED_WITH_PROPERTIES;
Builder() {}
@@ -147,15 +117,11 @@ public final class EventStrategy extends AbstractTraversalStrategy<TraversalStra
}
/**
- * Configures the method of detachment for element provided in mutation callback events. If configured with
- * {@code null} for no detachment with a transactional graph, be aware that accessing the evented elements
- * after {@code commit()} will likely open new transactions.
- *
- * @param factoryClass must be either {@code null} (for no detachment), {@link ReferenceFactory} for elements
- * with no properties or {@link DetachedFactory} for elements with properties.
+ * Configures the method of detachment for element provided in mutation callback events. The default is
+ * {@link Detachment#DETACHED_WITH_PROPERTIES}.
*/
- public Builder detach(final Class<?> factoryClass) {
- detachmentFactory = factoryClass;
+ public Builder detach(final Detachment detachment) {
+ this.detachment = detachment;
return this;
}
@@ -165,6 +131,60 @@ public final class EventStrategy extends AbstractTraversalStrategy<TraversalStra
}
/**
+ * A common interface for detachment.
+ */
+ public interface Detacher {
+ public Object detach(final Object object);
+ }
+
+ /**
+ * Options for detaching elements from the graph during eventing.
+ */
+ public enum Detachment implements Detacher {
+ /**
+ * Does not detach the element from the graph. It should be noted that if this option is used with
+ * transactional graphs new transactions may be opened if these elements are accessed after a {@code commit()}
+ * is called.
+ */
+ NONE {
+ @Override
+ public Object detach(final Object object) {
+ return object;
+ }
+ },
+
+ /**
+ * Uses {@link DetachedFactory} to detach and includes properties of elements that have them.
+ */
+ DETACHED_WITH_PROPERTIES {
+ @Override
+ public Object detach(final Object object) {
+ return DetachedFactory.detach(object, true);
+ }
+ },
+
+ /**
+ * Uses {@link DetachedFactory} to detach and does not include properties of elements that have them.
+ */
+ DETACHED_NO_PROPERTIES {
+ @Override
+ public Object detach(final Object object) {
+ return DetachedFactory.detach(object, false);
+ }
+ },
+
+ /**
+ * Uses {@link ReferenceFactory} to detach which only includes id and label of elements.
+ */
+ REFERENCE {
+ @Override
+ public Object detach(final Object object) {
+ return ReferenceFactory.detach(object);
+ }
+ }
+ }
+
+ /**
* Gathers messages from callbacks and fires them to listeners. When the event is sent to the listener is
* up to the implementation of this interface.
*/
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/7963fdfb/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/decoration/EventStrategyTest.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/decoration/EventStrategyTest.java b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/decoration/EventStrategyTest.java
index 27d9b7e..911ef41 100644
--- a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/decoration/EventStrategyTest.java
+++ b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/decoration/EventStrategyTest.java
@@ -23,7 +23,6 @@ import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.DefaultGraphTrav
import org.apache.tinkerpop.gremlin.process.traversal.step.Mutating;
import org.apache.tinkerpop.gremlin.process.traversal.step.util.event.ConsoleMutationListener;
import org.apache.tinkerpop.gremlin.process.traversal.step.util.event.MutationListener;
-import org.apache.tinkerpop.gremlin.structure.Direction;
import org.apache.tinkerpop.gremlin.structure.util.empty.EmptyGraph;
import org.junit.Test;
import org.junit.runner.RunWith;
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/7963fdfb/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/decoration/EventStrategyProcessTest.java
----------------------------------------------------------------------
diff --git a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/decoration/EventStrategyProcessTest.java b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/decoration/EventStrategyProcessTest.java
index e7a392c..b527340 100644
--- a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/decoration/EventStrategyProcessTest.java
+++ b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/decoration/EventStrategyProcessTest.java
@@ -32,7 +32,6 @@ import org.apache.tinkerpop.gremlin.structure.util.detached.DetachedEdge;
import org.apache.tinkerpop.gremlin.structure.util.detached.DetachedVertex;
import org.apache.tinkerpop.gremlin.structure.util.detached.DetachedVertexProperty;
import org.apache.tinkerpop.gremlin.structure.util.reference.ReferenceEdge;
-import org.apache.tinkerpop.gremlin.structure.util.reference.ReferenceFactory;
import org.apache.tinkerpop.gremlin.structure.util.reference.ReferenceVertex;
import org.apache.tinkerpop.gremlin.structure.util.reference.ReferenceVertexProperty;
import org.apache.tinkerpop.gremlin.util.iterator.IteratorUtils;
@@ -634,8 +633,7 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
assertThat(element, instanceOf(DetachedVertexProperty.class));
assertEquals(label, element.label());
assertEquals(value, element.value());
- assertEquals(null, oldValue.value());
- assertEquals("new", oldValue.key());
+ assertEquals(Property.empty(), oldValue);
assertEquals("yay!", setValue);
triggered.set(true);
}
@@ -752,8 +750,7 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
assertEquals(label, element.label());
assertEquals(inId, element.inVertex().id());
assertEquals(outId, element.outVertex().id());
- assertEquals(null, oldValue.value());
- assertEquals("new", oldValue.key());
+ assertEquals(Property.empty(), oldValue);
assertEquals("yay!", setValue);
triggered.set(true);
}
@@ -928,44 +925,7 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
assertThat(element, instanceOf(DetachedVertex.class));
assertEquals(label, element.label());
assertEquals(id, element.id());
- assertEquals("new", oldValue.key());
- assertEquals(null, oldValue.value());
- assertEquals("dah", setValue);
- triggered.set(true);
- }
- };
- final EventStrategy.Builder builder = EventStrategy.build().addListener(listener);
-
- if (graph.features().graph().supportsTransactions())
- builder.eventQueue(new EventStrategy.TransactionalEventQueue(graph));
-
- final EventStrategy eventStrategy = builder.create();
- final GraphTraversalSource gts = create(eventStrategy);
-
- gts.V(v).property(VertexProperty.Cardinality.single, "new", "dah").iterate();
- tryCommit(graph);
-
- assertEquals(2, IteratorUtils.count(g.V(v).properties()));
- assertThat(triggered.get(), is(true));
- }
-
- @Test
- @FeatureRequirementSet(FeatureRequirementSet.Package.VERTICES_ONLY)
- public void shouldDetachVertexPropertyWhenNewDeprecated() {
- final AtomicBoolean triggered = new AtomicBoolean(false);
- final Vertex v = graph.addVertex();
- final String label = v.label();
- final Object id = v.id();
- v.property("old","blah");
-
- final MutationListener listener = new AbstractMutationListener() {
- @Override
- public void vertexPropertyChanged(final Vertex element, final Property oldValue, final Object setValue, final Object... vertexPropertyKeyValues) {
- assertThat(element, instanceOf(DetachedVertex.class));
- assertEquals(label, element.label());
- assertEquals(id, element.id());
- assertEquals("new", oldValue.key());
- assertEquals(null, oldValue.value());
+ assertEquals(VertexProperty.empty(), oldValue);
assertEquals("dah", setValue);
triggered.set(true);
}
@@ -1069,7 +1029,7 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
triggered.set(true);
}
};
- final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(ReferenceFactory.class);
+ final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(EventStrategy.Detachment.REFERENCE);
if (graph.features().graph().supportsTransactions())
builder.eventQueue(new EventStrategy.TransactionalEventQueue(graph));
@@ -1109,7 +1069,7 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
triggered.set(true);
}
};
- final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(ReferenceFactory.class);
+ final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(EventStrategy.Detachment.REFERENCE);
if (graph.features().graph().supportsTransactions())
builder.eventQueue(new EventStrategy.TransactionalEventQueue(graph));
@@ -1142,13 +1102,12 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
assertThat(element, instanceOf(ReferenceVertexProperty.class));
assertEquals(label, element.label());
assertEquals(value, element.value());
- assertEquals(null, oldValue.value());
- assertEquals("new", oldValue.key());
+ assertEquals(Property.empty(), oldValue);
assertEquals("yay!", setValue);
triggered.set(true);
}
};
- final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(ReferenceFactory.class);
+ final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(EventStrategy.Detachment.REFERENCE);
if (graph.features().graph().supportsTransactions())
builder.eventQueue(new EventStrategy.TransactionalEventQueue(graph));
@@ -1187,7 +1146,7 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
triggered.set(true);
}
};
- final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(ReferenceFactory.class);
+ final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(EventStrategy.Detachment.REFERENCE);
if (graph.features().graph().supportsTransactions())
builder.eventQueue(new EventStrategy.TransactionalEventQueue(graph));
@@ -1227,7 +1186,7 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
triggered.set(true);
}
};
- final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(ReferenceFactory.class);
+ final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(EventStrategy.Detachment.REFERENCE);
if (graph.features().graph().supportsTransactions())
builder.eventQueue(new EventStrategy.TransactionalEventQueue(graph));
@@ -1260,13 +1219,12 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
assertEquals(label, element.label());
assertEquals(inId, element.inVertex().id());
assertEquals(outId, element.outVertex().id());
- assertEquals(null, oldValue.value());
- assertEquals("new", oldValue.key());
+ assertEquals(Property.empty(), oldValue);
assertEquals("yay!", setValue);
triggered.set(true);
}
};
- final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(ReferenceFactory.class);
+ final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(EventStrategy.Detachment.REFERENCE);
if (graph.features().graph().supportsTransactions())
builder.eventQueue(new EventStrategy.TransactionalEventQueue(graph));
@@ -1302,7 +1260,7 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
triggered.set(true);
}
};
- final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(ReferenceFactory.class);
+ final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(EventStrategy.Detachment.REFERENCE);
if (graph.features().graph().supportsTransactions())
builder.eventQueue(new EventStrategy.TransactionalEventQueue(graph));
@@ -1335,7 +1293,7 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
triggered.set(true);
}
};
- final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(ReferenceFactory.class);
+ final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(EventStrategy.Detachment.REFERENCE);
if (graph.features().graph().supportsTransactions())
builder.eventQueue(new EventStrategy.TransactionalEventQueue(graph));
@@ -1369,7 +1327,7 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
triggered.set(true);
}
};
- final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(ReferenceFactory.class);
+ final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(EventStrategy.Detachment.REFERENCE);
if (graph.features().graph().supportsTransactions())
builder.eventQueue(new EventStrategy.TransactionalEventQueue(graph));
@@ -1408,7 +1366,7 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
triggered.set(true);
}
};
- final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(ReferenceFactory.class);
+ final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(EventStrategy.Detachment.REFERENCE);
if (graph.features().graph().supportsTransactions())
builder.eventQueue(new EventStrategy.TransactionalEventQueue(graph));
@@ -1425,7 +1383,7 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
@Test
@FeatureRequirementSet(FeatureRequirementSet.Package.VERTICES_ONLY)
- public void shouldRefereceVertexPropertyWhenNew() {
+ public void shouldReferenceVertexPropertyWhenNew() {
final AtomicBoolean triggered = new AtomicBoolean(false);
final Vertex v = graph.addVertex();
final String label = v.label();
@@ -1438,49 +1396,12 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
assertThat(element, instanceOf(ReferenceVertex.class));
assertEquals(label, element.label());
assertEquals(id, element.id());
- assertEquals("new", oldValue.key());
- assertEquals(null, oldValue.value());
- assertEquals("dah", setValue);
- triggered.set(true);
- }
- };
- final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(ReferenceFactory.class);
-
- if (graph.features().graph().supportsTransactions())
- builder.eventQueue(new EventStrategy.TransactionalEventQueue(graph));
-
- final EventStrategy eventStrategy = builder.create();
- final GraphTraversalSource gts = create(eventStrategy);
-
- gts.V(v).property(VertexProperty.Cardinality.single, "new", "dah").iterate();
- tryCommit(graph);
-
- assertEquals(2, IteratorUtils.count(g.V(v).properties()));
- assertThat(triggered.get(), is(true));
- }
-
- @Test
- @FeatureRequirementSet(FeatureRequirementSet.Package.VERTICES_ONLY)
- public void shouldReferenceVertexPropertyWhenNewDeprecated() {
- final AtomicBoolean triggered = new AtomicBoolean(false);
- final Vertex v = graph.addVertex();
- final String label = v.label();
- final Object id = v.id();
- v.property("old","blah");
-
- final MutationListener listener = new AbstractMutationListener() {
- @Override
- public void vertexPropertyChanged(final Vertex element, final Property oldValue, final Object setValue, final Object... vertexPropertyKeyValues) {
- assertThat(element, instanceOf(ReferenceVertex.class));
- assertEquals(label, element.label());
- assertEquals(id, element.id());
- assertEquals("new", oldValue.key());
- assertEquals(null, oldValue.value());
+ assertEquals(VertexProperty.empty(), oldValue);
assertEquals("dah", setValue);
triggered.set(true);
}
};
- final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(ReferenceFactory.class);
+ final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(EventStrategy.Detachment.REFERENCE);
if (graph.features().graph().supportsTransactions())
builder.eventQueue(new EventStrategy.TransactionalEventQueue(graph));
@@ -1512,7 +1433,7 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
triggered.set(true);
}
};
- final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(ReferenceFactory.class);
+ final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(EventStrategy.Detachment.REFERENCE);
if (graph.features().graph().supportsTransactions())
builder.eventQueue(new EventStrategy.TransactionalEventQueue(graph));
@@ -1541,7 +1462,7 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
triggered.set(true);
}
};
- final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(ReferenceFactory.class);
+ final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(EventStrategy.Detachment.REFERENCE);
if (graph.features().graph().supportsTransactions())
builder.eventQueue(new EventStrategy.TransactionalEventQueue(graph));
@@ -1578,7 +1499,7 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
triggered.set(true);
}
};
- final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(ReferenceFactory.class);
+ final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(EventStrategy.Detachment.REFERENCE);
if (graph.features().graph().supportsTransactions())
builder.eventQueue(new EventStrategy.TransactionalEventQueue(graph));
@@ -1618,7 +1539,7 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
triggered.set(true);
}
};
- final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(ReferenceFactory.class);
+ final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(EventStrategy.Detachment.REFERENCE);
if (graph.features().graph().supportsTransactions())
builder.eventQueue(new EventStrategy.TransactionalEventQueue(graph));
@@ -1651,13 +1572,12 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
assertEquals(vp, element);
assertEquals(label, element.label());
assertEquals(value, element.value());
- assertEquals(null, oldValue.value());
- assertEquals("new", oldValue.key());
+ assertEquals(Property.empty(), oldValue);
assertEquals("yay!", setValue);
triggered.set(true);
}
};
- final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(ReferenceFactory.class);
+ final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(EventStrategy.Detachment.REFERENCE);
if (graph.features().graph().supportsTransactions())
builder.eventQueue(new EventStrategy.TransactionalEventQueue(graph));
@@ -1696,7 +1616,7 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
triggered.set(true);
}
};
- final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(ReferenceFactory.class);
+ final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(EventStrategy.Detachment.REFERENCE);
if (graph.features().graph().supportsTransactions())
builder.eventQueue(new EventStrategy.TransactionalEventQueue(graph));
@@ -1736,7 +1656,7 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
triggered.set(true);
}
};
- final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(ReferenceFactory.class);
+ final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(EventStrategy.Detachment.REFERENCE);
if (graph.features().graph().supportsTransactions())
builder.eventQueue(new EventStrategy.TransactionalEventQueue(graph));
@@ -1769,13 +1689,12 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
assertEquals(label, element.label());
assertEquals(inId, element.inVertex().id());
assertEquals(outId, element.outVertex().id());
- assertEquals(null, oldValue.value());
- assertEquals("new", oldValue.key());
+ assertEquals(Property.empty(), oldValue);
assertEquals("yay!", setValue);
triggered.set(true);
}
};
- final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(ReferenceFactory.class);
+ final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(EventStrategy.Detachment.REFERENCE);
if (graph.features().graph().supportsTransactions())
builder.eventQueue(new EventStrategy.TransactionalEventQueue(graph));
@@ -1811,7 +1730,7 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
triggered.set(true);
}
};
- final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(ReferenceFactory.class);
+ final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(EventStrategy.Detachment.REFERENCE);
if (graph.features().graph().supportsTransactions())
builder.eventQueue(new EventStrategy.TransactionalEventQueue(graph));
@@ -1845,7 +1764,7 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
triggered.set(true);
}
};
- final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(ReferenceFactory.class);
+ final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(EventStrategy.Detachment.REFERENCE);
if (graph.features().graph().supportsTransactions())
builder.eventQueue(new EventStrategy.TransactionalEventQueue(graph));
@@ -1880,7 +1799,7 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
triggered.set(true);
}
};
- final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(ReferenceFactory.class);
+ final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(EventStrategy.Detachment.REFERENCE);
if (graph.features().graph().supportsTransactions())
builder.eventQueue(new EventStrategy.TransactionalEventQueue(graph));
@@ -1917,7 +1836,7 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
triggered.set(true);
}
};
- final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(ReferenceFactory.class);
+ final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(EventStrategy.Detachment.REFERENCE);
if (graph.features().graph().supportsTransactions())
builder.eventQueue(new EventStrategy.TransactionalEventQueue(graph));
@@ -1947,49 +1866,12 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
assertEquals(v, element);
assertEquals(label, element.label());
assertEquals(id, element.id());
- assertEquals("new", oldValue.key());
- assertEquals(null, oldValue.value());
+ assertEquals(VertexProperty.empty(), oldValue);
assertEquals("dah", setValue);
triggered.set(true);
}
};
- final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(ReferenceFactory.class);
-
- if (graph.features().graph().supportsTransactions())
- builder.eventQueue(new EventStrategy.TransactionalEventQueue(graph));
-
- final EventStrategy eventStrategy = builder.create();
- final GraphTraversalSource gts = create(eventStrategy);
-
- gts.V(v).property(VertexProperty.Cardinality.single, "new", "dah").iterate();
- tryCommit(graph);
-
- assertEquals(2, IteratorUtils.count(g.V(v).properties()));
- assertThat(triggered.get(), is(true));
- }
-
- @Test
- @FeatureRequirementSet(FeatureRequirementSet.Package.VERTICES_ONLY)
- public void shouldUseActualVertexPropertyWhenNewDeprecated() {
- final AtomicBoolean triggered = new AtomicBoolean(false);
- final Vertex v = graph.addVertex();
- final String label = v.label();
- final Object id = v.id();
- v.property("old","blah");
-
- final MutationListener listener = new AbstractMutationListener() {
- @Override
- public void vertexPropertyChanged(final Vertex element, final Property oldValue, final Object setValue, final Object... vertexPropertyKeyValues) {
- assertEquals(v, element);
- assertEquals(label, element.label());
- assertEquals(id, element.id());
- assertEquals("new", oldValue.key());
- assertEquals(null, oldValue.value());
- assertEquals("dah", setValue);
- triggered.set(true);
- }
- };
- final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(ReferenceFactory.class);
+ final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(EventStrategy.Detachment.REFERENCE);
if (graph.features().graph().supportsTransactions())
builder.eventQueue(new EventStrategy.TransactionalEventQueue(graph));
@@ -2021,7 +1903,7 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
triggered.set(true);
}
};
- final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(ReferenceFactory.class);
+ final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(EventStrategy.Detachment.REFERENCE);
if (graph.features().graph().supportsTransactions())
builder.eventQueue(new EventStrategy.TransactionalEventQueue(graph));
@@ -2051,7 +1933,7 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
triggered.set(true);
}
};
- final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(ReferenceFactory.class);
+ final EventStrategy.Builder builder = EventStrategy.build().addListener(listener).detach(EventStrategy.Detachment.REFERENCE);
if (graph.features().graph().supportsTransactions())
builder.eventQueue(new EventStrategy.TransactionalEventQueue(graph));
@@ -2083,7 +1965,7 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
}
@Override
- public void vertexPropertyChanged(final Vertex element, final Property oldValue, final Object setValue, final Object... vertexPropertyKeyValues) {
+ public void vertexPropertyChanged(final Vertex element, final VertexProperty oldValue, final Object setValue, final Object... vertexPropertyKeyValues) {
}
@@ -2211,11 +2093,6 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
}
@Override
- public void vertexPropertyChanged(final Vertex element, final Property oldValue, final Object setValue, final Object... vertexPropertyKeyValues) {
- // do nothing - deprecated
- }
-
- @Override
public void vertexPropertyChanged(final Vertex element, final VertexProperty oldValue, final Object setValue, final Object... vertexPropertyKeyValues) {
vertexPropertyChangedEvent.incrementAndGet();
order.add("v-property-changed-" + element.id());
[09/12] tinkerpop git commit: Merged vtslab recipe for connected
components
Posted by sp...@apache.org.
Merged vtslab recipe for connected components
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/d913143e
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/d913143e
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/d913143e
Branch: refs/heads/TINKERPOP-1967
Commit: d913143e69ae76a110eb9378aab6312ac5a55cd7
Parents: 607c37a
Author: HadoopMarc <vt...@xs4all.nl>
Authored: Mon May 21 14:03:54 2018 +0200
Committer: Stephen Mallette <sp...@genoprime.com>
Committed: Fri Jun 15 08:25:01 2018 -0400
----------------------------------------------------------------------
docs/src/recipes/connected-components.asciidoc | 123 +++++++++++++-------
1 file changed, 83 insertions(+), 40 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/d913143e/docs/src/recipes/connected-components.asciidoc
----------------------------------------------------------------------
diff --git a/docs/src/recipes/connected-components.asciidoc b/docs/src/recipes/connected-components.asciidoc
index 70abdbd..edbeec5 100644
--- a/docs/src/recipes/connected-components.asciidoc
+++ b/docs/src/recipes/connected-components.asciidoc
@@ -14,17 +14,31 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
////
+
+// @author Daniel Kuppitz (anwer on gremlin user list)
+// @author Robert Dale (answer on gremlin user list)
+// @author Marc de Lignie
+
[[connected-components]]
== Connected Components
Gremlin can be used to find link:https://en.wikipedia.org/wiki/Connected_component_(graph_theory)[connected components]
-in a graph. As of TinkerPop 3.4.0, the process has been simplified to the `connectedComponent()`-step which is
-described in more detail in the link:
-link:http://tinkerpop.apache.org/docs/x.y.z/reference/#connectedcomponent-step[Reference Documentation].
+in a graph. In a directed graph like in TinkerPop, components can be weakly or strongly connected. This recipe is
+restricted to finding link:https://en.wikipedia.org/wiki/Directed_graph#Directed_graph_connectivity[weakly
+connected components], in which the direction of edges is not taken into account.
+
+Depending on the size of the graph, three solution regimes can be discriminated:
+
+1. Small graphs that fit in the memory of a single machine
+
+2. Medium graphs backed by storage for which a linear scan is still feasible. This regime is left to third party
+TinkerPop implementations, since TinkerPop itself has no storage-backed reference implementations. The idea is that
+component membership is stored in the graph, rather than in memory.
-The `connectedComponent()`-step replaces the original recipe described below from earlier versions of TinkerPop,
-however the algorithm from that old recipe remains interesting for educational purposes and has thus not been removed.
-The original recipe considers the following graph which has three connected components:
+3. Large graphs requiring an OLAP approach to yield results in a reasonable time.
+
+
+These regimes are discussed separately using the following graph with three weakly connected components:
image:connected-components.png[width=600]
@@ -41,46 +55,75 @@ g.addV().property(id, "A").as("a").
addE("link").from("d").to("e").iterate()
----
-One way to detect the various subgraphs would be to do something like this:
+
+===== Small graphs
+
+Connected components in a small graph can be determined with both an OLTP traversal and the OLAP
+`connectedComponent()`-step. The `connectedComponent()`-step is available as of TinkerPop 3.4.0 and is
+described in more detail in the
+link:http://tinkerpop.apache.org/docs/x.y.z/reference/#connectedcomponent-step[Reference Documentation].
+
+A straightforward way to detect the various subgraphs with an OLTP traversal is to do this:
[gremlin-groovy,existing]
----
-g.V().emit(cyclicPath().or().not(both())).repeat(both()).until(cyclicPath()). <1>
- path().aggregate("p"). <2>
- unfold().dedup(). <3>
- map(__.as("v").select("p").unfold(). <4>
- filter(unfold().where(eq("v"))).
- unfold().dedup().order().by(id).fold()).
- dedup() <5>
+g.V().emit(cyclicPath().or().not(both())). <1>
+ repeat(__.where(without('a')).store('a').both()).until(cyclicPath()). <2>
+ group().by(path().unfold().limit(1)). <3>
+ by(path().unfold().dedup().fold()). <4>
+ select(values).unfold() <5>
----
-<1> Iterate all vertices and repeatedly traverse over both incoming and outgoing edges (TinkerPop doesn't support
-unidirectional graphs directly so it must be simulated by ignoring the direction with `both`). Note the use of `emit`
-prior to `repeat` as this allows for return of a single length path.
-<2> Aggregate the `path()` of the emitted vertices to "p". It is within these paths that the list of connected
-components will be identified. Obviously the paths list are duplicative in the sense that they contains different
-paths traveled over the same vertices.
-<3> Unroll the elements in the path list with `unfold` and `dedup`.
-<4> Use the first vertex in each path to filter against the paths stored in "p". When a path is found that has the
-vertex in it, dedup the vertices in the path, order it by the identifier. Each path output from this `map` step
-represents a connected component.
-<5> The connected component list is duplicative given the nature of the paths in "p", but now that the vertices within
-the paths are ordered, a final `dedup` will make the list of connective components unique.
-
-NOTE: This is a nice example of where running smaller pieces of a large Gremlin statement make it easier to see what
-is happening at each step. Consider running this example one line at a time (or perhaps even in a step at a time) to
-see the output at each point.
-
-While the above approach returns results nicely, the traversal doesn't appear to work with OLAP. A less efficient
-approach, but one more suited for OLAP execution looks quite similar but does not use `dedup` as heavily (thus
-`GraphComputer` is forced to analyze far more paths):
+<1> The initial emit() step allows for output of isolated vertices, in addition to the discovery of
+components as described in (2).
+
+<2> The entire component to which the first returned vertex belongs, is visited. To allow for components of any
+structure, a repeat loop is applied that only stops for a particular branch of the component when it detects a cyclic
+path. Collection `'a'` is used to keep track of visited vertices, for both subtraversals within a component
+and new traversals resulting from the `g.V()` linear scan.
+
+<3> While `'a'` nicely keeps track of vertices already visited, the actual components need to be extracted from the
+path information of surviving traversers. The `path().unfold().limit(1)` closure provides the starting vertex
+of surviving traversers, which can be used to group the components.
+
+<4> This clause collects the unique vertices from all paths with the same starting vertex, thus from the same
+weak component.
+
+<5> The values of the groupby map contain the lists of vertices making up the requested components.
+
+This algorithm completes in linear time with the number of vertices and edges, because a traversal is started for each
+vertex and each edge with its associated out-vertex is visited exactly once.
+
+
+==== Large graphs
+
+Large graphs require an OLAP solution with a custom VertexProgram that can be run using a graph implementation's
+GraphComputer, in particular `SparkGraphComputer` on a `HadoopGraph`. The OLAP solution also runs faster for most
+medium-sized graphs, that is when these graph have a 'natural' structure with a limited maximum path length.
+
+The TinkerPop library of vertex programs contains the `WeakComponentsVertexProgram` which can be run in the same
+way as the link:http://tinkerpop.apache.org/docs/x.y.z/reference/#peerpressurevertexprogram[PeerPressureVertexProgram]:
[gremlin-groovy,existing]
----
-g.withComputer().V().emit(cyclicPath().or().not(both())).repeat(both()).until(cyclicPath()).
- aggregate("p").by(path()).cap("p").unfold().limit(local, 1).
- map(__.as("v").select("p").unfold().
- filter(unfold().where(eq("v"))).
- unfold().dedup().order().by(id).fold()
- ).toSet()
+result = g.getGraph().compute().
+ program(WeakComponentsVertexProgram.build().maxIterations(100).create()).
+ mapReduce(ClusterPopulationMapReduce.build().create()).
+ mapReduce(ClusterCountMapReduce.build().create()).
+ submit().get()
+result.memory().clusterPopulation
+gResult = result.graph().traversal()
+gResult.V().valueMap(true)
----
+
+The vertex program has interconnected vertices exchange id's and store the lowest id until no vertex receives a
+lower id. This algorithm is commonly applied in
+link:https://en.wikipedia.org/wiki/Bulk_synchronous_parallel[bulk synchronous parallel] systems, e.g. in
+link:https://spark.apache.org/graphx[Apache Spark GraphX].
+
+==== Scalability
+
+ToDo:
+ - limits and run time regime 1
+ - test of friendster graph regime 3
+ - discuss: link:http://www.vldb.org/pvldb/vol7/p1821-yan.pdf[http://www.vldb.org/pvldb/vol7/p1821-yan.pdf]
\ No newline at end of file
[07/12] tinkerpop git commit: Fixed spacing error in CHANGELOG CTR
Posted by sp...@apache.org.
Fixed spacing error in CHANGELOG CTR
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/61238bfe
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/61238bfe
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/61238bfe
Branch: refs/heads/TINKERPOP-1967
Commit: 61238bfe6f4d5710d796e3b520157b86036af4d4
Parents: 4a7cdb5
Author: Stephen Mallette <sp...@genoprime.com>
Authored: Wed Jun 13 06:05:10 2018 -0400
Committer: Stephen Mallette <sp...@genoprime.com>
Committed: Wed Jun 13 06:05:10 2018 -0400
----------------------------------------------------------------------
CHANGELOG.asciidoc | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/61238bfe/CHANGELOG.asciidoc
----------------------------------------------------------------------
diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc
index e74d6a6..5e36f57 100644
--- a/CHANGELOG.asciidoc
+++ b/CHANGELOG.asciidoc
@@ -38,7 +38,7 @@ This release also includes changes from <<release-3-3-3, 3.3.3>>.
* Changed the order of `select()` scopes. The order is now: maps, side-effects, paths.
* Removed previously deprecated Credentials DSL infrastructure.
* Moved `TraversalEngine` to `gremlin-test` as it has long been only used in testing infrastructure.
-* Events from `EventStrategy`raised from "new" mutations will now return a true "empty" property from `VertexProperty.empty()` or `Property.empty()` as is appropriate.
+* Events from `EventStrategy` raised from "new" mutations will now return a true "empty" property from `VertexProperty.empty()` or `Property.empty()` as is appropriate.
* `MutationListener#vertexPropertyChanged(Vertex, VertexProperty, Object, Object...)` no longer has a default implementation.
* Removed previously deprecated `MutationListener#vertexPropertyChanged(Vertex, Property, Object, Object...)`.
* Removed previously deprecated `OpSelectorHandler` constructor.
[04/12] tinkerpop git commit: Merge branch 'TINKERPOP-1518'
Posted by sp...@apache.org.
Merge branch 'TINKERPOP-1518'
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/ad19a117
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/ad19a117
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/ad19a117
Branch: refs/heads/TINKERPOP-1967
Commit: ad19a1179d241340e81413fd951c00dd9cb82058
Parents: f3506ee 221fda5
Author: Stephen Mallette <sp...@genoprime.com>
Authored: Wed Jun 13 05:44:54 2018 -0400
Committer: Stephen Mallette <sp...@genoprime.com>
Committed: Wed Jun 13 05:44:54 2018 -0400
----------------------------------------------------------------------
CHANGELOG.asciidoc | 1 +
docs/src/dev/provider/index.asciidoc | 4 +
docs/src/upgrade/release-3.4.x.asciidoc | 20 +++++
.../tinkerpop/gremlin/structure/Graph.java | 1 -
.../tinkerpop/gremlin/AbstractGremlinTest.java | 78 +++++++++++++-------
.../apache/tinkerpop/gremlin/GraphManager.java | 5 ++
.../apache/tinkerpop/gremlin/GraphProvider.java | 16 ++++
.../neo4j/AbstractNeo4jGraphProvider.java | 23 +++++-
8 files changed, 121 insertions(+), 27 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/ad19a117/docs/src/upgrade/release-3.4.x.asciidoc
----------------------------------------------------------------------
[06/12] tinkerpop git commit: Merge branch 'TINKERPOP-1831'
Posted by sp...@apache.org.
Merge branch 'TINKERPOP-1831'
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/4a7cdb58
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/4a7cdb58
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/4a7cdb58
Branch: refs/heads/TINKERPOP-1967
Commit: 4a7cdb589b8985b814bd8f02064a5fc78e835df5
Parents: 89c23bd 7963fdf
Author: Stephen Mallette <sp...@genoprime.com>
Authored: Wed Jun 13 05:49:20 2018 -0400
Committer: Stephen Mallette <sp...@genoprime.com>
Committed: Wed Jun 13 05:49:20 2018 -0400
----------------------------------------------------------------------
CHANGELOG.asciidoc | 3 +
docs/src/upgrade/release-3.4.x.asciidoc | 22 +++
.../bulkloading/BulkLoaderVertexProgram.java | 6 -
.../step/sideEffect/AddPropertyStep.java | 6 +-
.../util/event/ConsoleMutationListener.java | 5 -
.../step/util/event/MutationListener.java | 11 +-
.../strategy/decoration/EventStrategy.java | 108 +++++-----
.../strategy/decoration/EventStrategyTest.java | 1 -
.../decoration/EventStrategyProcessTest.java | 197 ++++---------------
9 files changed, 130 insertions(+), 229 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/4a7cdb58/CHANGELOG.asciidoc
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/4a7cdb58/docs/src/upgrade/release-3.4.x.asciidoc
----------------------------------------------------------------------
[03/12] tinkerpop git commit: TINKERPOP-1518 Minor refactoring -
extracted requirement check function
Posted by sp...@apache.org.
TINKERPOP-1518 Minor refactoring - extracted requirement check function
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/221fda58
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/221fda58
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/221fda58
Branch: refs/heads/TINKERPOP-1967
Commit: 221fda584c6baf7c65eb4e6eebf72bb0ba251de4
Parents: aec1709
Author: Stephen Mallette <sp...@genoprime.com>
Authored: Fri Jun 8 07:45:29 2018 -0400
Committer: Stephen Mallette <sp...@genoprime.com>
Committed: Fri Jun 8 07:45:29 2018 -0400
----------------------------------------------------------------------
.../tinkerpop/gremlin/AbstractGremlinTest.java | 52 ++++++++++----------
1 file changed, 26 insertions(+), 26 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/221fda58/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/AbstractGremlinTest.java
----------------------------------------------------------------------
diff --git a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/AbstractGremlinTest.java b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/AbstractGremlinTest.java
index 8f55329..7ca44ba 100644
--- a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/AbstractGremlinTest.java
+++ b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/AbstractGremlinTest.java
@@ -37,6 +37,7 @@ import org.junit.rules.TestName;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
+import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.util.ArrayList;
import java.util.Arrays;
@@ -64,8 +65,6 @@ import static org.junit.Assume.assumeThat;
public abstract class AbstractGremlinTest {
private static final Logger logger = LoggerFactory.getLogger(AbstractGremlinTest.class);
- protected static final Map<Pair<Class<?>, String>, Boolean> featureCache = new HashMap<>();
-
protected Graph graph;
protected GraphTraversalSource g;
protected Configuration config;
@@ -84,20 +83,10 @@ public abstract class AbstractGremlinTest {
graphProvider = GraphManager.getGraphProvider();
+ // pre-check if available from graph provider to avoid graph creation
final Optional<Graph.Features> staticFeatures = graphProvider.getStaticFeatures();
if (staticFeatures.isPresent()) {
- for (FeatureRequirement fr : featureRequirementSet) {
- try {
- assumeThat(String.format("StaticFeatures of the graph do not support all of the features required by this test so it will be ignored: %s.%s=%s",
- fr.featureClass().getSimpleName(), fr.feature(), fr.supported()),
- staticFeatures.get().supports(fr.featureClass(), fr.feature()), is(fr.supported()));
- } catch (NoSuchMethodException nsme) {
- throw new NoSuchMethodException(String.format("[supports%s] is not a valid feature on %s", fr.feature(), fr.featureClass()));
- } catch (UnsupportedOperationException uoe) {
- // no worries - it just means that we can't use the cache to support this check and will have to
- // incur the cost of instantiating a graph instance directly
- }
- }
+ assumeRequirementsAreMetForTest(featureRequirementSet, staticFeatures.get(), true);
}
graphProvider.getTestListener().ifPresent(l -> l.onTestStart(this.getClass(), name.getMethodName()));
@@ -110,18 +99,10 @@ public abstract class AbstractGremlinTest {
graph = graphProvider.openTestGraph(config);
g = graphProvider.traversal(graph);
- for (FeatureRequirement fr : featureRequirementSet) {
- try {
- // even if we checked static features above it's of little cost to recheck again with the real graph
- // once it is instantiated. the real cost savings is preventing graph creation in the first place so
- // let's double check that all is legit.
- assumeThat(String.format("%s does not support all of the features required by this test so it will be ignored: %s.%s=%s",
- graph.getClass().getSimpleName(), fr.featureClass().getSimpleName(), fr.feature(), fr.supported()),
- graph.features().supports(fr.featureClass(), fr.feature()), is(fr.supported()));
- } catch (NoSuchMethodException nsme) {
- throw new NoSuchMethodException(String.format("[supports%s] is not a valid feature on %s", fr.feature(), fr.featureClass()));
- }
- }
+ // even if we checked static features earlier it's of little cost to recheck again with the real graph
+ // once it is instantiated. the real cost savings is preventing graph creation in the first place so
+ // let's double check that all is legit.
+ assumeRequirementsAreMetForTest(featureRequirementSet, graph.features(), false);
beforeLoadGraphWith(graph);
@@ -131,6 +112,25 @@ public abstract class AbstractGremlinTest {
afterLoadGraphWith(graph);
}
+ private static void assumeRequirementsAreMetForTest(final Set<FeatureRequirement> featureRequirementSet,
+ final Graph.Features features, final boolean staticCheck)
+ throws IllegalAccessException, InvocationTargetException, NoSuchMethodException {
+ for (FeatureRequirement fr : featureRequirementSet) {
+ try {
+ assumeThat(String.format("Features of the graph do not support all of the features required by this test so it will be ignored: %s.%s=%s",
+ fr.featureClass().getSimpleName(), fr.feature(), fr.supported()),
+ features.supports(fr.featureClass(), fr.feature()), is(fr.supported()));
+ } catch (NoSuchMethodException nsme) {
+ throw new NoSuchMethodException(String.format("[supports%s] is not a valid feature on %s", fr.feature(), fr.featureClass()));
+ } catch (UnsupportedOperationException uoe) {
+ // no worries if this is a check of static features - it just means that we can't use the cache to
+ // support this check and will have to incur the cost of instantiating a graph instance directly. but,
+ // if this is not a static check then something else is amiss and we should throw.
+ if (staticCheck) throw uoe;
+ }
+ }
+ }
+
protected void beforeLoadGraphWith(final Graph g) throws Exception {
// do nothing
}
[02/12] tinkerpop git commit: TINKERPOP-1518 GraphProvider allows for
caching of graph features
Posted by sp...@apache.org.
TINKERPOP-1518 GraphProvider allows for caching of graph features
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/aec17098
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/aec17098
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/aec17098
Branch: refs/heads/TINKERPOP-1967
Commit: aec170982aa03d1a07248721cb045ba95e11188b
Parents: ae2f304
Author: Stephen Mallette <sp...@genoprime.com>
Authored: Thu Jun 7 20:15:53 2018 -0400
Committer: Stephen Mallette <sp...@genoprime.com>
Committed: Thu Jun 7 20:15:53 2018 -0400
----------------------------------------------------------------------
CHANGELOG.asciidoc | 1 +
docs/src/dev/provider/index.asciidoc | 4 ++
docs/src/upgrade/release-3.4.x.asciidoc | 20 ++++++
.../tinkerpop/gremlin/structure/Graph.java | 1 -
.../tinkerpop/gremlin/AbstractGremlinTest.java | 64 ++++++++++++++------
.../apache/tinkerpop/gremlin/GraphManager.java | 5 ++
.../apache/tinkerpop/gremlin/GraphProvider.java | 16 +++++
.../neo4j/AbstractNeo4jGraphProvider.java | 23 ++++++-
8 files changed, 114 insertions(+), 20 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/aec17098/CHANGELOG.asciidoc
----------------------------------------------------------------------
diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc
index e886107..a846363 100644
--- a/CHANGELOG.asciidoc
+++ b/CHANGELOG.asciidoc
@@ -33,6 +33,7 @@ This release also includes changes from <<release-3-3-3, 3.3.3>>.
* `min()` and `max()` now support all types implementing `Comparable`.
* Change the `toString()` of `Path` to be standardized as other graph elements are.
* `hadoop-gremlin` no longer generates a test artifact.
+* Allowed `GraphProvider` to expose a cached `Graph.Feature` object so that the test suite could re-use them to speed test runs.
* Fixed a bug in `ReducingBarrierStep`, that returned the provided seed value despite no elements being available.
* Changed the order of `select()` scopes. The order is now: maps, side-effects, paths.
* Removed previously deprecated Credentials DSL infrastructure.
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/aec17098/docs/src/dev/provider/index.asciidoc
----------------------------------------------------------------------
diff --git a/docs/src/dev/provider/index.asciidoc b/docs/src/dev/provider/index.asciidoc
index f6a964b..896f85c 100644
--- a/docs/src/dev/provider/index.asciidoc
+++ b/docs/src/dev/provider/index.asciidoc
@@ -662,6 +662,10 @@ should validate that the ignored tests are appropriately bypassed and that there
definitions. Moreover, implementers should consider filling gaps in their own test suites, especially when
IO-related tests are being ignored.
+TIP: If it is expensive to construct a new `Graph` instance, consider implementing `GraphProvider.getStaticFeatures()`
+which can help by caching a static feature set for instances produced by that `GraphProvider` and allow the test suite
+to avoid that construction cost if the test is ignored.
+
The only test-class that requires any code investment is the `GraphProvider` implementation class. This class is a
used by the test suite to construct `Graph` configurations and instances and provides information about the
implementation itself. In most cases, it is best to simply extend `AbstractGraphProvider` as it provides many
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/aec17098/docs/src/upgrade/release-3.4.x.asciidoc
----------------------------------------------------------------------
diff --git a/docs/src/upgrade/release-3.4.x.asciidoc b/docs/src/upgrade/release-3.4.x.asciidoc
index 851d458..741fb3c 100644
--- a/docs/src/upgrade/release-3.4.x.asciidoc
+++ b/docs/src/upgrade/release-3.4.x.asciidoc
@@ -241,6 +241,26 @@ See: link:https://issues.apache.org/jira/browse/TINKERPOP-1522[TINKERPOP-1522]
==== Graph Database Providers
+===== Caching Graph Features
+
+For graph implementations that have expensive creation times, it can be time consuming to run the TinkerPop test suite
+as each test run requires a `Graph` instance even if the test is ultimately ignored becaue it doesn't pass the feature
+checks. To possibly help alleviate this problem, the `GraphProvider` interface now includes this method:
+
+[source,java]
+----
+public default Optional<Graph.Features> getStaticFeatures() {
+ return Optional.empty();
+}
+----
+
+This method can be implemented to return a cacheable set of features for a `Graph` generated from that `GraphProvider`.
+Assuming this method is faster than the cost of creating a new `Graph` instance, the test suite should execute
+significantly faster depending on how many tests end up being ignored.
+
+See: link:https://issues.apache.org/jira/browse/TINKERPOP-1518[TINKERPOP-1518],
+link:
+
===== Configuring Interface
There were some changes to interfaces that were related to `Step`. A new `Configuring` interface was added that was
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/aec17098/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/Graph.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/Graph.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/Graph.java
index 2fbfe03..dc14cc6 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/Graph.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/Graph.java
@@ -40,7 +40,6 @@ import java.lang.reflect.InvocationTargetException;
import java.util.Collections;
import java.util.Iterator;
import java.util.Map;
-import java.util.NoSuchElementException;
import java.util.Optional;
import java.util.Set;
import java.util.UUID;
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/aec17098/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/AbstractGremlinTest.java
----------------------------------------------------------------------
diff --git a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/AbstractGremlinTest.java b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/AbstractGremlinTest.java
index 25d7a55..8f55329 100644
--- a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/AbstractGremlinTest.java
+++ b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/AbstractGremlinTest.java
@@ -29,9 +29,9 @@ import org.apache.tinkerpop.gremlin.structure.Graph;
import org.apache.tinkerpop.gremlin.structure.Vertex;
import org.apache.tinkerpop.gremlin.structure.VertexProperty;
import org.apache.tinkerpop.gremlin.util.iterator.IteratorUtils;
+import org.javatuples.Pair;
import org.junit.After;
import org.junit.Before;
-import org.junit.BeforeClass;
import org.junit.Rule;
import org.junit.rules.TestName;
import org.slf4j.Logger;
@@ -40,9 +40,11 @@ import org.slf4j.LoggerFactory;
import java.lang.reflect.Method;
import java.util.ArrayList;
import java.util.Arrays;
+import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
-import java.util.Random;
+import java.util.Map;
+import java.util.Optional;
import java.util.Set;
import java.util.function.Consumer;
import java.util.stream.Collectors;
@@ -61,6 +63,9 @@ import static org.junit.Assume.assumeThat;
*/
public abstract class AbstractGremlinTest {
private static final Logger logger = LoggerFactory.getLogger(AbstractGremlinTest.class);
+
+ protected static final Map<Pair<Class<?>, String>, Boolean> featureCache = new HashMap<>();
+
protected Graph graph;
protected GraphTraversalSource g;
protected Configuration config;
@@ -75,8 +80,26 @@ public abstract class AbstractGremlinTest {
final LoadGraphWith[] loadGraphWiths = testMethod.getAnnotationsByType(LoadGraphWith.class);
final LoadGraphWith loadGraphWith = loadGraphWiths.length == 0 ? null : loadGraphWiths[0];
final LoadGraphWith.GraphData loadGraphWithData = null == loadGraphWith ? null : loadGraphWith.value();
+ final Set<FeatureRequirement> featureRequirementSet = getFeatureRequirementsForTest(testMethod, loadGraphWiths);
graphProvider = GraphManager.getGraphProvider();
+
+ final Optional<Graph.Features> staticFeatures = graphProvider.getStaticFeatures();
+ if (staticFeatures.isPresent()) {
+ for (FeatureRequirement fr : featureRequirementSet) {
+ try {
+ assumeThat(String.format("StaticFeatures of the graph do not support all of the features required by this test so it will be ignored: %s.%s=%s",
+ fr.featureClass().getSimpleName(), fr.feature(), fr.supported()),
+ staticFeatures.get().supports(fr.featureClass(), fr.feature()), is(fr.supported()));
+ } catch (NoSuchMethodException nsme) {
+ throw new NoSuchMethodException(String.format("[supports%s] is not a valid feature on %s", fr.feature(), fr.featureClass()));
+ } catch (UnsupportedOperationException uoe) {
+ // no worries - it just means that we can't use the cache to support this check and will have to
+ // incur the cost of instantiating a graph instance directly
+ }
+ }
+ }
+
graphProvider.getTestListener().ifPresent(l -> l.onTestStart(this.getClass(), name.getMethodName()));
config = graphProvider.standardGraphConfiguration(this.getClass(), name.getMethodName(), loadGraphWithData);
@@ -87,24 +110,11 @@ public abstract class AbstractGremlinTest {
graph = graphProvider.openTestGraph(config);
g = graphProvider.traversal(graph);
- // get feature requirements on the test method and add them to the list of ones to check
- final FeatureRequirement[] featureRequirement = testMethod.getAnnotationsByType(FeatureRequirement.class);
- final List<FeatureRequirement> frs = new ArrayList<>(Arrays.asList(featureRequirement));
-
- // if the graph is loading data then it will come with it's own requirements
- if (loadGraphWiths.length > 0) frs.addAll(loadGraphWiths[0].value().featuresRequired());
-
- // if the graph has a set of feature requirements bundled together then add those
- final FeatureRequirementSet[] featureRequirementSets = testMethod.getAnnotationsByType(FeatureRequirementSet.class);
- if (featureRequirementSets.length > 0)
- frs.addAll(Arrays.stream(featureRequirementSets)
- .flatMap(f -> f.value().featuresRequired().stream()).collect(Collectors.toList()));
-
- // process the unique set of feature requirements
- final Set<FeatureRequirement> featureRequirementSet = new HashSet<>(frs);
for (FeatureRequirement fr : featureRequirementSet) {
try {
- //System.out.println(String.format("Assume that %s meets Feature Requirement - %s - with %s", fr.featureClass().getSimpleName(), fr.feature(), fr.supported()));
+ // even if we checked static features above it's of little cost to recheck again with the real graph
+ // once it is instantiated. the real cost savings is preventing graph creation in the first place so
+ // let's double check that all is legit.
assumeThat(String.format("%s does not support all of the features required by this test so it will be ignored: %s.%s=%s",
graph.getClass().getSimpleName(), fr.featureClass().getSimpleName(), fr.feature(), fr.supported()),
graph.features().supports(fr.featureClass(), fr.feature()), is(fr.supported()));
@@ -263,6 +273,24 @@ public abstract class AbstractGremlinTest {
AbstractGremlinTest.verifyUniqueStepIds(traversal, 0, new HashSet<>());
}
+ private static Set<FeatureRequirement> getFeatureRequirementsForTest(final Method testMethod, final LoadGraphWith[] loadGraphWiths) {
+ // get feature requirements on the test method and add them to the list of ones to check
+ final FeatureRequirement[] featureRequirement = testMethod.getAnnotationsByType(FeatureRequirement.class);
+ final List<FeatureRequirement> frs = new ArrayList<>(Arrays.asList(featureRequirement));
+
+ // if the graph is loading data then it will come with it's own requirements
+ if (loadGraphWiths.length > 0) frs.addAll(loadGraphWiths[0].value().featuresRequired());
+
+ // if the graph has a set of feature requirements bundled together then add those
+ final FeatureRequirementSet[] featureRequirementSets = testMethod.getAnnotationsByType(FeatureRequirementSet.class);
+ if (featureRequirementSets.length > 0)
+ frs.addAll(Arrays.stream(featureRequirementSets)
+ .flatMap(f -> f.value().featuresRequired().stream()).collect(Collectors.toList()));
+
+ // process the unique set of feature requirements
+ return new HashSet<>(frs);
+ }
+
private static void verifyUniqueStepIds(final Traversal.Admin<?, ?> traversal, final int depth, final Set<String> ids) {
for (final Step step : traversal.asAdmin().getSteps()) {
/*for (int i = 0; i < depth; i++) System.out.print("\t");
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/aec17098/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/GraphManager.java
----------------------------------------------------------------------
diff --git a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/GraphManager.java b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/GraphManager.java
index 6886465..7506226 100644
--- a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/GraphManager.java
+++ b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/GraphManager.java
@@ -183,6 +183,11 @@ public class GraphManager {
if (innerGraphProvider instanceof AutoCloseable)
((AutoCloseable) innerGraphProvider).close();
}
+
+ @Override
+ public Optional<Graph.Features> getStaticFeatures() {
+ return innerGraphProvider.getStaticFeatures();
+ }
}
}
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/aec17098/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/GraphProvider.java
----------------------------------------------------------------------
diff --git a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/GraphProvider.java b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/GraphProvider.java
index c785cfc..1fcb147 100644
--- a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/GraphProvider.java
+++ b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/GraphProvider.java
@@ -294,6 +294,22 @@ public interface GraphProvider {
}
/**
+ * Gets a {@link Graph.Features} implementation that contains graph feature configuration that will never change
+ * from execution to execution of the tests given this current {@code GraphProvider} implementation. Implementing
+ * this method will allow the test suite to avoid creation of a {@link Graph} instance and thus speed up the
+ * execution of tests if that creation process is expensive. It is important that this static set of features be
+ * representative of what the {@link Graph} instance produced by this {@code GraphProvider} can actually do or
+ * else the cost of {@link Graph} instantiation will be incurred when it doesn't need to be. It is also important
+ * that this method be faster than the cost of {@link Graph} creation in the first place or there really won't be
+ * any difference in execution speed. In cases where some features are static and others are not, simply throw
+ * an {@code UnsupportedOperationException} from those features to let the test suite know that it cannot rely
+ * on them and the test suite will revert to using a constructed {@link Graph} instance.
+ */
+ public default Optional<Graph.Features> getStaticFeatures() {
+ return Optional.empty();
+ }
+
+ /**
* An annotation to be applied to a {@code GraphProvider} implementation that provides additional information
* about its intentions. The {@code Descriptor} is required by those {@code GraphProvider} implementations
* that will be assigned to test suites that use
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/aec17098/neo4j-gremlin/src/test/java/org/apache/tinkerpop/gremlin/neo4j/AbstractNeo4jGraphProvider.java
----------------------------------------------------------------------
diff --git a/neo4j-gremlin/src/test/java/org/apache/tinkerpop/gremlin/neo4j/AbstractNeo4jGraphProvider.java b/neo4j-gremlin/src/test/java/org/apache/tinkerpop/gremlin/neo4j/AbstractNeo4jGraphProvider.java
index 9226822..514ef2b 100644
--- a/neo4j-gremlin/src/test/java/org/apache/tinkerpop/gremlin/neo4j/AbstractNeo4jGraphProvider.java
+++ b/neo4j-gremlin/src/test/java/org/apache/tinkerpop/gremlin/neo4j/AbstractNeo4jGraphProvider.java
@@ -32,6 +32,7 @@ import org.apache.tinkerpop.gremlin.structure.Graph;
import java.io.File;
import java.util.HashSet;
+import java.util.Optional;
import java.util.Random;
import java.util.Set;
@@ -49,6 +50,26 @@ public abstract class AbstractNeo4jGraphProvider extends AbstractGraphProvider {
add(Neo4jVertexProperty.class);
}};
+ protected Graph.Features features = null;
+
+ @Override
+ public Graph openTestGraph(final Configuration config) {
+ final Graph graph = super.openTestGraph(config);
+
+ // we can just use the initial set of features taken from the first graph generated from the provider because
+ // neo4j feature won't ever change. don't think there is any danger of keeping this instance about even if
+ // the original graph instance goes out of scope.
+ if (null == features) {
+ this.features = graph.features();
+ }
+ return graph;
+ }
+
+ @Override
+ public Optional<Graph.Features> getStaticFeatures() {
+ return Optional.ofNullable(features);
+ }
+
@Override
public void clear(final Graph graph, final Configuration configuration) throws Exception {
if (null != graph) {
@@ -56,7 +77,7 @@ public abstract class AbstractNeo4jGraphProvider extends AbstractGraphProvider {
graph.close();
}
- if (configuration.containsKey(Neo4jGraph.CONFIG_DIRECTORY)) {
+ if (configuration != null && configuration.containsKey(Neo4jGraph.CONFIG_DIRECTORY)) {
// this is a non-in-sideEffects configuration so blow away the directory
final File graphDirectory = new File(configuration.getString(Neo4jGraph.CONFIG_DIRECTORY));
deleteDirectory(graphDirectory);
[08/12] tinkerpop git commit: TINKERPOP-1831 Added
KeyedProperty/VertexProperty for EventStrategy
Posted by sp...@apache.org.
TINKERPOP-1831 Added KeyedProperty/VertexProperty for EventStrategy
This is a minor but necessary change from the previous approach that used EmptyProperty/VertexProperty for "new" property additions. Use of "empty" was not sufficient because it didn't allow the MutationListener to know the key being updated. CTR
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/fc866751
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/fc866751
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/fc866751
Branch: refs/heads/TINKERPOP-1967
Commit: fc866751e11768666ed347f772d86d888a2dec16
Parents: 61238bf
Author: Stephen Mallette <sp...@genoprime.com>
Authored: Wed Jun 13 10:13:15 2018 -0400
Committer: Stephen Mallette <sp...@genoprime.com>
Committed: Wed Jun 13 10:15:55 2018 -0400
----------------------------------------------------------------------
CHANGELOG.asciidoc | 2 +-
docs/src/upgrade/release-3.4.x.asciidoc | 4 +-
.../step/sideEffect/AddPropertyStep.java | 8 +-
.../util/empty/EmptyVertexProperty.java | 2 +-
.../structure/util/keyed/KeyedProperty.java | 85 ++++++++++++++
.../util/keyed/KeyedVertexProperty.java | 114 +++++++++++++++++++
.../decoration/EventStrategyProcessTest.java | 29 +++--
7 files changed, 228 insertions(+), 16 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/fc866751/CHANGELOG.asciidoc
----------------------------------------------------------------------
diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc
index 5e36f57..2320162 100644
--- a/CHANGELOG.asciidoc
+++ b/CHANGELOG.asciidoc
@@ -38,7 +38,7 @@ This release also includes changes from <<release-3-3-3, 3.3.3>>.
* Changed the order of `select()` scopes. The order is now: maps, side-effects, paths.
* Removed previously deprecated Credentials DSL infrastructure.
* Moved `TraversalEngine` to `gremlin-test` as it has long been only used in testing infrastructure.
-* Events from `EventStrategy` raised from "new" mutations will now return a true "empty" property from `VertexProperty.empty()` or `Property.empty()` as is appropriate.
+* Events from `EventStrategy` raised from "new" mutations will now return a `KeyedVertexProperty` or `KeyedProperty` as is appropriate.
* `MutationListener#vertexPropertyChanged(Vertex, VertexProperty, Object, Object...)` no longer has a default implementation.
* Removed previously deprecated `MutationListener#vertexPropertyChanged(Vertex, Property, Object, Object...)`.
* Removed previously deprecated `OpSelectorHandler` constructor.
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/fc866751/docs/src/upgrade/release-3.4.x.asciidoc
----------------------------------------------------------------------
diff --git a/docs/src/upgrade/release-3.4.x.asciidoc b/docs/src/upgrade/release-3.4.x.asciidoc
index 68f7cd3..8829569 100644
--- a/docs/src/upgrade/release-3.4.x.asciidoc
+++ b/docs/src/upgrade/release-3.4.x.asciidoc
@@ -129,8 +129,8 @@ have originally been the correct signature of `vertexPropertyChanged(Vertex, Ver
prior versions when this method and its related `edgePropertyChanged()` and `vertexPropertyPropertyChanged()` were
triggered by way of the addition of a new property a "fake" property was included with a `null` value for the
"oldValue" argument to these methods (as it did not exist prior to this event). That was a bit awkward to reason about
-when dealing with that event. To make this easier, the event now raises with a `VertexProperty.empty()` or
-`Property.empty()` instance, which can be evaluated with `equals()` easily to determine if the property is new or not.
+when dealing with that event. To make this easier, the event now raises with a `KeyedVertexProperty` or
+`KeyedProperty` instance, which only contains a property key and no value in them.
link:https://issues.apache.org/jira/browse/TINKERPOP-1831[TINKERPOP-1831]
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/fc866751/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/sideEffect/AddPropertyStep.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/sideEffect/AddPropertyStep.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/sideEffect/AddPropertyStep.java
index 3589c0c..6b814c3 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/sideEffect/AddPropertyStep.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/sideEffect/AddPropertyStep.java
@@ -37,6 +37,8 @@ import org.apache.tinkerpop.gremlin.structure.T;
import org.apache.tinkerpop.gremlin.structure.Vertex;
import org.apache.tinkerpop.gremlin.structure.VertexProperty;
import org.apache.tinkerpop.gremlin.structure.util.StringFactory;
+import org.apache.tinkerpop.gremlin.structure.util.keyed.KeyedProperty;
+import org.apache.tinkerpop.gremlin.structure.util.keyed.KeyedVertexProperty;
import java.util.List;
import java.util.Set;
@@ -97,17 +99,17 @@ public final class AddPropertyStep<S extends Element> extends SideEffectStep<S>
if (element instanceof Vertex)
evt = new Event.VertexPropertyChangedEvent(eventStrategy.detach((Vertex) element),
newProperty ?
- VertexProperty.empty() :
+ new KeyedVertexProperty(key) :
eventStrategy.detach((VertexProperty) currentProperty), value, vertexPropertyKeyValues);
else if (element instanceof Edge)
evt = new Event.EdgePropertyChangedEvent(eventStrategy.detach((Edge) element),
newProperty ?
- Property.empty() :
+ new KeyedProperty(key) :
eventStrategy.detach(currentProperty), value);
else if (element instanceof VertexProperty)
evt = new Event.VertexPropertyPropertyChangedEvent(eventStrategy.detach((VertexProperty) element),
newProperty ?
- Property.empty() :
+ new KeyedProperty(key) :
eventStrategy.detach(currentProperty), value);
else
throw new IllegalStateException(String.format("The incoming object cannot be processed by change eventing in %s: %s", AddPropertyStep.class.getName(), element));
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/fc866751/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/util/empty/EmptyVertexProperty.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/util/empty/EmptyVertexProperty.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/util/empty/EmptyVertexProperty.java
index a77fd6f..c7a5b44 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/util/empty/EmptyVertexProperty.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/util/empty/EmptyVertexProperty.java
@@ -90,7 +90,7 @@ public final class EmptyVertexProperty<V> implements VertexProperty<V> {
}
@Override
- public <U> Iterator<Property<U>> properties(String... propertyKeys) {
+ public <U> Iterator<Property<U>> properties(final String... propertyKeys) {
return Collections.emptyIterator();
}
}
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/fc866751/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/util/keyed/KeyedProperty.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/util/keyed/KeyedProperty.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/util/keyed/KeyedProperty.java
new file mode 100644
index 0000000..5532f06
--- /dev/null
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/util/keyed/KeyedProperty.java
@@ -0,0 +1,85 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.gremlin.structure.util.keyed;
+
+import org.apache.tinkerpop.gremlin.structure.Element;
+import org.apache.tinkerpop.gremlin.structure.Property;
+import org.apache.tinkerpop.gremlin.structure.util.StringFactory;
+
+import java.util.NoSuchElementException;
+
+/**
+ * A utility implementation of a {@link Property} that only has a key but no value.
+ *
+ * @author Stephen Mallette (http://stephen.genoprime.com)
+ */
+public final class KeyedProperty<V> implements Property<V> {
+
+ private final String key;
+
+ public KeyedProperty(final String key) {
+ if (null == key || key.isEmpty()) throw new IllegalArgumentException("key cannot be null");
+ this.key = key;
+ }
+
+ @Override
+ public String key() {
+ return this.key;
+ }
+
+ @Override
+ public V value() throws NoSuchElementException {
+ throw Exceptions.propertyDoesNotExist();
+ }
+
+ @Override
+ public boolean isPresent() {
+ return false;
+ }
+
+ @Override
+ public Element element() {
+ throw Exceptions.propertyDoesNotExist();
+ }
+
+ @Override
+ public void remove() {
+
+ }
+
+ @Override
+ public String toString() {
+ return StringFactory.propertyString(this);
+ }
+
+ @Override
+ public boolean equals(final Object o) {
+ if (this == o) return true;
+ if (o == null || getClass() != o.getClass()) return false;
+
+ final KeyedProperty<?> that = (KeyedProperty<?>) o;
+
+ return key.equals(that.key);
+ }
+
+ @Override
+ public int hashCode() {
+ return key.hashCode();
+ }
+}
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/fc866751/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/util/keyed/KeyedVertexProperty.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/util/keyed/KeyedVertexProperty.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/util/keyed/KeyedVertexProperty.java
new file mode 100644
index 0000000..53cba69
--- /dev/null
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/util/keyed/KeyedVertexProperty.java
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.gremlin.structure.util.keyed;
+
+import org.apache.tinkerpop.gremlin.structure.Graph;
+import org.apache.tinkerpop.gremlin.structure.Property;
+import org.apache.tinkerpop.gremlin.structure.Vertex;
+import org.apache.tinkerpop.gremlin.structure.VertexProperty;
+import org.apache.tinkerpop.gremlin.structure.util.StringFactory;
+
+import java.util.Collections;
+import java.util.Iterator;
+import java.util.NoSuchElementException;
+
+/**
+ * A utility implementation of a {@link Property} that only has a key but no value and no meta-properties.
+ *
+ * @author Stephen Mallette (http://stephen.genoprime.com)
+ */
+public final class KeyedVertexProperty<V> implements VertexProperty<V> {
+
+ private final String key;
+
+ public KeyedVertexProperty(final String key) {
+ if (null == key || key.isEmpty()) throw new IllegalArgumentException("key cannot be null");
+ this.key = key;
+ }
+
+ @Override
+ public Vertex element() {
+ throw Property.Exceptions.propertyDoesNotExist();
+ }
+
+ @Override
+ public Object id() {
+ throw Property.Exceptions.propertyDoesNotExist();
+ }
+
+ @Override
+ public Graph graph() {
+ throw Property.Exceptions.propertyDoesNotExist();
+ }
+
+ @Override
+ public <U> Property<U> property(String key) {
+ return Property.<U>empty();
+ }
+
+ @Override
+ public <U> Property<U> property(String key, U value) {
+ return Property.<U>empty();
+ }
+
+ @Override
+ public String key() {
+ return this.key;
+ }
+
+ @Override
+ public V value() throws NoSuchElementException {
+ throw Property.Exceptions.propertyDoesNotExist();
+ }
+
+ @Override
+ public boolean isPresent() {
+ return false;
+ }
+
+ @Override
+ public void remove() {
+
+ }
+
+ @Override
+ public String toString() {
+ return StringFactory.propertyString(this);
+ }
+
+ @Override
+ public <U> Iterator<Property<U>> properties(final String... propertyKeys) {
+ return Collections.emptyIterator();
+ }
+
+ @Override
+ public boolean equals(Object o) {
+ if (this == o) return true;
+ if (o == null || getClass() != o.getClass()) return false;
+
+ final KeyedVertexProperty<?> that = (KeyedVertexProperty<?>) o;
+
+ return key.equals(that.key);
+ }
+
+ @Override
+ public int hashCode() {
+ return key.hashCode();
+ }
+}
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/fc866751/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/decoration/EventStrategyProcessTest.java
----------------------------------------------------------------------
diff --git a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/decoration/EventStrategyProcessTest.java b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/decoration/EventStrategyProcessTest.java
index b527340..0992eae 100644
--- a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/decoration/EventStrategyProcessTest.java
+++ b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/decoration/EventStrategyProcessTest.java
@@ -31,6 +31,8 @@ import org.apache.tinkerpop.gremlin.structure.VertexProperty;
import org.apache.tinkerpop.gremlin.structure.util.detached.DetachedEdge;
import org.apache.tinkerpop.gremlin.structure.util.detached.DetachedVertex;
import org.apache.tinkerpop.gremlin.structure.util.detached.DetachedVertexProperty;
+import org.apache.tinkerpop.gremlin.structure.util.keyed.KeyedProperty;
+import org.apache.tinkerpop.gremlin.structure.util.keyed.KeyedVertexProperty;
import org.apache.tinkerpop.gremlin.structure.util.reference.ReferenceEdge;
import org.apache.tinkerpop.gremlin.structure.util.reference.ReferenceVertex;
import org.apache.tinkerpop.gremlin.structure.util.reference.ReferenceVertexProperty;
@@ -633,7 +635,8 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
assertThat(element, instanceOf(DetachedVertexProperty.class));
assertEquals(label, element.label());
assertEquals(value, element.value());
- assertEquals(Property.empty(), oldValue);
+ assertThat(oldValue, instanceOf(KeyedProperty.class));
+ assertEquals("new", oldValue.key());
assertEquals("yay!", setValue);
triggered.set(true);
}
@@ -750,7 +753,8 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
assertEquals(label, element.label());
assertEquals(inId, element.inVertex().id());
assertEquals(outId, element.outVertex().id());
- assertEquals(Property.empty(), oldValue);
+ assertThat(oldValue, instanceOf(KeyedProperty.class));
+ assertEquals("new", oldValue.key());
assertEquals("yay!", setValue);
triggered.set(true);
}
@@ -925,7 +929,8 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
assertThat(element, instanceOf(DetachedVertex.class));
assertEquals(label, element.label());
assertEquals(id, element.id());
- assertEquals(VertexProperty.empty(), oldValue);
+ assertThat(oldValue, instanceOf(KeyedVertexProperty.class));
+ assertEquals("new", oldValue.key());
assertEquals("dah", setValue);
triggered.set(true);
}
@@ -1102,7 +1107,8 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
assertThat(element, instanceOf(ReferenceVertexProperty.class));
assertEquals(label, element.label());
assertEquals(value, element.value());
- assertEquals(Property.empty(), oldValue);
+ assertThat(oldValue, instanceOf(KeyedProperty.class));
+ assertEquals("new", oldValue.key());
assertEquals("yay!", setValue);
triggered.set(true);
}
@@ -1219,7 +1225,8 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
assertEquals(label, element.label());
assertEquals(inId, element.inVertex().id());
assertEquals(outId, element.outVertex().id());
- assertEquals(Property.empty(), oldValue);
+ assertThat(oldValue, instanceOf(KeyedProperty.class));
+ assertEquals("new", oldValue.key());
assertEquals("yay!", setValue);
triggered.set(true);
}
@@ -1396,7 +1403,8 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
assertThat(element, instanceOf(ReferenceVertex.class));
assertEquals(label, element.label());
assertEquals(id, element.id());
- assertEquals(VertexProperty.empty(), oldValue);
+ assertThat(oldValue, instanceOf(KeyedVertexProperty.class));
+ assertEquals("new", oldValue.key());
assertEquals("dah", setValue);
triggered.set(true);
}
@@ -1572,7 +1580,8 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
assertEquals(vp, element);
assertEquals(label, element.label());
assertEquals(value, element.value());
- assertEquals(Property.empty(), oldValue);
+ assertThat(oldValue, instanceOf(KeyedProperty.class));
+ assertEquals("new", oldValue.key());
assertEquals("yay!", setValue);
triggered.set(true);
}
@@ -1689,7 +1698,8 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
assertEquals(label, element.label());
assertEquals(inId, element.inVertex().id());
assertEquals(outId, element.outVertex().id());
- assertEquals(Property.empty(), oldValue);
+ assertThat(oldValue, instanceOf(KeyedProperty.class));
+ assertEquals("new", oldValue.key());
assertEquals("yay!", setValue);
triggered.set(true);
}
@@ -1866,7 +1876,8 @@ public class EventStrategyProcessTest extends AbstractGremlinProcessTest {
assertEquals(v, element);
assertEquals(label, element.label());
assertEquals(id, element.id());
- assertEquals(VertexProperty.empty(), oldValue);
+ assertThat(oldValue, instanceOf(KeyedVertexProperty.class));
+ assertEquals("new", oldValue.key());
assertEquals("dah", setValue);
triggered.set(true);
}
[10/12] tinkerpop git commit: TINKERPOP-1967 Minor text cleanup for
connectedComponent() docs
Posted by sp...@apache.org.
TINKERPOP-1967 Minor text cleanup for connectedComponent() docs
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/2469d56f
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/2469d56f
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/2469d56f
Branch: refs/heads/TINKERPOP-1967
Commit: 2469d56f076fd297322dd208280eb62833a9116c
Parents: 93231a1
Author: Stephen Mallette <sp...@genoprime.com>
Authored: Fri Jun 15 08:24:22 2018 -0400
Committer: Stephen Mallette <sp...@genoprime.com>
Committed: Fri Jun 15 08:25:01 2018 -0400
----------------------------------------------------------------------
docs/src/recipes/connected-components.asciidoc | 16 +++++-----------
1 file changed, 5 insertions(+), 11 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/2469d56f/docs/src/recipes/connected-components.asciidoc
----------------------------------------------------------------------
diff --git a/docs/src/recipes/connected-components.asciidoc b/docs/src/recipes/connected-components.asciidoc
index 850c31f..e6d0f7a 100644
--- a/docs/src/recipes/connected-components.asciidoc
+++ b/docs/src/recipes/connected-components.asciidoc
@@ -37,7 +37,6 @@ component membership is stored in the graph, rather than in memory.
3. Large graphs requiring an approach with `HadoopGraph` and `SparkGraphComputer` to yield results in a reasonable time.
-
These regimes are discussed separately using the following graph with three weakly connected components:
image:connected-components.png[width=600]
@@ -70,6 +69,7 @@ g.withComputer().V().connectedComponent().
----
A straightforward way to detect the various subgraphs with an OLTP traversal is to do this:
+
[gremlin-groovy,existing]
----
g.V().emit(cyclicPath().or().not(both())). <1>
@@ -95,8 +95,6 @@ weak component.
<5> The values of the groupby map contain the lists of vertices making up the requested components.
-
-
==== Small graph scalability
The scalability of the OLTP traversal and the `connectedComponent()`-step for in-memory graphs is shown in the figures
@@ -118,7 +116,6 @@ every cycle each vertex has to be checked for being
pure depth-first-search or breadth-first-search implementations, connected-component algotithms should scale
as [.big]##O##(V+E). For the traversals in the figure above this is almost the case.
-
[[cc-scale-ratio]]
.Run times for finding connected components in a randomly generated graph with 10 components, each consisting of 6400 vertices
image::cc-scale-ratio.png[width=600]
@@ -130,7 +127,6 @@ characteristics show clearly from the graph. Indeed, for a given number of verti
`connectedComponent()`-step does not depend on the number of edges, but rather on the maximum shortest path length in
the graph.
-
==== Large graphs
Large graphs in TinkerPop require distributed processing by `SparkGraphComputer` to get results in a reasonable time (OLAP
@@ -142,10 +138,8 @@ either with the `gremlin.hadoop.defaultGraphComputer` property or as part of the
Scalability of the the `connectedComponent()`-step with `SparkGraphComputer` is high, but note that:
-* the graph should fit in the memory of the Spark cluster to allow the VertexProgram to run its cycles without spilling
-intermediate results to disk and loosing most of the gains from the distributed processing
-
-* as discussed for small graphs, the BSP algorithm does not play well with graphs having a large shortest path between
+* The graph should fit in the memory of the Spark cluster to allow the VertexProgram to run its cycles without spilling
+intermediate results to disk and loosing most of the gains from the distributed processing.
+* As discussed for small graphs, the BSP algorithm does not play well with graphs having a large shortest path between
any pair of vertices. Overcoming this limitation is still a
-link:http://www.vldb.org/pvldb/vol7/p1821-yan.pdf[subject of academic research].
-
+link:http://www.vldb.org/pvldb/vol7/p1821-yan.pdf[subject of academic research].
\ No newline at end of file
[11/12] tinkerpop git commit: Extended the connected-components recipe
Posted by sp...@apache.org.
Extended the connected-components recipe
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/93231a1d
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/93231a1d
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/93231a1d
Branch: refs/heads/TINKERPOP-1967
Commit: 93231a1d94ede6263b9412e1b1e91491a78c67ad
Parents: d913143
Author: HadoopMarc <vt...@xs4all.nl>
Authored: Sun Jun 10 15:17:17 2018 +0200
Committer: Stephen Mallette <sp...@genoprime.com>
Committed: Fri Jun 15 08:25:01 2018 -0400
----------------------------------------------------------------------
docs/src/recipes/connected-components.asciidoc | 94 ++++++++++++--------
docs/static/images/cc-scale-ratio.png | Bin 0 -> 14393 bytes
docs/static/images/cc-scale-size.png | Bin 0 -> 12220 bytes
3 files changed, 58 insertions(+), 36 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/93231a1d/docs/src/recipes/connected-components.asciidoc
----------------------------------------------------------------------
diff --git a/docs/src/recipes/connected-components.asciidoc b/docs/src/recipes/connected-components.asciidoc
index edbeec5..850c31f 100644
--- a/docs/src/recipes/connected-components.asciidoc
+++ b/docs/src/recipes/connected-components.asciidoc
@@ -31,11 +31,11 @@ Depending on the size of the graph, three solution regimes can be discriminated:
1. Small graphs that fit in the memory of a single machine
-2. Medium graphs backed by storage for which a linear scan is still feasible. This regime is left to third party
+2. Medium-sized graphs backed by storage for which an OLTP linear scan is still feasible. This regime is left to third party
TinkerPop implementations, since TinkerPop itself has no storage-backed reference implementations. The idea is that
component membership is stored in the graph, rather than in memory.
-3. Large graphs requiring an OLAP approach to yield results in a reasonable time.
+3. Large graphs requiring an approach with `HadoopGraph` and `SparkGraphComputer` to yield results in a reasonable time.
These regimes are discussed separately using the following graph with three weakly connected components:
@@ -55,16 +55,21 @@ g.addV().property(id, "A").as("a").
addE("link").from("d").to("e").iterate()
----
+==== Small graph traversals
-===== Small graphs
-
-Connected components in a small graph can be determined with both an OLTP traversal and the OLAP
+Connected components in a small graph can be determined with either an OLTP traversal or the OLAP
`connectedComponent()`-step. The `connectedComponent()`-step is available as of TinkerPop 3.4.0 and is
described in more detail in the
link:http://tinkerpop.apache.org/docs/x.y.z/reference/#connectedcomponent-step[Reference Documentation].
+The traversal looks like:
+[gremlin-groovy,existing]
+----
+g.withComputer().V().connectedComponent().
+ group().by('gremlin.connectedComponentVertexProgram.component').
+ select(values).unfold()
+----
A straightforward way to detect the various subgraphs with an OLTP traversal is to do this:
-
[gremlin-groovy,existing]
----
g.V().emit(cyclicPath().or().not(both())). <1>
@@ -73,7 +78,6 @@ g.V().emit(cyclicPath().or().not(both())). <1
by(path().unfold().dedup().fold()). <4>
select(values).unfold() <5>
----
-
<1> The initial emit() step allows for output of isolated vertices, in addition to the discovery of
components as described in (2).
@@ -83,7 +87,7 @@ path. Collection `'a'` is used to keep track of visited vertices, for both subt
and new traversals resulting from the `g.V()` linear scan.
<3> While `'a'` nicely keeps track of vertices already visited, the actual components need to be extracted from the
-path information of surviving traversers. The `path().unfold().limit(1)` closure provides the starting vertex
+path information. The `path().unfold().limit(1)` closure provides the starting vertex
of surviving traversers, which can be used to group the components.
<4> This clause collects the unique vertices from all paths with the same starting vertex, thus from the same
@@ -91,39 +95,57 @@ weak component.
<5> The values of the groupby map contain the lists of vertices making up the requested components.
-This algorithm completes in linear time with the number of vertices and edges, because a traversal is started for each
-vertex and each edge with its associated out-vertex is visited exactly once.
-
-==== Large graphs
-Large graphs require an OLAP solution with a custom VertexProgram that can be run using a graph implementation's
-GraphComputer, in particular `SparkGraphComputer` on a `HadoopGraph`. The OLAP solution also runs faster for most
-medium-sized graphs, that is when these graph have a 'natural' structure with a limited maximum path length.
+==== Small graph scalability
-The TinkerPop library of vertex programs contains the `WeakComponentsVertexProgram` which can be run in the same
-way as the link:http://tinkerpop.apache.org/docs/x.y.z/reference/#peerpressurevertexprogram[PeerPressureVertexProgram]:
+The scalability of the OLTP traversal and the `connectedComponent()`-step for in-memory graphs is shown in the figures
+below.
-[gremlin-groovy,existing]
-----
-result = g.getGraph().compute().
- program(WeakComponentsVertexProgram.build().maxIterations(100).create()).
- mapReduce(ClusterPopulationMapReduce.build().create()).
- mapReduce(ClusterCountMapReduce.build().create()).
- submit().get()
-result.memory().clusterPopulation
-gResult = result.graph().traversal()
-gResult.V().valueMap(true)
-----
+[[cc-scale-size]]
+.Run times for finding connected components in a randomly generated graph with 10 components of equal size and with an edge/vertex ratio of 6
+image::cc-scale-size.png[width=600, side=bottom]
-The vertex program has interconnected vertices exchange id's and store the lowest id until no vertex receives a
-lower id. This algorithm is commonly applied in
+In general, the `connectedComponent()`-step is almost a factor two faster than the OLTP traversal. Only, for very
+small graphs the overhead of running the ConnectedComponentVertexProgram is larger than that of the OLTP traversal.
+The vertex program works by having interconnected vertices exchange id's and store the lowest id until no vertex
+receives a lower id. This algorithm is commonly applied in
link:https://en.wikipedia.org/wiki/Bulk_synchronous_parallel[bulk synchronous parallel] systems, e.g. in
-link:https://spark.apache.org/graphx[Apache Spark GraphX].
+link:https://spark.apache.org/graphx[Apache Spark GraphX]. Overhead for the vertex program arises because it has to run
+as many cycles as the largest length of the shortest paths between any two vertices in a component of the graph. In
+every cycle each vertex has to be checked for being
+"halted". Overhead of the OLTP traversal consists of each traverser having to carry complete path information. For
+pure depth-first-search or breadth-first-search implementations, connected-component algotithms should scale
+as [.big]##O##(V+E). For the traversals in the figure above this is almost the case.
+
+
+[[cc-scale-ratio]]
+.Run times for finding connected components in a randomly generated graph with 10 components, each consisting of 6400 vertices
+image::cc-scale-ratio.png[width=600]
+
+The random graphs used for the scalability tests can be modulated with the edge/vertex ratio. For small ratios the
+components generated are more lint-like and harder to process by the `connectedComponent()`-step. For high ratios
+the components are more mesh-like and the ConnectedComponentVertexProgram needs few cycles to process the graph. These
+characteristics show clearly from the graph. Indeed, for a given number of vertices, the run time of the
+`connectedComponent()`-step does not depend on the number of edges, but rather on the maximum shortest path length in
+the graph.
+
+
+==== Large graphs
+
+Large graphs in TinkerPop require distributed processing by `SparkGraphComputer` to get results in a reasonable time (OLAP
+approach). This means that the graph must be available as `HadoopGraph` (third party TinkerPop implementations often
+allow to make a graph available as an `HadoopGraph` by providing an Hadoop `InputFormat`). Running the
+`connectedComponent()`-step on
+an `HadoopGraph` works the same as for a small graph, provided that `SparkGraphComputer` is specified as the graph computer,
+either with the `gremlin.hadoop.defaultGraphComputer` property or as part of the `withComputer()`-step.
+
+Scalability of the the `connectedComponent()`-step with `SparkGraphComputer` is high, but note that:
+
+* the graph should fit in the memory of the Spark cluster to allow the VertexProgram to run its cycles without spilling
+intermediate results to disk and loosing most of the gains from the distributed processing
-==== Scalability
+* as discussed for small graphs, the BSP algorithm does not play well with graphs having a large shortest path between
+any pair of vertices. Overcoming this limitation is still a
+link:http://www.vldb.org/pvldb/vol7/p1821-yan.pdf[subject of academic research].
-ToDo:
- - limits and run time regime 1
- - test of friendster graph regime 3
- - discuss: link:http://www.vldb.org/pvldb/vol7/p1821-yan.pdf[http://www.vldb.org/pvldb/vol7/p1821-yan.pdf]
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/93231a1d/docs/static/images/cc-scale-ratio.png
----------------------------------------------------------------------
diff --git a/docs/static/images/cc-scale-ratio.png b/docs/static/images/cc-scale-ratio.png
new file mode 100644
index 0000000..33a842d
Binary files /dev/null and b/docs/static/images/cc-scale-ratio.png differ
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/93231a1d/docs/static/images/cc-scale-size.png
----------------------------------------------------------------------
diff --git a/docs/static/images/cc-scale-size.png b/docs/static/images/cc-scale-size.png
new file mode 100644
index 0000000..2b08a89
Binary files /dev/null and b/docs/static/images/cc-scale-size.png differ
[12/12] tinkerpop git commit: TINKERPOP-1967 Added
connectedComponent() step
Posted by sp...@apache.org.
TINKERPOP-1967 Added connectedComponent() step
Deprecated the recipe for "Connected Components" but left the old content present as I felt it had educational value.
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/607c37ac
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/607c37ac
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/607c37ac
Branch: refs/heads/TINKERPOP-1967
Commit: 607c37ac4c962ec5d1fc784654021d145d2aab6d
Parents: fc86675
Author: Stephen Mallette <sp...@genoprime.com>
Authored: Thu May 17 14:44:01 2018 -0400
Committer: Stephen Mallette <sp...@genoprime.com>
Committed: Fri Jun 15 08:25:01 2018 -0400
----------------------------------------------------------------------
CHANGELOG.asciidoc | 1 +
docs/src/recipes/connected-components.asciidoc | 8 +-
docs/src/reference/the-graphcomputer.asciidoc | 6 +
docs/src/reference/the-traversal.asciidoc | 35 +++
docs/src/upgrade/release-3.4.x.asciidoc | 32 +++
.../tinkerpop/gremlin/jsr223/CoreImports.java | 4 +
.../ConnectedComponentVertexProgram.java | 234 +++++++++++++++++++
.../traversal/step/map/ConnectedComponent.java | 38 +++
.../ConnectedComponentVertexProgramStep.java | 98 ++++++++
.../gremlin/process/remote/RemoteGraph.java | 4 +
.../traversal/dsl/graph/GraphTraversal.java | 15 ++
.../traversal/dsl/graph/GraphTraversalTest.java | 2 +-
.../Process/Traversal/GraphTraversal.cs | 9 +
.../lib/process/graph-traversal.js | 10 +
.../gremlin_python/process/graph_traversal.py | 4 +
.../gremlin/process/ProcessComputerSuite.java | 2 +
.../step/map/ConnectedComponentTest.java | 117 ++++++++++
.../computer/SparkHadoopGraphProvider.java | 2 +
18 files changed, 619 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/607c37ac/CHANGELOG.asciidoc
----------------------------------------------------------------------
diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc
index 2320162..f2e2c5b 100644
--- a/CHANGELOG.asciidoc
+++ b/CHANGELOG.asciidoc
@@ -29,6 +29,7 @@ This release also includes changes from <<release-3-3-3, 3.3.3>>.
* Replaced `Parameterizing.addPropertyMutations()` with `Configuring.configure()`.
* Changed interface hierarchy for `Parameterizing` and `Mutating` interfaces as they are tightly related.
* Introduced the `with()` step modulator which can supply configuration options to `Configuring` steps.
+* Added `connectedComponent()` step and related `VertexProgram`.
* Added `supportsUpsert()` option to `VertexFeatures` and `EdgeFeatures`.
* `min()` and `max()` now support all types implementing `Comparable`.
* Change the `toString()` of `Path` to be standardized as other graph elements are.
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/607c37ac/docs/src/recipes/connected-components.asciidoc
----------------------------------------------------------------------
diff --git a/docs/src/recipes/connected-components.asciidoc b/docs/src/recipes/connected-components.asciidoc
index 46d61eb..70abdbd 100644
--- a/docs/src/recipes/connected-components.asciidoc
+++ b/docs/src/recipes/connected-components.asciidoc
@@ -18,7 +18,13 @@ limitations under the License.
== Connected Components
Gremlin can be used to find link:https://en.wikipedia.org/wiki/Connected_component_(graph_theory)[connected components]
-in a graph. Consider the following graph which has three connected components:
+in a graph. As of TinkerPop 3.4.0, the process has been simplified to the `connectedComponent()`-step which is
+described in more detail in the link:
+link:http://tinkerpop.apache.org/docs/x.y.z/reference/#connectedcomponent-step[Reference Documentation].
+
+The `connectedComponent()`-step replaces the original recipe described below from earlier versions of TinkerPop,
+however the algorithm from that old recipe remains interesting for educational purposes and has thus not been removed.
+The original recipe considers the following graph which has three connected components:
image:connected-components.png[width=600]
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/607c37ac/docs/src/reference/the-graphcomputer.asciidoc
----------------------------------------------------------------------
diff --git a/docs/src/reference/the-graphcomputer.asciidoc b/docs/src/reference/the-graphcomputer.asciidoc
index 4bf39d0..566651b 100644
--- a/docs/src/reference/the-graphcomputer.asciidoc
+++ b/docs/src/reference/the-graphcomputer.asciidoc
@@ -403,6 +403,12 @@ g.V().peerPressure().by('cluster').valueMap()
g.V().peerPressure().by(outE('knows')).by('cluster').valueMap()
----
+[[connectedcomponentvertexprogram]]
+=== ConnectedComponentVertexProgram
+
+The `ConnectedComponentVertexProgram` identifies link:https://en.wikipedia.org/wiki/Connected_component_(graph_theory)[Connected Component]
+instances in a graph. See <<connectedcomponent-step,`connectedComponent()`>>-step for more information.
+
[[bulkdumpervertexprogram]]
=== BulkDumperVertexProgram
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/607c37ac/docs/src/reference/the-traversal.asciidoc
----------------------------------------------------------------------
diff --git a/docs/src/reference/the-traversal.asciidoc b/docs/src/reference/the-traversal.asciidoc
index 0589203..5e3b456 100644
--- a/docs/src/reference/the-traversal.asciidoc
+++ b/docs/src/reference/the-traversal.asciidoc
@@ -608,6 +608,41 @@ g.V().coin(1.0)
link:++http://tinkerpop.apache.org/javadocs/x.y.z/core/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversal.html#coin-double-++[`coin(double)`]
+[[connectedcomponent-step]]
+=== ConnectedComponent Step
+
+The `connectedComponent()` step performs a computation to identify link:https://en.wikipedia.org/wiki/Connected_component_(graph_theory)[Connected Component]
+instances in a graph. When this step completes, the vertices will be labelled with a component identifier to denote
+the component to which they are associated.
+
+IMPORTANT: The `connectedComponent()`-step is a `VertexComputing`-step and as such, can only be used against a graph
+that supports `GraphComputer` (OLAP).
+
+[gremlin-groovy,modern]
+----
+g = graph.traversal().withComputer()
+g.V().
+ connectedComponent().
+ with(ConnectedComponent.propertyName, 'component').
+ project('name','component').
+ by('name').
+ by('component')
+g.V().hasLabel('person').
+ connectedComponent().
+ with(ConnectedComponent.propertyName, 'component').
+ with(ConnectedComponent.edges, outE('knows')).
+ project('name','component').
+ by('name').
+ by('component')
+----
+
+Note the use of the `with()` modulating step which provides configuration options to the algorithm. It takes
+configuration keys from the `ConnectedComponent` class and is automatically imported to the Gremlin Console.
+
+*Additional References*
+
+link:++http://tinkerpop.apache.org/javadocs/x.y.z/core/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversal.html#connectedComponent--++[`connectedComponent()`]
+
[[constant-step]]
=== Constant Step
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/607c37ac/docs/src/upgrade/release-3.4.x.asciidoc
----------------------------------------------------------------------
diff --git a/docs/src/upgrade/release-3.4.x.asciidoc b/docs/src/upgrade/release-3.4.x.asciidoc
index 8829569..7f5ad40 100644
--- a/docs/src/upgrade/release-3.4.x.asciidoc
+++ b/docs/src/upgrade/release-3.4.x.asciidoc
@@ -65,6 +65,38 @@ release where breaking changes are allowed.
See: link:https://issues.apache.org/jira/browse/TINKERPOP-1975[TINKERPOP-1975],
link:http://tinkerpop.apache.org/docs/3.4.0/reference/#with-step[Reference Documentation]
+==== connectedComponent() Step
+
+In prior version of TinkerPop, it was recommended that the identification of
+link:https://en.wikipedia.org/wiki/Connected_component_(graph_theory)[Connected Component] instances in a graph be
+computed by way of a reasonably complex bit of Gremlin that looked something like this:
+
+[source,groovy]
+----
+g.V().emit(cyclicPath().or().not(both())).repeat(both()).until(cyclicPath()).
+ path().aggregate("p").
+ unfold().dedup().
+ map(__.as("v").select("p").unfold().
+ filter(unfold().where(eq("v"))).
+ unfold().dedup().order().by(id).fold()).
+ dedup()
+----
+
+The above approach had a number of drawbacks that included a large execution cost as well as incompatibilities in OLAP.
+To simplify usage of this commonly use graph algorithm, TinkerPop 3.4.0 introduces the `connectedComponent()` step
+which reduces the above operation to:
+
+[source,groovy]
+----
+g.withComputer().V().connectedComponent()
+----
+
+It is important to note that this step does require the use of a `GraphComputer` to work, as it utilizes a
+`VertexProgram` behind the scenes.
+
+See: link:https://issues.apache.org/jira/browse/TINKERPOP-1967[TINKERPOP-1967],
+link:http://tinkerpop.apache.org/docs/x.y.z/reference/#connectedcomponent-step[Reference Documentation]
+
==== Removal of Giraph Support
Support for Giraph has been removed as of this version. There were a number of reasons for this decision which were
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/607c37ac/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/CoreImports.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/CoreImports.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/CoreImports.java
index 558d6ca..4d90c76 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/CoreImports.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/CoreImports.java
@@ -41,6 +41,7 @@ import org.apache.tinkerpop.gremlin.process.computer.bulkloading.BulkLoader;
import org.apache.tinkerpop.gremlin.process.computer.bulkloading.BulkLoaderVertexProgram;
import org.apache.tinkerpop.gremlin.process.computer.bulkloading.IncrementalBulkLoader;
import org.apache.tinkerpop.gremlin.process.computer.bulkloading.OneTimeBulkLoader;
+import org.apache.tinkerpop.gremlin.process.computer.clustering.connected.ConnectedComponentVertexProgram;
import org.apache.tinkerpop.gremlin.process.computer.clustering.peerpressure.ClusterCountMapReduce;
import org.apache.tinkerpop.gremlin.process.computer.clustering.peerpressure.ClusterPopulationMapReduce;
import org.apache.tinkerpop.gremlin.process.computer.clustering.peerpressure.PeerPressureVertexProgram;
@@ -48,6 +49,7 @@ import org.apache.tinkerpop.gremlin.process.computer.ranking.pagerank.PageRankMa
import org.apache.tinkerpop.gremlin.process.computer.ranking.pagerank.PageRankVertexProgram;
import org.apache.tinkerpop.gremlin.process.computer.traversal.MemoryTraversalSideEffects;
import org.apache.tinkerpop.gremlin.process.computer.traversal.TraversalVertexProgram;
+import org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.ConnectedComponent;
import org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.PageRank;
import org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.PeerPressure;
import org.apache.tinkerpop.gremlin.process.computer.traversal.strategy.decoration.VertexProgramStrategy;
@@ -248,6 +250,8 @@ public final class CoreImports {
// graph computer
CLASS_IMPORTS.add(Computer.class);
CLASS_IMPORTS.add(ComputerResult.class);
+ CLASS_IMPORTS.add(ConnectedComponent.class);
+ CLASS_IMPORTS.add(ConnectedComponentVertexProgram.class);
CLASS_IMPORTS.add(GraphComputer.class);
CLASS_IMPORTS.add(Memory.class);
CLASS_IMPORTS.add(VertexProgram.class);
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/607c37ac/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/clustering/connected/ConnectedComponentVertexProgram.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/clustering/connected/ConnectedComponentVertexProgram.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/clustering/connected/ConnectedComponentVertexProgram.java
new file mode 100644
index 0000000..de718f1
--- /dev/null
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/clustering/connected/ConnectedComponentVertexProgram.java
@@ -0,0 +1,234 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.gremlin.process.computer.clustering.connected;
+
+import org.apache.commons.configuration.BaseConfiguration;
+import org.apache.commons.configuration.Configuration;
+import org.apache.commons.configuration.ConfigurationUtils;
+import org.apache.tinkerpop.gremlin.process.computer.GraphComputer;
+import org.apache.tinkerpop.gremlin.process.computer.Memory;
+import org.apache.tinkerpop.gremlin.process.computer.MemoryComputeKey;
+import org.apache.tinkerpop.gremlin.process.computer.MessageScope;
+import org.apache.tinkerpop.gremlin.process.computer.Messenger;
+import org.apache.tinkerpop.gremlin.process.computer.VertexComputeKey;
+import org.apache.tinkerpop.gremlin.process.computer.VertexProgram;
+import org.apache.tinkerpop.gremlin.process.computer.traversal.TraversalVertexProgram;
+import org.apache.tinkerpop.gremlin.process.computer.util.AbstractVertexProgramBuilder;
+import org.apache.tinkerpop.gremlin.process.traversal.Operator;
+import org.apache.tinkerpop.gremlin.process.traversal.Traversal;
+import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__;
+import org.apache.tinkerpop.gremlin.process.traversal.util.PureTraversal;
+import org.apache.tinkerpop.gremlin.structure.Direction;
+import org.apache.tinkerpop.gremlin.structure.Edge;
+import org.apache.tinkerpop.gremlin.structure.Graph;
+import org.apache.tinkerpop.gremlin.structure.Vertex;
+import org.apache.tinkerpop.gremlin.structure.VertexProperty;
+
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.Set;
+
+/**
+ * Identifies "Connected Component" instances in a graph by assigning a component identifier (the lexicographically
+ * least string value of the vertex in the component) to each vertex.
+ *
+ * @author Stephen Mallette (http://stephen.genoprime.com)
+ * @author Daniel Kuppitz (http://gremlin.guru)
+ */
+public class ConnectedComponentVertexProgram implements VertexProgram<String> {
+
+ public static final String COMPONENT = "gremlin.connectedComponentVertexProgram.component";
+ private static final String PROPERTY = "gremlin.connectedComponentVertexProgram.property";
+ private static final String EDGE_TRAVERSAL = "gremlin.pageRankVertexProgram.edgeTraversal";
+ private static final String HAS_HALTED = "gremlin.connectedComponentVertexProgram.hasHalted";
+ private static final String VOTE_TO_HALT = "gremlin.connectedComponentVertexProgram.voteToHalt";
+
+ private static final Set<MemoryComputeKey> MEMORY_COMPUTE_KEYS = Collections.singleton(MemoryComputeKey.of(VOTE_TO_HALT, Operator.and, false, true));
+ private MessageScope.Local<?> scope = MessageScope.Local.of(__::bothE);
+ private Set<MessageScope> scopes;
+ private String property = COMPONENT;
+ private boolean hasHalted = false;
+ private PureTraversal<Vertex, Edge> edgeTraversal = null;
+ private Configuration configuration;
+
+ private ConnectedComponentVertexProgram() {}
+
+ @Override
+ public void loadState(final Graph graph, final Configuration config) {
+ configuration = new BaseConfiguration();
+ if (config != null) {
+ ConfigurationUtils.copy(config, configuration);
+ }
+
+ if (configuration.containsKey(EDGE_TRAVERSAL)) {
+ this.edgeTraversal = PureTraversal.loadState(configuration, EDGE_TRAVERSAL, graph);
+ this.scope = MessageScope.Local.of(() -> this.edgeTraversal.get().clone());
+ }
+
+ scopes = new HashSet<>(Collections.singletonList(scope));
+
+ this.property = configuration.getString(PROPERTY, COMPONENT);
+ this.hasHalted = configuration.getBoolean(HAS_HALTED, false);
+ }
+
+ @Override
+ public void storeState(final Configuration config) {
+ VertexProgram.super.storeState(config);
+ if (configuration != null) {
+ ConfigurationUtils.copy(configuration, config);
+ }
+ }
+
+ @Override
+ public void setup(final Memory memory) {
+ memory.set(VOTE_TO_HALT, true);
+ }
+
+ @Override
+ public void execute(final Vertex vertex, final Messenger<String> messenger, final Memory memory) {
+ if (memory.isInitialIteration()) {
+ // on the first pass, just initialize the component to its own id then pass it to all adjacent vertices
+ // for evaluation
+ vertex.property(VertexProperty.Cardinality.single, property, vertex.id().toString());
+
+ // no need to send messages from traversers that were filtered - happens for stuff like
+ // g.V().hasLabel('software').connectedComponent() (i.e. when there is a TraversalVertexProgram in the mix
+ // halting traversers. only want to send messages from traversers that are still hanging about after
+ // the filter. the unfiltered vertices can only react to messages sent to them. of course, this can
+ // lead to weirdness in results.
+ if (vertex.edges(Direction.BOTH).hasNext() && !(hasHalted && !vertex.property(TraversalVertexProgram.HALTED_TRAVERSERS).isPresent())) {
+ // since there was message passing we don't want to halt on the first round. this should only trigger
+ // a single pass finish if the graph is completely disconnected (technically, it won't even really
+ // work in cases where halted traversers come into play
+ messenger.sendMessage(scope, vertex.id().toString());
+ memory.add(VOTE_TO_HALT, false);
+ }
+ } else {
+ // by the second iteration all vertices that matter should have a component assigned
+ String currentComponent = vertex.value(property);
+ boolean different = false;
+
+ // iterate through messages received and determine if there is a component that has a lesser value than
+ // the currently assigned one
+ final Iterator<String> componentIterator = messenger.receiveMessages();
+ while(componentIterator.hasNext()) {
+ final String candidateComponent = componentIterator.next();
+ if (candidateComponent.compareTo(currentComponent) < 0) {
+ currentComponent = candidateComponent;
+ different = true;
+ }
+ }
+
+ // if there is a better component then assign it and notify adjacent vertices. triggering the message
+ // passing should not halt future executions
+ if (different) {
+ vertex.property(VertexProperty.Cardinality.single, property, currentComponent);
+ messenger.sendMessage(scope, currentComponent);
+ memory.add(VOTE_TO_HALT, false);
+ }
+ }
+ }
+
+ @Override
+ public Set<VertexComputeKey> getVertexComputeKeys() {
+ return new HashSet<>(Collections.singletonList(VertexComputeKey.of(property, false)));
+ }
+
+ @Override
+ public Set<MemoryComputeKey> getMemoryComputeKeys() {
+ return MEMORY_COMPUTE_KEYS;
+ }
+
+ @Override
+ public boolean terminate(final Memory memory) {
+ final boolean voteToHalt = memory.<Boolean>get(VOTE_TO_HALT);
+ if (voteToHalt) {
+ return true;
+ } else {
+ // it is basically always assumed that the program will want to halt, but if message passing occurs, the
+ // program will want to continue, thus reset false values to true for future iterations
+ memory.set(VOTE_TO_HALT, true);
+ return false;
+ }
+ }
+
+ @Override
+ public Set<MessageScope> getMessageScopes(final Memory memory) {
+ return scopes;
+ }
+
+ @Override
+ public GraphComputer.ResultGraph getPreferredResultGraph() {
+ return GraphComputer.ResultGraph.NEW;
+ }
+
+ @Override
+ public GraphComputer.Persist getPreferredPersist() {
+ return GraphComputer.Persist.VERTEX_PROPERTIES;
+ }
+
+
+ @Override
+ @SuppressWarnings("CloneDoesntCallSuperClone,CloneDoesntDeclareCloneNotSupportedException")
+ public ConnectedComponentVertexProgram clone() {
+ return this;
+ }
+
+ @Override
+ public Features getFeatures() {
+ return new Features() {
+ @Override
+ public boolean requiresLocalMessageScopes() {
+ return true;
+ }
+
+ @Override
+ public boolean requiresVertexPropertyAddition() {
+ return true;
+ }
+ };
+ }
+
+ public static ConnectedComponentVertexProgram.Builder build() {
+ return new ConnectedComponentVertexProgram.Builder();
+ }
+
+ public static final class Builder extends AbstractVertexProgramBuilder<ConnectedComponentVertexProgram.Builder> {
+
+ private Builder() {
+ super(ConnectedComponentVertexProgram.class);
+ }
+
+ public ConnectedComponentVertexProgram.Builder hasHalted(final boolean hasHalted) {
+ this.configuration.setProperty(HAS_HALTED, hasHalted);
+ return this;
+ }
+
+ public ConnectedComponentVertexProgram.Builder edges(final Traversal.Admin<Vertex, Edge> edgeTraversal) {
+ PureTraversal.storeState(this.configuration, EDGE_TRAVERSAL, edgeTraversal);
+ return this;
+ }
+
+ public ConnectedComponentVertexProgram.Builder property(final String key) {
+ this.configuration.setProperty(PROPERTY, key);
+ return this;
+ }
+ }
+}
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/607c37ac/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/step/map/ConnectedComponent.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/step/map/ConnectedComponent.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/step/map/ConnectedComponent.java
new file mode 100644
index 0000000..85558bc
--- /dev/null
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/step/map/ConnectedComponent.java
@@ -0,0 +1,38 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.gremlin.process.computer.traversal.step.map;
+
+import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal;
+import org.apache.tinkerpop.gremlin.structure.Graph;
+
+/**
+ * Configuration options to be passed to the {@link GraphTraversal#with(String, Object)} step on
+ * {@link GraphTraversal#connectedComponent()} ()}.
+ */
+public class ConnectedComponent {
+ /**
+ * Configures the edge to traverse when calculating the pagerank.
+ */
+ public static final String edges = Graph.Hidden.hide("tinkerpop.connectedComponent.edges");
+
+ /**
+ * Configures the name of the property within which to store the pagerank value.
+ */
+ public static final String propertyName = Graph.Hidden.hide("tinkerpop.connectedComponent.propertyName");
+}
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/607c37ac/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/step/map/ConnectedComponentVertexProgramStep.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/step/map/ConnectedComponentVertexProgramStep.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/step/map/ConnectedComponentVertexProgramStep.java
new file mode 100644
index 0000000..edeb497
--- /dev/null
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/step/map/ConnectedComponentVertexProgramStep.java
@@ -0,0 +1,98 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.tinkerpop.gremlin.process.computer.traversal.step.map;
+
+import org.apache.tinkerpop.gremlin.process.computer.GraphFilter;
+import org.apache.tinkerpop.gremlin.process.computer.Memory;
+import org.apache.tinkerpop.gremlin.process.computer.clustering.connected.ConnectedComponentVertexProgram;
+import org.apache.tinkerpop.gremlin.process.computer.traversal.TraversalVertexProgram;
+import org.apache.tinkerpop.gremlin.process.traversal.Traversal;
+import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategies;
+import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__;
+import org.apache.tinkerpop.gremlin.process.traversal.step.Configuring;
+import org.apache.tinkerpop.gremlin.process.traversal.step.TraversalParent;
+import org.apache.tinkerpop.gremlin.process.traversal.step.util.Parameters;
+import org.apache.tinkerpop.gremlin.process.traversal.util.PureTraversal;
+import org.apache.tinkerpop.gremlin.structure.Edge;
+import org.apache.tinkerpop.gremlin.structure.Graph;
+import org.apache.tinkerpop.gremlin.structure.Vertex;
+import org.apache.tinkerpop.gremlin.structure.util.StringFactory;
+
+/**
+ * @author Stephen Mallette (http://stephen.genoprime.com)
+ */
+public final class ConnectedComponentVertexProgramStep extends VertexProgramStep implements TraversalParent, Configuring {
+
+ private Parameters parameters = new Parameters();
+ private PureTraversal<Vertex, Edge> edgeTraversal;
+ private String clusterProperty = ConnectedComponentVertexProgram.COMPONENT;
+
+ public ConnectedComponentVertexProgramStep(final Traversal.Admin traversal) {
+ super(traversal);
+ this.configure(ConnectedComponent.edges, __.<Vertex>bothE());
+ }
+
+ @Override
+ public void configure(final Object... keyValues) {
+ if (keyValues[0].equals(ConnectedComponent.edges)) {
+ if (!(keyValues[1] instanceof Traversal))
+ throw new IllegalArgumentException("ConnectedComponent.edges requires a Traversal as its argument");
+ this.edgeTraversal = new PureTraversal<>(((Traversal<Vertex,Edge>) keyValues[1]).asAdmin());
+ this.integrateChild(this.edgeTraversal.get());
+ } else if (keyValues[0].equals(ConnectedComponent.propertyName)) {
+ if (!(keyValues[1] instanceof String))
+ throw new IllegalArgumentException("ConnectedComponent.propertyName requires a String as its argument");
+ this.clusterProperty = (String) keyValues[1];
+ } else {
+ this.parameters.set(this, keyValues);
+ }
+ }
+
+ @Override
+ public Parameters getParameters() {
+ return parameters;
+ }
+
+ @Override
+ public int hashCode() {
+ return super.hashCode() ^ this.clusterProperty.hashCode();
+ }
+
+ @Override
+ public String toString() {
+ return StringFactory.stepString(this, this.clusterProperty, new GraphFilter(this.computer));
+ }
+
+ @Override
+ public ConnectedComponentVertexProgram generateProgram(final Graph graph, final Memory memory) {
+ final Traversal.Admin<Vertex, Edge> detachedTraversal = this.edgeTraversal.getPure();
+ detachedTraversal.setStrategies(TraversalStrategies.GlobalCache.getStrategies(graph.getClass()));
+ return ConnectedComponentVertexProgram.build().
+ hasHalted(memory.exists(TraversalVertexProgram.HALTED_TRAVERSERS)).
+ edges(detachedTraversal).
+ property(this.clusterProperty).create(graph);
+ }
+
+ @Override
+ public ConnectedComponentVertexProgramStep clone() {
+ return (ConnectedComponentVertexProgramStep) super.clone();
+ }
+
+}
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/607c37ac/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/remote/RemoteGraph.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/remote/RemoteGraph.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/remote/RemoteGraph.java
index da12114..d2093e4 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/remote/RemoteGraph.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/remote/RemoteGraph.java
@@ -57,6 +57,10 @@ import java.util.Iterator;
method = "*",
reason = "hmmmm")
@Graph.OptOut(
+ test = "org.apache.tinkerpop.gremlin.process.traversal.step.map.ConnectedComponentTest",
+ method = "*",
+ reason = "hmmmm")
+@Graph.OptOut(
test = "org.apache.tinkerpop.gremlin.process.traversal.step.map.PageRankTest",
method = "*",
reason = "hmmmm")
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/607c37ac/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversal.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversal.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversal.java
index 7d1e7e4..6fbe61d 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversal.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversal.java
@@ -19,6 +19,8 @@
package org.apache.tinkerpop.gremlin.process.traversal.dsl.graph;
import org.apache.tinkerpop.gremlin.process.computer.VertexProgram;
+import org.apache.tinkerpop.gremlin.process.computer.clustering.connected.ConnectedComponentVertexProgram;
+import org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.ConnectedComponentVertexProgramStep;
import org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.PageRankVertexProgramStep;
import org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.PeerPressureVertexProgramStep;
import org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.ProgramVertexProgramStep;
@@ -2422,6 +2424,18 @@ public interface GraphTraversal<S, E> extends Traversal<S, E> {
}
/**
+ * Executes a Connected Component algorithm over the graph.
+ *
+ * @return the traversal with the appended {@link ConnectedComponentVertexProgram}
+ * @see <a href="http://tinkerpop.apache.org/docs/${project.version}/reference/#connectedcomponent-step" target="_blank">Reference Documentation - ConnectedComponent Step</a>
+ * @since 3.4.0
+ */
+ public default GraphTraversal<S, E> connectedComponent() {
+ this.asAdmin().getBytecode().addStep(Symbols.connectedComponent);
+ return this.asAdmin().addStep((Step<E, E>) new ConnectedComponentVertexProgramStep(this.asAdmin()));
+ }
+
+ /**
* Executes an arbitrary {@link VertexProgram} over the graph.
*
* @return the traversal with the appended {@link ProgramVertexProgramStep}
@@ -2817,6 +2831,7 @@ public interface GraphTraversal<S, E> extends Traversal<S, E> {
public static final String pageRank = "pageRank";
public static final String peerPressure = "peerPressure";
+ public static final String connectedComponent = "connectedComponent";
public static final String program = "program";
public static final String by = "by";
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/607c37ac/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversalTest.java
----------------------------------------------------------------------
diff --git a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversalTest.java b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversalTest.java
index 9009d0b..7ef9027 100644
--- a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversalTest.java
+++ b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversalTest.java
@@ -42,7 +42,7 @@ import static org.junit.Assert.assertEquals;
public class GraphTraversalTest {
private static final Logger logger = LoggerFactory.getLogger(GraphTraversalTest.class);
- private static Set<String> NO_GRAPH = new HashSet<>(Arrays.asList("asAdmin", "by", "with", "option", "iterate", "to", "from", "profile", "pageRank", "peerPressure", "program", "none"));
+ private static Set<String> NO_GRAPH = new HashSet<>(Arrays.asList("asAdmin", "by", "with", "option", "iterate", "to", "from", "profile", "pageRank", "connectedComponent", "peerPressure", "program", "none"));
private static Set<String> NO_ANONYMOUS = new HashSet<>(Arrays.asList("start", "__"));
private static Set<String> IGNORES_BYTECODE = new HashSet<>(Arrays.asList("asAdmin", "iterate", "mapValues", "mapKeys"));
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/607c37ac/gremlin-dotnet/src/Gremlin.Net/Process/Traversal/GraphTraversal.cs
----------------------------------------------------------------------
diff --git a/gremlin-dotnet/src/Gremlin.Net/Process/Traversal/GraphTraversal.cs b/gremlin-dotnet/src/Gremlin.Net/Process/Traversal/GraphTraversal.cs
index 537cdbe..a1936a7 100644
--- a/gremlin-dotnet/src/Gremlin.Net/Process/Traversal/GraphTraversal.cs
+++ b/gremlin-dotnet/src/Gremlin.Net/Process/Traversal/GraphTraversal.cs
@@ -401,6 +401,15 @@ namespace Gremlin.Net.Process.Traversal
}
/// <summary>
+ /// Adds the connectedComponent step to this <see cref="GraphTraversal{SType, EType}" />.
+ /// </summary>
+ public GraphTraversal<S, E> ConnectedComponent ()
+ {
+ Bytecode.AddStep("connectedComponent");
+ return Wrap<S, E>(this);
+ }
+
+ /// <summary>
/// Adds the constant step to this <see cref="GraphTraversal{SType, EType}" />.
/// </summary>
public GraphTraversal<S, E2> Constant<E2> (E2 e)
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/607c37ac/gremlin-javascript/src/main/javascript/gremlin-javascript/lib/process/graph-traversal.js
----------------------------------------------------------------------
diff --git a/gremlin-javascript/src/main/javascript/gremlin-javascript/lib/process/graph-traversal.js b/gremlin-javascript/src/main/javascript/gremlin-javascript/lib/process/graph-traversal.js
index f143542..b18f515 100644
--- a/gremlin-javascript/src/main/javascript/gremlin-javascript/lib/process/graph-traversal.js
+++ b/gremlin-javascript/src/main/javascript/gremlin-javascript/lib/process/graph-traversal.js
@@ -343,6 +343,16 @@ class GraphTraversal extends Traversal {
}
/**
+ * Graph traversal connectedComponent method.
+ * @param {...Object} args
+ * @returns {GraphTraversal}
+ */
+ connectedComponent(...args) {
+ this.bytecode.addStep('connectedComponent', args);
+ return this;
+ }
+
+ /**
* Graph traversal constant method.
* @param {...Object} args
* @returns {GraphTraversal}
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/607c37ac/gremlin-python/src/main/jython/gremlin_python/process/graph_traversal.py
----------------------------------------------------------------------
diff --git a/gremlin-python/src/main/jython/gremlin_python/process/graph_traversal.py b/gremlin-python/src/main/jython/gremlin_python/process/graph_traversal.py
index bb81d87..2410366 100644
--- a/gremlin-python/src/main/jython/gremlin_python/process/graph_traversal.py
+++ b/gremlin-python/src/main/jython/gremlin_python/process/graph_traversal.py
@@ -189,6 +189,10 @@ class GraphTraversal(Traversal):
self.bytecode.add_step("coin", *args)
return self
+ def connectedComponent(self, *args):
+ self.bytecode.add_step("connectedComponent", *args)
+ return self
+
def constant(self, *args):
self.bytecode.add_step("constant", *args)
return self
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/607c37ac/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/ProcessComputerSuite.java
----------------------------------------------------------------------
diff --git a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/ProcessComputerSuite.java b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/ProcessComputerSuite.java
index 3d51a94..c19d4c2 100644
--- a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/ProcessComputerSuite.java
+++ b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/ProcessComputerSuite.java
@@ -49,6 +49,7 @@ import org.apache.tinkerpop.gremlin.process.traversal.step.filter.TailTest;
import org.apache.tinkerpop.gremlin.process.traversal.step.filter.WhereTest;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.AddEdgeTest;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.CoalesceTest;
+import org.apache.tinkerpop.gremlin.process.traversal.step.map.ConnectedComponentTest;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.ConstantTest;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.CountTest;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.FlatMapTest;
@@ -140,6 +141,7 @@ public class ProcessComputerSuite extends AbstractGremlinSuite {
// map
CoalesceTest.Traversals.class,
+ ConnectedComponentTest.Traversals.class,
ConstantTest.Traversals.class,
CountTest.Traversals.class,
FlatMapTest.Traversals.class,
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/607c37ac/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/ConnectedComponentTest.java
----------------------------------------------------------------------
diff --git a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/ConnectedComponentTest.java b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/ConnectedComponentTest.java
new file mode 100644
index 0000000..25e618a
--- /dev/null
+++ b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/ConnectedComponentTest.java
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.gremlin.process.traversal.step.map;
+
+import org.apache.tinkerpop.gremlin.LoadGraphWith;
+import org.apache.tinkerpop.gremlin.process.computer.clustering.connected.ConnectedComponentVertexProgram;
+import org.apache.tinkerpop.gremlin.process.AbstractGremlinProcessTest;
+import org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.ConnectedComponent;
+import org.apache.tinkerpop.gremlin.process.traversal.Traversal;
+import org.apache.tinkerpop.gremlin.structure.Vertex;
+import org.junit.Test;
+
+import static org.apache.tinkerpop.gremlin.LoadGraphWith.GraphData.MODERN;
+import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.bothE;
+import static org.junit.Assert.assertEquals;
+
+/**
+ * @author Stephen Mallette (http://stephen.genoprime.com)
+ */
+public abstract class ConnectedComponentTest extends AbstractGremlinProcessTest {
+
+ public abstract Traversal<Vertex, Vertex> get_g_V_connectedComponent();
+
+ public abstract Traversal<Vertex, Vertex> get_g_V_hasLabelXsoftwareX_connectedComponent();
+
+ public abstract Traversal<Vertex, Vertex> get_g_V_connectedComponent_withXedges_bothEXknowsXX_withXpropertyName_clusterX();
+
+ @Test
+ @LoadGraphWith(MODERN)
+ public void g_V_connectedComponent() {
+ final Traversal<Vertex, Vertex> traversal = get_g_V_connectedComponent();
+ printTraversalForm(traversal);
+ int counter = 0;
+ while (traversal.hasNext()) {
+ final Vertex vertex = traversal.next();
+ counter++;
+ assertEquals("1", vertex.value(ConnectedComponentVertexProgram.COMPONENT));
+ }
+ assertEquals(6, counter);
+ }
+
+ @Test
+ @LoadGraphWith(MODERN)
+ public void g_V_hasLabelXsoftwareX_connectedComponent() {
+ final Traversal<Vertex, Vertex> traversal = get_g_V_hasLabelXsoftwareX_connectedComponent();
+ printTraversalForm(traversal);
+ int counter = 0;
+ while (traversal.hasNext()) {
+ final Vertex vertex = traversal.next();
+ final String name = vertex.value("name");
+ switch (name) {
+ case "lop":
+ case "ripple":
+ assertEquals("3", vertex.value(ConnectedComponentVertexProgram.COMPONENT));
+ break;
+ }
+ counter++;
+ }
+ assertEquals(2, counter);
+ }
+
+ @Test
+ @LoadGraphWith(MODERN)
+ public void g_V_connectedComponent_withXEDGES_bothEXknowsXX_withXPROPERTY_NAME_clusterX() {
+ final Traversal<Vertex, Vertex> traversal = get_g_V_connectedComponent_withXedges_bothEXknowsXX_withXpropertyName_clusterX();
+ printTraversalForm(traversal);
+ int counter = 0;
+ while (traversal.hasNext()) {
+ final Vertex vertex = traversal.next();
+ final String name = vertex.value("name");
+ switch (name) {
+ case "marko":
+ case "vadas":
+ case "josh":
+ assertEquals("1", vertex.value("cluster"));
+ break;
+ case "peter":
+ assertEquals("6", vertex.value("cluster"));
+ break;
+ }
+ counter++;
+ }
+ assertEquals(4, counter);
+ }
+
+ public static class Traversals extends ConnectedComponentTest {
+ @Override
+ public Traversal<Vertex, Vertex> get_g_V_connectedComponent() {
+ return g.V().connectedComponent();
+ }
+
+ @Override
+ public Traversal<Vertex, Vertex> get_g_V_hasLabelXsoftwareX_connectedComponent() {
+ return g.V().hasLabel("software").connectedComponent();
+ }
+ @Override
+ public Traversal<Vertex, Vertex> get_g_V_connectedComponent_withXedges_bothEXknowsXX_withXpropertyName_clusterX() {
+ return g.V().hasLabel("person").connectedComponent().with(ConnectedComponent.edges, bothE("knows")).with(ConnectedComponent.propertyName, "cluster");
+ }
+ }
+}
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/607c37ac/spark-gremlin/src/test/java/org/apache/tinkerpop/gremlin/spark/process/computer/SparkHadoopGraphProvider.java
----------------------------------------------------------------------
diff --git a/spark-gremlin/src/test/java/org/apache/tinkerpop/gremlin/spark/process/computer/SparkHadoopGraphProvider.java b/spark-gremlin/src/test/java/org/apache/tinkerpop/gremlin/spark/process/computer/SparkHadoopGraphProvider.java
index c778c6d..2f727c8 100644
--- a/spark-gremlin/src/test/java/org/apache/tinkerpop/gremlin/spark/process/computer/SparkHadoopGraphProvider.java
+++ b/spark-gremlin/src/test/java/org/apache/tinkerpop/gremlin/spark/process/computer/SparkHadoopGraphProvider.java
@@ -39,6 +39,7 @@ import org.apache.tinkerpop.gremlin.process.computer.Computer;
import org.apache.tinkerpop.gremlin.process.computer.GraphComputer;
import org.apache.tinkerpop.gremlin.process.computer.util.ComputerGraph;
import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource;
+import org.apache.tinkerpop.gremlin.process.traversal.step.map.ConnectedComponentTest;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.PageRankTest;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.PeerPressureTest;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.ProgramTest;
@@ -115,6 +116,7 @@ public class SparkHadoopGraphProvider extends AbstractFileGraphProvider {
if (null != loadGraphWith &&
!test.equals(ProgramTest.Traversals.class) &&
!test.equals(PageRankTest.Traversals.class) &&
+ !test.equals(ConnectedComponentTest.Traversals.class) &&
!test.equals(PeerPressureTest.Traversals.class) &&
!test.equals(FileSystemStorageCheck.class) &&
!testMethodName.equals("shouldSupportJobChaining") && // GraphComputerTest.shouldSupportJobChaining
[05/12] tinkerpop git commit: Removed a non-link in docs CTR
Posted by sp...@apache.org.
Removed a non-link in docs CTR
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/89c23bd9
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/89c23bd9
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/89c23bd9
Branch: refs/heads/TINKERPOP-1967
Commit: 89c23bd9c1d925e0b4e2e6d20d77301b723a506d
Parents: ad19a11
Author: Stephen Mallette <sp...@genoprime.com>
Authored: Wed Jun 13 05:48:28 2018 -0400
Committer: Stephen Mallette <sp...@genoprime.com>
Committed: Wed Jun 13 05:48:28 2018 -0400
----------------------------------------------------------------------
docs/src/upgrade/release-3.4.x.asciidoc | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/89c23bd9/docs/src/upgrade/release-3.4.x.asciidoc
----------------------------------------------------------------------
diff --git a/docs/src/upgrade/release-3.4.x.asciidoc b/docs/src/upgrade/release-3.4.x.asciidoc
index 5035831..d44f69d 100644
--- a/docs/src/upgrade/release-3.4.x.asciidoc
+++ b/docs/src/upgrade/release-3.4.x.asciidoc
@@ -63,7 +63,7 @@ Note that the `by()` modulators still work, but should be considered deprecated
release where breaking changes are allowed.
See: link:https://issues.apache.org/jira/browse/TINKERPOP-1975[TINKERPOP-1975],
-link:http://tinkerpop.apache.org/docs/current/reference/#with-step[Reference Documentation]
+link:http://tinkerpop.apache.org/docs/3.4.0/reference/#with-step[Reference Documentation]
==== Removal of Giraph Support
@@ -258,8 +258,7 @@ This method can be implemented to return a cacheable set of features for a `Grap
Assuming this method is faster than the cost of creating a new `Graph` instance, the test suite should execute
significantly faster depending on how many tests end up being ignored.
-See: link:https://issues.apache.org/jira/browse/TINKERPOP-1518[TINKERPOP-1518],
-link:
+See: link:https://issues.apache.org/jira/browse/TINKERPOP-1518[TINKERPOP-1518]
===== Configuring Interface