You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tinkerpop.apache.org by sp...@apache.org on 2016/05/18 13:11:14 UTC
[07/16] incubator-tinkerpop git commit: Updated the gephi plugin to
work on Gephi 0.9.x
Updated the gephi plugin to work on Gephi 0.9.x
Improved output of HTTP request errors to gephi. It is likely that this change will not allow Gephi to work with 0.8.x anymore.
Project: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/commit/82899ec3
Tree: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/tree/82899ec3
Diff: http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/diff/82899ec3
Branch: refs/heads/TINKERPOP-1274
Commit: 82899ec3ec5cce9db367570e50ce5226e4b51155
Parents: efe0360
Author: Stephen Mallette <sp...@genoprime.com>
Authored: Tue May 10 12:55:32 2016 -0400
Committer: Stephen Mallette <sp...@genoprime.com>
Committed: Fri May 13 08:59:49 2016 -0400
----------------------------------------------------------------------
CHANGELOG.asciidoc | 1 +
.../src/reference/gremlin-applications.asciidoc | 31 +++++----
.../upgrade/release-3.2.x-incubating.asciidoc | 8 +++
.../console/plugin/GephiRemoteAcceptor.groovy | 71 +++++++-------------
.../GephiRemoteAcceptorIntegrateTest.java | 26 +++----
5 files changed, 64 insertions(+), 73 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/82899ec3/CHANGELOG.asciidoc
----------------------------------------------------------------------
diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc
index fcd0a1b..8a5c54f 100644
--- a/CHANGELOG.asciidoc
+++ b/CHANGELOG.asciidoc
@@ -30,6 +30,7 @@ TinkerPop 3.2.1 (NOT OFFICIALLY RELEASED YET)
* Added tests to ensure that threaded transactions cannot be re-used.
* `GraphFilter` helper methods are now more intelligent when determining edge direction/label legality.
* Added `GraphFilterStrategy` to automatically construct `GraphFilters` via traversal introspection in OLAP.
+* Updated the Gephi Plugin to support Gephi 0.9.x.
* Increased the testing and scope of `TraversalHelper.isLocalStarGraph()`.
* Changed signature of `get_g_VXlistXv1_v2_v3XX_name` and `get_g_VXlistX1_2_3XX_name` of `VertexTest` to take arguments for the `Traversal` to be constructed by extending classes.
* Added `VertexProgramInterceptor` interface as a general pattern for `GraphComputer` providers to use for bypassing `GraphComputer` semantics where appropriate.
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/82899ec3/docs/src/reference/gremlin-applications.asciidoc
----------------------------------------------------------------------
diff --git a/docs/src/reference/gremlin-applications.asciidoc b/docs/src/reference/gremlin-applications.asciidoc
index ea79621..ab4ea71 100644
--- a/docs/src/reference/gremlin-applications.asciidoc
+++ b/docs/src/reference/gremlin-applications.asciidoc
@@ -1630,18 +1630,18 @@ This plugin imports the appropriate classes for managing the credentials graph.
Gephi Plugin
~~~~~~~~~~~~
-image:gephi-logo.png[width=200, float=left] link:http://gephi.github.io/[Gephi] is an interactive visualization,
-exploration, and analysis platform for graphs. The link:https://marketplace.gephi.org/plugin/graph-streaming/[Graph Streaming]
-plugin for Gephi provides an link:https://wiki.gephi.org/index.php/Graph_Streaming[API] that can be leveraged to
-stream graphs and visualize traversals interactively through the Gremlin Gephi Plugin.
+image:gephi-logo.png[width=200, float=left] link:http://gephi.org/[Gephi] is an interactive visualization,
+exploration, and analysis platform for graphs. The link:https://gephi.org/plugins/#/plugin/graphstreaming[Graph Streaming]
+plugin for Gephi provides an API that can be leveraged to stream graph data to a running Gephi application. The Gephi
+plugin for Gremlin Console utilizes this API to allow for graph and traversal visualization.
-The following instructions assume that Gephi has been download and installed. It further assumes that the Graph
-Streaming plugin has been installed (`Tools > Plugins`). The following instructions explain how to visualize a `Graph`
-and `Traversal`.
+The Gephi plugin for Gremlin Console is compatible with Gephi 0.9.x. The following instructions assume that Gephi
+has been download and installed. It further assumes that the Graph Streaming plugin has been installed
+(`Tools > Plugins`). The following instructions explain how to visualize a `Graph` and `Traversal`.
In Gephi, create a new project with `File > New Project`. In the lower left view, click the "Streaming" tab, open the
Master drop down, and right click `Master Server > Start` which starts the Graph Streaming server in Gephi and by
-default accepts requests at `http://localhost:8080/workspace0`:
+default accepts requests at `http://localhost:8080/workspace1`:
image::gephi-start-server.png[width=800]
@@ -1686,12 +1686,15 @@ that it requires use of the `visualTraversal` option on the `config` function of
:remote config visualTraversal graph <1>
traversal = vg.V(2).in().out('knows').
has('age',gt(30)).outE('created').
- has('weight',gt(0.5d)).inV();null
-:> traversal <2>
+ has('weight',gt(0.5d)).inV();[] <2>
+:> traversal <3>
----
-<1> Configure a "visual traversal" from your "graph" - this must be a `Graph` instance.
-<2> Submit the `Traversal` to visualize to Gephi.
+<1> Configure a "visual traversal" from your "graph" - this must be a `Graph` instance. This command will create a
+new `TraversalSource` called "vg" that must be used to visualize any spawned traversals in Gephi.
+<2> Define the traversal to be visualized. Note that ending the line with `;[]` simply prevents iteration of
+the traversal before it is submitted.
+<3> Submit the `Traversal` to visualize to Gephi.
When the `:>` line is called, each step of the `Traversal` that produces or filters vertices generates events to
Gephi. The events update the color and size of the vertices at that step with `startRGBColor` and `startSize`
@@ -1723,7 +1726,7 @@ Gephi plugin configuration parameters as accepted via the `:remote config` comma
[width="100%",cols="3,10,^2",options="header"]
|=========================================================
|Parameter |Description |Default
-|workspace |The name of the workspace that your Graph Streaming server is started for. |workspace0
+|workspace |The name of the workspace that your Graph Streaming server is started for. |workspace1
|host |The host URL where the Graph Streaming server is configured for. |localhost
|port |The port number of the URL that the Graph Streaming server is listening on. |8080
|sizeDecrementRate |The rate at which the size of an element decreases on each step of the visualization. |0.33
@@ -1732,7 +1735,7 @@ Gephi plugin configuration parameters as accepted via the `:remote config` comma
|startSize |The size an element should be when it is most recently visited. |20
|colorToFade |A single char from the set `{r,g,b,R,G,B}` determining which color to fade for vertices visited in prior steps |g
|colorFadeRate |A float value in the range `(0.0,1.0]` which is multiplied against the current `colorToFade` value for prior vertices; a `1.0` value effectively turns off the color fading of prior step visited vertices |0.7
-|visualTraversal |Creates a `TraversalSource` variable in the Console named `vg` which can be used for visualizing traversals. This configuration option takes two parameters. The first is required and is the name of the `Graph` instance variable that will generate the `TraversalSource`. The second parameter is the variable name that the `TraversalSource` should have when referenced in the Console. If left unspecified, this value defaults to `vg`.
+|visualTraversal |Creates a `TraversalSource` variable in the Console named `vg` which can be used for visualizing traversals. This configuration option takes two parameters. The first is required and is the name of the `Graph` instance variable that will generate the `TraversalSource`. The second parameter is the variable name that the `TraversalSource` should have when referenced in the Console. If left unspecified, this value defaults to `vg`. |vg
|=========================================================
[[server-plugin]]
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/82899ec3/docs/src/upgrade/release-3.2.x-incubating.asciidoc
----------------------------------------------------------------------
diff --git a/docs/src/upgrade/release-3.2.x-incubating.asciidoc b/docs/src/upgrade/release-3.2.x-incubating.asciidoc
index cc50c43..1b5aa51 100644
--- a/docs/src/upgrade/release-3.2.x-incubating.asciidoc
+++ b/docs/src/upgrade/release-3.2.x-incubating.asciidoc
@@ -32,6 +32,14 @@ Please see the link:https://github.com/apache/incubator-tinkerpop/blob/3.2.1-inc
Upgrading for Users
~~~~~~~~~~~~~~~~~~~
+Gephi Plugin
+^^^^^^^^^^^^
+
+The Gephi Plugin has been updated to support Gephi 0.9.x. Please upgrade to this latest version to use the Gephi Plugin
+for Gremlin Console.
+
+See: link:https://issues.apache.org/jira/browse/TINKERPOP-1297[TINKERPOP-1297]
+
TraversalVertexProgram
^^^^^^^^^^^^^^^^^^^^^^
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/82899ec3/gremlin-console/src/main/groovy/org/apache/tinkerpop/gremlin/console/plugin/GephiRemoteAcceptor.groovy
----------------------------------------------------------------------
diff --git a/gremlin-console/src/main/groovy/org/apache/tinkerpop/gremlin/console/plugin/GephiRemoteAcceptor.groovy b/gremlin-console/src/main/groovy/org/apache/tinkerpop/gremlin/console/plugin/GephiRemoteAcceptor.groovy
index 1cd3584..4198444 100644
--- a/gremlin-console/src/main/groovy/org/apache/tinkerpop/gremlin/console/plugin/GephiRemoteAcceptor.groovy
+++ b/gremlin-console/src/main/groovy/org/apache/tinkerpop/gremlin/console/plugin/GephiRemoteAcceptor.groovy
@@ -20,6 +20,8 @@ package org.apache.tinkerpop.gremlin.console.plugin
import groovy.json.JsonOutput
import groovy.transform.CompileStatic
+import org.apache.http.client.methods.CloseableHttpResponse
+import org.apache.http.client.methods.HttpUriRequest
import org.apache.http.client.methods.RequestBuilder
import org.apache.http.entity.StringEntity
import org.apache.http.impl.client.CloseableHttpClient
@@ -27,19 +29,13 @@ import org.apache.http.impl.client.HttpClients
import org.apache.http.util.EntityUtils
import org.apache.tinkerpop.gremlin.groovy.plugin.RemoteAcceptor
import org.apache.tinkerpop.gremlin.groovy.plugin.RemoteException
-import org.apache.tinkerpop.gremlin.process.traversal.Path
import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource
-import org.apache.tinkerpop.gremlin.structure.Direction
import org.apache.tinkerpop.gremlin.structure.Edge
import org.apache.tinkerpop.gremlin.structure.Graph
import org.apache.tinkerpop.gremlin.structure.Vertex
import groovy.json.JsonSlurper
-import org.apache.tinkerpop.gremlin.util.iterator.IteratorUtils
import org.codehaus.groovy.tools.shell.Groovysh
import org.codehaus.groovy.tools.shell.IO
-import org.javatuples.Pair
-
-import java.util.stream.Collectors
/**
* @author Stephen Mallette (http://stephen.genoprime.com)
@@ -49,11 +45,12 @@ class GephiRemoteAcceptor implements RemoteAcceptor {
private String host = "localhost"
private int port = 8080
- private String workspace = "workspace0"
+ private String workspace = "workspace1"
private final Groovysh shell
private final IO io
+ private final Random rand = new Random();
boolean traversalSubmittedForViz = false
long vizStepDelay
private float[] vizStartRGBColor
@@ -76,7 +73,7 @@ class GephiRemoteAcceptor implements RemoteAcceptor {
vizDefaultRGBColor = [0.6f, 0.6f, 0.6f] // light grey
vizColorToFade = 'g' // will fade so blue is strongest
vizColorFadeRate = 0.7 // the multiplicative rate to fade visited vertices
- vizStartSize = 20
+ vizStartSize = 10
vizSizeDecrementRate = 0.33f
}
@@ -193,7 +190,7 @@ class GephiRemoteAcceptor implements RemoteAcceptor {
currentColor *= vizColorFadeRate
int currentSize = attrs["size"]
- currentSize = Math.max(1, currentSize - (currentSize * vizSizeDecrementRate))
+ currentSize = Math.max(vizStartSize, currentSize - (currentSize * vizSizeDecrementRate))
vertexAttributes.get(vertexId).color = currentColor
vertexAttributes.get(vertexId).size = currentSize
@@ -202,27 +199,6 @@ class GephiRemoteAcceptor implements RemoteAcceptor {
}
}
- def touch(Vertex v) {
- vertexAttributes.get(v.id().toString()).touch++
- }
-
- def applyRelativeSizingInGephi() {
- def touches = vertexAttributes.values().collect{it["touch"]}
- def max = touches.max()
- def min = touches.min()
-
- vertexAttributes.each { k, v ->
- double touch = v.touch
-
- // establishes the relative size of the node with a lower limit of 0.25 of vizStartSize
- def relative = Math.max((touch - min) / Math.max((max - min).doubleValue(), 0.00001), 0.25)
- int size = Math.max(1, vizStartSize * relative)
- v.size = size
-
- changeVertexAttributes(k)
- }
- }
-
def changeVertexAttributes(def String vertexId) {
def props = [:]
props.put(vizColorToFade.toString(), vertexAttributes[vertexId].color)
@@ -233,28 +209,17 @@ class GephiRemoteAcceptor implements RemoteAcceptor {
/**
* Visit a vertex traversed and initialize its color and size.
*/
- def visitVertexInGephi(def Vertex v, def size = vizStartSize) {
+ def visitVertexInGephi(def Vertex v) {
def props = [:]
props.put('r', vizStartRGBColor[0])
props.put('g', vizStartRGBColor[1])
props.put('b', vizStartRGBColor[2])
- props.put('size', size)
+ props.put('size', vizStartSize * 2.5)
props.put('visited', 1)
updateGephiGraph([cn: [(v.id().toString()): props]])
- vertexAttributes[v.id().toString()] = [color: vizStartRGBColor[fadeColorIndex()], size: size, touch: 1]
- }
-
- def visitEdgeInGephi(def Edge e) {
- def props = [:]
- props.put('r', vizStartRGBColor[0])
- props.put('g', vizStartRGBColor[1])
- props.put('b', vizStartRGBColor[2])
- props.put('size', vizStartSize)
- props.put('visited', 1)
-
- updateGephiGraph([ce: [(e.id().toString()): props]])
+ vertexAttributes[v.id().toString()] = [color: vizStartRGBColor[fadeColorIndex()], size: vizStartSize * 2.5, touch: 1]
}
def fadeColorIndex() {
@@ -273,6 +238,9 @@ class GephiRemoteAcceptor implements RemoteAcceptor {
props.put('r', vizDefaultRGBColor[0])
props.put('g', vizDefaultRGBColor[1])
props.put('b', vizDefaultRGBColor[2])
+ props.put('x', rand.nextFloat())
+ props.put('y', rand.nextFloat())
+ props.put('size', 10)
props.put('visited', 0)
// only add if it does not exist in graph already
@@ -318,7 +286,8 @@ class GephiRemoteAcceptor implements RemoteAcceptor {
def requestBuilder = RequestBuilder.get("http://$host:$port/$workspace")
queryArgs.each { requestBuilder = requestBuilder.addParameter(it.key, it.value) }
- def resp = EntityUtils.toString(httpclient.execute(requestBuilder.build()).entity)
+ def httpResp = makeRequest(requestBuilder.build())
+ def resp = EntityUtils.toString(httpResp.entity)
// gephi streaming plugin does not set the content type or respect the Accept header - treat as text
if (resp.isEmpty())
@@ -332,7 +301,17 @@ class GephiRemoteAcceptor implements RemoteAcceptor {
.addParameter("format", "JSON")
.addParameter("operation", "updateGraph")
.setEntity(new StringEntity(JsonOutput.toJson(postBody)))
- EntityUtils.consume(httpclient.execute(requestBuilder.build()).entity)
+ EntityUtils.consume(makeRequest(requestBuilder.build()).entity)
+ }
+
+ private CloseableHttpResponse makeRequest(HttpUriRequest request) {
+ def httpResp = httpclient.execute(request)
+ if (httpResp.getStatusLine().getStatusCode() == 200) {
+ return httpResp
+ } else {
+ def resp = EntityUtils.toString(httpResp.entity)
+ throw new RuntimeException("Unsuccessful request to Gephi - [${httpResp.getStatusLine().getStatusCode()}] ${httpResp.getStatusLine().getReasonPhrase()} - $resp")
+ }
}
@Override
http://git-wip-us.apache.org/repos/asf/incubator-tinkerpop/blob/82899ec3/gremlin-console/src/test/java/org/apache/tinkerpop/gremlin/console/groovy/plugin/GephiRemoteAcceptorIntegrateTest.java
----------------------------------------------------------------------
diff --git a/gremlin-console/src/test/java/org/apache/tinkerpop/gremlin/console/groovy/plugin/GephiRemoteAcceptorIntegrateTest.java b/gremlin-console/src/test/java/org/apache/tinkerpop/gremlin/console/groovy/plugin/GephiRemoteAcceptorIntegrateTest.java
index 6af1641..afda528 100644
--- a/gremlin-console/src/test/java/org/apache/tinkerpop/gremlin/console/groovy/plugin/GephiRemoteAcceptorIntegrateTest.java
+++ b/gremlin-console/src/test/java/org/apache/tinkerpop/gremlin/console/groovy/plugin/GephiRemoteAcceptorIntegrateTest.java
@@ -82,31 +82,31 @@ public class GephiRemoteAcceptorIntegrateTest {
@Test
public void shouldConnectWithDefaults() throws RemoteException {
- assertThat(acceptor.connect(Collections.emptyList()).toString(), startsWith("Connection to Gephi - http://localhost:" + port + "/workspace0"));
+ assertThat(acceptor.connect(Collections.emptyList()).toString(), startsWith("Connection to Gephi - http://localhost:" + port + "/workspace1"));
}
@Test
public void shouldSubmitGraph() throws RemoteException {
- stubFor(post(urlPathEqualTo("/workspace0"))
+ stubFor(post(urlPathEqualTo("/workspace1"))
.withQueryParam("format", equalTo("JSON"))
.withQueryParam("operation", equalTo("updateGraph"))
.willReturn(aResponse()
.withStatus(200)));
- stubFor(get(urlPathEqualTo("/workspace0"))
+ stubFor(get(urlPathEqualTo("/workspace1"))
.withQueryParam("operation", equalTo("getNode"))
.willReturn(aResponse()
.withStatus(200)));
acceptor.submit(Arrays.asList("g = org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerGraph.open();g.addVertex();g"));
- wireMockRule.verify(4, postRequestedFor(urlPathEqualTo("/workspace0")));
- wireMockRule.verify(1, getRequestedFor(urlPathEqualTo("/workspace0")));
+ wireMockRule.verify(4, postRequestedFor(urlPathEqualTo("/workspace1")));
+ wireMockRule.verify(1, getRequestedFor(urlPathEqualTo("/workspace1")));
}
@Test
public void shouldSubmitTraversalAfterConfigWithDefaultG() throws RemoteException {
- stubFor(post(urlPathEqualTo("/workspace0"))
+ stubFor(post(urlPathEqualTo("/workspace1"))
.withQueryParam("format", equalTo("JSON"))
.withQueryParam("operation", equalTo("updateGraph"))
.willReturn(aResponse()
@@ -118,12 +118,12 @@ public class GephiRemoteAcceptorIntegrateTest {
acceptor.submit(Arrays.asList(
"vg.V(2).in('knows').out('knows').has('age',org.apache.tinkerpop.gremlin.process.traversal.P.gt(30)).outE('created').has('weight',org.apache.tinkerpop.gremlin.process.traversal.P.gt(0.5d)).inV().iterate()"));
- wireMockRule.verify(18, postRequestedFor(urlPathEqualTo("/workspace0")));
+ wireMockRule.verify(18, postRequestedFor(urlPathEqualTo("/workspace1")));
}
@Test
public void shouldSubmitTraversalAfterConfigWithDefinedG() throws RemoteException {
- stubFor(post(urlPathEqualTo("/workspace0"))
+ stubFor(post(urlPathEqualTo("/workspace1"))
.withQueryParam("format", equalTo("JSON"))
.withQueryParam("operation", equalTo("updateGraph"))
.willReturn(aResponse()
@@ -135,12 +135,12 @@ public class GephiRemoteAcceptorIntegrateTest {
acceptor.submit(Arrays.asList(
"x.V(2).in('knows').out('knows').has('age',org.apache.tinkerpop.gremlin.process.traversal.P.gt(30)).outE('created').has('weight',org.apache.tinkerpop.gremlin.process.traversal.P.gt(0.5d)).inV().iterate()"));
- wireMockRule.verify(18, postRequestedFor(urlPathEqualTo("/workspace0")));
+ wireMockRule.verify(18, postRequestedFor(urlPathEqualTo("/workspace1")));
}
@Test
public void shouldSubmitTraversalOverRepeat() throws RemoteException {
- stubFor(post(urlPathEqualTo("/workspace0"))
+ stubFor(post(urlPathEqualTo("/workspace1"))
.withQueryParam("format", equalTo("JSON"))
.withQueryParam("operation", equalTo("updateGraph"))
.willReturn(aResponse()
@@ -152,12 +152,12 @@ public class GephiRemoteAcceptorIntegrateTest {
acceptor.submit(Arrays.asList(
"vg.V(1).repeat(org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.__().out()).times(2).iterate()"));
- wireMockRule.verify(13, postRequestedFor(urlPathEqualTo("/workspace0")));
+ wireMockRule.verify(13, postRequestedFor(urlPathEqualTo("/workspace1")));
}
@Test
public void shouldClearGraph() throws RemoteException {
- stubFor(post(urlPathEqualTo("/workspace0"))
+ stubFor(post(urlPathEqualTo("/workspace1"))
.withQueryParam("format", equalTo("JSON"))
.withQueryParam("operation", equalTo("updateGraph"))
.withRequestBody(equalToJson("{\"dn\":{\"filter\":\"ALL\"}}"))
@@ -166,7 +166,7 @@ public class GephiRemoteAcceptorIntegrateTest {
acceptor.submit(Arrays.asList("clear"));
- wireMockRule.verify(1, postRequestedFor(urlPathEqualTo("/workspace0")));
+ wireMockRule.verify(1, postRequestedFor(urlPathEqualTo("/workspace1")));
}
private static int pickOpenPort() {