You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by vasia <gi...@git.apache.org> on 2015/03/30 17:23:24 UTC
[GitHub] flink pull request: [FLINK-1694] [gelly] added IterationConfigurat...
GitHub user vasia opened a pull request:
https://github.com/apache/flink/pull/547
[FLINK-1694] [gelly] added IterationConfiguration as a way to configure ...
...a VertexCentricIteration and avoid the split between create and run methods.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/vasia/flink vertex-centric-configuration
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/547.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #547
----
commit a377fba6b9a6b249525fffb0eda876687e7e7c0f
Author: vasia <va...@gmail.com>
Date: 2015-03-29T21:39:08Z
[FLINK-1694] [gelly] added IterationConfiguration as a way to configure a VertexCentricIteration
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1694] [gelly] added IterationConfigurat...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/547#discussion_r27955651
--- Diff: flink-staging/flink-gelly/src/main/java/org/apache/flink/graph/Graph.java ---
@@ -1118,30 +1119,52 @@ public boolean filter(Edge<K, EV> edge) {
}
/**
- * Create a Vertex-Centric iteration on the graph.
- *
+ * Runs a Vertex-Centric iteration on the graph.
+ * No configuration options are provided.
+
* @param vertexUpdateFunction the vertex update function
* @param messagingFunction the messaging function
* @param maximumNumberOfIterations maximum number of iterations to perform
- * @return
+ *
+ * @return the updated Graph after the vertex-centric iteration has converged or
+ * after maximumNumberOfIterations.
*/
- public <M> VertexCentricIteration<K, VV, M, EV> createVertexCentricIteration(
+ public <M> Graph<K, VV, EV> runVertexCentricIteration(
VertexUpdateFunction<K, VV, M> vertexUpdateFunction,
MessagingFunction<K, VV, M, EV> messagingFunction,
int maximumNumberOfIterations) {
- return VertexCentricIteration.withEdges(edges, vertexUpdateFunction,
- messagingFunction, maximumNumberOfIterations);
+
--- End diff --
Function call could be forwarded to the configured `runVertexCentricIteration` to reduce code duplication.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1694] [gelly] added IterationConfigurat...
Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen commented on the pull request:
https://github.com/apache/flink/pull/547#issuecomment-90883704
I like the change as well! Good to go!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1694] [gelly] added IterationConfigurat...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/547
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1694] [gelly] added IterationConfigurat...
Posted by vasia <gi...@git.apache.org>.
Github user vasia commented on the pull request:
https://github.com/apache/flink/pull/547#issuecomment-90745094
Hi,
any opinions on this?
I would like to merge it soon because it changes the api and vertex-centric related issues depend on it.
Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1694] [gelly] added IterationConfigurat...
Posted by vasia <gi...@git.apache.org>.
Github user vasia commented on the pull request:
https://github.com/apache/flink/pull/547#issuecomment-91048962
Thanks so much for the review @fhueske, @StephanEwen!
I'll make the suggested changes and merge :-)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1694] [gelly] added IterationConfigurat...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on the pull request:
https://github.com/apache/flink/pull/547#issuecomment-90863498
LGTM
I browsed the diffs and didn't find anything crucial. Just a typo and one possible simplification.
The change makes vertex iterations much easier to use as indicated by the implemented algorithms.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1694] [gelly] added IterationConfigurat...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/547#discussion_r27955499
--- Diff: docs/gelly_guide.md ---
@@ -343,29 +343,86 @@ Vertex-centric Iterations
Gelly wraps Flink's [Spargel API](spargel_guide.html) to provide methods for vertex-centric iterations.
Like in Spargel, the user only needs to implement two functions: a `VertexUpdateFunction`, which defines how a vertex will update its value
based on the received messages and a `MessagingFunction`, which allows a vertex to send out messages for the next superstep.
-These functions are given as parameters to Gelly's `createVertexCentricIteration`, which returns a `VertexCentricIteration`.
-The user can configure this iteration (set the name, the parallelism, aggregators, etc.) and then run the computation, using the `runVertexCentricIteration` method:
+These functions and the maximum number of iterations to run are given as parameters to Gelly's `runVertexCentricIteration`.
+This method will execute the vertex-centric iteration on the input Graph and return a new Graph, with updated vertex values:
{% highlight java %}
Graph<Long, Double, Double> graph = ...
-// create the vertex-centric iteration
-VertexCentricIteration<Long, Double, Double, Double> iteration =
- graph.createVertexCentricIteration(
+// run Single-Source-Shortest-Paths vertex-centric iteration
+Graph<Long, Double, Double> result =
+ graph.runVertexCentricIteration(
new VertexDistanceUpdater(), new MinDistanceMessenger(), maxIterations);
+// user-defined functions
+public static final class VertexDistanceUpdater {...}
+public static final class MinDistanceMessenger {...}
+
+{% endhighlight %}
+
+### Configuring a Vertex-Centric Iteration
+A vertex-centric iteration can be configured using an `IterationConfiguration` object.
+Currently, the following parameters can be specified:
+
+* <strong>Name</strong>: The name for the vertex-centric iteration. The name is displayed in logs and messages
+and can be specified using the `setName()` method.
+
+* <strong>Parallelism</strong>: The parallelism for the iteration. It can be set using the `setParallelism()` method.
+
+* <strong>Solution set in unmanaged memory</strong>: Defines whether the solution set is kept in managed memory (Flink's internal way of keeping object in serialized form) or as a simple object map. By default, the solution set runs in managed memory. This property can be set using the `setSolutionSetUnmanagedMemory()` method.
--- End diff --
Typo: "... of keeping objectS in ..."
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---